How I Moved the Interface Out of a Borrowed Machine

Introduction

A. A borrowed GPU, and a page that went with it

My laptop's GPU has 4 GB of video memory, which is not enough for the models behind several of my projects: image generation, document conversion, speech-to-text. So the heavy part of each runs on a Google Colab runtime with a T4 and about 15 GB of VRAM — a machine borrowed for the length of a session.

Each of these projects needs an interface. Something has to let me upload material, start work, watch progress, and look at results. The most natural place to put that interface was right next to the work, inside the same runtime: a small web server in the notebook, serving a page, with Colab's own port proxy turning it into a URL I could open in a browser. All three projects started that way.

It worked, and for a first version it was hard to beat. There was nothing to install and nothing to host, and the interface could call the model directly because it lived in the same process. But everything about it was borrowed along with the GPU. When the session ended, the page ended with it. Every new runtime produced a new URL. And the arrangement itself — serving a web application out of a Colab runtime through its proxy — turned out to be something Colab did not welcome. Even looking at yesterday's results meant starting a GPU session first, spending quota on nothing but opening a page.

The challenge is to move the interface out of the borrowed machine — so that it no longer dies with the runtime or depends on what Colab permits — while the runtime keeps doing the GPU work.

Easy to state, awkward to build. Moving a page is simple. What is not simple is everything the web server in the middle had been doing without anyone noticing: answering requests, holding the state of the work, translating errors into responses, and being the single place that both sides agreed was right. Take the server away and every one of those jobs has to go somewhere else.

B. What changes when there is no server in the middle

The shape the projects moved to has three parts, none of them a web server. The runtime is a pure worker: it loads the model, waits for work, does it, and writes the results out. The interface runs on my laptop, in the browser. Between them sit two things that both sides reach outward to: object storage, which holds every file and every record, and a message broker, which carries short notifications — "start this", "that one is done" — between them.

That removes the server, and with it the thing that used to guarantee the two sides agreed. When a server answers a request, it defines what the answer means. With no server, the interface and the worker are two separate codebases, in two languages, that must independently agree on where files go, what a finished item looks like, which topics carry which messages, and who is the source of truth when they disagree. Nothing enforces that agreement except the agreement itself. The worker is Python in a notebook; the interface is TypeScript in a browser. They share no code, no process, and no compiler that could notice a mismatch between them.

So the work of moving the interface out was mostly the work of making that agreement explicit, testable, and hard to break — and of noticing which jobs are better done in neither place. The reward for doing it is an interface that is simply there, whether or not a runtime happens to be alive behind it.

C. Eight obstacles in the way

The first interface lived inside the runtime. It was the simplest thing that worked, and it tied the life of the page to the life of a borrowed session.

The proxy itself was the risk. Serving a web application through the runtime's port proxy drew warnings and restrictions from Colab, and could put the account at risk.

The runtime has to become a pure worker. Taking the interface out meant removing a web framework, a page, and eighteen endpoints from a notebook, without disturbing the code that actually generated anything.

Without a server, something else has to serve the data. The interface still needs every image and record, and the notifications channel is the wrong place to carry them.

Without a server, the contract is only an agreement. Two codebases in two languages must compute the same folder names and topics, and a mismatch fails silently.

A command must survive a lost message. A notification can be dropped. A request that exists only in that notification is dropped with it.

Work moved into the browser can land in the wrong place. Preparing media in the browser seemed like a way to spare the runtime a job. It hung, and the fix was to stop doing it at all.

Sometimes the best interface is none. For a quick job, the whole arrangement of interface, storage, and broker is more machinery than the job deserves.

The order follows the move itself: why the interface had to leave, what the runtime became once it had left, what replaced the server's jobs, and finally two lessons about where work belongs — including the case where the answer is to have no interface at all.

Method

A. The interface that lived in the runtime

Here is the answer to the first obstacle, starting with what it looked like before. The first version of each project ran a web server inside the notebook and asked Colab for a URL to it:

app = FastAPI()

@app.get("/", response_class=HTMLResponse)
...
url = eval_js(f"google.colab.kernel.proxyPort({PORT})")
uvicorn.run(app, host="0.0.0.0", port=PORT, log_level="warning")

The page and every action behind it lived in one process.

The costs were structural rather than bugs. The interface could only exist while a runtime existed, so reviewing results meant starting a GPU session. The URL was new each time, so there was no stable address to return to. And the heaviest part of the notebook was not the model code but the interface: in the slideshow project, the page alone was 48 KB of HTML, CSS, and JavaScript embedded in a Python string.

B. When the arrangement itself is the risk

This answers the second obstacle, and it was what forced the change. Colab warned about and restricted serving a web application out of the runtime through its proxy. The assessment written down at the time was blunt: the arrangement risked violating Google's policy, with the account itself potentially banned.

That reframes the decision. Moving the interface out was not a refactor for tidiness that could wait. The borrowed machine sets terms for what may run on it, and an interface hosted inside it was on the wrong side of those terms. Everything else in this article is the work of doing without the thing that had to go.

The replacement was not designed from scratch. The PDF converter project had already made the move and been used in practice, so the slideshow and speech projects copied its shape and several of its files, including the storage client, verbatim. A proven arrangement was a better starting point than a new one.

C. Turning the runtime into a pure worker

This answers the third obstacle. Removing the interface from the slideshow notebook was done as a separate commit containing nothing but deletions, so that it could be reviewed as exactly that:

# one commit, deletions only
endpoints removed          18
embedded page removed      48 KB
application cell           139 KB  ->  78 KB
functions                  112     ->  87
lines removed              1526

What stayed was every function that did actual work — loading models, generating, tracking status. The only change to that layer was in how it reported failure. Errors had been HTTP exceptions, because an HTTP endpoint was the caller. With no HTTP left, a status code meant nothing:

class WorkerError(Exception):
    # a domain error: the interface needs a readable message, not a status code
    pass

raise WorkerError(...)   # e.g. "project not found"

Fourteen raises were converted. Each one describes something about the work — a project that is not open, an item already being generated — and travels to the interface as a readable message on the item's status. The worker that remains has no web server at all. It loads the model, waits, works, and writes.

D. Letting the storage serve the data

This answers the fourth obstacle. The server used to hand the interface every image and record. Without it, the interface reads them straight from object storage, and the broker carries only notifications:

[laptop]  interface --directly--> storage <--directly-- [Colab]  worker
              |                                             |
              +---------- broker (MQTT, triggers only) -----+

The rule behind that picture is that the broker never carries a file's bytes. Messages say that something happened; the thing itself is fetched from storage. That keeps messages small and fast, and it means a message that is lost costs a notification, never data.

Reading storage directly from a browser has its own requirements. Each image is fetched through a presigned URL — a link that grants temporary access to one object — and the lifetime of those links was chosen from how the interface is used:

// longer than the storage default (900 s): sessions last hours,
// and a link expiring mid-scroll makes images fail to load
presignTtl: 3600,

Cross-origin access was checked directly rather than assumed: a preflight request against the storage showed it reflecting any origin, so both the new interface and an older one could run side by side without changing the server's configuration.

E. A contract nobody enforces

This answers the fifth obstacle. In the speech project, the rules both sides must share were written down as a table, because nothing else would hold them together:

The reason it is a table and not a comment is the way it fails. If the two versions of the folder-name function disagree, the worker writes to one folder and the interface reads another. If the topic functions disagree, each side listens on a topic the other never uses. Neither produces an error. Both produce silence, which looks exactly like "the worker is slow".

So each pair of functions is tested against the same inputs. The three pure functions produced identical output across 21 test cases before the interface was built on them. The table says what must change together; the tests prove it did.

F. Writing the request before sending the notice

This answers the sixth obstacle. When I ask the speech worker to transcribe something, two things happen: the request is recorded, and the worker is told about it. The order is the whole design:

await store.requestConvert(props.name, model.value, language.value, force)
await job
  .notifyConvert(props.name, cur.stem, model.value, language.value, force)
  .catch(() => {
    // broker unreachable: the request is already in the manifest,
    // so the worker still finds it when it scans — only later
  })
startPoll()

The request goes into the project's manifest in storage first. Only then is a notification sent. If the notification is lost, nothing is lost with it: the worker finds the request the next time it scans, and a worker that receives the notification is guaranteed to find a complete request waiting.

The interface, in turn, does not depend on notifications to show results. It polls storage every five seconds while work is in progress. That was tested end to end with the broker producing no messages at all, and the transcript still appeared as soon as it was written. The notifications make things faster. The storage makes them correct.

G. The job that belonged to neither side

This answers the seventh obstacle. The speech interface originally extracted the audio from a video in the browser before uploading it, using a WebAssembly build of ffmpeg — the common command-line program for converting audio and video, compiled into a form a browser can run. The idea was sound: upload less, give the runtime less to do.

That library can hang forever on "extracting audio". It starts its worker as a module worker, where the call it uses to load its own core is not allowed. The worker fails while it is still being set up — before it has attached the handler that would report the failure — so no error ever comes back, and the promise waits indefinitely, in several versions of the library alike.

Instead of patching around it, the step was removed, because it turned out never to have been needed. The transcription library decodes video directly:

# the same 10-minute, 40 MB recording
decoded straight from the video    9600058 samples   0.6 s
decoded from the extracted audio   9600058 samples   0.6 s

The transcripts from both were identical word for word, across every container format tried. The only cost of uploading the whole video is upload time — about seven seconds more for a 31 MB file — and it bought back 32 MB of WebAssembly, a dependency on a content delivery network, and a class of bug that fails without saying anything.

Moving work into the interface is only an improvement when the interface is a good place for that work. Here the best place was neither the browser nor a new step on the worker, but a step that did not need to exist.

H. The interface that is not there

This answers the eighth obstacle. For the speech project there is also a headless variant: no interface, no storage, no broker. It runs entirely inside the Colab runtime. Media is dragged into a folder through Colab's file panel, the notebook lists what has not been transcribed yet, and I choose by number:

print(...)   # "leave empty and press Enter to cancel"
pilihan = parse_pilihan(input(...), len(antre))

if not pilihan:
    print(...)   # nothing was started

An empty answer means cancel, not "all" — pressing Enter by accident must not start hours of transcription. Status is not stored anywhere. It is derived from whether a subtitle file already sits next to each recording, the same way the full worker derives it from what exists in storage, so a failed file simply gets offered again next time.

I reach for it in two situations: one or two recordings that are needed quickly, where the whole chain of interface, storage, and broker is more machinery than the job warrants; and when the interface or the storage is unavailable, as a way to keep working anyway.

Its existence says something about the rest of the design. Moving the interface out was the right answer to one set of needs — long jobs, many items, review over time. It is not the only answer, and for a short job with one file, an interface that does not exist cannot fail.

Results and Discussion

A. What the eight decisions add up to

The slideshow and speech projects now run with the interface on the laptop and a worker in Colab that has no web server. The interface stays open while runtimes come and go, can be used to review results without a GPU session, and no longer depends on anything Colab might restrict. The PDF converter made the same move first.

B. The agreement is the server now

A web server does a quiet job that is easy to overlook until it is gone: it is the one place both sides trust. Remove it, and that job does not disappear. It has to be taken over by something, and in this design it is taken over by agreements — about folder names, topics, write order, and which copy of the truth wins.

Most of the obstacles are that transfer happening. The storage takes over holding the truth, which is why a request is written there before anyone is told about it. The broker takes over only the part of the server's job that was about timing, which is why it carries triggers and never data. The contract table and its tests take over the part that was about meaning, because without a server nothing else would notice two codebases drifting apart.

The general lesson applies well beyond Colab. When a system loses its middle, whatever the middle was guaranteeing has to be reassigned deliberately — or it is not guaranteed any more, and the first sign is usually silence.

C. What this does not cover

An interface that reads and writes storage directly is a design for an internal tool used by one person on their own machine, and only there. An interface exposed to anyone else goes through a server again — or at least through a service that hands out short-lived, narrowly scoped access.

Conclusion

A 4 GB GPU put the heavy work of several projects on a borrowed Colab runtime, and the natural first step was to put the interface there too. It worked, but it tied the page to the life of the session and relied on serving a web application through Colab's proxy — an arrangement Colab warned against and restricted.

Moving the interface out turned the runtime into a pure worker, stripped of eighteen endpoints and a 48 KB page, reporting domain errors instead of HTTP ones. Storage took over serving data, with the broker limited to triggers. A contract table and paired tests took over what the server used to guarantee about meaning. Requests are recorded before anyone is notified, so a lost message costs time rather than work. A job that belonged to neither side was dropped instead of patched. And for a quick job, the right interface turned out to be none at all.

What carries over is to ask, whenever the middle of a system is removed, what it was quietly guaranteeing — and to give each of those guarantees a new home on purpose.

Terms

port proxy

A Colab feature that exposes a port inside the runtime as a URL in the browser, which is how a web server running in a notebook can be reached.

pure worker

A process that only does the work — load, wait, compute, write — with no web server or interface of its own.

presigned URL

A link to one object in storage, signed to grant temporary access without sharing the storage credentials with whoever uses the link.

manifest

A record in storage describing a project and its items, including any request waiting to be processed. Here it is the source of truth for what has been asked.

headless

Running without any interface. The work is started and chosen from the notebook itself, and results are read directly from where they are written.

module worker

A browser worker started as a JavaScript module. Some older ways of loading code, such as importScripts(), are not allowed inside one.