Answers with the slide attached

1The numbers on a laptop

Full ingest: 52 seconds, which is 42.6× realtime, at $0.042. The transcript lane finishes first, so the video is answerable at 37 seconds while the captions are still being written. Retrieval came out at 0.917 recall@10 on the fused lanes, the judge scored correctness 4.33 out of 5, answers came back at 4.5 seconds p95, and it declined both of the unanswerable questions rather than inventing something.

Declining is the result I care most about. A video question-answering system that will not say 'that is not in this video' is not usable in front of anyone.

2The cloud run that was slower than the laptop

I expected the GPU machine to win easily. The full pipeline on a spot L4 took 207 seconds against the laptop's 52 — 10.7× realtime versus 42.6×, on hardware that costs money.

The cause was not the GPU. That machine pairs its L4 with four weak vCPUs, and decoding and filtering frames for scene detection is CPU work: 145 seconds there against 12.6 on the laptop. Hardware-accelerated decode did not help, because the crop and scale filters still run on the CPU. The bottleneck had moved and the instinct to buy a bigger GPU would have made it worse.

Speech recognition alone on the same machine was excellent: 36.8 minutes of audio in 5.3 seconds warm, 417× realtime, at $0.0005 per video-hour. The widely quoted figures around 3,000× are A100-class with heavy batching; 417× is the honest number for this card at batch 8.

Two things happened during those runs that no benchmark would have told me. A spot instance was preempted mid-ingest and the checkpoints resumed cleanly, which is the first time that design has been tested for real. And the caption provider began throttling after about 500 calls in a day, turning a 33-second stage into 305 seconds. Anything that goes to production needs a second caption provider, and I would not have known that from a clean run.

3What the economics say

An idle GPU is the most expensive GPU. Below roughly 30 video-hours a day, scaling to zero and summoning an ephemeral machine per upload beats owning an always-on box, and it is not close. The shape that fell out: a serverless front end that costs nothing at rest, which drops an upload in a bucket and starts a spot machine; the machine ingests, writes its index and keyframes back to the bucket, and deletes itself. Questions afterwards need no GPU at all, at about $0.002 each.

Caption cost per kept keyframe came in at $0.00024 using an 8B open vision model, measured rather than estimated.

4A retrieval idea that did not survive

Late-interaction retrieval straight over page pixels is fashionable and I wanted it to win. I grew the golden set to 28 questions, 15 of them visual, and ran it honestly.

The shipped dense lane scored 0.933 on the visual questions. Retrieval over pixels scored 0.867. Adding it to the fusion changed nothing at all. And the variant I was really testing — dropping captions entirely and relying on transcript plus pixels — dropped visual recall to 0.733. The captions earn their four cents a video. I kept the experiment in a copy and left the shipped stack alone.

5The voice half

The same corpus, asked out loud. What matters here is perceived latency rather than model latency: first audio back within about 500 ms feels instant, 800 ms is fine, and past two seconds the conversation is broken. You get there by streaming every stage and by acknowledging immediately, not by finding one fast model.

A fully open-source path lands in the 400–800 ms band. A speech-native open model gets to roughly 250 ms with real function calling. On cost, a stack assembled from hosted components runs about one to three cents per conversation-minute; the well-known commercial voice stacks run ten to thirty times that.

This half is still exploring. I have the teardown and the latency budget and a prototype, and no measured golden-set numbers of my own yet.

6What is still open

The retrieval and ingest numbers are from a small golden set — fourteen questions, grown to twenty-eight for the pixel comparison. That is enough to reject an idea and not enough to rank two good ones.

I have not built the sharded I-frame-only decode that would fix the 145-second CPU stage, so every cloud figure here is worse than the design allows. Also worth recording: the machine class I wanted was out of stock across three regions on the day I tried to provision it, which is a real constraint on this design and not usually in anyone's benchmark.

7Evidence

This thread has no lab notebook yet. What exists is the working folder below and the measurements in Table 1; until the runs are written up as claims with verdicts, treat the numbers as this page states them and no further.

Where the work lives:

• ~/company/wsz-research/video-audio-frames-summarization — video QA experiments.
• ~/company/wsz-research/voice-bridge-research.md — realtime voice, 20k words of notes.
• ~/company/wsz-research/voxbridge — the voice prototype.
• ~/company/wsz-research/research.md — the cloud benchmark, every number measured.