A Go worker pool converts MP4 uploads into HLS segments in parallel, coordinating FFmpeg jobs, database status updates and S3 uploads without serial bottlenecks. This article walks through the architecture demonstrated by Full Cycle during its March 2025 Golang Tech Week, including the domain model, migrations, integration tests and the AI-assisted workflow used to build it.
What the video conversion platform does
The platform uploads MP4 files and converts them to HLS so browsers can stream them in small pieces instead of downloading one large file. The project was built live by Full Cycle, a Brazilian software education company, during its Golang Tech Week streamed in March 2025.
HLS, HTTP Live Streaming, splits a video into short .ts segments plus an M3U8 manifest that indexes them; the player reads the manifest and fetches the next segment as needed. In the live demo the conversion produced 10-second segments using FFmpeg parameters such as -hls_time 10, the H.264 video codec and AAC audio, all inside a Docker container because the host machine had no FFmpeg installed.
The hard part is not a single conversion. It is coordinating several conversions and uploads at once: while worker one processes video one, worker two processes video two, and a finished video immediately starts sending up to a thousand segment files to storage. Doing this serially is easy; doing it concurrently requires deliberate structure.
How the Go worker pool works
The Go worker pool in this project is a generic package that executes any function with a configurable number of lightweight goroutines managed by the Go runtime. Go's own documentation on concurrency describes goroutines and channels as the standard way to coordinate this kind of work.
The flow demonstrated on day one of the event, using a simple number processor as the example, follows five steps:
- Define a
Jobstruct holding the input data and the function to execute. - Create the pool, passing the processing function and the worker count (three in the demo).
- Start the pool with an input channel that receives jobs in the background.
- Send jobs into the channel; the pool dispatches each one to a free worker.
- Workers execute the function and return results through a result channel.
Because jobs flow through channels, the caller keeps running while workers process in parallel. The same pattern was reused later in the project: the video converter service feeds ConversionJob structs into the pool, and each result carries the video ID, a success flag, an error, the produced output files and the processing duration.
A worker pool behaves like a load balancer inside the application. The pipeline chains pools together: a conversion pool turns MP4 files into segments, and a second stage uploads each segment, keeping both phases saturated.
DDD layers: domain, application, infrastructure
The project uses a simplified domain-driven design with three layers: domain, application and infrastructure. The presenter said he deliberately avoids creating a trillion interfaces, keeping DDD light so the demo stays readable.
Layer responsibilities in this codebase:
| Layer | Component | Responsibility |
|---|---|---|
| Domain | Video entity, VideoRepository interface | Holds status transitions and invariants |
| Application | FFmpeg service, video converter service | Orchestrates conversion via the worker pool |
| Infrastructure | PostgreSQL repository, migrations, LocalStack | Persists data and simulates S3 locally |
The Video entity carries an ID, title, file path, HLS path, manifest path, S3 URL, manifest URL, status, upload status, error message and timestamps. Expressive methods such as MarkAsProcessing, MarkAsCompleted, MarkAsFailed and IsCompleted keep domain rules explicit instead of raw field updates, a point a viewer identified only as the DDD-focused participant in the chat pressed during the stream.
The repository interface lives in the domain layer because, in DDD terms, a repository is a domain service: the application implements and uses it, but the domain guarantees invariants. The presenter noted he intentionally returns only error from Create and UpdateStatus rather than the entity plus error, because callers do not need the value back.
Persistence: migrations and integration tests
Persistence uses plain SQL with database/sql instead of an ORM, and schema changes run through golang-migrate, the standard migration tool for Go projects. The migration creates a videos table derived from the Video entity, with the ID typed as a UUID.
Migrations execute inside the application container with docker compose exec app, not inside the PostgreSQL container, a rule written into the project's Cursor rules after the AI agent twice tried the wrong path. The Docker Compose stack includes the Go 1.24 app image with FFmpeg installed, PostgreSQL on port 8080 for the API, and LocalStack, a fake local cloud provider that simulates S3 for development and tests.
Integration tests hit the real PostgreSQL database, guarded by a Go build tag so they can be skipped with -tags=integration off. The presenter caught the AI generating database tests without the tag and corrected it, separating unit tests from tests that need external resources. He then verified everything himself by running go test inside the container before marking the task complete.
The FFmpeg service and the video converter service
Two services divide the conversion work. The FFmpeg service converts one video; the video converter service manages the whole flow, including database status updates. Keeping them separate means the conversion primitive stays reusable by many use cases.
The FFmpeg service exposes a ConvertToHLS method that validates the context, creates the output directory, runs the conversion and returns the list of output files. Each output file records its path and a type: manifest for .m3u8 files or segment for .ts files. The presenter simplified the AI's first draft, which had over-engineered codec structs, down to codec, bitrate and a single resolution.
Instead of shelling out to the FFmpeg binary directly, the service uses the go-ffmpeg wrapper library, which the presenter described as a convenience layer so Go code does not need raw exec calls. The AI initially ignored this and called FFmpeg directly until pointed back to the project's golibs documentation file.
The video converter service then ties everything together. It depends on the FFmpeg service, the VideoRepository, the worker pool and a log/slog logger, Go's structured logging package from the standard library. Its Process function marks the video as processing, converts it, classifies each output file, updates the HLS path and status in PostgreSQL, and returns a ConversionResult. When tests hung, the cause was a misused pool integration; the presenter swapped in his documented reference worker pool implementation and asked the agent to guarantee compatibility.
Scoping AI: baby steps, context files and fresh chats
The workflow treats AI-assisted coding like TDD: step size matters. A step too wide and the model wanders; a step too small and you interact forever. The presenter, working in Cursor with Claude.7 Sonnet during the March 2025 session, asked the model to draft a plan in chat before implementing anything, then corrected signatures before code existed.
Context lives in Markdown files inside a project folder: an overview, architecture notes, a golibs file naming every library, and per-feature task lists with checkboxes, acceptance criteria and results the agent updates as it works. Cursor rules add conventions: tests live beside the code, integration tests carry a build tag, all commands run in Docker, and never upgrade a library without asking.
His practical rules from the live session:
- Ask how the model plans to solve something before letting it write code.
- Never request implementations directly; ask for its proposal first, then critique it.
- Finish one reasonably sized feature per chat, then OpenAI new chat that re-reads the project state.
- When the agent drifts, point it back to the relevant rule file instead of re-explaining.
- Document manual work in the state file so a later chat knows what happened.
The failure mode appeared on stream: after too many requests in one chat, the agent started running commands outside the container and inventing scripts. A fresh chat with instructions to read the project state fixed it.
Community, courses and where this fits
The session drew working developers. One attendee, Gustavo Dev Doido, said the demo changed how he plans to work: define architecture conceptually first, let the AI generate code, then review it. Another business owner watching said his company bought Full Cycle's Crazystack Typescript course content for the same architecture-first approach, and a manager noted his team is trading senior handwriting time for AI-piloted delivery. Viewers of the Bootcamp do Dev Doido and similar programs will recognize the same tension the chat debated: whether piloting counts as programming, and what juniors need to know before they can steer a model well.
The presenter's answer was blunt: fundamentals decide quality. System design, concurrency, memory management and domain modeling determine whether the generated code holds up, with or without AI. For Portuguese speakers he also noted he writes all prompts and specs in Portuguese and translates outputs to English on demand. Course details for the Go Expert postgraduate program mentioned live are on crazystack.com.br.
Frequently Asked Questions
- Why use a worker pool instead of one goroutine per video?
A fixed pool caps concurrency so memory, CPU and FFmpeg processes stay predictable under load. Unbounded goroutines can spawn thousands of FFmpeg processes and exhaust the machine.
- Why run FFmpeg inside Docker for this project?
The host in the demo had no FFmpeg installed, so the Dockerfile installs FFmpeg alongside Go 1.24. Tests and migrations then run via docker compose exec, keeping the environment identical for everyone and for the AI agent.
- What does LocalStack replace in development?
LocalStack simulates S3 locally, so uploads go to a fake bucket without real AWS credentials or network cost. Production code targets real S3 through the AWS SDK with the endpoint configurable.
- Why keep the repository interface in the domain layer?
In domain-driven design the repository is a domain service that protects invariants. The infrastructure layer implements it with PostgreSQL, but the domain defines the contract so persistence code cannot break domain rules.
- How were integration tests separated from unit tests?
Integration tests that touch PostgreSQL or run real FFmpeg conversions carry a build tag, so go test skips them by default and CI runs them explicitly with -tags=integration.
Turn your own stream into an article
A three-hour live stream holds the same kind of architecture decisions, corrections and viewer debates you just read about, but almost nobody has time to rewatch it. If your channel or your team records sessions like this one, the insights deserve a written form that search engines and new developers can actually find.
Skala Blog turns a YouTube URL into a structured article: paste the link, get the transcript, and generate a publishable draft you review before it goes live.
Fork this article
Start a new branch from the same video, shaped your way. You keep the credit; the original keeps the attribution.
A fork in another language is filed as a translation of this article, so the two pages point at each other. You can unlink it later from the editor.
0/240
You are creating
- Format
- For
- Language
- Source
- Your angle
You will be asked to sign in before it is generated.
Buy credits