The whole design, in writing
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What is an online judge, specifically?
Users submit arbitrary code — sometimes buggy, sometimes deliberately malicious — and expect it to be run, measured, and graded, at contest scale, fairly, without ever letting one submission compromise the host, starve other users, or get an inconsistent verdict on a rerun.
We'll build it around the one property that has to hold above all others: untrusted code execution must never become a security or fairness incident. Everything — sandboxing, queueing, resource limits, fair scheduling — flows from that.
What the new pieces do
- Userclient
- Submits a solution and waits — sometimes for milliseconds, sometimes, during a packed contest, for longer — to find out if it passed.
Step 1 · The baseline
Run the code right here
Simplest version: the API server receives a submission and just... runs it, directly, in-process or as a plain subprocess, to get the output. Why is this dangerous?
The API server directly executes arbitrary user-submitted code. What's the danger?
Performance is a real but secondary concern here — the primary issue is far more severe than speed.
Running untrusted code with no isolation on the same infrastructure serving every other user's requests means a single malicious submission can potentially read filesystem data it shouldn't, make outbound network calls, fork-bomb the host, or otherwise compromise a shared production system — this isn't a hypothetical, it's the defining risk of the entire product category.
CPU cost for legitimate code is a real, separate concern (addressed later via resource limits) — it's not the primary danger of running UNISOLATED, potentially malicious code on shared infrastructure.
Running submitted code directly, with no isolation, on infrastructure shared with everything else is a full arbitrary-code-execution vulnerability. Nothing else in this design matters if this foundational problem isn't solved first: untrusted code must never run unsandboxed on shared infrastructure.
Why this piece earns its place
The tempting middle ground is to keep running submissions on the API host and just refuse the dangerous ones — scan the source for socket, exec, open, block a list of imports, reject anything that looks hostile. That loses, and it’s worth saying why out loud: a denylist is a claim that you can enumerate every route a Turing-complete language has to the operating system, made against people whose whole hobby is finding the one you missed. You only learn it was incomplete from the incident. Isolation is the opposite shape of claim — it doesn’t care what the code tries, because the capability isn’t there to reach for. The other half people skip is the build. Compiling is already running untrusted input through a toolchain: includes, macros, template expansion and build scripts all execute at compile time, and a submission can exhaust memory or spin forever before a single line of its own program runs. So the boundary isn’t “sandbox the program”, it’s “sandbox everything that touches the submission”, compiler included.
What the new pieces do
- Submission APIgateway
- Accepts a submission and hands it off — never executes anything itself.
Step 2 · Real sandboxing
A container alone isn't enough
Isolate execution in some kind of sandbox — a container, a VM, a purpose-built sandbox runtime. But a plain container with default settings still shares the host kernel, has no CPU/memory ceiling by default, and can still open network connections or fork endlessly. What does a sandbox actually need to guarantee?
What does "sandboxed execution" actually need to guarantee, beyond just running in a container?
A default container alone doesn't cap CPU time, doesn't cap memory, doesn't block network access, and doesn't prevent a fork bomb from exhausting host process limits — process isolation is necessary but nowhere near sufficient by itself.
A wall-clock timeout alone stops an infinite loop eventually, but does nothing about memory exhaustion, network access, filesystem access outside the sandbox, or a fork bomb consuming process-table entries before the timeout even fires.
Real isolation needs multiple, layered guarantees together: bounded CPU time (kills runaway loops), bounded memory (kills memory exhaustion), no network (prevents exfiltration or attacking other systems), restricted filesystem access, and a cap on process/thread creation (prevents fork bombs) — and the whole environment is thrown away after each run so nothing persists between submissions.
Real sandboxing means several guarantees enforced together: hard CPU-time and memory limits via cgroups, no network access, minimal filesystem access, a cap on process/fork creation (stopping fork bombs), and a fully ephemeral environment destroyed after every single run so nothing persists or leaks between submissions.
Why this piece earns its place
Ephemeral teardown quietly assumes a fresh environment is cheap to create. It isn’t — provisioning a sandbox, its filesystem and its toolchain can cost as much as the run it wraps, and that cost lands on the critical path of every verdict, worst of all in the minute a contest opens. The pressure that creates is the dangerous one: someone proposes keeping a sandbox warm between submissions, and single-use isolation is gone. The way out is to pre-create rather than reuse. Keep a pool of fresh, never-used sandboxes, hand one out per submission, and refill the pool asynchronously behind the burst — each sandbox still serves exactly one run and is destroyed after it, only the waiting moved earlier. It’s also worth naming this layer’s ceiling honestly: cgroups, namespaces and seccomp filters all rest on one shared kernel, so a single kernel bug defeats several of them at once. Where the threat model justifies the cost, the answer is a separate kernel per run — a microVM — not more flags on the same one.
What the new pieces do
- Sandbox Workersservice
- Runs submitted code in strict isolation — no network, hard CPU/memory limits, torn down after every single run.
Step 3 · Async execution at contest scale
Submission and execution are two different moments
Execution can take real time — seconds, sometimes more for slow submissions — and during a live contest, thousands of submissions can land within the same few seconds. Should the API request block until execution completes?
A contest produces thousands of near-simultaneous submissions, each taking real time to execute. Block the request until done?
Holding open thousands of simultaneous HTTP connections, each waiting on a multi-second sandboxed execution, doesn't scale — the API server itself becomes the bottleneck, tied up waiting rather than accepting the next submission.
Decoupling "accepted" from "judged" via a durable queue means the API can accept submissions as fast as they arrive, completely independent of how many workers are currently busy executing code. Worker capacity can scale up during a contest burst and down afterward, without the API path itself ever being the bottleneck.
Simply rejecting submissions during exactly the highest-traffic, highest-stakes moment (a live contest) is the worst possible user experience — the system should absorb the burst, not refuse it.
Accept submissions immediately and place them on a durable Job Queue; a pool of Sandbox Workers pulls and executes asynchronously, independent of submission rate. This decouples "accepted" from "judged" — the API stays fast and available even during a massive contest-start burst, and worker capacity scales to match demand.
Why this piece earns its place
Accepting instantly turns a request into a promise, and a queue is allowed to break that promise quietly. Durable delivery is at-least-once, not exactly-once: a worker that dies mid-execution gets its job redelivered, which is exactly what you want — right up until the submission is the reason the worker died. Then it kills the next worker, and the next, and one adversarial program walks the entire pool while looking like an infrastructure problem. So bound it: count deliveries on the job itself, and after a small number stop redelivering and record an explicit system-error verdict with an alert, rather than letting it circle forever. At-least-once also means two workers can legitimately judge the same submission at once, so the verdict write is keyed by submission id and idempotent instead of appended. And because a genuinely lost job produces no error anywhere — just a submission that says judging forever — I’d age the pending set and alarm on anything older than the worst plausible judging time, since nothing else will tell you.
- asyncsubmission accepted instantly
- worker poolscales with contest load
- 0 blockingon the API request path
What the new pieces do
- Job Queueevents
- Decouples "submission accepted" from "execution finished" — nothing is lost even if every worker is briefly unavailable.
Step 4 · Determining the verdict
Hidden tests, checkers, and short-circuiting
A submission runs against multiple hidden test cases. Some problems have exactly one correct output per input (exact match); others have multiple valid outputs (floating-point tolerance, multiple valid orderings). How does the judge decide pass or fail?
Run the submission against each hidden test case in Test Cases, comparing actual output to expected — either an exact match, or, for problems where multiple outputs can be valid, a custom checker program that validates correctness rather than doing a literal diff. The judge typically short-circuits on the first failing test (reporting that specific verdict — Wrong Answer, Time Limit Exceeded, Memory Limit Exceeded, Runtime Error) rather than always running every test, saving compute on submissions that are already known to fail.
Why this piece earns its place
Short-circuiting makes the verdict a function of test order, which is a contract rather than an implementation detail. A submission that’s wrong on one test and too slow on another gets Wrong Answer or Time Limit Exceeded purely according to which hidden test the runner reached first. That’s fine while the order is fixed — but it means test order is part of the judge’s state. Shuffling the tests, inserting a case at the front, or racing tests in parallel can change a published verdict for code nobody touched, and that shows up later as exactly the flaky-looking rerun a judge can’t afford. So the order is stored and versioned with the test set, and parallel execution still reports the lowest-index failure rather than the first one to finish. The place the assumption stops holding altogether is partial scoring: once a problem awards points per subtask, every subtask has to be attempted, and “stop at the first failure” shrinks to an optimization applied inside a subtask that has already failed.
What the new pieces do
- Test Casesdata
- The hidden inputs, expected outputs, and (where needed) custom checker logic that actually determines a verdict.
- Verdictsdata
- The durable record of every submission's outcome — what the user ultimately sees, and what leaderboards are built from.
Step 5 · Precise, fair resource limits
CPU time, not wall-clock
Enforce a time limit on each submission. If the judge is under heavy load (many workers executing simultaneously, contending for the same host's CPU), a wall-clock timeout can unfairly penalize a submission that's algorithmically fine but simply got less CPU time due to contention. What should the time limit actually measure?
The judge is under heavy contest load. What should the time limit for each submission actually measure?
Wall-clock time is affected by host contention: the same submission can take noticeably longer under heavy concurrent load purely because it received less CPU time, not because its algorithm is actually slower — this makes verdicts inconsistent and unfair depending on unrelated system load.
CPU-time limits (enforced via cgroups) measure the actual computational work done, regardless of how contended the host was at that moment — a submission that needs 500ms of CPU work gets judged on that 500ms whether it ran alone or alongside a thousand other contest submissions, keeping verdicts fair and consistent.
An unbounded execution time means an infinite loop (or a deliberately slow submission) never terminates, tying up a sandbox worker indefinitely and defeating the entire point of a time limit as a correctness/efficiency signal.
Enforce limits on actual CPU time consumed (via cgroups CPU accounting), not wall-clock duration. This keeps a submission's verdict consistent and fair regardless of how contended the host happens to be at that moment — the same algorithmic solution gets the same time-limit outcome whether it runs alone or during the busiest minute of a contest.
Why this piece earns its place
The limit is a number a human chooses, per problem, and choosing it is most of the work. The usual method is to measure the intended solution’s CPU time on judge hardware and multiply — enough headroom that a competent implementation of the right algorithm passes comfortably, not so much that the wrong complexity slips through. Both directions cost something real. Too tight and the problem stops testing algorithms and starts testing constant factors and language choice: the same correct approach passes compiled and fails interpreted, which is why judges carry per-language multipliers instead of one global number. Too loose and the problem loses its teeth — a quadratic solution passes where a linearithmic one was the entire point, and the problem no longer distinguishes what it was written to distinguish. The consequence for problem setting is that intended and unintended complexities should be separated by a wide margin at the largest input, because any submission sitting right on the limit is being decided by noise no matter how cleanly you account for the CPU.
- CPU-time limitvia cgroups accounting
- contention-independentsame verdict under any load
- memory limitreal OOM kill, not a soft check
Step 6 · Determinism
The same submission should get the same verdict, always
A judge that gives a different verdict on a rerun of the exact same code — sometimes Accepted, sometimes a flaky Time Limit Exceeded — destroys trust, especially in a competitive contest setting where a verdict has real stakes.
CPU-time limiting (Step 5) already removes the dominant source of flakiness — host contention no longer affects the measured time. Combine it with fully ephemeral, freshly-provisioned sandboxes per run (no shared, potentially-degraded state carried between submissions) and, for genuinely borderline cases, an explicit rejudge mechanism that reruns a submission under controlled conditions if its verdict is disputed. Determinism isn't achieved by hoping the system behaves consistently — it's achieved by removing every source of non-determinism one at a time.
Why this piece earns its place
There’s one source of non-determinism left that fresh sandboxes don’t touch: the environment is new every time, but it isn’t necessarily the same. Compiler version, optimization flags, standard library, base image, kernel and the limit configuration all feed the outcome, and all of them get upgraded. Bump the compiler and code that just fit inside its limit can fall outside it — same submission, same judge, different verdict, months apart. That bites hardest on the mechanism this step just introduced, because a rejudge run under today’s toolchain isn’t a rerun of the original judgement, it’s a fresh one. So I’d version the judge environment as a single artifact — image digest, compiler versions, limit config — and stamp every verdict with the version that produced it. A rejudge then means rerunning under the recorded version, and upgrading becomes a deliberate event with a decision attached: leave old verdicts alone, or mass-rejudge a problem set under the new one. That choice is a policy call about standings, and it wants answering before a contest rather than during an appeal.
Step 7 · Fair queueing under contest load
One user shouldn't starve everyone else
A contest opens. Some users submit rapidly (retrying slightly different solutions), others submit once and wait. Should the Job Queue be strictly first-come-first-served across ALL submissions from ALL users?
During a contest, should worker capacity be allocated strictly first-come-first-served across everyone's submissions?
Strict global FIFO actually allows an unfair outcome: a user who happens to submit rapidly (retrying many near-identical attempts) can consume a disproportionate share of near-term worker capacity, delaying users who each submitted only once but happened to submit a moment later.
Per-user fairness (a fair-queueing discipline, or a rate limit on how many of one user's submissions can be "in flight" being judged at once) ensures worker capacity is distributed roughly evenly across active users, regardless of how many times any single user has submitted — closer to what contest participants actually expect as "fair."
Random shuffling doesn't target the actual fairness problem (one user consuming disproportionate share) — it just adds unpredictability without guaranteeing any particular user gets a fair, bounded share of throughput.
Implement fair queueing: each user gets a bounded share of worker throughput — a per-user submission-in-flight cap or a fair-share scheduling discipline within the overall queue — so rapid resubmission by one user can't crowd out judging turnaround for everyone else during a shared, high-stakes contest window. Combine this with worker autoscaling to absorb the aggregate burst.
Why this piece earns its place
A per-user cap has an obvious setting that’s wrong. One submission in flight per user is maximally fair and hurts everybody, because it idles workers: if a handful of users are active and most worker slots are free, a strict cap leaves them doing nothing while jobs sit in the queue — fairness paid for by throwing away capacity nobody was competing for. So the share has to be work-conserving. The cap binds only while there’s contention; when workers would otherwise go idle, a user is let past it. A fair share is a floor on what you’re guaranteed when everyone wants capacity, not a ceiling on what you may use when nobody does. What’s left to tune is how hard it binds under contention, and turning that down slides you smoothly back toward the FIFO this step just rejected. The assumption underneath is that the account is the unit of fairness, which holds only while accounts are expensive to create — otherwise the key has to be something with a cost attached, like contest registration, or three sock puppets beat the discipline outright.
What the new pieces do
- Fair Queueingservice
- Ensures one user's submission flood can't starve worker capacity that the rest of a live contest needs.
Step 8 · The sharp edges
Escape attempts, queue backlogs, and cleanup
Real-world stress tests: a submission deliberately trying to escape the sandbox, a queue backlog large enough that even fair-queued wait times become noticeable, and making sure every sandbox is genuinely, completely cleaned up between runs.
Defense in depth (Step 2) means an escape attempt has to defeat CPU/memory limits, no-network, restricted filesystem access, AND process caps simultaneously — a much higher bar than defeating any single mechanism. Queue backlogs are absorbed by worker autoscaling first, and, if a backlog still grows past a threshold, users see an honest estimated wait rather than a silent stall. And every sandbox is provisioned fresh and fully destroyed — not reset, not reused — after each run, so there's no path for one submission's leftover state to affect the next.
Why this piece earns its place
Cleanup is the one step that runs after the adversary has had the machine, which makes “destroyed after every run” the claim most likely to be quietly false. A submission can leave processes that outlive the one the worker spawned, hold a mount busy so it won’t unmount, or fill the disk the sandbox was carved out of. If teardown is best-effort code inside the same worker that just ran the program, that failure is silent — a line in a log, and the worker picks up the next job on a host that is no longer clean. Two things make the claim real. Teardown happens at the boundary and from outside: kill the whole cgroup and drop the namespace and its filesystem with it, so there’s nothing to enumerate and nothing to overlook. And a teardown that doesn’t confirm is a fatal worker error, not a warning — the worker stops taking jobs, the one it was holding goes back for redelivery, and the host is recycled. A judge that can’t prove the last sandbox is gone has no business starting the next one.
You did it
You just designed an online judge.
Everything you assembled, in order
- Running untrusted code unsandboxed on shared infrastructure is a full security vulnerability — nothing else matters until this is solved.
- Real sandboxing layers CPU/memory limits, no network access, restricted filesystem access, and process caps together, with a fully ephemeral environment per run.
- A durable job queue decouples "submission accepted" from "execution finished," letting worker capacity scale independently of submission rate.
- Verdicts come from comparing output against hidden test cases — either exact match or a checker program for problems with multiple valid outputs.
- CPU-time limits (via cgroups), not wall-clock time, keep verdicts fair and consistent regardless of host contention.
- Determinism is engineered by removing sources of non-determinism one at a time — CPU-time limiting, ephemeral sandboxes, and rejudge for disputed cases.
- Fair, per-user queueing prevents one user's submission volume from starving everyone else's judging turnaround during a live contest.