Vibe Engines
YouTube

INVESTIGATION 02 · FOLLOW THE WAITING WORK

More workers.
Same bottleneck.

Traffic triples. Requests pile up. You double the application workers, and the timeouts barely move. Find the limit, test a repair, and defend what it costs.

2 practice workloads3 changed conditionsReproducible event simulationOverload practice 45–75 min · retries 25–35 min

A synthetic service model that runs in your browser. Every result is calculated from the workload, settings, and versioned assumptions below.

01 · THE CUSTOMER BRIEF

The queue is a symptom.

Harbor’s fictional support API normally receives 60 requests per second. A launch pushes that to 180 for six seconds. The application accepts work, processes it briefly, then either serves a cached read or waits for the database.

The baseline has twelve application worker slots and four database service slots. One database operation takes about 40 ms; each worker stays occupied while waiting for its operation. An additional application worker can join that wait without increasing database throughput.

The decision you owe the customer

Serve at least 95% of all offered requests within 300 ms, using at most 28 capacity points. Count rejected and timed-out requests against that target. The client gives up at 1000 ms, even when the server is still busy.

  1. 1 · AdmissionAccept work or reject it immediately when the configured limit is reached.
  2. 2 · ApplicationA FIFO ingress queue feeds a bounded number of worker slots.
  3. 3 · Cache or databaseA cache hit completes a read. Other work waits for a database service slot.
  4. 4 · The clientA useful response must arrive within the objective; a deadline can expire first.

02 · DIAGNOSE, THEN CHANGE ONE THING

Where would you add capacity?

Inspect the baseline first. Predict which queue should shrink, change one control, and save a run. A fast response for the few users admitted can hide a poor experience for everyone else.

Choose how to work
Start with a worked example

At baseline, 12 application slots wait on 4 database service slots. Compare 24 application slots with 8 database slots while keeping the arrival stream fixed. The first adds waiting capacity; the second changes modeled service capacity. Compare timely responses and capacity points.

Your independent task: Try a cache-based repair on the read-heavy workload, then test the write-heavy and cold-cache conditions. Explain which assumption fails.

Predict first. Compare one change, inspect an actual passage/request/event, then explain a condition where the repair fails. The changed-case check removes these hints.

Your next experiment

Advanced admission and queue controls
20/28 capacity points

Within the illustrative capacity budget. One point per worker, two per DB service slot, and three if caching is enabled.

Selected workload: 670/1800 → 670/1800 users timely; 20 → 20 capacity points. Rejected and timed-out users remain in the denominator.

Compared settings: No setting differences. Both columns use the same case set.

Measured results

Practice workloads

The baseline is already simulated. Save an experiment to add your comparison.

The launch burst · all offered requests count
MeasureReferenceSelected run
Useful responses ≤300 msThe denominator includes rejections and timeouts 670/1800
37.2%
—
Completed responsesMay include responses slower than the objective932—
Client timeoutsOne-second deadline from arrival868—
Rejected immediatelyAdmission or full ingress queue0—
Successful-response p95Excludes timed-out and rejected requests 878 ms —
Peak ingress / DB waitWaiting requests, recorded at every event173 / 8 —
Database busy timeAverage slot utilization over the arrival window79% —
Queries finishing after deadlineWork consumed without a successful client response486—
Capacity pointsIllustrative budget ≤2820—

Times are simulated milliseconds, independent of your browser’s speed. Capacity points are a teaching budget. They are not cloud prices.

Keep this observation with your decision

Choose a measured passage, workload, or event. The attachment keeps its run and version when newer experiments are saved. Each record holds up to six observations; existing attachments are never evicted automatically.

Open decision record →

Follow the waiting work.

An average hides the burst. Compare the queues over time, then inspect which resource was occupied while callers waited.

ORIGINAL BASELINE · The launch burst

A support API rises from 60 to 180 requests/second for six seconds, then returns to 60. Eighty percent of requests are reads eligible for caching.

Find where requests wait. More application concurrency cannot create more database slots, and a queue stores unfinished work rather than creating throughput.

1800 offered requests1418 database starts0 cached responses
Ingress and database waiting queues over time Reference peak ingress queue 173. Selected run peak ingress queue 173, database wait queue 8. Exact samples are in the table below. 1720waiting requestsload change at 4 seconds 0 s 4 s 10 s 18 s
Reference ingressReference ingressReference DB wait

Event markers: load change at 4 s · Inspect an affected request ↓

Compare capacity trade-offs and seed sensitivity

These saved configurations use the same selected workload. A dominated row has no better timely count and no lower capacity cost than another row. This comparison covers only these two objectives.

RunCapacity pointsTimely / offeredDominated by

The seed changes the specified read mix and bounded service jitter, not the arrival-rate phases. This is deterministic model sensitivity, not a production confidence interval.

Inspect exact timeline samples

Samples are spaced 250 ms apart. Peak counters use every event, so a short peak can occur between samples. The final samples include draining after arrivals stop.

Time (s)Ingress waitDB waitWorkers occupiedDB slots busyCompletedTimeoutsRejected
0.000010000
0.2500221300
0.5000332700
0.7500434200
1.0000225800
1.2500437200
1.5000438700
1.75002210300
2.00002211800
2.25003213300
2.50003214800
2.75004316200
3.00003217800
3.25002219300
3.50002220800
3.75002222300
4.00003223800
4.2515812425900
4.5034712428500
4.7555712430900
5.0075712433400
5.2594712436000
5.50115812438400
5.75133612441100
6.00153612443600
6.251726124456100
6.501726124456550
6.7517151244561000
7.0017261244561450
7.2517271244561900
7.5017271244562350
7.7517041244562800
8.0017151244563250
8.2517261244563700
8.5017261244564150
8.7517261244564600
9.0017151244565050
9.2517261244565500
9.5017261244565950
9.7517271244566400
10.0017261244566850
10.2514261244567300
10.5011271244567750
10.758261244568200
11.005261244568650
11.254061244768680
11.503081245018680
11.752071245268680
12.00961245528680
12.25051045788680
12.5000326008680
12.7500436148680
13.0000436298680
13.2500326458680
13.5000436598680
13.7500326758680
14.0000436898680
14.2500327058680
14.5000437198680
14.7500437348680
15.0000327508680
15.2500437648680
15.5000327808680
15.7500327958680
16.0000438098680
16.2500438248680
16.5000438398680
16.7500328558680
17.0000438698680
17.2500328858680
17.5000329008680
17.7500439148680
18.0000229308680
18.2500009328680
18.5000009328680
18.7500009328680
19.0000009328680
19.2500009328680
19.5000009328680
Inspect individual request examples

Up to eight failed and eight completed requests, in arrival order within each group. A timeout during a database operation leaves the operation running until completion.

Arrival 5,200 msApp 6,091 msDB 6,168 msDeadline 6,200 ms

#456 · timed-out

Arrived 5,200.0 ms · read · database path

Database wait 67.0 ms.

Deadline expired at stage: database.

Arrival 5,206 msApp 6,104 msDB 6,180 msDeadline 6,206 ms

#457 · timed-out

Arrived 5,205.6 ms · write · database path

Database wait 65.7 ms.

Deadline expired at stage: database.

Arrival 5,211 msApp 6,115 msDB 6,181 msDeadline 6,211 ms

#458 · timed-out

Arrived 5,211.1 ms · read · database path

Database wait 57.0 ms.

Deadline expired at stage: database.

Arrival 5,217 msApp 6,117 msDB 6,189 msDeadline 6,217 ms

#459 · timed-out

Arrived 5,216.7 ms · read · database path

Database wait 60.5 ms.

Deadline expired at stage: database.

Arrival 5,222 msApp 6,130 msDB 6,212 msDeadline 6,222 ms

#460 · timed-out

Arrived 5,222.2 ms · read · database path

Database wait 73.0 ms.

Deadline expired at stage: database.

Arrival 5,228 msApp 6,144 msDB 6,215 msDeadline 6,228 ms

#461 · timed-out

Arrived 5,227.8 ms · read · database path

Database wait 59.4 ms.

Deadline expired at stage: database.

Arrival 5,233 msApp 6,148 msDB 6,220 msDeadline 6,233 ms

#462 · timed-out

Arrived 5,233.3 ms · read · database path

Database wait 61.8 ms.

Deadline expired at stage: database.

Arrival 5,239 msApp 6,152 msDB 6,235 msDeadline 6,239 ms

#463 · timed-out

Arrived 5,238.9 ms · write · database path

Database wait 72.9 ms.

Deadline expired at stage: database.

Arrival 0 msApp 0 msDB 11 msDeadline 1,000 ms

#0 · completed

Arrived 0.0 ms · read · database path

Response after 57.1 ms.

Database wait 0.0 ms.

Arrival 17 msApp 17 msDB 27 msDeadline 1,017 ms

#1 · completed

Arrived 16.7 ms · read · database path

Response after 43.1 ms.

Database wait 0.0 ms.

Arrival 33 msApp 33 msDB 41 msDeadline 1,033 ms

#2 · completed

Arrived 33.3 ms · read · database path

Response after 51.5 ms.

Database wait 0.0 ms.

Arrival 50 msApp 50 msDB 62 msDeadline 1,050 ms

#3 · completed

Arrived 50.0 ms · write · database path

Response after 47.3 ms.

Database wait 0.0 ms.

Arrival 67 msApp 67 msDB 76 msDeadline 1,067 ms

#4 · completed

Arrived 66.7 ms · write · database path

Response after 43.3 ms.

Database wait 0.0 ms.

Arrival 83 msApp 83 msDB 92 msDeadline 1,083 ms

#5 · completed

Arrived 83.3 ms · write · database path

Response after 45.8 ms.

Database wait 0.0 ms.

Arrival 100 msApp 100 msDB 112 msDeadline 1,100 ms

#6 · completed

Arrived 100.0 ms · read · database path

Response after 48.4 ms.

Database wait 0.0 ms.

Arrival 117 msApp 117 msDB 125 msDeadline 1,117 ms

#7 · completed

Arrived 116.7 ms · read · database path

Response after 55.7 ms.

Database wait 0.0 ms.

03 · CHANGE THE CONDITIONS

A repair has assumptions.

Keep your configuration fixed for the first attempt at three new conditions: a cold cache, a slower database, and a graceful reduction in application concurrency. Make a prediction before revealing their results.

If you retune against these public scenarios, they become practice. Record what broke and which capacity assumption you would validate with a production load test.

Save a practice run first. These public fixtures are a learning exercise; the scenario outcome is a property of this model.

04 · DEFEND THE DESIGN

Your decision record.

0/4 checkpoints recorded. The practice objective is calculated from the simulation; written reasoning remains self-reported.

  1. Hypothesis tested Still to do

    A saved prediction can be compared with an actual event-simulation run.

  2. Practice objective measured Still to do

    The latest practice configuration serves at least 95% of all requests within 300 ms in both practice workloads, at no more than 28 capacity points.

  3. Changed conditions tested Still to do

    The three fresh scenarios have been attempted. Passing the practice objective does not imply that these conditions are safe.

  4. Decision recorded Still to do

    Diagnosis, repair, and remaining risk are written down. A reviewer still needs to inspect the reasoning.

Progress and notes save in this browser.

Exports preserve the latest 20 runs with their predictions and settings. Share a configuration to make the decision reproducible; ask another person to challenge its assumptions.

THE MECHANISM

What happens between two points on the chart?

A small event calendar, not a screenshot of a real service

The simulator schedules each arrival, application completion, database completion, and client deadline. It processes events in time order. Events that share a timestamp use a fixed order: a completion wins a tie with its deadline. A seeded hash assigns read eligibility, cache hits, and service-time variation to each request. Changing capacity preserves the incoming requests and their service requirements.

The potential cache path is assigned at arrival; a cold-cache interval applies to those arriving requests. Only a completed cached response counts as a cache hit. A database slowdown applies when a query starts, and a worker reduction restricts new dispatches while existing requests drain. These boundaries make the experiment reproducible.

The default 18-second arrival window has four seconds at 60 requests/second, six seconds at 180, then eight seconds at 60. It offers 1,800 requests. Application work has a nominal duration of 10 ms; database work has a nominal duration of 40 ms. Both vary by up to twenty percent per request. These are authored assumptions, not measurements from a cloud service.

Two queues, two different constraints

The ingress queue waits for a worker. Once dispatched, a noncached request keeps that worker while waiting for a database slot. Adding workers can move waiting work from ingress to the database queue without making the database faster. The timeline’s database-slot counter helps distinguish those states.

The nominal four-slot database can execute about 100 operations/second at 40 ms each. That estimate motivates a hypothesis; the event simulation determines actual outcomes with jitter, deadlines, workload mix, and queueing. The average utilization over the whole window can hide a fully busy burst interval.

Deadlines can waste scarce capacity

A deadline expires one second after arrival. A request still queued or doing application work is canceled in this model, releasing its worker if needed. A database operation already running continues to completion and holds both resources. Its client receives a timeout, even when the database operation later finishes successfully.

This is why completed-response latency is reported separately from the fraction of all requests served within 300 ms. A configuration that admits very few requests can look fast while disappointing most users. Read the rejection and timeout counts alongside p95.

Capacity, cacheability, and load shedding

The illustrative budget charges one point per application slot, two per database service slot, and three when caching is enabled. It represents a reserved-capacity trade-off, rather than a currency. A higher real database connection limit may increase contention instead of creating the ideal parallel capacity assumed here.

The cache applies only to eligible reads. Writes always need the database. The read-heavy practice scenario and the write-heavy practice scenario deliberately share the arrival curve, so a cache-based repair has to survive a different mix. A fixed-window admission ceiling rejects excess work before queueing; the bounded ingress queue can reject it too.

Why might twice as many workers make almost no difference?

If the database slots remain saturated, extra workers mostly wait for that same resource. Test it with unchanged database capacity and compare DB wait, useful responses, and capacity points.

Why is a low p95 insufficient?

Here p95 includes successful responses only. It leaves rejected and timed-out users out. The all-arrivals objective makes that missing experience visible.

What does a larger queue buy?

Buffering and additional waiting time, not service capacity. A larger queue may absorb a short burst, but may also keep work until its client has given up.

Common mistakes and where this shows up

Autoscaling a stateless application while its shared database is saturated, sizing for a warm cache, increasing connection pools without measuring database capacity, and judging overload only by successful-response latency all create misleading design reviews. The same reasoning appears in API gateways, webhook consumers, worker farms, and interview follow-up questions about the first bottleneck.

This model deliberately leaves out retries, cache invalidation correctness, transaction conflicts, network failures, variable payload costs, and real database contention. It holds service-slot capacity independent and does not simulate a CPU scheduler. Use it to form a testable operational hypothesis, then measure those omitted mechanisms in the actual environment.

Inspect and download the complete workload model

Version harbor-service-2026-11-13.1. Each workload uses its own fixed seed. Requests arrive evenly within each phase; read classification and service-time jitter are deterministic. Worker reduction drains existing work rather than killing it.

The launch burst

practice · read-burst · seed 127

A support API rises from 60 to 180 requests/second for six seconds, then returns to 60. Eighty percent of requests are reads eligible for caching.

Find where requests wait. More application concurrency cannot create more database slots, and a queue stores unfinished work rather than creating throughput.

The write-heavy integration

practice · write-burst · seed 211

The traffic curve stays the same, but a bulk integration changes the mix: only ten percent of requests are cacheable reads.

An apparent fix based on a high cache hit rate may stop working when the workload changes. Compare database starts and the fraction of all users served promptly.

The cache starts cold

transfer · cold-cache · seed 307

The read-heavy launch burst returns. During seconds 4–8 the cache has no hits, even if caching is configured.

A hit-rate assumption is a capacity dependency. Check the worst interval and explain what happens while the cache warms.

The database slows down

transfer · slow-database · seed 401

The same read-heavy arrival stream meets database operations that take twice as long during seconds 4–10. Existing queries keep the duration assigned when they started.

A slower dependency lowers service capacity even with the same number of connections. Timeouts after query start leave capacity occupied until the query finishes.

Half the worker slots drain

transfer · worker-drain · seed 503

During seconds 4–8, only half the configured application worker slots can accept new work. In-flight requests finish; this models a graceful capacity reduction.

Database headroom alone cannot rescue an application-concurrency bottleneck. A worker waiting for the database is still occupied.

Can load shedding be the right decision even if it misses this objective?

Yes. Rejection can protect a system and preserve useful responses when capacity is unavailable. This customer’s objective still counts those users honestly. Explain the incident response separately from the long-term capacity needed to meet the objective.

Does a passing configuration prove production readiness?

It demonstrates this specified workload and event model. Validate request mix, cache behavior, dependency service times, deadlines, and resource scaling under real load, then test changed conditions. A private production workload can differ materially from these public fixtures.

Primary sources

The principles below motivate the investigation. The fictional workload, capacity points, and simulated timings are authored separately.

Handling Overload · Google SRE →Using load shedding to avoid overload · AWS Builders’ Library →

NEXT LAYER · RETRIES ARE OFFERED LOAD

One customer operation. Several attempts.

Compare no retry, immediate retry, and bounded backoff with jitter against a finite read service. A timeout does not cancel service already started. Useful completions count original customer operations once.

Worked example and model scope

If 100 user operations each produce three attempts, the dependency sees 300 attempts, or 3× amplification. More attempts can recover transient failures and can also consume capacity while obsolete attempts are still running. Predict the effect before comparing policies.

This is a separate read-only capacity model, with FIFO waiting, 24 queue entries, an 1,800 ms original-operation budget, and seeded 0.4–1.6× service-time variation. It does not implement writes or payment deduplication. A write retry requires a valid idempotency protocol; a client timeout alone cannot establish whether a write committed.

Predict which policy will help, then run all three against the same initial operation stream.

Version read-retry-2026-11-17.1. Compare the measured result with your prediction and explain a condition that reverses your choice.

PUT THE DECISION BACK INTO THE ARCHITECTURE

Carry the constraint forward.

Use these results in the capacity decision checkpoint →