22 Points and Two Blind Spots: Estimating the Same Four Bugs Twice

The same four bugs, estimated twice. The disagreements were worth more than the totals.

Two independent estimates of one backlog — one from an LLM-systems perspective, one from an SRE perspective. They converged on three structural calls, split on two numbers, and each identified a severe risk the other never mentioned.

The Four Bugs

All four found while standing up a two-machine Apple Silicon inference cluster.

	Bug	Why it matters
A	Activation rejects the default macOS computer name	Setup Assistant names Macs ‘s using a curly apostrophe (U+2019). The node-ID validator allows ASCII alphanumerics, dot, underscore, hyphen. Every un-renamed Mac fails at activation.
B	Correctly installed peer reported as “runtime not installed”	Interpreter discovery looks for a filename the product does not ship. The fallback path hardcodes a “not installed” verdict, which then surfaces as a diagnosis. A budgets endpoint 503s on every dashboard poll.
C	Non-coordinator answers requests silently	Every node is a full standalone server. A request to the wrong node returns a valid 200 from that node’s own model, indistinguishable from a distributed answer.
D	Coordinator loss ends the deployment, undocumented	No failover, no shard rebalance. Ranks detect the dead launcher and exit rather than hang. Correct behaviour — but written down nowhere.

D was deliberately filed as three things: document the semantics, surface the crash event, and an optional assisted re-activation. That split turned out to matter.

Two Estimates

Same inputs, same scale, no contact between them.

Effort Estimate By Lens — Story Points

Totals are not directly comparable, and that is worth being precise about. The systems lens gave 22 points, roughly 9–10 engineer-days including the re-activation feature. The SRE lens gave about 5.75 engineer-days explicitly excluding it, on the grounds that an unscoped feature should not be inside a total that someone might commit to a sprint. Strip re-activation from both and they land within about a day and a half of each other.

Where They Converged

Three structural calls, reached independently. That is the useful signal.

Split the assisted re-activation out

Both rated it 8 points — the “split it” value on the scale — and both said the same thing about why: bundling a new feature with a documentation task means the whole ticket blocks on its hardest fraction. One framed the risk as restarting a collective rendezvous after a hard process exit, with socket reuse and stale rank state unproven. The other framed it as needing a sleep-aware reachability precheck, since these machines sleep and “the same node set as last time” may not all be awake. Different reasons, same verdict.

Do not ship the refusal behaviour with the observability fix

Bug C offered three designs: response headers, a health-endpoint role field, or having a non-coordinator refuse requests outright with a redirect. Both lenses said the first two are additive and near-zero risk, and the third is a breaking behaviour change wearing an observability costume. One added the sharper point: a future implementer will pick the refusal precisely because it looks like the most complete option.

Do not build a general Unicode slugifier

The reported failure is one specific naming pattern. The operating system already computes a sanitised host name for exactly this purpose. Both said: prefer that value, treat any general-purpose slugify as a narrow fallback, and do not invite emoji and non-Latin edge cases that nobody reported.

Three independent agreements on scope, from two angles that share no vocabulary, is a stronger signal than either estimate’s arithmetic. All three were things the original tickets got wrong.

Where They Split

Two disagreements. In both cases one side was simply better informed.

Bug A — is there a migration problem?

The systems lens priced in a migration risk: node IDs are persisted in deployment records, so changing how they are derived could make a stored rank point at a different physical host. The SRE lens said flatly that no migration is needed, because activation rejects invalid IDs before anything is persisted — a bad ID could never have reached the stored record in the first place.

That is correct, and it is the sharper observation. But it does not fully dispose of the concern. The risk is not about previously-invalid IDs; it is about previously-valid ones. A machine that was manually renamed already has a valid ID today, and switching the derivation to prefer a different name source could yield a different valid ID tomorrow. Both are right about different inputs. The resolution both would accept: never silently remap — force the change through the existing content-hash approval gate, which is loud, safe and already built.

Bug C — 3 points or 5?

The systems lens called it mostly plumbing, on the assumption that role and deployment-membership state already exists and just needs exposing. The SRE lens priced the hard part explicitly: correctly distinguishing “enrolled as a non-coordinator in an active deployment” from “standalone node that was never part of one”. Without that distinction, any refusal behaviour breaks people deliberately running nodes standalone.

Having gone looking, I could find no membership signal on the worker side at all. The higher estimate is the better grounded one, and it is higher precisely because it questioned an assumption rather than inheriting it.

Blind Spot One: Fluent Wrong Answers

Seen only by the systems lens. The most severe item in the whole set.

Rank assignment is a coordination artifact. The collective library does not validate it semantically. There is no runtime check that the weights resident on rank 2 are actually shard 2.

That single observation reclassifies bug A. It looked like a cosmetic validator complaint about a curly apostrophe. It is actually the entry point to a silent-corruption path, and it is the reason any ambiguity in ID derivation has to fail loudly through the approval gate rather than being reconciled cleverly.

The same lens flagged a second one, out of scope but adjacent: fixing the runtime probe restores the check that a peer has the runtime, but nothing verifies that peers agree on library versions. Two ranks on mismatched builds in a ring collective can corrupt results or hang, rather than failing with a clean version-mismatch error.

Blind Spot Two: Fixing The Symptom Leaves The Gap

Seen only by the SRE lens. Changes what “done” means for bug B.

Bug B has an obvious fix: teach interpreter discovery about the file the product actually ships. That removes the false “runtime not installed” verdict and stops the dashboard flapping.

It does nothing about the design underneath. A peer that is genuinely asleep or offline still gets hit on a ten-second timer by every open dashboard tab, because health is evaluated on demand with no cache and no backoff. Fixing the discovery bug removes the visible symptom and leaves the cost in place — which is the ideal conditions for the work to be quietly dropped.

Two Problems Wearing One Symptom

Doing The Poll Maths

The number I had read as noise turned out to be a load statement.

I had counted 297 failed probes in one log and treated it as a count. The SRE reading turned it into a rate: at a ten-second cadence, 297 failures is roughly fifty minutes of continuous failure, from one dashboard tab on a two-node cluster. It scales with node count and with open tabs, because each poll fans out to every peer.

Then the part I had not considered at all — what each probe actually costs:

• Each probe is a full remote session: authentication, shell initialisation, and a subprocess spawned to hunt for an interpreter. Tens to low hundreds of milliseconds, not the sub-millisecond fabric latency elsewhere in the system.
• That overhead runs on the machine being monitored, competing for CPU with the inference process — on the 8 GB node, the smaller of the two.
• Authentication log entries accumulate on every peer at that rate, adding noise an operator has to filter past while debugging something unrelated.

One caveat on my own arithmetic: 297 is the total across the whole log file. The window I actually observed directly spanned a few minutes. The fifty-minute figure assumes a constant cadence across the file, which is a reasonable inference but not something I measured end to end.

What Changed As A Result

Four tickets went in. Four things about them were wrong.

Ticket	What the reviews changed
A	Reframed from a validator nit to the entry point of a silent-corruption path. Two “candidate fixes” collapsed to one preferred source with a narrow fallback. Migration framing dropped in favour of failing loudly through the existing approval gate.
B	Split conceptually in two: the discovery fix, and the health-cache and backoff work that must be ticketed separately so it does not vanish when the flapping stops.
C	Acceptance criteria narrowed to the two additive designs. Estimate raised, because the hard part is distinguishing an enrolled non-coordinator from a standalone node — and no membership signal appears to exist.
D	Split into three. Documentation is a quarter-day. Event surfacing is a day and depends on B’s plumbing. Assisted re-activation is an unscoped feature that should never have shared a ticket with a writing task.

The totals were the least interesting output. What the two lenses were actually good for was catching what the other could not see: one found a path to confidently wrong answers hiding inside a typography bug, the other found that the obvious fix would conceal its own sibling. Neither found both. A single reviewer, however senior, would have shipped with one of those two still in the backlog and no idea it was there.