Quality gates that fail quietly, and how to find yours

Stefan-Iulian Tesoi · 21 September 2026 · 6 min read

A check exits zero without checking anything when its input is missing, when its status is swallowed by a pipe, when its selector matches nothing, or when it catches its own failure and downgrades it to a warning. The pipeline is green in all four cases and nothing was verified.

Quality gates earn trust in proportion to how rarely anyone reads their output, which is exactly the wrong direction. A gate that has printed green every day for a year is either working or absent, and from the outside those look identical.

What is a quietly failing quality gate?

One that reports success for a reason unrelated to the property it exists to protect. It is not a false negative in the usual sense — the check did not miss a defect, it never looked.

The distinction matters because the two have different repairs. A gate that misses a defect needs a better rule. A gate that never ran needs to be told that not running is a failure, and almost none of them are.

This site's build has the pattern in it, documented in the script that does it. A guard verifies the MCP tool reference against the backend's server registry, finding the backend by relative path. The repository was renamed from chatbotai_backend to backend, and the rename turned the guard into a skip that prints in green. It stayed that way until someone picked up a story whose criteria said "no SKIPPED".

Nobody disabled the check. Nobody argued for disabling it. A directory changed name, the check stopped checking, and the only visible consequence was that the build got slightly faster.

How do quality gates exit zero without checking?

Four ways, and they account for most silent build failures worth the name.

The pipe case catches experienced people, because the shell is behaving as specified. A pipeline's exit status is the status of its last command unless pipefail is set, so pnpm run ci | tail reports whether tail worked. This repository's own tooling warns about it in writing, which is a fair measure of how easy it is to do by accident.

The empty-selector case is quieter still. A runner given a glob matching nothing runs zero tests and exits 0, and 0 passing looks like every other line in a scrolling log. A rename or a typo in a workspace filter produces it, and the suite reports success with nothing in it.

How do you test a gate?

Break the thing it checks and confirm it goes red. That is the whole method, and a gate nobody has ever seen fail is a gate nobody has tested.

The exercise takes a few minutes per check and is worth scheduling rather than intending:

• Delete a required field from a config the gate validates. Run the gate. It should fail, and the message should name the field.
• Move the directory the gate reads from. This is the one that catches skips, and it is the failure that actually happened here.
• Introduce a deliberate defect of the kind the gate exists to catch — a broken link, an unsorted list, a missing translation — and confirm the failure message would let someone fix it without reading the script.
• Count what ran. A check that reports how many things it inspected turns an empty selector into a visible number. 59 url(s), 8 doc(s), 25 post(s) is a far better line than ok.

That last point is the cheapest change available. A gate that prints a count converts CI checks that pass wrongly into checks that pass suspiciously, and a suspicious number is something a person notices in a log they were already skimming.

What should a gate do when its inputs are missing?

It should fail, unless someone has decided in writing that it may skip and named where it is not allowed to. Skipping is a legitimate choice for a local build and a bad default for the run that gates a merge.

Both options cost something. A guard that hard-fails without a sibling checkout makes the repository unusable on its own, a real tax on anyone cloning one repo to fix one typo. A guard that skips keeps that case working and accepts that its protection is conditional. The mistake is not choosing the second; it is choosing it silently, so a green run means two different things depending on where it ran.

Three things make a conditional gate safe to keep:

1. The skip is loud and names what was not verified — not "skipping", but which file went unchecked against which source.
2. The permissive path is impossible in CI. Set the environment variable, or make the runner's absent dependency a hard error.
3. Someone reads for skips before believing a green run. This is the weakest of the three, because it relies on attention, and it is the one most teams are relying on exclusively.

Why does this matter more when an agent reads the result?

Because an agent asked whether the build passed will read the exit code, and the exit code is the artefact the quiet failure corrupts. A person scrolling a log has a chance of noticing the word SKIPPED in passing. A process that branches on $? has none.

That gap is not a reason to distrust agents. It is a reason to stop treating a status code as a summary of a log it no longer summarises. It is also the load-bearing assumption under definition of done when an agent wrote the code and what a coding agent should return with its work: a verification gate that cannot fail proves nothing, whoever reads it.

Laimonade closes work on evidence rather than assertion, which rests the whole arrangement on the underlying checks — and a check that cannot go red is an assertion wearing a command's clothes. A person still decides what is done. The sprint workflow covers where that sits, recurring failures are in troubleshooting, and the item-level form of the discipline is in a backlog an agent can read.

Frequently asked questions

Should a gate fail when a dependency is absent?

In CI, yes. A merge-blocking run with a conditional check is a run whose meaning depends on the machine it happened on. Locally, a loud skip is defensible, because the alternative makes a single-repo clone unusable. The rule that keeps both working is that the permissive path must be unreachable on the runner, not merely discouraged.

How often should gates be re-tested?

Whenever something moves, and on a schedule otherwise. Renames, directory moves and workspace changes are what turn checks into skips, so a repository reorganisation is the moment to re-run the break-it-on-purpose pass. Absent a trigger, once a quarter finds the ones that decayed without anyone touching them.

Can an agent be trusted to report a red gate?

Yes, and that is not the risk. An agent reports what the command returned, faithfully, including a zero that means nothing happened. The failure is upstream in the gate, and asking the agent to paste the command and its output rather than a verdict is what makes the difference visible to whoever reads it next.

Is a slow gate better than a skipped one?

Usually, though it depends on what the slowness displaces. A four-minute check people keep is worth more than an instant one that silently stopped running. A check so slow it gets moved to a nightly run nobody reads has become a skip with extra steps.