A flaky test - one that is now green, now red with no code change - is more dangerous than a failing one. A failing test honestly points at a problem; a flaky one undermines trust in the whole suite: the team gets used to rerunning until green and, along with the noise, misses real regressions. So flakiness is not 'tolerated' but investigated, and almost always behind it stands a concrete leak of non-determinism, not mysticism.
Diagnosis starts from the symptom. A test green alone and red in the suite is almost certainly shared state: one test left behind a mutated module, a write in a shared object, an unclosed mock. This is checked by permuting the order via sequence.shuffle with a fixed seed: if the failure reproduces on a specific seed, the cause is order dependence, not the test itself.
Another symptom is a suite that does not finish, hanging after the last test. This is an open handle: an unclosed database connection, a live timer, an event subscription forgotten to be removed. The test could have run and gone green, but the process holds a resource. This is cured by teardown discipline - in afterEach/afterAll you close everything you opened - and by finding the source through tools that show exactly what keeps the process from exiting.
A special category is failing only in CI, all green locally. Almost always this is the environment: a different timezone that breaks a date test; fewer cores that expose a race invisible on a fast machine; a different locale or order. This is cured by removing the dependence on the environment: pin TZ in the config, replace time-based waits with explicit event waits rather than sleep, remove implicit assumptions about parallelism.
Performance stands separately. A slow suite is a defect too: if tests take minutes, people stop running them locally, and the feedback vanishes. A frequent cause is heavy setup repeated in beforeEach where beforeAll would suffice, or a real database under unit tests of rules. The --logHeapUsage flag shows tests bloating memory and helps find leaks because of which the suite degrades toward the end of the run.
A retry is a treatment of the symptom, not the cause, and must be applied narrowly. test.retry masks flakiness: a test green on the second try is still non-deterministic, the noise is merely hidden. A retry is justified in rare places that honestly cannot be made stable - usually in E2E depending on an external environment - and there it keeps them in quarantine rather than pretending there is no problem. In unit tests a retry almost always hides a real bug.
The common method is the same as in debugging: reproduce, then fix. Non-determinism is reproduced deterministically - by fixing the order seed and the input data - and only after catching a stable failure do you remove its source: isolate state, close handles, remove the dependence on time and environment. Flakiness defeated by reproduction does not return; flakiness smeared over by a retry waits for the worst moment.
The typical failures gather into one habit - putting up with flakiness. Rerunning CI until green instead of investigating; hanging test.retry on a unit, hiding a race; leaving sleep instead of waiting for an event and being surprised by failures on a loaded runner; keeping a real database under tests of rules and complaining about a slow suite. Every flake has a cause - find it through seed and order, rather than muffling it with a rerun.
// vitest.config.ts -> test:
sequence: { shuffle: true, seed: 12345 }, // catch order dependence
// afterEach closes everything opened - otherwise the suite hangs on a handle
afterEach(async () => {
await db.close()
vi.useRealTimers()
})