You push a small change, the pipeline runs, and one test fails. You rerun it with zero code changes and it passes. Nothing about your app is different between those two runs. That's a flaky test, and if you've spent any real time with Playwright, you already know the feeling in your stomach when you see it.
Here's the thing nobody tells junior QA engineers: flaky usually has nothing to do with your app being broken. It means a test made an assumption about timing that only holds sometimes.
It's Rarely the App's Fault
When a test fails intermittently, the instinct is to blame the feature. Nine times out of ten, the real problem is the test clicking a button before it's actually clickable, or checking for text before the API call that produces it has resolved. The app worked fine. The test just didn't wait for it to finish working.
This distinction matters because it changes what you fix. Chasing a phantom bug in application code that isn't actually broken burns a whole afternoon for nothing.
The Usual Suspects
A few patterns show up again and again once you start tracing flaky tests back to their root cause:
- Race conditions : The test interacts with an element before the page has finished rendering it, or before a network request has settled. This is the big one, easily 70% of what I run into.
- Selectors tied to styling : A test that grabs
.btn-primary-v2breaks the moment a developer renames a class. Nothing about the feature changed, but the test did anyway. - Shared state between tests : Test B assumes the database is in whatever state Test A left it. Run them in a different order, or run them in parallel, and Test B fails for reasons that have nothing to do with Test B.
- Environment noise : Slower CI runners, cold caches, third-party APIs having a bad day. These are real, but they're also the excuse people reach for when the actual cause is one of the three above.
Fix the Waits First
Playwright already solved most of the timing problem for you, and a lot of flaky suites don't actually use that part of it. The classic mistake:
await page.click('.submit-button');
await page.waitForTimeout(3000);
expect(await page.textContent('.success-message')).toBe('Order confirmed');That waitForTimeout(3000) is a guess. Sometimes 3 seconds is plenty, sometimes the server is slow that day and it isn't, and either way you're either wasting time or still racing the page. Playwright's locators auto-wait for you, so this does the same job properly:
await page.getByRole('button', { name: 'Submit' }).click();
await expect(page.getByText('Order confirmed')).toBeVisible();expect(locator).toBeVisible() polls until the element shows up or a timeout is hit. No arbitrary sleep, no guessing. I'd guess half the flaky tests I've debugged in the last year came down to swapping a waitForTimeout for an actual assertion like this.
Isolate Your Test Data
If your login test and your checkout test both reach for test@example.com, you've built a dependency between two tests that have no business knowing about each other. One test's leftover cart data quietly wrecks the other.
The fix is unglamorous: generate unique test data per run. A timestamp in the email, a fresh user created in a beforeEach, whatever fits your setup. It's more code up front. It also means you can run your suite in parallel without tests tripping over each other, which on a CI budget matters more than it sounds like it should.
Retries Are a Band-Aid, Not a Cure
Playwright lets you configure retries in CI, and I use them, but I want to be honest about what they actually do: they hide the problem, not fix it. A test that fails once and passes on retry is still a flaky test. It just failed to fail loudly enough for anyone to notice.
I still keep retries on in CI, because a red pipeline blocking a deploy over a genuine timing hiccup is its own kind of waste. But I track how often each test needs a retry to pass. A test that retries constantly gets flagged and rewritten, not left alone because "it eventually goes green."
Where This Actually Gets You
None of this is complicated once you see it, but it took me longer than I'd like to admit to stop treating flaky tests as background noise. A test suite people don't trust is worse than no test suite at all, because at least with no tests nobody has false confidence. The moment someone on your team says "oh that test just fails sometimes, ignore it," you've lost the thing tests are supposed to give you.
Fix the waits, isolate the data, and treat retries as a signal instead of a solution. It's not glamorous work, but it's the difference between a test suite you trust and one you route around.