Browser tests that pass on a developer laptop and fail in CI are not mysterious, they are under-observed. The problem is usually not that the test is “just flaky”, it is that the CI run is missing the artifacts needed to explain why the browser behaved differently. When you treat these failures as observability problems, you stop guessing, stop overusing retries, and start collecting evidence that can separate application bugs from environment issues, timing races, and brittle test design.

If your team wants to log browser test failures in CI in a way that actually helps, the goal is not to dump everything. It is to capture the smallest set of artifacts that reconstruct the failure path, without turning every pipeline into a storage problem. That means logging the right browser state, the right network data, the right environment metadata, and the right test runner signals, then tying them together with a run ID you can search later.

A useful rule of thumb: if you cannot answer “what changed between local and CI?” from the logs and artifacts, you do not have enough observability yet.

Why browser tests pass locally but fail in CI

The phrase “browser tests pass locally but fail in CI” hides a few different failure classes:

  • Timing differences, the CI machine is slower, the page needs more time to render, or the app is waiting on async work that finishes just fast enough on a workstation.
  • Environment differences, different viewport, locale, time zone, fonts, GPU availability, OS image, browser version, or container constraints.
  • Data differences, local test data is warmed up or manually seeded, while CI starts from a blank slate.
  • Network differences, API latency, DNS issues, third-party calls, service worker behavior, or requests blocked by a proxy.
  • Isolation differences, local state leaks through cookies, local storage, browser cache, or a previously authenticated session.
  • Test design issues, overly specific selectors, assumptions about ordering, reliance on animation completion, or hidden dependencies on real time.

The key point is that these are all diagnosable if you capture enough context. Without that context, the usual response is a retry, then a test rewrite, then a quiet acceptance of instability. That is expensive, and it often masks the actual production risk.

What a good CI failure record should answer

Before choosing tools or configuring plugins, define the questions each failed browser run should answer:

  1. Which test failed, exactly?
  2. At what step did it fail?
  3. What was on the page at the moment of failure?
  4. What browser and environment was used?
  5. What network calls happened, and which ones were slow or failed?
  6. Did the app emit console errors, exceptions, or hydration issues?
  7. Was the failure deterministic or only observed after a timeout?
  8. Can another engineer reproduce it with the same inputs?

If your logs cannot answer those questions, you are probably logging test framework noise instead of operational evidence.

The minimum artifact set worth keeping

For most teams, the useful baseline is a bundle of five artifacts per failed test, or per failed test attempt if you retry in CI:

  • Test metadata
  • Browser and environment metadata
  • Screenshot or video
  • DOM or page source snapshot
  • Console, network, and error logs

You do not need every artifact for every stack, but you should be able to reconstruct the failure from the bundle. Let’s break that down.

1) Test metadata, make the failure searchable

A failure record without good metadata is hard to correlate across jobs, branches, and retries. At minimum, log:

  • repository and branch
  • commit SHA
  • pipeline or workflow run ID
  • job name and matrix entry
  • test suite, file, and test name
  • retry count
  • timestamp with timezone
  • whether the run was local, CI, or a developer replay

This metadata is not glamorous, but it makes the difference between “we had a failure once” and “we can group all failures of the checkout flow on Chrome 124 in Linux CI last night.”

A practical format is structured JSON in a log line or artifact manifest:

{ “run_id”: “gha-88421”, “commit_sha”: “a1b2c3d”, “branch”: “main”, “job”: “e2e-chrome-linux”, “test”: “checkout.spec.ts::guest can apply discount”, “retry”: 1, “browser”: “chromium”, “browser_version”: “124.0”, “timestamp”: “2026-07-07T12:18:44Z” }

Use the same fields across all test jobs. Consistency matters more than completeness.

2) Browser and environment metadata, because CI is a different machine

If a test fails only in CI, the environment is part of the evidence. Log the exact browser and host details you ran with:

  • browser name and version
  • driver version, if applicable
  • OS image or container base image
  • CPU and memory limits
  • viewport size and device scale factor
  • locale, time zone, and language headers
  • headless or headed mode
  • whether GPU, sandboxing, or hardware acceleration is disabled
  • relevant environment variables, especially feature flags and endpoint URLs

This often exposes issues that are invisible locally. A “works on my machine” case may really be a viewport bug, a time zone assumption, or a CSS layout change triggered by a different font stack.

For example, a UI test that assumes a button is visible may pass on a large local monitor and fail in a smaller CI viewport because the element is pushed below the fold. That is not flakiness, it is an untested layout condition.

3) Screenshot, the fastest way to see state at failure time

A screenshot is the first artifact most engineers want, and for good reason. It answers a question that logs often cannot: what did the user actually see?

Capture screenshots:

  • on every failed test step if the stack supports it cheaply
  • at least once on failure, at the point of failure
  • after any major state transition, if the test is long and complex

A screenshot is especially useful for:

  • modal dialogs that block clicks
  • spinners that never finish
  • missing data or empty states
  • layout regressions that hide controls
  • auth redirects or unexpected login screens

Keep filenames deterministic and searchable:

text checkout.spec.ts__guest-can-apply-discount__retry-1.png

If you only capture screenshots after a final failure, you may miss transient states that explain the problem. Sometimes the test fails after the page already navigated or the app state changed, so the “final” screenshot is too late. In that case, capture a screenshot immediately before the action that tends to fail, or upon the first assertion failure.

4) Video, useful for timing and hover state problems

Video costs more than screenshots, but it can be the difference between speculation and certainty. Use it when the issue involves:

  • animations or transitions
  • drag-and-drop
  • hover menus
  • disappearing elements
  • delayed rendering
  • scroll-related behavior
  • intermittent modal overlays

A short failure clip is often enough. If your runner supports recording the whole test, make sure the artifact is easy to find and is only retained for failures or quarantined tests. Unbounded video retention becomes expensive quickly.

Video is less useful if you do not also have good timestamps and step logs. Otherwise you end up watching a failure without knowing which assertion triggered it.

5) DOM snapshot or page source, the state behind the screenshot

Screenshots show pixels, but many browser test failures are really about structure. Capture the DOM or rendered HTML around the failure point, ideally including the root app container and the elements under test.

This helps you diagnose:

  • a button exists in the DOM but is hidden
  • the wrong element matched a selector
  • the page contains an error boundary or fallback UI
  • the app rendered an empty list because the API payload was empty
  • the same text appears multiple times and your locator is ambiguous

For modern apps, a DOM snapshot is often more useful than raw page source, because the source after hydration tells you what the browser actually rendered. If the failure happens before full hydration, source plus client logs can show the gap.

A simple snapshot could be a trimmed HTML capture, not the whole document:

javascript

const html = await page.locator('main').evaluate((node) => node.outerHTML);
console.log(JSON.stringify({ main_html: html.slice(0, 5000) }));

Trim large snapshots, but do not trim away the actual problem.

6) Console logs, JavaScript errors, and uncaught exceptions

Browser tests often fail because the page itself is broken, but the test runner only reports a timeout or a failed click. You need to capture client-side diagnostics:

  • console warnings and errors
  • uncaught exceptions
  • unhandled promise rejections
  • page errors
  • hydration mismatch warnings
  • CSP violations
  • blocked resource or mixed content warnings

These are especially valuable in React, Vue, Angular, and other SPA stacks where a frontend error can leave the DOM in a partially interactive state.

In Playwright, for example, you can wire listeners early in the test:

page.on('console', (msg) => {
  if (msg.type() === 'error') console.error(`console: ${msg.text()}`);
});
page.on('pageerror', (err) => {
  console.error(`pageerror: ${err.message}`);
});

Do not rely only on test assertions to reveal browser-side errors. The test may fail later for an apparently unrelated reason, and you want the first error, not the final symptom.

7) Network logs, because many “UI bugs” are really data issues

When browser tests pass locally but fail in CI, network variability is a common culprit. Log the following for failed runs:

  • request URL and method
  • response status
  • request duration
  • failed requests and error text
  • slow responses above a threshold
  • redirects
  • API payloads for the specific request under test, if safe to store
  • service worker or cache involvement

This data reveals whether the UI failed because the server returned a 500, a 401, a 429, a malformed payload, or simply responded too slowly for the test’s assumptions.

A useful compromise is to capture the network trace for the failing test window, not the entire suite. In Playwright, tracing can be a strong fit when failures are intermittent, because it gives you screenshots, DOM snapshots, network data, and console messages together.

If you use a recorder or trace viewer, make sure it is part of the failure flow, not a separate manual step people forget to run. A trace that exists only when someone remembers to enable it is not reliable observability.

8) Step-level logs, not just start and end markers

A lot of CI logs are too coarse. They tell you the test started and the test failed, but not what happened in between. Add step-level logging around meaningful actions:

  • navigate
  • login
  • seed data
  • click primary action
  • wait for route change
  • assert visible content
  • submit form

Good step logs should record:

  • the action being attempted
  • the selector or locator used
  • the wait condition
  • the elapsed time
  • the result or error

Example:

typescript console.log(‘step=click_checkout_button selector=[data-testid=”checkout”]’);

await page.locator('[data-testid="checkout"]').click();
console.log('step=wait_for_confirmation');
await page.locator('text=Order confirmed').waitFor({ timeout: 5000 });

This sounds basic, but it becomes valuable when a failure happens after the action, not at the action. If the click succeeded and the page navigated to an error page, you need the step trail to see that sequence.

9) Retry history, because one failure is not always one bug

If you retry tests in CI, log every attempt separately. Do not collapse retries into a single pass/fail line. At minimum, record:

  • which attempt failed
  • whether the failure was identical
  • whether the failure shifted to a different step
  • whether artifacts differ between attempts

This helps you separate true flakes from environmental instability. If attempt one fails on a click, attempt two passes, and attempt three fails on an assertion, those are not the same event. They may represent a timing issue, a data race, or a hidden state leak.

Repeatedly retrying without preserving attempt-level evidence can make the original bug harder to reproduce than the first failure did.

10) Test data and state, the hidden source of many CI-only failures

A browser test may fail in CI because it depends on state that is not actually guaranteed. Log the data setup for the test:

  • seed version or fixture name
  • generated IDs or test user identifiers
  • pre-existing authentication state
  • feature flags enabled for the test
  • backend record counts or key entities
  • browser storage state, if relevant

If a test expects a shopping cart, invoice, or inbox to have specific contents, capture what was created before the browser opened. When the failure is due to missing or polluted data, the browser screenshot is only half the story.

A common pattern is to log a compact fixture summary:

{ “seed”: “checkout_v4”, “user”: “test-user-183”, “feature_flags”: [“new_checkout”, “tax_recalc”], “order_id”: “ord_92f1” }

When possible, generate fresh test data per run and record the identifiers. Shared test data is one of the fastest ways to create CI-only failures that are hard to reproduce locally.

What not to log, or at least not by default

Observability is not a license to capture everything. Some things are noisy, expensive, or risky to store.

Avoid or restrict:

  • full HARs for every passing test, unless you truly need them
  • unlimited videos and screenshots for green runs
  • secrets, access tokens, and session cookies in raw logs
  • entire application payloads when a small request/response excerpt would do
  • unbounded DOM dumps from very large pages

Use redaction and retention policies. A failure bundle that leaks credentials is worse than a weak failure bundle.

A practical failure bundle structure

A clean structure makes triage faster. One simple pattern is to group artifacts by run and test name:

text artifacts/ gha-88421/ checkout.spec.ts__guest-can-apply-discount/ metadata.json screenshot.png video.webm dom.html console.log network.json trace.zip

This layout works with local repros and CI uploads. It also makes it easy to attach the bundle to an issue or link it from a test report.

If your CI system has artifact retention limits, prioritize by value:

  1. metadata
  2. screenshot
  3. console and page errors
  4. network summary
  5. DOM snapshot
  6. video or trace

The exact order can change based on your stack, but the idea is to keep the cheapest, highest-signal artifacts first.

How to wire this into Playwright

Playwright is one of the easiest stacks for good CI diagnostics because it supports screenshots, traces, videos, and event hooks. A typical failure-first setup might look like this:

import { test } from '@playwright/test';

test.beforeEach(async ({ page }, testInfo) => { page.on(‘pageerror’, (err) => console.error(pageerror: ${err.message})); page.on(‘console’, (msg) => { if (msg.type() === ‘error’) console.error(console: ${msg.text()}); }); console.log(test=${testInfo.title}); });

test('guest can apply discount', async ({ page }) => {
  await page.goto('https://example.test/checkout');
  await page.locator('[data-testid="discount-code"]').fill('SAVE10');
  await page.locator('[data-testid="apply-discount"]').click();
});

In Playwright config, you can also retain artifacts on failure only:

import { defineConfig } from '@playwright/test';

export default defineConfig({ use: { screenshot: ‘only-on-failure’, video: ‘retain-on-failure’, trace: ‘retain-on-failure’ } });

That gives you a strong default without making every green run expensive.

How to wire this into Selenium

Selenium setups often need a bit more manual instrumentation. In Python, capture screenshots and browser logs on exception:

from selenium import webdriver
from selenium.webdriver.common.by import By

browser = webdriver.Chrome() try: browser.get(‘https://example.test/checkout’) browser.find_element(By.CSS_SELECTOR, ‘[data-testid=”apply-discount”]’).click() except Exception: browser.save_screenshot(‘failure.png’) print(browser.get_log(‘browser’)) raise finally: browser.quit()

Selenium can absolutely support good diagnostics, but you often need to build more of the log bundle yourself. That is fine, as long as the bundle is consistent and automatic.

CI configuration matters as much as test code

If your CI job does not preserve artifacts on failure, the best test instrumentation in the world will not help. Make sure your pipeline does three things:

  • uploads artifacts even when the job fails
  • keeps enough retention to analyze patterns across runs
  • labels artifacts with the same run ID used in logs

A GitHub Actions example for artifact upload:

- name: Upload failure artifacts
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: browser-failure-$
    path: artifacts/

Also log the browser container image or machine image used in the run. If the browser version changes under you, the failure profile may change too.

A triage workflow that uses the logs well

Once you have the artifacts, the triage path should be standardized:

  1. Confirm the failure step using step logs.
  2. Check the screenshot for obvious state differences.
  3. Inspect console errors for frontend exceptions.
  4. Review network failures or slowness.
  5. Compare DOM snapshot to selector expectations.
  6. Check environment metadata for viewport, browser version, and locale differences.
  7. Re-run locally with the same seed and flags.
  8. Decide whether this is a product bug, test bug, or environment bug.

The order matters. Many teams jump straight to rewriting the locator, but if the console shows a JavaScript exception or the network log shows a 401, the locator is not the real problem.

When a failure is probably a test problem

Some clues point to test design rather than application instability:

  • the same selector fails in multiple unrelated tests
  • the failure disappears when you slow the test down, but the app is not actually asynchronous in that area
  • the test assumes a specific order in a list that is not guaranteed
  • the test depends on invisible timing, like animation completion, instead of explicit state
  • the page is ready, but the locator is too broad or too narrow

If logs show that the app is healthy and the UI is stable, improve the test. Add explicit waits for business-relevant state, not arbitrary sleep calls. Make locators more semantic. Reduce cross-test coupling.

When a failure is probably an app or environment problem

Other clues point away from the test:

  • browser console shows an exception
  • network request fails or returns unexpected status
  • screenshot shows a login redirect or error page
  • DOM snapshot is empty because the backend did not return data
  • only one CI image or browser version is affected

In those cases, the test is doing its job by surfacing a real issue. The logs help you prove it.

A simple policy that improves flake diagnostics quickly

If you want one practical policy to start with, use this:

  • on any failed browser test in CI, capture metadata, screenshot, console logs, network summary, and DOM snapshot
  • on retries, preserve attempt-level artifacts separately
  • on repeated failures, add trace or video
  • never require a human to re-run a failure manually just to get the basics

This alone will dramatically improve your ability to log browser test failures in CI in a useful way.

The real goal is not logging, it is decision quality

Good failure logging does not just make debugging faster. It changes how teams decide what to do next. With enough evidence, you can tell whether to fix the application, harden the test, normalize the CI environment, or remove a bad dependency on external state.

That is why observability belongs at the center of browser testing. A flaky run is not solved by opinion, and it is rarely solved by a blind retry. It is solved when the evidence bundle is good enough to explain the failure without guesswork.

For teams working in software testing, test automation, and continuous integration, this is the difference between a pipeline that merely runs and a pipeline that teaches you something when it breaks.

Checklist: what to capture before you retry or rewrite

Use this as a quick review when a browser test fails in CI:

  • test name, file, and retry number
  • commit SHA and workflow run ID
  • browser, version, OS, viewport, locale, and time zone
  • screenshot at failure time
  • video or trace for intermittent or complex failures
  • console errors and page errors
  • network failures, slow requests, and status codes
  • DOM or rendered HTML snapshot
  • seed data, flags, and test user identifiers
  • artifact links that can be opened from a bug report

If you already capture most of these, the next improvement is not more logging. It is making sure the logs are structured, searchable, and consistent across all browser jobs.

The best CI test logs are not the longest ones, they are the ones that make the next debugging decision obvious.