Debugging HTTP: tools, logs and observability
curl and API clients, browser network panel analysis, HAR capture, server access logs, tracing and correlation headers, Server-Timing, and a catalogue of common failure signatures.
curl and the network panel
# show the whole exchange, headers included, and throw the body away
curl -sv https://api.example.com/v1/orders -o /dev/null
# status, timings and size in one line, for a script
curl -sS -o body.json -w '%{http_code} %{time_connect} %{time_total} %{size_download}\n' \
https://api.example.com/v1/orders
# test a specific backend before DNS or the load balancer changes
curl --resolve api.example.com:443:203.0.113.5 https://api.example.com/health
# send a JSON body and ask for compressed responses
curl -sS -X POST https://api.example.com/v1/orders \
-H 'Content-Type: application/json' -H 'Authorization: Bearer $TOKEN' \
--data '{"sku":"A-1","qty":2}' --compressed
# force a protocol version and inspect header handling
curl -v --http2 https://example.com/ -o /dev/null| curl flag | Answers |
|---|---|
-v / --trace-ascii | What was sent and received, byte for byte |
-i | Status line and response headers with the body |
-I | HEAD request — headers only |
-w | Machine-readable metrics: status, timings, sizes |
--resolve | Point a hostname at a chosen IP |
-L / --max-redirs | Follow redirects, and how far |
-k | Skip certificate verification (debugging only) |
--compressed | Send Accept-Encoding and decode the response |
- In the browser panel, look at timing first: DNS, connect, TLS, waiting and content download separate a network problem from a slow backend.
- A preflight
OPTIONSthat fails shows up as a CORS error, not as a 4xx in the console. Filter the panel by method to see it. - The panel shows the response after decompression and after redirects, so "it works in the browser" can hide a broken intermediary.
HAR capture, logs and tracing
| Artifact | Contains | Handle with care |
|---|---|---|
| HAR file | Every request, header, body and timing from a browser session | Frequently carries cookies, tokens and personal data — redact before sharing |
| Access log | One line per request: method, path, status, duration, client | Never log query strings blindly; tokens end up there |
| Server-Timing | Per-phase duration the server chooses to expose | Cheap way to show where time went, visible in the browser panel |
| Trace context | A trace id propagated across services | Needs consistent propagation or traces stop at the first hop |
HTTP/1.1 200 OK
Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
Server-Timing: db;dur=53.2, cache;desc="MISS";dur=1.7, app;dur=12.4
X-Request-Id: 01J8Y7Q2ZK3M4N5P6R7S8T9V0W203.0.113.7 - - [18/Sep/2026:10:04:11 +0000] "GET /v1/orders?limit=50 HTTP/1.1" 200 1843 0.042 "https://app.example.com/" "Mozilla/5.0" rid=01J8Y7Q2ZK3M4N5P6R7S8T9V0W u=user-42- Log a request id on every line and return it in the response; a user can then quote it and you can find the exact exchange.
- Record the status and the duration, not just the fact that a request happened — the distribution of durations is what finds the slow endpoint.
- Propagate the trace header across service boundaries, including into message queues, or the trace ends where the RPC does.
- Redact
Authorization,CookieandSet-Cookieat the logging layer rather than trusting every developer to remember.
A catalogue of failure signatures
| Symptom | First thing to check |
|---|---|
| Works in curl, fails in the browser | CORS headers and the preflight OPTIONS response |
| Works over HTTP, breaks over HTTPS | Mixed content, HSTS, a redirect loop, or a missing intermediate certificate |
| Request hangs with no response | Framing: Content-Length mismatch, or a chunked body missing its terminating chunk |
502 from the proxy, nothing in app logs | The backend never accepted the connection — check upstream host, port and listen address |
504 only under load | A read timeout shorter than the tail of the backend's latency distribution |
| Random logouts | SameSite or Secure dropping the cookie on some navigations |
| Old content after a deploy | Cache-Control on HTML, or a cache key that ignores a changed Vary |
| Duplicates after a timeout | A non-idempotent request retried without an idempotency key |
| Truncated or corrupted response | Compression negotiated twice, or an intermediary re-encoding the body |
- Read the first error, not the last. A cascade of timeouts usually starts with one dependency that went slow, and the later errors are consequences.
- Reproduce with the smallest possible request before changing application code: the difference between a failing and a working curl command is often the entire diagnosis.
- Check the boundary in both directions: what the client sent versus what the server logged. When they disagree, an intermediary is rewriting the request.
💡
Almost every confusing HTTP problem is answered by three questions: what exactly was sent, what exactly came back, and what sat in between. Capture all three before theorising — the answer is usually visible in
curl -v or a HAR file.FAQ
Is it safe to share a HAR file with a colleague?
Not unedited. HAR files routinely contain session cookies, bearer tokens and personal data. Strip the sensitive headers and bodies, or reproduce the request with curl instead.
Why does the same request sometimes fail and sometimes succeed?
Usually load balancing across backends with different state, or an expiry boundary. Add the request id to the logs and compare a failing and a successful attempt side by side — the backend identity and the timing usually explain it.
Related
Message anatomy and framing Proxies, load balancers and connection management
Last refreshed 2026-09-18.