Monitoring, uptime and log management

Health endpoints, uptime checks and alerting, log aggregation and retention, error tracking, and the difference between uptime and correctness.

Health endpoints

// two endpoints with different jobs
app.get("/healthz", (req, res) => res.status(200).send("ok"));   // liveness: is the process alive

app.get("/readyz", async (req, res) => {                          // readiness: can it serve?
  const checks = await Promise.allSettled([
    db.query("select 1"),
    cache.ping(),
    queue.ping()
  ]);

  const failed = checks.filter((c) => c.status === "rejected").map((c) => String(c.reason));
  if (failed.length) return res.status(503).json({ ok: false, failed });

  return res.json({ ok: true });
});

• Never point a liveness probe at an endpoint that checks the database. A brief database blip would restart every instance at once, turning a degradation into an outage.
• A readiness check should be fast and cheap. It runs every few seconds on every instance.
• Keep the health endpoint out of the public route if it exposes dependency names, or return a minimal body.

Uptime checks and alerting

1. Check from outside the provider's network. A monitor inside the same platform shares its fate and will report green during an edge failure.
2. Check the user journey, not just the homepage: a login, a search, one write that is rolled back. A homepage that returns 200 while checkout is broken is not uptime.
3. Alert on symptoms users feel - error rate, latency percentiles, failed checkouts - not on every CPU spike.
4. Set thresholds that a single blip cannot cross. An alert that fires for 30 seconds of noise gets muted within a week.
5. Route alerts to a person, with a runbook link in the alert body. An alert with no owner is decoration.
6. Review the alert list monthly and delete the ones nobody acts on.

Alerting sketch

  page       error rate above 1 percent for 5 minutes
             p95 latency above 2 s for 10 minutes
             readiness failing on more than half the instances
             certificate expiring in less than 14 days
             backup job failed
             queue depth growing for 15 minutes

  ticket     disk above 80 percent
             a single failed deploy
             elevated 404 rate on a key route

  dashboard  request rate, error rate, latency, saturation - and nothing else

# an external check you can run from anywhere
curl -fsS -o /dev/null -w "%{http_code} %{time_total}s\n" https://example.com/readyz

# and one that exercises a real path
curl -fsS -o /dev/null -w "%{http_code} %{time_total}s\n" \
  "https://example.com/api/search?q=test"

Logs and error tracking

// structured logs: one JSON object per line, with the fields you will search on
logger.info({
  event: "order.created",
  orderId: order.id,
  customerId: order.customerId,
  amount: order.total,
  durationMs: Date.now() - started
});

// never log secrets, tokens or full personal data
logger.info({ event: "auth.login", userId: user.id, ip: req.ip });   // not the password, not the token

Metrics answer "how much" cheaply and over time; logs answer "what exactly happened" expensively; traces answer "where did the time go" across services. Sending everything to logs and nothing to metrics is the most common and most expensive mistake.

FAQ

Related

Databases, object storage and backups Scaling: load balancing, autoscaling and stateless design

Last refreshed 2026-09-18.