Email, cron jobs and background workers

Transactional email providers and the DNS records they need, scheduled jobs and their alternatives, worker processes, queue basics, and delivery monitoring.

Transactional email

Do not send application email from your own web server. Deliverability depends on reputation and on a provider maintaining it, and a bad batch of password resets can damage the domain that your invoices also use.

Records the provider will ask for

  MX      for the sending subdomain, if it has a dedicated return path
  SPF     include the provider on the sending domain
  DKIM    CNAMEs or TXT records with the public key
  DMARC   starting at p=none, with an aggregate report address
  CNAME   click and open tracking domains, if you enable them

Use a subdomain for sending, e.g. mail.example.com, so a reputation
problem does not affect the root domain.

• Bounce handling is mandatory. Continuing to send to hard-bounced addresses damages the sending reputation for every customer.
• Never put a secret or a session token in a URL that gets logged in a mail client. Use a single-use token with a short expiry.
• Send through the API with a queue, not inline in a request handler. A slow mail provider should not slow down a checkout.

Scheduled jobs

# crontab on a VPS: five fields, then the command
# minute hour day-of-month month day-of-week
SHELL=/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin
[email protected]

0 3 * * *   /srv/app/bin/backup.sh >> /var/log/backup.log 2>&1
*/5 * * * * /srv/app/bin/worker-tick >> /var/log/worker.log 2>&1

1. Set the PATH explicitly. Cron's environment is minimal, and a command that works in your shell fails in cron for that reason more often than any other.
2. Redirect output somewhere. Cron mails output to the local mailbox, which on a server nobody reads is where failures go to die.
3. Use a lock so a slow run cannot overlap with the next one.
4. Make every job idempotent and safe to re-run - a machine reboot during a job should not require manual cleanup.
5. Log what the job did, not just that it ran. "Completed" with no counts is not monitoring.

# a cron wrapper with a lock and a real exit path
#!/usr/bin/env bash
set -euo pipefail

exec 9>/var/lock/app-report.lock
flock -n 9 || { echo "already running"; exit 0; }

echo "start $(date -Is)"
/srv/app/bin/report --since "1 day ago"
echo "done $(date -Is)"

Workers and queues

// a worker loop: claim, do, acknowledge, and survive a crash
while (running) {
  const job = await queue.claim({ visibilityTimeout: 300 });
  if (!job) { await sleep(1000); continue; }

  try {
    await handle(job);          // idempotent: the same job may arrive twice
    await queue.ack(job.id);
  } catch (err) {
    await queue.fail(job.id, { attempts: job.attempts + 1, error: String(err) });
    logger.error({ jobId: job.id, type: job.type, err });
  }
}

• A visibility timeout shorter than the job duration guarantees the job is delivered twice. Set it longer than the slowest expected run.
• After a few attempts, move the job to a dead-letter queue and alert. Infinite retries hide a real bug behind a growing backlog.
• Scale workers on queue depth, not on CPU. A queue that grows while CPU is idle means the workers are waiting on something external.
• Log the job id with every line the job emits, or a failure is impossible to reconstruct.

FAQ

Related

Databases, object storage and backups Monitoring, uptime and log management

Last refreshed 2026-09-18.