Runbook

Operations

Use this page as the deployment and production-readiness checklist for the service.

Configuration

Setting Purpose
AUTH_JWT_AUDIENCE Audience claim expected by downstream services.
AUTH_JWT_KEY_ID JWT key identifier exposed as kid in tokens and JWKS.
AUTH_JWT_PRIVATE_KEY_PATH Path to the RSA private key used for signing tokens.
AUTH_JWT_PUBLIC_KEY_PATH Path to the RSA public key exposed through JWKS.
AUTH_REFRESH_TOKEN_PEPPER Secret pepper for HMAC-SHA256 refresh token hashes. Changing it invalidates existing refresh sessions unless rotated deliberately.
worker-email-outbox-threads Dedicated task processor size for SMTP delivery work.
AUTH_SMTP_HOST SMTP relay hostname, for example smtp.gmail.com for local Gmail testing.
AUTH_SMTP_PORT SMTP relay port. Use 465 with implicit TLS or 587 with STARTTLS.
AUTH_SMTP_TLS_MODE tls, starttls, or none. Do not use none outside isolated local testing.
AUTH_SMTP_USERNAME SMTP authentication user. For Gmail this is normally the mailbox address.
AUTH_SMTP_APP_PASSWORD SMTP secret or app password. Store it only in the secret store and rotate it after exposure.
AUTH_SMTP_FROM_EMAIL Envelope/from address authorized by the SMTP provider and domain policy.
AUTH_SMTP_FROM_NAME Human-readable sender name shown by mail clients.

SMTP and Email Verification

Production should use a controlled SMTP relay or transactional email provider rather than a personal mailbox. Gmail app passwords are acceptable for local testing, but rotate them immediately if they are pasted into chat, committed, logged, or included in screenshots.

  • Use AUTH_SMTP_PORT: 465 with AUTH_SMTP_TLS_MODE: tls for implicit TLS providers.
  • Use AUTH_SMTP_PORT: 587 with AUTH_SMTP_TLS_MODE: starttls only when the provider greets in plain SMTP and upgrades with STARTTLS.
  • Keep AUTH_SMTP_FROM_EMAIL aligned with provider authorization, SPF, DKIM, and DMARC policy.
  • Do not log verification codes, raw outbox payloads, SMTP passwords, refresh cookies, refresh token pepper, or JWT private keys.
  • After SMTP config changes, restart the service process or container. Rebuild only when code changed.

Request a fresh verification code with the resend endpoint:

curl -i -X POST https://auth.example.com/auth/v0/verify-email/resend \
  -H 'Content-Type: application/json' \
  -d '{"email":"user@example.com"}'

Inspect outbox health in a local or restricted admin session:

SELECT id, to_email, status, attempts, next_attempt_at, locked_until,
       left(coalesce(last_error, ''), 300) AS last_error,
       created_at, updated_at
FROM email_outbox
ORDER BY created_at DESC
LIMIT 10;

Alert on sustained pending backlog, repeated retryable SMTP failures, and jobs that reach dead-letter state. Treat a growing backlog as user-facing because new users cannot complete verification.

Metrics and Alerts

Prometheus-compatible metrics are available from /service/monitor on the private monitor listener. userver supplies HTTP, task processor, PostgreSQL, and logging metrics. The service adds bounded email outbox metrics without user identifiers or other high-cardinality labels.

  • auth.email-outbox.deliveries.total uses the fixed result values sent, retry_scheduled, dead, and lease_lost.
  • auth.email-outbox.errors.total uses the fixed stage values claim and persist.
  • auth.email-outbox.processing-duration-seconds measures complete processing time for a claimed job.

Alert on dead deliveries, persistent claim or persistence errors, lease loss, and abnormal processing latency. Keep the monitor listener private and allow access only from health and metrics infrastructure.

Deployment Checklist

  • Serve the service only behind TLS.
  • Keep JWT private keys, refresh token pepper, SMTP credentials, and database passwords in a secret store.
  • Mount or generate configs/config_vars.yaml at deploy time; do not bake real secrets into the image.
  • Run the smirkly-migrate job before starting new auth service containers.
  • Run production containers with configs/static_config.prod.yaml and release binaries built with SMIRKLY_AUTH_ENABLE_TEST_CONTROL=OFF.
  • Expose the monitor listener only on a private network. Use /health/live and /health/ready for health, and /service/monitor for metrics.
  • Use config-service keys SMIRKLY_AUTH_RUNTIME_CONFIG and SMIRKLY_EMAIL_OUTBOX_RUNTIME_CONFIG for auth/outbox policy, including sign-in limits and outbox retry policy; do not put secrets, DSNs, ports, or thread counts there.
  • Seed new environments from configs/dynamic_config/values.example.json. The live config-service response must follow the shape documented in configs/dynamic_config/config_service_response.example.json.
  • Monitor dynamic config updater health and keep the cache path writable so restarts can fall back to the last successful snapshot.
  • Set CPU_LIMIT in production deployments so congestion-control can enforce real limits.
  • Configure reverse proxy request-size limits and rate limits for auth endpoints.
  • Set auth-http.trusted-proxy-cidrs to the exact ingress/load balancer CIDR ranges before trusting forwarded client IP headers.
  • Keep access logs free of passwords, refresh tokens, verification codes, and raw cookies.
  • Monitor sign-in failures, refresh-token reuse events, SMTP failures, and outbox dead jobs.

Security Controls

Edge

Apply IP and path rate limits at the proxy. Treat proxy headers as trusted only when they come from controlled infrastructure.

Application

Add account-aware limits for username, email, phone, user id, and verification target when the backing store is available.

Tokens

Rotate refresh tokens, revoke token families on reuse, and rotate JWT signing keys through JWKS with overlapping validity windows.

GitHub Pages

The repository includes a Pages workflow that publishes the static docs site. It copies docs/ and the current openapi/auth-v0.yaml into the Pages artifact.

  1. Commit docs/, openapi/auth-v0.yaml, and the Pages workflow.
  2. In repository settings, enable GitHub Pages with GitHub Actions as the source.
  3. Push to master or run the workflow manually.

Release Gate

cmake --build cmake-build-debug -j$(nproc) --target smirkly-auth smirkly-auth_unittest
# Run from a non-root user; testsuite PostgreSQL initdb refuses root.
ctest --test-dir cmake-build-debug --output-on-failure
ruby -e "require 'yaml'; YAML.load_file('openapi/auth-v0.yaml')"