Every Custom Domain Routing Mistake We Made (And How We Fixed All of Them)

Building custom domain routing for a multi-tenant SaaS sounds like a solved problem. Point a CNAME, provision an SSL cert, done. It is not. This is the account of several failures in our custom domain routing multi-tenant SaaS setup - including one that silently deleted every customer domain route on every CI deploy, with no errors and no alerts.

The Setup: One Backend, 500+ Custom Domains

Every customer on our platform gets their own domain - rega.studio, defensebrief.co, and so on - but we have exactly one backend: a single Cloud Run service. The naive read is that this is simple. One origin, many CNAMEs. What it actually means is that all routing logic moves to the edge, and the edge is where things go quiet when they break.

The full routing chain looks like this:

Browser → rega.studio (Cloudflare zone, NS delegated to us)
        → CNAME: customers.contentagents.dev (proxied)
        → CNAME: proxy-fallback.contentagents.dev (proxied)
        → A: 216.239.32.21 (Google Cloud Run Load Balancer)
        → Cloud Run: content-agents-saas (React SPA + nginx)

A Cloudflare Worker sits at the edge of every customer domain. Every request passes through it. The Worker's job is specific: strip the Host header (which would confuse the origin), forward the original hostname as X-Forwarded-Host, and proxy to Cloud Run. Simple in theory. The failures came from the details.

How the Cloudflare Worker Routes Requests

The Worker itself is straightforward. Here's the full implementation:

// infra/cloudflare/tenant-router-worker.ts
export default {
  async fetch(request: Request, env: { ORIGIN_BASE_URL: string }): Promise<Response> {
    const url = new URL(request.url);
    const originalHost = request.headers.get("host") || url.hostname;
    const originUrl = new URL(url.pathname + url.search, env.ORIGIN_BASE_URL);
    const headers = new Headers(request.headers);
    headers.set("X-Forwarded-Host", originalHost);
    headers.delete("Host");
    return fetch(originUrl.toString(), { method: request.method, headers, body: request.body });
  }
};

A browser request hits rega.studio. The Worker intercepts it, reads the Host header (rega.studio), stores it, deletes the original Host header, and proxies the request to Cloud Run with X-Forwarded-Host: rega.studio attached. Cloud Run never sees a confusing hostname. The origin always knows which tenant it's serving.

This architecture is cheaper and simpler than per-tenant infrastructure. But it pushes all routing intelligence to the edge - which means when something breaks, the breakage is invisible from the origin.

The Silent Failure: CI Deploy Deletes Every Domain Route

This was the worst one. A deploy ran cleanly. No errors in the pipeline, no alerts, no obvious signal. Every custom domain went down. Customers couldn't reach their sites.

The cause: wrangler deploy. When it runs, it reads wrangler.toml, finds the [[routes]] entries, and deletes any route not listed there. This means every customer-zone route added programmatically - rega.studio/*, defensebrief.co/*, every domain we'd provisioned - gets silently removed on every CI run. No warning. No diff. Just gone.

The fix was to stop using wrangler deploy in CI entirely. Instead, we deploy the Worker script directly via the Cloudflare API, which updates the script without touching routes:

npx esbuild@latest tenant-router-worker.ts --bundle --platform=browser --format=esm --outfile=bundle.js
curl -s -X PUT \
  "https://api.cloudflare.com/client/v4/accounts/${CF_ACCOUNT_ID}/workers/scripts/content-agents-tenant-router" \
  -H "Authorization: Bearer ${CLOUDFLARE_WORKER_TOKEN}" \
  -F 'metadata={"main_module":"bundle.js","compatibility_date":"2026-05-06","bindings":[{"type":"plain_text","name":"ORIGIN_BASE_URL","text":"https://content-agents-saas-[...].run.app"}]};type=application/json' \
  -F "[email protected];type=application/javascript+module" \
  | jq '{success: .success, errors: .errors}'

Customer-zone routes added via API are now permanent across deploys. wrangler.toml still exists, but only for local development - with a comment in the file making clear that CI deployment goes through the Workers Script API, not wrangler.

The X-Original-Host Problem

Cloudflare strips the Host header when proxying between zones. The Worker forwards X-Forwarded-Host, but nginx on the origin receives Host: content-agents-saas-[...].run.app. The real customer hostname needs to reach Supabase edge functions for tenant resolution - and it wasn't getting there.

The nginx fix maps the forwarded host to a variable that gets passed downstream:

map $http_x_forwarded_host $tenant_public_host {
  default $http_x_forwarded_host;
  ""      $host;
}
# Then in every proxy_pass block:
proxy_set_header X-Original-Host $tenant_public_host;

Edge functions then read x-original-host for tenant resolution and canonical URL generation:

const incomingHost = req.headers.get("x-original-host") || req.headers.get("x-forwarded-host");
const base = incomingHost && !incomingHost.endsWith(".supabase.co") ? `https://${incomingHost}` : domain.siteUrl;

Three More Failures Worth Naming

CF Error 1413 - custom_metadata on hostname creation. Cloudflare's Custom Hostnames API rejects requests that include custom_metadata and returns error 1413 without explanation. The fix: remove custom_metadata entirely and store the tenant mapping in your own database.

Orange-to-Orange routing and SSL 525 errors. When a customer zone's CNAME points to our platform zone and both are on Cloudflare, CF uses Orange-to-Orange routing. O2O requires explicit Worker routes on the customer zone to avoid SSL negotiation failures. Our reconciler adds rega.studio/* and www.rega.studio/* Worker routes after NS delegation is confirmed.

Apex CNAME conflicts. Apex records can't coexist with an A record if you want a CNAME. Many domains arrive with A records from a previous registrar's parking page. The provisioner must delete conflicting A records before adding the apex CNAME:

const existing = await cfClient.listDnsRecords(zoneId, { type: "A", name: domain });
for (const record of existing) {
  await cfClient.deleteDnsRecord(zoneId, record.id);
}
await cfClient.addDnsRecord(zoneId, { type: "CNAME", name: domain, content: "customers.contentagents.dev", proxied: true });

The Reconciler: Accepting That Distributed Provisioning Fails Partially

When a customer connects a domain, seven parallel provisioning operations kick off: Cloudflare custom hostname SSL, SendGrid domain authentication, GSC site verification, GSC owner addition, GA4-GSC link creation, Worker route registration, and inbound mail parse setup. Any one of these can fail, stall, or succeed out of order. Assuming they all complete on the first pass is optimistic in a way that causes real problems.

The solution is an idempotent reconciler that re-checks and re-tries only the steps that haven't converged yet, running every five minutes via GitHub Actions cron. Each step reads a dns_state JSON column, runs only if its flag isn't already set, and patches the column after completion. The next run knows exactly where to pick up.

# GitHub Actions workflow, every 5 minutes:
on:
  schedule:
    - cron: '*/5 * * * *'
  workflow_dispatch:
jobs:
  reconcile:
    runs-on: ubuntu-latest
    steps:
      - name: Run reconcile
        run: |
          curl -s -X POST "${{ secrets.SUPABASE_FUNCTION_URL }}/tenant-domain-reconcile" \
            -H "x-cron-secret: ${{ secrets.CRON_SECRET }}" -d '{}'

Auth for the cron uses a shared secret rather than a JWT - the edge function checks x-cron-secret against an env variable before running:

const cronSecret = Deno.env.get("CRON_SECRET");
const providedCronSecret = req.headers.get("x-cron-secret");
const isCronCall = cronSecret && providedCronSecret === cronSecret;
if (!isCronCall && !isPlatformAdmin) {
  return json({ error: "Forbidden" }, 403);
}

One reconcile step confirms NS delegation via DNS-over-HTTPS against Cloudflare's resolver before adding Worker routes to the customer zone. If NS hasn't propagated yet, the step skips and retries on the next pass. No manual intervention, no stuck domains.

Why This Matters: Edge Routing Is Invisible Until It Breaks

Custom domain routing in multi-tenant SaaS looks like a solved problem from the outside. CNAME plus SSL cert - what else is there? The edge layer introduces failure modes that don't show up in origin logs, don't fire alerts, and don't block deploys. They just quietly stop working.

When routing logic lives at the edge - Cloudflare Workers, Lambda@Edge, any similar system - a deploy can break routing for every customer simultaneously, and nothing in your standard observability stack will catch it. The origin is healthy. The Worker is deployed. The routes are gone.

If you're building multi-tenant SaaS with custom domains, assume your CI/CD will eventually break routing. The architecture that prevents it isn't complicated: separate configuration from code, deploy the Worker script independently of route management, and build a reconciler that catches the partial failures that distributed provisioning guarantees.

Key Takeaways

• Never use wrangler deploy in CI if you manage Worker routes programmatically. It will delete every route not listed in wrangler.toml on every run, silently. Deploy the Worker script via the Cloudflare Workers Script API instead, and keep wrangler.toml for local development only.

• Preserve the original hostname through every hop. The Worker captures Host, forwards it as X-Forwarded-Host, nginx maps it to X-Original-Host, and edge functions read x-original-host for tenant resolution. Any break in that chain means the origin can't identify which tenant it's serving.

• Cloudflare's Custom Hostnames API returns error 1413 if you include custom_metadata in the request. It gives no helpful explanation. Remove it and store tenant mappings in your own database.

• Build an idempotent reconciler for domain provisioning. SSL issuance, DNS propagation, and third-party verifications are all asynchronous and all fail sometimes. An idempotent cron that re-runs only incomplete steps - reading and writing state to a JSON column - handles partial failures without manual intervention.

• Monitor at the edge, not just the origin. If your origin health checks are green but customer domains are down, the problem is in the routing layer. Add synthetic checks that hit actual customer domains from outside your network, and treat a 5xx from a customer domain as a routing incident even when the origin looks healthy.