Multi-Region Deployment for nfyio

When your users span continents, a single-region deployment adds latency. Multi-region nfyio gives you data locality, lower latency, and a disaster recovery site that can take over if the primary fails.

Architecture

                    ┌─── DNS (latency-based) ───┐
                    │                            │
              ┌─────▼─────┐              ┌───────▼──────┐
              │  US-EAST   │              │   EU-WEST    │
              │            │              │              │
              │  Gateway   │◄────────────►│  Gateway     │
              │  Storage   │  replication │  Storage     │
              │  Agents    │              │  Agents      │
              │  PG + Redis│              │  PG + Redis  │
              └────────────┘              └──────────────┘
                    │                            │
                    └───────── VPC Peering ──────┘

Prerequisites

• Two (or more) server clusters in different regions
• DNS provider with latency-based or geo-routing (Cloudflare, Route 53, etc.)
• VPN or VPC peering between regions
• Identical nfyio versions deployed in each region

Step 1: Deploy the Primary Region (US-East)

Follow the standard installation guide or Kubernetes guide for the first region.

# US-East deployment
ssh us-east-server

git clone https://github.com/your-org/nfyio-infra.git
cd nfyio-infra

# Set region-specific config
cp .env.example .env

# .env (US-East)
REGION=us-east-1
SITE_ID=primary
PUBLIC_URL=https://us.nfyio.example.com
DATABASE_URL=postgresql://nfyio:password@pg-us-east:5432/nfyio
REDIS_URL=redis://redis-us-east:6379
SEAWEEDFS_MASTER=seaweedfs-master-us-east:9333
REPLICATION_PEERS=eu-west.nfyio.internal:4001

docker compose up -d

Step 2: Deploy the Secondary Region (EU-West)

ssh eu-west-server

git clone https://github.com/your-org/nfyio-infra.git
cd nfyio-infra
cp .env.example .env

# .env (EU-West)
REGION=eu-west-1
SITE_ID=secondary
PUBLIC_URL=https://eu.nfyio.example.com
DATABASE_URL=postgresql://nfyio:password@pg-eu-west:5432/nfyio
REDIS_URL=redis://redis-eu-west:6379
SEAWEEDFS_MASTER=seaweedfs-master-eu-west:9333
REPLICATION_PEERS=us-east.nfyio.internal:4001

docker compose up -d

Step 3: PostgreSQL Logical Replication

Set up bidirectional logical replication between regions.

On US-East (publisher):

-- Enable logical replication
ALTER SYSTEM SET wal_level = logical;
ALTER SYSTEM SET max_replication_slots = 4;
ALTER SYSTEM SET max_wal_senders = 4;

-- Restart PostgreSQL
-- Then create publication
CREATE PUBLICATION nfyio_pub FOR TABLE
  buckets, objects, access_keys, embeddings, user_profiles;

On EU-West (subscriber):

CREATE SUBSCRIPTION nfyio_sub
  CONNECTION 'host=us-east.nfyio.internal port=5432 dbname=nfyio user=replication password=reppassword'
  PUBLICATION nfyio_pub
  WITH (copy_data = true, synchronous_commit = 'remote_apply');

Bidirectional (EU-West publishes back):

-- On EU-West
CREATE PUBLICATION nfyio_pub FOR TABLE
  buckets, objects, access_keys, embeddings, user_profiles;

-- On US-East
CREATE SUBSCRIPTION nfyio_eu_sub
  CONNECTION 'host=eu-west.nfyio.internal port=5432 dbname=nfyio user=replication password=reppassword'
  PUBLICATION nfyio_pub;

Verify replication status:

-- On publisher
SELECT * FROM pg_stat_replication;

-- On subscriber
SELECT * FROM pg_stat_subscription;

Step 4: SeaweedFS Cross-Region Replication

SeaweedFS supports async replication between datacenters.

# On US-East master, configure replication to EU-West
weed master \
  -defaultReplication=010 \
  -peers=seaweedfs-master-eu-west:9333

Replication codes:

Code	Meaning
000	No replication
001	Replicate once in same rack
010	Replicate once in different rack (same DC)
100	Replicate once in different DC
110	Different DC + different rack

For cross-region, use 100 or 110:

weed master -defaultReplication=100

Step 5: DNS Configuration

Cloudflare (Latency-Based)

# Create latency-based DNS records
# US-East
curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records" \
  -H "Authorization: Bearer $CF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "A",
    "name": "api.nfyio.example.com",
    "content": "203.0.113.10",
    "proxied": true,
    "comment": "US-East gateway"
  }'

# EU-West
curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records" \
  -H "Authorization: Bearer $CF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "A",
    "name": "api.nfyio.example.com",
    "content": "198.51.100.20",
    "proxied": true,
    "comment": "EU-West gateway"
  }'

Enable Cloudflare Load Balancing with latency-based steering for automatic routing.

AWS Route 53

aws route53 change-resource-record-sets --hosted-zone-id $ZONE_ID \
  --change-batch '{
    "Changes": [{
      "Action": "CREATE",
      "ResourceRecordSet": {
        "Name": "api.nfyio.example.com",
        "Type": "A",
        "SetIdentifier": "us-east",
        "Region": "us-east-1",
        "TTL": 60,
        "ResourceRecords": [{"Value": "203.0.113.10"}]
      }
    }, {
      "Action": "CREATE",
      "ResourceRecordSet": {
        "Name": "api.nfyio.example.com",
        "Type": "A",
        "SetIdentifier": "eu-west",
        "Region": "eu-west-1",
        "TTL": 60,
        "ResourceRecords": [{"Value": "198.51.100.20"}]
      }
    }]
  }'

Step 6: Health Checks and Failover

Configure health checks that trigger failover:

# Health check script (runs on both regions)
#!/bin/bash
RESPONSE=$(curl -sf http://localhost:3000/health)
STATUS=$(echo "$RESPONSE" | jq -r '.status')

if [ "$STATUS" != "healthy" ]; then
  # Notify DNS provider to remove this region
  curl -X PATCH "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID" \
    -H "Authorization: Bearer $CF_TOKEN" \
    -d '{"proxied": false, "comment": "UNHEALTHY - removed from rotation"}'
  
  echo "ALERT: Region $(hostname) marked unhealthy"
  exit 1
fi

Step 7: Verify Multi-Region Setup

Test from different locations:

# From US
curl -s https://api.nfyio.example.com/health | jq '.region'
# "us-east-1"

# From Europe
curl -s https://api.nfyio.example.com/health | jq '.region'
# "eu-west-1"

# Verify data replication (create object in US, read from EU)
# 1. Create in US
curl -X PUT https://us.nfyio.example.com/api/v1/buckets/test/objects/hello.txt \
  -H "Authorization: Bearer $JWT" \
  -d "Hello from US-East"

# 2. Wait for replication
sleep 5

# 3. Read from EU
curl https://eu.nfyio.example.com/api/v1/buckets/test/objects/hello.txt \
  -H "Authorization: Bearer $JWT"
# "Hello from US-East"

Replication Lag Monitoring

Track replication lag between regions:

-- On subscriber
SELECT
  subname,
  received_lsn,
  latest_end_lsn,
  pg_size_pretty(pg_wal_lsn_diff(latest_end_lsn, received_lsn)) AS lag_bytes
FROM pg_stat_subscription;

Set up an alert if lag exceeds threshold:

# Prometheus alert rule
- alert: ReplicationLagHigh
  expr: pg_replication_lag_seconds > 30
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: "Cross-region replication lag is {{ $value }}s"

Cost Considerations

Component	Single Region	Two Regions	Notes
Compute	3 servers	6 servers	2x server cost
Storage	500 GB	1 TB	Replicated data
Network	Minimal	~$0.02/GB	Cross-region transfer
DNS	Standard	Load balancing	Cloudflare free / Route 53 $0.50/zone

Multi-region roughly doubles infrastructure cost but provides sub-100ms global latency and disaster recovery.

Key Takeaways

• PostgreSQL logical replication syncs metadata and embeddings between regions with sub-second lag
• SeaweedFS cross-datacenter replication handles object storage with configurable replication factors
• Latency-based DNS routing sends users to the nearest region automatically
• Health checks trigger automatic failover when a region becomes unhealthy
• Bidirectional replication enables active-active deployments where both regions accept writes
• Budget for approximately 2x infrastructure cost for a two-region setup

For VPC networking between regions, see the networking page. For monitoring replication health, see the Prometheus & Grafana guide.