AWS RDS to Self-Hosted PostgreSQL Migration: A 15-Minute Reality

The Challenge: Minimal-Downtime RDS Migration with Enterprise Security

When you're tasked with migrating a production AWS RDS PostgreSQL database to self-hosted infrastructure, the stakes are high. My 16GB production database needed to be moved from RDS to a production server with:

- Minimal application downtime during the migration window

- IP-based access control similar to RDS security groups

- Production-grade security with encrypted connections

- Minimal performance impact on the source system during backup

- Complete automation for future migrations

The challenge? My initial local backup took 2 hours for a 16GB database. That meant 2 hours where the database would need to be offline during migration. That was unacceptable.

The Reality of Database Migration Downtime

Let's be honest about what migration downtime means:

When you migrate a database, your application can't write to the old database while you're copying data. The migration process requires:

1. Backup phase: Stop writes to source database, copy all data

2. Transfer phase: Move backup to new server

3. Restore phase: Import data into new database

4. Cutover phase: Update application connection strings

5. Validation phase: Verify everything works

This entire process is where your application sees downtime. It's not about the database being "online" technically—it's about the application being able to read and write data.

With my original 2-hour backup time, the entire migration window would be 2+ hours. That's unacceptable for production.

The Journey: From 2 Hours to 15 Minutes

Phase 1: The Standard Approach (2+ Hours Total Downtime)

My first attempt was the straightforward path - standard pg_dump from my local machine:

pg_dump \
  -h my-rds-instance.region.rds.amazonaws.com \
  -U myuser \
  -d mydb \
  --format=custom \
  --compress=6 \
  --file=backup.dump

Result: 2 hours for the backup phase alone. Total migration time: 2+ hours of application downtime.

The bottleneck was obvious:

- Single-threaded compression

- Network latency from my home internet (50 Mbps)

- No parallel processing

- CPU cores sitting idle

Phase 2: Parallel Processing Optimization (45 Minutes Backup)

I discovered PostgreSQL's parallel dump capabilities. The --jobs parameter distributes work across multiple CPU cores:

pg_dump \
  --format=directory \
  --jobs=4 \
  --compress=3 \
  --file=./backup_dir

Result: Reduced backup to ~45 minutes. Total migration time would be ~1 hour.

Better, but still a long maintenance window.

Phase 3: Compression Strategy Refinement (30 Minutes Backup)

I experimented with different compression levels to find the sweet spot:

# Test with higher compression (level 6)
pg_dump \
  --format=directory \
  --jobs=8 \
  --compress=6 \
  --file=./backup_dir

# Test with lower compression (level 3)
pg_dump \
  --format=directory \
  --jobs=8 \
  --compress=3 \
  --file=./backup_dir

Result: ~30 minutes with level 3 compression. Network bandwidth remained the limiting factor.

Key insight: Higher compression doesn't mean faster backups. Level 3 gave us 20% compression but was 40% faster than level 6.

Total migration time would be ~50 minutes of downtime. Still too long.

Phase 4: Infrastructure as the Real Solution (1.5 Minutes Backup)

The real improvement didn't come from code optimization. It came from understanding where the bottlenecks actually were.

I moved the backup process to my production server infrastructure:

- 8 CPU cores

- 1Gbps network connection to AWS

- NVMe SSD storage

Running the same commands on proper infrastructure:

pg_dump \
  --format=directory \
  --jobs=16 \
  --compress=3 \
  --verbose \
  --no-owner \
  --no-privileges \
  --no-comments \
  --file=./backup_dir

Result: 1.5 minutes for backup. Total migration time: 15 minutes.

This wasn't innovation. This was simply choosing appropriate infrastructure for the job.

Complete Migration Timeline (16GB Database)

1. Backup from RDS: 1.5 minutes (this is the critical part)

2. Transfer to production server: 30 seconds (local network)

3. Start Docker PostgreSQL: 15 seconds

4. Parallel restore: 8 minutes

5. Security configuration: 2 minutes

6. Testing & validation: 3 minutes

Total migration time: 15 minutes

With proper planning, your application can be down for just this 15-minute window. Everything else happens while the old database is still serving traffic.

Technical Architecture

System Overview

Docker PostgreSQL Stack (Production Configuration)

services:
  postgres:
    image: postgres:${POSTGRES_VERSION:-16}-alpine
    container_name: ${CONTAINER_NAME:-postgres_db}
    restart: unless-stopped

    environment:
      POSTGRES_USER: ${POSTGRES_USER:-postgres}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?required}
      POSTGRES_DB: ${POSTGRES_DB:-mydb}

    ports:
      - "${POSTGRES_PORT:-5432}:5432"

    volumes:
      - ./config/postgresql.conf:/etc/postgresql/postgresql.conf:ro
      - ./config/pg_hba.conf:/etc/postgresql/pg_hba.conf:ro
      - postgres_data:/var/lib/postgresql/data
      - ./backups:/backups:ro

    command: >
      postgres
      -c config_file=/etc/postgresql/postgresql.conf
      -c hba_file=/etc/postgresql/pg_hba.conf

    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-postgres}"]
      interval: 10s
      timeout: 5s
      retries: 5

    networks:
      - postgres_net

volumes:
  postgres_data:

networks:
  postgres_net:
    driver: bridge

Parallel Backup Implementation

#!/bin/bash
# Parallel backup from RDS to production server

set -euo pipefail

RDS_HOST="${RDS_HOST:?required}"
RDS_USER="${RDS_USER:?required}"
RDS_DB="${RDS_DB:?required}"
BACKUP_DIR="${BACKUP_DIR:-./backups}"
PARALLEL_JOBS="${PARALLEL_JOBS:-8}"

mkdir -p "$BACKUP_DIR"

BACKUP_NAME="backup_${RDS_DB}_$(date +%Y%m%d_%H%M%S)"
BACKUP_PATH="$BACKUP_DIR/$BACKUP_NAME"

echo "Starting parallel backup from RDS..."
START_TIME=$(date +%s)

PGPASSWORD="${RDS_PASSWORD}" pg_dump \
    --host="$RDS_HOST" \
    --username="$RDS_USER" \
    --dbname="$RDS_DB" \
    --format=directory \
    --jobs="$PARALLEL_JOBS" \
    --compress=3 \
    --verbose \
    --no-owner \
    --no-privileges \
    --no-comments \
    --file="$BACKUP_PATH"

END_TIME=$(date +%s)
DURATION=$((END_TIME - START_TIME))

echo "✓ Backup Complete!"
echo "Duration: ${DURATION}s ($(($DURATION / 60))m $(($DURATION % 60))s)"
echo "Location: $BACKUP_PATH"

Parallel Restore Process

#!/bin/bash
# Parallel restore into Docker PostgreSQL container

set -euo pipefail

CONTAINER_NAME="${CONTAINER_NAME:-postgres_db}"
POSTGRES_USER="${POSTGRES_USER:-postgres}"
POSTGRES_PASSWORD="${POSTGRES_PASSWORD:?required}"
TARGET_DB="${TARGET_DB:?required}"
BACKUP_FILE="${BACKUP_FILE:?required}"
PARALLEL_JOBS="${PARALLEL_JOBS:-4}"

echo "Starting parallel restore..."
START_TIME=$(date +%s)

docker exec \
    -e PGPASSWORD="$POSTGRES_PASSWORD" \
    "$CONTAINER_NAME" \
    pg_restore \
        -U "$POSTGRES_USER" \
        -d "$TARGET_DB" \
        --no-owner \
        --no-privileges \
        --jobs="$PARALLEL_JOBS" \
        --verbose \
        "/backups/$(basename "$BACKUP_FILE")"

END_TIME=$(date +%s)
DURATION=$((END_TIME - START_TIME))

echo "✓ Restore Complete!"
echo "Duration: ${DURATION}s"

Security: Enterprise-Grade Access Control

UFW Firewall Configuration

#!/bin/bash
# Configure UFW firewall for PostgreSQL access control

set -euo pipefail

# Disable Docker iptables manipulation
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json > /dev/null <<EOF
{
  "iptables": false,
  "ip-forward": true
}
EOF

sudo systemctl restart docker

# UFW Defaults - deny all incoming traffic
sudo ufw default deny incoming
sudo ufw default allow outgoing

# SSH with rate limiting
sudo ufw limit 22/tcp comment "SSH rate limited"

# PostgreSQL - Allow only specific application IPs
ALLOWED_IPS=(
    "10.0.1.50/32"         # App server 1
    "10.0.2.50/32"         # App server 2
    "192.168.1.100/32"     # Local development (if needed)
)

for IP in "${ALLOWED_IPS[@]}"; do
    sudo ufw allow from "$IP" to any port 5432 comment "PostgreSQL from $IP"
done

# Enable UFW
sudo ufw --force enable

# Verify configuration
echo "UFW Status:"
sudo ufw status numbered

pg_hba.conf Configuration

# PostgreSQL Client Authentication Configuration
# Deny by default, allow specific IPs

# Local connections
local   all             all                                     trust
host    all             all             127.0.0.1/32            scram-sha-256
host    all             all             ::1/128                 scram-sha-256

# Docker internal network
host    all             all             172.16.0.0/12           scram-sha-256

# Application servers (production)
host    mydb            appuser         10.0.1.50/32            scram-sha-256
host    mydb            appuser         10.0.2.50/32            scram-sha-256
host    mydb            readonly        10.0.0.0/8              scram-sha-256

# Deny everything else
host    all             all             0.0.0.0/0               reject
host    all             all             ::/0                    reject

CLI Tool Usage

# Start PostgreSQL container
./pg start
# Output: ✓ Container started: postgres_db

# Backup from RDS
./pg backup:rds
# Output: Duration: 90s (1m 30s)

# Restore to Docker
./pg restore
# Output: Duration: 480s (8m 0s)

# Add application server IP to whitelist
./pg ip:add 10.0.1.50/32
# Output: ✓ IP authorized

# Verify migration and health
./pg health
# Output: ✓ Container: postgres_db running
# ✓ Database: mydb ready
# ✓ Security: configured

Key Insights from This Migration

1. Infrastructure Decisions Matter More Than Code Optimization

The single biggest performance improvement came from choosing appropriate infrastructure. Moving from home internet (50 Mbps) to production-grade connectivity (1Gbps) provided an 80x improvement. No code change could achieve that.

When optimizing, look at infrastructure first. Database migrations are I/O bound, not CPU bound.

2. Parallel Processing Requires the Right Format

PostgreSQL's pg_dump doesn't parallelize with custom format. You must use directory format:

- Custom format: --format=custom (cannot parallelize)

- Directory format: --format=directory (parallelizes with --jobs=N)

3. Compression is a Trade-Off

More compression ≠ faster backups:

- Level 6: 25% file size, but slower compression

- Level 3: 20% file size, 40% faster compression

- Level 0: no compression, fastest but large files

For backups that will be transferred, level 3-5 is the sweet spot.

4. Security Requires Layering

Self-hosted PostgreSQL needs multiple security layers:

- Network level: UFW firewall rules

- Database level: pg_hba.conf access control

- Transport level: SCRAM-SHA-256 authentication

- Application level: IP whitelisting in code

No single layer is sufficient. All four are necessary.

5. Docker Networking Has Security Implications

Docker's default behavior with iptables can bypass firewall rules. Setting "iptables": false in the Docker daemon configuration is critical for proper security.

Planning Your Migration Window

Here's what a realistic migration timeline looks like:

Pre-migration (done before downtime):
- Test backup and restore on non-prod data
- Configure production server and Docker
- Prepare rollback plan
- Notify stakeholders

Migration window (15 minutes):
- Stop writes to RDS (~2 minute setup)
- Run parallel backup (1.5 minutes)
- Restore to new server (8 minutes)
- Update application connection strings (2 minutes)
- Verify data integrity (2 minutes)

Post-migration:
- Monitor for 24 hours
- Run full validation queries
- Decommission RDS instance (after confidence period)

A 15-minute migration window is practical for most applications. It's long enough to be safe, short enough to be acceptable for business hours.

Common Mistakes to Avoid

❌ Don't: Use custom format with --jobs (it won't parallelize)

✅ Do: Use directory format for parallel backups

❌ Don't: Run backups from a slow network connection

✅ Do: Run from infrastructure with good connectivity to the source database

❌ Don't: Set compression to maximum

✅ Do: Use level 3-5 for the best speed-to-compression ratio

❌ Don't: Rely on firewall alone for security

✅ Do: Layer security at multiple levels

❌ Don't: Assume Docker networking is secure by default

✅ Do: Explicitly configure firewall rules for containers

Getting Started

1. Clone the repository:

git clone https://github.com/shankarlmc/db-migration.git

cd db-migration

2. Configure your environment:

cp .env.example .env

# Edit with your RDS credentials and settings

nano .env

3. Run the migration:

chmod +x pg

./pg backup:rds

./pg start

./pg restore

4. Secure your deployment:

./pg ip:add YOUR_APP_SERVER_IP/32

./pg health

Conclusion

This is a straightforward approach to database migration. The key isn't complexity—it's understanding your bottlenecks and choosing appropriate solutions:

- Parallel processing for utilizing available CPU cores

- Proper infrastructure for network connectivity

- Layered security for protecting production data

- Automation for repeatability and reliability

With these fundamentals in place, you can migrate production PostgreSQL databases with a 15-minute maintenance window and enterprise-grade security.

For your first migration, test on non-critical data. The process is solid, but every environment is different.

---

Resources

- GitHub Repository: https://github.com/shankarlmc/db-migration

- PostgreSQL Backup Documentation: https://www.postgresql.org/docs/current/backup.html

- Docker PostgreSQL: https://hub.docker.com/_/postgres

- UFW Firewall Guide: https://help.ubuntu.com/community/UFW

---

Have you migrated PostgreSQL databases? What challenges did you face?

Share your experience in the comments—I'd like to hear about your setup and what worked for you.

---

Last updated: January 9, 2026

Found this approach useful? Share it with your DevOps team.

---