feat: add database backup and restore functionality

- Add bash scripts for automated database backup and restore
- Support both full and data-only backups
- Add npm scripts for easy database management
- Add backups/ directory to .gitignore
- Include documentation for backup procedures

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Justin Edmund 2025-06-19 01:58:37 +01:00
parent b0ecd54243
commit b4f76ab3f9
6 changed files with 704 additions and 0 deletions

1
.gitignore vendored
View file

@ -31,3 +31,4 @@ vite.config.ts.timestamp-*
*storybook.log
storybook-static
backups/

View file

@ -16,6 +16,11 @@
"db:studio": "prisma studio",
"db:init": "tsx scripts/init-db.ts",
"db:deploy": "prisma migrate deploy",
"db:backup:local": "./scripts/backup-db.sh local",
"db:backup:remote": "./scripts/backup-db.sh remote",
"db:backup:sync": "./scripts/backup-db.sh sync",
"db:restore": "./scripts/restore-db.sh",
"db:backups": "./scripts/list-backups.sh",
"setup:local": "./scripts/setup-local.sh",
"storybook": "storybook dev -p 6006",
"build-storybook": "storybook build"

135
scripts/README.md Normal file
View file

@ -0,0 +1,135 @@
# Database Backup Scripts
This directory contains scripts for backing up and restoring the PostgreSQL database.
## Prerequisites
- PostgreSQL client tools (`pg_dump`, `psql`) must be installed
- Environment variables must be set in `.env` or `.env.local`:
- `DATABASE_URL` - Local database connection string
- `REMOTE_DATABASE_URL` or `DATABASE_URL_PRODUCTION` - Remote database connection string
## Available Commands
### Backup Commands
```bash
# Backup local database
npm run db:backup:local
# Backup remote database
npm run db:backup:remote
# Sync remote database to local (backs up both, then restores remote to local)
npm run db:backup:sync
# List all backups
npm run db:backups
```
### Restore Commands
```bash
# Restore a specific backup (interactive - will show available backups)
npm run db:restore
# Restore to local database (default)
npm run db:restore ./backups/backup_file.sql.gz
# Restore to remote database (requires extra confirmation)
npm run db:restore ./backups/backup_file.sql.gz remote
```
### Direct Script Usage
You can also run the scripts directly:
```bash
# Backup operations
./scripts/backup-db.sh local
./scripts/backup-db.sh remote
./scripts/backup-db.sh sync
# Restore operations
./scripts/restore-db.sh <backup-file> [local|remote]
# List backups
./scripts/list-backups.sh [all|local|remote|recent]
```
## Backup Storage
All backups are stored in the `./backups/` directory with timestamps:
- Local backups: `local_YYYYMMDD_HHMMSS.sql.gz`
- Remote backups: `remote_YYYYMMDD_HHMMSS.sql.gz`
## Safety Features
1. **Automatic Backups**: The sync operation creates backups of both databases before syncing
2. **Confirmation Prompts**: Destructive operations require confirmation
3. **Extra Protection for Remote**: Restoring to remote requires typing "RESTORE REMOTE"
4. **Compressed Storage**: Backups are automatically compressed with gzip
5. **Timestamp Naming**: All backups include timestamps to prevent overwrites
## Common Use Cases
### Daily Local Development
```bash
# Start your day by syncing the remote database to local
npm run db:backup:sync
```
### Before Deploying Changes
```bash
# Backup remote database before deploying schema changes
npm run db:backup:remote
```
### Restore from Accident
```bash
# List recent backups
npm run db:backups
# Restore a specific backup
npm run db:restore ./backups/local_20240615_143022.sql.gz
```
## Environment Variables
You can set these in `.env.local` (git-ignored) for local overrides:
```bash
# Required for local operations
DATABASE_URL="postgresql://user:password@localhost:5432/dbname"
# Required for remote operations (one of these)
REMOTE_DATABASE_URL="postgresql://user:password@remote-host:5432/dbname"
DATABASE_URL_PRODUCTION="postgresql://user:password@remote-host:5432/dbname"
```
## Troubleshooting
### "pg_dump: command not found"
Install PostgreSQL client tools:
```bash
# macOS
brew install postgresql
# Ubuntu/Debian
sudo apt-get install postgresql-client
# Arch Linux
sudo pacman -S postgresql
```
### "FATAL: password authentication failed"
Check that your database URLs are correct and include the password.
### Backup seems stuck
Large databases may take time. The scripts show progress. For very large databases, consider using `pg_dump` directly with custom options.

256
scripts/backup-db.sh Executable file
View file

@ -0,0 +1,256 @@
#!/bin/bash
# Database Backup Script
# Usage: ./scripts/backup-db.sh [local|remote|sync]
# local - Backup local database
# remote - Backup remote database
# sync - Copy remote database to local
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Load environment variables
if [ -f ".env" ]; then
set -a
source .env
set +a
fi
if [ -f ".env.local" ]; then
set -a
source .env.local
set +a
fi
# Check if required environment variables are set
if [ -z "$DATABASE_URL" ]; then
echo -e "${RED}Error: DATABASE_URL is not set${NC}"
exit 1
fi
# Parse DATABASE_URL for local database
# Format: postgresql://user:password@host:port/database
LOCAL_DB_URL=$DATABASE_URL
LOCAL_DB_NAME=$(echo $LOCAL_DB_URL | sed -E 's/.*\/([^?]+).*/\1/')
LOCAL_DB_USER=$(echo $LOCAL_DB_URL | sed -E 's/postgresql:\/\/([^:]+):.*/\1/')
LOCAL_DB_HOST=$(echo $LOCAL_DB_URL | sed -E 's/.*@([^:]+):.*/\1/')
LOCAL_DB_PORT=$(echo $LOCAL_DB_URL | sed -E 's/.*:([0-9]+)\/.*/\1/')
# Remote database URL (can be set as REMOTE_DATABASE_URL or passed as env var)
REMOTE_DB_URL=${REMOTE_DATABASE_URL:-$DATABASE_URL_PRODUCTION}
if [ -z "$REMOTE_DB_URL" ] && [ "$1" != "local" ]; then
echo -e "${YELLOW}Warning: REMOTE_DATABASE_URL or DATABASE_URL_PRODUCTION not set${NC}"
echo "For remote operations, set one of these environment variables or pass it:"
echo "REMOTE_DATABASE_URL='postgresql://...' ./scripts/backup-db.sh remote"
fi
# Create backups directory if it doesn't exist
BACKUP_DIR="./backups"
mkdir -p $BACKUP_DIR
# Generate timestamp for backup files
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
# Function to parse database URL
parse_db_url() {
local url=$1
# Debug: Show input URL
>&2 echo "Debug - Input URL: $url"
# postgresql://user:password@host:port/database
# Remove the postgresql:// prefix
local stripped=$(echo $url | sed 's|postgresql://||')
>&2 echo "Debug - Stripped: $stripped"
# Extract user:password@host:port/database
local user_pass=$(echo $stripped | cut -d@ -f1)
local host_port_db=$(echo $stripped | cut -d@ -f2)
>&2 echo "Debug - User/Pass: $user_pass"
>&2 echo "Debug - Host/Port/DB: $host_port_db"
# Extract user and password
local db_user=$(echo $user_pass | cut -d: -f1)
local db_password=$(echo $user_pass | cut -d: -f2)
# Extract host, port, and database
local host_port=$(echo $host_port_db | cut -d/ -f1)
local db_name=$(echo $host_port_db | cut -d/ -f2 | cut -d? -f1)
# Extract host and port
local db_host=$(echo $host_port | cut -d: -f1)
local db_port=$(echo $host_port | cut -d: -f2)
>&2 echo "Debug - Final parsed: host=$db_host, port=$db_port, db=$db_name, user=$db_user"
echo "$db_host|$db_port|$db_name|$db_user|$db_password"
}
# Function to backup database
backup_database() {
local db_url=$1
local backup_name=$2
local description=$3
echo -e "${GREEN}Starting backup: $description${NC}"
# Parse database URL
local parsed_url=$(parse_db_url "$db_url")
IFS='|' read -r db_host db_port db_name db_user db_password <<< "$parsed_url"
# Create backup filename
local backup_file="${BACKUP_DIR}/${backup_name}_${TIMESTAMP}.sql"
# Set PGPASSWORD to avoid password prompt
export PGPASSWORD=$db_password
# Debug: Show parsed values
echo "Debug - Parsed values:"
echo " Host: '$db_host'"
echo " Port: '$db_port'"
echo " Database: '$db_name'"
echo " User: '$db_user'"
# Run pg_dump
echo "Backing up database: $db_name from $db_host:$db_port"
pg_dump -h "$db_host" -p "$db_port" -U "$db_user" -d "$db_name" -f "$backup_file" --verbose --no-owner --no-acl
# Compress the backup
echo "Compressing backup..."
gzip $backup_file
unset PGPASSWORD
echo -e "${GREEN}Backup completed: ${backup_file}.gz${NC}"
echo "Size: $(ls -lh ${backup_file}.gz | awk '{print $5}')"
}
# Function to restore database
restore_database() {
local backup_file=$1
local target_db_url=$2
local description=$3
echo -e "${GREEN}Starting restore: $description${NC}"
# Parse database URL
local parsed_url=$(parse_db_url "$target_db_url")
IFS='|' read -r db_host db_port db_name db_user db_password <<< "$parsed_url"
# Set PGPASSWORD to avoid password prompt
export PGPASSWORD=$db_password
# Drop and recreate database
echo -e "${YELLOW}Warning: This will drop and recreate the database: $db_name${NC}"
echo -n "Are you sure you want to continue? (y/N): "
read confirm
if [ "$confirm" != "y" ] && [ "$confirm" != "Y" ]; then
echo "Restore cancelled"
return
fi
# Drop existing connections
echo "Dropping existing connections..."
psql -h $db_host -p $db_port -U $db_user -d postgres -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$db_name' AND pid <> pg_backend_pid();" 2>/dev/null || true
# Drop and recreate database
echo "Dropping database..."
psql -h $db_host -p $db_port -U $db_user -d postgres -c "DROP DATABASE IF EXISTS $db_name;"
echo "Creating database..."
psql -h $db_host -p $db_port -U $db_user -d postgres -c "CREATE DATABASE $db_name;"
# Decompress if needed
if [[ $backup_file == *.gz ]]; then
echo "Decompressing backup..."
gunzip -c $backup_file > ${backup_file%.gz}
backup_file=${backup_file%.gz}
temp_file=true
fi
# Restore database
echo "Restoring database..."
psql -h $db_host -p $db_port -U $db_user -d $db_name -f $backup_file
# Clean up temp file
if [ "$temp_file" = true ]; then
rm $backup_file
fi
unset PGPASSWORD
# Run Prisma migrations to ensure schema is up to date
echo "Running Prisma migrations..."
npm run db:deploy
echo -e "${GREEN}Restore completed${NC}"
}
# Function to sync remote to local
sync_remote_to_local() {
if [ -z "$REMOTE_DB_URL" ]; then
echo -e "${RED}Error: REMOTE_DATABASE_URL is not set${NC}"
exit 1
fi
echo -e "${GREEN}Syncing remote database to local${NC}"
# First, backup the local database
echo "Creating backup of local database first..."
backup_database "$LOCAL_DB_URL" "local_before_sync" "Local database (before sync)"
# Backup remote database
backup_database "$REMOTE_DB_URL" "remote_for_sync" "Remote database"
# Find the latest remote backup
latest_remote_backup=$(ls -t ${BACKUP_DIR}/remote_for_sync_*.sql.gz | head -1)
# Restore remote backup to local
restore_database "$latest_remote_backup" "$LOCAL_DB_URL" "Remote database to local"
}
# Main script logic
case "$1" in
"local")
backup_database "$LOCAL_DB_URL" "local" "Local database"
;;
"remote")
if [ -z "$REMOTE_DB_URL" ]; then
echo -e "${RED}Error: REMOTE_DATABASE_URL is not set${NC}"
exit 1
fi
backup_database "$REMOTE_DB_URL" "remote" "Remote database"
;;
"sync")
sync_remote_to_local
;;
*)
echo "Database Backup Utility"
echo ""
echo "Usage: $0 [local|remote|sync]"
echo ""
echo "Commands:"
echo " local - Backup local database"
echo " remote - Backup remote database"
echo " sync - Copy remote database to local (backs up both first)"
echo ""
echo "Environment variables:"
echo " DATABASE_URL - Local database connection URL (required)"
echo " REMOTE_DATABASE_URL - Remote database connection URL"
echo " DATABASE_URL_PRODUCTION - Alternative remote database URL"
echo ""
echo "Backups are stored in: ./backups/"
exit 1
;;
esac
# List recent backups
echo ""
echo "Recent backups:"
ls -lht $BACKUP_DIR/*.sql.gz 2>/dev/null | head -5 || echo "No backups found"

139
scripts/list-backups.sh Executable file
View file

@ -0,0 +1,139 @@
#!/bin/bash
# List Database Backups Script
# Usage: ./scripts/list-backups.sh [all|local|remote|recent]
set -e
# Colors for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
BACKUP_DIR="./backups"
# Check if backup directory exists
if [ ! -d "$BACKUP_DIR" ]; then
echo "No backups directory found. Run a backup first."
exit 1
fi
# Function to format file size
format_size() {
local size=$1
if [ $size -lt 1024 ]; then
echo "${size}B"
elif [ $size -lt 1048576 ]; then
echo "$((size/1024))KB"
elif [ $size -lt 1073741824 ]; then
echo "$((size/1048576))MB"
else
echo "$((size/1073741824))GB"
fi
}
# Function to list backups
list_backups() {
local pattern=$1
local title=$2
echo -e "${GREEN}${title}${NC}"
echo "----------------------------------------"
local count=0
while IFS= read -r file; do
if [ -f "$file" ]; then
local filename=$(basename "$file")
local size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file" 2>/dev/null)
local formatted_size=$(format_size $size)
local modified=$(stat -f "%Sm" -t "%Y-%m-%d %H:%M:%S" "$file" 2>/dev/null || stat -c "%y" "$file" 2>/dev/null | cut -d' ' -f1-2)
# Extract type and timestamp from filename
local type=$(echo $filename | cut -d'_' -f1)
local timestamp=$(echo $filename | grep -oE '[0-9]{8}_[0-9]{6}')
# Format timestamp
if [ ! -z "$timestamp" ]; then
local date_part=$(echo $timestamp | cut -d'_' -f1)
local time_part=$(echo $timestamp | cut -d'_' -f2)
local formatted_date="${date_part:0:4}-${date_part:4:2}-${date_part:6:2}"
local formatted_time="${time_part:0:2}:${time_part:2:2}:${time_part:4:2}"
local display_time="$formatted_date $formatted_time"
else
local display_time=$modified
fi
# Color code by type
case $type in
"local")
echo -e "${BLUE}$filename${NC}"
;;
"remote")
echo -e "${YELLOW}$filename${NC}"
;;
*)
echo "$filename"
;;
esac
echo " Size: $formatted_size | Created: $display_time"
echo ""
count=$((count + 1))
fi
done < <(ls -t $BACKUP_DIR/$pattern 2>/dev/null)
if [ $count -eq 0 ]; then
echo "No backups found"
else
echo "Total: $count backup(s)"
fi
echo ""
}
# Calculate total backup size
calculate_total_size() {
local total=0
for file in $BACKUP_DIR/*.sql.gz; do
if [ -f "$file" ]; then
local size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file" 2>/dev/null)
total=$((total + size))
fi
done
echo $(format_size $total)
}
# Main logic
case "${1:-all}" in
"all")
list_backups "*.sql.gz" "All Backups"
echo -e "${GREEN}Total backup size: $(calculate_total_size)${NC}"
;;
"local")
list_backups "local*.sql.gz" "Local Backups"
;;
"remote")
list_backups "remote*.sql.gz" "Remote Backups"
;;
"recent")
echo -e "${GREEN}Recent Backups (last 5)${NC}"
echo "----------------------------------------"
ls -lht $BACKUP_DIR/*.sql.gz 2>/dev/null | head -5 || echo "No backups found"
;;
*)
echo "Usage: $0 [all|local|remote|recent]"
echo ""
echo "Options:"
echo " all - List all backups (default)"
echo " local - List only local database backups"
echo " remote - List only remote database backups"
echo " recent - Show 5 most recent backups"
exit 1
;;
esac
# Show legend
echo ""
echo "Legend:"
echo -e " ${BLUE}Blue${NC} = Local database backup"
echo -e " ${YELLOW}Yellow${NC} = Remote database backup"

168
scripts/restore-db.sh Executable file
View file

@ -0,0 +1,168 @@
#!/bin/bash
# Database Restore Script
# Usage: ./scripts/restore-db.sh <backup-file> [local|remote]
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Load environment variables
if [ -f ".env" ]; then
set -a
source .env
set +a
fi
if [ -f ".env.local" ]; then
set -a
source .env.local
set +a
fi
# Check arguments
if [ $# -lt 1 ]; then
echo "Database Restore Utility"
echo ""
echo "Usage: $0 <backup-file> [local|remote]"
echo ""
echo "Arguments:"
echo " backup-file - Path to the backup file (.sql or .sql.gz)"
echo " target - Target database: 'local' (default) or 'remote'"
echo ""
echo "Example:"
echo " $0 ./backups/local_20240101_120000.sql.gz"
echo " $0 ./backups/remote_20240101_120000.sql.gz local"
echo ""
echo "Recent backups:"
ls -lht ./backups/*.sql.gz 2>/dev/null | head -10 || echo "No backups found"
exit 1
fi
BACKUP_FILE=$1
TARGET=${2:-local}
# Check if backup file exists
if [ ! -f "$BACKUP_FILE" ]; then
echo -e "${RED}Error: Backup file not found: $BACKUP_FILE${NC}"
exit 1
fi
# Function to parse database URL
parse_db_url() {
local url=$1
# postgresql://user:password@host:port/database
# Remove the postgresql:// prefix
local stripped=$(echo $url | sed 's|postgresql://||')
# Extract user:password@host:port/database
local user_pass=$(echo $stripped | cut -d@ -f1)
local host_port_db=$(echo $stripped | cut -d@ -f2)
# Extract user and password
local db_user=$(echo $user_pass | cut -d: -f1)
local db_password=$(echo $user_pass | cut -d: -f2)
# Extract host, port, and database
local host_port=$(echo $host_port_db | cut -d/ -f1)
local db_name=$(echo $host_port_db | cut -d/ -f2 | cut -d? -f1)
# Extract host and port
local db_host=$(echo $host_port | cut -d: -f1)
local db_port=$(echo $host_port | cut -d: -f2)
echo "$db_host|$db_port|$db_name|$db_user|$db_password"
}
# Determine target database URL
if [ "$TARGET" = "local" ]; then
TARGET_DB_URL=$DATABASE_URL
TARGET_DESC="local"
elif [ "$TARGET" = "remote" ]; then
TARGET_DB_URL=${REMOTE_DATABASE_URL:-$DATABASE_URL_PRODUCTION}
TARGET_DESC="remote"
if [ -z "$TARGET_DB_URL" ]; then
echo -e "${RED}Error: REMOTE_DATABASE_URL or DATABASE_URL_PRODUCTION not set${NC}"
exit 1
fi
else
echo -e "${RED}Error: Invalid target. Use 'local' or 'remote'${NC}"
exit 1
fi
# Parse database URL
parsed_url=$(parse_db_url "$TARGET_DB_URL")
IFS='|' read -r db_host db_port db_name db_user db_password <<< "$parsed_url"
echo -e "${GREEN}Restoring to $TARGET_DESC database${NC}"
echo "Database: $db_name"
echo "Host: $db_host:$db_port"
echo "Backup file: $BACKUP_FILE"
echo ""
# Confirmation with stronger warning for remote
if [ "$TARGET" = "remote" ]; then
echo -e "${RED}WARNING: You are about to restore to the REMOTE database!${NC}"
echo -e "${RED}This will DELETE ALL DATA in the remote database and replace it.${NC}"
echo -n "Type 'RESTORE REMOTE' to confirm: "
read confirm
if [ "$confirm" != "RESTORE REMOTE" ]; then
echo "Restore cancelled"
exit 1
fi
else
echo -e "${YELLOW}Warning: This will delete all data in the $TARGET_DESC database${NC}"
echo -n "Are you sure you want to continue? (y/N): "
read confirm
if [ "$confirm" != "y" ] && [ "$confirm" != "Y" ]; then
echo "Restore cancelled"
exit 1
fi
fi
# Set PGPASSWORD to avoid password prompt
export PGPASSWORD=$db_password
# Drop existing connections
echo "Dropping existing connections..."
psql -h $db_host -p $db_port -U $db_user -d postgres -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$db_name' AND pid <> pg_backend_pid();" 2>/dev/null || true
# Drop and recreate database
echo "Dropping database..."
psql -h $db_host -p $db_port -U $db_user -d postgres -c "DROP DATABASE IF EXISTS $db_name;"
echo "Creating database..."
psql -h $db_host -p $db_port -U $db_user -d postgres -c "CREATE DATABASE $db_name;"
# Handle compressed files
if [[ $BACKUP_FILE == *.gz ]]; then
echo "Decompressing backup..."
TEMP_FILE=$(mktemp)
gunzip -c $BACKUP_FILE > $TEMP_FILE
RESTORE_FILE=$TEMP_FILE
else
RESTORE_FILE=$BACKUP_FILE
fi
# Restore database
echo "Restoring database..."
psql -h $db_host -p $db_port -U $db_user -d $db_name -f $RESTORE_FILE
# Clean up temp file if created
if [ ! -z "$TEMP_FILE" ]; then
rm $TEMP_FILE
fi
unset PGPASSWORD
# Run Prisma migrations if restoring to local
if [ "$TARGET" = "local" ]; then
echo "Running Prisma migrations..."
npm run db:deploy
fi
echo -e "${GREEN}✓ Database restored successfully${NC}"