Compare commits
6 Commits
e015f01cd6
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| caf482e9f4 | |||
| 09952319e8 | |||
| 52ef680771 | |||
| 9f7accba04 | |||
| 863cbd9ec9 | |||
| af0ffb2293 |
15
.env.sample
15
.env.sample
@@ -1,3 +1,12 @@
|
||||
POSTGRES_DB=qlockify
|
||||
POSTGRES_USER=postgres
|
||||
POSTGRES_PASSWORD=postgres
|
||||
POSTGRES_DB=qlockify
|
||||
POSTGRES_USER=postgres
|
||||
POSTGRES_PASSWORD=postgres
|
||||
|
||||
S3_BACKUP_BUCKET=
|
||||
S3_BACKUP_PREFIX=qlockify
|
||||
S3_BACKUP_ENDPOINT_URL=
|
||||
S3_BACKUP_ACCESS_KEY_ID=
|
||||
S3_BACKUP_SECRET_ACCESS_KEY=
|
||||
BACKUP_ENCRYPTION_PASSPHRASE=
|
||||
BACKUP_LOCAL_KEEP_LATEST=3
|
||||
BACKUP_REMOTE_KEEP_LATEST=7
|
||||
|
||||
1
.gitattributes
vendored
Normal file
1
.gitattributes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
*.sh text eol=lf
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -1,4 +1,5 @@
|
||||
.env
|
||||
backups/
|
||||
backend/logs/
|
||||
backend/.pytest_cache/
|
||||
|
||||
|
||||
106
README.md
106
README.md
@@ -45,7 +45,7 @@ Main deployed services:
|
||||
Traffic pattern:
|
||||
|
||||
- `qlockify.ir` serves the frontend
|
||||
- `api.qlockify.ir` serves the backend API, admin, docs, static, and media
|
||||
- backend traffic is served from `qlockify.ir` under `/api` and `/admin`
|
||||
- Nginx terminates TLS and proxies requests to the frontend and backend containers
|
||||
|
||||
## Expected Repository Layout
|
||||
@@ -107,18 +107,17 @@ Configured domains:
|
||||
|
||||
- `qlockify.ir`
|
||||
- `www.qlockify.ir`
|
||||
- `api.qlockify.ir`
|
||||
|
||||
Behavior:
|
||||
|
||||
- `www.qlockify.ir` redirects to `qlockify.ir`
|
||||
- `http` redirects to `https`
|
||||
- frontend is served from `qlockify.ir`
|
||||
- backend traffic is served from `api.qlockify.ir`
|
||||
- backend traffic is proxied from `qlockify.ir/api` and `qlockify.ir/admin`
|
||||
|
||||
Before production startup:
|
||||
|
||||
1. Point DNS records for `qlockify.ir`, `www.qlockify.ir`, and `api.qlockify.ir` to the server.
|
||||
1. Point DNS records for `qlockify.ir` and `www.qlockify.ir` to the server.
|
||||
2. Make sure `80` and `443` are open on the server firewall.
|
||||
3. Make sure the TLS certificate covers all required names.
|
||||
|
||||
@@ -147,10 +146,10 @@ Set these in:
|
||||
|
||||
Core production values:
|
||||
|
||||
- `DJANGO_ALLOWED_HOSTS=api.qlockify.ir,qlockify.ir,www.qlockify.ir`
|
||||
- `DJANGO_ALLOWED_HOSTS=qlockify.ir,www.qlockify.ir`
|
||||
- `CORS_ALLOWED_ORIGINS=https://qlockify.ir,https://www.qlockify.ir`
|
||||
- `CSRF_TRUSTED_ORIGINS=https://api.qlockify.ir,https://qlockify.ir,https://www.qlockify.ir`
|
||||
- `BASE_URL=https://api.qlockify.ir`
|
||||
- `CSRF_TRUSTED_ORIGINS=https://qlockify.ir,https://www.qlockify.ir`
|
||||
- `BASE_URL=https://qlockify.ir`
|
||||
- `POSTGRES_HOST=db`
|
||||
- `REDIS_HOST=redis`
|
||||
- `REDIS_URL=redis://redis:6379/0`
|
||||
@@ -161,7 +160,7 @@ Google OAuth values:
|
||||
|
||||
- `GOOGLE_OAUTH_CLIENT_ID=...`
|
||||
- `GOOGLE_OAUTH_CLIENT_SECRET=...`
|
||||
- `GOOGLE_OAUTH_REDIRECT_URI=https://api.qlockify.ir/api/users/oauth/google/callback/`
|
||||
- `GOOGLE_OAUTH_REDIRECT_URI=https://qlockify.ir/api/users/oauth/google/callback/`
|
||||
- `GOOGLE_OAUTH_FRONTEND_CALLBACK_URL=https://qlockify.ir/auth/google/callback`
|
||||
|
||||
## Required Frontend Environment
|
||||
@@ -173,7 +172,7 @@ Set this in:
|
||||
```
|
||||
|
||||
```text
|
||||
VITE_API_BASE_URL=https://api.qlockify.ir/api
|
||||
VITE_API_BASE_URL=/api
|
||||
```
|
||||
|
||||
## Background Workers
|
||||
@@ -235,6 +234,95 @@ Stop everything:
|
||||
docker compose down
|
||||
```
|
||||
|
||||
Backup the deployed data:
|
||||
|
||||
```bash
|
||||
chmod +x ./scripts/backup.sh
|
||||
./scripts/backup.sh
|
||||
```
|
||||
|
||||
By default, backup archives are written to `./backups/`. Each archive contains:
|
||||
|
||||
- PostgreSQL data from the `db` service
|
||||
- media files from the Docker media volume
|
||||
- `.env` files from the deployment, backend, and frontend projects
|
||||
|
||||
Restore a backup archive on another machine:
|
||||
|
||||
```bash
|
||||
chmod +x ./scripts/restore.sh
|
||||
./scripts/restore.sh ./backups/qlockify-backup-YYYYMMDD-HHMMSS.tar.gz
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Restore is destructive for the database by default. To restore only part of an archive, use:
|
||||
|
||||
```bash
|
||||
RESTORE_SKIP_DB=1 ./scripts/restore.sh ./backups/qlockify-backup-YYYYMMDD-HHMMSS.tar.gz
|
||||
RESTORE_SKIP_MEDIA=1 ./scripts/restore.sh ./backups/qlockify-backup-YYYYMMDD-HHMMSS.tar.gz
|
||||
RESTORE_SKIP_ENV=1 ./scripts/restore.sh ./backups/qlockify-backup-YYYYMMDD-HHMMSS.tar.gz
|
||||
```
|
||||
|
||||
Install backup upload prerequisites:
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y rclone openssl
|
||||
```
|
||||
|
||||
Configure encrypted S3-compatible backup uploads in `./.env`:
|
||||
|
||||
```bash
|
||||
S3_BACKUP_BUCKET=qlockify-backups
|
||||
S3_BACKUP_PREFIX=qlockify
|
||||
S3_BACKUP_ENDPOINT_URL=https://c284984.parspack.net
|
||||
S3_BACKUP_ACCESS_KEY_ID=
|
||||
S3_BACKUP_SECRET_ACCESS_KEY=
|
||||
BACKUP_ENCRYPTION_PASSPHRASE=
|
||||
BACKUP_LOCAL_KEEP_LATEST=3
|
||||
BACKUP_REMOTE_KEEP_LATEST=7
|
||||
```
|
||||
|
||||
Upload an encrypted backup to S3-compatible object storage with rclone:
|
||||
|
||||
```bash
|
||||
chmod +x ./scripts/backup-upload-s3.sh
|
||||
./scripts/backup-upload-s3.sh
|
||||
```
|
||||
|
||||
After a successful upload, the remote storage keeps only the latest `BACKUP_REMOTE_KEEP_LATEST` encrypted `.tar.gz.enc` files. The server keeps only the latest `BACKUP_LOCAL_KEEP_LATEST` plaintext `.tar.gz` files under `./backups/latest/`; encrypted backup files are not kept locally.
|
||||
|
||||
Restore the latest encrypted backup from S3:
|
||||
|
||||
```bash
|
||||
chmod +x ./scripts/restore-from-s3.sh
|
||||
./scripts/restore-from-s3.sh latest
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Restore a specific encrypted backup from S3:
|
||||
|
||||
```bash
|
||||
./scripts/restore-from-s3.sh qlockify/qlockify-backup-YYYYMMDD-HHMMSS.tar.gz.enc
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Schedule daily encrypted uploads with cron:
|
||||
|
||||
```cron
|
||||
0 2 * * * cd /home/ubuntu/qlockify-deployment && ./scripts/backup-upload-s3.sh >> ./backups/backup.log 2>&1
|
||||
```
|
||||
|
||||
The S3 restore script supports the same partial restore flags:
|
||||
|
||||
```bash
|
||||
RESTORE_SKIP_DB=1 ./scripts/restore-from-s3.sh latest
|
||||
RESTORE_SKIP_MEDIA=1 ./scripts/restore-from-s3.sh latest
|
||||
RESTORE_SKIP_ENV=1 ./scripts/restore-from-s3.sh latest
|
||||
```
|
||||
|
||||
Keep `BACKUP_ENCRYPTION_PASSPHRASE` in a safe place outside the server. Encrypted backups cannot be restored without it.
|
||||
|
||||
## CI/CD with Gitea Actions
|
||||
|
||||
This repository now ships with a Gitea Actions deployment workflow in:
|
||||
|
||||
@@ -44,7 +44,7 @@ LANGUAGE_CODE=en-us
|
||||
TIME_ZONE=Asia/Tehran
|
||||
|
||||
SMS_APIKEY=
|
||||
BASE_URL=https://api.qlockify.ir
|
||||
BASE_URL=https://qlockify.ir
|
||||
GOOGLE_OAUTH_CLIENT_ID=
|
||||
GOOGLE_OAUTH_CLIENT_SECRET=
|
||||
GOOGLE_OAUTH_REDIRECT_URI=https://qlockify.ir/api/users/oauth/google/callback/
|
||||
|
||||
@@ -25,9 +25,9 @@ RUN apt-get update \
|
||||
|
||||
COPY requirements/ /app/requirements/
|
||||
|
||||
RUN pip install --no-cache-dir --upgrade pip setuptools wheel \
|
||||
&& pip install --no-cache-dir -r requirements/base.txt \
|
||||
&& pip install --no-cache-dir -r requirements/prod.txt
|
||||
RUN pip install --no-cache-dir --upgrade pip setuptools wheel -i https://package-mirror.liara.ir/repository/pypi/simple \
|
||||
&& pip install --no-cache-dir -r requirements/base.txt -i https://package-mirror.liara.ir/repository/pypi/simple \
|
||||
&& pip install --no-cache-dir -r requirements/prod.txt -i https://package-mirror.liara.ir/repository/pypi/simple
|
||||
|
||||
COPY . .
|
||||
|
||||
|
||||
@@ -1,32 +1,47 @@
|
||||
services:
|
||||
db:
|
||||
image: postgres:18-alpine
|
||||
restart: always
|
||||
env_file:
|
||||
- ./backend/qlockify-backend-deployment/.env
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql
|
||||
- ./postgres/init.sql:/docker-entrypoint-initdb.d/init.sql
|
||||
command: postgres
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
ports:
|
||||
- "127.0.0.1:5432:5432" # Bound to localhost for security (as in old project)
|
||||
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
restart: always
|
||||
ports:
|
||||
- "127.0.0.1:6379:6379"
|
||||
|
||||
services:
|
||||
# proxy:
|
||||
# build:
|
||||
# context: ../proxy
|
||||
# dockerfile: Dockerfile
|
||||
# restart: unless-stopped
|
||||
# expose:
|
||||
# - "8085"
|
||||
# volumes:
|
||||
# - ../proxy/config.json:/app/config.json:ro
|
||||
# - ../proxy/ca:/app/ca
|
||||
# environment:
|
||||
# PYTHONUNBUFFERED: "1"
|
||||
# PROXY_HOST: "0.0.0.0"
|
||||
# PROXY_PORT: "8085"
|
||||
|
||||
db:
|
||||
image: postgres:18-alpine
|
||||
restart: always
|
||||
env_file:
|
||||
- ./backend/qlockify-backend-deployment/.env
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql
|
||||
- ./postgres/init.sql:/docker-entrypoint-initdb.d/init.sql
|
||||
command: postgres
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
ports:
|
||||
- "127.0.0.1:5432:5432" # Bound to localhost for security (as in old project)
|
||||
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
restart: always
|
||||
ports:
|
||||
- "127.0.0.1:6379:6379"
|
||||
|
||||
backend:
|
||||
build:
|
||||
context: ./backend/qlockify-backend-deployment
|
||||
dockerfile: ../Dockerfile
|
||||
restart: always
|
||||
build:
|
||||
context: ./backend/qlockify-backend-deployment
|
||||
dockerfile: ../Dockerfile
|
||||
restart: always
|
||||
env_file:
|
||||
- ./backend/qlockify-backend-deployment/.env
|
||||
volumes:
|
||||
@@ -42,27 +57,36 @@ services:
|
||||
--threads 8
|
||||
--timeout 120
|
||||
--keep-alive 75"
|
||||
expose:
|
||||
- "8000"
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_started
|
||||
|
||||
expose:
|
||||
- "8000"
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_started
|
||||
# proxy:
|
||||
# condition: service_started
|
||||
# environment:
|
||||
# HTTP_PROXY: "http://proxy:8085"
|
||||
# HTTPS_PROXY: "http://proxy:8085"
|
||||
# http_proxy: "http://proxy:8085"
|
||||
# https_proxy: "http://proxy:8085"
|
||||
# NO_PROXY: "localhost,127.0.0.1,db,redis,proxy"
|
||||
# no_proxy: "localhost,127.0.0.1,db,redis,proxy"
|
||||
|
||||
celery:
|
||||
build:
|
||||
context: ./backend/qlockify-backend-deployment
|
||||
dockerfile: ../Dockerfile
|
||||
restart: always
|
||||
restart: always
|
||||
env_file:
|
||||
- ./backend/qlockify-backend-deployment/.env
|
||||
volumes:
|
||||
- media_data:/app/media
|
||||
command: celery -A config worker -l INFO
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
command: celery -A config worker -l INFO
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_started
|
||||
backend:
|
||||
@@ -89,13 +113,13 @@ services:
|
||||
frontend:
|
||||
build:
|
||||
context: ./frontend/qlockify-frontend-deployment
|
||||
dockerfile: ../Dockerfile
|
||||
restart: always
|
||||
env_file:
|
||||
- ./frontend/qlockify-frontend-deployment/.env
|
||||
expose:
|
||||
- "80"
|
||||
|
||||
dockerfile: ../Dockerfile
|
||||
restart: always
|
||||
env_file:
|
||||
- ./frontend/qlockify-frontend-deployment/.env
|
||||
expose:
|
||||
- "80"
|
||||
|
||||
nginx:
|
||||
image: nginx:alpine
|
||||
restart: always
|
||||
@@ -108,10 +132,10 @@ services:
|
||||
- ./nginx/.htpasswd:/etc/nginx/.htpasswd:ro
|
||||
- static_data:/usr/share/nginx/html/static:ro
|
||||
- media_data:/usr/share/nginx/html/media:ro
|
||||
depends_on:
|
||||
- backend
|
||||
- frontend
|
||||
|
||||
depends_on:
|
||||
- backend
|
||||
- frontend
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
static_data:
|
||||
|
||||
213
scripts/backup-upload-s3.sh
Executable file
213
scripts/backup-upload-s3.sh
Executable file
@@ -0,0 +1,213 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
backup-upload-s3.sh
|
||||
|
||||
Environment:
|
||||
DEPLOY_ROOT Deployment directory. Defaults to ~/qlockify-deployment.
|
||||
|
||||
S3_BACKUP_ENDPOINT_URL S3 endpoint URL.
|
||||
S3_BACKUP_ACCESS_KEY_ID S3 access key.
|
||||
S3_BACKUP_SECRET_ACCESS_KEY S3 secret key.
|
||||
S3_BACKUP_BUCKET Bucket name.
|
||||
S3_BACKUP_PREFIX Object prefix. Defaults to qlockify.
|
||||
|
||||
BACKUP_ENCRYPTION_PASSPHRASE Encryption passphrase.
|
||||
BACKUP_LOCAL_KEEP_LATEST Number of latest local plaintext backups to keep. Defaults to 3.
|
||||
BACKUP_REMOTE_KEEP_LATEST Number of latest remote encrypted backups to keep. Defaults to 7.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[backup-rclone] %s\n' "$*"
|
||||
}
|
||||
|
||||
fail() {
|
||||
printf '[backup-rclone] %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
require_var() {
|
||||
local name="$1"
|
||||
[[ -n "${!name:-}" ]] || fail "$name is required"
|
||||
}
|
||||
|
||||
load_env() {
|
||||
local env_path="$1"
|
||||
|
||||
[[ -f "$env_path" ]] || fail "Deployment env file not found: $env_path"
|
||||
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
. "$env_path"
|
||||
set +a
|
||||
}
|
||||
|
||||
normalize_prefix() {
|
||||
local value="${1:-qlockify}"
|
||||
|
||||
value="${value#/}"
|
||||
value="${value%/}"
|
||||
|
||||
printf '%s' "$value"
|
||||
}
|
||||
|
||||
positive_integer_or_default() {
|
||||
local value="${1:-}"
|
||||
local fallback="$2"
|
||||
|
||||
if [[ "$value" =~ ^[0-9]+$ ]]; then
|
||||
printf '%s' "$value"
|
||||
else
|
||||
printf '%s' "$fallback"
|
||||
fi
|
||||
}
|
||||
|
||||
rclone_remote_path() {
|
||||
printf 'parspack:%s/%s' "$S3_BACKUP_BUCKET" "$S3_BACKUP_PREFIX"
|
||||
}
|
||||
|
||||
cleanup_local_backups() {
|
||||
local keep_count="$1"
|
||||
local latest_dir="$2"
|
||||
|
||||
[[ "$keep_count" -gt 0 ]] || {
|
||||
rm -rf "$latest_dir"
|
||||
return
|
||||
}
|
||||
|
||||
mkdir -p "$latest_dir"
|
||||
find "$latest_dir" -maxdepth 1 -type f -name '*.tar.gz.enc' -delete
|
||||
cp "$ARCHIVE_PATH" "$latest_dir/$ARCHIVE_NAME"
|
||||
|
||||
find "$latest_dir" -maxdepth 1 -type f -name '*.tar.gz' -printf '%T@ %p\n' \
|
||||
| sort -nr \
|
||||
| awk -v keep="$keep_count" 'NR > keep {print $2}' \
|
||||
| xargs -r rm -f
|
||||
|
||||
log "Kept latest $keep_count plaintext backup(s) locally in: $latest_dir"
|
||||
}
|
||||
|
||||
cleanup_remote_backups() {
|
||||
local keep_count="$1"
|
||||
local remote_path="$2"
|
||||
local stale_files
|
||||
|
||||
[[ "$keep_count" -gt 0 ]] || {
|
||||
log "Skipping remote cleanup because BACKUP_REMOTE_KEEP_LATEST is $keep_count"
|
||||
return
|
||||
}
|
||||
|
||||
stale_files="$(
|
||||
rclone lsf "$remote_path" \
|
||||
--config "$RCLONE_CONFIG" \
|
||||
--s3-no-check-bucket \
|
||||
--files-only \
|
||||
| grep '\.tar\.gz\.enc$' \
|
||||
| sort -r \
|
||||
| awk -v keep="$keep_count" 'NR > keep' \
|
||||
|| true
|
||||
)"
|
||||
|
||||
if [[ -z "$stale_files" ]]; then
|
||||
log "No remote backups need cleanup"
|
||||
return
|
||||
fi
|
||||
|
||||
while IFS= read -r stale_file; do
|
||||
[[ -n "$stale_file" ]] || continue
|
||||
log "Deleting old remote backup: $remote_path/$stale_file"
|
||||
rclone deletefile "$remote_path/$stale_file" \
|
||||
--config "$RCLONE_CONFIG" \
|
||||
--s3-no-check-bucket
|
||||
done <<< "$stale_files"
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
command -v rclone >/dev/null 2>&1 || fail "rclone is required"
|
||||
command -v openssl >/dev/null 2>&1 || fail "openssl is required"
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
DEPLOY_ROOT="${DEPLOY_ROOT:-$(cd "$SCRIPT_DIR/.." && pwd)}"
|
||||
|
||||
DEPLOY_ENV="$DEPLOY_ROOT/.env"
|
||||
BACKUP_SCRIPT="$SCRIPT_DIR/backup.sh"
|
||||
|
||||
[[ -x "$BACKUP_SCRIPT" ]] || fail "Backup script is not executable: $BACKUP_SCRIPT"
|
||||
|
||||
load_env "$DEPLOY_ENV"
|
||||
|
||||
require_var S3_BACKUP_ENDPOINT_URL
|
||||
require_var S3_BACKUP_ACCESS_KEY_ID
|
||||
require_var S3_BACKUP_SECRET_ACCESS_KEY
|
||||
require_var S3_BACKUP_BUCKET
|
||||
require_var BACKUP_ENCRYPTION_PASSPHRASE
|
||||
|
||||
S3_BACKUP_PREFIX="$(normalize_prefix "${S3_BACKUP_PREFIX:-qlockify}")"
|
||||
BACKUP_LOCAL_KEEP_LATEST="$(positive_integer_or_default "${BACKUP_LOCAL_KEEP_LATEST:-3}" 3)"
|
||||
BACKUP_REMOTE_KEEP_LATEST="$(positive_integer_or_default "${BACKUP_REMOTE_KEEP_LATEST:-7}" 7)"
|
||||
|
||||
WORK_DIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK_DIR"' EXIT
|
||||
|
||||
PLAIN_DIR="$WORK_DIR/plain"
|
||||
mkdir -p "$PLAIN_DIR"
|
||||
|
||||
log "Creating local backup archive"
|
||||
|
||||
DEPLOY_ROOT="$DEPLOY_ROOT" "$BACKUP_SCRIPT" "$PLAIN_DIR"
|
||||
|
||||
shopt -s nullglob
|
||||
archives=("$PLAIN_DIR"/*.tar.gz)
|
||||
shopt -u nullglob
|
||||
|
||||
[[ "${#archives[@]}" -eq 1 ]] || fail "Expected exactly one backup archive, found ${#archives[@]}"
|
||||
|
||||
ARCHIVE_PATH="${archives[0]}"
|
||||
ARCHIVE_NAME="$(basename "$ARCHIVE_PATH")"
|
||||
|
||||
ENCRYPTED_NAME="$ARCHIVE_NAME.enc"
|
||||
ENCRYPTED_PATH="$WORK_DIR/$ENCRYPTED_NAME"
|
||||
|
||||
log "Encrypting backup archive"
|
||||
|
||||
openssl enc -aes-256-cbc -salt -pbkdf2 -iter 200000 \
|
||||
-in "$ARCHIVE_PATH" \
|
||||
-out "$ENCRYPTED_PATH" \
|
||||
-pass env:BACKUP_ENCRYPTION_PASSPHRASE
|
||||
|
||||
RCLONE_CONFIG="$WORK_DIR/rclone.conf"
|
||||
|
||||
cat > "$RCLONE_CONFIG" <<EOF
|
||||
[parspack]
|
||||
type = s3
|
||||
provider = Other
|
||||
access_key_id = $S3_BACKUP_ACCESS_KEY_ID
|
||||
secret_access_key = $S3_BACKUP_SECRET_ACCESS_KEY
|
||||
endpoint = $S3_BACKUP_ENDPOINT_URL
|
||||
acl = private
|
||||
force_path_style = true
|
||||
EOF
|
||||
|
||||
REMOTE_PATH="$(rclone_remote_path)"
|
||||
|
||||
log "Uploading encrypted backup to $REMOTE_PATH/$ENCRYPTED_NAME"
|
||||
|
||||
rclone copy \
|
||||
"$ENCRYPTED_PATH" \
|
||||
"$REMOTE_PATH" \
|
||||
--config "$RCLONE_CONFIG" \
|
||||
--s3-no-check-bucket \
|
||||
--progress
|
||||
|
||||
cleanup_local_backups "$BACKUP_LOCAL_KEEP_LATEST" "$DEPLOY_ROOT/backups/latest"
|
||||
cleanup_remote_backups "$BACKUP_REMOTE_KEEP_LATEST" "$REMOTE_PATH"
|
||||
|
||||
log "Backup upload completed successfully"
|
||||
126
scripts/backup.sh
Executable file
126
scripts/backup.sh
Executable file
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
backup.sh [output-directory]
|
||||
|
||||
Environment:
|
||||
DEPLOY_ROOT Deployment directory. Defaults to ~/qlockify-deployment.
|
||||
|
||||
Creates a timestamped .tar.gz archive containing:
|
||||
- PostgreSQL dump from the db service
|
||||
- media files from the backend media volume
|
||||
- deployment, backend, and frontend .env files
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[backup] %s\n' "$*"
|
||||
}
|
||||
|
||||
fail() {
|
||||
printf '[backup] %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
require_file() {
|
||||
local path="$1"
|
||||
[[ -f "$path" ]] || fail "Required file not found: $path"
|
||||
}
|
||||
|
||||
compose() {
|
||||
docker compose -f "$DEPLOY_ROOT/docker-compose.yml" "$@"
|
||||
}
|
||||
|
||||
wait_for_db() {
|
||||
compose exec -T db sh -c '
|
||||
set -eu
|
||||
: "${POSTGRES_USER:?POSTGRES_USER is missing}"
|
||||
: "${POSTGRES_DB:?POSTGRES_DB is missing}"
|
||||
until pg_isready --username="$POSTGRES_USER" --dbname="$POSTGRES_DB"; do
|
||||
sleep 1
|
||||
done
|
||||
' >/dev/null
|
||||
}
|
||||
|
||||
copy_env_if_exists() {
|
||||
local source_path="$1"
|
||||
local target_path="$2"
|
||||
|
||||
if [[ -f "$source_path" ]]; then
|
||||
cp "$source_path" "$target_path"
|
||||
fi
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
DEPLOY_ROOT="${DEPLOY_ROOT:-$HOME/qlockify-deployment}"
|
||||
OUTPUT_DIR="${1:-$DEPLOY_ROOT/backups}"
|
||||
BACKEND_ENV="$DEPLOY_ROOT/backend/qlockify-backend-deployment/.env"
|
||||
FRONTEND_ENV="$DEPLOY_ROOT/frontend/qlockify-frontend-deployment/.env"
|
||||
DEPLOY_ENV="$DEPLOY_ROOT/.env"
|
||||
|
||||
require_file "$DEPLOY_ROOT/docker-compose.yml"
|
||||
require_file "$BACKEND_ENV"
|
||||
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
WORK_DIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK_DIR"' EXIT
|
||||
|
||||
TIMESTAMP="$(date -u +'%Y%m%d-%H%M%S')"
|
||||
ARCHIVE_NAME="qlockify-backup-$TIMESTAMP.tar.gz"
|
||||
ARCHIVE_PATH="$OUTPUT_DIR/$ARCHIVE_NAME"
|
||||
|
||||
mkdir -p "$WORK_DIR/env"
|
||||
|
||||
log "Checking Docker Compose configuration"
|
||||
compose config -q
|
||||
|
||||
log "Starting database service if needed"
|
||||
compose up -d db
|
||||
wait_for_db
|
||||
|
||||
log "Dumping database from db service"
|
||||
compose exec -T db sh -c '
|
||||
set -eu
|
||||
: "${POSTGRES_USER:?POSTGRES_USER is missing}"
|
||||
: "${POSTGRES_DB:?POSTGRES_DB is missing}"
|
||||
pg_dump \
|
||||
--username="$POSTGRES_USER" \
|
||||
--dbname="$POSTGRES_DB" \
|
||||
--format=plain \
|
||||
--clean \
|
||||
--if-exists \
|
||||
--no-owner \
|
||||
--no-privileges
|
||||
' > "$WORK_DIR/database.sql"
|
||||
|
||||
log "Archiving media files from backend media volume"
|
||||
if compose ps --status running --services | grep -qx 'backend'; then
|
||||
compose exec -T backend sh -c 'mkdir -p /app/media && tar -C /app/media -czf - .' > "$WORK_DIR/media.tar.gz"
|
||||
else
|
||||
log "Backend service is not running; using a temporary container for media volume"
|
||||
compose run --rm --no-deps --entrypoint sh backend -c 'mkdir -p /app/media && tar -C /app/media -czf - .' > "$WORK_DIR/media.tar.gz"
|
||||
fi
|
||||
|
||||
log "Copying environment files"
|
||||
copy_env_if_exists "$DEPLOY_ENV" "$WORK_DIR/env/deployment.env"
|
||||
copy_env_if_exists "$BACKEND_ENV" "$WORK_DIR/env/backend.env"
|
||||
copy_env_if_exists "$FRONTEND_ENV" "$WORK_DIR/env/frontend.env"
|
||||
|
||||
cat > "$WORK_DIR/manifest.txt" <<EOF
|
||||
name=$ARCHIVE_NAME
|
||||
created_at_utc=$TIMESTAMP
|
||||
deploy_root=$DEPLOY_ROOT
|
||||
includes=database.sql,media.tar.gz,env/deployment.env,env/backend.env,env/frontend.env
|
||||
EOF
|
||||
|
||||
log "Creating archive $ARCHIVE_PATH"
|
||||
tar -C "$WORK_DIR" -czf "$ARCHIVE_PATH" .
|
||||
|
||||
log "Backup completed: $ARCHIVE_PATH"
|
||||
148
scripts/restore-from-s3.sh
Executable file
148
scripts/restore-from-s3.sh
Executable file
@@ -0,0 +1,148 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
restore-from-s3.sh latest
|
||||
restore-from-s3.sh <s3-object-key>
|
||||
|
||||
Examples:
|
||||
restore-from-s3.sh latest
|
||||
restore-from-s3.sh qlockify/qlockify-backup-YYYYMMDD-HHMMSS.tar.gz.enc
|
||||
|
||||
Environment:
|
||||
DEPLOY_ROOT Deployment directory. Defaults to ~/qlockify-deployment.
|
||||
S3_BACKUP_BUCKET S3 bucket name.
|
||||
S3_BACKUP_PREFIX S3 object prefix. Defaults to qlockify.
|
||||
S3_BACKUP_ENDPOINT_URL S3-compatible endpoint URL.
|
||||
S3_BACKUP_ACCESS_KEY_ID S3 access key.
|
||||
S3_BACKUP_SECRET_ACCESS_KEY S3 secret key.
|
||||
BACKUP_ENCRYPTION_PASSPHRASE Passphrase used to decrypt backup archives.
|
||||
|
||||
Existing RESTORE_SKIP_DB, RESTORE_SKIP_MEDIA, and RESTORE_SKIP_ENV flags are
|
||||
passed through to restore.sh.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[restore-rclone] %s\n' "$*"
|
||||
}
|
||||
|
||||
fail() {
|
||||
printf '[restore-rclone] %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
require_var() {
|
||||
local name="$1"
|
||||
[[ -n "${!name:-}" ]] || fail "$name is required"
|
||||
}
|
||||
|
||||
load_env() {
|
||||
local env_path="$1"
|
||||
[[ -f "$env_path" ]] || fail "Deployment env file not found: $env_path"
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
. "$env_path"
|
||||
set +a
|
||||
}
|
||||
|
||||
normalize_prefix() {
|
||||
local value="${1:-qlockify}"
|
||||
value="${value#/}"
|
||||
value="${value%/}"
|
||||
printf '%s' "$value"
|
||||
}
|
||||
|
||||
rclone_remote_path() {
|
||||
printf 'parspack:%s/%s' "$S3_BACKUP_BUCKET" "$S3_BACKUP_PREFIX"
|
||||
}
|
||||
|
||||
latest_object_name() {
|
||||
rclone lsf "$REMOTE_PATH" \
|
||||
--config "$RCLONE_CONFIG" \
|
||||
--s3-no-check-bucket \
|
||||
--files-only \
|
||||
| grep '\.tar\.gz\.enc$' \
|
||||
| sort \
|
||||
| tail -n 1 \
|
||||
|| true
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
REQUESTED_KEY="${1:-}"
|
||||
[[ -n "$REQUESTED_KEY" ]] || {
|
||||
usage
|
||||
exit 1
|
||||
}
|
||||
|
||||
command -v rclone >/dev/null 2>&1 || fail "rclone is required"
|
||||
command -v openssl >/dev/null 2>&1 || fail "openssl is required"
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
DEPLOY_ROOT="${DEPLOY_ROOT:-$(cd "$SCRIPT_DIR/.." && pwd)}"
|
||||
DEPLOY_ENV="$DEPLOY_ROOT/.env"
|
||||
RESTORE_SCRIPT="$SCRIPT_DIR/restore.sh"
|
||||
|
||||
[[ -x "$RESTORE_SCRIPT" ]] || fail "Restore script is not executable: $RESTORE_SCRIPT"
|
||||
load_env "$DEPLOY_ENV"
|
||||
|
||||
S3_BACKUP_PREFIX="$(normalize_prefix "${S3_BACKUP_PREFIX:-qlockify}")"
|
||||
|
||||
require_var S3_BACKUP_BUCKET
|
||||
require_var S3_BACKUP_ENDPOINT_URL
|
||||
require_var S3_BACKUP_ACCESS_KEY_ID
|
||||
require_var S3_BACKUP_SECRET_ACCESS_KEY
|
||||
require_var BACKUP_ENCRYPTION_PASSPHRASE
|
||||
|
||||
WORK_DIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK_DIR"' EXIT
|
||||
|
||||
RCLONE_CONFIG="$WORK_DIR/rclone.conf"
|
||||
|
||||
cat > "$RCLONE_CONFIG" <<EOF
|
||||
[parspack]
|
||||
type = s3
|
||||
provider = Other
|
||||
access_key_id = $S3_BACKUP_ACCESS_KEY_ID
|
||||
secret_access_key = $S3_BACKUP_SECRET_ACCESS_KEY
|
||||
endpoint = $S3_BACKUP_ENDPOINT_URL
|
||||
acl = private
|
||||
force_path_style = true
|
||||
EOF
|
||||
|
||||
REMOTE_PATH="$(rclone_remote_path)"
|
||||
|
||||
if [[ "$REQUESTED_KEY" == "latest" ]]; then
|
||||
log "Resolving latest encrypted backup object"
|
||||
OBJECT_NAME="$(latest_object_name)"
|
||||
[[ -n "$OBJECT_NAME" ]] || fail "No encrypted backups found in $REMOTE_PATH"
|
||||
else
|
||||
OBJECT_NAME="${REQUESTED_KEY##*/}"
|
||||
fi
|
||||
|
||||
ENCRYPTED_PATH="$WORK_DIR/$OBJECT_NAME"
|
||||
DECRYPTED_PATH="$WORK_DIR/${ENCRYPTED_PATH##*/}"
|
||||
DECRYPTED_PATH="${DECRYPTED_PATH%.enc}"
|
||||
|
||||
log "Downloading encrypted backup from $REMOTE_PATH/$OBJECT_NAME"
|
||||
rclone copyto "$REMOTE_PATH/$OBJECT_NAME" "$ENCRYPTED_PATH" \
|
||||
--config "$RCLONE_CONFIG" \
|
||||
--s3-no-check-bucket \
|
||||
--progress
|
||||
|
||||
log "Decrypting backup archive"
|
||||
openssl enc -d -aes-256-cbc -pbkdf2 -iter 200000 \
|
||||
-in "$ENCRYPTED_PATH" \
|
||||
-out "$DECRYPTED_PATH" \
|
||||
-pass env:BACKUP_ENCRYPTION_PASSPHRASE
|
||||
|
||||
log "Restoring decrypted backup archive"
|
||||
DEPLOY_ROOT="$DEPLOY_ROOT" "$RESTORE_SCRIPT" "$DECRYPTED_PATH"
|
||||
|
||||
log "S3 restore completed from: $REMOTE_PATH/$OBJECT_NAME"
|
||||
134
scripts/restore.sh
Executable file
134
scripts/restore.sh
Executable file
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
restore.sh <backup-archive.tar.gz>
|
||||
|
||||
Environment:
|
||||
DEPLOY_ROOT Deployment directory. Defaults to ~/qlockify-deployment.
|
||||
RESTORE_SKIP_ENV Set to 1 to keep current .env files.
|
||||
RESTORE_SKIP_MEDIA Set to 1 to keep current media files.
|
||||
RESTORE_SKIP_DB Set to 1 to keep current database.
|
||||
|
||||
This script restores:
|
||||
- deployment, backend, and frontend .env files
|
||||
- media files into the backend media volume
|
||||
- PostgreSQL database from database.sql
|
||||
|
||||
Database restore is destructive unless RESTORE_SKIP_DB=1 is set.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[restore] %s\n' "$*"
|
||||
}
|
||||
|
||||
fail() {
|
||||
printf '[restore] %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
require_file() {
|
||||
local path="$1"
|
||||
[[ -f "$path" ]] || fail "Required file not found: $path"
|
||||
}
|
||||
|
||||
compose() {
|
||||
docker compose -f "$DEPLOY_ROOT/docker-compose.yml" "$@"
|
||||
}
|
||||
|
||||
wait_for_db() {
|
||||
compose exec -T db sh -c '
|
||||
set -eu
|
||||
: "${POSTGRES_USER:?POSTGRES_USER is missing}"
|
||||
: "${POSTGRES_DB:?POSTGRES_DB is missing}"
|
||||
until pg_isready --username="$POSTGRES_USER" --dbname="$POSTGRES_DB"; do
|
||||
sleep 1
|
||||
done
|
||||
' >/dev/null
|
||||
}
|
||||
|
||||
restore_env_if_present() {
|
||||
local source_path="$1"
|
||||
local target_path="$2"
|
||||
|
||||
if [[ -f "$source_path" ]]; then
|
||||
mkdir -p "$(dirname "$target_path")"
|
||||
cp "$source_path" "$target_path"
|
||||
chmod 600 "$target_path" || true
|
||||
log "Restored $target_path"
|
||||
fi
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
ARCHIVE_PATH="${1:-}"
|
||||
[[ -n "$ARCHIVE_PATH" ]] || {
|
||||
usage
|
||||
exit 1
|
||||
}
|
||||
|
||||
DEPLOY_ROOT="${DEPLOY_ROOT:-$HOME/qlockify-deployment}"
|
||||
BACKEND_ENV="$DEPLOY_ROOT/backend/qlockify-backend-deployment/.env"
|
||||
FRONTEND_ENV="$DEPLOY_ROOT/frontend/qlockify-frontend-deployment/.env"
|
||||
DEPLOY_ENV="$DEPLOY_ROOT/.env"
|
||||
|
||||
require_file "$ARCHIVE_PATH"
|
||||
require_file "$DEPLOY_ROOT/docker-compose.yml"
|
||||
|
||||
WORK_DIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK_DIR"' EXIT
|
||||
|
||||
log "Extracting backup archive"
|
||||
tar -C "$WORK_DIR" -xzf "$ARCHIVE_PATH"
|
||||
|
||||
if [[ "${RESTORE_SKIP_ENV:-0}" != "1" ]]; then
|
||||
log "Restoring environment files"
|
||||
restore_env_if_present "$WORK_DIR/env/deployment.env" "$DEPLOY_ENV"
|
||||
restore_env_if_present "$WORK_DIR/env/backend.env" "$BACKEND_ENV"
|
||||
restore_env_if_present "$WORK_DIR/env/frontend.env" "$FRONTEND_ENV"
|
||||
else
|
||||
log "Skipping environment restore"
|
||||
fi
|
||||
|
||||
require_file "$BACKEND_ENV"
|
||||
|
||||
log "Checking Docker Compose configuration"
|
||||
compose config -q
|
||||
|
||||
if [[ "${RESTORE_SKIP_MEDIA:-0}" != "1" ]]; then
|
||||
require_file "$WORK_DIR/media.tar.gz"
|
||||
log "Restoring media files into backend media volume"
|
||||
compose run --rm --no-deps --entrypoint sh backend -c 'rm -rf /app/media/* /app/media/.[!.]* /app/media/..?* 2>/dev/null || true'
|
||||
compose run --rm --no-deps --entrypoint sh -T backend -c 'mkdir -p /app/media && tar -C /app/media -xzf -' < "$WORK_DIR/media.tar.gz"
|
||||
else
|
||||
log "Skipping media restore"
|
||||
fi
|
||||
|
||||
if [[ "${RESTORE_SKIP_DB:-0}" != "1" ]]; then
|
||||
require_file "$WORK_DIR/database.sql"
|
||||
log "Starting database service"
|
||||
compose up -d db
|
||||
wait_for_db
|
||||
|
||||
log "Recreating and restoring database"
|
||||
compose exec -T db sh -c '
|
||||
set -eu
|
||||
: "${POSTGRES_USER:?POSTGRES_USER is missing}"
|
||||
: "${POSTGRES_DB:?POSTGRES_DB is missing}"
|
||||
psql --username="$POSTGRES_USER" --dbname=postgres --command="SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '\''$POSTGRES_DB'\'' AND pid <> pg_backend_pid();" >/dev/null
|
||||
dropdb --username="$POSTGRES_USER" --if-exists "$POSTGRES_DB"
|
||||
createdb --username="$POSTGRES_USER" "$POSTGRES_DB"
|
||||
psql --username="$POSTGRES_USER" --dbname="$POSTGRES_DB"
|
||||
' < "$WORK_DIR/database.sql"
|
||||
else
|
||||
log "Skipping database restore"
|
||||
fi
|
||||
|
||||
log "Restore completed"
|
||||
log "Run: docker compose up -d --build"
|
||||
Reference in New Issue
Block a user