restic：快速、安全、高效的备份程序

jetsung

restic：快速、安全、高效的备份程序

Restic 是一个现代备份程序，可以备份您的文件：

从 Linux、BSD、Mac 和 Windows 到许多不同的存储类型，包括自托管和在线服务
很容易，成为无需服务器或复杂设置即可运行的单个可执行文件
实际上，只传输您备份的文件中实际更改的部分
通过在过程的每个部分谨慎使用加密技术，确保安全
可验证，使您能够确保可以在需要时恢复文件
免费 - Restic 完全免费使用且完全开源

https://github.com/restic/restic
https://restic.net/

jetsung

教程（AI 问答）

restic 和 rclone 使用教程

restic 使用教程

1. 安装 restic

# Ubuntu/Debian
sudo apt install restic

# macOS
brew install restic

# Windows (Chocolatey)
choco install restic

# 或者直接下载二进制文件
wget https://github.com/restic/restic/releases/latest/download/restic_xxx_linux_amd64.bz2
bunzip2 restic_xxx_linux_amd64.bz2
chmod +x restic_xxx_linux_amd64
sudo mv restic_xxx_linux_amd64 /usr/local/bin/restic

2. 初始化仓库

# 本地存储
restic init --repo /path/to/backup

# S3 存储
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret
restic init --repo s3:s3.amazonaws.com/bucket-name/path

# Backblaze B2
export B2_ACCOUNT_ID=your-id
export B2_ACCOUNT_KEY=your-key
restic init --repo b2:bucket-name:path

3. 设置密码

# 方法1：环境变量
export RESTIC_PASSWORD="your-strong-password"

# 方法2：密码文件
echo "your-strong-password" > ~/.restic-password
export RESTIC_PASSWORD_FILE=~/.restic-password

# 方法3：交互输入
restic backup --repo /path/to/backup /home/user/Documents

4. 基本备份操作

# 备份整个目录
restic backup --repo /path/to/backup /home/user/Documents

# 备份多个路径
restic backup --repo /path/to/backup /home/user/Documents /home/user/Pictures

# 排除特定文件
restic backup --repo /path/to/backup \
  --exclude="*.tmp" \
  --exclude=".cache" \
  --exclude="node_modules" \
  /home/user

# 添加标签和说明
restic backup --repo /path/to/backup \
  --tag "daily" \
  --host "my-laptop" \
  --tag "important" \
  /home/user/Documents

5. 查看和恢复数据

# 查看备份快照
restic snapshots --repo /path/to/backup

# 查看特定标签的快照
restic snapshots --tag daily

# 查看快照详情
restic snapshots --repo /path/to/backup <snapshot-id>

# 浏览备份内容
restic ls --repo /path/to/backup latest

# 恢复最新备份
restic restore --repo /path/to/backup latest --target /tmp/restore

# 恢复特定快照到指定目录
restic restore --repo /path/to/backup abc12345 --target /home/user/restore

# 恢复特定文件
restic restore --repo /path/to/backup latest --include "/home/user/Documents/file.txt" --target /tmp

6. 管理快照

# 删除旧快照（保留最近7天）
restic forget --repo /path/to/backup --keep-last 7 --prune

# 保留策略
restic forget --repo /path/to/backup \
  --keep-daily 7 \
  --keep-weekly 4 \
  --keep-monthly 6 \
  --keep-yearly 2 \
  --prune

# 手动删除特定快照
restic forget --repo /path/to/backup snapshot-id --prune

# 清理无用数据
restic prune --repo /path/to/backup

7. 数据检查和维护

# 检查仓库完整性
restic check --repo /path/to/backup

# 重建索引
restic rebuild-index --repo /path/to/backup

# 显示统计信息
restic stats --repo /path/to/backup

8. 自动化脚本示例

#!/bin/bash
# backup.sh

# 设置环境变量
export RESTIC_REPOSITORY="/path/to/backup"
export RESTIC_PASSWORD_FILE="/home/user/.restic-password"

# 执行备份
restic backup \
  --exclude="*.tmp" \
  --exclude=".cache" \
  --tag "daily" \
  /home/user/Documents \
  /home/user/Pictures

# 清理旧备份
restic forget --keep-daily 7 --keep-weekly 4 --prune

# 检查仓库
restic check

echo "Backup completed at $(date)"

rclone 使用教程

1. 安装 rclone

# 官方安装脚本
curl https://rclone.org/install.sh | sudo bash

# 或者手动安装
wget https://downloads.rclone.org/rclone-current-linux-amd64.zip
unzip rclone-current-linux-amd64.zip
sudo cp rclone-*/rclone /usr/local/bin/
sudo chmod 755 /usr/local/bin/rclone

2. 配置远程存储

# 交互式配置
rclone config

# 或者手动配置
cat > ~/.config/rclone/rclone.conf << EOF
[google-drive]
type = drive
client_id = 
client_secret = 
scope = drive
token = {"access_token":"xxx","token_type":"Bearer","refresh_token":"xxx","expiry":"2023-01-01T00:00:00.000000000+00:00"}

[aws-s3]
type = s3
provider = AWS
env_auth = false
access_key_id = your-access-key
secret_access_key = your-secret-key
region = us-east-1
EOF

3. 基本文件操作

# 列出远程存储
rclone lsd remote:

# 列出文件
rclone ls remote:path/to/folder

# 上传文件
rclone copy /local/path remote:path/to/folder

# 同步目录（本地到远程）
rclone sync /local/path remote:path/to/folder

# 同步目录（双向）
rclone bisync /local/path remote:path/to/folder

# 下载文件
rclone copy remote:path/to/file /local/path

# 移动文件
rclone move /local/path remote:path/to/folder

# 删除文件
rclone delete remote:path/to/file

4. 高级功能

# 过滤文件
rclone copy /source remote:dest \
  --exclude "*.tmp" \
  --exclude ".git/**" \
  --include "*.jpg" \
  --include "*.png"

# 带宽限制
rclone copy /source remote:dest --bwlimit 1M

# 断点续传
rclone copy /large-file remote:dest --transfers 1

# 并行传输
rclone copy /source remote:dest --transfers 8

# 显示进度
rclone copy /source remote:dest --progress

# 模拟操作（不实际执行）
rclone copy --dry-run /source remote:dest

5. 云存储特定操作

# Google Drive
rclone lsd gdrive:MyDrive
rclone copy /local/path gdrive:MyDrive/backup

# Amazon S3
rclone ls s3:bucket-name/path
rclone copy /local/path s3:bucket-name/path

# OneDrive
rclone lsd onedrive:Documents
rclone copy /local/path onedrive:Documents/backup

# Dropbox
rclone ls dropbox:Backup
rclone copy /local/path dropbox:Backup

6. 挂载远程存储

# 挂载为本地文件系统
rclone mount gdrive: /mnt/gdrive --daemon

# 只读挂载
rclone mount gdrive: /mnt/gdrive --read-only --daemon

# 卸载
fusermount -u /mnt/gdrive

7. 自动化脚本示例

#!/bin/bash
# sync.sh

# 同步到多个云存储
rclone sync /home/user/backup gdrive:backup \
  --exclude "*.tmp" \
  --bwlimit 2M \
  --transfers 4 \
  --log-file=/var/log/rclone-sync.log

rclone sync /home/user/backup s3:my-backup-bucket \
  --exclude "*.tmp" \
  --bwlimit 5M \
  --transfers 8

echo "Sync completed at $(date)" | tee -a /var/log/sync.log

8. 定时任务设置

# 编辑 crontab
crontab -e

# 每天凌晨2点执行备份
0 2 * * * /path/to/backup.sh

# 每小时同步重要文件
0 * * * * /path/to/sync.sh

# 每周日凌晨3点清理旧快照
0 3 * * 0 /path/to/cleanup.sh

最佳实践建议

restic 最佳实践

定期检查：每周运行 restic check
保留策略：设置合理的快照保留规则
多地备份：将备份存储在不同位置
测试恢复：定期测试数据恢复流程

rclone 最佳实践

使用配置文件：避免重复输入认证信息
限制带宽：避免影响正常网络使用
启用日志：记录同步操作的详细信息
定期验证：检查同步数据的完整性

restic 和 rclone 结合使用教程

将 restic 和 rclone 结合使用可以发挥两者的优势：rclone 负责与云存储的高效通信，restic 负责专业的备份管理。

方案一：rclone 作为 restic 的后端存储

1. 使用 rclone 挂载作为 restic 存储

#!/bin/bash
# backup-with-mount.sh

# 设置变量
MOUNT_POINT="/tmp/restic-repo"
LOCAL_REPO="$MOUNT_POINT/backup"
SOURCE_PATH="/home/user/Documents"
RCLONE_REMOTE="gdrive:backup"

# 创建挂载点
mkdir -p $MOUNT_POINT

# 挂载 rclone 远程存储
rclone mount $RCLONE_REMOTE $MOUNT_POINT --daemon

# 等待挂载完成
sleep 5

# 初始化 restic 仓库（首次运行）
if [ ! -d "$LOCAL_REPO" ]; then
    export RESTIC_PASSWORD="your-strong-password"
    restic init --repo $LOCAL_REPO
fi

# 执行备份
export RESTIC_PASSWORD="your-strong-password"
restic backup --repo $LOCAL_REPO $SOURCE_PATH

# 清理旧快照
restic forget --repo $LOCAL_REPO --keep-daily 7 --keep-weekly 4 --prune

# 卸载
fusermount -u $MOUNT_POINT

2. 使用 rclone 的 REST 服务

#!/bin/bash
# backup-with-rest.sh

# 启动 rclone REST 服务
rclone rcd --rc-addr :5572 --rc-user admin --rc-pass password &

# 等待服务启动
sleep 3

# 使用 HTTP 后端初始化 restic
export RESTIC_PASSWORD="your-strong-password"
restic init --repo rest:http://admin:password@localhost:5572/

# 执行备份
restic backup --repo rest:http://admin:password@localhost:5572/ /home/user/Documents

# 清理
pkill rclone

方案二：rclone 传输 + restic 本地管理

3. 本地备份后同步到云端

#!/bin/bash
# hybrid-backup.sh

# 配置
LOCAL_REPO="/backup/local-restic"
CLOUD_REMOTE="gdrive:backup"
SOURCE_PATH="/home/user"
RETENTION_DAYS=30

# 设置密码
export RESTIC_PASSWORD_FILE="/home/user/.restic-password"

# 确保本地仓库存在
if [ ! -d "$LOCAL_REPO" ]; then
    mkdir -p "$LOCAL_REPO"
    restic init --repo "$LOCAL_REPO"
fi

# 执行本地备份
echo "开始本地备份..."
restic backup --repo "$LOCAL_REPO" \
    --exclude="*.tmp" \
    --exclude=".cache" \
    --exclude="node_modules" \
    --tag "daily-$(date +%Y%m%d)" \
    "$SOURCE_PATH"

# 清理本地旧快照
echo "清理本地旧快照..."
restic forget --repo "$LOCAL_REPO" \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 6 \
    --prune

# 同步到云端（使用 rclone）
echo "同步到云端..."
rclone sync "$LOCAL_REPO" "$CLOUD_REMOTE/restic-repo" \
    --transfers 4 \
    --checkers 8 \
    --bwlimit 2M \
    --progress \
    --log-file="/var/log/rclone-sync.log"

# 在云端也执行清理（可选）
echo "云端清理..."
rclone exec restic "$CLOUD_REMOTE/restic-repo" forget \
    --keep-daily 30 \
    --keep-weekly 12 \
    --keep-monthly 12 \
    --prune

echo "混合备份完成！"

方案三：分层备份策略

4. 快速同步 + 完整备份

#!/bin/bash
# tiered-backup.sh

# 配置
QUICK_REMOTE="gdrive:quick-backup"      # 快速同步（rclone）
FULL_LOCAL_REPO="/backup/full-restic"   # 完整备份（restic）
FULL_REMOTE="s3:full-backup-bucket"     # 完整备份云端存储
SOURCE_PATH="/home/user"

# 设置密码
export RESTIC_PASSWORD_FILE="/home/user/.restic-password"

# 第一层：快速文件同步（使用 rclone）
echo "=== 执行快速同步 ==="
rclone sync "$SOURCE_PATH" "$QUICK_REMOTE" \
    --exclude="*.tmp" \
    --exclude=".cache" \
    --transfers 8 \
    --checkers 16 \
    --progress

# 第二层：完整备份（使用 restic）
echo "=== 执行完整备份 ==="

# 确保本地仓库存在
if [ ! -d "$FULL_LOCAL_REPO" ]; then
    mkdir -p "$FULL_LOCAL_REPO"
    restic init --repo "$FULL_LOCAL_REPO"
fi

# 执行完整备份
restic backup --repo "$FULL_LOCAL_REPO" \
    --exclude="*.tmp" \
    --exclude=".cache" \
    --tag "full-$(date +%Y%m%d)" \
    --host "$(hostname)" \
    "$SOURCE_PATH"

# 完整备份清理策略
restic forget --repo "$FULL_LOCAL_REPO" \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 12 \
    --prune

# 将完整备份同步到云端
echo "=== 同步完整备份到云端 ==="
rclone sync "$FULL_LOCAL_REPO" "$FULL_REMOTE" \
    --transfers 4 \
    --checkers 8 \
    --bwlimit 5M

echo "分层备份完成！"

方案四：灾难恢复备份

5. 多地备份策略

#!/bin/bash
# disaster-recovery-backup.sh

# 配置多个云存储
PRIMARY_REMOTE="s3:primary-backup-bucket"
SECONDARY_REMOTE="gdrive:backup"
TERTIARY_REMOTE="onedrive:backup"

LOCAL_REPO="/backup/local-restic"
SOURCE_PATHS=("/home/user" "/etc" "/var/www")

# 设置密码
export RESTIC_PASSWORD_FILE="/home/user/.restic-password"

# 函数：执行备份到指定仓库
backup_to_repo() {
    local repo_path=$1
    local tag_suffix=$2
    
    echo "备份到: $repo_path"
    
    # 初始化仓库
    if [ ! -d "$repo_path" ]; then
        mkdir -p "$repo_path"
        restic init --repo "$repo_path"
    fi
    
    # 执行备份
    restic backup --repo "$repo_path" \
        --exclude="*.tmp" \
        --exclude=".cache" \
        --tag "backup-$tag_suffix-$(date +%Y%m%d)" \
        "${SOURCE_PATHS[@]}"
    
    # 清理策略
    restic forget --repo "$repo_path" \
        --keep-daily 7 \
        --keep-weekly 4 \
        --keep-monthly 6 \
        --prune
}

# 函数：同步到远程存储
sync_to_remote() {
    local local_repo=$1
    local remote_name=$2
    
    echo "同步到远程: $remote_name"
    rclone sync "$local_repo" "$remote_name" \
        --transfers 4 \
        --checkers 8 \
        --retries 3 \
        --low-level-retries 10
}

# 主备份流程
echo "=== 开始灾难恢复备份 ==="

# 本地备份
backup_to_repo "$LOCAL_REPO" "local"

# 同步到多个远程存储
sync_to_remote "$LOCAL_REPO" "$PRIMARY_REMOTE"
sync_to_remote "$LOCAL_REPO" "$SECONDARY_REMOTE" 
sync_to_remote "$LOCAL_REPO" "$TERTIARY_REMOTE"

echo "灾难恢复备份完成！"

方案五：增量同步 + 完整验证

6. 智能备份脚本

#!/bin/bash
# smart-backup.sh

# 配置
CONFIG_FILE="/etc/backup/config.json"
LOG_FILE="/var/log/backup.log"
TEMP_DIR="/tmp/backup-$(date +%s)"

# 加载配置
load_config() {
    if [ -f "$CONFIG_FILE" ]; then
        source "$CONFIG_FILE"
    else
        # 默认配置
        LOCAL_REPO="/backup/restic"
        REMOTE_STORAGE="gdrive:backup"
        SOURCE_DIRS=("/home" "/etc")
        EXCLUDE_PATTERNS=("*.tmp" ".cache" "node_modules")
        RETENTION_POLICY="--keep-daily 7 --keep-weekly 4 --keep-monthly 6"
    fi
}

# 日志函数
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"
}

# 检查存储空间
check_space() {
    local required_space=$(du -sb "${SOURCE_DIRS[@]}" | awk '{sum += $1} END {print sum}')
    local available_space=$(df "$LOCAL_REPO" | awk 'NR==2 {print $4*1024}')
    
    if [ "$required_space" -gt "$available_space" ]; then
        log "警告: 空间不足，需要 $(($required_space/1024/1024))MB，可用 $(($available_space/1024/1024))MB"
        return 1
    fi
    return 0
}

# 执行备份
perform_backup() {
    log "开始备份..."
    
    # 检查空间
    if ! check_space; then
        log "空间不足，备份终止"
        return 1
    fi
    
    # 构建排除参数
    local exclude_args=""
    for pattern in "${EXCLUDE_PATTERNS[@]}"; do
        exclude_args="$exclude_args --exclude='$pattern'"
    done
    
    # 初始化仓库
    if [ ! -d "$LOCAL_REPO" ]; then
        mkdir -p "$LOCAL_REPO"
        restic init --repo "$LOCAL_REPO"
        log "初始化仓库: $LOCAL_REPO"
    fi
    
    # 执行备份
    local backup_cmd="restic backup --repo '$LOCAL_REPO' $exclude_args --tag 'auto-$(date +%Y%m%d)'"
    for dir in "${SOURCE_DIRS[@]}"; do
        backup_cmd="$backup_cmd '$dir'"
    done
    
    log "执行命令: $backup_cmd"
    eval $backup_cmd
    
    if [ $? -eq 0 ]; then
        log "备份成功"
        return 0
    else
        log "备份失败"
        return 1
    fi
}

# 清理旧快照
cleanup_snapshots() {
    log "清理旧快照..."
    restic forget --repo "$LOCAL_REPO" $RETENTION_POLICY --prune
    if [ $? -eq 0 ]; then
        log "清理完成"
    else
        log "清理失败"
    fi
}

# 同步到云端
sync_to_cloud() {
    log "同步到云端: $REMOTE_STORAGE"
    
    # 先检查远程连接
    rclone lsd "$REMOTE_STORAGE" > /dev/null 2>&1
    if [ $? -ne 0 ]; then
        log "警告: 无法连接到远程存储"
        return 1
    fi
    
    # 执行同步
    rclone sync "$LOCAL_REPO" "$REMOTE_STORAGE" \
        --transfers 4 \
        --checkers 8 \
        --progress \
        --stats 1s \
        --log-file="$LOG_FILE"
    
    if [ $? -eq 0 ]; then
        log "同步成功"
        return 0
    else
        log "同步失败"
        return 1
    fi
}

# 验证备份完整性
verify_backup() {
    log "验证备份完整性..."
    
    # 检查仓库
    restic check --repo "$LOCAL_REPO"
    local check_result=$?
    
    # 验证最新快照
    local latest_snapshot=$(restic snapshots --repo "$LOCAL_REPO" --json | jq -r '.[-1].id')
    if [ "$latest_snapshot" != "null" ] && [ -n "$latest_snapshot" ]; then
        # 随机验证一些文件（可选）
        log "最新快照: $latest_snapshot"
    fi
    
    if [ $check_result -eq 0 ]; then
        log "验证通过"
        return 0
    else
        log "验证失败"
        return 1
    fi
}

# 主函数
main() {
    log "=== 智能备份开始 ==="
    
    # 加载配置
    load_config
    
    # 执行备份
    if ! perform_backup; then
        log "备份失败，终止流程"
        exit 1
    fi
    
    # 清理
    cleanup_snapshots
    
    # 同步到云端
    sync_to_cloud
    
    # 验证
    verify_backup
    
    # 清理临时文件
    rm -rf "$TEMP_DIR"
    
    log "=== 智能备份完成 ==="
}

# 执行主函数
main "$@"

配置文件示例

7. 配置文件 `/etc/backup/config.json`

# 备份配置
LOCAL_REPO="/backup/restic"
REMOTE_STORAGE="gdrive:backup"
SOURCE_DIRS=(
    "/home/user/Documents"
    "/home/user/Pictures"
    "/etc"
)
EXCLUDE_PATTERNS=(
    "*.tmp"
    "*.log"
    ".cache"
    "node_modules"
    "__pycache__"
    ".git"
)
RETENTION_POLICY="--keep-daily 7 --keep-weekly 4 --keep-monthly 6"

# 网络设置
MAX_BANDWIDTH="2M"
MAX_TRANSFERS=4

# 邮件通知（可选）
EMAIL_NOTIFY="user@example.com"
SMTP_SERVER="smtp.example.com"

定时任务设置

8. crontab 配置

# 编辑 crontab
crontab -e

# 每天凌晨2点执行快速同步
0 2 * * * /usr/local/bin/smart-backup.sh

# 每周日凌晨3点执行完整验证
0 3 * * 0 /usr/local/bin/smart-backup.sh --full-verify

# 每月1号执行深度清理
0 4 1 * * /usr/local/bin/deep-cleanup.sh

监控和告警

9. 监控脚本

#!/bin/bash
# backup-monitor.sh

# 检查备份状态
check_backup_status() {
    local repo_path=$1
    local max_age_hours=${2:-24}
    
    # 检查最近快照
    local latest_backup=$(restic snapshots --repo "$repo_path" --json | jq -r '.[-1].time')
    
    if [ "$latest_backup" == "null" ]; then
        echo "错误: 没有找到备份"
        return 1
    fi
    
    # 计算时间差
    local backup_time=$(date -d "$latest_backup" +%s)
    local current_time=$(date +%s)
    local age_hours=$(( (current_time - backup_time) / 3600 ))
    
    if [ $age_hours -gt $max_age_hours ]; then
        echo "警告: 备份已过期 (${age_hours}小时前)"
        return 1
    else
        echo "正常: 最近备份 ${age_hours}小时前"
        return 0
    fi
}

# 检查存储空间
check_storage_space() {
    local path=$1
    local threshold=${2:-90}
    
    local usage=$(df "$path" | awk 'NR==2 {print $5}' | sed 's/%//')
    
    if [ "$usage" -gt "$threshold" ]; then
        echo "警告: 存储空间使用率 ${usage}%"
        return 1
    else
        echo "正常: 存储空间使用率 ${usage}%"
        return 0
    fi
}

# 主监控函数
monitor_backup() {
    local repo_path="/backup/restic"
    
    echo "=== 备份监控报告 ==="
    
    # 检查备份状态
    check_backup_status "$repo_path"
    
    # 检查存储空间
    check_storage_space "$repo_path"
    
    # 检查云存储连接
    rclone lsd gdrive:backup > /dev/null 2>&1
    if [ $? -eq 0 ]; then
        echo "正常: 云存储连接正常"
    else
        echo "错误: 云存储连接失败"
    fi
    
    echo "=== 监控完成 ==="
}

monitor_backup

这种结合使用的方式充分发挥了两个工具的优势：

rclone: 高效的云存储传输和管理
restic: 专业的备份管理和数据保护

根据你的具体需求选择合适的方案！