Compare commits

...

36 Commits

Author SHA1 Message Date
Asif Bacchus 0fcab85da9 fix incorrect filename 2019-07-30 01:11:15 -06:00
Asif Bacchus ed62bb1e38 update logwatch integration 2019-07-30 00:44:37 -06:00
Asif Bacchus 3a759f39ac reworded redundant success msgs 2019-07-30 00:41:02 -06:00
Asif Bacchus 5cd023010c update wiki link, match details filename to script 2019-07-30 00:39:17 -06:00
Asif Bacchus 8e30cb4048 fix timestamp double-bracket 2019-07-26 03:16:55 -06:00
Asif Bacchus 8edd833544 fix final success msg output spacing err 2019-07-26 03:15:26 -06:00
Asif Bacchus dd49378678 check running as root 2019-07-26 03:14:12 -06:00
Asif Bacchus efe202d9a3 new directory structure 2019-07-26 03:11:53 -06:00
Asif Bacchus 9936cf7de3 add link to wiki 2019-07-20 18:14:14 -06:00
Asif Bacchus 0b1d220b5b update readme and license 2019-07-20 18:09:19 -06:00
Asif Bacchus a299cca365 move logwatch under etc dir to mirror real sys 2019-07-20 17:17:33 -06:00
Asif Bacchus 271f08d0e4 sample logrotate config 2019-07-20 17:16:58 -06:00
Asif Bacchus 55162342c7 add logwatch example files 2019-07-20 17:14:58 -06:00
Asif Bacchus fdb139afc9 change success msg for logwatch parsing 2019-07-20 17:14:44 -06:00
Asif Bacchus 93bfdbfeb3 enclose timestamps in square brackets 2019-07-20 17:06:20 -06:00
Asif Bacchus 3f1861fc01 update example default locations 2019-07-20 16:57:37 -06:00
Asif Bacchus 4e5bfa49b4 use dbnames from details file for dump 2019-06-20 03:39:57 -06:00
Asif Bacchus 6fec178fae check for null database names in details file 2019-06-20 03:37:17 -06:00
Asif Bacchus ca190c89bb rewrote help section 2019-06-19 23:08:07 -06:00
Asif Bacchus 26524c97ed func scriptHelp and contents 2019-06-19 02:13:21 -06:00
Asif Bacchus 85fe709e06 update function commentary 2019-06-18 23:11:47 -06:00
Asif Bacchus 13b75232bc fix trapExit error code 2019-06-18 22:32:33 -06:00
Asif Bacchus 1144eb6ae6 trap calls dedicated func 2019-06-18 22:30:22 -06:00
Asif Bacchus 91f0cdfa8e add trap for abnormal termination 2019-06-18 22:14:33 -06:00
Asif Bacchus ba27cff791 note exit on error at end of log for clarity 2019-06-18 22:06:26 -06:00
Asif Bacchus 57b13a3844 remove quotes on borgPrune vars, not sure why 2019-06-18 22:01:28 -06:00
Asif Bacchus 3fd8da1e26 fix borgSSHKey test, check for file not dir 2019-06-18 21:38:44 -06:00
Asif Bacchus 249209df99 update readme 2019-05-28 21:02:49 -06:00
Asif Bacchus 036cfdcdba rename script file 2019-05-27 07:26:30 -06:00
Asif Bacchus 3a12003803 remove old files from tansitional repo 2019-05-27 07:25:08 -06:00
Asif Bacchus 24f5af85c0 sample helper files 2019-05-27 07:24:50 -06:00
Asif Bacchus e7bf384cba annotated sample details file 2019-05-27 06:53:38 -06:00
Asif Bacchus 11afb719ff updated error code notation in comments 2019-05-27 06:20:05 -06:00
Asif Bacchus 5e7d2fe6d2 check SQL vars are defined in details file 2019-05-27 06:17:38 -06:00
Asif Bacchus 62107ed611 fixed missing space in sqldumpdir command 2019-05-27 06:13:03 -06:00
Asif Bacchus f33f4824fa syntax and var name corrections 2019-05-27 06:10:06 -06:00
18 changed files with 544 additions and 900 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License MIT License
Copyright (c) <year> <copyright holders> Copyright (c) 2019 Asif Bacchus
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal of this software and associated documentation files (the "Software"), to deal
+62 -2
View File
@@ -1,3 +1,63 @@
# seafileBackup # Backup a bare-metal Seafile installation using borgbackup <!-- omit in toc -->
Backup Seafile DB and data using borgbackup to rsync.net Simple POSIX-compliant script to backup a bare-metal Seafile installation using borgbackup to a remote SSH-capable storage system. I strongly suggest using rsync.net since they are very reliable, fast and even have special pricing for people using borgbackup/attic [(details here)](https://www.rsync.net/products/attic.html).
**THIS SCRIPT USES BORG [(borgbackup)](https://borgbackup.readthedocs.io/en/stable/) SO PLEASE ENSURE THAT IS INSTALLED CORRECTLY BEFORE TRYING TO USE THIS SCRIPT!**
- [Basic features of this script:](#Basic-features-of-this-script)
- [Check out the wiki](#Check-out-the-wiki)
- [Overview](#Overview)
- [Installation](#Installation)
- [Running the script](#Running-the-script)
- [*seafbackup.sh*](#seafbackupsh)
- [Final thoughts](#Final-thoughts)
## Basic features of this script:
- Backup your entire Seafile program and data directories.
- Dump Seafile SQL databases to a temporary directory, add them to the backup and then remove the temporary directory.
- Easily add additional directories/files to the backup by modifying a plain-text file.
- Easily exclude directories/files from being backed up (i.e. thumbnails) by modifying a plain-text file.
- Optionally stop Seafile services during the backup and restart them after.
- Optionally copy a 503 error page to your webserver so users know your server is unavailable due to a backup being performed. The 503 file is removed when the backup is completed.
- Run 'borg prune' to trim old backups according to your schedule so you don't pay storage fees for super-old or un-needed backups.
- Create a clear, easy to parse log file so you can keep an eye on warnings/errors, etc.
- The generated log file can be easily parsed by tools like *logwatch* so you can integrate backup-monitoring with your existing system-monitoring solutions.
## Check out the wiki
This readme is an overview of the script. For detailed instructions on how the script works, what it does and it's various options, please [read the wiki](https://git.asifbacchus.app/asif/seafileBackup/wiki)! I've tried to annotate the files as best I could with many many comments, but seriously, reading the [wiki](https://git.asifbacchus.app/asif/seafileBackup/wiki) if you're stuck is the best approach since I try to go over everything step-by-step.
## Overview
This script has NO mandatory parameters. If no parameters are specified, it will proceed using defaults which normally align with the recommended Seafile installation defaults. That being said, it is impossible to determine defaults for things like your backup-server settings, so please make sure you customize the *seafbackup.details* file with appropriate details!
By default, the script will dump Seafile SQL databases and back them up along with your entire Seafile program and data directories. You can add additional files to be backed up by adding paths to *xtraLocations.borg*.
## Installation
Simply clone this repo to the desired location. I recommend somewhere simple like *ROOT's home directory* so any sensitive information is restricted to the root user by default.
```bash
cd /root
sudo git clone https://git.asifbacchus.app/asif/seafileBackup.git
```
## Running the script
```bash
./seafbackup.sh [parameters]
# display abbreviated help
./seafbackup.sh --help
```
## *seafbackup.sh*
Please open this file in a text editor and follow the comments to fill in the parameters with the correct values for your environment and your backup server.
## Final thoughts
I hope this makes backing up your Seafile server easier and removes one more mundane administrative task from your schedule :-) If you have any comments, questions, concerns, suggestions for improvements or notice any bugs, please file an issue in this repo!
For more scripts and solutions like this, check out my blog at [My Techie-Thoughts](https://mytechiethoughts.com). Thanks!
-813
View File
@@ -1,813 +0,0 @@
#!/bin/bash
### Text formatting presets
normal="\e[0m"
bold="\e[1m"
default="\e[39m"
err="\e[1;31m"
warn="\e[1;93m"
ok="\e[32m"
lit="\e[93m"
op="\e[39m"
info="\e[96m"
note="\e[95m"
### Functions ###
function checkExist {
if [ "$1" = "ff" ]; then
# find file
if [ -f "$2" ]; then
# found
return 0
else
# not found
return 1
fi
elif [ "$1" = "fd" ]; then
# find directory
if [ -d "$2" ]; then
# found
return 0
else
# not found
return 1
fi
fi
}
### cleanup - cleanup files and directories created by this script
function cleanup {
## remove SQL dump file and directory
rm -rf "$sqlDumpDir" >> "$logFile" 2>&1
# verify directory is gone
checkExist fd "$sqlDumpDir"
checkResult="$?"
if [ "$checkResult" = "0" ]; then
# directory still exists
exitWarn+=("[$(stamp)]_111")
else
# directory removed
echo -e "${op}[$(stamp)] Removed SQL temp directory${normal}" \
>> "$logFile"
fi
## remove 503 error page
# check value of 'clean503' to see if this is necessary (=1) otherwise, skip
if [ "$clean503" = "1" ]; then
# proceed with cleanup
echo -e "${op}[$(stamp)] Removing 503 error page..." >> "$logFile"
rm -f "$webroot/$err503File" >> "$logFile" 2>&1
# verify file is actually gone
checkExist ff "$webroot/$err503File"
checkResult="$?"
if [ "$checkResult" = "0" ]; then
# file still exists
exitWarn+=("[$(stamp)]_5030")
else
# file removed
echo -e "${info}[$(stamp)] -- [INFO] 503 page removed from webroot" \
"--${normal}" >> "$logFile"
fi
else
echo -e "${op}[$(stamp)] 503 error page never copied to webroot," \
"nothing to cleanup" >> "$logFile"
fi
## Restart Seafile services
svcSeafile start
}
### scriptHelp -- display usage information for this script
function scriptHelp {
echo -e "${bold}${note}\n${scriptName} usage instructions:\n${normal}"
echo -e "${default}This script performs a backup of your Seafile system" \
"assuming a fairly standard"
echo -e "baremetal set up such as outlined at${lit}" \
"https://mytechiethoughts.com${default}."
echo -e "Details about this script can be found at that site."
echo -e "${bold}\nThe script performs the following tasks:" \
"${normal}${default}"
echo -e "1. Stops Seafile services and copies a 503 error page to" \
"your webroot to"
echo -e "\tprevent access to Seafile during the backup."
echo -e "2. Dumps Seafile SQL database to a temporary directory."
echo -e "3. Invokes borgbackup to backup your SQL data, Seafile program" \
"and data files,"
echo -e "\tand any other files you specify."
echo -e "3. Prunes old backups from borgbackup repo."
echo -e "4. Removes 503 error page, if necessary, and cleans up."
echo -e "\nThe readme file included in this script's git repo contains" \
"detailed usage"
echo -e "information. The following is a brief summary:\n"
echo -e "${bold}${note}Mandatory parameters:${normal}${default}"
echo -e "${lit}\n-d, Seafile data directory${default}"
echo -e "This is the physical location of your Seafile DATA."
echo -e "${lit}\n-p, Seafile program directory${default}"
echo -e "This is the physical location of your Seafile PROGRAM FILES."
echo -e "${lit}\n-u, Seafile system user account${default}"
echo -e "The system account Seafile is running under. Some actions must" \
"run as this"
echo -e "user due to file ownership restrictions."
echo -e "${bold}${note}\nOptional parameters:${normal}${default}"
echo -e "${lit}\n-5, Location of 503 error page file${default}"
echo -e "FULL PATH to the 503 error page HTML file you want copied to" \
"your webroot to"
echo -e "inform users the server is down during the backup. If you don't" \
"specify a path/"
echo -e "file, the default will be used. If the default cannot be found," \
"a warning will"
echo -e "be logged and the script will continue."
echo -e "${info}Default: ScriptPath/503.html${default}"
echo -e "${lit}\n-b, Location of file with borg repo details${default}"
echo -e "FULL PATH to the plain text file containing all information" \
"needed to connect"
echo -e "and process your borg repo. Details on the structure of this" \
"file are in the"
echo -e "readme and on ${lit}https://mytechiethoughts.com${default}"
echo -e "${info}Default: ScriptPath/seaf_borg.details${default}"
echo -e "${lit}\n-l, Location to save log file${default}"
echo -e "This script writes a detailed log file of all activities. It" \
"is structured in"
echo -e "a way easy for log parsers (like Logwatch) to read."
echo -e "${info}Default: ScriptPath/ScriptName.log${default}"
echo -e "${lit}\n-s, Location of file with mySQL details${default}"
echo -e "FULL PATH to the plain text file containing all information" \
"needed to connect"
echo -e "to your mySQL (mariaDB) server and Seafile databases. Details" \
"on the structure"
echo -e "of this file are in the readme and on${lit}" \
"https://mytechiethoughts.com${default}"
echo -e "${info}Default: ScriptPath/seaf_sql.details${default}"
echo -e "${lit}\n-v, Verbose output from borgbackup${default}"
echo -e "By default, this script will only log summary data from borg." \
"If you need/want"
echo -e "more detailed information, the verbose setting will list every" \
"file processed"
echo -e "along with their status. Note: Your log file can quickly get" \
"very very large"
echo -e "using this option!"
echo -e "${info}Default: NOT activated (standard logging)${default}"
echo -e "${lit}\n-w, webserver's webroot directory${default}"
echo -e "This is the location from which your webserver (NGINX, Apache," \
"etc.) physically"
echo -e "stores files to be served. This is NOT the configuration" \
"directory for your"
echo -e "webserver! It is the place where the actual" \
"HTML/PHP/CSS/JS/etc. files are"
echo -e "stored. NOTE: If you omit this option, then the entire 503 copy" \
"process will"
echo -e "be skipped regardless of the presence of a 503.html file. If" \
"you don't want to"
echo -e "use the 503 feature, omitting this is an easy way to skip it!"
echo -e "${info}Default: NONE${default}"
echo -e "${lit}\n-?, This help screen${default}\n"
echo -e "${bold}Please refer to the readme file and/or ${lit}https://mytechiethoughts.com${default}"
echo -e "for more information on this script.${normal}\n"
# exit with code 1 -- there is no use logging this
exit 1
}
### start and stop seafile services
function svcSeafile {
if [ "$1" = "start" ]; then
# start services
systemctl start seafile.service
statusSeaf=$?
systemctl start seahub.service
statusShub=$?
# check for errors
if [ $statusSeaf -eq 0 ] && [ $statusShub -eq 0 ]; then
echo -e "${info}[$(stamp)] -- [INFO] Seafile services started" \
"successfully --${normal}" >> "$logFile"
else
exitError+=("[$(stamp)] 100")
quit
fi
elif [ "$1" = "stop" ]; then
# stop services
systemctl stop seahub.service
statusShub=$?
systemctl stop seafile.service
statusSeaf=$?
# check for errors
if [ $statusSeaf -eq 0 ] && [ $statusShub -eq 0 ]; then
echo -e "${info}[$(stamp)] -- [INFO] Seafile services stopped" \
"successfully --${normal}" >> "$logFile"
else
# verify services are still actually running
pidShub=$(pgrep -f "seahub")
pidSeaf=$(pgrep -f seafile-controller)
# force stop seahub
if [ "$pidShub" ]; then
pkill -f "seahub"
exitWarn+=("[$(stamp)]_101")
fi
# force stop seafile
if [ "$pidSeaf" ]; then
pkill -f seafile-controller
exitWarn+=("[$(stamp)]_101")
fi
echo -e "${info}[$(stamp)] -- [INFO] Seafile services stopped" \
"successfully --${normal}" >> "$logFile"
fi
fi
}
### generate dynamic timestamps
function stamp {
(date +%F" "%T)
}
### quit -- exit the script after logging any errors, warnings, etc.
function quit {
# list generated warnings, if any
if [ ${#exitWarn[@]} -gt 0 ]; then
echo -e "${warn}${scriptName} generated the following warnings:" \
"${normal}" >> "$logFile"
for warnCode in "${exitWarn[@]}"; do
warnStamp="${warnCode%%_*}"
warnValue="${warnCode##*_}"
echo -e "${warn}${warnStamp} -- [WARNING]" \
"${warningExplain[$warnValue]} (code: ${warnValue}) --" \
"${normal}" >> "$logFile"
done
fi
if [ -z "${exitError}" ]; then
# exit cleanly
echo -e "${note}[$(stamp)] --- ${scriptName} completed" \
"---${normal}" >> "$logFile"
exit 0
else
# list generated errors and explanations then exit script with code 2
echo -e "${err}${scriptName} generated the following errors:" \
"${normal}" >> "$logFile"
for errCode in "${exitError[@]}"; do
errStamp="${errCode%%_*}"
errValue="${errCode##*_}"
echo -e "${err}${errStamp} -- [ERROR] ${errorExplain[$errValue]}" \
"(code: ${errValue}) --${normal}" >> "$logFile"
done
exit 2
fi
}
### End of Functions ###
### Default parameters
# store the logfile in the same directory as this script using the script's name
# with the extension .log
scriptPath="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
scriptName="$( basename ${0} )"
logFile="$scriptPath/${scriptName%.*}.log"
# set default 503 error page name and location in scriptPath
err503Path="$scriptPath/503.html"
err503File="${err503Path##*/}"
# set default sqlDetails path to scriptPath
sqlDetails="$scriptPath/seaf_sql.details"
# set default borgDetails path to scriptPath
borgDetails="$scriptPath/seaf_borg.details"
# set borg parameters to 'normal' verbosity
borgCreateParams='--stats'
borgPruneParams='--list'
### initialize array variables
exitError=()
errorExplain=()
exitWarn=()
warningExplain=()
borgConfig=()
xtraFiles=()
### Error codes
errorExplain[100]="Could not start Seafile service(s)"
errorExplain[200]="Could not dump Seafile SQL databases"
errorExplain[210]="Invalid or non-existant borg base directory specified (borg backup details file)"
errorExplain[211]="Invalid or non-existant path to borg SSH keyfile (borg backup details file)"
errorExplain[212]="Name of borg repo was not specified (borg backup details file)"
errorExplain[215]="Could not find/create 'tmp' directory within borg base directory. Please manually create it and ensure it's writable"
errorExplain[220]="Borg exited with a critical error. Please check this script's logfile for details"
errorExplain[221]="Borg prune exited with ERRORS. Please check this script's logfile for details"
### Warning codes & messages
warningExplain[101]="Could not stop Seafile service(s) normally, had to force"
warningExplain[111]="Could not remove SQL dump file and directory, please remove manually"
warningExplain[5030]="Could not remove 503 error page. This MUST be removed manually before NGINX will serve webclients!"
warningExplain[5031]="No webroot path was specified (-w parameter missing)"
warningExplain[5032]="The specified webroot (-w parameter) could not be found"
warningExplain[5033]="No 503 error page could be found. If not using the default located in the script directory, then check your -5 parameter"
warningExplain[5035]="Error copying 503 error page to webroot"
warn503="Web users will NOT be informed the server is down!"
warningExplain[2111]="No password used for access to remote borg repo. This is an insecure configuration"
warningExplain[2112]="No remote borg instance specified. Operations will be slower in this configuration"
warningExplain[2113]="The specified file containing extra files for inclusion in borgbackup could not be found"
warningExplain[2114]="The specified file containing exclusion patterns for borgbackup could not be found. Backup was performed as though NO exclusions were defined"
warningExplain[2115]="No paramters provided for borg prune. No repo pruning has taken place. You should reconsider this decision to control the size/history of your backups"
warningExplain[2116]="No additional locations are specified for inclusion in backup. ONLY Seafile DATA will be backed up (no configuration, etc). If this is unintentional, check the inclusion file referenced in your borgbackup settings"
warningExplain[2200]="Borg completed with warnings. Please check this script's logfile for details"
warningExplain[2201]="Borg exited with an unknown return-code. Please check this script's logfile for details"
warningExplain[2210]="Borg prune exited with warnings. Please check this script's logfile for details"
warningExplain[2212]="Borg prune exited with an unknown return-code. Please check this script's logfile for details"
### Process script parameters
# If parameters are provided but don't start with '-' then show the help page
# and exit with an error
if [ -n "$1" ] && [[ ! "$1" =~ ^- ]]; then
# show script help page
scriptHelp
fi
# use GetOpts to process parameters
while getopts ':5:b:d:l:p:s:u:v:w:' PARAMS; do
case "$PARAMS" in
5)
# Full path to 503 error page
err503Path="${OPTARG%/}"
err503File="${err503Path##*/}"
;;
b)
# path to file containing borgbackup settings and details
borgDetails="${OPTARG%/}"
;;
d)
# Seafile data directory
dataDir="${OPTARG%/}"
;;
l)
# use provided location for logFile
logFile="${OPTARG%/}"
;;
p)
# Seafile program directory
seafDir="${OPTARG%/}"
;;
s)
# path to file containing SQL login details
sqlDetails="${OPTARG%/}"
;;
u)
# Seafile system user
seafUser="${OPTARG}"
;;
v)
# verbose output from Borg
borgCreateParams='--list --stats'
borgPruneParams='--list'
;;
w)
# path to webserver webroot to copy 503 error page
webroot="${OPTARG%/}"
;;
?)
# unrecognized parameters trigger scriptHelp
scriptHelp
;;
esac
done
### Verify script pre-requisites
## If not running as root, display error on console and exit
if [ $(id -u) -ne 0 ]; then
echo -e "\n${err}This script MUST be run as ROOT. Exiting.${normal}"
exit 3
fi
## Check Seafile program directory
# Ensure Seafile program directory is provided
if [ -z "$seafDir" ]; then
echo -e "\n${err}The Seafile program directory must be specified" \
"(-p parameter)${normal}\n"
exit 1
# Ensure Seafile program directory exists
else
checkExist fd "$seafDir"
checkResult="$?"
if [ "$checkResult" -eq 1 ]; then
# Specified Seafile program directory could not be found
echo -e "\n${err}The provided Seafile program directory" \
"(-p parameter) does not exist.${normal}\n"
exit 1
fi
fi
## Check Seafile user account
# Ensure Seafile user account is provided
if [ -z "$seafUser" ]; then
echo -e "\n${err}The user account running Seafile must be provided" \
"(-u parameter)${normal}\n"
exit 1
# Check if supplied seafUser account exists
else
user_exists=$(id -u "$seafUser" > /dev/null 2>&1; echo $?)
if [ "$user_exists" -ne 0 ]; then
echo -e "\n${err}The supplied user account (-u parameter) does not" \
"exist.${normal}\n"
exit 1
fi
fi
## Ensure sqlDetails file exists
checkExist ff "$sqlDetails"
checkResult="$?"
if [ "$checkResult" = "1" ]; then
# sqlDetails file cannot be found
echo -e "\n${err}The file containing your SQL details does not exist" \
"(-s parameter)${normal}\n"
exit 1
fi
## Ensure borgDetails file exists
checkExist ff "$borgDetails"
checkResult="$?"
if [ "$checkResult" = "1" ]; then
# borgDetails file cannot be found
echo -e "\n${err}The file containing your borgbackup details does not" \
"exist (-b parameter)${normal}\n"
exit 1
fi
## Check Seafile data directory
# Ensure Seafile data directory is provided
if [ -z "$dataDir" ]; then
echo -e "\n${err}The Seafile data directory must be specified" \
"(-d parameter)${normal}\n"
exit 1
# Ensure Seafile data directory exists
else
checkExist fd "$dataDir"
checkResult="$?"
if [ "$checkResult" = "1" ]; then
# Specified Seafile data directory could not be found
echo -e "\n${err}The provided Seafile data directory" \
"(-d parameter) does not exist.${normal}\n"
exit 1
fi
fi
### Log start of script operations
echo -e "${note}[$(stamp)] --- Start $scriptName execution ---${normal}" \
>> "$logFile"
echo -e "${info}[$(stamp)] -- [INFO] Log file located at ${lit}${logFile}" \
"${info}--${normal}" >> "$logFile"
### Export logFile variable for use by Borg
export logFile="$logFile"
### Create sqlDump temporary directory and sqlDumpFilenames
sqlDumpDir=$( mktemp -d )
sqlDump_ccnet="backup-$(date +%Y%m%d_%H%M%S)_ccnet-db.sql"
sqlDump_seafile="backup-$(date +%Y%m%d_%H%M%S)_seafile-db.sql"
sqlDump_seahub="backup-$(date +%Y%m%d_%H%M%S)_seahub-db.sql"
echo -e "${info}[$(stamp)] -- [INFO] mySQL dump files will be temporarily" \
"stored in ${lit}${sqlDumpDir}/${info} --${normal}" >> "$logFile"
### 503 error page: If you don't plan on using the auto-copied 503 then comment
### this entire section starting with '--- Begin 503 section ---' until
### '--- End 503 section ---' to suppress generated warnings
### --- Begin 503 section ---
## Check if webroot has been specified, if not, skip this entire section since there is nowhere to copy the 503 file.
if [ -z "$webroot" ]; then
# no webroot path provided
echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --${normal}" \
>> "$logFile"
exitWarn+=("[$(stamp)]_5031")
clean503=0
else
# verify webroot actually exists
checkExist fd "$webroot"
checkResult="$?"
if [ "$checkResult" -eq 1 ]; then
# webroot directory specified could not be found
echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --${normal}" \
>> "$logFile"
exitWarn+=("{$stamp}_5032")
clean503=0
else
# webroot exists
echo -e "${op}[$(stamp)] Using webroot: ${lit}${webroot}${normal}" \
>> "$logFile"
# Verify 503 file existence at given path
checkExist ff "$err503Path"
checkResult="$?"
if [ "$checkResult" -eq 1 ]; then
# 503 file could not be found
echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --${normal}" \
>> "$logFile"
exitWarn+=("[$(stamp)]_5033")
clean503=0
else
# 503 file exists and webroot is valid. Let's copy it!
echo -e "${op}[$(stamp)] ${err503File} found at" \
"${lit}${err503Path}${normal}" >> "$logFile"
echo -e "${op}[$(stamp)] Copying 503 error page to webroot..." \
"${normal}" >> "$logFile"
cp "${err503Path}" "$webroot/" >> "$logFile" 2>&1
copyResult="$?"
# verify copy was successful
if [ "$copyResult" -eq 1 ]; then
# copy was unsuccessful
echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --" \
"${normal}" >> "$logFile"
exitWarn+=("[$(stamp)]_5035")
clean503=0
else
# copy was successful
echo -e "${info}[$(stamp)] -- [INFO] 503 error page" \
"successfully copied to webroot --${normal}" >> "$logFile"
clean503=1
fi
fi
fi
fi
### --- End 503 section ---
### Stop Seafile services
svcSeafile stop
### Get SQL info from sqlDetails
mapfile -t sqlParams < "$sqlDetails"
### Dump SQL
echo -e "${op}[$(stamp)] Dumping Seafile SQL databases...${normal}" \
>> "$logFile"
mysqldump -h"${sqlParams[0]}" -u"${sqlParams[1]}" -p"${sqlParams[2]}" --opt \
ccnet-db > "${sqlDumpDir}/${sqlDump_ccnet}" 2>> "$logFile"
dumpResult_ccnet=$?
mysqldump -h"${sqlParams[0]}" -u"${sqlParams[1]}" -p"${sqlParams[2]}" --opt \
seafile-db > "${sqlDumpDir}/${sqlDump_seafile}" 2>> "$logFile"
dumpResult_seafile=$?
mysqldump -h"${sqlParams[0]}" -u"${sqlParams[1]}" -p"${sqlParams[2]}" --opt \
seahub-db > "${sqlDumpDir}/${sqlDump_seahub}" 2>> "$logFile"
dumpResult_seahub=$?
# verify
if [ "$dumpResult_ccnet" -eq 0 ] && [ "$dumpResult_seafile" -eq 0 ] && \
[ "$dumpResult_seahub" -eq 0 ]; then
echo -e "${ok}[$(stamp)] -- [SUCCESS] SQL dumped successfully --${normal}" \
>> "$logFile"
else
exitError+=("[$(stamp)]_200")
cleanup
quit
fi
### Call borgbackup to copy actual files
echo -e "${op}[$(stamp)] Pre-backup tasks completed, calling borgbackup..." \
"${normal}" >> "$logFile"
## Get borgbackup settings and repo details
# read definition file and map to array variable
mapfile -t borgConfig < "$borgDetails"
## check if any required borg configuration variables in definition file are
## empty and exit with error, otherwise, map array items to variables
# check: borg base directory
echo -e "${op}[$(stamp)] Verifying supplied borg configuration variables..." \
"${normal}" >> "$logFile"
if [ -z "${borgConfig[0]}" ]; then
exitError+=("[$(stamp)]_210")
cleanup
quit
else
# verify the path actually exists
checkExist fd "${borgConfig[0]}"
checkResult="$?"
if [ "$checkResult" = "1" ]; then
# borg base directory specified could not be found
exitError+=("[$(stamp)]_210")
cleanup
quit
fi
echo -e "${op}[$(stamp)] Borg base dir... OK${normal}" >> "$logFile"
export BORG_BASE_DIR="${borgConfig[0]%/}"
fi
# check: path to SSH keyfile
if [ -z "${borgConfig[1]}" ]; then
exitError+=("[$(stamp)]_211")
cleanup
quit
else
checkExist ff "${borgConfig[1]}"
checkResult="$?"
if [ "$checkResult" = 1 ]; then
# SSH keyfile specified could not be found
exitError+=("[$(stamp)]_211")
cleanup
quit
fi
echo -e "${op}[$(stamp)] Borg SSH key... OK${normal}" >> "$logFile"
export BORG_RSH="ssh -i ${borgConfig[1]}"
fi
# check: name of borg repo
if [ -z "${borgConfig[2]}" ]; then
exitError+=("[$(stamp)]_212")
cleanup
quit
else
echo -e "${op}[$(stamp)] Borg REPO name... OK${normal}" >> "$logFile"
export BORG_REPO="${borgConfig[2]}"
fi
# repo password
if [ -n "${borgConfig[3]}" ]; then
echo -e "${op}[$(stamp)] Borg REPO password... OK${normal}" >> "$logFile"
export BORG_PASSPHRASE="${borgConfig[3]}"
else
exitWarn+=("[$(stamp)]_2111")
# if the password was omitted by mistake, export a dummy password so borg
# fails with an error instead of sitting and waiting for input
export BORG_PASSPHRASE="DummyPasswordSoBorgFails"
fi
# additional files to be backed up
borgXtra="${borgConfig[4]}"
# file with pattern definition for excluded files
borgExclude="${borgConfig[5]}"
# parameters for borg prune
borgPrune="${borgConfig[6]}"
# export: borg remote path (if not blank)
if [ -n "${borgConfig[7]}" ]; then
echo -e "${op}[$(stamp)] Borg REMOTE path... OK${normal}" >> "$logFile"
export BORG_REMOTE_PATH="${borgConfig[7]}"
else
exitWarn+=("[$(stamp)]_2112")
fi
## If borgXtra exists, map contents to an array variable
if [ -n "$borgXtra" ]; then
echo -e "${op}[$(stamp)] Processing referenced extra files list for" \
"borgbackup to include in backup${normal}" >> "$logFile"
checkExist ff "$borgXtra"
checkResult="$?"
if [ "$checkResult" = "0" ]; then
echo -e "${op}[$(stamp)] Found ${lit}${borgXtra}${normal}" >> "$logFile"
mapfile -t xtraFiles < "$borgXtra"
echo -e "${op}[$(stamp)] Processed extra files list for inclusion in" \
"borgbackup${normal}" >> "$logFile"
else
exitWarn+=("[$(stamp)]_2113")
fi
else
# no extra locations specified
echo -e "${op}[$(stamp)] No additional locations specified for backup." \
"Only Seafile data files will be backed up${normal}" >> "$logFile"
exitWarn+=("[$(stamp)]_2116")
fi
## Check if borgExclude exists since borg will throw an error if it's missing
if [ -n "$borgExclude" ]; then
checkExist ff "$borgExclude"
checkResult="$?"
if [ "$checkResult" = "0" ]; then
echo -e "${op}[$(stamp)] Found ${lit}${borgExclude}${normal}" \
>> "$logFile"
else
# file not found, unset the variable so it's like it was not specified
# in the first place and continue with backup
unset borgExclude
exitWarn+=("[$(stamp)]_2114")
fi
else
echo -e "${op}[$(stamp)] Exclusion pattern file not specified." \
"No exclusions will be processed${normal}" >> "$logFile"
fi
## Export TMPDIR environment variable for borg via python
## Python requires a writable temporary directory when unpacking borg and
## executing commands. This defaults to /tmp but many systems mount /tmp with
## the 'noexec' option for security. Thus, we will use/create a 'tmp' folder
## within the BORG_BASE_DIR and instruct python to use that instead of /tmp
# check if BORG_BASE_DIR/tmp exists, if not, create it
echo -e "${op}[$(stamp)] Checking for tmp directory at ${lit}${BORG_BASE_DIR}" \
"${normal}" >> "$logFile"
checkExist fd "$BORG_BASE_DIR/tmp"
checkResult="$?"
if [ "$checkResult" = "1" ]; then
# folder not found
echo -e "${op}[$(stamp)] tmp folder not found... creating${lit}" \
"${BORG_BASE_DIR}/tmp${normal}" >> "$logFile"
mkdir "$BORG_BASE_DIR/tmp" 2>> "$logFile"
# verify folder created
checkExist fd "$BORG_BASE_DIR/tmp"
checkResult="$?"
if [ "$checkResult" = "0" ]; then
# folder exists
echo -e "${op}[$(stamp)] tmp folder created within borg base directory" \
"${normal}" >> "$logFile"
else
# problem creating folder and script will exit
exitError+=("[$(stamp)]_215")
cleanup
quit
fi
else
# folder found
echo -e "${op}[$(stamp)] tmp folder found within borg base directory" \
"${normal}" >> "$logFile"
fi
# export TMPDIR environment variable
export TMPDIR="${BORG_BASE_DIR}/tmp"
## Generate and execute borg
# commandline depends on whether borgExclude is empty or not
if [ -z "$borgExclude" ]; then
# borgExclude is empty
echo -e "${info}[$(stamp)] --[INFO] Executing borg without exclusions --" \
"${normal}" >> "$logFile"
borg --show-rc create ${borgCreateParams} ::"$(date +%Y-%m-%d_%H%M%S)" \
"${xtraFiles[@]}" \
"${seafDir}" \
"${dataDir}" \
"${sqlDumpDir}" \
2>> "$logFile"
else
# borgExclude is not empty
echo -e "${info}[$(stamp)] --[INFO] Executing borg with exclusions --" \
"${normal}" >> "$logFile"
borg --show-rc create ${borgCreateParams} --exclude-from "${borgExclude}" \
::"$(date +%Y-%m-%d_%H%M%S)" \
"${xtraFiles[@]}" \
"${seafDir}" \
"${dataDir}" \
"${sqlDumpDir}" \
2>> "$logFile"
fi
## Check status of borg operation
borgResult="$?"
if [ "$borgResult" -eq 0 ]; then
echo -e "${ok}[$(stamp)] -- [SUCCESS] Borg backup completed successfully --" \
"${normal}" >> "$logFile"
elif [ "$borgResult" -eq 1 ]; then
exitWarn+=("[$(stamp)]_2200")
elif [ "$borgResult" -ge 2 ]; then
exitError+=("[$(stamp)]_220")
cleanup
quit
else
exitWarn+=("[$(stamp)]_2201")
fi
## Generate and execute borg prune
# command depends on whether or not parameters have been defined
if [ -n "$borgPrune" ]; then
# parameters defined
echo -e "${info}[$(stamp)] --[INFO] Executing borg prune operation --" \
"${normal}" >> "$logFile"
borg prune --show-rc -v ${borgPruneParams} ${borgPrune} \
2>> "$logFile"
# check return-status
pruneResult="$?"
if [ "$pruneResult" -eq 0 ]; then
echo -e "${ok}[$(stamp)] -- [SUCCESS] Borg prune completed successfully" \
"--${normal}" >> "$logFile"
elif [ "$pruneResult" -eq 1 ]; then
exitWarn+=("[$(stamp)]_2210")
elif [ "$pruneResult" -ge 2 ]; then
exitError+=("[$(stamp)]_221")
else
exitWarn+=("[$(stamp)]_2212")
fi
else
# parameters not defined... skip pruning
exitWarn+=("[$(stamp)]_2115")
fi
### borgbackup completed
echo -e "${op}[$(stamp)] Borgbackup completed... begin cleanup" \
"${normal}" >> "$logFile"
### Exit script
echo -e "${bold}${op}[$(stamp)] ***Normal exit process***${normal}" \
>> "$logFile"
cleanup
echo -e "${bold}${ok}[$(stamp)] -- [SUCCESS] All processes completed" \
"successfully --${normal}" >> "$logFile"
quit
# This code should not be executed since the 'quit' function should terminate
# this script. Therefore, exit with code 99 if we get to this point.
exit 99
-1
View File
@@ -1 +0,0 @@
EOF
-8
View File
@@ -1,8 +0,0 @@
<path to borgbackup base directory> /var/borgbackup
<path to SSH private key for remote server> /var/borgbackup/sshPrivate.key
<connection string to remote repo> user@servername.tld:repoName/
<password for repo> pAsSwOrd
<path to file listing extra files> /root/SeafBackup/xtraLocations.borg
<path to file with exclusions> /root/SeafBackup/excludeLocations.borg
<purge timeframe options> --keep-within=7d --keep-daily=30 --keep-weekly=12 --keep-monthly=-1
<location of borg remote instance> borg1
-3
View File
@@ -1,3 +0,0 @@
<name of machine hosting mySQL> localhost
<name of authorized user> seafile
<password for above user> pAsSwOrd
-28
View File
@@ -1,28 +0,0 @@
/etc/fstab
/etc/network/interfaces
/etc/network/interfaces.d/
/etc/systemd/timesyncd.conf
/etc/profile
/etc/bash.bashrc
/etc/skel/
/etc/nanorc
/etc/msmtprc
/etc/msmtp_aliases
/etc/apt/sources.list
/etc/apt/sources.list.d/
/etc/apt/listchanges.conf
/etc/apt/apt.conf.d/50unattended-upgrades
/etc/apt/apt.conf.d/20auto-upgrades
/etc/ssh/
/etc/logwatch/
/etc/letsencrypt/
/etc/fail2ban/fail2ban.local
/etc/fail2ban/jail.local
/etc/fail2ban/filter.d/seafile-auth.conf
/etc/fail2ban/jail.d/
/root/.bashrc
/root/SeafBackup/
/root/.ssh/
/etc/mysql/my.cnf
/etc/nginx/
/usr/share/nginx/html/
+15
View File
@@ -0,0 +1,15 @@
### Rotate backup log file
# location of log file (-l parameter of script file)
/path/to/backup.log {
# rotate log file weekly -- you could also use 'daily', 'monthly' or
# specify a size using 'size 100k', for example
weekly
# keep 4 weeks of old logs (4 files) and delete older ones
rotate 4
# compress old log files using gzip (default)
compress
}
@@ -0,0 +1,13 @@
# Location of your script's log file, -l parameter
LogFile = /path/to/your/seafbackup.log
# Format of logrotate archives for your script. Example assumes compression and
# extension preservation
Archive = /path/to/your/seafbackup.log.?.gz
# Apply the correct date/time filtering to match the format of the script's log
# We are using a custom pl script in /etc/logwatch/scripts/shared/ You don't
# need to change this unless you have altered the 'stamp' function in the backup
# script in which case you will want to update the regex in the custom pl script
# below
*sqFullStampAnywhere
@@ -0,0 +1,9 @@
# Name of the logfile group without any extension
LogFile = seafbackup
# Heading displayed on Logwatch's report for this service
Title = "System and Seafile Backup"
# Override the detail level for this service
# Remember the levels are: 0, 1-4, 5, 6+
# Detail = 0
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/perl
#############################################################################
# $Id$
#############################################################################
# Log: Seafile backup script (seafbackup, Logwatch group 'backup')
# Revision 1.1 2019/07/20
# Written by Asif Bacchus
#############################################################################
use strict;
### Get Logwatch detail level (default to 0)
my $detailLevel = $ENV{'LOGWATCH_DETAIL_LEVEL'} || 0;
### Declare variables
my $summaryErr;
my $summaryWarn;
my $summarySuccess;
my %reportHash = ();
my $key;
### Minimal detail level: provide summary data only
if ($detailLevel == 0) {
### process logfile and summarize message types
while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /\-- \[ERROR\] /) {
$summaryErr++;
}
elsif ($ThisLine =~ /\-- \[WARNING\] /) {
$summaryWarn++;
}
elsif ($ThisLine =~ /All processes completed/) {
$summarySuccess++;
}
}
### fill hash table with headings and summary counts
if ($summarySuccess > 0) {
$reportHash{"All processes successfully completed"} = $summarySuccess;
}
if ($summaryWarn > 0) {
$reportHash{"Warnings issued"} = $summaryWarn;
}
if ($summaryErr > 0) {
$reportHash{"Errors encountered"} = $summaryErr;
}
### print hash table
foreach $key (sort keys %reportHash) {
print "$key: $reportHash{$key}\n";
}
}
### Levels 1-4 provide the actual error, warning and success messages instead
### of a summary count
elsif ($detailLevel >= 1 && $detailLevel <= 4) {
while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /\-- \[ERROR\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[WARNING\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[SUCCESS\] /) {
print $ThisLine;
}
}
}
### Level 5 is similiar to levels 1-4 except it also reports informational
### messages such as the location of script created files, variable checks,
### etc. This is useful when verifying the script's operation.
elsif ($detailLevel == 5) {
while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /\-- \[ERROR\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[WARNING\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[SUCCESS\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[INFO\] /) {
print $ThisLine;
}
}
}
### Any level above 5 will echo the entire log including the debugging notes
### within the script meant for troubleshooting. Using this level of detail
### should only be done if you cannot view the actual log file directly for
### whatever reason. The actual log file is colour-coded for easier debugging.
elsif ($detailLevel > 5) {
while (defined(my $ThisLine = <STDIN>)) {
print $ThisLine;
}
}
### Exit gracefully
exit (0);
# vi: shiftwidth=3 tabstop=3 et
# Local Variables:
# mode: perl
# perl-indent-level: 3
# indent-tabs-mode: nil
# End:
@@ -0,0 +1,37 @@
#!/usr/bin/perl
##########################################################################
# $Id$
##########################################################################
###############################################################################
## Filter dates in full-date-time international format, surrounded by square
## brackets located anywhere on a given line
## Format: '[%Y-%m-%d %H:%M:%S]'
###############################################################################
use Logwatch ':dates';
my $Debug = $ENV{'LOGWATCH_DEBUG'} || 0;
### Specify the format of the date/time stamp itself
$SearchDate = TimeFilter('%Y-%m-%d %H:%M:%S');
if ( $Debug > 5 ) {
print STDERR "DEBUG: Inside FullDateTime...\n";
print STDERR "DEBUG: Looking For: " . $SearchDate . "\n";
}
while (defined($ThisLine = <STDIN>)) {
### specify the regex that defines how to find 'SearchDate'
if ($ThisLine =~ m/\[$SearchDate\] /o) {
print $ThisLine;
}
}
# vi: shiftwidth=3 syntax=perl tabstop=3 et
# Local Variables:
# mode: perl
# perl-indent-level: 3
# indent-tabs-mode: nil
# End:
-22
View File
@@ -1,22 +0,0 @@
#######
### Seafile backup script configuration details
###
### this file contains sensitive information, make sure you have protected
### it by restricting permissions!
### 'chown root:root ./seafbackup.details'
### 'chmod 600 ./seafbackup.details'
#######
### SQL server details
## database names
# if you are using the defaults, then you don't have to change anything here
ccnetDB_name="ccnet-db"
seafileDB_name="seafile-db"
seahubDB_name="seahub-db"
## database credentials
# you should be using the same credentials that SEAFILE uses!
sqlServer="localhost"
sqlUser="seafile"
sqlPass="password"
@@ -18,11 +18,8 @@
<body> <body>
<h1>Bad timing!</h1> <h1>Bad timing!</h1>
<p>Seems you're trying to access me during my daily backup window. Don't <p>Seems you're trying to access me during my daily backup window. Don't worry though, I should be up and running again very soon.</p>
worry though, I should be up and running again very soon.</p> <p>My average backup window duration is pretty short and I'm quite busy during that time copying your super-important files to my secure hiding place so they stay safe in case anything ever happens to me!</p>
<p>My average backup window duration is around 10 minutes during which time
I'm quite busy copying your super-important files to my ultra-secret hiding
place so they stay safe in case anything ever happens to me!</p>
<h3><em>I'm really sorry for the delay. Please try me again soon!</em></h3> <h3><em>I'm really sorry for the delay. Please try me again soon!</em></h3>
</body> </body>
+5
View File
@@ -0,0 +1,5 @@
'/var/seafile/seafile-data/httptemp/'
'/var/seafile/seafile-data/thumbnails/'
'/var/seafile/seafile-data/tmpfiles/'
'/var/seafile/seafile-data/webdavtmp/'
EOF
+80
View File
@@ -0,0 +1,80 @@
#######
### Seafile backup script configuration details
###
### This file contains sensitive information, make sure you have protected
### it by restricting permissions!
### Run the following in the directory where this file is located:
### chown root:root ./seafbackup.details
### chmod 600 ./seafbackup.details
###
### Do NOT include any commands in this file as they WILL be executed!!!
#######
### SQL server details
## database names
# if you are using the Seafile script installation defaults, then you don't
# have to change anything here
ccnetDB_name="ccnet-db"
seafileDB_name="seafile-db"
seahubDB_name="seahub-db"
## database credentials
# you should be specifying the same credentials that SEAFILE uses!
# DNS name or ip address of your sql server (mysql -h)
# this is most likely "localhost" or "127.0.0.1" assuming mysql/mariadb
# is installed on this machine along with Seafile
sqlServer="localhost"
# username that Seafile uses to connect to it's SQL databases (mysql -u)
# you set this up when Seafile was installed, likely via the mysql script
sqlUser="seafile"
# password for the above SQL user (mysql -p)
sqlPass="p@ssW0rd"
### borg details
# if you're unsure what to enter here, please consult the repo wiki and/or
# the borg documentation
# base configuration directory for borg, all borg parameters use this directory
# as their 'root'. I recommended setups with this being "/var/borgbackup", the
# default is "$HOME" or "~$USER" in that order. If you're unsure, try "$HOME"
borgBaseDir="/var/borgbackup"
# full path to the SSH key used to connect to your remote backup server
borgSSHKey="/var/borgbackup/private.key"
# connection string to access the borg repo on your remote backup server
# this is usually in the form user@servername.tld:repoName/
borgConnectRepo="jdoe123@borg.server.net:seafileBackup/"
# password to access repo
# this was set when the repo was initialized and, while optional, is HIGHLY
# recommended for security
borgRepoPassphrase="p@ssW0rd"
# OPTIONAL: path to text file containing a list (one per line) of files/
# directories to include in your backup along with Seafile data
# see repo wiki for more details
# leave blank if you only want to backup Seafile related files/directories.
borgXtraListPath="/root/seafileBackup/xtraLocations.borg"
# OPTIONAL: path to file containing files/directories or 'patterns' to be
# excluded in a BORG RECOGNIZED format
# see repo wiki for more details or consult borg documentation
# leave blank for no exclusions.
borgExcludeListPath="/root/seafileBackup/excludeLocations.borg"
# parameters to determine how borg deletes aged backups
# more details in the repo wiki and/or borg documentation
# leave blank to skip pruning altogether -- NOT recommended!
borgPruneSettings="--keep-within=7d --keep-daily=30 --keep-weekly=12 --keep-monthly=-1"
# location of borg instance on your remote backup server
# this is very often just "borg1"
borgRemote="borg1"
+141 -16
View File
@@ -12,7 +12,11 @@ warn=$(tput bold)$(tput setaf 3)
ok=$(tput setaf 2) ok=$(tput setaf 2)
yellow=$(tput setaf 3) yellow=$(tput setaf 3)
cyan=$(tput setaf 6) cyan=$(tput setaf 6)
mag=$(tput setaf 5) magenta=$(tput setaf 5)
### trap
trap trapExit 1 2 3 6
### functions ### functions
@@ -79,13 +83,93 @@ cleanup () {
fi fi
} }
# call cleanup and then exit with error report
exitError () { exitError () {
printf "${err}[%s] -- [ERROR] %s: %s --${norm}\n" \ printf "${err}[%s] -- [ERROR] %s: %s --${norm}\n" \
"$(stamp)" "$1" "$2" >> "$logFile" "$(stamp)" "$1" "$2" >> "$logFile"
cleanup cleanup
# note script completion with error
printf "${err}[%s] --- %s execution completed with error ---${norm}\n" \
"$(stamp)" "$scriptName" >> "$logFile"
exit "$1" exit "$1"
} }
# display script help information
scriptHelp () {
printf "\n${cyan}%s\n" "--------------------------------------------------------------------------------"
printf "Usage:${norm} %s [parameters]\n\n" "$scriptName"
printf "${cyan}Parameters ${yellow}(default value):${norm}\n"
printf "There are NO mandatory parameters. If a parameter is not "
printf "supplied, it's\n"
printf "default value will be used. In the case of a switch parameter, "
printf "it will\n"
printf "remain deactivated if not specified.\n\n"
printf "${magenta}script related\n"
printf "${cyan}-c, --config, --details${norm}\n"
printf "Path to the configuration key/value pair file for this script.\n"
printf "${yellow}(scriptPath/seafbackup.details)${norm}\n\n"
printf "${cyan}-h, -?, --help${norm}\n"
printf "This help screen.\n\n"
printf "${cyan}-l, --log${norm}\n"
printf "Path to write log file.\n"
printf "${yellow}(scriptPath/scriptName.log)${norm}\n\n"
printf "${cyan}-v, --verbose${norm}\n"
printf "Log borg output with increased verbosity (list all files). "
printf "Careful! Your\n"
printf "log file can get large very quickly! This is a switch value, "
printf "specifying it\n"
printf "will turn the option ON.\n"
printf "${yellow}(normal output, option is OFF)${norm}\n\n"
printf "${magenta}503 functionality\n"
printf "${cyan}-5, --use-503${norm}\n"
printf "Copy an 'error 503' page to your webroot for your webserver to "
printf "find. This is a\n"
printf "switch value, specifying it will instruct the script to copy the "
printf "error page.\n"
printf "${yellow}(do NOT copy, option is OFF)${norm}\n\n"
printf "${cyan}--503-path${norm}\n"
printf "Path to the file you want copied to your webroot as the 'error "
printf "503' page during\n"
printf "backup operations.\n"
printf "${yellow}(scriptPath/503_backup.html)${norm}\n\n"
printf "${cyan}-w, --webroot${norm}\n"
printf "Path to where the 'error 503' file should be copied during "
printf "backups.\n"
printf "${yellow}(/usr/share/nginx/html)${norm}\n\n"
printf "${magenta}seafile related\n"
printf "${cyan}-d, --data, --datadir, --seafdata${norm}\n"
printf "Path to your seafile DATA base/root directory."
printf "${yellow}(/var/seafile)${norm}\n\n"
printf "${cyan}-o, --offline${norm}\n"
printf "Stop your seafile systemd services and start them up again after "
printf "the backup.\n"
printf "This is a switch value.\n"
printf "${yellow}(leave seafile running, OFF)${norm}\n\n"
printf "${cyan}-p, --seaf, --seafdir${norm}\n"
printf "${norm}Path to your seafile PROGRAM base/root directory.\n"
printf "${yellow}(/opt/seafile)${norm}\n\n"
printf "${cyan}--seafile-service${norm}\n"
printf "The name of your seafile systemd service. Use with '--offline' "
printf "when using a\n"
printf "non-default service name.\n"
printf "${yellow}(seafile.service)${norm}\n\n"
printf "${cyan}--seahub-service${norm}\n"
printf "The name of your seahub (web) systemd service. Use with "
printf "'--offline' when using\n"
printf "a non-default service name.\n"
printf "${yellow}(seahub.service)${norm}\n\n"
printf "More details and examples of script usage can be found in the "
printf "repo wiki at\n"
printf "${yellow}https://git.asifbacchus.app/BacchusGroup/seafileBackup"
printf "/wiki${norm}.\n"
printf "${cyan}%s${norm}\n\n" "--------------------------------------------------------------------------------"
}
# control seafile services (systemd) # control seafile services (systemd)
seafSvc () { seafSvc () {
if [ "$1" = "start" ]; then if [ "$1" = "start" ]; then
@@ -122,6 +206,17 @@ stamp () {
(date +%F" "%T) (date +%F" "%T)
} }
# same as exitError but for signal captures
trapExit () {
printf "${err}[%s] -- [ERROR] 99: Caught signal --${norm}\n" \
"$(stamp)" >> "$logFile"
cleanup
# note script completion with error
printf "${err}[%s] --- %s execution was terminated via signal ---${norm}\n" \
"$(stamp)" "$scriptName" >> "$logFile"
exit 99
}
### end of functions ### end of functions
@@ -134,7 +229,7 @@ scriptPath="$( CDPATH='' cd -- "$( dirname -- "$0" )" && pwd -P )"
scriptName="$( basename "$0" )" scriptName="$( basename "$0" )"
logFile="$scriptPath/${scriptName%.*}.log" logFile="$scriptPath/${scriptName%.*}.log"
warnCount=0 warnCount=0
configDetails="$scriptPath/seafbackup.details" configDetails="$scriptPath/${scriptName%.*}.details"
err503Copied=0 err503Copied=0
sqlCopied=0 sqlCopied=0
includeXtra=0 includeXtra=0
@@ -292,6 +387,11 @@ done
### check pre-requisites and default values ### check pre-requisites and default values
# check if running as root, otherwise exit
if [ $( id -u ) -ne 0 ]; then
printf "\n${err}ERROR: script MUST be run as ROOT${norm}\n\n"
exit 2
fi
# does the details file exist? # does the details file exist?
if [ ! -f "$configDetails" ]; then if [ ! -f "$configDetails" ]; then
badParam dne "(--details default)" "$configDetails" badParam dne "(--details default)" "$configDetails"
@@ -330,7 +430,7 @@ fi
### start logging ### start logging
printf "${mag}[%s] --- Start %s execution ---${norm}\n" \ printf "${magenta}[%s] --- Start %s execution ---${norm}\n" \
"$(stamp)" "$scriptName" >> "$logFile" "$(stamp)" "$scriptName" >> "$logFile"
printf "${cyan}[%s] -- [INFO] Log located at ${yellow}%s${cyan} --${norm}\n" \ printf "${cyan}[%s] -- [INFO] Log located at ${yellow}%s${cyan} --${norm}\n" \
"$(stamp)" "$logFile" >> "$logFile" "$(stamp)" "$logFile" >> "$logFile"
@@ -381,6 +481,27 @@ printf "${cyan}[%s] -- [INFO] ${yellow}%s${cyan} imported --${norm}\n" \
### dump sql databases ### dump sql databases
## verify SQL variables are defined in details file
# sqlServer
if [ -z "${sqlServer}" ]; then
badDetails empty 'sqlServer'
fi
# sqlUser
if [ -z "${sqlUser}" ]; then
badDetails empty 'sqlUser'
fi
# sqlPass
if [ -z "${sqlPass}" ]; then
badDetails empty 'sqlPass'
fi
# database names
if [ -z "${ccnetDB_name}" ] || [ -z "${seafileDB_name}" ] || \
[ -z "${seahubDB_name}" ]; then
badDetails empty 'one or more mysql database names'
fi
## create tmp directory and generate dumpfile names
printf "${cyan}[%s] -- [INFO] Dumping SQL databases --${norm}\n" \ printf "${cyan}[%s] -- [INFO] Dumping SQL databases --${norm}\n" \
"$(stamp)" >> "$logFile" "$(stamp)" >> "$logFile"
# create temporary directory to dump files before borg backup # create temporary directory to dump files before borg backup
@@ -396,23 +517,26 @@ printf "\n${yellow}%s/${cyan} --${norm}\n" "$sqlDumpDir" >> "$logFile"
sqlDump_ccnet="backup-$(date +%Y%m%d_%H%M%S)_${ccnetDB_name}.sql" sqlDump_ccnet="backup-$(date +%Y%m%d_%H%M%S)_${ccnetDB_name}.sql"
sqlDump_seafile="backup-$(date +%Y%m%d_%H%M%S)_${seafileDB_name}.sql" sqlDump_seafile="backup-$(date +%Y%m%d_%H%M%S)_${seafileDB_name}.sql"
sqlDump_seahub="backup-$(date +%Y%m%d_%H%M%S)_${seahubDB_name}.sql" sqlDump_seahub="backup-$(date +%Y%m%d_%H%M%S)_${seahubDB_name}.sql"
## dump databases ## dump databases
# dump CCNET-DB # dump CCNET-DB
if ! mysqldump -h"${sqlServer}" -u"${sqlUser}" -p"${sqlPass}" \ if ! mysqldump -h"${sqlServer}" -u"${sqlUser}" -p"${sqlPass}" \
--opt ccnet-db > "${sqlDumpDir}/${sqlDump_ccnet}" 2>> "$logFile"; then --opt "${ccnetDB_name}" > "${sqlDumpDir}/${sqlDump_ccnet}" \
2>> "$logFile"; then
exitError 115 "Could not dump ${ccnetDB_name} database" exitError 115 "Could not dump ${ccnetDB_name} database"
fi fi
# dump SEAFILE-DB # dump SEAFILE-DB
if ! mysqldump -h"${sqlServer}" -u"${sqlUser}" -p"${sqlPass}" \ if ! mysqldump -h"${sqlServer}" -u"${sqlUser}" -p"${sqlPass}" \
--opt ccnet-db > "${sqlDumpDir}/${sqlDump_seafile}" 2>> "$logFile"; then --opt "${seafileDB_name}" > "${sqlDumpDir}/${sqlDump_seafile}" \
2>> "$logFile"; then
exitError 116 "Could not dump ${seafileDB_name} database" exitError 116 "Could not dump ${seafileDB_name} database"
fi fi
# dump CCNET-DB # dump CCNET-DB
if ! mysqldump -h"${sqlServer}" -u"${sqlUser}" -p"${sqlPass}" \ if ! mysqldump -h"${sqlServer}" -u"${sqlUser}" -p"${sqlPass}" \
--opt ccnet-db > "${sqlDumpDir}/${sqlDump_seahub}" 2>> "$logFile"; then --opt "${seahubDB_name}" > "${sqlDumpDir}/${sqlDump_seahub}" 2>> "$logFile"; then
exitError 117 "Could not dump ${seahubDB_name} database" exitError 117 "Could not dump ${seahubDB_name} database"
fi fi
printf "${ok}[%s] -- [SUCCESS] SQL databases dumped successfully --${norm}\n" \ printf "${ok}[%s] -- [SUCCESS] Dumped SQL databases --${norm}\n" \
"$(stamp)" >> "$logFile" "$(stamp)" >> "$logFile"
@@ -432,17 +556,17 @@ elif [ ! -d "${borgBaseDir}" ]; then
badDetails dne 'borgBaseDir' badDetails dne 'borgBaseDir'
fi fi
printf "${magenta}details:borgBaseDir ${norm}-- ${ok}[OK]${norm}\n" \ printf "${magenta}details:borgBaseDir ${norm}-- ${ok}[OK]${norm}\n" \
>> "$logfile" >> "$logFile"
export BORG_BASE_DIR="${borgBaseDir%/}" export BORG_BASE_DIR="${borgBaseDir%/}"
## check path to SSH keyfile ## check path to SSH keyfile
if [ -z "${borgSSHKey}" ]; then if [ -z "${borgSSHKey}" ]; then
badDetails empty 'borgSSHKey' badDetails empty 'borgSSHKey'
elif [ ! -d "${borgSSHKey}" ]; then elif [ ! -f "${borgSSHKey}" ]; then
badDetails dne 'borgSSHKey' badDetails dne 'borgSSHKey'
fi fi
printf "${magenta}details:borgSSHKey ${norm}-- ${ok}[OK]${norm}\n" \ printf "${magenta}details:borgSSHKey ${norm}-- ${ok}[OK]${norm}\n" \
>> "$logfile" >> "$logFile"
export BORG_RSH="ssh -i ${borgSSHKey}" export BORG_RSH="ssh -i ${borgSSHKey}"
## check borg repo connect string ## check borg repo connect string
@@ -450,13 +574,13 @@ if [ -z "${borgConnectRepo}" ]; then
badDetails empty 'borgConnectRepo' badDetails empty 'borgConnectRepo'
fi fi
printf "${magenta}details:borgConnectRepo ${norm}-- ${ok}[OK]${norm}\n" \ printf "${magenta}details:borgConnectRepo ${norm}-- ${ok}[OK]${norm}\n" \
>> "$logfile" >> "$logFile"
export BORG_REPO="${borgConnectRepo}" export BORG_REPO="${borgConnectRepo}"
## check borg repo password ## check borg repo password
if [ -n "${borgRepoPassphrase}" ]; then if [ -n "${borgRepoPassphrase}" ]; then
printf "${magenta}details:borgRepoPassphrase ${norm}-- ${ok}[OK]${norm}\n" \ printf "${magenta}details:borgRepoPassphrase ${norm}-- ${ok}[OK]${norm}\n" \
>> "$logfile" >> "$logFile"
export BORG_PASSPHRASE="${borgRepoPassphrase}" export BORG_PASSPHRASE="${borgRepoPassphrase}"
else else
# if passwd is blank intentionally, this is insecure # if passwd is blank intentionally, this is insecure
@@ -489,7 +613,7 @@ if [ -n "${borgXtraListPath}" ]; then
$( sed -e '/^\s*#.*$/d' -e '/^\s*$/d' "${borgXtraListPath}" ) $( sed -e '/^\s*#.*$/d' -e '/^\s*$/d' "${borgXtraListPath}" )
EOF EOF
printf "${magenta}details:borgXtraListPath ${norm}-- ${ok}[OK]${norm}\n" \ printf "${magenta}details:borgXtraListPath ${norm}-- ${ok}[OK]${norm}\n" \
>> "$logfile" >> "$logFile"
includeXtra=1 includeXtra=1
fi fi
@@ -625,7 +749,7 @@ cleanup
# note complete success, tally warnings and exit # note complete success, tally warnings and exit
printf "${ok}[%s] -- [SUCCESS] All processes completed --${norm}\n" \ printf "${ok}[%s] -- [SUCCESS] All processes completed --${norm}\n" \
"$(stamp)" >> "$logFile" "$(stamp)" >> "$logFile"
printf "${mag}[%s] --- %s execution completed ---${norm}\n" \ printf "${magenta}[%s] --- %s execution completed ---${norm}\n" \
"$(stamp)" "$scriptName" >> "$logFile" "$(stamp)" "$scriptName" >> "$logFile"
if [ "$warnCount" -gt 0 ]; then if [ "$warnCount" -gt 0 ]; then
printf "${warn}%s warnings issued!${norm}\n" "${warnCount}" >> "$logFile" printf "${warn}%s warnings issued!${norm}\n" "${warnCount}" >> "$logFile"
@@ -640,6 +764,7 @@ exit 0
# 1: parameter error # 1: parameter error
# 2: not run as root # 2: not run as root
# 3: borg not installed # 3: borg not installed
# 99: TERM signal trapped
# 100: could not start seafile service # 100: could not start seafile service
# 101: could not start seahub service # 101: could not start seahub service
# 102: could not stop seafile service # 102: could not stop seafile service
@@ -648,7 +773,7 @@ exit 0
# 115: could not dump CCNET-DB # 115: could not dump CCNET-DB
# 116: could not dump SEAFILE-DB # 116: could not dump SEAFILE-DB
# 117: could not dump SEAHUB-DB # 117: could not dump SEAHUB-DB
# 130: null borg configuration variable # 130: null configuration variable in details file
# 131: invalid borg configuration variable # 131: invalid configuration variable in details file
# 138: borg exited with a critical error # 138: borg exited with a critical error
# 139: borg prune exited with a critical error # 139: borg prune exited with a critical error
+66
View File
@@ -0,0 +1,66 @@
# Files and directories listed here will be included in your borg backup along
# with your Seafile program and data files.
# List the path to files/directories one per line.
# Any blank lines will be ignored.
# Any lines starting with '#' will be ignored as a comment.
# For consistency, you should include the trailing slash for directories.
# important system configuration files for reference during a restore
/etc/fstab
/etc/network/interfaces
/etc/network/interfaces.d/
/etc/systemd/timesyncd.conf
/etc/systemd/system/seafile.service
/etc/systemd/system/seahub.service
# ssh configuration and hostkeys
/etc/ssh/
# apt configuration (auto updates and source lists)
/etc/apt/sources.list
/etc/apt/sources.list.d/
/etc/apt/listchanges.conf
/etc/apt/apt.conf.d/50unattended-upgrades
/etc/apt/apt.conf.d/20auto-upgrades
# user profile defaults and configurations
/etc/profile
/etc/bash.bashrc
/etc/skel/
/etc/nanorc
# root user files
/root/.bashrc
/root/.ssh/
/root/seafileBackup/
### configuration files for programs related to Seafile
# letsencrypt
/etc/letsencrypt/
# mysql/mariadb
/etc/mysql/my.cnf
/etc/mysql/conf.d/
# NGINX
/etc/nginx/
/usr/share/nginx/html/
### other installed program configurations
# fail2ban
/etc/fail2ban/fail2ban.local
/etc/fail2ban/jail.local
/etc/fail2ban/filter.d/seafile-auth.conf
/etc/fail2ban/jail.d/
# logwatch
/etc/logwatch/
# msmtp
/etc/msmtprc
/etc/msmtp_aliases