Compare commits

...

16 Commits

Author SHA1 Message Date
Asif Bacchus 0fcab85da9 fix incorrect filename 2019-07-30 01:11:15 -06:00
Asif Bacchus ed62bb1e38 update logwatch integration 2019-07-30 00:44:37 -06:00
Asif Bacchus 3a759f39ac reworded redundant success msgs 2019-07-30 00:41:02 -06:00
Asif Bacchus 5cd023010c update wiki link, match details filename to script 2019-07-30 00:39:17 -06:00
Asif Bacchus 8e30cb4048 fix timestamp double-bracket 2019-07-26 03:16:55 -06:00
Asif Bacchus 8edd833544 fix final success msg output spacing err 2019-07-26 03:15:26 -06:00
Asif Bacchus dd49378678 check running as root 2019-07-26 03:14:12 -06:00
Asif Bacchus efe202d9a3 new directory structure 2019-07-26 03:11:53 -06:00
Asif Bacchus 9936cf7de3 add link to wiki 2019-07-20 18:14:14 -06:00
Asif Bacchus 0b1d220b5b update readme and license 2019-07-20 18:09:19 -06:00
Asif Bacchus a299cca365 move logwatch under etc dir to mirror real sys 2019-07-20 17:17:33 -06:00
Asif Bacchus 271f08d0e4 sample logrotate config 2019-07-20 17:16:58 -06:00
Asif Bacchus 55162342c7 add logwatch example files 2019-07-20 17:14:58 -06:00
Asif Bacchus fdb139afc9 change success msg for logwatch parsing 2019-07-20 17:14:44 -06:00
Asif Bacchus 93bfdbfeb3 enclose timestamps in square brackets 2019-07-20 17:06:20 -06:00
Asif Bacchus 3f1861fc01 update example default locations 2019-07-20 16:57:37 -06:00
12 changed files with 259 additions and 13 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License MIT License
Copyright (c) <year> <copyright holders> Copyright (c) 2019 Asif Bacchus
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal of this software and associated documentation files (the "Software"), to deal
+61 -5
View File
@@ -1,7 +1,63 @@
# seafileBackup # Backup a bare-metal Seafile installation using borgbackup <!-- omit in toc -->
Backup Seafile DB and data using borgbackup to rsync.net Simple POSIX-compliant script to backup a bare-metal Seafile installation using borgbackup to a remote SSH-capable storage system. I strongly suggest using rsync.net since they are very reliable, fast and even have special pricing for people using borgbackup/attic [(details here)](https://www.rsync.net/products/attic.html).
This readme will be an overiew of the purpose of the repo and BASIC usability **THIS SCRIPT USES BORG [(borgbackup)](https://borgbackup.readthedocs.io/en/stable/) SO PLEASE ENSURE THAT IS INSTALLED CORRECTLY BEFORE TRYING TO USE THIS SCRIPT!**
information. Detailed usage information will all be presented in the repo's
wiki. - [Basic features of this script:](#Basic-features-of-this-script)
- [Check out the wiki](#Check-out-the-wiki)
- [Overview](#Overview)
- [Installation](#Installation)
- [Running the script](#Running-the-script)
- [*seafbackup.sh*](#seafbackupsh)
- [Final thoughts](#Final-thoughts)
## Basic features of this script:
- Backup your entire Seafile program and data directories.
- Dump Seafile SQL databases to a temporary directory, add them to the backup and then remove the temporary directory.
- Easily add additional directories/files to the backup by modifying a plain-text file.
- Easily exclude directories/files from being backed up (i.e. thumbnails) by modifying a plain-text file.
- Optionally stop Seafile services during the backup and restart them after.
- Optionally copy a 503 error page to your webserver so users know your server is unavailable due to a backup being performed. The 503 file is removed when the backup is completed.
- Run 'borg prune' to trim old backups according to your schedule so you don't pay storage fees for super-old or un-needed backups.
- Create a clear, easy to parse log file so you can keep an eye on warnings/errors, etc.
- The generated log file can be easily parsed by tools like *logwatch* so you can integrate backup-monitoring with your existing system-monitoring solutions.
## Check out the wiki
This readme is an overview of the script. For detailed instructions on how the script works, what it does and it's various options, please [read the wiki](https://git.asifbacchus.app/asif/seafileBackup/wiki)! I've tried to annotate the files as best I could with many many comments, but seriously, reading the [wiki](https://git.asifbacchus.app/asif/seafileBackup/wiki) if you're stuck is the best approach since I try to go over everything step-by-step.
## Overview
This script has NO mandatory parameters. If no parameters are specified, it will proceed using defaults which normally align with the recommended Seafile installation defaults. That being said, it is impossible to determine defaults for things like your backup-server settings, so please make sure you customize the *seafbackup.details* file with appropriate details!
By default, the script will dump Seafile SQL databases and back them up along with your entire Seafile program and data directories. You can add additional files to be backed up by adding paths to *xtraLocations.borg*.
## Installation
Simply clone this repo to the desired location. I recommend somewhere simple like *ROOT's home directory* so any sensitive information is restricted to the root user by default.
```bash
cd /root
sudo git clone https://git.asifbacchus.app/asif/seafileBackup.git
```
## Running the script
```bash
./seafbackup.sh [parameters]
# display abbreviated help
./seafbackup.sh --help
```
## *seafbackup.sh*
Please open this file in a text editor and follow the comments to fill in the parameters with the correct values for your environment and your backup server.
## Final thoughts
I hope this makes backing up your Seafile server easier and removes one more mundane administrative task from your schedule :-) If you have any comments, questions, concerns, suggestions for improvements or notice any bugs, please file an issue in this repo!
For more scripts and solutions like this, check out my blog at [My Techie-Thoughts](https://mytechiethoughts.com). Thanks!
+15
View File
@@ -0,0 +1,15 @@
### Rotate backup log file
# location of log file (-l parameter of script file)
/path/to/backup.log {
# rotate log file weekly -- you could also use 'daily', 'monthly' or
# specify a size using 'size 100k', for example
weekly
# keep 4 weeks of old logs (4 files) and delete older ones
rotate 4
# compress old log files using gzip (default)
compress
}
@@ -0,0 +1,13 @@
# Location of your script's log file, -l parameter
LogFile = /path/to/your/seafbackup.log
# Format of logrotate archives for your script. Example assumes compression and
# extension preservation
Archive = /path/to/your/seafbackup.log.?.gz
# Apply the correct date/time filtering to match the format of the script's log
# We are using a custom pl script in /etc/logwatch/scripts/shared/ You don't
# need to change this unless you have altered the 'stamp' function in the backup
# script in which case you will want to update the regex in the custom pl script
# below
*sqFullStampAnywhere
@@ -0,0 +1,9 @@
# Name of the logfile group without any extension
LogFile = seafbackup
# Heading displayed on Logwatch's report for this service
Title = "System and Seafile Backup"
# Override the detail level for this service
# Remember the levels are: 0, 1-4, 5, 6+
# Detail = 0
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/perl
#############################################################################
# $Id$
#############################################################################
# Log: Seafile backup script (seafbackup, Logwatch group 'backup')
# Revision 1.1 2019/07/20
# Written by Asif Bacchus
#############################################################################
use strict;
### Get Logwatch detail level (default to 0)
my $detailLevel = $ENV{'LOGWATCH_DETAIL_LEVEL'} || 0;
### Declare variables
my $summaryErr;
my $summaryWarn;
my $summarySuccess;
my %reportHash = ();
my $key;
### Minimal detail level: provide summary data only
if ($detailLevel == 0) {
### process logfile and summarize message types
while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /\-- \[ERROR\] /) {
$summaryErr++;
}
elsif ($ThisLine =~ /\-- \[WARNING\] /) {
$summaryWarn++;
}
elsif ($ThisLine =~ /All processes completed/) {
$summarySuccess++;
}
}
### fill hash table with headings and summary counts
if ($summarySuccess > 0) {
$reportHash{"All processes successfully completed"} = $summarySuccess;
}
if ($summaryWarn > 0) {
$reportHash{"Warnings issued"} = $summaryWarn;
}
if ($summaryErr > 0) {
$reportHash{"Errors encountered"} = $summaryErr;
}
### print hash table
foreach $key (sort keys %reportHash) {
print "$key: $reportHash{$key}\n";
}
}
### Levels 1-4 provide the actual error, warning and success messages instead
### of a summary count
elsif ($detailLevel >= 1 && $detailLevel <= 4) {
while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /\-- \[ERROR\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[WARNING\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[SUCCESS\] /) {
print $ThisLine;
}
}
}
### Level 5 is similiar to levels 1-4 except it also reports informational
### messages such as the location of script created files, variable checks,
### etc. This is useful when verifying the script's operation.
elsif ($detailLevel == 5) {
while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /\-- \[ERROR\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[WARNING\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[SUCCESS\] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /\-- \[INFO\] /) {
print $ThisLine;
}
}
}
### Any level above 5 will echo the entire log including the debugging notes
### within the script meant for troubleshooting. Using this level of detail
### should only be done if you cannot view the actual log file directly for
### whatever reason. The actual log file is colour-coded for easier debugging.
elsif ($detailLevel > 5) {
while (defined(my $ThisLine = <STDIN>)) {
print $ThisLine;
}
}
### Exit gracefully
exit (0);
# vi: shiftwidth=3 tabstop=3 et
# Local Variables:
# mode: perl
# perl-indent-level: 3
# indent-tabs-mode: nil
# End:
@@ -0,0 +1,37 @@
#!/usr/bin/perl
##########################################################################
# $Id$
##########################################################################
###############################################################################
## Filter dates in full-date-time international format, surrounded by square
## brackets located anywhere on a given line
## Format: '[%Y-%m-%d %H:%M:%S]'
###############################################################################
use Logwatch ':dates';
my $Debug = $ENV{'LOGWATCH_DEBUG'} || 0;
### Specify the format of the date/time stamp itself
$SearchDate = TimeFilter('%Y-%m-%d %H:%M:%S');
if ( $Debug > 5 ) {
print STDERR "DEBUG: Inside FullDateTime...\n";
print STDERR "DEBUG: Looking For: " . $SearchDate . "\n";
}
while (defined($ThisLine = <STDIN>)) {
### specify the regex that defines how to find 'SearchDate'
if ($ThisLine =~ m/\[$SearchDate\] /o) {
print $ThisLine;
}
}
# vi: shiftwidth=3 syntax=perl tabstop=3 et
# Local Variables:
# mode: perl
# perl-indent-level: 3
# indent-tabs-mode: nil
# End:
@@ -62,13 +62,13 @@ borgRepoPassphrase="p@ssW0rd"
# directories to include in your backup along with Seafile data # directories to include in your backup along with Seafile data
# see repo wiki for more details # see repo wiki for more details
# leave blank if you only want to backup Seafile related files/directories. # leave blank if you only want to backup Seafile related files/directories.
borgXtraListPath="/root/seafBackup/xtraLocations.borg" borgXtraListPath="/root/seafileBackup/xtraLocations.borg"
# OPTIONAL: path to file containing files/directories or 'patterns' to be # OPTIONAL: path to file containing files/directories or 'patterns' to be
# excluded in a BORG RECOGNIZED format # excluded in a BORG RECOGNIZED format
# see repo wiki for more details or consult borg documentation # see repo wiki for more details or consult borg documentation
# leave blank for no exclusions. # leave blank for no exclusions.
borgExcludeListPath="/root/seafBackup/excludeLocations.borg" borgExcludeListPath="/root/seafileBackup/excludeLocations.borg"
# parameters to determine how borg deletes aged backups # parameters to determine how borg deletes aged backups
# more details in the repo wiki and/or borg documentation # more details in the repo wiki and/or borg documentation
@@ -165,9 +165,8 @@ scriptHelp () {
printf "More details and examples of script usage can be found in the " printf "More details and examples of script usage can be found in the "
printf "repo wiki at\n" printf "repo wiki at\n"
printf "${yellow}https://git.asifbacchus.app/asif/seafileBackup.git" printf "${yellow}https://git.asifbacchus.app/BacchusGroup/seafileBackup"
printf "/wiki${norm}.\n" printf "/wiki${norm}.\n"
printf "${cyan}%s${norm}\n\n" "--------------------------------------------------------------------------------" printf "${cyan}%s${norm}\n\n" "--------------------------------------------------------------------------------"
} }
@@ -230,7 +229,7 @@ scriptPath="$( CDPATH='' cd -- "$( dirname -- "$0" )" && pwd -P )"
scriptName="$( basename "$0" )" scriptName="$( basename "$0" )"
logFile="$scriptPath/${scriptName%.*}.log" logFile="$scriptPath/${scriptName%.*}.log"
warnCount=0 warnCount=0
configDetails="$scriptPath/seafbackup.details" configDetails="$scriptPath/${scriptName%.*}.details"
err503Copied=0 err503Copied=0
sqlCopied=0 sqlCopied=0
includeXtra=0 includeXtra=0
@@ -388,6 +387,11 @@ done
### check pre-requisites and default values ### check pre-requisites and default values
# check if running as root, otherwise exit
if [ $( id -u ) -ne 0 ]; then
printf "\n${err}ERROR: script MUST be run as ROOT${norm}\n\n"
exit 2
fi
# does the details file exist? # does the details file exist?
if [ ! -f "$configDetails" ]; then if [ ! -f "$configDetails" ]; then
badParam dne "(--details default)" "$configDetails" badParam dne "(--details default)" "$configDetails"
@@ -532,7 +536,7 @@ if ! mysqldump -h"${sqlServer}" -u"${sqlUser}" -p"${sqlPass}" \
--opt "${seahubDB_name}" > "${sqlDumpDir}/${sqlDump_seahub}" 2>> "$logFile"; then --opt "${seahubDB_name}" > "${sqlDumpDir}/${sqlDump_seahub}" 2>> "$logFile"; then
exitError 117 "Could not dump ${seahubDB_name} database" exitError 117 "Could not dump ${seahubDB_name} database"
fi fi
printf "${ok}[%s] -- [SUCCESS] SQL databases dumped successfully --${norm}\n" \ printf "${ok}[%s] -- [SUCCESS] Dumped SQL databases --${norm}\n" \
"$(stamp)" >> "$logFile" "$(stamp)" >> "$logFile"
@@ -33,7 +33,7 @@
# root user files # root user files
/root/.bashrc /root/.bashrc
/root/.ssh/ /root/.ssh/
/root/SeafBackup/ /root/seafileBackup/
### configuration files for programs related to Seafile ### configuration files for programs related to Seafile