update readme to reference wiki

update logwatch readme to point to wiki
include mailcow.conf directory in backup
2020-05-24 04:48:21 -06:00 · 2020-05-24 03:55:20 -06:00 · 2020-05-07 04:59:37 -06:00 · 2020-05-07 04:58:26 -06:00 · 2020-05-07 01:43:53 -06:00 · 2020-05-07 01:35:20 -06:00
10 changed files with 996 additions and 1731 deletions
@@ -1,569 +1,120 @@
 # Mailcow Backup Using borgbackup <!-- omit in toc -->

-This script automates backing up your Mailcow installation using borgbackup
-and a remote ssh-capable storage system.  I suggest using rsync.net since they
-have great speeds and a special pricing structure for borgbackup/attic users
-([details here](https://www.rsync.net/products/attic.html)).
+This script automates backing up your Mailcow installation using borgbackup and a remote ssh-capable storage system.  I suggest using rsync.net since they
+have great speeds and a special pricing structure for borgbackup/attic users ([details here](https://www.rsync.net/products/attic.html)).

 This script automates the following tasks:

- Optionally copies a 503 error page to your webserver so users know when your
-  server is unavailable due to backups being performed. The 503 file is removed
+- Optionally copies a 503 error page to your webserver so users know when your server is unavailable due to backups being performed. The 503 file is removed
  when the backup is completed so users can login again
 - Dumps the Mailcow mySQL database and adds it to the backup
- Handles stopping and re-starting mail-flow containers (postfix and dovecot) so
-  everything is in a consistent state during the backup
+- Handles stopping and re-starting mail-flow containers (postfix and dovecot) so everything is in a consistent state during the backup
 - Allows you to specify additional files you want backed up
 - Allows you to specify files/directories to exclude from your backups
 - Runs 'borg prune' to make sure you are trimming old backups on your schedule
- Creates a clear, easy to parse log file so you can keep an eye on your backups
-  and any errors/warnings
+- Creates a clear, easy to parse log file so you can keep an eye on your backups and any errors/warnings

 ## Contents <!-- omit in toc -->

- [Installation/copying](#installationcopying)
- [Environment notes](#environment-notes)
- [Why this script must be run as root](#why-this-script-must-be-run-as-root)
- [Script parameters](#script-parameters)
-  - [Optional parameters](#optional-parameters)
-    - [Docker container STOP timeout before error: -1 _number_](#docker-container-stop-timeout-before-error--1-number)
-    - [Docker container START timeout before error: -2 _number_](#docker-container-start-timeout-before-error--2-number)
-    - [Path to 503 error page: -5 _/path/to/filename.html_](#path-to-503-error-page--5-pathtofilenamehtml)
-    - [Path to borg details file: -b _/path/to/filename.file_](#path-to-borg-details-file--b-pathtofilenamefile)
-    - [File name of docker-compose configuration file: -d _filename.file_](#file-name-of-docker-compose-configuration-file--d-filenamefile)
-    - [Log file location: -l _/path/to/filename.file_](#log-file-location--l-pathtofilenamefile)
-    - [File name of Mailcow master configuration file: -m _filename.file_](#file-name-of-mailcow-master-configuration-file--m-filenamefile)
-    - [Verbose output from borg: -v (no arguments)](#verbose-output-from-borg--v-no-arguments)
-    - [Path to webroot: -w _/path/to/webroot/_](#path-to-webroot--w-pathtowebroot)
- [Borg details file](#borg-details-file)
-  - [Protect your borg details file](#protect-your-borg-details-file)
-  - [borg specific entries (lines 1-4)](#borg-specific-entries-lines-1-4)
-    - [Line 1: Path to borg base directory](#line-1-path-to-borg-base-directory)
-    - [Line 2: Path to SSH key for remote server](#line-2-path-to-ssh-key-for-remote-server)
-    - [Line 3: Connection string to remote repo](#line-3-connection-string-to-remote-repo)
-    - [Line 4: Password for borg repo/repo key](#line-4-password-for-borg-reporepo-key)
-  - [additional files/directories to backup](#additional-filesdirectories-to-backup)
-  - [exclusion patterns](#exclusion-patterns)
-  - [prune timeframe options](#prune-timeframe-options)
-  - [borg remote location](#borg-remote-location)
-  - [Examples](#examples)
- [503 functionality](#503-functionality)
-  - [Conditional forwarding by your webserver](#conditional-forwarding-by-your-webserver)
-    - [NGINX](#nginx)
-    - [Apache](#apache)
-    - [Disabling 503 functionality altogether](#disabling-503-functionality-altogether)
- [Scheduling: Cron](#scheduling-cron)
- [The log file](#the-log-file)
-  - [Using Logwatch](#using-logwatch)
-  - [Remember to rotate your logs](#remember-to-rotate-your-logs)
+- [quick start](#quick-start)
+- [configuration file](#configuration-file)
+- [running the script](#running-the-script)
+- [scheduling your backup via cron](#scheduling-your-backup-via-cron)
 - [Final notes](#final-notes)

-## Installation/copying
+## quick start

-Once you've either cloned this git or downloaded the release file, simply copy
-the files within the archive to whatever location(s) that work for your setup.
-I've stored the files in this git archive in a directory structure that should
-match most default setups.  I suggest keeping the contents of the
-*'/root/scripts'* folder in that location since the root user must execute the
-script anyways.  If you edit the 503.html and mc_borg.details files in place,
-then you don't have to specify their locations when running the script.
+Clone this repo or download a release file into a directory of your choosing. For all examples in this document, I will assume you will run the script from */scripts/backup*. Make sure the script file is executable and you protect the *.details* file since it contains things like your repo password:

-Remember to make the script executable!
+```bash
+# run commands as root
+sudo -s

-```Bash
+# find somewhere to clone the repo
+cd /usr/local/src
+
+# clone the repo from my server (best choice)
+git clone https://git.asifbacchus.app/asif/MailcowBackup.git
+# or clone from github
+git clone https://github.com/asifbacchus/MailcowBackup.git
+
+# make a home for your backup script
+mkdir -p /scripts/backup
+cd /scripts/backup
+
+# copy files from cloned repo to this new home
+cp /usr/local/src/MailcowBackup/backup/* ./
+
+# make script executable and protect your .details file
 chmod +x backup.sh
+chmod 600 backup.details
 ```

-In addition, you can rename this script file to anything you like.  The log file
-will use that same name by default when naming itself and any mention of this
-file in the logs will automatically use whatever name you choose to give it.
+## configuration file

-## Environment notes
+You will need to let the script know how to access your remote repo along with any passwords/keyfiles needed to encrypt data. This is all handled via the plain-text 'configuration details' file. By default, this file is named *backup.details*. The file itself is fully commented so setting it up should not be difficult. If you need more information, consult [page 4.0](https://git.asifbacchus.app/asif/MailcowBackup/wiki/4.0-Configuration-details-file) in this wiki.

-The script is designed to be easy to use but still be flexible enough to
-accommodate a wide range of Mailcow setups.  The script pulls nearly all it's
-configuration from the Mailcow configuration files themselves, so it adapts to
-nearly all customizations you may have in your environment.  The script accepts
-several optional parameters to override its default or detected settings.  In
-addition, it reads easy-to-edit external plain-text files for borg settings so
-you don't have to weed through the script code to supply things like passwords.
+## running the script

-**This script auto-detects the location of your Mailcow configuration file. If
-you have multiple files on your system with the same name as the configuration
-file, the script will likely get confused and exit with an error**
+After setting up the *.details* file correrctly and assuming you are running a default set up of mailcow according to the documentation, you just have to run the script and it will find everything on it's own. In particular, the defaults are set as follows:

-## Why this script must be run as root
+- mailcow.conf is located at */opt/mailcow-dockerized/mailcow.conf*
+- docker-compose file is located at */opt/mailcow-dockerized/docker-compose.yml*
+- the log file will be saved in the same directory as the script with the same name as the script but with the extension *.log*

-This script must be run by the root user and will exit with an error if you try
-running it otherwise.  This is because a default secured setup of borgbackup
-contains things like the repository private key that are locked out to root user
-access only.  In addition, the root user is guaranteed to have access to all
-files you might want to backup.
+To get a list of all configuration options with defaults:

-## Script parameters
-
-You can run the script with the *'-?'* parameter to access the built-in help
-which explains the parameters.  However, the following is a more detailed
-explanation of each parameter and how to use them. **Note that any parameters
-needing a directory (webroot, log file location, etc.) can be entered with or
-without the trailing '/' since it's stripped by the script anyways.**
-
-General usage:
-
-```Bash
-/path/to/script/scriptname.sh -parameter argument -parameter argument ...
+```bash
+./backup.sh --help
 ```

-### Optional parameters
+To run with defaults:

-#### Docker container STOP timeout before error: -1 _number_
-
-The amount of time, in seconds, to wait for a docker container to STOP
-gracefully before aborting, logging the error and exiting the script.\
-**Default: _120_**
-
-#### Docker container START timeout before error: -2 _number_
-
-The amount of time, in seconds, to wait for a docker container to START
-before aborting, logging the error and exiting the script.\
-**Default: _180_**
-
-#### Path to 503 error page: -5 _/path/to/filename.html_
-
-The path to an html file for the script to copy to your webroot during the
-backup process.  This file can be scanned by your webserver and a 503 error can
-be issued to users letting them know that your Mailcow is 'temporarily
-unavailable' while being backed up.  A sample 503 page is included for you.
-
-If you remove the default file or the one you specify is missing, a warning will
-be issued by the script but, it will continue executing.  More details on the
-503 notification can be found later in the [503
-functionality](#503-functionality) section of this document.\
-**Default: _scriptpath/503.html_**
-
-#### Path to borg details file: -b _/path/to/filename.file_
-
-This is a text file that lays out various borg options such as repo name,
-password, additional files to include, exclusion patters, etc.  A sample file is
-included for your reference.  More details, including the *required order* of
-entries can be found later in this document in the [borg details
-file](#borg-details-file) section.\
-**Default: _scriptpath/mc_borg.details_**
-
-#### File name of docker-compose configuration file: -d _filename.file_
-
-This is the file name of your docker-compose configuration file that is used to
-build/start/stop containers.  This script will only search for this file within
-the same directory where your Mailcow configuration file is found.\
-**Default: _docker-compose.yml_**
-
-#### Log file location: -l _/path/to/filename.file_
-
-If you have a particular place and filename you'd like this script to use for
-it's log, then you can specify it using this parameter.  I would recommend
-*'/var/log/backup.log'*. By default, the script will name the log file
-*scriptname*.log and will save it in the same directory as the script itself.\
-**Default: _scriptpath/scriptname.log_**
-
-#### File name of Mailcow master configuration file: -m _filename.file_
-
-This is the file name of the Mailcow master configuration file that was
-generated after installation and contains all information needed to run Mailcow
-(database user name, volume directory prefixes, etc.)  This script will search
-your computer for either the default file name or the one you have provided.
-Upon finding it, the script will derive the file path and use that as the path
-in which to run all Mailcow/docker commands.  **Please do not have multiple
-files on your system with this name, the script WILL get confused and exit with
-an error.\
-**Default: _mailcow.conf_**
-
-#### Verbose output from borg: -v (no arguments)
-
-By default, the script will ask borg to generate summary only output and record
-that in the script's log file.  If you are running the backup for the first time
-or are troubleshooting, you may want a detailed output of all files and their
-changed/unchanged/excluded status from borg.  In that case, specify the -v
-switch. **Note: This will make your log file very large very quickly since EVERY
-file being backed up is written to the log.**
-
-#### Path to webroot: -w _/path/to/webroot/_
-
-This is the path to the directory your webserver is using as it's default root.
-In other words, this is the directory that contains the html files served when
-someone browses to your server.  The correct webroot depends greatly on your
-particular setup.
-
-If you directly connect to Mailcow via Docker, then your webroot is by default
-*/opt/mailcow-dockerized/data/web*, unless you've made changes to your install
-locations.  If you are running behind a reverse-proxy, then your webroot is your
-webserver's webroot (*/var/www* or */usr/share/nginx/html*, for example).
-
-This is used exclusively for 503 functionality since the script has to know
-where to copy the 503 file.  If you don't want to use this functionality, you
-can omit this parameter and the script will issue a warning and move on.  More
-details can be found in the [503 functionality](#503-functionality) section
-later in this document.
-
-## Borg details file
-
-This file contains all the data needed to access your borg remote data repo.
-Each line must contain specific information in a specific order or **needs to be
-blank if that data is not required**.  The sample file includes this data and
-example entries.  The file must have the following information in the following
-order:
-
-    1. path to borg base directory **(required)**
-    2. path to ssh private key for remote server **(required)**
-    3. connection string to remote repo **(required)**
-    4. password for borg repo/repo key **(required)**
-    5. path to file listing additional files/directories to backup
-    6. path to file containing borg-specific exclusion patterns
-    7. prune timeframe options
-    8. location of borg remote instance
-
-### Protect your borg details file
-
-This file contains information on how to access and decrypt your borg repo,
-therefore, you **must** protect it.  You should lock it out for everyone but
-your root user. Putting it in your root folder is not enough!  Run the following
-commands to restrict access to the root user only (assuming filename is
-*mc_borg.details*):
-
-```Bash
-chown root:root mc_borg.details   # make root the owner of this file
-chmod 600 mc_borg.details   # grant access to root user only (read/write)
+```bash
+./backup.sh
 ```

-### borg specific entries (lines 1-4)
+To run with a custom log file name and location:

-If you need help with these options, then you should consult the borg
-documentation or search my blog at
-[https://mytechiethoughts.com](https://mytechiethoughts.com) for borg.  Here's a
-very brief overview:
-
-#### Line 1: Path to borg base directory
-
-This is primary directory on your local system where your borg configuration is
-located, **NOT* the path to your borg binary.  The base directory contains the
-borg configuration, cache, security files and keys.
-
-#### Line 2: Path to SSH key for remote server
-
-This is the SSH key used to connect to your remote (backup) server where your
-borg repo is located.  **This is NOT your borg repo key!**
-
-> Please note:  If you are planning on executing this script via cron or some
-> other form of automation, it is *highly recommended* that you use an SSH key
-> **without** a password!  SSH is designed such that passwords cannot simply be
-> passed to it via environment variables, etc. so this is something not easily
-> automated by a script such as this for security reasons.  As such, your
-> computer will sit and wait for you to enter the password and will NOT execute
-> the actual backup portion of the script until the SSH key password is provided.
-> 
-> If you really want/need to use an SSH key password, you will have to look into
-> somethign like GNOME keyring or SSH-agent to provide a secure automated way to
-> provide that password to SSH and allow this script to continue.
->
-> In practice, SSH keys without passwords are still quite safe since the key
-> must still be known in order to connect and most keys are quite long.  In
-> addition, they key only connects to the remote server, your actual information
-> within the borg repository is still encrypted and secured with both a key and
-> password.
-
-#### Line 3: Connection string to remote repo
-
-This is the full server and path required to connect to your borg repo on the
-remote server.  Very often it is the in the form of:
-
-```
-user@servername.tld:repo-name/
+```bash
+./backup.sh --log /var/log/mailcow_backup.log
 ```

-for rsync.net it is in the following form:
+To copy a 503 error page to your webroot:

-```
-username@server-number.rsync.net:repo-name/
+```bash
+# assuming default NGINX webroot (/usr/share/nginx/html)
+./backup.sh -5
+# custom webroot
+./backup.sh -5 -w /var/www/
 ```

-#### Line 4: Password for borg repo/repo key
+Common usage: custom log file and copy 503 to custom webroot

-This is the password needed to access and decrypt your *borg repo*.  Assuming
-you set up your borg repo using recommended practices, this will actually be the
-password for your *borg repo private key*.  **This is NOT your SSH key
-password!**
-
-### additional files/directories to backup
-
-This points to a plain-text file listing additional files and directories you'd
-like borg to include in the backup.  The sample file, *'xtraLocations.borg'*
-contains the most likely files you'd want to include assuming you're using a
-standard setup like I outline in my blog.
-
-The following would include all files in the home folder for users *'foo'* and
-*'bar'* and any conf files in *'/etc/someProgram'*:
-
-```Bash
-/home/foo/
-/home/bar/
-/etc/someProgram/*.conf
+```bash
+./backup.sh -l /var/log/mailcow_backup.log -5 -w /var/www/
 ```

-*You can leave this line blank* to tell borg to only backup your Mailcow data,
-configuration and the SQL dump.  However, this is pretty unusual since you would
-not be including any server configuration files, reverse-proxy configurations,
-etc. If you omit this line, the script will log a warning to remind you of this
-unusual situation.
+Non-default mailcow location (example: */var/mailcow*):

-### exclusion patterns
-
-This points to a plain-text file containing borg-specific patterns describing
-what files you'd like borg to ignore during the backup.  To specify exclusions,
-create a text file in any location you want and specify exclusions patterns, one
-per line.  Then update line 6 in your borg details file with the path to your
-new exclusion file.
-
-You need to run *'borg help patterns'* for help on how to specify exclusion
-patterns since the format is not always standard BASH format and only sometimes
-uses standard regex.
-
-If you leave this line blank, the script will note it is not processing any
-exclusions and will proceed with backing up all files specified.
-
-### prune timeframe options
-
-Here you can let borg prune know how you want to manage your backup history.
-Consult the borg documentation and then copy the relevant options directly into
-this line including any spaces, etc.  The example file contains the following as
-a staring point:
-
-```Ini
--keep-within=7d --keep-daily=30 --keep-weekly=12 --keep-monthly=-1
+```bash
+./backup.sh --docker-compose /var/mailcow/docker-compose.yml --mailcow-config /var/mailcow/mailcow.conf
 ```

-This would tell borg prune to keep ALL backups made for any reason within the
-last 7 days, keep 30 days worth of daily backups, 12 weeks of end-of-week
-backups and then an infinite amount of end-of-month backups.
+For more configuration options, see [page 3.0](https://git.asifbacchus.app/asif/MailcowBackup/wiki/3.0-Script-parameters) in the wiki and [page 4.4](https://git.asifbacchus.app/asif/MailcowBackup/wiki/4.4-Configuration-examples) for some configuration examples. Consult [section 7]() of the wiki for information about the log file and how to integrate it with logwatch.

-### borg remote location
+## scheduling your backup via cron

-If you're using rsync.net, then just have this say *'borg1'*.  If you are using
-another provider, you'll have to reference their locally installed copy of borg
-relative to your home directory.  You can also leave this blank if your provider
-does not run borg locally but your backups/restores will be slower.
+Edit your root user's crontab and add an entry like this which would run the script using defaults at 1:07am daily:

-### Examples
-
-Repo in directory *'MailcowBackup'*, all fields including pointers to additional
-files to backup, exclusion patterns and a remote borg path.  Prune: keep all
-backups made in the last 14 days.
-
-```Ini
-/var/borgbackup
-/var/borgbackup/SSHprivate.key
-myuser@usw-s001.rsync.net:MailcowBackup/
-myPaSsWoRd
-/root/scripts/xtraLocations.borg
-/root/scripts/excludeLocations.borg
--keep-within=14d
-borg1
+```ini
+7 1 * * * /scripts/backup/backup.sh -l /var/log/mailcow_backup.log > /dev/null 2>&1
 ```

-Repo in directory *'myBackup'*, no exclusions, keep 14 days end-of-day, 52 weeks
-end-of-week
-
-```Ini
-/var/borgbackup
-/root/keys/rsyncPrivate.key
-myuser@usw-s001.rsync.net:myBackup/
-PaSsWoRd
-/var/borgbackup/include.list
-
--keep-daily=14 --keep-weekly=52
-borg1
-```
-
-Repo in directory *'backup'*, no extra file locations, no exclusions, no remote
-borg installation. Keep last 30 backups.
-
-```Ini
-/root/.borg
-/root/.borg/private.key
-username@server.tld:backup/
-pAsSw0rD
-
-
--keep-within=30d
-
-```
-
-**Notice that the blank lines are very important!**
-
-## 503 functionality
-
-This script includes an entire section dedicated to copying an html file to act
-as an error 503 notification page.  Error 503 is by definition "service
-temporarily unavailable" which is exactly the case for your Mailcow server
-during a backup since the mail-flow containers have been disabled.
-
-The script copies whatever file is defined by the *'-5'* parameter (or the
-default located at *'scriptpath/503.html'*) to whatever path is defined as the
-'webroot' by the *'-w'* parameter.  This means that if you omit the *'-w'*
-parameter, the script will necessarily skip this entire process and just issue a
-warning to let you know about it.
-
-### Conditional forwarding by your webserver
-
-The script copying the file to the webroot is the easy part.  Your webserver has
-to look for the presence of that file and generate a 503 error in order for the
-magic to happen.  To do that, you have to include an instruction to that effect
-in your default server definition and/or your Mailcow virtual server definition
-file depending on your setup.
-
-#### NGINX
-
-You can copy the following code into the relevant server definition(s) on an
-NGINX server:
-
-```Perl
-server {
-    ...
-    if (-f /usr/share/nginx/html/503.html) {
-        return 503;
-    }
-...
-    error_page 503 @backup
-    location @backup {
-        root /usr/share/nginx/html;
-        rewrite ^(.*)$ /503.html break;
-    }
-}
-```
-
-This tells NGINX that if it finds the file *'503.html'* at the path
-*'/usr/share/nginx/html'* (webroot on reverse proxy) then return an error code
-503.  Upon encountering a 503 error, rewrite any url to *'domain.tld/503.html'*
-and thus, display the custom 503 error page.  On the other hand, if it can't
-find 503.html at the path specified (i.e. the script has deleted it because the
-backup is completed), then go about business as usual.
-
-#### Apache
-
-I don't use apache for anything, ever... so I'm not sure how exactly you'd do
-this but I think you'd have to use something like:
-
-```Perl
-RewriteEngine On
-RewriteCond %{ENV:REDIRECT_STATUS} !=503
-RewriteCond "/var/www/503.html" -f
-RewriteRule ^ - [R=503,L]
-...
-ErrorDocument 503 /503.html
-...
-```
-
-Let me know if that works and I'll update this document accordingly.  Like I
-said, I don't use Apache so I can't really test it very easily.
-
-#### Disabling 503 functionality altogether
-
-If you don't want to use the 503 functionality for whatever reason and don't
-want your log file junked up with warnings about it, then find the section of
-the script file that starts with *'--- Begin 503 section ---'* and either
-comment all the lines (put a *'#'* at the beginning of each line) or delete all
-the lines until you get to *'--- End 503 section ---'*.
-
-## Scheduling: Cron
-
-After running this script at least once manually to test your settings, you
-should schedule it to run automatically so things stay backed up.  This is
-easiest with a simple cron job.
-
-1. Open root's crontab:
-
-    ```Bash
-    sudo crontab -e
-    ```
-
-2. Add your script command line and set the time. I'm assuming your script is
-   located at *'/root/scripts'*, all files are at their default locations and
-   you want to run your backup at 1:07am daily.
-
-    ```Bash
-    7 1 * * * /root/scripts/backup.sh -l /var/log/backup.log -w /usr/share/nginx/html > /dev/null 2>&1
-    ```
-
-    The last part redirects all output to 'null' and forwards any errors to
-    'null' also.  You don't need output because the script creates a wonderfully
-    detailed log file that you can review :-)
-3. Save the file and exit.
-4. Confirm by listing the root user's crontab:
-
-    ```Bash
-    sudo crontab -l
-    ```
-
-## The log file
-
-The script creates a very detailed log file of all major operations along with
-any errors and warnings.  Everything is timestamped so you can see how long
-things take and when any errors took place.  The script includes debugging
-notes such as where temp files are located, where it's looking for data, whether
-it created/moved/copied files, etc.  All major operations are tagged *'-- [INFO]
-message here --'*.  Similarly, warnings are tagged *'-- [WARNING] message here
-(code: xxxx) --'* and errors are tagged *'-- [ERROR] message here (code: xxx)
--'*.  Successful operations generate a *'-- [SUCCESS] message here --'* stamp.
-
-Sections of the script are all colour-coded to make viewing it easier.  This
-means you should use something like *'cat backup.log | more'* or *'tail -n
-numberOfLines backup.log'* to view the file since the ansi colour codes
-would make it difficult to read in nano or vi.
-
-This tagging makes it easy for you to set up a log screening program to make
-keeping an eye on your backup results easier.  If you plan on using Logwatch
-(highly recommended, great program!) then I've done the work for you...
-
-### Using Logwatch
-
-Log-group, conf and service files are included so that you can easily setup
-Logwatch to monitor the script's log file and report at your desired detail
-level as follows:
-
-    1. 0: Summary of total successes, warnings & errors only
-    2. 1-4: Actual success, error & warning messages
-    3. 5: Same as above, but includes info messages
-    4. 6+: Dumps entire raw log file including debugging messages
-
-A detailed breakdown of the files and all options are included in a separate
-readme in the *'/etc/logwatch'* folder of this git archive.
-
-If you don't really care how it works, you can just copy the files from this
-archive to your *'/etc/logwatch'* directory.  The directory structure is correct
-for a default Logwatch install on Debian/Ubuntu.  You will have to update the
-log-group file to reflect the path to your script's log file.
-
-### Remember to rotate your logs
-
-The log file generated by this script is fairly detailed so it can grow quite
-large over time.  This is especially true if you are using verbose output from
-borg for any troubleshooting or for compliance/auditing.  I've included a sample
-commented *logrotate config file* in this git archive at *'/etc/logrotate.d'*
-which you can modify and drop into that same directory on your Debian/Ubuntu
-system.  If you are using another log rotating solution, then please remember to
-configure it so that your log files don't get overwhelmingly large should you
-need to parse them if something goes wrong with your backups.
-
 ## Final notes

-I think that's everything. If I've forgotten to document something, please let
-me know. I know this readme is long but, I hate how much stuff for linux and
-open-source programs/scripts in general are so poorly documented especially for
-newbies and I didn't want to make that same mistake.
+I think that's everything. For detailed information, please review the [wiki](https://git.asifbacchus.app/asif/MailcowBackup/wiki/_pages). If I've forgotten to document something there, please let me know. I know the wiki is long but, I hate how much stuff for linux and open-source programs/scripts in general are so poorly documented especially for newbies and I didn't want to make that same mistake.

-I don't script too often and I'm a horrible programmer, so if you see anything
-that can be/should be improved, please let me know or submit your changes!  I
-love learning new ways of doing things and getting feedback, so suggestions and
-comments are more than welcome.
+I don't script too often and I'm a horrible programmer, so if you see anything that can be/should be improved, please let me know by filing an issue or submit your changes via a pull request!  I love learning new ways of doing things and getting feedback, so suggestions and comments are more than welcome.

-If this has helped you out, then please visit my blog at
-[https://mytechiethoughts.com](https://mytechiethoughts.com) where I solve
-problems like this all the time on a shoe-string or zero budget.  Thanks!
+If this has helped you out, then please visit my blog at [https://mytechiethoughts.com](https://mytechiethoughts.com) where I solve problems like this all the time on a shoe-string or zero budget. Thanks!
@@ -18,11 +18,8 @@

 <body>
    <h1>Bad timing!</h1>
-    <p>Seems you're trying to access me during my daily backup window.  Don't
-    worry though, I should be up and running again very soon.</p>
-    <p>My average backup window duration is around 10 minutes during which time
-    I'm quite busy copying your super-important emails to my ultra-secret hiding
-    place so they stay safe in case anything ever happens to me!</p>
+    <p>Seems you're trying to access me during my daily backup window.  Don't worry though, I should be up and running again very soon.</p>
+    <p>My average backup window duration is pretty short and I'm quite busy during that time copying your super-important stuff to my secure hiding place so they stay safe in case anything ever happens to me!</p>
    <h3><em>I'm really sorry for the delay.  Please try me again soon!</em></h3>
 </body>

@@ -0,0 +1,61 @@
+#######
+### backup script configuration details
+###
+### This file contains sensitive information, make sure you have protected
+### it by restricting permissions!
+### Run the following in the directory where this file is located:
+###     chown root:root ./backup.details
+###     chmod 600 ./backup.details
+###
+### Do NOT include any commands in this file as they WILL be executed!!!
+#######
+
+
+### borg details
+# if you're unsure what to enter here, please consult the repo wiki and/or
+# the borg documentation
+
+# base configuration directory for borg, all borg parameters use this directory
+# as their 'root'.  I recommend setups with this being "/var/borgbackup", the
+# default is "$HOME" or "~$USER" in that order.  If you're unsure, try "$HOME"
+borgBaseDir="/var/borgbackup"
+
+# full path to the SSH key used to connect to your remote backup server
+borgSSHKey="/var/borgbackup/private.key"
+
+# connection string to access the borg repo on your remote backup server
+# this is usually in the form user@servername.tld:repoName/
+borgConnectRepo="jdoe123@borg.server.net:mailcow/"
+
+# password to access repo
+# this was set when the repo was initialized and, while optional, is HIGHLY
+# recommended for security
+borgRepoPassphrase="p@ssW0rd"
+
+# keyfile to access repo
+# FULL PATH where the associated keyfile for your repo is located -- relevant
+# only if your repo requires a keyfile (i.e. 'keyfile' vs 'repokey') and if you
+# are not using the default keyfile location
+borgKeyfileLocation="/var/borgbackup/.config/borg/keys/server_address__repo_name"
+
+# REQUIRED: path to text file containing a list (one per line) of files/
+# directories to include in your backup. Since this is a generic backup script,
+# nothing is defined by default. Therefore, ONLY files specified in this file
+# will be backed up!
+# see repo wiki for more details
+borgXtraListPath="/scripts/backup/xtraLocations.borg"
+
+# OPTIONAL: path to file containing files/directories or 'patterns' to be
+# excluded in a BORG RECOGNIZED format
+# see repo wiki for more details or consult borg documentation
+# leave blank for no exclusions.
+borgExcludeListPath="/scripts/backup/excludeLocations.borg"
+
+# parameters to determine how borg deletes aged backups
+# more details in the repo wiki and/or borg documentation
+# leave blank to skip pruning altogether -- NOT recommended!
+borgPruneSettings="--keep-within=14d --keep-daily=30 --keep-weekly=12 --keep-monthly=12"
+
+# location of borg instance on your remote backup server
+# this is very often just "borg1"
+borgRemote="borg1"
@@ -0,0 +1,797 @@
+#!/bin/sh
+
+#######
+### mailcow backup using borgbackup
+### this assumes three things:
+###     1. standard mailcow-dockerized setup as per the docs
+###     2. using borg to perform backups to ssh-capable remote server
+###     3. remote repo already set-up and configured
+#######
+
+
+### text formatting presents
+if command -v tput > /dev/null; then
+    bold=$(tput bold)
+    cyan=$(tput setaf 6)
+    err=$(tput bold)$(tput setaf 1)
+    magenta=$(tput setaf 5)
+    norm=$(tput sgr0)
+    ok=$(tput setaf 2)
+    warn=$(tput bold)$(tput setaf 3)
+    width=$(tput cols)
+    yellow=$(tput setaf 3)
+else
+    bold=""
+    cyan=""
+    err=""
+    magenta=""
+    norm=""
+    ok=""
+    warn=""
+    width=80
+    yellow=""
+fi
+
+
+### trap
+trap trapExit 1 2 3 6
+
+
+### functions
+
+# bad configuration value passed in details file
+badDetails() {
+    if [ "$1" = "empty" ]; then
+        exitError 130 "details:${2} cannot be NULL (undefined)"
+    elif [ "$1" = "dne" ]; then
+        exitError 131 "details:${2} file or directory does not exist."
+    fi
+}
+
+# bad parameter passed to script
+badParam() {
+    if [ "$1" = "dne" ]; then
+        printf "\n%sError: '%s %s'\n" "$err" "$2" "$3"
+        printf "file or directory does not exist.%s\n\n" "$norm"
+        exit 1
+    elif [ "$1" = "empty" ]; then
+        printf "\n%sError: '%s' cannot have a NULL (empty) value.\n" "$err" "$2"
+        printf "%sPlease use '--help' for assistance%s\n\n" "$cyan" "$norm"
+        exit 1
+    elif [ "$1" = "svc" ]; then
+        printf "\n%sError: '%s %s': Service does not exist!%s\n\n" \
+            "$err" "$2" "$3" "$norm"
+        exit 1
+    elif [ "$1" = "user" ]; then
+        printf "\n%sError: '%s %s': User does not exist!%s\n\n" \
+            "$err" "$2" "$3" "$norm"
+        exit 1
+    fi
+}
+
+# cleanup
+cleanup() {
+    # cleanup 503 if copied
+    if [ "$err503Copied" -eq 1 ]; then
+        if ! rm -f "$webroot/$err503File" 2>>"$logFile"; then
+            printf "%s[%s] -- [WARNING] Could not remove 503 error page." \
+                "$warn" "$(stamp)" >> "$logFile"
+            printf " Web interface will not function until this file is " \
+                >> "$logFile"
+            printf "removed --%s\n" "$norm" >> "$logFile"
+            warnCount=$((warnCount+1))
+        else
+            printf "%s[%s] -- [INFO] 503 error page removed --%s\n" \
+                "$cyan" "$(stamp)" "$norm" >> "$logFile"
+        fi        
+    fi
+    # cleanup SQL dump directory if created
+    if [ "$sqlDumpDirCreated" -eq 1 ]; then
+        if ! rm -rf "$sqlDumpDir" 2>>"$logFile"; then
+            printf "%s[%s] -- [WARNING] Could not remove temporary SQL-dump directory. Sorry for the mess. --%s\n" \
+                "$warn" "$(stamp)" "$norm" >> "$logFile"
+        else
+            printf "%s[%s] -- [INFO] Temporary SQL-dump directory removed successfully --%s\n" \
+                "$cyan" "$(stamp)" "$norm" >> "$logFile"
+        fi
+    fi
+    # start docker containers (no harm if they are already running)
+    doDocker start postfix
+    if [ "$dockerResultState" = "true" ]; then
+        printf "%s[%s] -- [INFO] POSTFIX container is running --%s\n" \
+            "$cyan" "$(stamp)" "$norm" >> "$logFile"
+    else
+        exitError 102 'Could not start POSTFIX container.'
+    fi
+    doDocker start dovecot
+    if [ "$dockerResultState" = "true" ]; then
+        printf "%s[%s] -- [INFO] DOVECOT container is running --%s\n" \
+            "$cyan" "$(stamp)" "$norm" >> "$logFile"
+    else
+        exitError 102 'Could not start DOVECOT container.'
+    fi
+}
+
+doDocker() {
+    containerName="$( docker ps -a --format '{{ .Names }}' --filter name=${COMPOSE_PROJECT_NAME}_${2}-mailcow_1 )"
+
+    # determine action to take
+    if [ "$1" = "stop" ]; then
+        printf "%s[%s] -- [INFO] Stopping %s-mailcow container --%s\n" \
+            "$cyan" "$(stamp)" "$2" "$norm" >> "$logFile"
+        docker-compose -f "$mcDockerCompose" stop --timeout "$dockerStopTimeout" "$2-mailcow" 2>> "$logFile"
+        # set result vars
+        dockerResultState="$( docker inspect -f '{{ .State.Running }}' $containerName )"
+        dockerResultExit="$( docker inspect -f '{{ .State.ExitCode }}' $containerName )"
+    elif [ "$1" = "start" ]; then
+        printf "%s[%s] -- [INFO] Starting %s-mailcow container --%s\n" \
+            "$cyan" "$(stamp)" "$2" "$norm" >> "$logFile"
+        docker-compose -f "$mcDockerCompose" start "$2-mailcow" 2>> "$logFile"
+        # set result vars
+        dockerResultState="$( docker inspect -f '{{ .State.Running }}' $containerName )"
+    fi
+}
+
+# call cleanup and then exit with error report
+exitError() {
+    printf "%s[%s] -- [ERROR] %s: %s --%s\n" \
+            "$err" "$(stamp)" "$1" "$2" "$norm" >> "$logFile"
+    cleanup
+    # note script completion with error
+    printf "%s[%s] --- %s execution completed with error ---%s\n" \
+        "$err" "$(stamp)" "$scriptName" "$norm" >> "$logFile"
+    exit "$1"
+}
+
+# display script help information
+scriptHelp() {
+    newline
+    printf "%sUsage: %s [parameters]%s\n\n" "$bold" "$scriptName" "$norm"
+    textblock "There are NO mandatory parameters. If a parameter is not supplied, its default value will be used. In the case of a switch parameter, it will remain DEactivated if NOT specified."
+    newline
+    textblock "Parameters are listed then followed by a description of their effect on the following line. Finally, if a default value exists, it will be listed on the next line in (parentheses)."
+    newline
+    textblock "${magenta}--- script related parameters ---${norm}"
+    newline
+    switchTextblock "-c | --config | --details"
+    textblock "Path to the configuration key/value-pair file for this script."
+    defaultsTextblock "(scriptPath/scriptName.details)"
+    newline
+    switchTextblock "-h | -? | --help"
+    textblock "This help screen"
+    newline
+    switchTextblock "-l | --log"
+    textblock "Path to write log file"
+    defaultsTextblock "(scriptPath/scriptName.log)"
+    newline
+    switchTextblock "[SWITCH] -v | --verbose"
+    textblock "Log borg output with increased verbosity (list all files). Careful! Your log file can get very large very quickly!"
+    defaultsTextblock "(normal output, option is OFF)"
+    newline
+    textblock "${magenta}--- 503 functionality ---${norm}"
+    newline
+    switchTextblock "[SWITCH] -5 | --use-503"
+    textblock "Copy an 'error 503' page/indicator file to your webroot for your webserver to find. Specifying this option will enable other 503 options."
+    defaultsTextblock "(do NOT copy, option is OFF)"
+    newline
+    switchTextblock "--503-path"
+    textblock "Path to the file you want copied to your webroot as the 'error 503' page."
+    defaultsTextblock "(scriptPath/503_backup.html)"
+    newline
+    switchTextblock "-w | --webroot"
+    textblock "Path to where the 'error 503' file should be copied."
+    defaultsTextblock "(/usr/share/nginx/html/)"
+    newline
+    textblock "${magenta}--- mailcow related ---${norm}"
+    newline
+    switchTextblock "-d | --docker-compose"
+    textblock "Path to mailcow's 'docker-compose.yml' file."
+    defaultsTextblock "(/opt/mailcow-dockerized/docker-compose.yml)"
+    newline
+    switchTextblock "-m | --mailcow-config"
+    textblock "Path to mailcow configuration file ('mailcow.conf')."
+    defaultsTextblock "(/opt/mailcow-dockerized/mailcow.conf)"
+    newline
+    switchTextblock "-t1 | --dockerStartTimeout"
+    textblock "Seconds to wait for docker containers to start."
+    defaultsTextblock "(180)"
+    newline
+    switchTextblock "-t2 | --dockerStopTimeout"
+    textblock "Seconds to wait for docker containers to stop."
+    defaultsTextblock "(120)"
+    newline
+    textblock "More details and examples of script usage can be found in the repo wiki at ${yellow}https://git.asifbacchus.app/asif/MailcowBackup/wiki${norm}"
+    newline
+}
+
+# generate dynamic timestamps
+stamp() {
+    (date +%F" "%T)
+}
+
+textblock() {
+    printf "%s\n" "$1" | fold -w "$width" -s
+}
+
+defaultsTextblock() {
+    printf "%s%s%s\n" "$yellow" "$1" "$norm"
+}
+
+switchTextblock() {
+    printf "%s%s%s\n" "$cyan" "$1" "$norm"
+}
+
+# print a blank line
+newline() {
+    printf "\n"
+}
+
+# same as exitError but for signal captures
+trapExit() {
+    printf "%s[%s] -- [ERROR] 99: Caught signal --%s\n" \
+            "$err" "$(stamp)" "$norm" >> "$logFile"
+    cleanup
+    # note script completion with error
+    printf "%s[%s] --- %s execution was terminated via signal ---%s\n" \
+        "$err" "$(stamp)" "$scriptName" "$norm" >> "$logFile"
+    exit 99
+}
+
+### end of functions
+
+
+### default variable values
+
+## script related
+# store logfile in the same directory as this script file using the same file
+# name as the script but with the extension '.log'
+scriptPath="$( CDPATH='' cd -- "$( dirname -- "$0" )" && pwd -P )"
+scriptName="$( basename "$0" )"
+logFile="$scriptPath/${scriptName%.*}.log"
+warnCount=0
+configDetails="$scriptPath/${scriptName%.*}.details"
+err503Copied=0
+exclusions=0
+# borg output verbosity -- normal
+borgCreateParams='--stats'
+borgPruneParams='--list'
+
+# 503 related
+use503=0
+err503Path="$scriptPath/503_backup.html"
+err503File="${err503Path##*/}"
+webroot="/usr/share/nginx/html"
+
+# mailcow/docker related
+mcConfig='/opt/mailcow-dockerized/mailcow.conf'
+mcDockerCompose="/opt/mailcow-dockerized/docker-compose.yml"
+dockerStartTimeout=180
+dockerStopTimeout=120
+
+
+### process startup parameters
+while [ $# -gt 0 ]; do
+    case "$1" in
+        -h|-\?|--help)
+            # display help
+            scriptHelp
+            exit 0
+            ;;
+        -l|--log)
+            # set log file location
+            if [ -n "$2" ]; then
+                logFile="${2%/}"
+                shift
+            else
+                badParam empty "$@"
+            fi
+            ;;
+        -c|--config|--details)
+            # location of config details file
+            if [ -n "$2" ]; then
+                if [ -f "$2" ]; then
+                    configDetails="${2%/}"
+                    shift
+                else
+                    badParam dne "$@"
+                fi
+            else
+                badParam empty "$@"
+            fi
+            ;;
+        -v|--verbose)
+            # set verbose logging from borg
+            borgCreateParams='--list --stats'
+            borgPruneParams='--list'
+            ;;
+        -5|--use-503)
+            # enable copying 503 error page to webroot
+            use503=1
+            ;;
+        --503-path)
+            # FULL path to 503 file
+            if [ -n "$2" ]; then
+                if [ -f "$2" ]; then
+                    err503Path="${2%/}"
+                    err503File="${2##*/}"
+                    shift
+                else
+                    badParam dne "$@"
+                fi
+            else
+                badParam empty "$@"
+            fi
+            ;;
+        -w|--webroot)
+            # path to webroot (copy 503)
+            if [ -n "$2" ]; then
+                if [ -d "$2" ]; then
+                    webroot="${2%/}"
+                    shift
+                else
+                    badParam dne "$@"
+                fi
+            else
+                badParam empty "$@"
+            fi
+            ;;
+        -d|--docker-compose)
+            # FULL path to mailcow docker-compose file
+            if [ -n "$2" ]; then
+                if [ -f "$2" ]; then
+                    mcDockerCompose="$2"
+                    shift
+                else
+                    badParam dne "$@"
+                fi
+            else
+                badParam empty "$@"
+            fi
+            ;;
+        -m|--mailcow-config)
+            # FULL path to mailcow configuration file
+            if [ -n "$2" ]; then
+                if [ -f "$2" ]; then
+                    mcConfig="$2"
+                    shift
+                else
+                    badParam dne "$@"
+                fi
+            else
+                badParam empty "$@"
+            fi
+            ;;
+        -t1|--timeout-start)
+            if [ -z "$2" ]; then
+                badParam empty "$@"
+            else
+                dockerStartTimeout="$2"
+            fi
+            ;;
+        -t2|--timeout-stop)
+            if [ -z "$2" ]; then
+                badParam empty "$@"
+            else
+                dockerStopTimeout="$2"
+            fi
+            ;;
+        *)
+            printf "\n%sUnknown option: %s\n" "$err" "$1"
+            printf "%sUse '--help' for valid options.%s\n\n" "$cyan" "$norm"
+            exit 1
+            ;;
+    esac
+    shift
+done
+
+
+### check pre-requisites and default values
+# check if running as root, otherwise exit
+if [ "$( id -u )" -ne 0 ]; then
+    printf "\n%sERROR: script MUST be run as ROOT%s\n\n" "$err" "$norm"
+    exit 2
+fi
+# does the details file exist?
+if [ ! -f "$configDetails" ]; then
+    badParam dne "(--details default)" "$configDetails"
+fi
+# is borg installed?
+if ! command -v borg > /dev/null; then
+    printf "\n%sERROR: BORG is not installed on this system!%s\n\n" "$err" "$norm"
+    exit 3
+fi
+# if 503 functionality is enabled, do 503 related files exist?
+if [ "$use503" -eq 1 ]; then
+    if [ ! -f "$err503Path" ]; then
+        badParam dne "(--503-path default)" "$err503Path"
+    elif [ ! -d "$webroot" ]; then
+        badParam dne "(--webroot default)" "$webroot"
+    fi
+fi
+# verify mailcow.conf location and extract path
+if [ ! -f "$mcConfig" ]; then
+    badParam dne "(--mailcow-config)" "$mcConfig"
+fi
+# verify docker-compose file exists
+if [ ! -f "$mcDockerCompose" ]; then
+    badParam dne "(--docker-compose)" "$mcDockerCompose"
+fi
+
+
+### read mailcow.conf and set vars as needed
+. "$mcConfig"
+export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+export COMPOSE_HTTP_TIMEOUT="$dockerStartTimeout"
+
+
+### start logging
+printf "%s[%s] --- Start %s execution ---%s\n" \
+    "$magenta" "$(stamp)" "$scriptName" "$norm" >> "$logFile"
+printf "%s[%s] -- [INFO] Log located at %s%s%s --%s\n" \
+    "$cyan" "$(stamp)" "$yellow" "$logFile" "$cyan" "$norm" >> "$logFile"
+
+
+### get location of docker volumes
+dockerVolumeMail=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_vmail-vol-1)
+printf "%s[%s] -- [INFO] Using MAIL volume: %s --%s\n" \
+    "$cyan" "$(stamp)" "$dockerVolumeMail" "$norm" >> "$logFile"
+dockerVolumeRspamd=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_rspamd-vol-1)
+printf "%s[%s] -- [INFO] Using RSPAMD volume: %s --%s\n" \
+    "$cyan" "$(stamp)" "$dockerVolumeRspamd" "$norm" >> "$logFile"
+dockerVolumePostfix=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_postfix-vol-1)
+printf "%s[%s] -- [INFO] Using POSTFIX volume: %s --%s\n" \
+    "$cyan" "$(stamp)" "$dockerVolumePostfix" "$norm" >> "$logFile"
+dockerVolumeRedis=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_redis-vol-1)
+printf "%s[%s] -- [INFO] Using REDIS volume: %s --%s\n" \
+    "$cyan" "$(stamp)" "$dockerVolumeRedis" "$norm" >> "$logFile"
+dockerVolumeCrypt=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_crypt-vol-1)
+printf "%s[%s] -- [INFO] Using MAILCRYPT volume: %s --%s\n" \
+    "$cyan" "$(stamp)" "$dockerVolumeCrypt" "$norm" >> "$logFile"
+
+
+### read details file to get variables needed run borg
+# check if config details file was provided as a relative or absolute path
+case "${configDetails}" in
+    /*)
+        # absolute path, no need to rewrite variable
+        . "${configDetails}"
+        ;;
+    *)
+        # relative path, prepend './' to create absolute path
+        . "./${configDetails}"
+        ;;
+esac
+printf "%s[%s] -- [INFO] %s%s%s imported --%s\n" \
+    "$cyan" "$(stamp)" "$yellow" "$configDetails" "$cyan" "$norm" >> "$logFile"
+
+
+### Run borg variable checks
+printf "%s[%s] -- [INFO] Verifying supplied borg details --%s\n" \
+    "$cyan" "$(stamp)" "$norm" >> "$logFile"
+
+## verify borg base directory
+if [ -z "${borgBaseDir}" ]; then
+    badDetails empty 'borgBaseDir'
+elif [ ! -d "${borgBaseDir}" ]; then
+    badDetails dne 'borgBaseDir'
+fi
+printf "%sdetails:borgBaseDir %s-- %s[OK]%s\n" \
+    "$magenta" "$norm" "$ok" "$norm" >> "$logFile"
+export BORG_BASE_DIR="${borgBaseDir%/}"
+
+## check path to SSH keyfile
+if [ -z "${borgSSHKey}" ]; then
+    badDetails empty 'borgSSHKey'
+elif [ ! -f "${borgSSHKey}" ]; then
+    badDetails dne 'borgSSHKey'
+fi
+printf "%sdetails:borgSSHKey %s-- %s[OK]%s\n" \
+    "$magenta" "$norm" "$ok" "$norm" >> "$logFile"
+export BORG_RSH="ssh -i ${borgSSHKey}"
+
+## check borg repo connect string
+if [ -z "${borgConnectRepo}" ]; then
+    badDetails empty 'borgConnectRepo'
+fi
+printf "%sdetails:borgConnectRepo %s-- %s[OK]%s\n" \
+    "$magenta" "$norm" "$ok" "$norm" >> "$logFile"
+export BORG_REPO="${borgConnectRepo}"
+
+## check borg repo password
+if [ -n "${borgRepoPassphrase}" ]; then
+    printf "%sdetails:borgRepoPassphrase %s-- %s[OK]%s\n" \
+    "$magenta" "$norm" "$ok" "$norm" >> "$logFile"
+    export BORG_PASSPHRASE="${borgRepoPassphrase}"
+else
+    # if passwd is blank intentionally, this is insecure
+    printf "%s-- [WARNING] Using a borg repo without a password is an " \
+        "$warn" >> "$logFile"
+    printf "insecure configuration --%s\n" "$norm">> "$logFile"
+    warnCount=$((warnCount+1))
+    # if this was an accident, we need to provide a bogus passwd so borg fails
+    # otherwise it will sit forever just waiting for input
+    export BORG_PASSPHRASE="DummyPasswordSoBorgFails"
+fi
+
+## check borg repository keyfile location
+if [ -z "${borgKeyfileLocation}" ]; then
+    printf "%sdetails:borgKeyfileLocation %s-- %s[DEFAULT]%s\n" "$magenta" "$norm" "$ok" "$norm" >> "$logFile"
+else
+    # check if keyfile location exists
+    if [ ! -f "${borgKeyfileLocation}" ]; then
+        badDetails dne 'borgKeyfileLocation'
+    fi
+    printf "%sdetails:borgKeyfileLocation %s-- %s[OK]%s\n" "$magenta" "$norm" "$ok" "$norm" >> "$logFile"
+    export BORG_KEY_FILE="${borgKeyfileLocation}"
+fi
+
+## export borg remote path, if specified
+if [ -n "${borgRemote}" ]; then export BORG_REMOTE_PATH="${borgRemote}"; fi
+
+## check if exlusion list file is specified
+if [ -n "${borgExcludeListPath}" ]; then
+    # check if the file actually exists
+    if [ ! -f "${borgExcludeListPath}" ]; then
+        badDetails dne 'borgExcludeListPath'
+    fi
+exclusions=1
+fi
+
+## read additional files
+if [ -z "${borgXtraListPath}" ]; then
+    xtraList=''
+else
+    # check if file actually exists
+    if [ ! -f "${borgXtraListPath}" ]; then
+        badDetails dne 'borgXtraListPath'
+    fi
+    # read file contents into concatenated list for echo to cmdline
+    while read -r xtraItem; do
+        if [ -z "${xtraList}" ]; then
+            xtraList="${xtraItem}"
+        else
+            xtraList="${xtraList} ${xtraItem}"
+        fi
+    done <<EOF
+    $( sed -e '/^\s*#.*$/d' -e '/^\s*$/d' "${borgXtraListPath}" )
+EOF
+printf "%sdetails:borgXtraListPath %s-- %s[OK]%s\n" \
+    "$magenta" "$norm" "$ok" "$norm" >> "$logFile"
+fi
+
+
+### set location of sql dump
+# this is done before resetting default TMP dir for borg
+if ! sqlDumpDir=$( mktemp -d 2>/dev/null ); then
+    exitError 115 'Unable to create temp directory for SQL dump.'
+else
+    sqlDumpFile="backup-$( date +%Y%m%d_%H%M%S ).sql"
+    sqlDumpDirCreated=1
+    printf "%s[%s] -- [INFO] SQL dump file will be stored at: %s --%s\n" \
+        "$cyan" "$(stamp)" "$sqlDumpDir/$sqlDumpFile" "$norm" >> "$logFile"
+fi
+
+
+### create borg temp dir:
+## python requires a writable temporary directory when unpacking borg and
+## executing commands.  This defaults to /tmp but many systems mount /tmp with
+## the 'noexec' option for security.  Thus, we will use/create a 'tmp' folder
+## within the BORG_BASE_DIR and instruct python to use that instead of /tmp
+
+# check if BORG_BASE_DIR/tmp exists, if not, create it
+if [ ! -d "${borgBaseDir}/tmp" ]; then
+    if ! mkdir "${borgBaseDir}/tmp"; then
+        exitError 132 "Unable to create borg ${borgBaseDir}/tmp directory"
+    else
+        printf "%s[%s] -- [INFO] Created %s%s/tmp " \
+            "$cyan" "$(stamp)" "$yellow" "${borgBaseDir}" >> "$logFile"
+        printf "%s--%s\n" "$cyan" "$norm">> "$logFile"
+    fi
+fi
+export TMPDIR="${borgBaseDir}/tmp"
+
+
+### 503 functionality
+if [ "$use503" -eq 1 ]; then
+    printf "%s[%s] -- [INFO] Copying 503 error page to " \
+        "$cyan" "$(stamp)" >> "$logFile"
+    printf "webroot -- %s\n" "$norm">> "$logFile"
+    if ! cp --force "${err503Path}" "${webroot}/${err503File}" 2>> "$logFile"
+        then
+        printf "%s[%s] -- [WARNING] Failed to copy 503 error page. " \
+            "$warn" "$(stamp)" >> "$logFile"
+        printf "Web users will NOT be notified --%s\n" "$norm" >> "$logFile"
+        warnCount=$((warnCount+1))
+    else
+        printf "%s[%s] -- [SUCCESS] 503 error page copied --%s\n" \
+            "$ok" "$(stamp)" "$norm" >> "$logFile"
+        # set cleanup flag
+        err503Copied=1
+    fi
+fi
+
+### change to mailcow directory so docker commands execute properly
+cd "${mcConfig%/*}" || exitError 100 'Could not change to mailcow directory.'
+
+### stop postfix and dovecot mail containers to prevent mailflow during backup
+doDocker stop postfix
+if [ "$dockerResultState" = "false" ] && [ "$dockerResultExit" -eq 0 ]; then
+    printf "%s[%s] -- [INFO] POSTFIX container stopped --%s\n" \
+        "$cyan" "$(stamp)" "$norm" >> "$logFile"
+else
+    exitError 101 'Could not stop POSTFIX container.'
+fi
+doDocker stop dovecot
+if [ "$dockerResultState" = "false" ] && [ "$dockerResultExit" -eq 0 ]; then
+    printf "%s[%s] -- [INFO] DOVECOT container stopped --%s\n" \
+        "$cyan" "$(stamp)" "$norm" >> "$logFile"
+else
+    exitError 101 'Could not stop DOVECOT container.'
+fi
+
+
+### dump SQL
+printf "%s[%s] -- [INFO] Dumping mailcow SQL database --%s\n" \
+    "$cyan" "$(stamp)" "$norm" >> "$logFile"
+docker-compose exec -T mysql-mailcow mysqldump --default-character-set=utf8mb4 \
+    -u${DBUSER} -p${DBPASS} ${DBNAME} > "$sqlDumpDir/$sqlDumpFile" 2>> "$logFile"
+dumpResult=$( docker-compose exec -T mysql-mailcow echo "$?" )
+if [ "$dumpResult" -eq 0 ]; then
+    printf "%s[%s] -- [INFO] SQL database dumped successfully --%s\n" \
+        "cyan" "$(stamp)" "$norm" >> "$logFile"
+else
+    exitError 118 'There was an error dumping the mailcow SQL database.'
+fi
+
+
+### dump redis inside container
+# delete old redis dump if it exists
+if [ -f "$dockerVolumeRedis/dump.rdb" ]; then
+    rm -f "$dockerVolumeRedis/dump.rdb"
+fi
+# dump redis
+printf "%s[%s] -- [INFO] Dumping mailcow redis database --%s\n" \
+    "$cyan" "$(stamp)" "$norm" >> "$logFile"
+docker-compose exec -T redis-mailcow redis-cli save >> "$logFile" 2>&1
+rdumpResult=$( docker-compose exec -T redis-mailcow echo "$?" )
+if [ "$rdumpResult" -eq 0 ]; then
+    printf "%s[%s] -- [INFO] mailcow redis dumped successfully --%s\n" \
+        "cyan" "$(stamp)" "$norm" >> "$logFile"
+else
+    exitError 119 'There was an error dumping the mailcow redis database.'
+fi
+
+
+### execute borg depending on whether exclusions are defined
+printf "%s[%s] -- [INFO] Pre-backup tasks completed, calling borgbackup --%s\n" "$cyan" "$(stamp)" "$norm" >> "$logFile"
+
+## construct the proper borg commandline
+# base command
+if [ "$exclusions" -eq 0 ]; then
+    borgCMD="borg --show-rc create ${borgCreateParams} \
+        ::$(date +%Y-%m-%d_%H%M%S) \
+        ${mcConfig%/*} \
+        ${sqlDumpDir} \
+        ${dockerVolumeMail} \
+        ${dockerVolumeRspamd} \
+        ${dockerVolumePostfix} \
+        ${dockerVolumeRedis} \
+        ${dockerVolumeCrypt} \
+        ${xtraList}"
+elif [ "$exclusions" -eq 1 ]; then
+    borgCMD="borg --show-rc create ${borgCreateParams} \
+        --exclude-from ${borgExcludeListPath} \
+        ::$(date +%Y-%m-%d_%H%M%S) \
+        ${mcConfig%/*} \
+        ${sqlDumpDir} \
+        ${dockerVolumeMail} \
+        ${dockerVolumeRspamd} \
+        ${dockerVolumePostfix} \
+        ${dockerVolumeRedis} \
+        ${dockerVolumeCrypt} \
+        ${xtraList}"
+fi
+
+# execute borg
+printf "%s[%s] -- [INFO] Executing borg backup operation --%s\n" \
+    "$cyan" "$(stamp)" "$norm" >> "$logFile"
+${borgCMD} 2>> "$logFile"
+borgResult="$?"
+
+## check borg exit status
+if [ "$borgResult" -eq 0 ]; then
+    printf "%s[%s] -- [SUCCESS] Borg backup completed --%s\n" \
+        "$ok" "$(stamp)" "$norm" >> "$logFile"
+elif [ "$borgResult" -eq 1 ]; then
+    printf "%s[%s] -- [WARNING] Borg completed with warnings. " \
+        "$warn" "$(stamp)" >> "$logFile"
+    printf "Review this logfile for details --%s\n" "$norm">> "$logFile"
+    warnCount=$((warnCount+1))
+elif [ "$borgResult" -ge 2 ]; then
+    err_1="Borg exited with a critical error. Please review this log file"
+    err_2="for details."
+    exitError 138 "$err_1 $err_2"
+else
+    printf "%s[%s] -- [WARNING] Borg exited with unknown return code. " \
+        "$warn" "$(stamp)" >> "$logFile"
+    printf "Review this logfile for details --%s\n" "$norm">> "$logFile"
+    warnCount=$((warnCount+1))
+fi
+
+
+### execute borg prune if paramters are provided, otherwise skip with a warning
+if [ -n "${borgPruneSettings}" ]; then
+    printf "%s[%s] -- [INFO] Executing borg prune operation --%s\n" \
+        "$cyan" "$(stamp)" "$norm" >> "$logFile"
+    borg prune --show-rc -v ${borgPruneParams} ${borgPruneSettings} \
+        2>> "$logFile"
+    borgPruneResult="$?"
+else
+    printf "%s[%s] -- [WARNING] No prune parameters provided. " \
+        "$warn" "$(stamp)" >> "$logFile"
+    printf "Your archive will continue growing with each backup --%s\n" \
+        "$norm" >> "$logFile"
+    warnCount=$((warnCount+1))
+fi
+
+## report on prune operation if executed
+if [ -n "${borgPruneResult}" ]; then
+    if [ "${borgPruneResult}" -eq 0 ]; then
+        printf "%s[%s] -- [SUCCESS] Borg prune completed --%s\n" \
+            "$ok" "$(stamp)" "$norm" >> "$logFile"
+    elif [ "$borgPruneResult" -eq 1 ]; then
+        printf "%s[%s] -- [WARNING] Borg prune completed with warnings. " \
+        "$warn" "$(stamp)" >> "$logFile"
+        printf "Review this logfile for details --%s\n" "$norm" >> "$logFile"
+        warnCount=$((warnCount+1))
+    elif [ "$borgPruneResult" -ge 2 ]; then
+        err_1="Borg prune exited with a critical error. Please review this"
+        err_2="log file for details."
+        exitError 139 "$err_1 $err_2"
+    else
+        printf "%s[%s] -- [WARNING] Borg prune exited with an unknown " \
+            "$warn" "$(stamp)" >> "$logFile"
+        printf "return code. Review this logfile for details --%s\n" \
+            "$norm" >> "$logFile"
+        warnCount=$((warnCount+1))
+    fi
+fi
+
+
+### all processes successfully completed, cleanup and exit gracefully
+
+# note successful completion of borg commands
+printf "%s[%s] -- [SUCCESS] Backup operations completed --%s\n" \
+    "$ok" "$(stamp)" "$norm" >> "$logFile"
+
+# cleanup
+cleanup
+
+# note complete success, tally warnings and exit
+printf "%s[%s] -- [SUCCESS] All processes completed --%s\n" \
+    "$ok" "$(stamp)" "$norm" >> "$logFile"
+printf "%s[%s] --- %s execution completed ---%s\n" \
+    "$magenta" "$(stamp)" "$scriptName" "$norm" >> "$logFile"
+if [ "$warnCount" -gt 0 ]; then
+    printf "%s%s warnings issued!%s\n" "$warn" "${warnCount}" "$norm" >> "$logFile"
+else
+    printf "%s0 warnings issued.%s\n" "$ok" "$norm" >> "$logFile"
+fi
+exit 0
+
+
+### error codes
+# 1: parameter error
+# 2: not run as root
+# 3: borg not installed
+# 99: TERM signal trapped
+# 100: could not change to mailcow-dockerized directory
+# 101: could not stop container(s)
+# 102: could not start container(s)
+# 115: unable to create temp dir for SQL dump
+# 118: error dumping SQL database
+# 119: error dumping redis database
+# 130: null configuration variable in details file
+# 131: invalid configuration variable in details file
+# 138: borg exited with a critical error
+# 139: borg prune exited with a critical error
@@ -0,0 +1,2 @@
+
+EOF
@@ -0,0 +1,60 @@
+# Files and directories listed here will be included in your borg backup
+#
+# Good candidates for inclusion would be things like your mailcow configuration
+# files, customized docker-compose overrides, your SSL certificates, etc.
+#
+# List the path to files/directories one per line.
+# Any blank lines will be ignored.
+# Any lines starting with '#' will be ignored as a comment.
+# For consistency, you should include the trailing slash for directories.
+
+# these examples are for a very basic Debian machine hosting mailcow
+
+
+### important system configuration files
+
+# basic configuration
+/etc/fstab
+/etc/network/interfaces
+/etc/network/interfaces.d/
+/etc/systemd/timesyncd.conf
+
+# ssh configuration and host keys
+/etc/ssh/
+
+# apt configuration
+/etc/apt/sources.list
+/etc/apt/sources.list.d/
+/etc/apt/listchanges.conf
+/etc/apt/apt.conf.d/50unattended-upgrades
+/etc/apt/apt.conf.d/20auto-upgrades
+
+# user profile defaults and configurations
+/etc/profile
+/etc/bash.bashrc
+/etc/skel/
+/etc/nanorc
+
+# selected root user files
+/root/.bashrc
+/root/.ssh/
+
+# scripts
+/scripts/
+
+
+### important programs and configurations
+
+# name of program for reference
+# include the paths to important configuration files/directories and/or
+# data directories
+
+# mailcow configuration (example)
+/opt/mailcow-dockerized/
+
+# NGINX (example)
+/etc/nginx/
+/usr/share/nginx/html/
+
+# LetsEncrypt (example)
+/etc/letsencrypt/
@@ -1,300 +1,13 @@
-# Using Logwatch to monitor backup script <!-- omit in toc -->
+# Using Logwatch to monitor the backup script

-The backup script's log file has been set up so that utilities like Logwatch can
-easily parse it.  In order to make that happen, a LogFile Group file, Service
-and Script have to be created for Logwatch to generate reports.  The correct
-(general) directory structure has been created in this git archive already.
-Below are the details of each file.
+## quick start

-**If you don't care about how it works, you can simply copy this folder to your
-Logwatch configuration directory (_/etc/logwatch/_ by default).  Everything is
-already in the proper directory structure for a default Debian/Ubuntu
-installation.**
+Simply copy the contents of this folder to your logwatch configuration directory (*/etc/logwatch/* by default). The directory structure is already correct for a default Debian/Ubuntu logwatch installation. You **must** update the paths in */etc/logwatch/conf/logfiles/backup.conf* to point to your script's log file, but that's the only required change. Please consult [page 7.1.5](https://git.asifbacchus.app/asif/MailcowBackup/wiki/7.1.5-Testing) in the wiki for information on how to test logwatch using this new configuration.

-*If you need help installing or setting up Logwatch, please see my blog at
-[https://mytechiethoughts.com](https://mytechiethoughts.com) and search for
-'_logwatch_'*.  These instructions assume you already have Logwatch setup correctly.
+## more information

-## Contents <!-- omit in toc -->
+Please consult [section 7.1](https://git.asifbacchus.app/asif/MailcowBackup/wiki/7.1-Using-logwatch) in the wiki for detailed information about each logwatch configuration file contained within this section of the git repo and how to customize them for your environment.

- [LogFile Group file (/etc/logwatch/conf/logfiles/backup.conf)](#logfile-group-file-etclogwatchconflogfilesbackupconf)
-    - [Log file location](#log-file-location)
-    - [Archive location and name format](#archive-location-and-name-format)
-    - [External script for timestamp processing](#external-script-for-timestamp-processing)
- [Service definition file (/etc/logwatch/conf/services/backup.conf)](#service-definition-file-etclogwatchconfservicesbackupconf)
-    - [LogFile Group file definition](#logfile-group-file-definition)
-    - [Report title](#report-title)
-    - [Detail level](#detail-level)
- [Service script (/etc/logwatch/scripts/services/backup)](#service-script-etclogwatchscriptsservicesbackup)
-    - [Detail levels](#detail-levels)
- [Timestamp processing script (/etc/logwatch/scripts/shared/sqfullstampanywhere)](#timestamp-processing-script-etclogwatchscriptssharedsqfullstampanywhere)
-    - [The time format specification](#the-time-format-specification)
-    - [The search REGEX](#the-search-regex)
- [Testing](#testing)
- [Final thoughts](#final-thoughts)
+## final thoughts

-## LogFile Group file (/etc/logwatch/conf/logfiles/backup.conf)
-
-### Log file location
-
-Update this as needed to point to the location and name of the log file
-generated by the backup script.  Remember, by default, the log file is created
-in the same directory as the script itself.
-
-```Ini
-LogFile = /path/to/your/backup.log
-...
-```
-
-Best practices suggest you use the backup script's *-l* flag to change this
-location to something like */var/log/backup.log*, for example.  In that case,
-the entry would look like:
-
-```Ini
-LogFile = /var/log/backup.log
-...
-```
-
-### Archive location and name format
-
-If you want Logwatch to process old (archived) log files generated by something
-like *Logrotate*, then you have to specify the location and file name format of
-those files.  I've included the generalized compressed format of such rotated
-files as the default in the script.  Suppose you store your log files in the
-recommended location (*/var/log/*) and are using *Logrotate* with compression
-enabled, the archive line would look like:
-
-```Ini
-...
-Archive = /var/log/backup.log.?.gz
-...
-```
-
-This would tell Logwatch, when the archive option is set to true, that your
-*backup.log* files are archived as: *backup.log.1.gz*, *backup.log.2.gz*, etc.
-and are all located in */var/log/*.
-
-**Note: This line is totally optional and only used if you set the archive
-option in Logwatch to true (default).  You can comment/delete this line if you
-wish.**
-
-### External script for timestamp processing
-
-Since the log file uses a non-standard (according to Logwatch) method of
-datestamping, a custom filter had to be created.  See the
-[relevant](#timestamp-processing-script-etclogwatchscriptssharedsqfullstampanywhere)
-section of this document for more information.
-
-The script file is called with an *\** before the filename.
-
-```Ini
-...
-*sqFullStampAnywhere
-...
-```
-
-If you change the name of this file, you will have to change this line.
-Remember that whatever you type here as a name is converted to all-lowercase
-so your filename should be all lowercase also.
-
-## Service definition file (/etc/logwatch/conf/services/backup.conf)
-
-### LogFile Group file definition
-
-The service file needs to know what group of log files it is responsible for
-processing.  This MUST match the name of your *LogFile Group file*:
-
-```Ini
-LogFile = backup
-...
-```
-
-If you change your LogFile Group filename, then update it here too without the
-*.conf* extension.
-
-### Report title
-
-The Logwatch output file (html or text) is divided into sections.  You can
-define the title to be anything that has meaning for you.  I have arbitrarily
-chosen *"System and Mailcow Backup"* but you can change it to anything you want by
-modifying the line:
-
-```Ini
-...
-Title = "System and Mailcow Backup"
-```
-
-### Detail level
-
-If you want to set the *detail* level of this service differently from your
-other services (which will use the *--detail* switch value or the value in your
-*logwatch.conf*), then you can define that level here.  By default, it appears
-like this in the service configuration file:
-
-```Ini
-...
-# Override the detail level for this service
-# Remember the levels are: 0, 1-4, 5, 6+
-# Detail = 0
-```
-
-Simply change it to the value you want enforced.  For example, here I'm setting
-it to output level 5 regardless of whatever settings everything else is using.
-
-```Ini
-# Override the detail level for this service
-# Remember the levels are: 0, 1-4, 5, 6+
-Detail = 5
-```
-
-## Service script (/etc/logwatch/scripts/services/backup)
-
-Logwatch calls any script with a name that **matches the service name**.  You'll
-notice that I just named everything *backup* to keep things simple.  You can
-change this to whatever you want, however.  If you changed the service name to
-*"MailcowBackup*.conf", for example, you would have to rename this script file
-to "*MailcowBackup*" with no extension.\
-*Note: The script is a PERL file (note the shebang) but it can be written in any
-language.*
-
-**In essence, Logwatch just spits out the log file(s) defined in the LogFile
-Group file as standard input (STDIN) for the script and then takes whatever is
-output (STDOUT) from the script to assemble into it's report.**
-
-### Detail levels
-
-The script supports four (4) detail levels as follows:
-
- **Level 0: Summary output only**
-  - This will display an aggregate total of certain logged elements.  It will
-    display the total number of overall successful script executions, total
-    generated warnings and total errors encountered that stopped the normal
-    execution of the script.  All totals are relative to the reporting period
-    Logwatch is using (--range parameter).
-
-    **This is the recommended reporting level.**  It does not take up much space
-    and is quick to read.  If you notice warnings and/or errors, you should
-    consult the full log.
- **Levels 1-4 (all the same): Critical messages**
-  - This uses the data which is summarized by Level 0 but outputs the actual
-    messages in the log file.  For example, you will see the actual text of the
-    errors logged instead of just a total number of errors.  This level of
-    reporting is useful when *initially* monitoring the script's operation since
-    you can see the actual text of any generated warnings/errors.
-
- **Level 5: Verbose (debugging) output**
-  - Like the previous level, this outputs the actual messages found in the log
-    file.  However, it also includes *[INFO] tags* which contain logged
-    operational messages such as created temporary directories,
-    starting/stopping docker containers, whether the 503 page was copied, etc.
-    This level of reporting is useful in diagnosing why errors are occurring or
-    if you just want more insight into how the script works.
-
-    **This level of output will make your Logwatch reports longer and consume
-    more of your time to review. You should not use this level day-to-day.**
- **Levels 6+ (all the same): Complete log file dump**
-  - Any number greater than 5 passed as a detail level will trigger the script
-    to dump the entire log file out to Logwatch line-by-line.  This is useful
-    only if you are debugging an issue and cannot get access to the actual raw
-    log file itself.  The actual log file is colour-coded which makes it much
-    easier to read for debugging purposes.
-
-    **Use this detail level only when you need to see the entire log file and
-    cannot otherwise access the log file.**
-
-## Timestamp processing script (/etc/logwatch/scripts/shared/sqfullstampanywhere)
-
-This is basically a modified version of the '*applyeurodate*' script that comes
-with Logwatch.  It had to be modified to search within [square brackets] and to
-accept characters coming before the stamp (i.e. ANSI colour codes).  If you
-change the '**stamp**' variable in the backup script to update the timestamp to
-your liking (which to totally fine!) then you'll probably have to update this
-file.  There are two lines you need to modify to suit your new '**stamp**'
-variable.
-
-### The time format specification
-
-'*$SearchDate*' is the variable used in the PERL script to do exactly what it
-says, search for the date stamp.  I have it set up to look for the format
-'*year-month-date hour:minute:second*'.  Note, we don't care about brackets or
-anything here, we're just defining the format of the date/time stamp.
-
-```Perl
-...
-$SearchDate = TimeFilter('%Y-%m-%d %H:%M:%S');
-...
-```
-
-If you changed the '**stamp**' variable so it was formatted as '*month/day/year
-hour:minute*' (ex: '*[09/27/2018 18:38]*') then you'd update the **$SearchDate**
-variable as follows (note: no mention of the square brackets!):
-
-```Perl
-...
-$SearchDate = TimeFilter('%m/%d/%Y %H:%M');
-...
-```
-
-### The search REGEX
-
-The PERL script uses a '*regular expression*' (REGEX) to search within the log file for
-'*$SearchDate*'.  For the default datestamp, this specification looks like:
-
-```Perl
-...
-if ($ThisLine =~ m/\[$SearchDate\] /o) {
-...
-```
-
-The REGEX appears between '*m/*' and '*/o*'.  In this case, it searches for
-'*$SearchDate*' inside [square brackets] appearing anywhere on the line.  This
-is because ANSI colour-codes often appear before the datestamp in the default
-log file.  If you have modified this so that your datestamp appears at the
-beginning of the line and in the example format in the section above (using
-slashes instead of dashes) then you'd rewrite this REGEX as follows:
-
-```Perl
-...
-if ($ThisLine =~ m/^\[$SearchDate\] /o) {
-...
-```
-
-or using regular brackets anywhere on the line:
-
-```Perl
-...
-if ($ThisLine =~ m/\($SearchDate\) /o) {
-...
-```
-
-or without any brackets but appearing at the beginning of the line:
-
-```Perl
-...
-if ($ThisLine =~ m/^$SearchDate /o) {
-...
-```
-
-## Testing
-
-Run *logwatch --help* and note the options.  You can test just this service
-locally on your screen with the following command (assuming you kept default
-names for everything):
-
-```Bash
-# Summary output, entire duration of log file
-logwatch --service backup --output stdout --format text --range all --detail 0
-
-# Minimal detail, yesterday only
-logwatch --service backup --output stdout --format text --range yesterday --detail 3
-
-# Verbose output, today only
-logwatch --service backup --output stdout --format text --range today --detail 5
-```
-
-## Final thoughts
-
-That's it!  I'm a horrible PERL programmer so if anyone can optimize/improve the
-script file used for Logwatch then please do it!  Otherwise, I hope this made
-sense and helped you integrate the backup script with Logwatch for easy
-monitoring :-)
+I hope this helps you get your mailcow backup integrated with logwatch easily and quickly. If you have any suggestions/improvements, drop me a line in the issues section!
@@ -1,881 +0,0 @@
-#!/bin/bash
-
-
-### Text formatting presets
-normal="\e[0m"
-bold="\e[1m"
-default="\e[39m"
-err="\e[1;31m"
-warn="\e[1;93m"
-ok="\e[32m"
-lit="\e[93m"
-op="\e[39m"
-info="\e[96m"
-note="\e[95m"
-
-
-### Functions ###
-
-### scriptHelp -- display usage information for this script
-function scriptHelp {
-    echo -e "${bold}${note}\n${scriptName} usage instructions:\n${normal}"
-    echo -e "${default}This script performs a backup of your Mailcow system"
-    echo -e "assuming a fairly standard set up such as outlined at"
-    echo -e "${lit}https://mytechiethoughts.com${default}.  Full details about"
-    echo -e "this script can be found at that site."
-    echo -e "${bold}\nThe script performs the following tasks:${normal}${default}"
-    echo -e "1. Stops mailflow (postfix & dovecot) containers."
-    echo -e "2. Optionally copies a 503 error page to your webroot."
-    echo -e "3. Dumps Mailcow's SQL DB to a temporary directory."
-    echo -e "4. Invokes borgbackup to backup your SQL info, Mailcow settings"
-    echo -e "\tand (raw) data files along with any other files you specify."
-    echo -e "5. Removes temp files, the 503 error page and restores"
-    echo -e "\tMailcow to operational status."
-    echo -e "\nThe readme file included in this script's git contains detailed"
-    echo -e "usage information. The following is a brief summary:\n"
-    echo -e "${bold}***You MUST provide the full path to your mailcow"
-    echo -e "configuration file using the '-m' parameter***${normal}${default}"
-    echo -e "${bold}${note}\nMandatory parameters:${normal}${default}"
-    echo -e "${lit}\n-m, File name of the Mailcow build configuration file${default}"
-    echo -e "FULL PATH to your Mailcow master build configuration file containing"
-    echo -e "all variables and configuration info unique to your Mailcow setup."
-    echo -e "The path specified here is also used for all docker-related"
-    echo -e "operations in this script."
-    echo -e "${info}Default: <none>${default}"
-    echo -e "${bold}${note}\nOptional parameters:${normal}${default}"
-    echo -e "${lit}\n-1, Timeout for containers to STOP before error${default}"
-    echo -e "The number of seconds to wait for a docker container to STOP"
-    echo -e "before aborting the procedure and exiting this script with an"
-    echo -e "error."
-    echo -e "${info}Default: 120 seconds${default}"
-    echo -e "${lit}\n-2, Timeout for containers to START before error${default}"
-    echo -e "The number of seconds to wait for a docker container to START"
-    echo -e "before aborting the procedure and exiting this script with an"
-    echo -e "error."
-    echo -e "${info}Default: 180 seconds${default}"
-    echo -e "${lit}\n-5, Location of 503 error page file${default}"
-    echo -e "FULL PATH to the 503 error page HTML file you want copied to your"
-    echo -e "webroot to inform users the server is down during the backup. If"
-    echo -e "you don't specify a path/file, the default will be used. If the"
-    echo -e "default cannot be found, a warning will be logged and the script"
-    echo -e "will continue."
-    echo -e "${info}Default: ScriptPath/503.html${default}"
-    echo -e "${lit}\n-b, Location of file with borg repo details${default}"
-    echo -e "FULL PATH to the plain text file containing all information needed"
-    echo -e "to connect and process your borg repo. Details on the structure of"
-    echo -e "this file are in the readme and on ${lit}https://mytechiethoughts.com${default}"
-    echo -e "${info}Default: ScriptPath/nc_borg.details${default}"
-    echo -e "${lit}\n-d, File name of the docker-compose configuration file${default}"
-    echo -e "Name of the docker-compose configuration file that Mailcow uses"
-    echo -e "to build/start/stop all containers.  This will only be searched"
-    echo -e "for in the path provided for your mailcow configuration file."
-    echo -e "${info}Default: docker-compose.yml${default}"
-    echo -e "${lit}\n-l, Location to save log file${default}"
-    echo -e "This script writes a detailed log file of all activities.  It is"
-    echo -e "structured in an way easy for log parsers (like Logwatch) to read."
-    echo -e "${info}Default: ScriptPath/ScriptName.log${default}"
-    echo -e "${lit}\n-v, Verbose output from borgbackup${default}"
-    echo -e "By default, this script will only log summary data from borg."
-    echo -e "If you need/want more detailed information, the verbose setting"
-    echo -e "will list every file processed along with their status. Note: Your"
-    echo -e "log file can quickly get very very large using this option!"
-    echo -e "${info}Default: NOT activated (standard logging)${default}"
-    echo -e "${lit}\n-w, webserver's webroot directory${default}"
-    echo -e "This is the location from which your webserver (NGINX, Apache,"
-    echo -e "etc.) physically stores files to be served.  This is NOT the"
-    echo -e "configuration directory for your webserver!  It is the place"
-    echo -e "where the actual HTML/PHP/CSS/JS/etc. files are stored."
-    echo -e "NOTE: If you omit this option, then the entire 503 copy process"
-    echo -e "will be skipped regardless of the presence of a 503.html file."
-    echo -e "If you don't want to use the 503 feature, omitting this is an easy"
-    echo -e "way to skip it!"
-    echo -e "${info}Default: NONE${default}"
-    echo -e "${lit}\n-?, This help screen${default}\n"
-    echo -e "${bold}Please refer to the readme file and/or ${lit}https://mytechiethoughts.com${default}"
-    echo -e "for more information on this script.${normal}\n"
-    # exit with code 1 -- there is no use logging this
-    exit 1
-}
-
-### Generate dynamic timestamp
-function stamp {
-    echo `date +%F" "%T`
-}
-
-### quit -- exit the script after logging any errors, warnings, etc.
-function quit {
-    # list generated warnings, if any
-    if [ ${#exitWarn[@]} -gt 0 ]; then
-        echo -e "\n${warn}${scriptName} generated the following warnings:" \
-            "${normal}" >> "$logFile"
-        for warnCode in "${exitWarn[@]}"; do
-            warnStamp="${warnCode%%_*}"
-            warnValue="${warnCode##*_}"
-            echo -e "${warn}${warnStamp} -- [WARNING]" \
-                "${warningExplain[$warnValue]} (code: ${warnValue}) --" \
-                "${normal}" >> "$logFile"
-        done
-    fi
-    if [ -z "${exitError}" ]; then
-        # exit cleanly
-        echo -e "${note}[$(stamp)] -- ${scriptName} completed" \
-            "--${normal}" >> "$logFile"
-        exit 0
-    else
-        # list generated errors and explanations then exit script with code 2
-        echo -e "\n${err}${scriptName} generated the following errors:" \
-            "${normal}" >> "$logFile"
-        for errCode in "${exitError[@]}"; do
-            errStamp="${errCode%%_*}"
-            errValue="${errCode##*_}"
-            echo -e "${err}${errStamp} -- [ERROR] ${errorExplain[$errValue]}" \
-                "(code: ${errValue}) --${normal}" >> "$logFile"
-        done
-        exit 2
-    fi
-}
-
-function checkExist {
-    if [ "$1" = "ff" ]; then
-        # find file
-        if [ -f "$2" ]; then
-            # found
-            return 0
-        else
-            # not found
-            return 1
-        fi
-    elif [ "$1" = "fs" ]; then
-        # find file > 0 bytes
-        if [ -s "$2" ]; then
-            # found
-            return 0
-        else
-            # not found
-            return 1
-        fi
-    elif [ "$1" = "fd" ]; then
-        # find directory
-        if [ -d "$2" ]; then
-            # found
-            return 0
-        else
-            # not found
-            return 1
-        fi
-    fi
-}
-
-### cleanup - cleanup files and directories created by this script
-function cleanup {
-    ## remove SQL dump file and directory
-    rm -rf "$sqlDumpDir" >> "$logFile" 2>&1
-    # verify directory is gone
-    checkExist fd "$sqlDumpDir"
-    checkResult="$?"
-    if [ "$checkResult" = "0" ]; then
-        # directory still exists
-        exitWarn+=("[$(stamp)]_111")
-    else
-        # directory removed
-        echo -e "${op}[$(stamp)] Removed SQL temp directory${normal}" \
-            >> "$logFile"
-    fi
-
-    ## remove 503 error page
-    # check value of 'clean503' to see if this is necessary (=1) otherwise, skip
-    if [ "$clean503" = "1" ]; then
-        # proceed with cleanup
-        echo -e "${op}[$(stamp)] Removing 503 error page..." >> "$logFile"
-        rm -f "$webroot/$err503File" >> "$logFile" 2>&1
-        # verify file is actually gone
-        checkExist ff "$webroot/$err503File"
-        checkResult="$?"
-        if [ "$checkResult" = "0" ]; then
-            # file still exists
-            exitWarn+=("[$(stamp)]_5030")
-        else
-            # file removed
-            echo -e "${info}[$(stamp)] -- [INFO] 503 page removed from webroot" \
-                "--${normal}" >> "$logFile"
-        fi
-    else
-        echo -e "${op}[$(stamp)] 503 error page never copied to webroot," \
-            "nothing to cleanup" >> "$logFile"
-    fi
-
-    ## restart mailflow docker containers
-    # start and verify postfix
-    operateDocker start postfix
-    if [ "$dockerResultState" = "true" ]; then
-        echo -e "${info}[$(stamp)] -- [INFO] Postfix container is running --" \
-            "${normal}" >> "$logFile"
-    else
-        exitError+=("[$(stamp)]_103")
-    fi
-    # start and verify dovecot
-    operateDocker start dovecot
-    if [ "$dockerResultState" = "true" ]; then
-        echo -e "${info}[$(stamp)] -- [INFO] Dovecot container is running --" \
-            "${normal}" >> "$logFile"
-    else
-        exitError+=("[$(stamp)]_104")
-    fi
-}
-
-### operate docker containers
-function operateDocker {
-    containerName="$(docker ps -a --format '{{ .Names }}' --filter name=${COMPOSE_PROJECT_NAME}_${2}-mailcow_1)"
-
-    # determine action to take
-    if [ "$1" = "stop" ]; then
-        echo -e "${op}[$(stamp)] Stopping ${2}-mailcow container...${normal}" \
-            >> "$logFile"
-        docker-compose stop --timeout ${dockerStopTimeout} ${2}-mailcow \
-            2>> "$logFile"
-        # verify container stopped (should return true)
-        dockerResultState="$(docker inspect -f '{{ .State.Running }}' $containerName)"
-        # verify clean stop (exit code 0)
-        dockerResultExit="$(docker inspect -f '{{ .State.ExitCode }}' $containerName)"
-    elif [ "$1" = "start" ]; then
-        echo -e "${op}[$(stamp)] Starting ${2}-mailcow container...${normal}" \
-            >> "$logFile"
-        docker-compose start ${2}-mailcow 2>> "$logFile"
-        # verify
-        dockerResultState="$(docker inspect -f '{{ .State.Running }}' $containerName)"
-    fi
-}
-
-### End of Functions ###
-
-
-### Default parameters
-
-# store the logfile in the same directory as this script using the script's name
-# with the extension .log
-scriptPath="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
-scriptName="$( basename ${0} )"
-logFile="$scriptPath/${scriptName%.*}.log"
-
-# Set default docker-compose filename
-dockerComposeFile=docker-compose.yml
-
-# set default 503 error page name and location in scriptPath
-err503Path="$scriptPath/503.html"
-err503File="${err503Path##*/}"
-
-# Set default docker-compose stop timeout
-dockerStopTimeout=120
-
-# Set default docker-compose start timeout
-dockerStartTimeout=180
-
-# set default borgDetails path to scriptPath
-borgDetails="$scriptPath/mc_borg.details"
-
-# set borg parameters to 'normal' verbosity
-borgCreateParams='--stats'
-borgPruneParams='--list'
-
-
-### Set script parameters to null and initialize array variables
-unset mailcowConfigFilePath
-unset mailcowPath
-unset dockerComposeFilePath
-unset PARAMS
-unset sqlDumpDir
-unset webroot
-unset clean503
-unset borgXtra
-unset borgExclude
-unset borgPrune
-unset BORG_BASE_DIR
-unset BORG_RSH
-unset BORG_REPO
-unset BORG_PASSPHRASE
-unset BORG_REMOTE_PATH
-unset TMPDIR
-unset dockerVolumeMail
-unset dockerVolumeRspamd
-unset dockerVolumePostfix
-unset dockerVolumeRedis
-unset dockerVolumeCrypt
-exitError=()
-errorExplain=()
-exitWarn=()
-warningExplain=()
-borgConfig=()
-xtraFiles=()
-
-### Error codes
-errorExplain[101]="Could not stop Postfix container. Please check docker logs"
-errorExplain[102]="Could not stop Dovecot container. Please check docker logs"
-errorExplain[103]="Could not start Postfix container. Please check docker logs"
-errorExplain[104]="Could not start Dovecot container. Please check docker logs"
-errorExplain[201]="There was a problem dumping the SQL database. It has NOT been backed up"
-errorExplain[202]="There was a problem saving redis state information. It has NOT been backed up"
-errorExplain[210]="Invalid or non-existant borg base directory specified (borg backup details file)"
-errorExplain[211]="Invalid or non-existant path to borg SSH keyfile (borg backup details file)"
-errorExplain[212]="Name of borg repo was not specified (borg backup details file)"
-errorExplain[215]="Could not find/create 'tmp' directory within borg base directory. Please manually create it and ensure it's writable"
-errorExplain[220]="Borg exited with a critical error. Please check this script's logfile for details"
-errorExplain[221]="Borg prune exited with ERRORS. Please check this script's logfile for details"
-
-
-### Warning codes & messages
-warningExplain[111]="Could not remove SQL dump file and directory, please remove manually"
-warningExplain[5030]="Could not remove 503 error page. This MUST be removed manually before NGINX will serve webclients!"
-warningExplain[5031]="No webroot path was specified (-w parameter missing)"
-warningExplain[5032]="The specified webroot (-w parameter) could not be found"
-warningExplain[5033]="No 503 error page could be found. If not using the default located in the script directory, then check your -5 parameter"
-warningExplain[5035]="Error copying 503 error page to webroot"
-warn503="Web users will NOT be informed the server is down!"
-warningExplain[2111]="No password used for SSH keys or access to remote borg repo. This is an insecure configuration"
-warningExplain[2112]="No remote borg instance specified. Operations will be slower in this configuration"
-warningExplain[2113]="The specified file containing extra files for inclusion in borgbackup could not be found"
-warningExplain[2114]="The specified file containing exclusion patterns for borgbackup could not be found. Backup was performed as though NO exclusions were defined"
-warningExplain[2115]="No paramters provided for borg prune. No repo pruning has taken place. You should reconsider this decision to control the size/history of your backups"
-warningExplain[2116]="No additional locations are specified for inclusion in backup. ONLY Mailcow data and config files will be backed up (NO system files, etc). If this is unintentional, check the inclusion file referenced in your borgbackup settings"
-warningExplain[2200]="Borg completed with warnings. Please check this script's logfile for details"
-warningExplain[2201]="Borg exited with an unknown return-code. Please check this script's logfile for details"
-warningExplain[2210]="Borg prune exited with warnings. Please check this script's logfile for details"
-warningExplain[2212]="Borg prune exited with an unknown return-code. Please check this script's logfile for details"
-
-
-### Process script parameters
-
-# If parameters are provided but don't start with '-' then show the help page
-# and exit with an error
-if [ -n "$1" ] && [[ ! "$1" =~ ^- ]]; then
-    # show script help page
-    scriptHelp
-fi
-
-# use GetOpts to process parameters
-while getopts ':l:v5:w:b:m:d:1:2:' PARAMS; do
-    case "$PARAMS" in
-        l)
-            # use provided location for logFile
-            logFile="${OPTARG%/}"
-            ;;
-        v)
-            # verbose output from Borg
-            borgCreateParams='--list --stats'
-            borgPruneParams='--list'
-            ;;
-        5)
-            # Full path to 503 error page
-            err503Path="${OPTARG%/}"
-            err503File="${err503Path##*/}"
-            ;;
-        w)
-            # path to webserver webroot to copy 503 error page
-            webroot="${OPTARG%/}"
-            ;;
-        b)
-            # path to file containing borgbackup settings and details
-            borgDetails="${OPTARG%/}"
-            ;;
-        m)
-            # full path to mailcow.conf configuration file
-            mailcowConfigFilePath="${OPTARG%/}"
-            ;;
-        d)
-            # name of docker-compose configuration file
-            dockerComposeFile="${OPTARG}"
-            ;;
-        1)
-            # docker-compose stop timeout in seconds
-            dockerStopTimeout="${OPTARG}"
-            ;;
-        2)
-            # docker-compose start timeout in seconds
-            dockerStartTimeout="${OPTARG}"
-            ;;
-        ?)
-            # unrecognized parameters trigger scriptHelp
-            scriptHelp
-            ;;
-    esac
-done
-
-
-### Verify script pre-requisties
-
-## If not running as root, display error on console and exit
-if [ $(id -u) -ne 0 ]; then
-    echo -e "\n${err}This script MUST be run as ROOT. Exiting.${normal}"
-    exit 3
-fi
-
-## verify mailcow.conf location provided
-if [ -z "$mailcowConfigFilePath" ]; then
-    echo -e "\n${err}You MUST provide the full path to your mailcow.conf" \
-        "configuration file. Exiting.${normal}"
-    exit 1
-fi
-
-## verify mailcow.conf and extract path
-if checkExist ff "$mailcowConfigFilePath"; then
-    # extract directory name
-    case $mailcowConfigFilePath in
-        */*)
-            mailcowPath=${mailcowConfigFilePath%/*}
-            ;;
-        *)
-            mailcowPath="."
-            ;;
-    esac
-else
-    echo -e "\n${err}Could not locate the specified mailcow configuration" \
-        "file: ${lit}${mailcowConfigFilePath}${normal}"
-    exit 1
-fi
-
-# verify docker-compose.yml exists at location of mailcow.conf (standard setup)
-dockerComposeFilePath="${mailcowPath}/${dockerComposeFile}"
-if ! checkExist ff "$dockerComposeFilePath"; then
-    echo -e "\n${err}Could not locate docker-compose configuration file:" \
-        "${lit}${dockerComposeFilePath}${normal}"
-    exit 1
-fi
-
-## Ensure borgDetails file exists
-checkExist ff "$borgDetails"
-checkResult="$?"
-if [ "$checkResult" = "1" ]; then
-    # sqlDetails file cannot be found
-    echo -e "\n${err}The file containing your borgbackup details does not" \
-        "exist (-b parameter)${normal}\n"
-    exit 1
-fi
-
-
-### Log start of script operations
-echo -e "${note}[$(stamp)] --- Start $scriptName execution ---${normal}" \
-    >> "$logFile"
-echo -e "${info}[$(stamp)] -- [INFO] using ${lit}${mailcowConfigFilePath}" \
-    >> "$logFile"
-echo -e "${info}[$(stamp)] -- [INFO] using ${lit}${dockerComposeFilePath}" \
-    >> "$logFile"
-
-
-### Import additional variables from mailcow configuration file
-source "$mailcowConfigFilePath"
-
-### Export PATH so this script can access all docker and docker-compose commands
-export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
-
-### Export logFile variable for use by Borg
-export logFile="$logFile"
-
-### Export docker container startup timeout variable
-export COMPOSE_HTTP_TIMEOUT=${dockerStartTimeout}
-
-## Get docker volume paths on filesystem
-dockerVolumeMail=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_vmail-vol-1)
-dockerVolumeRspamd=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_rspamd-vol-1)
-dockerVolumePostfix=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_postfix-vol-1)
-dockerVolumeRedis=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_redis-vol-1)
-dockerVolumeCrypt=$(docker volume inspect -f '{{ .Mountpoint }}' ${COMPOSE_PROJECT_NAME}_crypt-vol-1)
-
-
-### Create sqlDump temporary directory and sqlDumpFile name
-sqlDumpDir=$( mktemp -d )
-sqlDumpFile="backup-`date +%Y%m%d_%H%M%S`.sql"
-echo -e "${info}[$(stamp)] -- [INFO] mySQL dump file will be stored" \
-    "at: ${lit}${sqlDumpDir}/${sqlDumpFile}${normal}" >> "$logFile"
-
-
-### 503 error page: If you dont' plan on using the auto-copied 503 then comment
-### this entire section starting with '--- Begin 503 section ---' until
-### '--- End 503 section ---' to suppress generated warnings
-
-### --- Begin 503 section ---
-
-## Check if webroot has been specified, if not, skip this entire section since there is nowhere to copy the 503 file.
-if [ -z "$webroot" ]; then
-    # no webroot path provided
-    echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --${normal}" \
-        >> "$logFile"
-    exitWarn+=("[$(stamp)]_5031")
-    clean503=0
-else
-    # verify webroot actually exists
-    checkExist fd "$webroot"
-    checkResult="$?"
-    if [ "$checkResult" = "1" ]; then
-        # webroot directory specified could not be found
-        echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --${normal}" \
-            >> "$logFile"
-        exitWarn+=("[$(stamp)]_5032")
-        clean503=0
-    else
-        # webroot exists
-        echo -e "${op}[$(stamp)] Using webroot: ${lit}${webroot}${normal}" \
-            >> "$logFile"
-        # Verify 503 file existance at given path
-        checkExist ff "$err503Path"
-        checkResult="$?"
-        if [ "$checkResult" = "1" ]; then
-            # 503 file could not be found
-            echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --${normal}" \
-                >> "$logFile"
-            exitWarn+=("[$(stamp)]_5033")
-            clean503=0
-        else
-            # 503 file exists and webroot is valid. Let's copy it!
-            echo -e "${op}[$(stamp)] ${err503File} found at ${lit}${err503Path}" \
-                "${normal}" >> "$logFile"
-            echo -e "${op}[$(stamp)] Copying 503 error page to webroot..." \
-                "${normal}" >> "$logFile"
-            cp "${err503Path}" "$webroot/" >> "$logFile" 2>&1
-            copyResult="$?"
-            # verify copy was successful
-                if [ "$copyResult" = "1" ]; then
-                    # copy was unsuccessful
-                    echo -e "${info}[$(stamp)] -- [INFO] ${warn503} --${normal}" \
-                        >> "$logFile"
-                    exitWarn+=("[$(stamp)]_5035")
-                    clean503=0
-                else
-                # copy was successful
-                echo -e "${info}[$(stamp)] -- [INFO] 503 error page" \
-                    "successfully copied to webroot --${normal}" >> "$logFile"
-                clean503=1
-                fi
-        fi
-    fi
-fi
-
-### --- End 503 section ---
-
-
-### Change directory to mailcowPath
-cd "$mailcowPath"
-
-
-### Stop postfix and dovecot so mailflow is stopped until backup is completed
-## Stop postfix-mailcow container
-operateDocker stop postfix
-# process result
-if [ "$dockerResultState" = "false" ] && [ "$dockerResultExit" -eq 0 ]; then
-    echo -e "${info}[$(stamp)] -- [INFO] Postfix container stopped --${normal}" \
-        >> "$logFile"
-else
-    exitError+=("[$(stamp)]_101")
-    cleanup
-    quit
-fi
-## Stop dovecot-mailcow container
-operateDocker stop dovecot
-# process result
-if [ "$dockerResultState" = "false" ] && [ "$dockerResultExit" -eq 0 ]; then
-    echo -e "${info}[$(stamp)] -- [INFO] Dovecot container stopped --${normal}" \
-        >> "$logFile"
-else
-    exitError+=("[$(stamp)]_102")
-    cleanup
-    quit
-fi
-
-
-### Dump SQL
-echo -e "${op}[$(stamp)] Dumping mailcow SQL database...${normal}" >> "$logFile"
-docker-compose exec -T mysql-mailcow mysqldump --default-character-set=utf8mb4 \
-    -u${DBUSER} -p${DBPASS} ${DBNAME} > "$sqlDumpDir/$sqlDumpFile" \
-    2>> "$logFile"
-dumpResult=$(docker-compose exec -T mysql-mailcow echo "$?")
-## very mysqldump completed successfully
-if [ "$dumpResult" = "0" ]; then
-    echo -e "${info}[$(stamp)] -- [INFO] mySQLdump completed successfully --" \
-        "${normal}" >> "$logFile"
-else
-    exitError+=("[$(stamp)]_201")
-fi
-## verify the dump file was actually written to disk
-checkExist fs "$sqlDumpDir/$sqlDumpFile"
-    checkResult="$?"
-    if [ "$checkResult" = "0" ]; then
-        echo -e "${ok}[$(stamp)] -- [SUCCESS] SQL successfully dumped --" \
-            "${normal}" >> "$logFile"
-    else
-        exitError+=("[$(stamp)]_201")
-    fi
-
-### Save redis state
-## Delete any existing redis dump file otherwise our file check will be useless
-echo -e "${op}[$(stamp)] Cleaning up old redis state backup...${normal}" \
-    >> "$logFile"
-checkExist ff "$dockerVolumeRedis/dump.rdb"
-    checkResult="$?"
-    if [ "$checkResult" = "0" ]; then
-        echo -e "${lit}[$(stamp)] Old redis backup found. ${op}Deleting..." \
-            "${normal}" >> "$logFile"
-        rm -f "$dockerVolumeRedis/dump.rdb" 2>> "$logFile"
-        echo -e "${op}[$(stamp)] ...done${normal}" >> "$logFile"
-    else
-        echo -e "${op}[$(stamp)] No old redis backup found${normal}" \
-            >> "$logFile"
-    fi
-## Export redis
-echo -e "${op}[$(stamp)] Saving redis state information...${normal}" >> "$logFile"
-docker-compose exec -T redis-mailcow redis-cli save >> "$logFile" 2>&1
-saveResult=$(docker-compose exec -T redis-mailcow echo "$?")
-# verify save operation completed successfully
-if [ "$saveResult" = "0" ]; then
-    echo -e "${info}[$(stamp)] -- [INFO] redis save-state successful --" \
-        "${normal}" >> "$logFile"
-else
-    exitError+=("[$(stamp)]_202")
-fi
-## verify save-file written to disk
-checkExist fs "$dockerVolumeRedis/dump.rdb"
-    checkResult="$?"
-    if [ "$checkResult" = "0" ]; then
-        echo -e "${ok}[$(stamp)] -- [SUCCESS] redis state saved --${normal}" \
-        >> "$logFile"
-    else
-        exitError+=("[$(stamp)]_202")
-    fi
-
-
-### Call borgbackup to copy actual files
-echo -e "${op}[$(stamp)] Pre-backup tasks completed, calling borgbackup..." \
-    "${normal}" >> "$logFile"
-
-## Get borgbackup settings and repo details
-# read definition file and map to array variable
-mapfile -t borgConfig < "$borgDetails"
-## check if any required borg configuration variables in defintion file are
-## empty and exit with error, otherwise, map array items to variables
-# check: borg base directory
-echo -e "${op}[$(stamp)] Verifying supplied borg configuration variables..." \
-    "${normal}" >> "$logFile"
-if [ -z "${borgConfig[0]}" ]; then
-    exitError+=("[$(stamp)]_210")
-    cleanup
-    quit
-else
-    # verify the path actually exists
-    checkExist fd "${borgConfig[0]}"
-    checkResult="$?"
-    if [ "$checkResult" = "1" ]; then
-        # borg base directory specified could not be found
-        exitError+=("[$(stamp)]_210")
-        cleanup
-        quit
-    fi
-    echo -e "${op}[$(stamp)] Borg base dir... OK${normal}" >> "$logFile"
-    export BORG_BASE_DIR="${borgConfig[0]%/}"
-fi
-# check: path to SSH keyfile
-if [ -z "${borgConfig[1]}" ]; then
-    exitError+=("[$(stamp)]_211")
-    cleanup
-    quit
-else
-    checkExist ff "${borgConfig[1]}"
-    checkResult="$?"
-    if [ "$checkResult" = 1 ]; then
-        # SSH keyfile specified could not be found
-        exitError+=("[$(stamp)]_211")
-        cleanup
-        quit
-    fi
-    echo -e "${op}[$(stamp)] Borg SSH key... OK${normal}" >> "$logFile"
-    export BORG_RSH="ssh -i ${borgConfig[1]}"
-fi
-# check: name of borg repo
-if [ -z "${borgConfig[2]}" ]; then
-    exitError+=("[$(stamp)]_212")
-    cleanup
-    quit
-else
-    echo -e "${op}[$(stamp)] Borg REPO name... OK${normal}" >> "$logFile"
-    export BORG_REPO="${borgConfig[2]}"
-fi
-# repo password
-if [ -n "${borgConfig[3]}" ]; then
-    echo -e "${op}[$(stamp)] Borg SSH/REPO password... OK${normal}" >> "$logFile"
-    export BORG_PASSPHRASE="${borgConfig[3]}"
-else
-    exitWarn+=("[$(stamp)]_2111")
-    # if the password was omitted by mistake, export a dummy password so borg
-    # fails with an error instead of sitting and waiting for input
-    export BORG_PASSPHRASE="DummyPasswordSoBorgFails"
-fi
-# additional files to be backed up
-borgXtra="${borgConfig[4]}"
-# file with pattern definition for excluded files
-borgExclude="${borgConfig[5]}"
-# parameters for borg prune
-borgPrune="${borgConfig[6]}"
-# export: borg remote path (if not blank)
-if [ -n "${borgConfig[7]}" ]; then
-    echo -e "${op}[$(stamp)] Borg REMOTE path... OK${normal}" >> "$logFile"
-    export BORG_REMOTE_PATH="${borgConfig[7]}"
-else
-    exitWarn+=("[$(stamp)]_2112")
-fi
-
-## If borgXtra exists, map contents to an array variable
-if [ -n "$borgXtra" ]; then
-    echo -e "${op}[$(stamp)] Processing referenced extra files list for" \
-        "borgbackup to include in backup${normal}" >> "$logFile"
-    checkExist ff "$borgXtra"
-    checkResult="$?"
-    if [ "$checkResult" = "0" ]; then
-        echo -e "${op}[$(stamp)] Found ${lit}${borgXtra}${normal}" >> "$logFile"
-        mapfile -t xtraFiles < "$borgXtra"
-        echo -e "${op}[$(stamp)] Processed extra files list for inclusion in" \
-            "borgbackup${normal}" >> "$logFile"
-    else
-        exitWarn+=("[$(stamp)]_2113")
-    fi
-else
-    # no extra locations specified
-    echo -e "${op}[$(stamp)] No additional locations specified for backup." \
-        "Only Mailcow data and config files will be backed up.${normal}" \
-            >> "$logFile"
-    exitWarn+=("[$(stamp)]_2116")
-fi
-
-## Check if borgExclude exists since borg will throw an error if it's missing
-if [ -n "$borgExclude" ]; then
-    checkExist ff "$borgExclude"
-    checkResult="$?"
-    if [ "$checkResult" = "0" ]; then
-            echo -e "${op}[$(stamp)] Found ${lit}${borgExclude}${normal}" \
-                >> "$logFile"
-    else
-        # file not found, unset the variable so it's like it was not specified
-        # in the first place and continue with backup
-        unset borgExclude
-        exitWarn+=("[$(stamp)]_2114")
-    fi
-else
-    echo -e "${op}[$(stamp)] Exclusion pattern file not specified." \
-        "No exclusions will be processed${normal}" >> "$logFile"
-fi
-
-
-## Export TMPDIR environment variable for borg via python
-## Python requires a writable temporary directory when unpacking borg and
-## executing commands.  This defaults to /tmp but many systems mount /tmp with
-## the 'noexec' option for security.  Thus, we will use/create a 'tmp' folder
-## within the BORG_BASE_DIR and instruct python to use that instead of /tmp
-# check if BORG_BASE_DIR/tmp exists, if not, create it
-echo -e "${op}[$(stamp)] Checking for tmp directory at ${lit}${BORG_BASE_DIR}" \
-    "${normal}" >> "$logFile"
-checkExist fd "$BORG_BASE_DIR/tmp"
-checkResult="$?"
-if [ "$checkResult" = "1" ]; then
-    # folder not found
-    echo -e "${op}[$(stamp)] tmp folder not found... creating${lit}" \
-        "${BORG_BASE_DIR}/tmp${normal}" >> "$logFile"
-    mkdir "$BORG_BASE_DIR/tmp" 2>> "$logFile"
-    # verify folder created
-    checkExist fd "$BORG_BASE_DIR/tmp"
-    checkResult="$?"
-    if [ "$checkResult" = "0" ]; then
-        # folder exists
-        echo -e "${op}[$(stamp)] tmp folder created within borg base directory" \
-            "${normal}" >> "$logFile"
-    else
-        # problem creating folder and script will exit
-        exitError+=("[$(stamp)]_215")
-        cleanup
-        quit
-    fi
-else
-    # folder found
-    echo -e "${op}[$(stamp)] tmp folder found within borg base directory" \
-        "${normal}" >> "$logFile"
-fi
-# export TMPDIR environment variable
-export TMPDIR="${BORG_BASE_DIR}/tmp"
-
-
-## Generate and execute borg
-# commandline depends on whether borgExclude is empty or not
-if [ -z "$borgExclude" ]; then
-    # borgExclude is empty
-    echo -e "${bold}${op}[$(stamp)] Executing borg without exclusions${normal}" \
-        >> "$logFile"
-    borg --show-rc create ${borgCreateParams} ::`date +%Y-%m-%d_%H%M%S` \
-        "${xtraFiles[@]}" \
-        "${sqlDumpDir}" \
-        "${dockerVolumeMail}" "${dockerVolumeRspamd}" "${dockerVolumePostfix}" \
-        "${dockerVolumeRedis}" "${dockerVolumeCrypt}" \
-        2>> "$logFile"
-else
-    # borgExclude is not empty
-    echo -e "${bold}${op}[$(stamp)] Executing borg with exclusions${normal}" \
-        >> "$logFile"
-    borg --show-rc create ${borgCreateParams} --exclude-from "${borgExclude}" \
-        ::`date +%Y-%m-%d_%H%M%S` \
-        "${xtraFiles[@]}" \
-        "${sqlDumpDir}" \
-        "${dockerVolumeMail}" "${dockerVolumeRspamd}" "${dockerVolumePostfix}" \
-        "${dockerVolumeRedis}" "${dockerVolumeCrypt}" \
-        2>> "$logFile"
-fi
-
-## Check status of borg operation
-borgResult="$?"
-if [ "$borgResult" -eq 0 ]; then
-    echo -e "${ok}[$(stamp)] -- [SUCCESS] Borg backup completed successfully --" \
-        "${normal}" >> "$logFile"
-elif [ "$borgResult" -eq 1 ]; then
-    exitWarn+=("[$(stamp)]_2200")
-elif [ "$borgResult" -ge 2 ]; then
-    exitError+=("[$(stamp)]_220")
-    cleanup
-    quit
-else
-    exitWarn+=("[$(stamp)]_2201")
-fi
-
-## Generate and execute borg prune
-# command depends on whether or not parameters have been defined
-if [ -n "$borgPrune" ]; then
-    # parameters defined
-    echo -e "${bold}${op}[$(stamp)] Executing borg prune operation${normal}" \
-        >> "$logFile"
-    borg prune --show-rc -v ${borgPruneParams} ${borgPrune} \
-        2>> "$logFile"
-    # check return-status
-    pruneResult="$?"
-    if [ "$pruneResult" -eq 0 ]; then
-        echo -e "${ok}[$(stamp)] -- [SUCCESS] Borg prune completed successfully" \
-            "--${normal}" >> "$logFile"
-    elif [ "$pruneResult" -eq 1 ]; then
-        exitWarn+=("[$(stamp)]_2210")
-    elif [ "$pruneResult" -ge 2 ]; then
-        exitError+=("[$(stamp)]_221")
-    else
-        exitWarn+=("[$(stamp)]_2212")
-    fi
-else
-    # parameters not defined... skip pruning
-    exitWarn+=("[$(stamp)]_2115")
-fi
-
-
-### borgbackup completed
-echo -e "${op}[$(stamp)] Borgbackup completed... begin cleanup" \
-    "${normal}" >> "$logFile"
-
-
-### Exit script
-echo -e "${bold}${op}[$(stamp)] ***Normal exit process***${normal}" \
-    >> "$logFile"
-cleanup
-echo -e "${bold}${ok}[$(stamp)] -- [SUCCESS] All processes completed" \
-    "successfully --${normal}" >> "$logFile"
-quit
-
-# This code should not be executed since the 'quit' function should terminate
-# this script.  Therefore, exit with code 99 if we get to this point.
-exit 99
@@ -1,8 +0,0 @@
-<path to borgbackup base directory>          /var/borgbackup
-<path to SSH private key for remote server>  /var/borgbackup/sshPrivate.key
-<connection string to remote repo>           user@servername.tld:repoName/
-<password for repo>                          pAsSwOrd
-<path to file listing extra files>           /root/scripts/xtraLocations.borg
-<path to file with exclusions>               /root/scripts/excludeLocations.borg
-<purge timeframe options>                    --keep-within=7d --keep-daily=30 --keep-weekly=12 --keep-monthly=-1
-<location of borg remote instance>           borg1
@@ -1,27 +0,0 @@
-/etc/network/interfaces
-/etc/network/interfaces.d/
-/etc/systemd/timesyncd.conf
-/etc/profile
-/etc/bash.bashrc
-/etc/skel/
-/etc/nanorc
-/etc/msmtprc
-/etc/msmtp_aliases
-/etc/apt/sources.list
-/etc/apt/sources.list.d/
-/etc/apt/listchanges.conf
-/etc/apt/apt.conf.d/50unattended-upgrades
-/etc/apt/apt.conf.d/20auto-upgrades
-/etc/ssh/
-/etc/logwatch/
-/etc/letsencrypt/
-/etc/fail2ban/fail2ban.local
-/etc/fail2ban/jail.local
-/etc/fail2ban/action.d/ufw.conf
-/etc/fail2ban/filter.d/ufw-probe.conf
-/etc/fail2ban/jail.d/
-/root/.bashrc
-/root/scripts/
-/root/.ssh/
-/etc/nginx/
-/usr/share/nginx/html/
Author	SHA1	Message	Date
Asif Bacchus	8680f91572	update readme to reference wiki	2020-05-24 04:48:21 -06:00
Asif Bacchus	b6c05e4ccb	update logwatch readme to point to wiki	2020-05-24 03:55:20 -06:00
Asif Bacchus	7293a29a05	include mailcow.conf directory in backup	2020-05-07 04:59:37 -06:00
Asif Bacchus	8486c4b7d2	make xtraList optional	2020-05-07 04:58:26 -06:00
Asif Bacchus	f57841b69a	update in-script help	2020-05-07 01:43:53 -06:00
Asif Bacchus	4d407887d5	change default docker-compose path	2020-05-07 01:35:20 -06:00
Asif Bacchus	6f59804cfc	docker-compose and mailcow-config accept full path	2020-05-07 01:34:43 -06:00
Asif Bacchus	b5247a085e	switch to mailcow directory to allow docker cmds	2020-05-07 01:10:26 -06:00
Asif Bacchus	2432139f3d	simplify mcConfig existance check	2020-05-07 01:08:36 -06:00
Asif Bacchus	5f0eddf22a	add mailcow.conf location in docker-compose cmds	2020-05-07 01:00:14 -06:00
Asif Bacchus	09c1d903ba	move sql dump earlier in script Borg requires the default TMP dir be changed to avoid permissions issue. Moving earlier means SQL dumps to system default TMP location.	2020-05-07 00:58:27 -06:00
Asif Bacchus	51928d4327	fix error in default docker-compose location	2020-05-07 00:47:34 -06:00
Asif Bacchus	a7f506efb0	add mailcow as example xtra file location	2020-05-07 00:42:56 -06:00
Asif Bacchus	0993083a7b	mark script executable	2020-05-07 00:33:50 -06:00
Asif Bacchus	5448848c36	update default sample paths	2020-05-07 00:32:58 -06:00
Asif Bacchus	5667204378	remove old bash script version	2020-05-07 00:22:41 -06:00
Asif Bacchus	ca1186d855	reset bookmarks	2020-05-07 00:18:50 -06:00
Asif Bacchus	f8561a3aab	restart docker containers in cleanup function	2020-05-07 00:17:41 -06:00
Asif Bacchus	1b68616e54	fix wrong container name	2020-05-07 00:16:01 -06:00
Asif Bacchus	51120491d5	add mc locations to borgbackup cmd	2020-05-07 00:08:47 -06:00
Asif Bacchus	abdbb44922	dump redis db	2020-05-07 00:01:21 -06:00
Asif Bacchus	ca62d81368	dump sql	2020-05-06 23:54:31 -06:00
Asif Bacchus	72e396fa64	fix error in printf format string	2020-05-06 23:53:02 -06:00
Asif Bacchus	992f8ef16e	stop containers before backup starts	2020-05-06 23:38:26 -06:00
Asif Bacchus	9d7b7e1a94	add func doDocker	2020-05-06 23:32:53 -06:00
Asif Bacchus	4fb6315643	clean up function names for consistency	2020-05-06 23:18:01 -06:00
Asif Bacchus	d9c662bb45	move sql temp dir creation and 503-copy moved to after borg configuration checks to consoldate checks vs actions	2020-05-06 23:02:07 -06:00
Asif Bacchus	e9884a347f	note err code for sql temp dir creation fail	2020-05-06 22:59:35 -06:00
Asif Bacchus	8a5a87db12	add SQL directory to cleanup	2020-05-06 22:57:41 -06:00
Asif Bacchus	0d32aaf40a	change sql temp dir creation process	2020-05-06 22:56:52 -06:00
Asif Bacchus	bb952a3aba	get and report mc volumes, sql backup location	2020-05-06 22:30:58 -06:00
Asif Bacchus	dccf5e17d0	source and use mailcow.conf	2020-05-06 22:30:19 -06:00
Asif Bacchus	35ce452dec	fix var declaration	2020-05-06 22:30:03 -06:00
Asif Bacchus	604d3da07d	add mc params, defaults and pre-req checks	2020-05-06 22:15:55 -06:00
Asif Bacchus	a5bc4549e9	basic import from generic backup script	2020-05-06 21:29:48 -06:00