asif/MailcowBackup
1
0
Fork 0
Code Issues Releases 10 Wiki Activity

add restore documentation

Asif Bacchus 2021-02-10 07:48:19 -07:00
parent b035a2f045
commit 8e0cbe307c
6 changed files with 340 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
8.0-Restore-overview.md Normal file

@ -0,0 +1,38 @@
# Restoring Backups
Restoring mailcow backups made by the backup script in this git repo is very much like doing a migration between servers. The notable exception is that the SQL database is restored from a dump file instead of straight-copied. The majority of these instructions will discuss a semi-automated restore using the new `restore.sh` script. If you want to do a fully manual restore, please refer to [page 8.3.1](https://git.asifbacchus.app/asif/MailcowBackup/wiki/8.3.1-Restore-manual-restore).
## Overview
First off, **please remember to run everything as root!** This will allow borg to restore files with proper permissions, ensure that docker doesn’t run into any problems and generally make your life easier.
The basic restore process is as follows:
1. Download (‘extract’) the borg backup archive you want to restore from your remote repository to a local directory.
2. Basic first-install of mailcow:
<ol type="a">
<li>Download mailcow git repo</li>
<li>Run `generate.sh` to create configuration, CA list and snake-oil certificates</li>
</ol>
3. Copy configuration files from restored backup to mailcow directory (i.e. */opt/mailcow-dockerized*):
- mailcow.conf
- customization files (usually in *data/conf/…*)
- certificates
- etc.
4. Start mailcow normally once to generate default configurations and volumes.
5. Browse to mailcow admin panel and/or SOGo (front page) to verify everything is working with the defaults and the install is working before data restore.
6. Run `restore.sh` script to restore data.
If this overview is enough for you to figure things out, then you can get started. Remember, you can run the restore script with the `--help` parameter for a quick overview on how to use it:
```bash
./restore.sh --help
```
If you need a little more guidance, the following pages will go through each step.

64
8.1-Restore-extract.md Normal file

@ -0,0 +1,64 @@
# Restoring Backups - extract borg backup
The first step in restoring your data is extracting it from your borg archive that the backup script created for you. Let’s go through that process.
> Reminder: **Run your entire restore process as ROOT!**
First, why are we extracting the data locally instead of pulling it directly from our remote server? Well, a few reasons:
- Downloading it locally does not require any additional programs to be installed. The restore script uses *tar* which is installed on pretty much every Linux distro by default. Restoring directly from a remote repo would require something like *rsync* which is an excellent tool but I wanted to focus on staying as POSIX-compliant as possible. If you’d like to see *rsync* support, please file an issue requesting it and I’ll add it based on interest.
- Restoring data doesn’t always go smoothly the first time around. By having a local copy, you do not have to re-run lengthy and potentially costly downloads multiple times. Remember, many backup storage services charge you for downloading your data!
- Of course, the downside is that you require twice the storage space available - space for the downloaded backup and space to copy it to mailcow.
## Set up borg
Obviously, you need to have borg installed and functional before going any further. To access your remote borg repository, you’ll probably also need your SSH key and likely your borg repository key (unless you set up a *repokey* type repository in which case your key is stored in the repo itself and you only need your password). Once you have those things, you need to set some environment variables to get borg to work.
If you have access to the *backup.details* file used by *backup.sh* from your previous set up then this step is really easy. Just ‘source’ the file to import the environment variables into your working environment. I am assuming your borg set up is the same as your old system and **you are already running as root**.
```bash
# assuming the old backup.details file is in your home directory
source /root/backup.details
```
If you don’t have that file, we can still do things manually.
```bash
# assuming your borg base directory is /var/borgbackup
export BORG_BASE_DIR='/var/borgbackup'
export BORG_REMOTE_PATH=borg1
export BORG_RSH='ssh -i /path/to/your/ssh.key'
export BORG_REPO='user@remote.server.tld:repoName'
export BORG_PASSPHRASE='repository_password'
# if you are using a keyfile style repo, export this variable also
export BORG_KEY_FILE='/path/to/borg/key.file'
```
## Extract backup archive
Let’s see what our options are first – get a list of available archives in our borg repository:
```bash
borg list
```
If you get any errors running this command, double-check your environment variables in the previous step and reset them as necessary.
Each of your archives should be listed in the first column with their names being numerical date stamps. Review your options and choose the one you want to restore. Remember that borg extracts archives to the current directory, so we’ll make a directory first and switch to it.
```bash
# create directory for our files and switch to it
mkdir /var/mcRestore
cd /var/mcRestore
# extract borg archive from repository
borg extract --list ::archiveName
```
I’m using the `--list` parameter to output each file name to the console. This is optional but I like seeing what’s going on.
> N.B. If you do *not* run as root, borg **cannot** restore file permissions and ownership! If you run as a regular user, you’ll notice that all your files are restored with that user as the owner. This will **not** work since mailcow requires particular files to have particular owners within its various containers.
>
> Bottom line: **RUN BORG EXTRACT AS ROOT!**
Assuming the extraction is successful, you’re ready for the next step.

56
8.2-Restore-install-mailcow.md Normal file

@ -0,0 +1,56 @@
# Restoring Backups - install mailcow
We need to do a basic vanilla installation of mailcow so that we have something to restore data into. Follow the [official mailcow documentation](https://mailcow.github.io/mailcow-dockerized-docs/i_u_m_install/) but **do not** start mailcow yet!
> N.B. DO NOT start mailcow via `docker-compose up -d` just yet! We need to do some stuff before. You can, however, feel free to pull the containers.
> Remember to run *generate_config.sh* so that snake-oil certificates and the CA file is created. This is important regardless of whether or not you are using your own certificates. Since we will be using our old mailcow.conf, it doesn’t really matter what options you choose when running the generate script.
## Copy old configuration
Before we start mailcow, we need to copy our old configuration files so that everything is created using the correct *project name*, *hostname*, etc. This is simple since we have our backup already extracted.
### mailcow.conf
When you ran *generate_config.sh* a mailcow.conf file was created. Since we don’t need that file, you can delete it. Next, we’ll copy our old configuration from our backup:
```bash
cd /opt/mailcow-dockerized
rm mailcow.conf
cp /var/mcRestore/opt/mailcow-dockerized/mailcow.conf ./
```
## Customizations and certificates
If you have any custom postfix/unbound settings, certificates and docker-compose overrides, we need to copy those so that the environment is consistent with our backup. I’ll provide a few examples here, but ultimately only you know your own particular setup. Alternatively, if you have no idea what I’m talking about, you can likely skip this section entirely :stuck_out_tongue_winking_eye:
```bash
# assuming you are in the mailcow home directory (e.g. /opt/mailcow-dockerized)
# copy certificates
mkdir /certs
chown root:root /certs
chmod 700 /certs
cp /var/mcRestore/certs/* /certs/
# copy postfix extra.cf
cp /var/mcRestore/opt/mailcow-dockerized/data/conf/postfix/extra.cf ./data/conf/postfix/
# copy docker-compose override
cp /var/mcRestore/opt/mailcow-dockerized/docker-compose.override.yml ./
```
## Start mailcow
Now that our configuration is copied, we can start mailcow. We need to do this for two very important reasons:
1. Ensure that mailcow runs properly with our restored settings but without any data; and
2. Create empty default volumes for us to copy our data into!
```bash
docker-compose up -d && docker-compose logs -f
```
Quickly check the logs for errors, there shouldn’t be any. When you are done reviewing your real-time logs, press `ctrl-c` to exit. Now, open a browser and verify that you can get to the mailcow admin page and, optionally, the SOGo homepage. If you copied certificates, it should be using those assuming you set them up properly. We only need to test front pages, it is **not** necessary to login or anything.
Ok, we’re ready to restore our data!

103
8.3-Restore-restoring-data.md Normal file

@ -0,0 +1,103 @@
# Restoring Backups - restore data
Before restoring data, please ensure you have completed the previous steps. Specifically, ensure that mailcow has been started at least once to create volumes otherwise the restore script will exit with an error. Assuming we’re all good with the previous steps, let’s finish the restore.
## The restore script
The restore script has sane defaults and only requires one argument, `--backup-location`, which is the location of your extracted borg archive on the local system. You can run the script with the `--help` parameter for a quick overview of all options but, let’s go over them in slightly more detail here:
| parameter | description | default | required? |
| :------------------------ | ------------------------------------------------------------ | ------------------------------------------ | :-------: |
| -b<br />--backup-location | Directory containing your extracted borg backup archive. | none | YES |
| -l<br />--log | Path to write log file.<br />If you specify a filename, it will be created in the current directory.<br />If you specify a path ending with a ‘/’, the log file will be created using the default filename in that directory.<br />If you specify a full path, the log file will be saved as such.<br />In any case, the script will attempt to create any paths as needed. | scriptPath/scriptName.log | NO |
| -v<br />--verbose | Enable verbose logging. This will output the paths of all processed (restored) files in the log file. *Warning:* This may lead to a large log file. | false | NO |
| --skip-mail | Skip restoring email and encryption key. | false | NO |
| --skip-sql | Skip restoring mailcow settings database (mySQL). | false | NO |
| --skip-postfix | Skip restoring postfix settings/configuration files. | false | NO |
| --skip-rspamd | Skip restoring rspamd settings, configuration and history. | false | NO |
| --skip-redis | Skip restoring the redis database. | false | NO |
| -d<br />--docker-compose | FULL path to the *docker-compose.yml* file used by mailcow. | /opt/mailcow-dockerized/docker-compose.yml | NO |
| -m<br />--mailcow-config | FULL path to mailcow’s configuration file (*mailcow.conf*). This path is also assumed to be the main mailcow directory. | /opt/mailcow-dockerized/mailcow.conf | NO |
| -t1<br />--timeout-start | Number of seconds to wait for docker-compose to bring the mailcow project up before throwing an error. | 180 | NO |
| -t2<br />--timeout-stop | Number of seconds to wait for docker-compose to bring the mailcow project down before throwing an error. | 120 | NO |
| -?<br />-h<br />--help | Show in-script help (basically this table). | n/a | n/a |
### Restore data
Based on the information above, we can restore our data easily as follows:
```bash
# run with verbose logging
./restore.sh --backup-location /var/mcRestore --log /var/log/mailcow-restore.log --verbose
# run with standard logging
./restore.sh --backup-location /var/mcRestore --log /var/log/mailcow-restore.log
```
In this case, I have specified our example backup location and have told the script where to place the log file. In the first example, I’ve requested verbose output so my log file will show each file that was restored and whether or not there is a problem. The verbose option is very useful when testing restore operations but, for a production server you could end up listing hundreds of thousands of mail files in your log which could be overwhelming and not very useful.
That’s it. The script will restore the SQL database then shut down mailcow and copy all the data to the proper volumes before restarting mailcow.
## Post-restore steps
As with any restore operation, you should review the log to verify no errors or warnings were issued and you should verify that mailcow restarted properly. Assuming everything worked, your mailcow is fully restored!
> N.B. For some reason, the Rspamd console *password* is *not* restored. All Rspamd data and history is restored, however, you must manually reset the password for Rspamd via your mailcow admin console.
>
> If someone figures out why this is, I’d love to know!
### Clean-up
After you’ve verified your backup was successfully restored, you can remove your extracted borg archive:
```bash
rm -rf /var/mcRestore
```
## More script examples
- Default log filename and location (same directory as *restore.sh* and named *restore.log*).
```bash
./restore.sh -b /var/mcRestore
```
Note that if you rename *restore.sh*, the default name for the logfile will change accordingly. For example, if the restore script is renamed to *mailcow-restore.sh*, the default logfile will be named *mailcow-restore.log*.
- Only restore email, encryption key and mailcow database.
```bash
./restore.sh -b /var/mcRestore -l /var/log/mailcow-restore.log --skip-postfix --skip-rspamd --skip-redis
```
- Restore settings database and settings for all containers, skip email and encryption key. This might be useful if you want a clean-start on a new server with the same email addresses, configuration etc. but no old email.
```bash
./restore.sh -b /var/mcRestore -l /logs/mailcow-restore.log --skip-mail
```
- A very common situation is restoring email, encryption key, mailcow settings and spam database.
```bash
./restore.sh -b /var/mcRestore -l /root/mc_restore.log --skip-postfix --skip-redis
```
- Customized docker-compose file (this does *not* affect *override* file processing).
- ```bash
./restore.sh -b /var/mcRestore -l /var/log/mc_restore.log --docker-compose /opt/mailcow-dockerized/myMailcow.yml
```
- Non-standard mailcow location.
```bash
./restore.sh -b /var/mcRestore -l /var/log/mc_restore.log --docker-compose /usr/local/mailcow/docker-compose.yml --mailcow-config /usr/local/mailcow/mailcow.conf
```
- On some systems, docker-compose may need more time to start and/or stop. I’ll show setting both timeouts at the same time, but you can specify only one or the other as needed.
```bash
./restore.sh -b /var/mcRestore -l /logs/mcRestore.log --timeout-start 90 --timeout-stop 300
```

73
8.3.1-Restore-manual-restore.md Normal file

@ -0,0 +1,73 @@
# Restoring Backups - manual restore
If you don’t want to use the restore script or cannot use it due to a non-standard set up, here is how you can manually restore your mailcow from the extracted borg archive.
First off, you still have to do all the steps as outlined on pages [8.1](https://git.asifbacchus.app/asif/MailcowBackup/wiki/8.1-Restore-extract) and [8.2](https://git.asifbacchus.app/asif/MailcowBackup/wiki/8.2-Restore-install-mailcow).
1. Switch to your mailcow directory (I will assume the default location for this example) and START MAILCOW. Then ‘source’ your *mailcow.conf* file so you can use environment variables in the subsequent commands. Wait a minute or two before proceeding to the next step so that mailcow has time to start properly.
```bash
cd /opt/mailcow-dockerized
docker-compose up -d
source mailcow.conf
```
2. Let’s restore the mailcow database first. Remember to change the location of your extracted backup as necessary. You can skip this step if you are not restoring the mailcow settings database.
```bash
SQLBACKUP=$(find /var/mcRestore/tmp -iname "*.sql")
docker exec -i "$(docker-compose ps -q mysql-mailcow)" mysql -u${DBUSER} -p${DBPASS} ${DBNAME} < "${SQLBACKUP}"
```
3. Stop all containers.
```bash
docker-compose down
```
4. Copy our extracted data to the proper volumes. You can skip any of these as per your requirements. Also, remember to change the location of your extracted backups as necessary.
```bash
# copy email
SRC=$(find /var/mcRestore -iname "*vmail*" -type d)
TGT=$(docker volume inspect -f '{{.Mountpoint}}' ${COMPOSE_PROJECT_NAME}_vmail-vol-1)
(cd "$SRC/_data" && tar -cf - .) | (cd "$TGT" && tar xvf -)
# copy encryption key
SRC=$(find /var/mcRestore -iname "*crypt*" -type d)
TGT=$(docker volume inspect -f '{{.Mountpoint}}' ${COMPOSE_PROJECT_NAME}_crypt-vol-1)
(cd "$SRC/_data" && tar -cf - .) | (cd "$TGT" && tar xvf -)
# copy postfix data
SRC=$(find /var/mcRestore -iname "*postfix*" -type d)
TGT=$(docker volume inspect -f '{{.Mountpoint}}' ${COMPOSE_PROJECT_NAME}_postfix-vol-1)
(cd "$SRC/_data" && tar -cf - .) | (cd "$TGT" && tar xvf -)
# copy rspamd data
SRC=$(find /var/mcRestore -iname "*rspamd*" -type d)
TGT=$(docker volume inspect -f '{{.Mountpoint}}' ${COMPOSE_PROJECT_NAME}_rspamd-vol-1)
(cd "$SRC/_data" && tar -cf - .) | (cd "$TGT" && tar xvf -)
# copy redis
SRC=$(find /var/mcRestore -iname "*redis*" -type d)
TGT=$(docker volume inspect -f '{{.Mountpoint}}' ${COMPOSE_PROJECT_NAME}_redis-vol-1)
(cd "$SRC/_data" && tar -cf - .) | (cd "$TGT" && tar xvf -)
```
5. Restart mailcow and monitor the logs. When you’re done monitoring the real-time log file, press `ctrl-c`
```bash
docker-compose up -d && docker-compose logs -f
```
6. Login to the mailcow admin console. Ensure your admin login works properly. Reset Rspamd console password (not restored for some reason).
7. Test out mailcow, ensure data is restored.
8. Clean-up extracted borg archive (optional but recommended to save space)
```bash
rm -rf /var/mcRestore
```
That’s it!

0
8.0-Final-notes.md → 9.0-Final-notes.md