Articles in this section

See more

Basic backup configuration

Avatar
Jakub Samek
  • Updated
Follow

It is more than wise to have an application, server or installation of any kind backed up to prevent a loss caused by a technical glitch, human error or a case rollback situation.

IceWarp backup in the general

IceWarp, in general, has two types of data:

  • Mails and documents are stored on a filesystem directly, basically as .imap files.
  • Other data, such as contacts and calendars (groupware) are stored in the database.

Based on the specification of the installation, best practice is to use a combination of backups by:

  • Snapshotting by virtualisation technology or by a dedicated storage mechanism.
  • Database by using scripts started manually/by the scheduler.
  • Email files by using scripts started manually/by scheduler..
  • Database and files using 3rd party technology.
  • Database by built-in IceWarp mechanism.
  • Configuration manually or by built-in tools.

Do not forget that the configuration of backups ends with the recovery test to ensure that configured backup mechanism is reliable. During the testing recovery section, try to restore not only IceWarp as a whole but also folder by folder or file by file because restoration of one missed file is a more probable scenario than restoring the whole server.

Some of the approaches described in this general section are described with the examples in the sections below.

Backup of the database and files by Linux bash script

One of the approaches described above is to use a script with standard file operations and tools. Below is the example of a bin/bash script for creating the backup on the Linux IceWarp installation - files and database. Such a script can be added into the crontab and start, for example, every night.

Warning. Read through the script and understand it fully before applying it. Do not forget to change all necessary variables where applicable to your specific installation, especially paths and directory names.

#!/bin/bash
source /etc/icewarp/icewarp.conf
maildirpath="$(${IWS_INSTALL_DIR}/tool.sh get system C_System_Storage_Dir_MailPath | grep -P '(?<=: ).*(?=/mail/)' -o)"
backuppath="${maildirpath}/backup"
scriptdir="$(cd $(dirname $0) && pwd)"
mkdir -p ${maildirpath}
mkdir -p ${backuppath}
mkdir -p ${scriptdir}/logs
backupsrvhost="${1}" # if we take backups from another host
backupsrvport="${2}" # than the one we connect to
logdate="$(date +%Y%m%d)"
logfile="${scriptdir}/logs/bck_${logdate}.log"

function log {
  echo $(date +%H:%M:%S) $1 >> ${logfile}
}

log "Starting backup."

utiltest="$(/usr/bin/find /usr/lib64 -type f -name "Entities.pm")"
if [[ -z "${utiltest}" ]]
  then
  log "Installing Entities.pm"
  /usr/bin/yum -y install epel-release
  /usr/bin/yum -y install perl-HTML-Encoding.noarch
fi

log "Reading DB access data."
dbpass="$(/usr/bin/cat ${IWS_INSTALL_DIR}/config/_webmail/server.xml | cat ${IWS_INSTALL_DIR}/config/_webmail/server.xml | egrep -o ".*" | perl -pe 's|^\<dbpass\>(.*)\</dbpass\>$|\1|' | perl -MHTML::Entities -pe 'decode_entities($_);')"
wcdbuser="$(/usr/bin/cat ${IWS_INSTALL_DIR}/config/_webmail/server.xml | egrep -o ".*" | perl -pe 's|^\<dbuser\>(.*)\</dbuser\>$|\1|')"
read -r dbhost dbport wcdbname <<< $(cat ${IWS_INSTALL_DIR}/config/_webmail/server.xml | egrep -o "mysql:host=.*;port=.*;dbname=.*" | perl -pe 's|^\<dbconn\>mysql:host=(.*);port=(.*);dbname=(.*)\</dbconn\>$|\1 \2 \3|')
read -r accdbname accdbuser <<< $(${IWS_INSTALL_DIR}/tool.sh get system c_system_storage_accounts_odbcconnstring | perl -pe 's|^c_system_storage_accounts_odbcconnstring: (.*);(.*);.*;.*;.*;.*$|\1 \2|')
read -r aspdbname aspdbuser <<< $(${IWS_INSTALL_DIR}/tool.sh get system c_as_challenge_connectionstring | perl -pe 's|^c_as_challenge_connectionstring: (.*);(.*);.*;.*;.*;.*$|\1 \2|')
read -r grwdbname grwdbuser <<< $(${IWS_INSTALL_DIR}/tool.sh get system c_gw_connectionstring | perl -pe 's|^c_gw_connectionstring: (.*);(.*);.*;.*;.*;.*$|\1 \2|')
read -r dcdbname dcdbuser <<< $(${IWS_INSTALL_DIR}/tool.sh get system c_accounts_global_accounts_directorycacheconnectionstring | perl -pe 's|^c_accounts_global_accounts_directorycacheconnectionstring: (.*);(.*);.*;.*;.*;.*$|\1 \2|')
easdbname="$(${IWS_INSTALL_DIR}/tool.sh get system c_activesync_dbconnection | ${IWS_INSTALL_DIR}/tool.sh get system c_activesync_dbconnection | egrep -o "mysql:host=.*;dbname=.*" | perl -pe 's|^mysql:host=.*;port=.*;dbname=(.*)$|\1|')"
easdbuser="$(${IWS_INSTALL_DIR}/tool.sh get system c_activesync_dbuser | perl -pe 's|^c_activesync_dbuser: (.*)$|\1|')"
easdbpass="$(${IWS_INSTALL_DIR}/tool.sh get system c_activesync_dbpass | perl -pe 's|^c_activesync_dbpass: (.*)$|\1|')"
if [ -z "${easdbpass}" ]; then easdbpass="easdbpass not discovered"; fi
if [[ "${dbpass}" =~ "^sqlite:.*" ]]; then dbpass="${easdbpass}"; fi
if [ -z "${dbpass}" ]; then dbpass="dbpass not discovered"; fi
if [ ! -z "${backupsrvhost}" ]; then dbhost="${backupsrvhost}"; fi # if we take backups from another host
if [ ! -z "${backupsrvport}" ]; then dbport="${backupsrvport}"; fi # than the one we connect to

/usr/bin/mysqldump --single-transaction -u ${accdbuser} -p${dbpass} -h${dbhost} -P ${dbport} ${accdbname} | gzip -c | cat > ${backuppath}/bck_db_acc_${accdbname}`date +%Y%m%d-%H%M`.sql.gz &
/usr/bin/mysqldump --single-transaction -u ${aspdbuser} -p${dbpass} -h${dbhost} -P ${dbport} ${aspdbname} | gzip -c | cat > ${backuppath}/bck_db_asp_${aspdbname}`date +%Y%m%d-%H%M`.sql.gz &
/usr/bin/mysqldump --single-transaction -u ${grwdbuser} -p${dbpass} -h${dbhost} -P ${dbport} ${grwdbname} | gzip -c | cat > ${backuppath}/bck_db_grw_${grwdbname}`date +%Y%m%d-%H%M`.sql.gz &
/usr/bin/mysqldump --single-transaction -u ${dcdbuser} -p${dbpass} -h${dbhost} -P ${dbport} ${dcdbname} | gzip -c | cat > ${backuppath}/bck_db_dc_${dcdbname}`date +%Y%m%d-%H%M`.sql.gz &
/usr/bin/mysqldump --single-transaction -u ${easdbuser} -p${easdbpass} -h${dbhost} -P ${dbport} ${easdbname} | gzip -c | cat > ${backuppath}/bck_db_eas_${easdbname}`date +%Y%m%d-%H%M`.sql.gz &
/usr/bin/mysqldump --single-transaction -u ${wcdbuser} -p${dbpass} -h${dbhost} -P ${dbport} ${wcdbname} | gzip -c | cat > ${backuppath}/bck_db_wc_${wcdbname}`date +%Y%m%d-%H%M`.sql.gz &

log "Finished DB backup."
wait ${!}
log "Starting IW config backup."
/usr/bin/tar -czf ${backuppath}/bck_cnf`date +%Y%m%d-%H%M`.tgz ${IWS_INSTALL_DIR}/config > /dev/null 2>&1
/usr/bin/tar -czf ${backuppath}/bck_cal`date +%Y%m%d-%H%M`.tgz ${IWS_INSTALL_DIR}/calendar > /dev/null 2>&1
${IWS_INSTALL_DIR}/tool.sh export account "*@*" u_backup > ${backuppath}/bck_acc_backup`date +%Y%m%d-%H%M`.csv
${IWS_INSTALL_DIR}/tool.sh export domain "*" d_backup > ${backuppath}/bck_dom_backup`date +%Y%m%d-%H%M`.csv
log "Finished IW config backup."
log "Cleaning old backups and logs."
/usr/bin/find ${backuppath}/ -type f -name "bck_*" -mtime +3 -delete > /dev/null 2>&1
/usr/bin/find ${scriptdir}/logs/ -type f -name "bck_*.log" -mtime +30 -delete > /dev/null 2>&1
log "All done."
exit 0

Backup of MySQL database in the Windows environment

The example above backs up the files and database entries. As mentioned previously, files such as emails are stored on the filesystem directly, and handling is very similar to the previous part. Handling Databases on Windows can be specific. Below is an example of MySQL backup using a .bat file and robocopy and 7zip. Both must be installed on the server which runs the script.

Warning. Read through the script and understand it fully before applying it. Do not forget to change all necessary variables where applicable to your specific installation, especially paths and directory names.

@echo off
 :: db admin user and password
 set dbUser=user
 set dbPassword=pass
 
 :: IceWarp installation dir
 set iwMainDir="C:\Program Files\IceWarp"
 
 :: backup retention temp dir
 set rtDir="C:\Program Files\IceWarp\backup\temp"
 
 :: how many older backups do we keep
 set rtDays=3
 
 :: directory where to store the backups
 set backupDir="C:\Program Files\IceWarp\backup\dbdump"
 
 :: MySQL data dir
 set mysqlDataDir="C:\Program Files\MariaDB 10.2\data"
 
 :: full path to mysqldump.exe binary
 set mysqldump="C:\Program Files\MariaDB 10.2\bin\mysqldump.exe"
 
 :: full path to 7z.exe binary
 set zip="C:\Program Files\7-Zip\7z.exe"

 if not exist %backupDir% mkdir %backupDir%
 if not exist %rtDir% mkdir %rtDir%

 :: get date ( on windows in cs lang, change delim from / to ., ie.: for /F "tokens=2-4 delims=. )
 for /F "tokens=2-4 delims=/ " %%i in ('date /t') do (
      set yy=%%i
      set mon=%%j
      set dd=%%k
 )

 :: get time 
 for /F "tokens=5-8 delims=:. " %%i in ('echo.^| time ^| find "current" ') do (
      set hh=%%i
      set min=%%j
 )

 echo dirName=%yy%%mon%%dd%_%hh%%min%
 set dirName=%yy%%mon%%dd%_%hh%%min%
 
 :: switch to the "data" folder
 pushd %mysqlDataDir%

 :: iterate over the folder structure in the "data" folder to get the databases
 for /d %%f in (*) do (

 if not exist %backupDir%\%dirName%\ (
      mkdir %backupDir%\%dirName%
 )

 %mysqldump% --host="localhost" --user=%dbUser% --password=%dbPassword% --single-transaction --add-drop-table --databases %%f > %backupDir%\%dirName%\%%f.sql

 %zip% a -tgzip %backupDir%\%dirName%\%%f.sql.gz %backupDir%\%dirName%\%%f.sql
 del %backupDir%\%dirName%\%%f.sql
 )
 popd

 :: backup server settings
 %iwMainDir%\tool.exe export account "*@*" u_backup > %backupDir%\%dirName%\acc_u_backup.csv
 %iwMainDir%\tool.exe export domain "*" d_backup > %backupDir%\%dirName%\dom_d_backup.csv
 %zip% a -r %backupDir%\%dirName%\cfg.7z %iwMainDir%\config
 %zip% a -r %backupDir%\%dirName%\cal.7z %iwMainDir%\calendar

 :: remove backups older than 3 days
 ROBOCOPY %backupDir% %rtDir% /mov /minage:%rtDays%
 del "%rtDir%" /q

Recovery test - folder of more files

  1. Find out when the missing file last occurred. For example, in the snapshots in the directory  /mnt/data/.zfs/snapshot/
  2. Then rsync them back to prod. maildir, for example.: rsync -av --include="*/" --include="*.imap" --exclude="*" /mnt/data/.zfs/snapshot/20220228-0000/icewarp/mail/domainexample.com/exampleaccount/inbox/ /mnt/data/icewarp/mail/domainexample.com/examleaccount/
  3. If necessary, run the chown command on icewar:icewarp or any other user account in which context IW runs: chown -R icewarp:icewarp /mnt/data/icewarp/mail/domainexample.com/exampleaccount/inbox
  4. Important - refresh directory cache: /opt/icewarp/tool.sh set account "exampleaccount@domainexample.com" u_directorycache_refreshnow 1
  5. If refreshing the directory cache isn't enough, also reset the WebClient cache https://<DOMAIN>/wccm/main.php or maybe https://<DOMAIN>/webmail/wccm/main.php
  6. In case of any further issues: Restart IMAP -> refresh directory cache -> refresh WebClient cache

Recovery test - specific file

In this example, is restoration considered again from the path /mnt/data/.zfs/snapshot/

  1. Find out where the missing file last occurred. It can be done by searching through the filesystem using grep with regexgrep -Rl "Hello, my name is Alejandro" /mnt/data/.zfs/snapshot/20220228-0000/icewarp/mail/domainexample.com/exampleaccount/inbox/ will search for that string recursively in the exampleaccount/inbox folder
  2. Copy the mail, for example: cp /mnt/data/.zfs/snapshot/20220228-0000/icewarp/mail/domainexample.com/exampleaccount/inbox/2022052512562547.imap /mnt/data/icewarp/mail/domainexample.com/exampleaccount/inbox
  3. If necessary, run the chown command on icewar:icewarp or any other user account in which context IW runs: chown -R icewarp:icewarp /mnt/data/icewarp/mail/domainexample.com/exampleaccount/inbox
  4. Refresh directory cache: /opt/icewarp/tool.sh set account "exampleaccount@domainexample.com" u_directorycache_refreshnow 1
  5. If refreshing the directory cache isn't enough, also reset the WebClient cache https://<DOMAIN>/wccm/main.php or maybe https://<DOMAIN>/webmail/wccm/main.php
  6. In case of any further issues: Restart IMAP -> refresh directory cache -> refresh WebClient cache

Configuration backup using a built-in tool

The IceWarp Server configuration data is stored in these folders:

{InstallDirectory}/config/

{InstallDirectory}/spam/

{InstallDirectory}/calendar/

{InstallDirectory}/_webmail/config/

These folders contain the following information:

  • All system settings
  • Users, domains, and all settings associated with each
  • The Calendar and Spam folders contain the default SQLite databases for groupware and Antispam. These hold the data for Contacts, Calendars, and White and Blacklists.)

Backup can be done by using the remote console:  File > Backup settings menu option. It can also be automated by using the Tools > System Backup feature.

If using external databases (MySQL, Microsoft SQL Server, etc.) for accounts, groupware or Antispam, backup of the database must be done manually or by using the option in the System > Tools > System Backup > Database section to synchronize your databases to another database server.

For more information, visit this article.

Backup of the files using a built-in tool

It is not recommended to use a built-in tool for backing up the mail files. The internal tool creates one big zip file, which can be difficult to manipulate using technology/server limitations, especially on big installations.

Final hints

Unless it is not related to IceWarp directly, do not forget to estimate the RTO (recovery time objective) and RPO (recovery point objective) times. It is important to understand that recovery takes time, and it is reasonable to know what this time is in your environment. More about RTO and RPO. It easily says RTO is how fast data can be recovered, and RPO is to which point. Do not forget, if the backup is performed once in a day, RPO is 24 hours. It means RPO is the amount of time in the worst-case scenario.

Test the backups regularly. Run manual backup before every update

Check the consistency of the backup in the context of the data space allocated to backup files. In many real-life scenarios, backups were not made because of non-sufficient dataspace left. Do not forget to add backups creation itself into the monitoring system.

If necessary, only if license can be exported. See this article.

 

Related articles

Comments

0 comments

Please sign in to leave a comment.