Is there a need to create a backup copy of a directory according to a schedule and store it on an external storage?

This can be done quite easily using KeepData as a remote cloud storage. Moreover, such a process can be fully automated.

The solution varies depending on the OS, but the general algorithm consists of running an executable file on a designated schedule. Let’s look at examples.

Creating backups in Windows

In Windows, this can be done using an executable file in the .bat format and the Windows task scheduler.

Let’s look at the step-by-step instructions:

1. For the correct execution of the future .bat file, we will need to download and install the WinSCP program. This can be done on their official website. You will not need to perform any actions in the program itself.

2. Create a new text file in any convenient location. This will be our new backup script.

3. The most important step is filling the script. Below is an example with explanations. The highlighted variables must be filled with your information:

@echo off

"C:\Program Files (x86)\WinSCP\WinSCP.exe" ^
  /ini=nul ^
  /command ^
    "open ftp://login:password@server.keepdata.thehost.ua/" ^
    "lcd C:\Folder\Work" ^
    "cd /backups-work/" ^
    "mkdir %%TIMESTAMP#yyyy-mm-dd%%" ^
    "cd /backups-work/%%TIMESTAMP#yyyy-mm-dd%%" ^
    "put File1" ^
    "lcd C:\" ^
    "put Folder2" ^
    "lcd C:\Users\User" ^
    "put File3" ^
    "exit"

set WINSCP_RESULT=%ERRORLEVEL%
if %WINSCP_RESULT% equ 0 (
  echo Success
) else (
  echo Error
)

exit /b %WINSCP_RESULT%

Now, to the explanations:

• "C:\Program Files (x86)\WinSCP\WinSCP.exe" ^ – specify the path to the WinSCP .exe file. The example specifies the path for a standard installation.
• "open ftp://login:password@server.keepdata.thehost.ua/" ^ – this line connects to your Cloud Storage. Here you need to specify the login, password and connection server. You can find all this data in the Letter about opening the service.
• "lcd C:\Folder\Work" ^ – this command goes to the dedicated path on the local PC.
• "cd /backups-work/" ^ – this command goes to the dedicated path on your Cloud Storage.
• "mkdir %%TIMESTAMP#yyyy-mm-dd%%" ^ – creates a directory in KeepData with a name in the format year-month-day.
• "cd /backups-work/%%TIMESTAMP#yyyy-mm-dd%%" ^ – goes to the specified directory.
• "put File1" ^ – uploads File1 from its current location on the local PC (D:\Folder\Work in this example) to this directory.
• "lcd C:\" ^ – goes to the new path on the local PC.
• "put Folder2" ^ – uploads Folder2 directory and all of its data from the new path to Cloud Storage.
• "lcd C:\Users\User" ^ – navigates to a new path on the local PC.
• "put File3" ^ – uploads the file File3 from the new path to the Cloud Storage.

Thus, we have a script that connects to KeepData, creates a new folder in the specified directory and uploads three files from different locations on the local PC. The created folder will have a name similar to the time of its creation, which will facilitate further work with the existing backups. If necessary, the specified script can be edited to suit your needs.

4. Save our text file in the .bat format. To do this, in the file editor, instead of Save, click Save as. When saving, select All files and specify the full file name, along with the .bat format at the end:

Now we have a written script that performs a backup. How can this process be automated? The Windows Task Scheduler will help us with this.

5. Open the Task Scheduler using the Start menu or alternatively by pressing Win + R and running taskschd.msc:

6. Right-click on Task Scheduler Library to open the context menu and click Create task:

7. Go to the Triggers submenu and click Create. In the open form, specify the schedule for the task to be executed. For example, every day at 12:00:

8. Similarly, go to the Actions submenu and click Create again. In the open form, select Run program and specify the path to our script, manually or using the Browse button:

9. The task has been created. It will be executed according to the specified schedule. It is highly recommended to test its execution immediately by manually starting it – by right-clicking on the task and pressing Run. If the test execution is unsuccessful – contact our Support Service providing as much information as possible, including the text of your script.

Creating backups in Linux

In Linux, the process is similar, with an adjustment for the tools and OS dependencies. Also, there will be two scripts for performing a backup – for the files themselves and for the databases.

The dependency for the backup executable file in Linux is the console FTP client LFTP.

Let’s look at the setup instructions using a step-by-step example:

1. Install LFTP.

• You can install it in Ubuntu/Debian like this:

sudo apt update
sudo apt install lftp

• In CentOS/AlmaLinux/RockyLinux:

sudo yum install lftp

2. In Linux, we will need two scripts – for backing up certain directories and for backing up certain databases. In our example they are called backup-dir.sh and backup-mysql.sh, their creation is dedicated to steps 3 and 4 of this instruction. If you need only one – create only one. Creation of future executable files in .sh format in the desired directory is performed as follows:

touch backup-dir.sh
touch backup-mysql.sh

3. Let’s start with the script that backs up the files of the desired directory, in this example it is called backup-dir.sh. Open it with any text editor, for example, with nano backup-dir.sh. Below is an example of the script with explanations. Highlighted variables must be filled with your information:

#!/bin/bash
echo "Directory Backuper by TheHost"

#Defining variables and parameters for accessing the FTP server
TMP_PATH="/tmp"
BACKUP_DIRS="/home/user/data /home/user/data2 /home/user/data3"
FTP_HOST="ftp://server.keepdata.thehost.ua"
FTP_USER="ftp_user"
FTP_PASS="ftp_password"

#Creating a new backup locally, in a temporary directory
DATE=`date +%Y-%m-%d`
DATE_TIME=`date +%Y-%m-%d-%H-%M-%S`
mkdir -p "$TMP_PATH/DIR_BACKUP-$DATE"
cd $TMP_PATH/DIR_BACKUP-$DATE
GZIPFILE="backup-$DATE_TIME.tgz"
echo -n "Creating archive $GZIPFILE... "
tar -czf $GZIPFILE $BACKUP_DIRS && echo "OK" || echo -e "Failed!\a"

#Uploading the created archive to FTP storage
echo -n "Checking $FTP_HOST for today's archive... "
if [[ -z "`lftp -u $FTP_USER,$FTP_PASS -e "cd $DATE && renlist backup-$DATE-*.tgz; bye" $FTP_HOST 2>/dev/null`" ]]; then
  echo "not found"
  echo "Uploading archive to $FTP_HOST"
  lftp -u $FTP_USER,$FTP_PASS -e "cd $DATE || mkdir $DATE; cd /$DATE && put $GZIPFILE; bye" $FTP_HOST >/dev/null 2>&1;
else
  echo "already exist!";
fi


#Removing temporary directory
cd $TMP_PATH
rm -rf $TMP_PATH/DIR_BACKUP-$DATE

echo "Done."
exit 0

Despite the cumbersome appearance of the script, it requires very little modification. Namely:

• TMP_PATH="/tmp" – sets a temporary directory for storing the created backup copy before uploading it to the remote server. You can leave the default value /tmp.
• BACKUP_DIRS="/home/user/data /home/user/data2 /home/user/data3" – which directories will be backed up to the archive. You can specify one.
• FTP_HOST="ftp://server.keepdata.thehost.ua" – the address of your Cloud Storage.
• FTP_USER="ftp_user" – FTP user of your Cloud Storage.
• FTP_PASS="ftp_password" – password of FTP user of your Cloud Storage.

Such a script will create an archive in .tgz format of the directories specified in BACKUP_DIRS=, save them in TMP_PATH= locally and then try to upload the archive to the remote FTP storage, to the directory it created with the name in the form of the current date. If successful, it will delete the local copy of the archive. After editing the script, save the file, be sure to use the .sh format.

4. Now we edit the script for the databases, in our example it is backup-mysql.sh. Let’s open it with any text editor, for example, using nano backup-mysql.sh. Below is an example of the script with explanations. Highlighted variables must be filled with your information:

#!/bin/bash
echo "Database Backuper by TheHost"

#Defining variables and access parameters
DATABASES='database1 database2 database3'
TMP_PATH="/tmp"
BACKUP_USER="mysql_user"
BACKUP_PASS="mysql_password"
FTP_HOST="ftp://server.keepdata.thehost.ua"
FTP_USER="ftp_user"
FTP_PASS="ftp_password"

#Creating a new backup
DATE=`date +%Y-%m-%d`
DATE_TIME=`date +%Y-%m-%d-%H-%M-%S`
mkdir -p "$TMP_PATH/DB_BACKUP-$DATE"
cd $TMP_PATH/DB_BACKUP-$DATE
for DB in $DATABASES; do
  echo -n "Dumping $DB... "
  mysqldump -E -R -u"$BACKUP_USER" -p"$BACKUP_PASS" $DB > noc-$DATE_TIME.$DB.sql && echo "OK" || echo -e "Failed!\a"
done
GZIPDB="backup-$DATE_TIME.tgz"
echo -n "Creating archive $GZIPDB... "
tar -czf $GZIPDB --remove-files backup-$DATE_TIME.*.sql && echo "OK" || echo -e "Failed!\a"

#Downoading backup on a remote storage
echo -n "Checking $FTP_HOST for today's archive... "
if [[ -z "`lftp -u $FTP_USER,$FTP_PASS -e "cd $DATE && renlist backup-$DATE-*.tgz; bye" $FTP_HOST 2>/dev/null`" ]]; then
  echo "not found"
  echo "Uploading archive to $FTP_HOST"
  lftp -u $FTP_USER,$FTP_PASS -e "cd $DATE || mkdir $DATE; cd /$DATE && put $GZIPDB; bye" $FTP_HOST >/dev/null 2>&1;
else
  echo "already exist!";
fi

#Removing the temporary directory
cd $TMP_PATH
rm -rf $TMP_PATH/DB_BACKUP-$DATE

echo "Done."
exit 0

Explanations:

• DATABASES='database1 database2 database3 – list of databases that the task will affect. Separate with spaces.
• TMP_PATH="/tmp" – sets a temporary directory for storing the created backup copy before uploading it to the remote server. You can leave the default value of /tmp.
• BACKUP_USER="mysql_user" – user to log in to MySQL.
• BACKUP_PASS="mysql_password" – password of the specified user.
• FTP_HOST="ftp://server.keepdata.thehost.ua" – the address of your Cloud Storage.
• FTP_USER="ftp_user" – the FTP user of your Cloud Storage.
• FTP_PASS="ftp_password" – password of the FTP user of your Cloud Storage.

This script will create database dumps DATABASES= when launched and compress them into an archive in .tgz format, save it in TMP_PATH= locally and then try to upload it to the remote FTP storage, into the directory it created with the name in the form of the current date. If successful, it will delete the local copy of the archive. After editing the script, save the file, be sure to use the .sh format.

5. Grant the scripts execution rights:

chmod +x /path/to/backup-dir.sh
chmod +x /path/to/backup-mysql.sh

Where /path/to/ replace with the paths to the created scripts.

6. The scripts are created, it remains to automate their execution. For this, we use a CRON task.

Open the crontab file with the command:

crontab -e

Add new tasks for executing your scripts. For example:

30 3 * * * /path/to/backup-dir.sh
30 3 * * * /path/to/backup-mysql.sh

Replace /path/to/ with the paths to the created scripts, then save the file. The provided examples will be executed daily at 3:30 am.

7. Done! The scripts have been created and edited with your data, the CRON task for their execution is already waiting for the required date. We also recommend that you run the script manually for testing. This can be done with the following command:

./path/to/backup-dir.sh

Where /path/to/ is replaced with the path to the script. If the test was successful, then you have configured everything correctly. If the desired result was not achieved, contact our Support Service providing as much information as possible, including the contents of your scripts.