You may have important files in your computer such as a password manager database, gpg key, bitcoin private keys. There is a risk of loss or modification of these files for multiple possible reason such as hardware failure. If that happen it may have a big impact for you. In this article some simple backup method for Linux will be presented as well as how to automatize them.

Importance of backup

Losing data can happen for multiple reasons such as:

accidental delete
mechanical or electronic defect on device
random errors on file

For this reason, important data and especially private key, password manager, etc., should be backed up:

Some methods of backup

In this part different method for backing up files with Linux will be presented. First a custom bash script is created. In a second part, an attempt of a one-liner will be presented.

Custom scripts

Let's make a custom bash script to backing up files into another folder

Attention: This script creates and delete files permanently. You can use it or modify it to your needs, but I cannot be held responsible for any bad manipulation you could do. I also recommend using rm -i when testing script using rm to have interactive deletion and avoid accidental deletion.

# Change following strings to adapt to your case
file='/path/to/my/important/data/'
number_backup=6 # the maximum number of copies we want
# end
now="$(date +%Y_%m_%d)" # retrieve the current day in the format YYYY_MM_DD
new_name="$file""_$now"".bak"
cp "$file" "$new_name"

# To delete the old
# Manage the number of copies and delete when there are too many
counter=$(find "$(dirname $file)" -name "$(basename $file)_*.bak" | wc -l)
if [ "$counter" -le "$number_backup" ]
then
    echo "only $counter (<=$number_backup) copy of the file"
    echo "Nothing to do. Exiting"
    exit 0
else
    diff=$(( "$counter" - "$number_backup"))
    to_delete=$(find "$(dirname $file)" -name "$(basename $file)_*" | sort | head -n "$diff")
    rm -i "$to_delete"
    # When you finish testing it
    # you can replace it by rm "$to_delete"
fi

Of course this script can be adapted for example to add logs or any other actions

This script has some problems which are:

the script is long
it cannot be used remotely easily
it would be even better if it can be done in one line (linux powa (^^)/)

One liner with rsync

In this part I will present the rsync command and how to make the backup command in one line.

Quick presentation of rsync

From the man page of rsync:

rsync - a fast, versatile, remote (and local) file-copying tool

Indeed, rsync is a command that can be easily used to make copies and synchronize folders locally or remotely. It is easy to use:

rsync [OPTION] SRC DST

Let's see some options that are useful (by reading man rsync):

#[useful command to debug]
--verbose, -v            increase verbosity 
--dry-run, -n            perform a trial run with no changes made

# For Backups
--backup, -b             make backups (see --suffix & --backup-dir)
--rsh=COMMAND, -e        specify the remote shell to use

# Commands that are useful when doing system backup and being able to restore the permission
--perms, -p              preserve permission
--acls, -A               preserve ACLs (implies --perms)
--xattrs, -X             preserve extended attributes
--owner, -o              preserve owner (super-user only)
--group, -g              preserve group

So for example if I want to back up a file in a remote server, I can use the following command:

rsync --verbose --backup -e "ssh -i <private_key>" <file> <username>@<server address>:/path/to/bakup/file>

You should, of course, give correct access permission to the private key file, especially if you do not password protect it for automating the backup. In this case, I advise using a key that will be only be used to for the backing up data.

Let's back it up

From the previous part, it is easy to adapt the first script by using rsync instead of cp:

rsync --backup <file> <user>@<address>:</path/to/backup/file>`date +'.%Y_%m_%d.bak'`

or if we want to back it up remotely

rsync --backup -e "ssh -i </path/to/private/key>" <file> <user>@<address>:</path/to/backup/file>`date +'.%Y_%m_%d.bak'`

Now let’s try to improve the deletions of the old files.

# The files older than n days 
# /!\ deleted files are deleted forever. You should test it carefully
find . -name '*.bak' -mtime +<n> -delete
# Or only keep the <n> more recent files
ls -1t *.bak | tail -n +$((<n>+1)) | xargs -d '\n' rm -i # replace by rm when needed

The first command is not to difficult to understand. Let's explain the second command:

ls -1t *.bak : Find all files ending by .bak and sort them in one column
tail -n +$((<n>+1)) : keep only the lines after line n
xargs -d '\n' rm : consider each entry separated by a newline '\n' as a file name and delete it

From it, the backup can become a one-liner:

rsync --backup <file> </path/to/backup/file>`date +'.%Y_%m_%d.bak'` && 
find . -name '*.bak' -mtime +<n> -delete 
# or
rsync --backup <file> </path/to/backup/file>`date +'.%Y_%m_%d.bak'` &&
ls -1t *.bak | tail -n +$((<n>+1)) | xargs -d '\n' rm -i

And in case it is done remotely, we should use ssh command to execute it remotely or launch the deletion script directly on the machine:

key=</path/to/private/key> && 
rsync --backup -e "ssh -i $key" <file> <user>@<address>:</path/to/backup/file>`date +'.%Y_%m_%d.bak'` && 
ssh -i $key <user>@<address> "find . -name '*.bak' -mtime +<n> -delete"
# or
key=</path/to/private/key> &&
rsync --backup -e "ssh -i $key" <file> <user>@<address>:</path/to/backup/file>`date +'.%Y_%m_%d.bak'` && 
ssh -i $key <user>@<address> "ls -1t *.bak | tail -n +$((<n>+1)) | xargs -d '\n' rm -i"

Of course it is better to put all this one-liner in a file to execute it.

Let's automatize it

With the previous scripts, backups become easy to make. However, it would be better if there is no need to remember to do backup and that they are done automatically. Cronjob is a very good option to automatize script execution

Once you have installed cron in your machine, let’s see how to use it.

To list all the cronjob (all automatic task) simply use:

crontab -l

New automatic tasks can be added by:

crontab -e

Each task as the following format

# Minute(0-59) Hour(0-24) monthday(1-31) Month(1-12) weekday(0-6) Command

Which corresponds to the following possibilities

# Execute every minute
* * * * * /path/to/script/to/execute

# Execute two hours
* */2 * * * /path/to/script/to/execute

# Execute every 3rd day of the month
* * 3 * * /path/to/script/to/execute

# Every week on Monday at 1a.m.
* 1 * * 1 /path/to/script/to/execute
# or
@weekly /path/to/script/to/execute

You can execute your script at the time you want. To verify that the execution is correctly done, it is advised to test it by executing the script every minute before adjusting to your needs.

Conclusion

Backing up important data is important to avoid any future problem. By making your scripts or using command such as rsync, you can back up easily your data. By using crontab it can be automated easily. Therefore you should put a backup policy for your personal data.

Thank you for reading this article. Hope to see you soon.

Sinople.

social