System Maintenance

RSYNC is probably the best backup tool I know. the speed of which it can compare two directories with over 250GB of data and 100,000’s of files to determine what the differences are is incredible; it can take literally seconds!!!

Below are two commands I use for backing up onto an ext3 partition and FAT32 parition

First of all you need to know the drive mount point of the directory you want to back up and the mount point of the directory you want to back up to. I have all my data in /data/data directory. The external ext3 formatted hd is mounted as /media/xxxxxxxxxx, its full path is /media/xxxxxxxxxx/Data/data.

Trail run;

rsync -vrutn –progress –delete /data/data/ /media/xxxxxxxxxx/Data/data

The real deal;

rsync -vrut –progress –delete /data/data/ /media/xxxxxxxxxx/Data/data

/data/data/ refers to the contents of the directory /data/data would reference the directory itself.
rsync is the application
We are simply replicating the data on another drive; not compressing the data.
v is verbose mode. Gives you lots of information as to what it is doing.
r is recursive. It will back up the contents of every directory found in /data/data irrelevant of where it is found in relation to /data/data. Omitting this will only backup the files within/data/data
u is update only. Files which are newer or the same file are not backed up. This means that we are only adding in any changes from the last backup.
t preserves the modification times. This means that the target files will take on the time stamp of the source files and not the date/time they are copied onto the destination disk. If you were syncing both ways omitting this would leave you with constant changes as each copy would add the files with a later time stamp.
–progress provides a good indication of how far through the progress we are.
–delete will delete files and directories on the destination disk which do not exist on the source disk.
n allows a dry run (trial). Along with verbose it allows you to understand what will be copied and deleted as part of the update.

Fat 32 / NTFS might not behave as nicely. Time stamps across file systems is not as reliable. For example according to the rsync man page FAT32 has a time resolution of two seconds. The ext3 filesystem according to the ls –full-size command reports a lot more precise time recording; 17:18:01.943927692. This is 1 billionth of a second; Surely this is not right? Can anyone enlighten me?

We have a few options. You only need use one of these. If you don’t you will find that rsync appears to completely back up your data every time and you lose the benefit of the pushing only the differences.

–size-only compares the sizes of files and nothing else. 1 character in a text file adds 1 byte; enough to trigger the change. I use this option and it works fine for me.
–modify-window=x where x is a positive integer. This allows a time difference of up to x seconds between the source and target files without them being considered different. A value of 1 would work for FAT32, maybe 2 for good measure.
–checksum will force rsync to provide a 128-bit checksum on both the source and destination file and update files where the checksums are different. This obviously adds a lot of overhead onto the backup process and the time to back up will significantly be longer.

Rsync normally determines file differences between file sizes and time stamp variations. –size-only option removes the time stamp check and only looks at file sizes. This might sound a little dodgy but it is your only option. For a sanity check and for some reassurance create a new text file and add a few characters. In the terminal expect its size via ‘ls -l’. Add a character into the file and check the size again. 1 char is one byte. If you need more precision then I would recommend formatting

luke wickstead

snippets for your thoughts….

FSCK

rsync to back up a linux system to ext3 or FAT32/NTFS