Tentacled Monkey

While at work I had to backup a number of build templates and resource files to another computer on the network, a NAS device, for safe storage. Previously, our backup schedules were fairly non-existent for this set of data so I really had free reign to do what I liked to make sure it just got backed up simply.

I knew I wanted to use xcopy to do the actual data transfer over the network, but I wasn't so keen on simply copying raw data in excess of 3GB on a weekly basis. So I started digging around for a command line interface compression tool. Oh, and it had to be free, since I was at work. After a few minutes of searching I came across 7-Zip which touted better compression ratios than WinZIP and a command line interface.

7-Zip Command Line

The command line of 7-Zip is pretty complete, and allows for include lists and excludes using wildcards, all sorts of shite really. It did take me about 30 minutes of jacking around to make it grab the exact files that I wanted but I was able to get it to grab exactly what I wanted.

Setting up Your Backup System

Well, first head over to the 7-Zip download page and get a copy of the program for WinBlows. They actually have a version for damn near every OS out there but since I am focusing on Windows just get that.

After installing 7-Zip browse to the install directory and copy the 7z.exe file into your System32 folder. This will allow us to access the command line from anywhere. I suppose that we could have added the install directory to the system path, but my way is much easier.

Since I was looking at including a large number of files spread across a series of directories we are going to include the files in a separate list file. Also, since batch scripts are much more forgiving when ran from the root of the C: drive, create a backup_list.txt file there and list out all of the directories that you want backed up one per line. Here’s an examples of the file:

c:\Build_Files\V12Data
c:\Build_Files\system_files
etc...


Now you need to actually create the bat file that will run our 7-Zip & xcopy commands, let's call this backup.bat. Here is mine so far:

7z a -tzip c:\archive.zip @c:\backup_list.txt -scsWIN

This command will create a ZIP file (-tzip) called archive.zip in the root of the C: drive and add the directories listed in the C:\backup_list.txt file. Since I just used ANSI and not UTF-8 for the formatting I had to use the -scsWIN flag to tell 7-Zip, otherwise the command will die.

Now just running this command would be fine for most but I had a number of subdirectories that I did not want to include. These subdirectories are located at the same level and would all be similarly named. So to exclude these directories we need to add an exclusion to the command we already have. So now we have something like this:

7z a -tzip c:\archive.zip @c:\backup_list.txt -scsWIN -x!Build_Files\*\backup*\

This will exclude any folders that start with "backup" on the third level. Now I also needed to exclude another particular folder so my script came out to this:

7z a -tzip c:\archive.zip @c:\backup_list.txt -scsWIN -x!Build_Files\*\backup*\ -x!Build_Files\V12Data\Streets\

Now that we have the compression command finalized we can worry about the actual xcopy.

xcopy c:\archive.zip Y:\Data\ /D /Y

I am using xcopy cuz it rocks and it is present on every XP machine, and I am not so sure RoboCopy is. Anyway, the xcopy command initiates a file copy from the compressed file we just created, c:\archive.zip, to a map drive location that I have already set up. The /D switch tells it to only update the file if it is newer than the one I am attempting to overwrite; while the /Y flag tells it to suppress any warning flags.

Next I want to get rid of the compressed file on the local machine so we run a simple remove command and then exit the script.

rem c:\archive.zip
exit

Now we can run the batch file and see if it is all kosher, it should be but if not generally your error will be in the 7-Zip line for some syntax error. Run the batch by right-clicking on it and left-clicking on "Run." You will see a command prompt window appear and start compressing your files. I would let it run one full time to make sure that your copy and rem command is working correctly.

Scheduling the Script

Once the script is finalized we can add it as a scheduled task on the local machine. I set mine up to run at 2:00AM on Sunday mornings. Again we will probably want to verify that it works by right-clicking on the task and choosing to run it. It will run in the background without showing a command prompt.

Do yourself a favor and make sure that you are backing up to either a separate machine or an external hard drive. There is nothing so ignorant as making sure you are backing up your stuff but then just placing it on a separate partition of your only hdd.

Anyway, I hope that this helps someone else trying to backup large chunks of data on a WinBlows system/Network.