| August 7, 2009 1:39 pm

Time-Drive-Icon[11]In part 1 of this article, I shared a few of the frustrations and reasons why I decided to write my own backup utility rather than submit to the tyranny of currently available solutions.  While some might find those ruminations interesting, the vast majority are probably far more interested in the end result.  There is a reason why “Get to the point” is one of the most important sentences in the English language.

Here’s the short version: After becoming tremendously frustrated by the state of backup on Linux, I decided to take matters into my own hands and create my own tool.

And though I only want a few things, I want that tool to do each very well.  First, I’m looking for a solution that can incrementally backup over the network and let me restore a file from an existing snapshot.  Second, those snapshots should be compressed, encrypted and secure.  Third, it should be easy to browse old backups for existing files and restoration should be a one-click affair.  Fourth, I want a backup system that can protect me from disaster, carelessness and pathological stupidity.Time Drive is based on the excellent command line program, Duplicity though it was also inspired by another backup utility, Back In Time.  Because it uses Duplicity for its backend, Time Drive plugs into the same industrial strength utilities used to backup and restore enormous sets of enterprise data.  It just removes the industrial level complexity.

Like Back In Time for Linux and Time Machine for Mac OS X, Time Drive not only keeps a copy of the most recent file, but it remembers how that file looked at many points in the past.  Put another way, it creates an incremental snapshot which lets you restore to a previous version should you so desire.

The Time Drive main window is designed to be simple and functional.  Options related to backing up files are on the left, options related to restoring files are on the right.  If you want to restore from a remote archive which is not currently specified in your settings, you can do so through the “Advanced Restore Options.”

Time Drive - Main Window

Over the Network or Across the World

Backups can be made to remote servers and web services that support any one of the following protocols: file transfer protocol (ftp), secure shell (ssh), secure copy (scp), rsync, webdav (http), and secure webdav (https).  Time Drive also supports backups to a locally attached drive or mounted volume.  This includes shares that are mounted with virtual filesystems like sshfs, smbfs, and s3fs.

You can choose where you want to backup your files under the "General" tab of the settings dialog.You can configure the remote backup by clicking on the general tab of the settings pane.  Then, move the option to “Use a remote directory (SCP, WebDav or FTP).”  From the drop down list, choose the communications protocol.

If making a backup to ftp or webdav (http), enter the name of the host and the path to your backup folder as a standard web url.  It should look like:

computer.name.com/path/to/folder

For example, the backup path to my webdav share is:

backup.oak-tree.us/RobOakes/Backup

If backing up by scp or ssh, you will need to use a second slash after the server name, for example:

backup.oak-tree.us//RobOakes/Backup

Next, enter the username and password that you use to access the server and some sort of passkey in the “Encrypt Files” box.  (While future versions of Time Drive will allow users to choose whether or not they wish to encrypt their files, this has not yet been implemented.)  When finished, you can check the connection by clicking on the “Check Connection” button to the right of url text box.  If Time Drive is able to connect to the store, a little green check mark will appear.

Restoring Files

Should you delete a file that you needed, you can quickly browse or search the archive to find it and restore it.  Need to restore a bunch of files?  You can add as many as needed to the queue and then restore them with a single click.  The restore dialog is accessed by clicking on the “Add files” button of the main window.  The restore dialog itself allows you to browse the folders available in your backup, and any subfolders of the archive.

Should you delete a file, Time Drive makes it easy to explore or search your archive.

But what if you need access to a file in your work archive and you happen to be at home?  Time Drive makes it easy to brose and restore from any archive, anywhere.  Simply type in the url and click “Add files” to enter the restore files dialog.

Time Drive - Advanced Restore[4]

Set It, Then Forget It

Time Drive can be used to backup as many, or as few, folders as you like.While Time Drive can be configured to work many different ways, file backup should ultimately be a very simple affair.  True, it’s nice to have more advanced tools available, but most things in life aren’t really all that complicated.

As a result, setting up Time Drive for the first time is designed to be as easy as possible.  First, pick where you want to perform your backups: either to the local machine, or across the network.  Next, enter in a passkey to encrypt your backup files and keep them safe.  Finally, choose which folders you want to see included in the job.  When finished, click “Ok.”

Because each folder is a run as a separate duplicity script, you can always re-add that folder at a later time and Time-Drive will update the existing snapshot with the most recent information.  All of the existing history will be there should you ever need to access it.

Time Drive can be scheduled so that it runs automatically and in the background.   You can also set it so that it notifies you when backup jobs start and when they finish.Time Drive also makes it easy to schedule automated backups through the use of the powerful crontab utility available on most *nix machines.  Because it uses cron, there is no need to have a dedicated background process.  (Though cron must be running for scheduled backups to work.)

Automated backups are configured by opening the settings dialog and navigating to the “Options” tab.  Then, just pick how frequently you would like the program to run: every hour, day, week or month.  When done, select “Ok.”

Notifications and Feedback

When looking at other solutions, I quickly became frustrated by the way which they handled user notifications.  In some cases, the program provided a level of detail that was simply overwhelming.  In others, it didn’t provide any feedback whatsoever; even when there was a problem.

With Time Drive, I’ve tried to split the difference.  It uses the notification system of Ubuntu 9.04 and will let a user know when new backup jobs start and it finishes, if there is a problem, it will let you know that too.  But for those who would rather not be bothered with such trivial things, the notifications can be  easily disabled through the settings tab.

Time Drive - Backup Started[7]

Beta Software

Even though Time Drive is relatively full featured and offers a convenient backup package, it is still beta software.  That means that it needs to be tested, beat upon and otherwise tortured.  As you do so, I would ask that you be a bit patient and mind the potholes.  If it doesn’t quite meet your needs, please leave a bug report or feature request on the bugtracker page.  Be as specific as possible and I will try to incorporate your ideas into the next release.

Conclusion

With Time Drive, I finally have a backup utility for Linux that does things right.  It can backup to a computer on the network or across the internet; keeps incremental snapshots of only the files that have changed; makes file restoration a one-click affair; and keeps my personal information safe.  What’s better, no tweaks or hacks are required to make it work the way that I want.  In short, I finally have something that “Just Works” without having to stick some stupid qualifier in front.
______________________________________________________________

If you are interested in downloading, testing or using Time Drive, you can find additional information on the project homepage.  This includes installation instructions for Time Drive, Duplicity and PyQt in addition to the program documentation.

Comments

7 Responses to “Backup for Linux, Done Right – Part 2: Time Drive 0.1”

Johan Folin wrote a comment on August 12, 2009

Hi, wonderful work! I haven’t had time to try it out yet, but I will in the next few days. I use an rsync-script at the moment which works quite well, but it’s cumbersome to manage. I have a question though: Is it possible to use a different port for ssh? I have a machine dedicated to backup storage that runs ssh on a non-standard port.
Keep up the good work!

kelly Householder wrote a comment on August 12, 2009

Looks like a great tool. Hope to see you add S3 into the mix at some point. I use S3 for much of my backups today so this would be a great way to leverage one tool for all platforms with a common storage.

Rob Oakes wrote a comment on August 12, 2009

Hi Johan, yes it is. I should probably add a dedicated port box to the user interface. You can specify a non-standard port for ssh by including in in the url.

computer.name.com:[port]//path/to/folder

If you use port 777 for ssh, this might look like:

backup.oak-tree.us:777//Backup/RobOakes

The same trick also works for ftp and the other communications protocols. Though I’ve broken them up into a couple of different boxes, Time Drive just combines the various pieces of information into a single unified Duplicity url. (I thought it would be a little easier that way.) So, you can use any of the tricks found on the duplicity man page.

http://duplicity.nongnu.org/duplicity.1.html

Rob Oakes wrote a comment on August 12, 2009

Hi Kelly. It’s actually in the works for the next version which should be released in the next few weeks (as soon as I can find some time to finish it). Duplicity already supports it (and it works wonderfully), but it’s handled slightly different than the other types of communications (ssh, ftp, etc). As a result, I need to add some additional code for S3 connections and it’s taken a little more time than I thought it would.

The next version will also include support for restoring files from older snapshots. I just finished up that particular bit last night and just need to test it.

Stan Armstrong wrote a comment on September 6, 2009

Last month my Ubuntu hard drive failed. I had been using Simple Backup. Unfortunately, sbackup was producing empty backup files every day. As we all know (now), SB does not notify you of anything. So I am looking for something I can actually depend upon. Time-Drive promises to be everything I want. However, even though I have installed Duplicity and PyQT and Time-Drive runs, it doesn’t do anything. It tells me it has started a backup, and then a few seconds later, it tell me it has successfully completed a backup. When I go to the backup site, what I find is empty directories. No backup has actually been performed. I don’t know how to begin to troubleshoot this problem.

Rob Oakes wrote a comment on September 6, 2009

@ Stan. I’d be happy to help you troubleshoot any problems. Unfortunately, Time-Drive is still brand new software (still 0.1.x), and as a result has a number of particularly sharp edges. I think you might have found one.

Just off the top of my head, I can think of two things. First, the easy one. Which version of Duplicity do you have installed? Go to the command line and type:

duplicity -version

It should be version 0.6.04 or greater. If it is version, 0.5.x, you’ll need to install the most recent version from:

32-Bit: https://launchpad.net/~duplicity-team/+archive/ppa/+files/duplicity_0.6.04-0jaunty1_i386.deb
64-Bit: https://launchpad.net/~duplicity-team/+archive/ppa/+files/duplicity_0.6.04-0jaunty1_i386.deb

If that isn’t the problem, this might be an instance of another known bug. Duplicity (and by extension Time Drive) really doesn’t like top level folders that have spaces in them (it backs up subfolders with spaces without problems). Thus, if your top level folder looks like this:

/Users/RobOakes/Documents

You will be fine, but if it looks like:

/Users/Rob Oakes/Documents and Settings/My Documents

Time Drive will fail silently. I’ve fixed this for the next version of Time-Drive, but other features (like Amazon S3 support and the deletion/renaming of snapshots) are not ready for prime-time.

Would you mind trying an experiment? Create a new directory without spaces and put some files in it. Then try and make a backup of that directory. If that doesn’t work, let me know and we’ll try and we can try and dig into the problem a bit more.

Cheers,

Rob

Matthai wrote a comment on December 23, 2009

Hi,

I tried time-drive under Ubuntu Karmic, but unfortunately it is not working – or I am doing sometnihg wrong (which is more likely).

I am trying to backup one directory to Ubuntu One, which is a local directoy (/home/user/Ubuntu One, but is actually cloud storage solution (similar to Dropbox). When you put a file in /home/user/Ubuntu One, it is synchronized with cloud.

So I went to settings and set:
Local Backup Directory: /home/matthai/Ubuntu One/time-drive-backup
Encrypt Files: “somepassword”. BTW: As far as I know, Duplicity enable to use GPG for encrypting. What about Time-drive?

Include (Folders in backup): /home/matthai/Desktop/important

BTW, “Auto-remove” tab is inactive in settings.

Then I clicked to a BackupNow button (diskette) and got a message that backup is completed.

Then I go to Restore files, create new directory (/home/matthai/Desktop/important2), TIme-drive says it has restored files, but /home/matthai/Desktop/important2 is empty.

What could be wrong? I am using Duplicity 0.6.06. BTW, help button for Time-drive is inactive – if you need help for writing Time-drive maual, please let me know. I also cannot rename or delete snapshot history and if I click to shapshot name (it is called Wed Dec 23 14:21:23 2009), I cannot see any files, actually hotnihg happens.

Any idea?

Care to comment?