| July 8, 2010 11:45 am

Time DriveInnovation happens in fits and starts.  No single person, no matter how brilliant, capable or amazing is able to consistently create awesome stuff day in and day out.  Whenever engaged in any creative endeavor, there will be hills and valleys in productivity.

These happen for lots of reasons: maybe you’ve hit the bottom of the ideas barrel, or lost the source of your inspiration.  But there is another cause: you’ve reached your goal or solved the original problem.

Reaching such a plateau is a good thing.  It means you’ve accomplished something.  It’s a moment for patting yourself on the back, looking over your accomplishments, and otherwise appreciating that you’ve done something.  But at the same time, it doesn’t mean that you should sit still or stagnate.  So, in addition to looking at what you’ve done, it’s also important to take stock of the path ahead and start formulating plans.  You might even want to find a new challenge.

And this is where Time Drive has been for the better part of six months.  Ever since releasing version 0.3, there have been few changes to Time Drive.  Sure, both Philippe DeLodder (the other amazing Time Drive Developer) and I have fixed a few bugs and made tweaks; but nothing really major.

My personal reason for this is simple: Time Drive meets my needs.  It does a good job of backing up my data.  It lets me copy things to an attached hard drive, or across the internet.  It makes it easy to search for and restore my files.  In short, I’ve reached my original goals.  I use Time Drive every day, and it’s saved my digital bacon a couple of times.

But that doesn’t mean that development is finished, or that I don’t have ideas.  Because, I do.

Even though much of the original problem has been solved, I’ve found a new one to work on; and given that you use the current version, I thought you might be curious to hear a little about it.

(In addition to introducing the new version of Time Drive and making a few other project announcements.)

New Features

But before I jump into any discussion of the future, I should probably mention that there is a new version of Time Drive.  This isn’t exactly news if you follow the announcements on the project homepage.  The source code for the new version (0.4) has been available since April 17t  (But, it’s always good to have a more formal announcement. )

Here’s what’s new:

  • Remote Folder Backup
  • Easier Installation for users of Ubuntu 10.04 and 9.10

Backup of Remote Folders

The big new feature is the ability to backup remote folders.  Time Drive has always allowed you to backup folders on your local machine, but there are times when you might also want to also include folders from another computer on your network or across the internet.

Well, now you can.  If you can connect to the computer via ssh, you can use Time Drive to backup any folder on that machine.  I regularly use this feature to make a backup of my website, but it can also be used to backup a server or shared drive to another redundant location (such as Amazon S3).

Note: Remote Folder Backup requires SSHFS.

Remote folders allow you to backup folders on other computers over ssh.  It requires sshfs.

Easier Installation for Users of Ubuntu 10.04 and 9.10

The second big feature is a much improved installation process.

If you’re using Ubuntu 9.10 or 10.04 (and really, you should be), you can now securely install Time Drive from the ppa with just three simple commands.  No longer will you need to  manually set up a software connection and then download the signing key.  Ubuntu has everything completely automated.  Here’s what it looks like from the command line:

sudo add-apt-repository ppa:time-drive-devel/stable
sudo apt-get update
sudo apt-get install time-drive

First you add the time drive ppa (first line), then you update your sources (second line) and then you tell Ubuntu to install Time Drive (third line).  All of the needed files, dependencies and packages will be downloaded for you.

Note: The process for installing from source remains the same and is explained on the Time Drive home page.

Time Drive and Mac OS X

With the new features out of the way, I need to make a project announcement.

For a while now, Time Drive and Mac OS X haven’t really gotten along.  I’d like to say that there is enough blame to go around, but well … that’s not true.  This is Apple’s fault.  With the release of Snow Leopard, Apple broke their entire operating system.  They completely abandoned any ideas about backwards compatibility and changed the internal workings of everything.

If Snow Leopard were a house, I would say that they gutted and completely renovated.  The plumbing got moved, the electrical rewired, and the heating revamped.  Very little was spared.

While these renovations have allowed them to make some nice improvements, they also come at a cost: a lot of third party software has stopped working.  By stopped working, I mean just that: it no longer runs.  (Or if it does run, it does very strange things.)

Included in this carnage are the Python bindings that Time Drive use for its GUI.  The entire situation is a deep technical mess.  Nokia (the makers of Qt) and Riverbank (the makers of PyQt) are trying to fix it, but the repairs have been a long time coming.  But it’s worse than that, the situation has become political, too.

It looks as though Apple has decided to stop supporting cross-platform toolkits, like PyQt.  The most obvious example is the banning of Flash from the App Store, but if you poke around, you can find evidence that what applies to iPhone also applies to Mac.  I’ve become so frustrated by the PyQt situation that I tried to contact Apple and get an answer about if/when/whether Nokia and Apple would resolve the current problems with PyQt.  Here’s what they said (paraphrased heavily):

Apple does not comment on or support third party toolkits.  The only supported tools for Mac development are XCode and the Cocoa frameworks.

Translation:

Apple does not support Qt, and you shouldn’t be using it.  If you want to be on our platform, you’ll use the tools we tell you to and be happy about it.  Otherwise, you’re free to go elsewhere.  And just to be clear, you probably won’t be missed.

I’ve waited the better part of a year and PyQt on Mac is still a mess.  More than that, Python on Mac is a mess.  It doesn’t work, and the only real solution involves compiling the whole technology stack from source.  This is painful, and I’m not going to do it.

I contacted Apple to ask about my concerns and got a lot of PR nonsense and ugly attitude.    Life is too short to put up with Apple’s bullshit, and I’m not going to.  Thus:

Mac OS X is no longer a supported platform for Time Drive.

To users of Time Drive on Mac, I really am sorry.  I’m a Mac user too (albeit a heavily disgruntled one), I think it’s a good platform, but I don’t want to spend more time fighting this particular battle.  If/when/whether Nokia and Riverbank get PyQt back up and running on Mac, I might reconsider; but right now, it’s not something I want to worry about.

The Future: File Synchronization

Right, so that takes care of new features and announcements, here’s where Time Drive will be headed in the future.

Backup and archiving files go hand in hand with another important task: file synchronization.  If you look carefully at the best online backup tools, Dropbox and Windows Live Mesh, they not only copy your data to the cloud, but also to other computers.  Synchronization really is the third leg of this particular stool, and it’s really important.

For that reason, future versions of Time Drive will support a new kind of repository: one that supports common backup and synchronization between different computers.  The other features of duplicity will also be maintained: it will still offer encryption, delta copy, and a slew of backend options.  But in addition, will allow you to sync your files anywhere and everywhere.

What’s even better, Time Drive won’t tie you to a particular service.  Do you have a server with space?  You can use that, no need for an additional account.  Want to backup to Amazon S3?  The new version of Time Drive will let you.  My goal is to give you control of your data.

I’ve been working on the code for a few months and should have a version ready for testing in the relatively near future.  Expect more information soon.

Comments

6 Responses to “Time Drive 0.4: Foundations and the Future”

Golan wrote a comment on July 9, 2010

I like Time Drive a lot, but there are a few of features I hope you can pencil into your future development plans:

1. Backup profiles. I want to back-up both to an external hard drive and a remote folder. I’d like the remote backup to be scheduled to run daily but the external hard drive manually when I connect it. Since remote backup/restore is slower, I’d like to have a smaller set of crucial files, while I can go pretty wild with the local backup. At the moment I have to change settings between backup methods, while having profiles would solve this problem very neatly.

2. Backup as root. In addition to my private files I also want to back up a few other files as well, such as my xorg.conf file or my /var/lib/mysql folder. I know I can use kdesudo or gksudo to run Time Drive as root, but I think it would be better if I could input my root password somewhere in Time Drive if I want to backup folders or files outside of my home folder. I’m not sure if it’s technically feasible, but it would be a nice feature.

3. Going along with #2, I’d also like to be able to choose to backup specific files rather than folders. For instance, In want to backup my hosts and fstab file, but I don’t need the whole /etc/ folder. It’s important (to me) that Time Drive keeps the folder structure, so if I backup fstab, hosts, xorg.conf and /etc/apache2/sites-available, I want them all to be under /etc in the file list.

Time Drive can already do things I want a backup program to do which others don’t (backup to ftp using ssh, for instance), I think you should look into a bit of marketing as well since I think more people should hear about Time Drive.

Golan.

Rob Oakes wrote a comment on July 9, 2010

Hi Golan,

Thanks for the feedback. All thoughts are very much appreciated. I think that these three features would be very nice additions. Both options 2 and options 3 would be relatively easy to implement as well. For that reason, I’ll open blueprints and try and get them shoehorned into the next version.

Option 1 is a bit trickier, but certainly doable. I’m already going to be reconfiguring how the advanced settings work (for the file synchronization support), and will try and figure a good way to support multiple backup profiles.

(I don’t suppose you have an example program that does what your looking for? It doesn’t have to be backup related. The best profiles management I’m aware of happens in Kate.)

The one concern that I have with profiles is that they often turn into a complicated mess. The best implementation of which I am aware is Kate, but even that doesn’t really approach “elegant.”

Rob

Golan wrote a comment on July 10, 2010

2 + 3 will definitely be nice as starters!

As for 1, I think calling it “profiles” was a bit misleading. It might be more correct to refer to it as “backup sets”. Grsync has something similar. If I often sync folder A to folder B and folder Y to folder Z, I can save them as Sessions and then select a session from a drop-down list instead of inputing the folders every time I want to sync.

What I’d like Time Drive to do is to create a backup sets, each one with its own settings, instead of having global settings like it has now. I’ll define the first set to backup to a remote server and therefore I would like a smaller set of crucial files to be backed up when using that set. Since I’m on a broadband I’d like this set to be scheduled to run daily.

Then I’d like to have a second set that backs up to an external hard drive. Since it’s faster I’d like to backup my entire home folder, but I want to run it manually whenever I have the hard drive attached.

Then, I would like Time Drive to have a list of my defined sets with a list of actions I can perform on each set (Edit, Delete, Run Now).

Thinking about it, the Symantec backup program (forgot how it’s called) allows you to do something similar and it calls it jobs. We had it in the office where I previously worked and although I think both jobs ran on the same set of files, one job ran a daily incremental backup to a smaller hard drive, and another job run monthly doing a full backup to a larger hard drive.

Does it make more sense now?

Golan

Rob Oakes wrote a comment on July 10, 2010

Hi Golan,

Thanks for explaining it in more detail. This makes a lot more sense. Nor do I think i would be all that difficult to implement. Here’s what I’m thinking. Right now, Time Drive determines the scope of the backup jobs based on the “Include” folder list.

For all intents and purposes, though, each folder is really it’s own little world. It runs in a separate duplicity job and can include it’s own list of exclusion criteria, files and folders. You can also specify where you want the backup to occur. You can either use the General Settings, specify a local backup directory, or use a remote directory (SSH, WebDav or FTP) (in addition to providing Folder Specific Encryption Keys).

The only thing that the folders have in common is that they run sequentially whenever Time Drive executes a backup command. After the backup command executes, however, it’s all duplicity from there.

If you wanted to, right this minute, you could include the same folder multiple times. Let’s say you wish to back up your home directory to both Amazon S3 and a local hard drive, for example. To do this, you would simply need to add it to the include list twice. Then, you modify the advanced settings of the folder to point at the different backup destinations.

The only thing that really seems to be missing is the ability to give the “Include” folder a custom name and to schedule when it will run. But in both cases, those are pretty easy modifications to make. As a final touch, we might just rename the tab to “Backup Jobs” instead of “Include” to make things a little more obvious and provide a “Run Now” button.

What do you think, does this get pretty close to what you need?

Cheers,

Rob

Golan wrote a comment on July 10, 2010

That sounds very good to me.

I never noticed the Advanced button in the Include tab, and even if I would I’d probably wouldn’t think it would have the option to define a different backup location. I will experiment with those settings in the next few days and see how far it takes me.

Thanks for taking the time to discuss that, and (for Time Drive generally).

Golan.

Sagar wrote a comment on June 26, 2011

Hey Rob,

I have been searching for a good backup utility for my Linux boxen for a while. Time drive seems a very good solution and I am raring to give it a go on my Ubuntu Lucid desktop. I just wanted to add that I found your blog posts about the design and releases very useful and, in a sense, reassuring [any guy who writes so much about what is on his mind can be trusted to a practical extent : ) ]

+1 for feature request #1 from Golan above. I realize it is a little late to jump into this conversation, but an open source software called fwbackups supports this feature, which is referred to as ‘backupsets’.

Another thing. Time Drive needs more publicity. I googled for linux backup solutions and found multiple ‘N best backup utilities for Linux’ but found no mention of Time Drive. I found about it while browsing through the comments for Deja Dup, which is a front-end for duplicity. This is a pity, since Time Drive sounds so much better.

Now I’ll shut my mouth and actually install and test-drive Time Drive.

Cheers,
Sagar

Care to comment?