When writing on a computer, the tool of choice for many writers (dare I say most?) is Microsoft Word. It’s everywhere and everyone has used it. It comes preinstalled on most computers and is a de-facto standard for exchanging written material with others.

Unfortunately, Word is not part of my preferred toolset. I prefer to write using LyX and an add-on I’ve written for it called LyX-Outline. But while I love my writing program, it makes it difficult to collaborate with other writers who use Word, as LyX doesn’t have a straightforward way to directly import Word files.

This isn’t a new problem and I’ve written about it before. I’ve even proposed a solutions. But while that solution was a good fit for me, it isn’t something that I would recommend to others.

For starters, it required a great deal of software to be installed. You needed a program to convert Microsoft Word documents to Open Office documents. You then had to use a second utility to convert it to HTML or LaTeX. After that, you used to a third utility to clean it up and import the LaTeX code into LyX. Three distinct steps, with a lot of places where things could go wrong.

Over the past few months, I’ve found that I need a better way, a tool that can directly import a Word document and cleanly translate its content. So, I decided to create one.

Project Goals

I wanted my tool to meet several important criteria:

• it should be easy to use and maintain
• it should maintain document structure including headers, styles and other elements
• it should successfully translate Word to LyX, meaning that I don’t need to spend time repairing double quotes or fixing em-dashes
• it should support tables, images, and footnotes
• it should be extensible, allowing me to create custom templates that support a wide variety of Word documents and LaTeX document classes

After several months of work, I’m now ready to release an alpha-quality version of that tool. I’m calling it word2lyx.

word2lyX

word2lyx is a python script to convert Microsoft Word documents to a format that can be imported into LyX. word2lyx only works on documents saved in the docx format, created using Word 2007 or 2010. Older in doc will have to be converted before word2lyx will be able to read them.

But here’s the important thing, it works directly on the Word file. It doesn’t rely on external parsers for translation. For that reason, you can specify exactly how you would like for your documents to be converted.

Currently, word2lyx supports:

• Translating Word paragraph and character styles to LyX paragraph and character styles. In cases where the character styles aren’t defined, it will write entries for them  in the local layout. These entries will include basic LaTeX font commands, created from the font properties.
• Importing Word tables, including those with merged rows or columns. it will also do its best with table borders.
• Enumerated and itemized lists
• Importing images from the Word document. Word image objects, such as Excel charts and graphs are skipped.
• The use of custom templates, which allow you to finely describe how you would like for your documents to be translated. There are two templates included with the script and supporting libraries: article.w2l and book.w2l. These can be found in the lyx/templates folder. (For more about templates, please see below.)

For best conversion, please make use of paragraph and character styles. These tags are  converted directly. Per word formatting is removed. This makes it easier to create  clean markup for LyX.

Usage

To use the script, use a variation of the command below:

python word2lyx.py “InputFile.docx” “OutputFile.lyx”

You can also specify a template you would like for it to use:

python word2lyx.py “InputFile.docx” “OutputFile.lyx” -t book

Right now, word2lyx includes two templates: one for articles and another for book chapters. These can serve as a base for importing your work into LyX. After you’ve got it loaded, you can then change the document class and begin to customize it from inside of LyX.

Downloads and Installation

If you think word2lyx sounds interesting, you can download it here. The download includes the script and two supporting libraries, one for reading docx files and one for writing LyX files. Additionally, it has a sample document which will show off most of the script features. To install word2lyx, put the files somewhere inside of your python path. (Or, when you run it, you can just type the full path to the script.)

Automating Document Conversion

Though using the script from the command line is pretty easy, a better way is to automate the conversion process. To do this, you need to create a new LyX document converter. Step-by-step instructions for adding additional file types and converters to LyX can be found in the older post I wrote on converting Word to LyX. Here, I’ll repeat the most important information.

First, go to the File Handling preferences (Tools > Preferences > File Handling > File Types), and add a new type called Word 2010. Set the file ending to be “docx.”

Next, go to Converters dialog and create a new “MS Word 2010” to “LyX” converter.

From format = “MS Word 2010”, To Format = “LyX”. Converter setting:
python “/path/to/word2lyx.py” $$I $$o –t article

Once you’ve added these settings, you can then import a document by going to File > Import > MS Word 2010.

word2lyx Templates

word2lyx templates are files which tell word2lyx how a particular type of document should be translated. They include lists of paragraph, character, and table styles and their LyX  equivalents. Consider the example below, which is taken from the article template:

ImageDir = ‘Images’
IgnoreStyles = FootnoteReference, EndnoteReference

[ParagraphStyles]
Title = Title
Author = Author
Part = Part
Heading1 = Chapter
…

The template file contains settings and sections. The example above includes two settings, ImageDir and IgnoreStyles, and a group of paragraph styles. The paragraph styles specify how Word styles (on the left), such as Heading1, should be translated into LyX styles, such as Chapter, Section, etc.

When you create a new template (or extend one of the existing options), you add additional styles so that word2lyx knows how you want those particular pieces of information to be processed.

Each word2lyx template includes several sections. These include:

• ParagraphStyles: Matched pairs that convert Word styles (on the left) to LyX styles (on the right).
• CharacterStyles: Matched pairs that convert Word charstyles (again, on the left) to LyX charstyles (again, specified on the right).
• TableStyles: Similar to paragraph and character styles, but used in the conversion of tables.

Sections are defined by a section tag, enclosed with square brackets. Values in the section are entered as matching pairs, offset by an equal sign.

If you choose, you can add an additional section called [DocOptions], where you specify information such as the document class, which types of fonts you would like the document to use, and so forth. (Please refer to the LyX documentation for avaialble options.)

If you create new templates, you should place them in the lyx/templates folder for word2lyx to be able to find them. They should have a ‘w2l’ file extension.

Conclusion

I’m posting this utility in the hope that people will find it useful. Please, download it and take it for a test drive on your documents. Let me know about any successes or failures that you might have. (Especially the failures, those are important to fix.) If there’s a feature you would like to see included, please leave a comment here. I’m looking forward to hearing if word2lyx helps make collaboration a little easier for you.

Importing Word Documents Into LyX (word2lyx 0.1) is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

Human lives are complicated things. When well lived, they encompasses moments of beauty, love, friendship, and happiness in addition to tragedy, lost opportunity, and regret. (And my grandfather’s life was well-live.) What’s more, we human beings are embodiments of paradox: a man may be both a great philanthropist and a brutal monopolist, who built his fortune upon the shattered careers of others. Or he may be a vicious murderer and anarchist who “loved sunflowers, eating yogurt with honey; who took his children to the beach, and let them sleep under the stars.” How do you capture all of it: the good, the bad, the ugly, the contemplative, the tragic, and the hilarious?

The only answer I have been able to find to that question was given by Neil Gaiman, while being interviewed by a reporter about someone still very much alive:

I spent half an hour yesterday talking to a reporter who was working on the obituary of someone who is currently very much alive, even in good health, and it was … well … very odd. I’d always known that obituaries aren’t just knocked up (or perhaps tossed off. Look, neither phrase sounds particularly wholesome) on the spur of the moment by some dusty, but hard working, obituarist whenever someone kicks the bucket, but they are written ahead of time, often rewritten many times over the years. The Daily Telegraphs are the best, I don’t why this is, but it’s true. Here is my favourite Telegraph obituary ever, because it contains:

Despite a brisk code of discipline, Singleton took a laissez faire approach out of the classroom. Every November 5, the smallest boy in the school was sent down a tunnel to light the very core of the bonfire. None, so far as anyone can recall, was ever lost.

Good obituaries (and by extension, eulogies), it appears, is much like any other sort of writing: a careful product of thought, time, review, and attention.

That is, perhaps, why they are such an art-form. Unlike other types of communication, they have to be composed in moments of pandemonium, distress, and chaos; at a time when it’s essential to find precisely the right words under conditions which make it all but impossible.

Manifestly unfair, if you want my opinion.

Obituaries and Eulogies is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

There is really only one thing left to talk about: conflicts, errors, and their resolution.

Conflicts: The Headaches of Version Control

Conflicts are incompatible changes made to the same file. They happen when someone modified one of the files you are working on and committed changes, or, when a file was moved, renamed, or deleted with changes still pending. They typically come up at the most inopportune of times, such as when you’re getting ready to head home for the day but want to commit the changes to your draft.

Here’s what happens: you’ve been working all day, maybe you’ve finished an important section needed by one of your co-authors. You’re excited to see your spouse and children, maybe you’ve got evening plans. But, when you try and commit your changes, you discover that you aren’t able to. Subversion tells you that other commits have been made and you need to update your files before you commit your changes. You run “SVN update,” which is successful … mostly. As you scan the log of the update, you see an error telling you that one of the files has a conflict, and you won’t be able to submit your revisions until the conflict has been resolved.

A “conflict” happens when incompatible changes are made to the same file. They have to be “resolved” by merging your changes with the most version recent of the file on the server.

When they happen, conflicts seem big and scary. They demand that you deal with them (since it’s impossible to submit additional commits until you have). But even so, they aren’t anything to be afraid of. Resolving most conflicts involves nothing more complicated than merging two files (the “trunk” version from from the repository and your local copy).

Indeed, you might even think of conflicts as a way of protecting your work. They prevent Subversion from doing mindless things. As an example, consider the screenshot below.

The file was moved to a new location in the repository, as part of a restructuring of the book it belonged to. However, this happened before I was able to commit changes to it. On the machine where I had made the changes, Subversion moved all of the files but kept my local copy of this one, protecting the work I’ve done. In this sense, I was very happy to have the conflict.

With that said,  it is better to prevent conflicts through the use of locks and good communication. Event though they’re meant to prevent problems, they’re something of a pain (and they always come up at inopportune times). But when they do come up, knowing how to resolve them without losing your work is a very important skill.

Types of Conflicts

In general, there are two types of conflicts:

• File conflicts, which happen when incompatible changes were made to the same file.
• Tree conflicts, which happen when one person has made changes and another moves it before those are committed.

The conflict from above is a tree conflict.

Resolving Conflicts

The path that you take to resolving the conflict depends on what the conflict is and what you would like to do about it. For file conflicts, it’s usually pretty simple:

• compare your copy of the file to the version from repository
• find the differences between the two using a diff tool or another program
• save a new copy of the file with both sets of changes in-tact

For tree conflicts, it’s similar, but with one additional step:

• figure out what caused the conflict (whether the file was moved, deleted, or renamed)
• determine what files need to be compared
• make the comparisons and merge the differences
• save a new copy of the file with the changes in-tact

In both cases, the key is to compare your copy of the file to the version from the repository. From there, you know what has changed and can generate a new file with the changes from both.

To learn more about the conflict and begin to resolve it, you can click on the “Edit conflicts” link from the TortoiseSVN context menu.

Text Files

If you are working with text files, this is usually pretty easy. You can right click on the file with the conflict (which will have a warning icon, as in the screenshot above), and then select “Edit conflicts” from the menu. TortoiseSVN will launch a merge tool, such as TortoiseMerge, which allows you to see what has changed and what has not.

Nearly all Subversion clients come with a merge tool, such as TortoiseMerge, which allows you to compare text files and see what has changed. These tools are helpful for resolving conflicts as they will allow you to incorporate changes from the repository file and from your local draft.

Mac OS X also includes a tool to merge differences, called DiffMerge. It works in much the same way as TortoiseMerge. It loads two text files side by side and you then choose the components that you wish to keep from both.

Using the merge tool, you then select which revisions you want to keep and those that you would like to discard. You can even go in and edit the conflict between the two files by hand. When you are finished, you can save the new file and mark it as “resolved.” You can then go about your business and commit your changes to the repository.

For programmers, this method of resolving conflicts works wonderfully, as they primarily work with text files. It’s less magical for those whose repositories are filled with Microsoft Word documents, images, SVG drawings, and other types of files.

Other (Binary) Types of Files

For other types of files, comparing the two documents and generating a list of changes is a little bit more involved than just using the built-in merge tool. It’s still nothing to be afraid of, though.

For every conflict, Subversion places three files in your directory. They are:

• filename.ext.mine: this is the file as it existed in your working copy before you made the update (and caused the conflict). It has your latest changes in it, all intact.
• filename.ext.rNEWREV: this is the file that was downloaded from the server during the update. It corresponds to the HEAD revision of the repository.
• filename.ext.rOLDREV: this is the file as it existed before you made your changes or it was updated in the repository.

filename.ext is the name of the file with the conflict.

These three files are used to merge the different versions and generate a new file without problems. Here’s how you do it:

1. Rename the files. Instead of filename.ext.mine, the file extensions should come after the revision information: filename.mine.ext,
2. Open two of the three files in a program that can compare them and make any required changes
3. Save the new file with the changes over the copy with the conflict

Again, the most important step is number two, where you compare the two files and make any required changes. What makes this step different for binary files, though, is that you won’t always use the same program. You need to use one that is capable of comparing the file type.

If you are working with Word documents, then you will use Word to compare the two files and make the changes. If the files are Photoshop objects, you’ll use a program like ComparePSD. If you write with LyX, you’ll use LyX. Because the file formats are more complicated than simple text, you have to use a program that can read and parse them correctly. If not, you’ll make the conflict worse.

To get the changes you want, you also have to compare the right two versions of the file. To see how the latest version from the repository differs from the original, you compare filename.rOLDREV.ext and filename.rNEWREV.ext (remember that you’ve changed the names so that the program will read them correctly). To see how your version and the repository differ, you compare filename.mine.ext and filename.rNEWREV.ext.

In most cases, you will want to compare filename.mine.ext and filename.rNEWREV.ext.

When there is a conflict in a file that isn’t text based, you need to use a tool that is capable of making comparisons. That means, if you write with LyX, that you use LyX; Microsoft Word documents should be loaded in Word; Photoshop documents in ComparePSD; and so forth.

Using a separate program to directly compare the different versions of the conflict will create a new file with the differences you are interested in.

Tree Conflicts

If you have a tree conflict with a binary file, the process is similar to resolving a file conflict. Except, you won’t have access to filename.ext.mine, filename.ext.rNEWVERSION, and filename.ext.rOLDVERSION. Instead, you will compare the version marked with the error and the version downloaded from repository.

If the file was moved, or renamed, Subversion can help you to find where it got moved to. If using TortoiseSVN, you can access this information by clicking on the “Check for Modifications” dialog. If the file was deleted in the repository, you will need to resolve the problem using the repository browser. In this case, it’s a good idea to move your (locally modified) version to another folder for safe keeping. After you restore the version in the repository (using the repo-browser), you can copy it back.

Marking a Conflict as Resolved

Once you’ve merged the new and old versions of the file, you’re ready to submit your commit to the repository and get back to your life … almost. There is just one last step, you have to mark the conflict as resolved. This is done by clicking on the (now merged) file, and selecting “Resolved…” from the context menu.

The last step required to resolve the conflict, commit your work, and return to life is to mark the commit as “resolved.”

Before marking a conflict as resolved, though, make sure that all of the problems have been fixed. When you click on “Resolved…”, it removes all of the other versions of the files (.rOLDVERSION, r.NEWVERSION, and .mine) and prepares it to be committed. If there is a conflict you haven’t mediated, then you might be in danger of losing your work.

Summary

Though they may look scary, conflicts are actually a good thing. They let you know when there is a problem with your project, provide a way to protect your work, and provide you with a way to merge your changes into those of others. The secret to successfully handling conflicts is not to be afraid of them. If a conflict happens after you update from the repository, or merge from a branch, just deal with it:

• figure out what caused the conflict
• compare the different files
• generate a new, merged copy
• mark it as resolved
• commit the changes

Then, you can get back to your life.

Getting Started with Subversion – Part 4: Conflicts and Errors is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

Here, I want to reiterate one important point: creating drafts that can be consumed by others is extremely important. It forces you as an author to find break points where you can send something definitive. Finding these points, where you can draw a line in the sand and say “draft …”,  causes you to solidify your thinking and take an important step toward completion. You may end up throwing the whole thing away because it was ineffectual, but that doesn’t mean the exercise was futile. The process of creating something, a draft, is an enormous step toward completion.  You’ll likely take many such steps, but each one results in a better manuscript.

SVN branches can be a huge help in drawing your lines in the sand.

Version Control Branches

In software development, a branch is used when you want your development process to go into two different directions. For example, let’s say you’ve been working on a program that makes it easier to layout ideas and write a draft of a novel. You’ve finished working on all of the major features and would like to get feedback on how they work. To do this, you might create a branch of your project called “testing” that can be compiled and released to users. But at the same time, you’re not completely finished with the software. So, you keep working on new features in the branch called “trunk.”

Both “testing” and “trunk” have a life of their own. Though they share a common history, at the time that you create the branch, that history splits. This gives you a separate log for each, the ability to check-in separate changes for each. In effect, a separate future for each.

The “testing” branch gets compiled and sent to the users. They’ll find problems with it and submit reports. As these reports come in, you can correct the problems and make commits to the “testing branch.” While that’s happening, though, you can continue your work in the main branch to develop your “big ideas.” When it’s time for the two to come together, you can import the changes in the “testing” branch back into the “trunk” using an operation called “merge.””

Writing projects can also benefit from branches. To see how, let’s consider a simple example.

Let’s say that you’re working on a novel, but you’re not quite sure where to take the plot. You’ve already written one draft of it, but aren’t completely satisfied with the direction it’s going. For that reason, perhaps you want to solicit comments from friends and family. At the same time, you want to begin working on an alternative ending.

To solve this problem, you can branch your work for others to review. In essence, you create a “testing” or “review” branch. Your friends, editor, or co-authors can check out the files, read through the novel, provide feedback, correct typos, and think about the work you’ve already done. At the same time, in the other branch, you work on the alternative version of the story. Both you and your reviewers can work at your own pace because you both have a separate copy of the files. Each copy has its own history and log.

If you don’t like where you end up heading, you can abandon the “experimental” branch you’ve created and resume work on the original branch, once the reviewers have finished with it. If you prefer the alternative ending, you can merge the changes from the “testing” branch into what you’re currently working on.

Creating Branches

Creating branches is easy. Subversion makes uses of “directory branches,” which means that a branch shows up in the working tree as a different directory. But though they look like folders, there’s one important difference: a branch is a “cheap copy.” Instead of making a complete copy of every file in the repository, an internal link is created without duplicating data. This means that it’s quick to create branches and they take up almost no extra space in the repository.

Note: Throughout the remainder of this tutorial, I will be using TortoiseSVN to show how to create and merge different branches. The process is very similar for other Subversion clients, KDE-SVN on Linux. It’s also possible to create and merge branches using the command line.

To create a new branch, you select the folder in your working copy which you want to copy. Then, you right click and select “TortoiseSVN > Branch/Tag…” from the context menu.

TortoiseSVN will load the Branch/Tag window and prompt you to add some additional information. First, it will want to know where you would like to store the branched copy. The default location will be the URL which the working copy is based off of. You should edit that URL to something else.

A good practice is to have a folder in the repository where you store all of your branches. Depending on how you use them, you may even want to have multiple folders. Below, you can see the directory structure for a repository for a pair of books I’m working on.

The working text can be found in the “Open Source Writing” and “Open Source Design” folders. Example files, used in both books, are found in the “Examples” folder. Extensions and plugins for the programs covered (some of which I’ve written) are kept in the “Programs and Plugins” folder. I keep branches in one of several top-level folders. If I want to send a review copy, I’ll create the branch in the “Review” folder and then send that to those I’m working with. If I’m soliciting revisions form a collaborator, I may create a branch in a “Collaborators” folder. This keeps them separate from the rest of the project and provides a simple URL that I can give to others.

For this reason, I need to update the destination URL that appears in the “To path:” field of the “Copy (Branch/Tag)” dialog. If I were creating a review copy, I might change it to “draft-20120123”. I would also add a message to the log stating that I had created a review branch. Because I’ll be sending the URL for people to download over an HTTP connection, it’s generally a good idea if the paths to your branches don’t include spaces.

After you specify where the branch should be saved, you then need to decide if you want to create a copy from the current “HEAD”, “a specific numbered revision”, or the local “working copy”:

• If you create the copy from the HEAD, it will save a snapshot of the repository as it exists now. You won’t need to transfer any information from your working copy. As a result, the branch will be created very quickly.
• If you create a branch from a specific revision, it will use an older revision than HEAD. This is useful if you want to get comments on something that may have already changed. If you can’t remember the revision number, you can click on the “Show Log” button help find it. Again, no data is transferred and the branch will be created very quickly.
• If you choose the “working copy” option, the new branch will be made from your local copy. If you have updated some files to an older revision or made other changes, these will be sent to the new branch.

Working With Branches

Once you’ve created the branch, you work with it in the exact same way that you would work with the main text. You can create a local copy of the files by checking it out from the repository,² providing the path to the branch.

Once the files have finished checking out, you can edit the draft and have all of your changes tracked. When ready to submit the revisions, you commit them back to the branch history. To download additions that others have made to the branch, you update from the repository. Everything works exactly the same as when working with the main body of your text, which is really nothing more than another branch.

Merging Changes

At some point, the branch you’ve created will have served its purpose. You will have gotten the feedback you were looking for, your co-authors will have finished with their sections, or you will have completed your alternative ending. At that point, you will be faced with an important decision: what do you want to do with it?

If you created the branch because you wanted to solicit feedback, then it’s likely that you will want to merge the changes back into the main text. If you created the branch to the experiment with the text, then you may not. Or, you might only want to merge a subset of the changes.

Once you’ve begun using branches to solicit review or to create different iterations of your work, it is possible to merge the changes of one branch into another. Subversion provides you with three different options: you can merge a range of revisions from one branch to another, reintegrate all of the changes of one branch into the main working text, or you might want to integrate changes from two different branches into the version you’re currently working on.

Subversion provides you with tools to do all three. Once a branch has been created, you can allow for it to grow as long as you would like. If there’s no good reason to merge two branches, you don’t have to. When it comes time to merge the branches, though, you can merge the entire history of one – back from the time it was split off – into any other, or, you can choose to merge a range of specific revisions.

To begin a merge, you should navigate to the working copy that you would like to hold all of your changes. (Merging always happens in a working copy, which means that if you want to merge changes into a branch you need to have a working copy for that branch checked out.) Then, right click and select “Merge” from the TortoiseSVN context menu. (It’s generally a good idea to merge changes into an unmodified working copy, this will greatly decrease the chance of errors.³) This will launch the Merge Wizard. On the first step of the Wizard, you will be accessed whether you want to merge a range of revisions, reintegrate a branch, or merge two different trees. In most cases, you will want to merge a range of revisions.

From there, you specify which branch you would like to merge the changes from. This goes in the “URL to merge from” field. Then, in the “Revision range to merge” field, you specify which revisions you would like to include. If you want to merge the entire history of the branch into your working copy, just leave the box empty.

If you’re unsure which changes you would like to include, the easiest way to select the range of revisions is to click on the “Show Log” button. This will list recent changes with the log comments.⁴ If you want to merge changes from several revisions, select the range while holding down the “Shift” key. When you click “Ok”, the list of revision numbers to merge will be filled in for you.

When you have specified which changes you would like to include, click “Next.” The last step is to specify any special options and, optionally, test the merge to see if there any problems. Usually, I just make use of the default merge options.

If will be merging in a large number of changes/edits, it’s generally a good idea to test a merge to see if there will be any conflicts. Knowing which files have conflicts can help you plan how to resolve them. Typically, dealing with “conflicts” in a writing project means that you compare the two files and generate a third which contains all of the changes in both. It is relatively easy to do, but requires that you know which files have to be looked at.

The last step in merging changes is to specify any special options and to test the merge. This will help you locate any conflicts and generate a strategy to deal with them.

If there are no conflicts, you can start the process by clicking on the “Merge” button.

Summary

Though the idea of “branching” may take a while to warm up to, it is tremendously powerful. It gives you a way to create a snapshot of your work that can you send to others for review.

For a solitary writer, this is important because it requires you to package the draft for consumption and pushes you toward completion. You branch when ready for specific feedback which gets provided on a copy meant for consumption. The result of which is a better critique, more thoughtful advice, and better writing.
__________________________

Notes

1. For TortoiseSVN users, you can open the repository browser by right clicking on the folder you have under version control and selecting “Repo-Browser” from the TortoiseSVN context menu. If opened from your local working copy, it will automatically connect to the repository from which you checked out the files. If you open up the repository browser from a folder not under version control, though, you will need to enter the path to where it can be found.
   For Mac OS X Versions users, you can access the repository by clicking on the repository bookmark in the navigation pane. The repository bookmark will usually be above the link to the working copy and uses a small globe icon.
   
2. For instructions on how to checkout files from a remote repository, Getting Started with Subversion, Part: Basics.
3. When you use Subversion for writing, handling errors and complex merges is a bit of an art form and requires a bit more discussion. For that reason, we’ll look at it in the next article in this series.
4. This is a good example of why you should submit comments in the log which explain what changes were made. Using the log to browse for specific changes is much faster than looking at specific change sets to find the version of the file that you are interested in.

Getting Started with Subversion – Part 3.4:
SVN Branches and the Simplification of Review is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

• it’s possible to distribute tasks according to individual strengths, meaning that the finished product will (probably) be more than a sum of the parts
• brainstorming is more effective, more people means more ideas
• not only will you have more ideas, but as you discuss, challenge, and research the topic amongst the group, you will have different ideas than you might develop on your own
• having many people working on a project gives it energy and focus, which is tremendously helpful upon entering the hinterland of any project commonly known as “middle”

Collaboration is good, but it is also complicated. It takes a great deal of work for a collaborative project to be success. You have to balance competing needs against one another. On the one hand, it is really important to provide an author the freedom and space required to own her ideas. At the same time, though, you need to make sure that everyone is clearly communicating about the project and where it is headed.

Making sure that everyone is on the same page and that efforts are coordinated is a complex challenge. It requires meaningful discussion happens; establishing a system for sharing documents and knowledge; and that goals, scope, audience, and purpose of the project are well defined. In many ways it shares much in common with another complex endeavor, coordinating the care of a medical patient.

Communication Challenges

Helping a single patient get the care needed to treat a disease is difficult. For any one procedure or treatment, there are dozens of people involved – nurses, technicians, pharmacists, and support staff – and these people are in addition to the doctor who proscribed it. For the treatment to happen, the whole team needs to to know what’s happening and what their role will be. They need to briefed on relevant test results and potential complications. This is a big job.

In general, the healthcare professions do a terrible job of sharing this information with one another (at least in the United States). You can see the symptoms of the problem every time you go to the doctor’s office or hospital: patients filling out the same forms multiple times during a single visit, answering the same questions to every provider that walks through the door, and repeating the same stories to a dozen different people. But though communication in healthcare is a shambles, it offers some instructive lessons. To explore these further, let’s consider a relatively simple example: a primary care doctor sends a patient to a specialist for an expert opinion.

To have a good outcome, where the patient learns what is wrong with him and the referring physician learns what she should do about it, three things need to happen:

1. An appointment with the specialist needs to be scheduled and a copy of the patient’s existing history and the notes must be sent for evaluation.
2. The patient should be seen and evaluated. Specialty diagnostic tests may be ordered.
3. The specialist will determine a diagnosis and communicate all of this back to the referring doctor, along with a copy of his notes, test results, and a plan of care.

Though this might seem pretty easy, this simple chain of events rarely happens,¹ consider the results form a study ² published in January of this year. When asked in a survey about how often they pass along important documentation, 69% of primary care physicians always or mostly pass on a patient’s history and reason for a consultation a specialist. Fewer than 35% of specialists, however, report receiving them. The same thing happens in reverse: 81% of specialists said that they send consult results back to the referring primary-care doctor, but only 62% of primary-care doctors say that they receive the information.

Think about what this means for a minute. We are talking about information that doctors not only should be sending to one another, but that is often mandated by law. Before making a diagnosis, a specialist needs to know the full medical history and the impressions of the other physicians taking care of the patient. This is not always material that can be obtained from the patient directly, it’s stuff required to do the job. Yet, the specialists receive it less than half the time. The same breakdown in communication then happens in reverse with the same implications to the patient’s life and health.

Even if the communication happens, doesn’t mean that it’s effective. Sending stacks of papers and notes is tremendously inefficient. A chart contains a great deal of important data, but not everything a doctor notices gets written down. Busy specialists may choose to skim the notes and comments, missing a vital piece of evidence. It would be much better if one doctor would call the other and briefly talk about the plan of care. This verbal communication might then be supplemented with what’s available in the notes. Yet, from my own personal experience – after nearly a decade of working in hospitals and around doctors – has been that this almost never happens.³ As a result, essential information – data that can save a life or result in death, such as whether a patient is allergic to a drug – is simply left unsaid, and mistakes occur,hundreds of thousands of them each year.

Lessons for Writers

Collaborative writing, like medical care, needs to be coordinated. Writers require more than just than the text of a manuscript to be effective. Like doctors, they have to talk to one another. They should meet, find ways to share documents and knowledge, and ensure that everyone remains on the same page. Doing these three simple things will go a long way to ensuring that as you write, your collaboration will be effective and fruitful.

Now for some bad news: for the first challenge, there isn’t much that Subversion can do to help you. At the end of the day, successful communication starts and ends with a group commitment, you just have to work at it.

SVN, though, can help with the latter two challenges: by making sure that everyone in the project has access to all of the files (working drafts, reference documents, citation libraries, etc.), and that you have a centralized location to provide updates on how the work is proceeding. In fact, it’s a brilliant way to address both.

This is because the features which enable effective collaboration are the same that allow you to synchronize your work amongst multiple computers and provide a log for the history of your changes.⁴ When working with others, though, the secret lies in how you use them. Below, you’ll find two suggestions for getting the most out of them.

Suggestion 1: Structure Your Files

You already know that Subversion can be used to synchronize nearly any type of file to everyone else working on the project. Here is where you need to put that feature to good use. In addition to the drafts of the manuscript and images, it’s also a good idea to add other assets to the repository. These might include specification documents, articles, or other material which everyone will need access to.

If working on an academic manuscript or a piece of non-fiction, it’s especially important to add reference databases. Using the reference databases that others have created can save you an enormous amount of time, especially as you don’t have to go out and look up the citations on your own. Different groups each have their own way for managing reference databases, but my experience has been that it’s better if each user keeps their own as a distinct file. In programs such as LyX, it’s easy to check for and remove duplicate references (which someone should probably do at a semi-regular interval). Other groups, though, prefer to use systems as Zotero and share their reference databases that way.

Whichever system you choose, make sure that the files are structured in a consistent manner. Reference documents and schematics might be added to a “References” folder, databases to a “Bibliography” folder, images or schematics to an “Images” folder, and so on. Ensure that everyone knows where to find important documents in the repository. When adding new material, explain it fully as part of comment in the log.

Suggestion 2: Centralize Communication Using the SVN Log

Which brings me to the second recommendation: other than long, meaningful discussion – which is best accompanied by food and drink – try and communicate using the SVN log. When you commit a change, leave a message. When you add a reference file, add an explanation. When updating your reference database, explain what’s new and why it’s there. If you’d like for another author to look at a file, let her know as part of your commit.

The SVN log is a powerful tool that can be used to coordinate and communicate the status and direction. When making changes, use it to explain those changes completely.

Short messages can communicate a great deal about what’s changed in a project and what still needs to change. It gets everyone on the same page and provides a place for coordinating work and starting longer conversations. And because it’s all in one place, you can always tell people to “Look at the log” in order to get up to speed.

There is also a wonderful side benefit to using the SVN log. When you consistently use good comments when committing changes, adding resources, and locking files, the repository contains not only every change you have ever made, but also contains an explanation of why those changes happened. Such records are invaluable as you forget things later on.

Summary

Effective collaboration is about meaningful communication. Subversion makes it easy to structure and send relevant assets and reference material for a particular project, you simply add it alongside the drafts and other material. This makes sure that everyone has access to the files needed for their part of the work. Through the use of the SVN log, you can coordinate your efforts and keep everyone abreast of where things are headed. This helps to keep everyone on the same page and informed, and that’s 80% of the battle.

In the next entry in this series, we’ll look at one final way that Subversion can streamline group work: making it possible to create snapshots of a project for review using branches.

__________________________

Notes

1. Physician to physician communication is something of a joke in the medical profession. It mostly doesn’t happen. As close as they usually get is encouraging one another to read the patient’s chart.
2. Ann S. O’Malley and James D. Reschovsky, “Referral and Consultation Communication Between Primary Care and Specialist Physicians: Finding Common Ground,” Arch Intern Med 171, no. 1 (January 10, 2011): 56-65.
3. After searching PubMed and other biomedical databases, I was unable to find quantitative studies looking at how often doctors speak to one another. Typically, the communication takes place through a nurse or other technician who receives an order for a consult and then relays it to the specialist physician.
4. For more information about how to create repositories, add files, and synchronize them between computers, see Part 1 of this series.

Getting Started with Subversion–Part 3.3:
Communication and Logs is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

In each case, these problems aren’t the result of malicious intent. Rather, they were unforeseen consequences of a transformative technology. When it was originally developed, email was a great way to quickly exchange letters with friends and colleagues. Its original designers never intended it to become the way in which a large number of people organize their daily lives. Nor was the introduction of text messaging or social networks meant to cause teenagers (or adults) to socially withdraw into an online world, but to provide an efficient and convenient way to keep people connected. This is also true in the changes that word processors and communications software have brought to the process of writing.

The Limitations of Word Processing

In the old days, writing was slow and methodical. It involved scratching out your words on a piece of paper, pecking them out on a typewriter, or engraving them into stone. Making changes to your draft was difficult (particularly if you chose to engrave). It required that you rewrite sections that you had already prepared. If you wanted a clean draft, you would have to recopy everything.

But all of the work also had a side effect: it forced you to plan, revise, and rewrite. In the process of re-copying the words, you also had to read them and to ask yourself questions, like: “Is this really what I intend to say?”, “Might this be better structured?”, or “Do I want to use such forceful language?”

Then, along came the computer and it became easy to modify the words on the page. Instead of laboriously copying, it was possible to word process.

In far too many ways to count, this change has been marvelous and miraculous. Instead of wasting effort recopying what has already been written, word processors gave writers the precious gift of time. Time to explore other ideas, to rethink the structure of a draft, or to polish the prose. Unfortunately, though, they also made it easy to be lazy by encouraging processing instead of to writing.

When we word process, it is tempting to sit down and start writing, usually without thinking through questions of purpose, audience, content, strategy, or effectiveness. Word processors provide a beautifully clean draft, which we may be tempted to send off without careful revision. In short, by making things easy, word processors encouraged the development of bad habits.

Even more unfortunately, these influences weren’t just limited to the solitary writer working on his own. They also changed the way in which we collaborate. Gerald Grow summarizes these changes well in his timeless essay, “How Computers Cause Bad Writing”:

Before computers, the usual form of collaboration consisted of dividing up the work so that different authors wrote different chapters; then they reviewed one another’s work. Writing with computers, though, collaborators can enter into one another’s work so readily and revise it so easily that, in effect, co-authors can mutually co-write each sentence.

Grow’s article was written in 1988, and in the past twenty-five years, it’s gotten even easier to enter another’s files and “co-write each sentence.” Tools such as Google Docs even encourage you to work this way by providing the ability to co-write in real-time. It’s not only possible to mess about with other’s words, but as the original author produces them! (I’ve shared my feelings about these real-time collaboration features before, so I won’t do so here.)

There is just one problem with such intensive co-writing, though:

This kind of collaborative writing can be difficult to read. No two writers have quite the same sense about punctuation, tone, rhythm, headings, sentence variation, and the like. In collaborative works, I sometimes find grammatical conventions changing from the beginning to the end of the same sentence–because one author started the sentence and the other finished it.

In the worst cases, collaborative writing becomes a colloid of conflicting styles. In a document I recently edited, one section was written by a psychologist with a propensity for theoretical language, another by a computer programmer concerned mainly with the technical characteristics of machinery, another by a manager recording the history of the project. To complicate things, each author had inserted a few sentences (in his own style) in the midst of the other sections. Every time I reached a new major heading, the narrator changed voice–and the voices occasionally jumped around from sentence to sentence. It was schizophrenic prose, with faults that had been amplified by the easy editing made possible by the word processor.

Put another way, intensely processed documents lead to poor writing and misunderstanding.

Ownership of Ideas

Luckily, there’s a pretty simple solution. Instead of encouraging co-writing, it’s a good idea to have each author own a particular portion of the text. Not only the ideas it contains, but the words used to describe them.

When you give someone the ownership of a section, you provide them with leeway to explore how the material might be presented. You allow them time to say something meaningful.

Unfortunately, we live in a rushed world. Too much writing happens to stringent deadlines and it’s not possible to provide such generous gifts of time to a single person. Co-authors may need and require access to a draft in development. But that doesn’t mean that you abandon the idea of ownership. A single mind working on a particular section is more likely to produce something beautiful than a committee.

At some level, this is intuitive. After all, the jokes about “design by committee” are legion. But it’s also something that you can see quantitatively. Let’s look at one of the best examples available, Wikipedia, the very definition of a collaborative writing project.

Wikipedia articles that are considered to be “high” in quality show a certain pattern of collaboration. For these articles authors more frequently write their own draft, provide their own references, and edit their own work. In comparison, articles rated “low” in quality split the work amongst more people: one group may have written the initial draft, another prompted for references and justifications, a third providing those references, and a fourth editing the prose for clarity and flow.¹ In other words, many “low quality articles” are more intensely co-processed.

Now, this doesn’t mean that the entire work should be written by one person laboring in solitude. Rather, that you need to carefully balance the benefits of collaboration with those of cohesion. There is some logical block of material – a section, a chapter, or a part – that exists as a cohesive whole. That is what one author should take ownership of. Once a cohesive draft has been completed, it then goes to others in the group to review. What you want to avoid is the intensive interference that can arise when the draft isn’t ready for editing or feedback.

SVN Locks

Subversion has a feature that can provide for this type of ownership, it is called a lock. A lock (also called a file lock) is exactly what it sounds like. It means that you’re the only person who can commit changes to that file. Commits and changes by other users will be blocked until you release the lock. What’s more, a locked file cannot be modified – meaning that it won’t be deleted, renamed or moved – except by the lock owner.

If you’re using Subversion to write software, locking a particular file is frowned upon. For writing projects, though, it is a powerful way to split ownership amongst multiple authors or editors. Consider the following workflow:

While the initial draft is being written, the file is locked. While others can review the text, they aren’t able to commit changes until the lock is removed. When the first draft is finished, the file is unlocked. If another author or editor will be working intensively on the text, a second lock is placed until finished, and so on.

In addition to enforcing accountability and ownership, using locks for writing projects also provides a technical benefit. Many of the programs commonly used by writers have a binary file instead of plain text file format – this includes Microsoft Word, OpenOffice, and Scrivenener. Subversion works best with plain text. In fact, it is the only file type were file merging is robust. When you lock a file, you prevent conflicts by ensuring that only one person at a time can commit changes. Though conflicts aren’t terribly difficult to resolve, they still require time and effort.²

Creating a Lock

Creating a lock in Subversion is easy. If using the Mac OS X client, Versions, you highlight the files that you want to lock (pressing Shift allows you to select a range of files, holding Ctrl will let you select multiple files at once that are not located right next to one another). Then, you right click and select “Lock” from the context menu.

Verisons will communicate with the server, and place a lock on those particular files. Files that have been locked are highlighted and have a lock icon appear next to them.

The process is very similar for TortoiseSVN. You locate the files that you want to lock in the file explorer. Then, you right click and select “Get Lock” from the TortoiseSVN options in the context menu.

When using TortoiseSVN, other users can see that a file has been locked by opening the repo-browser. The lock status won’t necessarily appear in their working folder, though, unless they frequently check for the current project status via an update. This means that a co-author might not realize that you’ve placed a lock until they have a commit fail.

Checking a Lock Status

To update your working copy with information about what files are locked, you can use the “Check for modifications” option to see what locks you’ve placed. Updating from the repository will let you know what files others have locked.

Breaking a Lock

When you commit changes, SVN will automatically remove a lock that you have on a file. If you’re not quite ready to remove the lock on a particular file, you can uncheck it and exclude it from your commit. It’s also possible to release a lock manually. To do this, you select the file in your working copy and select the “Release Lock” option from the TortoiseSVN context menu.

Summary

Though they’ve transformed the way that we work, computers have made it easy to word process rather than to write. They also have changed how we collaborate. If we aren’t careful, co-authors working on a project may be tempted to co-process the work of their colleagues. This can lead to poor writing and misunderstanding, and for that reason, should be avoided at all costs. Instead, it’s much better to encourage authors to take ownership of a particular section, both the ideas and the words used to describe them.

Subversion has a feature, called a lock, that can help facilitate this. A lock gives one person the exclusive ability edit and modify a file. This preserves the freedom and quiet needed to create beautiful writing. When ready, the lock can be broken and the draft is available to be edited and reviewed by others.

However, while locks are powerful, they are only the first feature that Subversion offers to help facilitate collaboration. In the next entry in this series, we’ll take a look at how the Subversion log can aid better communication.

__________________________

Notes

1. To read more about how collaborative patterns influence the quality of Wikipedia articles, see Jun Liu and Sudha Ram, “Who Does What: Collaboration Patterns in the Wikipedia and Their Impact on Data Quality,” SSRN eLibrary.
2. We’ll look at how to resolve conflicts in Part 4 of this series, “Dealing with catastrophe.”

Getting Started with Subversion – Part 3.2:
SVN Locks and Idea Ownership is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

But that’s really only the beginning. Hopefully, you’ve taken things to the next level and feel comfortable digging into the revision history to look at past drafts, make comparisons between documents, or to see how your work has evolved.

Mastering the basics of version control, followed by the finer points, is a fantastic way to be more productive as a writer. By relegating the job of backup and synchronization to a tool, you can spend more time actually writing (and who doesn’t want that). Having the ability to look at how you’re writing has evolved can make you more thoughtful. Both are powerful additions to the scrivener’s arsenal. If you can believe, it though, there is yet another level which allows Subversion to be even more helpful: using it to work collaboratively.

Not a Solitary Endeavor

As the acknowledgements section of any book or article can attest, good writing is the product of many hands. Family, friends, members of a local writing group, editors, research contacts, mentors, and many others are involved in the production of a finished manuscript. They offer feedback, provide support, serve as a sounding board and help to spot errors. Without the additional eyes, ears, and minds, it is unlikely that anything worth reading would ever be published.

With that said, writing collaboratively is frequently painful. Incompatible software makes it difficult to exchange files and collect feedback, important changes can be missed and manuscripts lost, and time is wasted due to editors having the wrong file versions. For all of these reasons, the world of letters has long relied on a time-tested, robust, and extremely slow collaboration process. They print out the draft and send it through the mail. The state of the art at the time of Jane Austen is still the state of the art.

There are many reasons for this (including the technical challenges noted above), but also others. It turns out, using paper for editing and revising is an effective way to work:

• The process of preparing, sending, and reviewing paper provides time to think carefully about a draft. One of technology’s greatest strengths – the way in which it speeds up communication – is also one of its greatest weaknesses. To develop a thoughtful and careful message sometimes requires thought and contemplation, and for that reason, cannot be rushed. This even goes for when preparing drafts. When you send a draft for someone to review, you are saying, “This is ready for consumption.” It’s not quite the same as just providing someone access to your files and having them go to town.
• Reading on paper encourages editors to make distinctions between large scale revision and small-scale editing, and, to provide feedback on both to their authors.
• It makes it easier to collaborate responsibly, with each author taking ownership of a particular section or chapter of the work and providing feedback on the others. When it becomes time to share different drafts, other collaborators are getting something that that’s thought out and cohesive.
• Reading on paper makes it clear when there might a problem of length. On a computer, it is easy to scan through text without recognizing just how long a particular piece of writing might be. This is usually not a problem when physically sorting pieces of paper.
• Though I’ve said it, it bears repeating. Sending paper forces an author to create a discrete version for review and for editors to provide feedback on a specific iteration. This difference is important. It results in a qualitatively different product, something that the author has thought through completely. Communicating the “big picture” to an editor or collaborator, before needing to defend decisions about comma placement, results in better writing. An unfortunate reality of editing on the computer is that it is a little too easy to “edit intensely,” that is, correct spelling and punctuation before understanding if there are architectural problems in the narrative.

Version control technologies, however, allow you to replicate many of the same features as a paper based workflow, and to supplement them with the strengths of a digital platform:

• A user can “lock” specific files, thereby taking ownership of them. When a file is locked, it isn’t possible to submit changes until the file has been unlocked by the original user. This allows for an author or editor to slow down edits and revisions to that one portion of the project while spending time with the draft. It also ensures that co-authors can’t begin providing feedback until the original author is ready to process it.
• Even though you may want to lock a particular file for ownership reasons, though, doesn’t mean that you don’t want co-authors to have access to it. Subversion can be used to check out an entire writing project at once and to download updates. This means that co-authors can have the most recent drafts of a file for their own reference without having to pester others in group to forward it.
• Finally, Subversion makes it easy to prepare snapshots of the work for review by others. Much in the same way that you might print out the current version on paper and ship it off to an editor. In the language of version control, these are called “branches.” When ready, a new “branch” of the writing project is created from the main collection of files (sometimes called the trunk) for others to check out and edit. This allows for editors and reviewers to see a cohesive body of work, but for the original authors to continue work on the trunk and then “merge” changes back in when the review has been finished.

When you use Subversion for writing, you get a wonderful hybrid of the paper and digital workflows. You can create the sort of contemplative quiet needed to produce a resonant message, but still have the speed and benefits of electronic communications at hand.

In the next three articles of this series, we’ll look at three of Subversion’s features that create this “wonderful hybrid”:

• Locks
• the SVN Log
• Branches

As we examine these features, we’ll also talk about best practices that can make them invaluable part of your workflow. First up: SVN locks.

Getting Started with Subversion – Part 3.1:
Collaboration Fundamentals is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

In the next few posts, I would like to dive a bit deeper and take a look at a few of Subversion’s more advanced features, such as:

• How to compare newer changes to older versions of a file
• How to use Subversion’s collaboration features to work with others
• How you can resolve errors that might arise from incompatible changes made to the same file

Though Subversion’s basic features are tremendously powerful, it’s the advanced options that make it indispensible. You know, the little things that live in the background most of the time, except when you really need them. This rest of this series is about how to leverage those. The first of those features we will look at is the revision history.

Living With History

In the first post in this series, I spent quite a bit of time talking about how version control systems can track every change to a file. If you’re working on a team, it will even remember who made the revisions. That’s all well and good, but all of these previous versions are of little help if you don’t know how to access them. Luckily, there are several ways to get at the history of your work: you can directly access previous versions using the repo-browser, you can use TortoiseSVN to launch programs and make comparisons, and (if the program supports it) you can directly compare differences from within the program you are using to write the draft.

Using the Repo Browser

The first method to get at the history of your work (and the most robust) is to directly access the change history.  Programs such as TortoiseSVN, KDESVN, and Versions make this easy through the use of a repository browser or “Repo-Browser.” Using the repo-browsers, you can look at any iteration of your file on the server, browse the history of your work, and look at what other people have done. You can download previous iterations in case you accidentally (or intentionally) deleted a section (and then regretted your decision).

To get an idea how the history appears, consider the screenshots below, which are taken from two separate programs:  TortoiseSVN (a Windows client) and Versions (a Mac OS X Subversions Tool). Both programs give you access to the same information, though they do it through a slightly different way.

TortoiseSVN (Windows)

Repository Browser (repo-browser) from TortoiseSVN. The repo-browser provides access to all current and past versions of the files. You can access previous versions by clicking on the “Head” button in the upper right hand corner and then selecting which revision you would like to look at. Subversion keeps track of changes as numbered revisions.

When you use TortoiseSVN, it places the emphasis on the file hierarchy. The screenshot above shows the repository browser, which lets you access remote versions of your files. At first glance, it may not seem obvious about how you get to the previous versions of your files.

But If you look carefully, you’ll notice a small button in the upper right hand corner of the window (which you can find by right clicking on a folder you’ve placed under version control and selecting “TortoiseSVN > Repo-Browser), there is a button which says “HEAD.” When you click on this button, it will open a new window that you can then specify which “revision”you would like to investigate. This is how you navigate back through your work.

Subversion keeps track of changes as numbered revision. The current revision is called the “head.” The first revision in the project is number 1. In the project that you’ll see in these screenshots, I’m on revision 358.

By specifying the number of the revision that you would like to investigate, you can effectively travel back in time. You can inspect the files as they look at that point in the project’s history and download a copy. But unless you know the entire history of each change (or commit), though, merely choosing a revision number might not be the most straightforward way to navigate your work.

You can determine which version you might want to look at by using the SVN log. TortoiseSVN keeps  this information in a slightly different location, which can be accessed by right clicking and then selecting “View Log” from the context menu. Here, you can browse messages and view a list of files that were changed when committing the revisions.

Luckily, though, Subversion keeps a log of all of the changes made. Each time that you make a commit, it will list all the files that were added, deleted, or modified. You will also be prompted to add a message regarding what was changed. (In general, it’s a very good idea to provide brief – albeit informative – summaries about what changes you made. This is not only useful for keeping track of the history, but can also be a powerful communication tool.) You can access the log by right clicking on the folder and then selecting “Log” from the TortoiseSVN entries in the context menu.

In the screenshot, you can see the log from a book that I’m currently writing. It shows the date of the commit, the message, and a list of the files that were modified, added or deleted. If you’re looking for a particular revision (perhaps when you decided to make a major editorial change), you can either browse the messages/file history, or search to locate the changes. If you want to look at how the files appeared at that point in time, you can launch the Repo-browser and automatically open it to the revision in question by right clicking and selecting “Browse Repository” from the context menu.

Once the repository is open you can do a number of very useful things, such as:

• download an older copy of a file that you’d like to look at
• locate an image that might have been accidentally deleted
• visualize the changes that have been made to the file over time
• Look at the specific changes made to any file in the project

these are all available from the context menu.

One of the many useful tools available tools from the repo-browser is the ability to visualize the history of a particular file. In the example above, the file was created at revision 94, renamed at revision 223, and split at revision 285 (where it was subsequently finalized). You can visualize any file’s history by right clicking from inside the repo-browser and clicking on “Revision Graph.”

Versions

Versions, in comparison to TortoiseSVN, emphasizes the history of the project. One of the two big navigation tabs you see when opening a working copy or repository allows you to browse the timeline and log. (Providing much of the same information seen in the TortoiseSVN log.) You see messages and the file history – what’s been modified, added, or deleted. If you want to look at the older version of the file, you click on the document’s name and Versions will download it for you.

Versions provides two views to look at the history of your project. The first is a “Timeline” view, which shows all of your revisions and notes. You can view older versions of files by clicking on the link in the log. Versions will download a copy from your repository and then display it.

If you need more information than what is available in the Timeline, Versions also allows you to browse the entire log. You can access this information by going to the “Browse” link, selecting a file or folder, and then clicking on the “History” button.

From the history pane, you can really drill down into what aspects of your work have changed. You can look at how file properties, for example, by clicking on the “Get Info” button. You can jump to the file location by clicking on the “Reveal” button. Or, you can analyze the line by line modifications by clicking on the “Show Changes” button.¹ This will launch a “Difference Viewer” that will show you exactly what has been added, deleted, or modified since the older version.

If you need more information than what is available in the Timeline, Versions also provides you with a detailed history view. You can access the history by highlighting the file or folder you would like to access and clicking on the “History” button in the toolbar. The History dialog allows you to browse the log, in the same way that TortoiseSVN allows you to browse the log.

By clicking on the “Show Changes” button, it is possible to see exactly which portions of your work have changed. If you prefer an older version, you can either restore your work manually (by downloading the file and copying the modified portion), or revert to the older version automatically.²

As with TortoiseSVN, you can also browse the repository by clicking on the repository bookmark in the left hand navigation pane, including the entire revision history. This is useful if you need to find a file or folder that might have been accidentally deleted.

Other Ways to View Differences

While manually using the repository browser to locate old versions of files is the most robust way to explore the changes to your work, there are other methods to see a project’s history. In the previous section, we looked at two: the “Revision Graph” offered by TortoiseSVN and the “Show Changes” option offered by Versions. But while these tools are useful, they are also somewhat limited. The Revision Graph only shows you some of the changes to the file – such as when it was created, renamed, or moved – and the “Show Changes” feature of Versions only works with text-based files.

Luckily, though, there are other ways to compare older versions of your work against the current draft. In the remainder of this article, we’ll be looking at two of those:

• using TortoiseSVN to launch programs, such as Microsoft Word, which allows you to compare two drafts of a file side by side
• using programs, such as LyX, that have support for Subversion built-in

Using TortoiseSVN to Compare Two Revisions of a File

Where Versions on the Mac only allows you to look at the differences in text files, TortoiseSVN goes an additional step. It helps you to look at differences for many types of binary files such as Microsoft Word documents, PowerPoint presentations, Excel worksheets, or OpenOffice documents. To make the comparison, you click on the file that you want to look at and select “Diff with Previous Version” from the “TortoiseSVN” options in the context menu.

For Microsoft Office documents and other supported file types, TortoiseSVN will launch the program, download the older file, and load them side by side. (Unfortunately, this feature is not available in the tools on other platforms.) The screenshot below shows that this looks like for a Microsoft Word document.

In addition to allowing you to compare text based files, TortoiseSVN also makes it easy to look at some types of binary files such as Microsoft Word documents, PowerPoint files, and Excel spreadsheets. You can compare different versions by right clicking on the file and choosing the “Diff with previous version” option from the TortoiseSVN context menu.

In addition to the default binary files, you can add additional file formats through the use of scripts. If you regularly use a program (such as Adobe Acrobat) that supports document comparison, it is possible to write a  script that can automate the process of downloading the previous version from the repository and load it alongside the current draft to make comparisons.

You can access the script settings by going to the “External Programs > Diff View” pane of the TortoiseSVN options. Then, click on the “Advanced” button. This will load the “Advanced diff settings” window (seen below). From the “Advanced diff settings” window, you can add new scripts, remove scripts, or edit the existing file formats.

TortoiseSVN allows you to specify which programs you would like to use to compare different types of files. If there is a program that you use frequently that supports comparisons, you can automate the process of downloading older versions and launching the program to compare them in a JavaScript or Visual Basic script. TortoiseSVN provides several examples.

Programs With Version Control Built-In

For all the power that repository browsers offer, though, the most convenient way to access a project’s history is from within a writing program. However, this is only possible if the program supports version control and subversion. LyX is one such program. It lets you pull up the history of a project, look at previous edits, commit your changes, and update your local working copy all without needing to ever leave the program.

When you want to look at a previous draft, you start the program and load the document you want to work on. Everything else happens from “Version Control,” underneath the “File” menu.

You can browse the history for the file, check in any changes you might have made, update the local working directory, revert changes that might have been made (which we’ll discuss more in depth in the next post), and compare with older revisions. In effect, you can use all of the features of version control that matter. Of these options, the one that I use most frequently is the “Compare With Older Revision…” and “Show History…”

Lyx provides built-in support for Version Control and Subversion. It allows you to browse the history of the file, look for specific changes, and compare with previous drafts. Above, it is possible to see the revision history for a book chapter that I have been working on.

The screenshot above shows the changes that I’ve made to a book chapter on publication design. This chapter has gone through several rather extensive revisions as I’ve tried to figure out the best way to present basic design principles to a novice audience. The message log captures some of these changes and how my thinking has evolved.

Because I haven’t been quite sure how to structure the writing, I’ve rewritten vast chunks of material. Some of the revisions have worked well, others have been less successful. For that reason, I frequently look at the older drafts next to the current draft.

To make comparisons, I select ‘Compare with Older Revision” from the “Version Control” options and then specify which revision I’d like to look at. (The “Version Control Log,” shown above, is invaluable when trying to figure out which revision.) I can either tell LyX how many “revisions back” I’d like to go, or specify a range.

Looking at a draft from one or two revisions back is useful if you happen to be working with multiple people. Perhaps you’re collaborating with a co-author and would like to see the editions she made to the document. Here, though, I want to compare my current work with one of my earlier drafts (revision 161).

Because the revision is quite old (and I know the specific number), instead of selecting “revisions back”, it’s easier to choose the“Between Revisions” option, and enter a range between 159 and 162. This encapsulates the revision I’m interested in and any other changes. I then tell LyX to pull up the old document and make the comparisons by clicking on the “OK”. It will look at the two drafts and open a third document with all of the differences highlighted which I can save to another location, or discard entirely.

When you compare with a previous revision, LyX will highlight all of the changes, so it is clear what has been added, modified, or deleted.

Summary

When working alone, version control  is a powerful way to see how my writing has changed over time. It gives you peace of mind and provides you with a way to capture every edit. Using either the repository browser or a program which supports it natively, you can keep track of every file’s history. You can make comparisons or get at old copies. It’s a wonderful system, and the subversion history is what makes it all possible.

What’s even better, though, is when you start to leverage Subversion to work collaboratively. Co-authors or editors check out a copy of the files in the project. They edit them at their leisure and when finished with their revisions, they commit the changes to the repository. There’s no need for dozens of emails or attachments. Everyone starts and remains on the same page, and headaches are kept to a minimum. In the next post in this series, we’ll look at how to leverage subversion in groups.
__________________________

Notes

1. While “Show Changes” is a tremendously powerful way to track how your work has changed, it only works with files that are text based. It isn’t possible to compare binary files like Microsoft Word documents, images, Adobe Illustrator illustrations, or Photoshop compositions. Additionally, if the file is text-based but composed of markup meant to be rendered by the computer, as is the case for SVG images or Adobe InDesign files, “Show Changes” will be of limited benefit. In such cases, it is usually better to download a copy of the old file and use a program’s built-in compare features directly.
2. In most cases, it’s better to restore the work manually. Reverting to an older copy will delete new portions of the file that you’ve added.

Getting Started With Subversion – Part 2: Advanced Stuff is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

Actually, that doesn’t go quite far enough in its praise. Devil is a magical book in almost every way that matters. It’s the sort of thing that (if you’re not careful) can swallow you up and send you into other worlds.

Indeed, that’s one of the reasons that I’ve spent so long reading it (eight months and counting). Larson keeps enchanting me down some of the 19th century’s most scenic, semi-forgotten by-ways. It can take some time to come back.

For example, I’ve learned about the sordid history of the Whitechapel Club, the polite hazing of women architects (with attendant nervous breakdowns) by prim society women, and a failed expedition to retrieve a tribe of Pygmies for public exhibition.

While these side stories only tangentially touch on Larson’s main narrative, the building of a world’s fair, they make the book. They help to flesh out the world and time in which the  Exposition took place and to place its events in context.

* * *

Unfortunately, in one way, the book underwhelms.

In many of the accounts Larson quotes (and others I’ve read elsewhere), it was more or less universal to comment on was the size and scope of the architecture and the visual spectacle of the city. Everything was enormous, pristine, and huge. The buildings gleamed and glittered. Many of the visitors described it as, “a paradise.”

The images included with “The Devil in the White City” provide only a glimpse of this wonder, though. The human details of the book thrill, but that was only half of the experience. The glorious White City of the Fair was also supposed to be a architectural marvel. So … I went digging to see what else I could find. And, luckily, there is a treasure trove available.

The Google Books project has digitized several wonderful albums with thousands of photographs. They show the buildings, the people, and help to capture something of the experience.

I enjoyed the images so much, I thought I would post a few of them here. (All of these were taken from a book called the “World’s Fair Album.”)

Image Sources

World’s Fair Album: Containing Photographic Views of Buildings at the World’s Columbian Exposition, Chicago 1893.

Gleaming Cities, Devils, and Dreams is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.

If you don’t believe me, take a look at this video from TED, which demonstrates a “next generation electronic book” called Our Choice, by Al Gore. (The video is only four minutes long and well worth your time.)

Though I may be succumbing to hyperbole, I really think that we are seeing the future of non-fiction. We live in a tremendously complex world and those of us who in the business of shaping and communicating ideas — scientists, engineers, idealists, philosophers, teachers, and so forth– face an enormous challenge: how do we take that complexity and make it understandable to others?

For centuries, books have done an excellent job of combining two types of media: text/narrative and images. But while you can communicate many ideas with text and images, there is still a limit. For example, who really thinks that math and certain scientific disciplines are best learned by reading a book? Even an excellent mathematical textbook is only an adequate tool, hardly an exceptional one. It simply leaves out too much of the logic necessary to understand how certain relationships are derived. In such cases, one of the best ways to understand those relationships is to watch them be derived in front of you. When done well, think Richard Feynman, it’s much more instructive than a text narrative could hope to be.

This is where electronic books might take up the slack. In addition to text and images, it’s possible to add video and even interactive elements. There are certain principles that are best explained by a narrative and video clip. There are other concepts where an interactive examples best illustrate an idea. And there are still others where trial, error, and feedback are the best way to teach the concept. With an electronic book, you can include all three. Indeed, just about anything you can do on the web is possible, which is really exciting!

For the electronic version of the Open Source Writing book, I’ve been experimenting with video. (Both ipad and the Nook Color provide rudimentary support for the HTML5 <video> tag.) In the process, I’ve learned an important point: motion and voice make it really easy to show certain points. For example, if talking about how to accomplish a certain task with a computer program, there is nothing more effective than showing how it is done. The rest of the text becomes supporting documentation.

And I think that’s cool because it expands the types of stories I can tell (and isn’t that generally the point of new technology). Cool tech without soul is just flash in the pan.

Which leads to the next important question: how do you create these amazing pieces of electronic art without breaking your wallet or mortgaging your soul? Turns out, the answer isn’t nearly so complex as you might think.

Digital Storytelling is a post from: Apolitically Incorrect. Copyright 2009 – 2010, Rob Oakes.