« Yes, Macworld Still Matters | Main | Apple Remote Desktop example scripts »

I realize that BootCamp and Parallels and virtualization are getting all the press, but I'd like to talk about something that's in every copy of Mac OS X 10.4, regardless of chip architecture, and something that has made, and will continue to make, a lot of people's lives much simpler: Sync Services.

Technorati Tags: Apple, Mac OS X, Sync Services, Technology

Way back during the Mac OS X 10.4 introduction, Apple was talking about a curious thing called Sync Services. This was an API for the plumbing that allowed iSync to work, and would allow for third party developers to gain access to things like Address Book data, iCal data, etc. They would also be able to sync this data with multiple Macs via .Mac. Sync Services, for the most part got a "Oh, that could be cool" reaction, but not much more.

Well, I was excited, because this was something I had talked about well before Mac OS X 10.4 was released, even before the first Apple WWDC that announced Mac OS X 10.4. In that article, entitled "Set iSync Free" I said that I didn't want everyone to have to use the Address Book database for contacts, or iCal for events. Instead, what I wanted was for Apple to allow any application to be able to have access to synchronization APIs so that any kind of data could be synchronized, automatically, on the fly.

What I learned about Sync Services made me quite happy. (No, I am not, for a second, even implying that my article is why we have Sync Services. For one, development realities say that's not how it worked, and two, I'm not the only smart person in the Mac community. It was just fortuitous timing, nothing more.) I was excited, because that was what I wanted. To have my Entourage data just replicate out to Address Book, iCal, or any other application that needed that data in the formats that the various applications needed. It wasn't about lockstep uniformity of file type, but transparent replication of data. I didn't want to care about what application I entered the data into. I just wanted contact data in my contact applications and calendar data in my calendar applications, and I didn't want to have to script my own solutions, or run manual scripts. This is basic data entry, and it's monkeywork, and it's what computers are for. This is what Sync Services handles for you, automatically and transparently.

At its heart, Sync Services is simple. The center of Sync Services is the Truth Database. The Truth Database contains a master copy of all the information used by the various Sync Services clients, such as Address Book and iCal. When an application syncs, it is the Truth Database that is the data store, not the other clients. This is important, as it removes most of the issues with data storage types, formatting, and the like. The clients can keep their own view of the "world" and the Truth Database can store the data without having to care how the clients want their data formatted.

On top of the Truth Database is the Sync Engine. The Sync Engine is where record changes, conflict handling, duplicate handling, and record changes/updates are controlled. It is able to synchronize at the field level, (i.e. the email address in a contact), not just at the record level, (the entire contact). So, if you change an email address for a contact in Address Book, and a phone number on your PDA for the same contact, only those changes are made in the various clients and the Truth Database. If there's a conflict, where the same fields are changed in different clients, then the user is asked how to handle it, and given the option to handle each conflict manually, or have them all handled in favor of one of the clients.

On a basic level, every sync is the same. A client pushes a state to the Sync Engine, the Sync Engine applies any changes to the Truth Database, and the client pulls back any changes made by other clients. This happens automatically for most clients on the same computer. .Mac sync can also be used on a timed schedule, or you can hit the "Sync Now" option. iSync is still a manual application, (and, in Mac OS X 10.4, is now solely used as an interface to external devices such as PDAs, Cell Phones, and iPods).

In a bit more detail, every sync has five steps:

1. Create the Sync Session. The client sends a message to the Sync Engine to initiate the sync. If other clients are able to use the data being pushed out, they're notified by the Sync Engine that they can join the sync. So, you make a change to a contact in Entourage, and the Sync Engine notifies Address Book so it can automatically join in the sync.


2. Negotiate how you will sync. There are five ways an application can sync, depending on the context of the sync. If the client is syncing for the first time, it will do what's called a slow sync, where it pushes all of its information up to the sync engine, and this data is integrated into the Truth Database. From then on, only changes are sent back and forth, and this is the most common type of sync, the fast sync. A variant of this is the trickle sync which is a fast sync that happens often without the user having to do anything, i.e. you change a contact in Entourage, and it just shows up in Address Book a few seconds later, without you manually initiating a sync. Sometimes, there may be a reason for a client to want to overwrite the Truth Database with its internal records, aka a push the truth sync. The opposite of this is where you have a client that wants to overwrite all its internal records with the data in the Truth Database, so it performs a pull the truth sync.


3. Once the sync type has been negotiated, the client that initiated the sync pushes its changed data up to the Sync Engine. It can either push the field changes, or the entire record. If the entire record is pushed, then the Sync Engine figures out what parts of the record have changed.


4. Once all the client pushes have completed, the Sync Engine merges the changes into the truth database, and handles any conflicts it can handle on its own, or notifies the user of conflicts it can't handle


5. Finally, clients that are participating in the sync pull changes back from the Sync Engine. This is where you would see the changes made in one application propagate out to other Sync Services clients.

Sync Services support took a while to start showing up. There were some applications that started using Sync Services, such as Yojimbo, from Bare Bones, and the Missing Sync for Palm from Mark/Space, even Transmit. But most of them, like Yojimbo and Transmit only really sync to themselves, because they're syncing data that's unique to those applications, and while cool, it's not useful in the way I wanted. The Missing Sync though, was a better start. No longer did you sync your Palm/Treo/whatever to Address Book or iCal. You sync'd them to Sync Services, and let that handle the contents. But there wasn't a lot to sync with outside of application to same application on a different computer, or iSync to a handheld device.

But then, Microsoft released the 11.2.3 update to Office, and with it, support for Spotlight and Sync Services in Entourage. Now, I could finally, on the fly, without having to remember anything, not only sync my Entourage information with .Mac, but with Sync Services running locally on my Mac. What did this mean? It meant that if I entered calendaring information in Entourage, that iCal automatically picked it up. If I added a new iChat buddy, and filled in things like email address, phone number, etc, that Entourage automatically picked that up too.

But it goes beyond .Mac and local applications. For example, I have a free Plaxo account. But for a long time, I didn't use it, because it only had an Address Book plugin, and for the way I work, Address Book was only a container for iChat data. For me to get my "real" contact data into Plaxo, I would have had to do imports, manually move the data, drag and drop .vcs files, pfah! With Sync Services, the only thing I have to do is the occasional sync with Plaxo's servers, and it's all there. Since Plaxo has a plugin for Thunderbird, I can add that in too, if I like. (In fact, I did add in the Plaxo extension to Thunderbird while I was writing this. A short install, restart Thunderbird, log it into Plaxo, hit the button, BAM! Now Thunderbird has all my contact info too. It's real simple.) But note in all of this, I only enter the data once. From there, various synchronization tools take over. If I want, I use the Outlook Plaxo plugin, and I get my contact data synched to Windows as well. Not bad for entering the data once.

It keeps getting better. Mark/Space just released an alpha of the next version of the Missing Sync for Windows Mobile 2.5, which now supports, like its older brother for Palm devices, Sync Services. Since my cell phone is a Windows Mobile device, this is a major convenience for me. Instead of synchronizing to Entourage, I sync to Sync Services. I'm no longer limited to picking applications to sync to, each with their own way of dealing with different data, even when it's almost the same. Sync Services has become my data repository, and really, that's not limiting. In fact, it's given me more options. If I'm forced to always sync to an application like Address Book, and I want to also sync to an application that doesn't use Address Book to keep its contacts, well, I have to play manual data coordinator. This is tedious, even with really good scripts to automate things. That's application-level syncing, and it's how things happened in Mac OS X 10.3 and earlier. Your synching was tied to a specific application.

But, if the applications I use support Sync Services, then I just use the applications I like, and the data replicates. The fact that Plaxo doesn't directly support Entourage makes no difference to me whatsoever. Thanks to Sync Services, it's all good. The fact that Entourage syncs with Microsoft Exchange far better than Address Book and iCal, (which doesn't sync with Exchange at all) is not any kind of issue. I let Entourage sync, and Sync Services pushes those changes out automatically. (Or not. Entourage is pretty cool in that I can decide to sync my Exchange or local data, so I can keep personal stuff and Exchange stuff separate if I choose.)

Sync Services has finally given me a good justification for .Mac beyond iPhoto, and WebDav. Face it, you can store pictures and files anywhere. But, with .Mac, I can take a brand new system, and on a decent Internet connection, have all my contact, event, and other Sync Services data downloaded on to it in minutes, and all I have to do is start the .Mac sync. So, one of the biggest bugaboos about multiple computers, keeping data up to date between them is now a no-brainer.

It is the transparency of operation that makes Sync Services so useful. It frees me to do stuff with my data that isn't monkeywork. I can just use it in the way I want to, not the way that a given application wants to. Like all the really good things Apple comes up with, it just works. The convenience and, well, brilliance of Sync Services is turning me into a bit of a snob. From now on, if I see a new release of an application that uses contact or other data that can be plugged into Sync Services, and it's not using Sync Services, then maybe I don't need to use that application. Yes, it's more work for the developer, but it gives them a way to make lives much easier for their audience, and that's a good thing last I checked, at least from a getting-people-to-pay point of view.

What I'd love to see is a .Mac sync tool for Windows and Linux too. Not just applications, but frameworks, APIs the whole thing. Being able to replicate data this easily between Linux, Windows, and Mac OS X would bring not just a major justification for .Mac to the table, but bring a major convenience to those of us living in a multiplatform world. But even without cross-platform Sync Services, it's still a heck of a (very underrated) tool, and one of the best reasons to upgrade to Mac OS X 10.4 if you're still living in a Mac OS X 10.3 world.

Comments

Warning for Notes users: The commenting system uses HTML.
I know this will be scary for some of you, especially Notes fans. However, open standards, rah-rah.
If you want to use less-than or greater-than signs, or other similar characters that HTML reserves,
you'll simply have to learn to do it the HTML way. Luckily, HTML is kind of popular, no matter what
your re-educators have told you, and you can easily find help on the intertubes.