Your App Needs to Sync

In the last few years, we’ve seen a pretty significant shift in how we use computers. We’ve gone from primarily using one Internet-enabled device (the PC) to using two (PC + phone) to using three (PC + phone + tablet), and who knows what else we’ll add in the next couple years. Not only are we looking up our data and documents on all these devices, we’re creating data and documents on them, and the time we’re spending to do it on the PC is getting smaller. Effortless and ubiquitous access to data is increasingly important to people.

If your app deals with user’s data, building cloud sync into your app should not be a feature you bolt on to an app - it is the feature. It’s why you will beat competitors or lose hard to them. It’s what will make your app feel effortless, thoughtless, and magical. It’s what will gain a user’s trust, and once you have that, they will sing your app’s praises and never give it up. But to earn that trust, you have to account for sync at every step of the design and engineering of your app.

Developers have a number of choices as to how to build an app around sync. You can use iCloud, you can use a hosted service like Parse, or you can build a custom sync service for your app. Each solution has trade offs. So what should you optimize for?

The reality is that every app is different, and each sync system must cater to the data that is syncing. While it is certainly the most work, it’s my belief that you should optimize for control. You should have total and complete control over when and how your app syncs its data. You should be handling errors and conflicts, not abstracting them behind black-box middleware. Without this level of integration, you’re bound to a system that can fail unreliably, leaving users to figure out what went wrong. And when they try, it’ll be your app that feels broken.

When you build it all yourself, you have absolute ownership over everything. When you decide your app should sync, it will. When you get the data, you can update the UI immediately. When there’s a problem or a network error, it’s not lost in some abstraction; it’s your code that can hear about it and deal with it or tell the user. You know when everything is working and when it isn’t. And if you know, you can empower the user to make decisions and set their expectations.

iCloud’s biggest problem is that it goes out of its way to obscure a lot of this detail from you. Their pitch is that creating apps with the document system and putting them in iCloud means they will all sync magically and you don’t have to worry and we’ll handle it for you thank you very much. But the reality of syncing data is that it’s tough, and network availability is not always reliable or fast (especially on mobile). You have to write a lot of nonobvious code to handle updates and problems. Building for iCloud once means you limit yourself to only Apple devices; you can never get that data synced to an Android device or make it accessible via the web (short of later building your own system, updating your apps, and making them push iCloud-stored data to your own server). And iCloud has not exactly gained notoriety for its stability or its friendliness to developers. The only real debugging tools you have are a web app that lets you see what’s in an iCloud folder and some rather verbose logging flags you can turn on that tell you some stuff about the syncing process. In other words, it’s not easy. I’ve tried to integrate iCloud no less than 6 times in various app prototypes, and every single time I’ve ran from it.

Note: I don’t mean this to denigrate the work of the iCloud team. What they’re doing is immensely difficult, and I have great respect for that team for trying to solve a problem of this magnitude in a way amenable to every app. But sympathy is not something you can build a business on.

Of course, the downside to the DIY solution is that it’s really, really hard. On top of building the app itself, you now have to build a database for data and the other half of a sync engine. You have to worry about security of users’ data. You have to back all this stuff up somewhere. You need a fallback strategy for when your main server has a problem. You have to know how to scale to lots of users and know what the hell terms like parititoning and sharding mean. Oh, and you have to know how to keep a server running and accessible. But the upsides are really valuable. You have limitless flexibility and control in what you use and how you use it. If something is running slow, you can fix it. You can compress your data into pushing everything important in a single network request in the most optimized form imaginable (which is a huge win for how fast your app syncs). You can manage your costs in a much more straightforward way. If you want to add some more advanced network features, like real-time sync, just do it. You own the whole stack, you can build what you want. You just have to build it.

Both extremes have upsides and downsides. And any system is a balance of tradeoffs. There is a middle ground in the form of services like Parse which handle a backend for you. You can push whatever data you want into their system and get it back later. You can build your own sync engine hooked up to their APIs. As you develop your app, you can update your data models without any effort. Their errors are well defined in documentation and you can handle them appropriately. But you are limited by what they can do (which is admittedly a lot, but not everything). No real-time updates, for example. You are limited in how you can package your data, making it tough to optimize network traffic. And as you grow, you’re bound by their pricing model, which can get far more expensive than hosting it yourself. And while the possibility is remote, they could disappear or get acquired by someone and shut the product down, leaving you to scramble to build the server stuff yourself in a tiny window of time.

In the end, it’s all about tradeoffs. If you have the technical chops and the time to build, run, and maintain servers, a self-assembled system will give you the most flexibility, best power, and lowest cost. If you are talented only in building apps, learning how to run a server is absolutely not a bad skill to have, but requires a different engineering mindset that takes time and experience to learn. In the meantime, a system like Parse is your best bet. iCloud, while a beautiful idea, is not production-ready for most apps, will cause endless headaches, and will lock you in to forever developing apps only for Apple products.

But whatever you decide, you need sync in your app. And you need to think about it at the beginning, not at the end.

If you enjoyed this post, please consider subscribing to Informal Protocol via RSS, App.net, or Twitter. Thanks!