Another coder at work
October 26, 2017

Improving File Syncing for TeX Writer

We are undertaking the second rewrite of file syncing engine in TeX Writer. The first rewrite was when Dropbox introduced its sync SDK. This time is because the sync SDK has been abandoned by Dropbox, and TeX Writer has been living on an extension period before syncing stops working eventually.

I don't want to blame Dropbox for the termination of sync SDK. It actually got me thinking how we can redesign a better syncing engine. Previously it's difficult to integrate another cloud storage provider, because the code has been structured around Dropbox sync SDK, and we have been reluctant to change code that is not broken. Now change is inevitable, and we have a chance to start from scratch.

We have been thinking and working on the new file syncing in the recent weeks. Many ideas come and go, and the following list is what we end up with.

Full cloud access

Currently only a small portion of Dropbox is visible to TeX Writer, that is files and folders under /Apps/TeX Writer. We wanted to play safe but that is extremely inconvenient for people who collaborate on folders outside the sandbox area. So this time we really want to make people's life easier by asking for full access to entire Dropbox folder.

The way TeX Writer compiler works requires the presence of all referenced files, especially when we need to work in offline mode, like on an airplane. We used to sync entire /Apps/TeX Writer folder down, but now it doesn't make sense to sync entire Dropbox. So we will let users to link a local folder on device to a remote folder on Dropbox (or any other storage provider). Only linked folders will be kept in sync. This is like the selective sync feature on desktop Dropbox client.

Auto merge

Currently if a file is changed locally and also has changes remotely, TeX Writer will require manual conflict resolution. You have to choose either local or remote copy, and it's painful. Most of the time, the changes happen on different portion of the same file and we should be able to merge automatically. We are not going to implement a version control system inside TeX Writer, but this kind of basic feature is useful when we are working on a shared folder with other collaborators.

OneDrive

Dropbox is still the best out there in my opinion, but people have been asking for alternatives. Many ask for iCloud. However, iCloud has a lower priority on our road map, because it is not a general purpose cloud storage. OneDrive is actually the better option. OneDrive provides web API, and its sdk is available on both iOS and android. Its SDK may not be as clean as Dropbox's, but is surely better than either iCloud or GoogleDrive.

Other things

TeX Writer has introduced Pro subscription so that subscribed users can use advanced features. Currently Pro subscription is not cross-platform, but eventually we will associate Pro subscription to one's Dropbox or OneDrive account, so that if one subscribes Pro on iOS, pro features should be avaiable on both texwriter.net and the Android version of TeX Writer, provided that the same Dropbox or OneDrive account is used. Likewise, if one have already purchased web app pro subscription, Pro features should be avaiable in apps too. This is on our list, but may take some time to implement because we need to think it through.

So far, progress has been slow, because file syncing requires extensive testing, and we want to be very careful with user files. The sync engine is implemented and tested separately and we need to integrate into the app in the next few weeks. We hope to deliver the new file syncing before the end of next month.