Overview
zsync is a file transfer program. It allows you to download a file from a remote server, where you have a copy of an older version of the file on your computer already. zsync downloads only the new parts of the file. It uses the same algorithm as rsync. However, where rsync is designed for synchronising data from one computer to another within an organisation, zsync is designed for file distribution, with one file on a server to be distributed to thousands of downloaders. zsync requires no special server software — just a web server to host the files — and imposes no extra load on the server, making it ideal for large scale file distribution.
zsync is open source, distributed under version 2 of the Artistic License. Feedback, bugs reports and patches are welcome.
Advantages of zsync
zsync fills a gap in the technology available for large-scale file distribution. Three key points explain why zsync provides a genuinely new technique for file distribution:
- Client-side rsync — zsync uses the rsync algorithm, but runs it on the client side, thus avoiding the high server load associated with rsync.
- Rsync over HTTP — zsync provides transfers that are nearly as efficient as rsync -z or cvsup, without the need to run a special server application. All that is needed is an HTTP/1.1-compliant web server. So it works through firewalls and on shared hosting accounts, and gives less security worries.
- Handling for compressed files — rsync is ineffective on compressed files, unless they are compressed with a patched version of gzip. zsync has special handling for gzipped files, which enables update transfers of files which are distributed in compressed form.
The special handling of gzip files is, as far as I know, entirely new and unique to zsync.
I have put up technical papers. There is a current paper, which reflects zsync's latest development, and which I update with new figures as I run new tests and comparisons; there are also older snapshots of the technical paper, included for reference and citation.
Practical Engineering
zsync is only useful if people offer zsync downloads. I have made a deliberate effort to make zsync simple and reliable in use, and easy to deploy.
- Single meta-file — zsync downloads are offered by building a .zsync file, which contains the meta-data needed by zsync. This file contains the precalculated checksums for the rsync algorithm; it is generated on the server, once, and is then used by any number of downloaders.
- Easy file format — .zsync files have a simple text header, so they can be edited with any good editor to change details like the download URL.
- Flexible use — The .zsync contains either a relative URL to the full content of the file, so you can either generate and offer it on the same server as hosts the full download (and so it can be distributed like any other file onto mirror sites). Or it can contains one or more absolute URLs (the client will try multiple URLs if necessary to find the content), meaning the .zsync can be generated and offered by a third party, while still pointing to the original distribution site for the content.
zsync can only be used to download files for which a .zsync file has been generated by someone who already has a copy of the file. So it requires that the download distributor, or third party, generates a .zsync file for the download.
If you are a server operator looking to use zsync for file distribution, see the server page, which gives some guidance on how to create and offer .zsync files.
Current Status
zsync was originally released in 2004, and has seen few changes in the last 15 years. While it never reached a v1.0 release, the program is stable and has been used by many projects for file distribution.
I am interested in hearing feedback about zsync. Bug reports are welcome. At this time I am not planning to make any large feature additions to zsync in its current form. Note in particular that:
- Multiple file support is not planned to be added. From the number of feature requests that I received for this, it is clear that there is or was a gap in the market for a directory synchronisation tool without the need for specialist server software like rsync; but this is not zsync's target.
- HTTPS support. Yes, I know that the internet has moved on and doing stuff over unencrypted HTTP looks old fashioned now. In particular the authentication support in zsync makes little sense without HTTPS support. But zsync's current implementation is not suited to adding HTTPS support. Really the HTTP layer in the program needs to be rewritten from hand-rolled to using a library. So HTTPS support is likely to happen only if and when I feel like rewriting a larger part of the client program.
