Overview
zsync is a file transfer program. It allows you to download a file from a remote server, where you have a copy of an older version of the file on your computer already. zsync downloads only the new parts of the file. It uses the same algorithm as rsync. However, where rsync is designed for synchronising data from one computer to another within an organisation, zsync is designed for file distribution, with one file on a server to be distributed to thousands of downloaders. zsync requires no special server software — just a web server to host the files — and imposes no extra load on the server, making it ideal for large scale file distribution.
zsync is open source, distributed under version 2 of the Artistic License. Feedback, bugs reports and patches are welcome.
Advantages of zsync
zsync fills a gap in the technology available for large-scale file distribution. Three key points explain why zsync provides a genuinely new technique for file distribution:
- Client-side rsync — zsync uses the rsync algorithm, but runs it on the client side, thus avoiding the high server load associated with rsync.
- Rsync over HTTP — zsync provides transfers that are nearly as efficient as rsync, without the need to run a special server application. All that is needed is an HTTP/1.1-compliant web server, and some extra metadata saved alongside the download. Downloads work through firewalls and from shared hosting accounts, and gives less security worries.
I have put up technical papers. There is a current paper, which reflects zsync's latest development, and which I update with new figures as I run new tests and comparisons; there are also older snapshots of the technical paper, included for reference and citation.
Practical Engineering
zsync is only useful if people offer zsync downloads. I have made a deliberate effort to make zsync simple and reliable in use, and easy to deploy.
- Single meta-file — zsync downloads are offered by building a .zsync file, which contains the meta-data needed by zsync. This file contains the precalculated checksums for the rsync algorithm; it is generated on the server, once, and is then used by any number of downloaders.
- Easy file format — .zsync files have a simple text header, so they can be edited with any good editor to change details like the download URL.
- Flexible use — The .zsync can contain a relative URL to the full content of the file, so you can generate and offer it on the same server as hosts the full download (and so it can be distributed like any other file onto mirror sites). Or it can contains one or more absolute URLs (the client will try multiple URLs if necessary to find the content), meaning the .zsync can be generated and offered by a third party, while still pointing to the original distribution site for the content.
zsync can only be used to download files for which a .zsync file has been generated by someone who already has a copy of the file. So it requires that the download distributor, or third party, generates a .zsync file for the download.
If you are a server operator looking to use zsync for file distribution, see the server page, which gives some guidance on how to create and offer .zsync files.
Current Status
zsync was originally released in 2004. While it never reached a v1.0 release, the program has been used by many projects for file distribution.
I do not do a lot of active development of zsync these days, but I do look at bug reports and pull requests via github and will make new releases if there are interesting changes to release.
Notes on likely and unlikely future changes:
- zsync 0.7.0 is a rewrite of the program, and so is less mature than the 0.6.x line of releases. So there is a higher chance of bugs and incompatibilities being discovered in use.
- Multiple file support is not planned to be added. From the number of feature requests that I received for this, it is clear that there is or was a gap in the market for a directory synchronisation tool without the need for specialist server software like rsync; but this is not zsync's target.
- Look-inside-compressed-file support is removed in the latest release. This was a feature added 20 years ago to ease early adoption of zsync, and because experimentally it was often slightly more efficient than `gzip --rsyncable`. But this support required a customized version of zlib to be shipped as part of the codebase, and that no longer seems like a good tradeoff. Users can still download and use 0.6.4 instead if they have an existing use case needing that feature.
