Making zsync Downloads

Note that zsync only (currently) supports single-file downloads. I am currently of the opinion that this is sufficient: it is a file distribution method, and, like .tar.gz, it is up to the downloader to worry about extraction, permisions, etc. zsync is not competing with rsync for synchronising directory trees between machines and over shell accounts.

There are three cases:

  1. The file to download is already gzipped, and you don't want to recompress it.
  2. The content of the file to download is already compressed or not worth compressing further (e.g. deb/rpm packages, ISOs of Linux/BSD distributions, mpegs etc).
  3. The file should be compressed for transfer.

Already Gzipped

Simply run zsyncmake on the target file. zsyncmake will automatically look inside the compressed file and produce a zsync for the content (this avoids the problem that rsync has, that it does not normally work with compressed files).

Use the -C option if you want the client to receive the uncompressed data, or if you want the download to work for versions of zsync before 0.3.1. Without -C, zsyncmake will include information to cause the client to compress the content again after it receives it — if you must have the client receive a .tar.gz identical to the original, specify -e.

Not Compressible & Not Gzipped

If the content of a file is compressed, but not in a simple gzip format, bear in mind that zsync may not be effective. Each compressed stream can typically only be efficiently updated via the rsync method if it is either completely unchanged, or the compression has been made rsync-friendly (with, for example, gzip --rsync).

So, for example, zsync is useless for individual Debian or RPM package files, and is useless for bzip2 files. But it might still be useful for an ISO containing lots of such files, if, say, some packages have changed but some are completely unchanged.

Just run zsyncmake over the file. It should choose a suitable block size for you.

To Be Compressed

You could just gzip the file first, but it is more efficient to use zsyncmake -z. This causes zsyncmake to build a gzip file for you; it constructs it in such a way that the zsync client will have less to download. It is this approach which gives transfer efficiencies approaching that of rsync -z. If you have control over the file being downloaded, it may even be worth decompressing it and recompressing with zsyncmake, to get that extra efficiency.

Distribution

The other consideration is how people will access the zsync download. Remember that people have to have access to both the .zsync meta-file, and the full download that it describes.

The server hosting the full download must support HTTP Range: requests (almost all do). It should support HTTP multipart/byteranges responses, HTTP/1.1 and pipelining. Currently it must not use Transfer-Encoding: chunked (yes, I know that makes zsync non-compliant), and should not be generating the content dynamically.

If you are hosting the download yourself, then simply put the .zsync in the same directory on the webserver as the download (this is the setup assumed by default by zsyncmake). zsyncmake inserts a relative URL by default, so this will work even if your download directory is then mirrored to other web servers. Remember that if you use zsyncmake -z, you must put the .gz file made by it on the server as well as the .zsync. If you want the .zsync to be in a different directory, specify the relative URL to the full download (for example, if the .zsync will go in a zsync/ subdirectory, specify -u ../orig_filename.tar.gz).

If the main download and the .zsync are on different servers, specify the full URL to the main download with -u. If you have a set of mirror sites, if you had only a single absolute URL in the .zsync then you would lose the benefit of the mirrors; so you can specify multiple -u options to provide multiple URLs, and the zsync client will choose between them (currently randomely, but it may change later to choose more intelligently).

You can even offer a .zsync for a download provided/hosted by someone else. You have to download the file yourself, and generate a .zsync from it. Specify the original URL with -u, and then put the .zsync on your own webserver.

Other Considerations

Serve .zsync files with a MIME type of application/x-zsync if possible. The zsync client does not need this, though. For example, for Apache, include

application/x-zsync             zsync

in your mime.types.

Set suitable content expiry headers on .zsync and other downloads, to help web proxies work out how long they can cache it. zsync itself always goes back to the server, although with the -k option it will use If-Modified-Since headers to avoid redownloading the .zsync unnecessarily.

Do not point the full download at content which is negotiated: it must always be exactly the same as the file supplied to (or generated by, with -z) zsyncmake.

See also the page on client/server issues, which contains useful info for webmasters and also explains errors that could be seen by users.