rsync Speed

Moving the work to the client relieves the server, but the client then has to deal with the problem of computing the rolling checksum over the old file. As explained in [[Rsync1998]], although it is necessary to calculate the weak checksum at every possible offset in the old file, due to the choice of checksum the checksum at offset x+1 can be calculated using the checksum at offset x in combination with the bytes at x and x+blocksize.

Despite this, when working on large files, for instance ISO files, the calculation can take some time — my Athlon XP 1600+ takes roughly 3 minutes to pass over an entire ISO file in zsync-0.1.6. One possible optimisation is skipping forward to the end of the block when a match is found — once one match is found, there is a good chance of a match at exactly one block further forward (where files have a section of more than once block in common), while a match before then is unlikely (this would mean that target file contained redundancy, with blocks containing similar but offset content). Skipping forward to the next block after a match can halve the time to process a file, if it has a lot in common with the target file, but may slightly increase the amount of data transfer required afterwards. More investigation is needed into this.