View Single Post
Old 2018-06-09, 13:49   #2
kriesel's Avatar
Mar 2017
US midwest

132816 Posts
Default Save file format as described by ewmayer

As posted at by ewmayer, except as updated in bold, first describing the V17.x format:, then planned V18 format additions:
Here is the current Mlucas file format:

o 1 byte for test type, numerically, i.e. 256 possible values, mapped to an internal table;
o 1 byte for modulus type, currently only Mersennes and Fermats supported;
o 8 bytes for iteration count of the residue stored in the file;
o ceiling(p/8) bytes for the residue R - i.e. maximally byte-compact, endian-and-FFT-length-of-run-independent;
o 8 bytes for Res64 = R (mod 2^64), which should match the leading 8 full-residue bytes in the above bytewise form);
o 5 bytes for R (mod 2^35-1);
o 5 bytes for R (mod 2^36-1) [these last 2 a.k.a. the Selfridge-Hurwitz residues, based on those guy's Fermat-number work, using a 36-bit-hardware-integer machine; SH also used R (mod 2^36), but that is just the low 36 bits of GIMPS' Res64];

After reading R, I directly compute the two SH residues and compare to the above file-stored checksums; this gives me an md5/sha1-style integrity check of the whole residue R, which the Res64 does not.

For v18, I am adding several new fields:
o 3 bytes (was 4) for FFT-length-in-K which the code was using at time of savefile write. This is so that if the code switches to a larger-than-default FFT length mid-run based on ROE behavior for the exponent in question, it will immediately resume using the larger FFT length on restart-from-interrupt, rather than resuming using the smaller default FFT length as the current release does.
o 8 bytes for circular-shift to apply to the (unshifted) residue read from the file. I include the shift-count-at-iteration-of-savefile-write because [a] the code will choose a random shift count at run-start time (i.e. since this is not specified by the Primenet server, it cannot be read from the worktodo file), and [b] it saves the need for taking an initial-shift value s from the savefile and computing s * 2^iter (mod p). I remove the shift from R prior to the savefile write, so in fact there's really no need to store s to the file (i.e. I could resume-from-interrupt using an entirely different random shift value, applied to R after reading it from the savefile), but for aesthetic reasons I like the idea of doing the whole run based on a single initial value of s, rather than as-many-values-of-s-as-there-were-run-interrupts.
For V19, LL save file format is the same as for V18, while for PRP, per, additionally, following those fields, are:
  • full-length residue byte-array (this one holding the accumulated Gerbicz checkproduct) ceiling(p/8) bytes for the value G - i.e. maximally byte-compact, endian-and-FFT-length-of-run-independent;
  • 8 bytes for G (mod 2^64), which should match the leading 8 full-residue bytes in the above bytewise form;
  • 5 bytes for G (mod 2^35-1);
  • 5 bytes for G (mod 2^36-1)
The residue byte-arrays are least significant byte first.

I note that the exponent p itself is in the file name, not in the contents. Also note mlucas V19 PRP implementation is type 1 residues only.

Save file names are p<exponent>, q<exponent>, p<exponent>.10M, etc. For example, for M332220523, p33220523, q33220523, p33220523.10M, and eventually p33220523.20M and so on.

File sizes derived from the preceding are:
  • V17.x: 28 + ceiling(p/8) bytes
  • V18, and V19 LL: 39 + ceiling(p/8) bytes
  • V19 PRP: 57 + 2 * ceiling(p/8) bytes
Size of the V17 p332220523 file is 41527594 bytes; check. (Size allocated on disk is larger due to the disk block size.)

(No data yet on V19.1, V20, etc.)

Top of reference tree:

Last fiddled with by kriesel on 2021-01-20 at 17:10 Reason: update for V19 PRP type etc
kriesel is online now