Current post processing status…
 

[CODE]S3m637: linear algebra completed 5090785 of 14819292 dimensions (34.4%, ETA 146h16m)
8647_61_minus1: linear algebra completed 46515 of 16385983 dimensions (0.3%, ETA 581h50m)[/CODE]We are using the .gz dat file for 8647_61_minus1. How much does this slow things down?

[QUOTE=Xyzzy;311912][CODE]S3m637: linear algebra completed 5090785 of 14819292 dimensions (34.4%, ETA 146h16m)
8647_61_minus1: linear algebra completed 46515 of 16385983 dimensions (0.3%, ETA 581h50m)[/CODE]We are using the .gz dat file for 8647_61_minus1. How much does this slow things down?[/QUOTE]
BL iterations don't even need the .dat ( or .dat.gz) files. So the answer is: N/A. (If you want to split hairs and add matrix building into linear algebra time, you could be off by +/- few minutes.)

Your 8647_61_minus1 must be heavily swapping. All things being equal, its ETA should have been expected to be (16385983/14819292)^2 * 146.25 hr / (1-0.344) = 272.5 hr

No swapping! (Seriously! There is no swap space!)

:mike:

[CODE]top - 21:47:29 up 4 days, 12:00, 3 users, load average: 3.71, 3.71, 3.71
Tasks: 155 total, 4 running, 151 sleeping, 0 stopped, 0 zombie
Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 12279364k total, 11744944k used, 534420k free, 4240k buffers
Swap: 0k total, 0k used, 0k free, 3924888k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14649 m 20 0 7333m 7.1g 632 R 399 60.8 781:54.18 ./msieve -s 8647_61_minus1.dat.gz -l 8647_61_minus1.log -i 8647_61_minus1.ini -nf 8647_61_minus1.fb -nc -t 4 -v
1676 root 20 0 153m 32m 4632 S 0 0.3 21:46.30 /usr/bin/Xorg :0 -br -verbose -audit 0 -novtswitch -auth /var/run/gdm3/auth-for-Debian-gdm-4GT2hc/database -nolisten tcp vt7
42 root 20 0 0 0 0 S 0 0.0 3:28.92 [kondemand/0]
4919 m 20 0 85360 1488 524 S 0 0.0 2:44.89 sshd: m@notty
43 root 20 0 0 0 0 S 0 0.0 2:06.99 [kondemand/1]
44 root 20 0 0 0 0 S 0 0.0 2:00.23 [kondemand/2]
45 root 20 0 0 0 0 R 0 0.0 1:52.55 [kondemand/3]
1989 m 20 0 408m 11m 4684 S 0 0.1 0:45.11 gnome-panel
1996 m 20 0 465m 9988 4300 S 0 0.1 0:41.34 nautilus
1974 m 20 0 211m 7340 2392 S 0 0.1 0:32.91 /usr/lib/gnome-settings-daemon/gnome-settings-daemon
4920 m 20 0 12648 996 432 S 0 0.0 0:26.77 /usr/lib/openssh/sftp-server
1990 root 20 0 46540 712 396 S 0 0.0 0:20.21 udisks-daemon: polling /dev/sr0
292 root 20 0 0 0 0 S 0 0.0 0:11.70 [scsi_eh_0]
2006 m 20 0 216m 5948 3248 S 0 0.0 0:10.43 update-notifier
339 root 20 0 0 0 0 S 0 0.0 0:08.36 [kjournald]
1994 m 20 0 131m 1292 848 S 0 0.0 0:06.27 /usr/lib/gvfs/gvfs-afc-volume-monitor
47 root 20 0 0 0 0 S 0 0.0 0:05.17 [kswapd0]
15 root 20 0 0 0 0 S 0 0.0 0:04.91 [events/0]
1509 root 20 0 62684 844 412 S 0 0.0 0:04.66 /usr/sbin/kerneloops
5257 m 20 0 25748 1760 684 S 0 0.0 0:03.62 SCREEN
281 root 20 0 0 0 0 S 0 0.0 0:03.03 [ksuspend_usbd]
1 root 20 0 8356 564 436 S 0 0.0 0:02.62 init [2]
16 root 20 0 0 0 0 S 0 0.0 0:02.42 [events/1]
18 root 20 0 0 0 0 R 0 0.0 0:02.35 [events/3]
17 root 20 0 0 0 0 S 0 0.0 0:02.32 [events/2]
1988 root 20 0 55044 2228 1512 S 0 0.0 0:02.15 /usr/lib/udisks/udisks-daemon
1971 m 20 0 199m 4656 2604 S 0 0.0 0:02.08 gnome-power-manager
1983 m 20 0 265m 7644 4176 S 0 0.1 0:01.80 /usr/bin/metacity
1965 m 20 0 46484 4904 1192 S 0 0.0 0:01.51 /usr/lib/libgconf2-4/gconfd-2
2017 m 20 0 156m 2496 1100 S 0 0.0 0:01.33 gnome-screensaver
2020 m 20 0 61180 1676 1064 S 0 0.0 0:00.99 /usr/lib/gvfs/gvfsd-trash --spawner :1.1 /org/gtk/gvfs/exec_spaw/0
13 root 20 0 0 0 0 S 0 0.0 0:00.78 [ksoftirqd/3]
5228 m 20 0 85184 1260 488 S 0 0.0 0:00.71 sshd: m@pts/0
4 root 20 0 0 0 0 S 0 0.0 0:00.64 [ksoftirqd/0]
1944 m 20 0 11880 380 72 S 0 0.0 0:00.58 /usr/bin/ssh-agent /usr/bin/dbus-launch --exit-with-session /usr/bin/seahorse-agent --execute x-session-manager
7 root 20 0 0 0 0 S 0 0.0 0:00.57 [ksoftirqd/1]
1549 avahi 20 0 33892 1112 800 S 0 0.0 0:00.56 avahi-daemon: running [4.local]
10 root 20 0 0 0 0 S 0 0.0 0:00.49 [ksoftirqd/2]
283 root 20 0 0 0 0 S 0 0.0 0:00.37 [khubd]
1513 messageb 20 0 24140 1548 596 S 0 0.0 0:00.37 /usr/bin/dbus-daemon --system
25 root 20 0 0 0 0 S 0 0.0 0:00.36 [bdi-default]
24 root 20 0 0 0 0 S 0 0.0 0:00.28 [sync_supers]
1986 m 20 0 63540 2108 1416 S 0 0.0 0:00.22 /usr/lib/gvfs/gvfs-gdu-volume-monitor
1236 root 20 0 22428 676 424 S 0 0.0 0:00.20 /usr/sbin/cron
1793 root 20 0 59588 2712 1532 S 0 0.0 0:00.18 /usr/lib/policykit-1/polkitd
2001 m 20 0 163m 3788 2332 S 0 0.0 0:00.17 /usr/lib/gnome-disk-utility/gdu-notification-daemon[/CODE]

What's the density, Kenneth?

So.......
One had 284M raw relations, and the other had 184M? I am surprized that even [STRIKE]kind of[/STRIKE] [STRIKE]sort of[/STRIKE] worked. ETA 581h50m... Jeeeeez.


[CODE]t6 diff 260 skew 1.00, size 9.005e-13, alpha 2.428, combined = 1.068e-13 rroots = 0
t8647 diff 240 skew 0.22, size 4.004e-12, alpha 1.666, combined = 3.350e-13 rroots = 2

#that's 20 digits easier!
[/CODE]

That was undersieved! But I am surprised how relatively little difference 100M relations had on the matrix size. I don't think the computers are equivalent, which is part of the slowdown but I didn't expect a factor of 2. Xyzzy, if you want we can sieve a bit more and bring the size of the matrix down, or we can simply wait 3 weeks for the factors. Whichever you prefer.

Lionel on post #222 warned that 8647_61_minus1 was very undersieved and I agree with him. I think more wu's are necessary even if Xyzzy is not using his fastest machine to do LA phase. 582 hours for LA is a lot!! Let the others do more sieving to reduce to at least half the LA time.

We don't mind having it run that long as long as it works in the end.

The box it is being run on has a SB i5.

We have plenty of boxes so this is no big deal.

The estimated time has improved a bit overnight.

[CODE]linear algebra completed 358639 of 16385983 dimensions (2.2%, ETA 567h 6m)[/CODE]

In my opinion you should wait for more relations but it is up to you.

A 16M matrix is horrendous, indeed. Worse than the hardest number ever sieved by RSALS, somewhat harder than this one, namely a GNFS difficulty 172.

"the cusp" of convergence
 

When someone will ever mention "the cusp" again, guys, know - this is it, exactly.

With a few days additional sieving, the resulting matrix will be below 200hrs. That's two weeks savings.