I don't understand these relation counts

fivemack · 2012-10-28, 22:22

Code:

Wed Oct  3 10:35:56 2012  found 28112108 hash collisions in 218362761 relations
Wed Oct  3 11:36:36 2012  found 22314936 duplicates and 197266829 unique relations
Wed Oct  3 15:33:43 2012  begin with 197266829 relations and 280985260 unique ideals
------
Sun Oct 28 19:00:41 2012  found 47095370 hash collisions in 288197220 relations
Sun Oct 28 19:41:43 2012  found 39606683 duplicates and 249809541 unique relations
Sun Oct 28 21:29:32 2012  begin with 249809541 relations and 264041178 unique ideals

The run today is using a strict superset of the relations from 3 October. I've added seventy million raw relations, and sixteen million unique ideals seem to have disappeared.

henryzz · 2012-10-28, 22:37

Have a lot of the ideals now got weights above 200 that hadn't before? Has the target excess shot up by 16M?

jasonp · 2012-10-29, 00:24

That's what I bet is happening. Either that or the large prime bound is much higher in the second case.

fivemack · 2012-10-29, 08:00

Yes, it was using a very different large-ideal bound the second time through, and I hadn't noticed because I read the log files with grep rather than from the top to the bottom. In the first case

Code:

Wed Oct  3 10:37:32 2012  commencing duplicate removal, pass 2
Wed Oct  3 11:36:36 2012  found 22314936 duplicates and 197266829 unique relations
Wed Oct  3 11:36:36 2012  memory use: 852.8 MB
Wed Oct  3 11:36:37 2012  reading ideals above 720000
Wed Oct  3 11:36:37 2012  commencing singleton removal, initial pass
Wed Oct  3 15:11:04 2012  memory use: 6024.0 MB
Wed Oct  3 15:11:06 2012  reading all ideals from disk
Wed Oct  3 15:19:46 2012  memory use: 8543.4 MB
Wed Oct  3 15:24:41 2012  keeping 280985260 ideals with weight <= 200, target excess is 982466
Wed Oct  3 15:30:00 2012  commencing in-memory singleton removal
Wed Oct  3 15:33:43 2012  begin with 197266829 relations and 280985260 unique ideals
Wed Oct  3 15:41:18 2012  reduce to 36019 relations and 0 ideals in 8 passes
Wed Oct  3 15:41:18 2012  max relations containing the same ideal: 0

and in the second

Code:

Sun Oct 28 19:01:29 2012  commencing duplicate removal, pass 2
Sun Oct 28 19:41:43 2012  found 39606683 duplicates and 249809541 unique relations
Sun Oct 28 19:41:43 2012  memory use: 1321.5 MB
Sun Oct 28 19:41:43 2012  reading ideals above 428670976
Sun Oct 28 19:41:43 2012  commencing singleton removal, initial pass
Sun Oct 28 21:25:25 2012  memory use: 6024.0 MB
Sun Oct 28 21:25:26 2012  reading all ideals from disk
Sun Oct 28 21:27:15 2012  memory use: 4233.5 MB
Sun Oct 28 21:28:25 2012  commencing in-memory singleton removal
Sun Oct 28 21:29:32 2012  begin with 249809541 relations and 264041178 unique ideals
Sun Oct 28 21:41:33 2012  reduce to 60249620 relations and 38031770 ideals in 34 passes
Sun Oct 28 21:41:33 2012  max relations containing the same ideal: 20
Sun Oct 28 21:41:51 2012  reading ideals above 720000
Sun Oct 28 21:41:51 2012  commencing singleton removal, initial pass
Sun Oct 28 22:43:21 2012  memory use: 1506.0 MB
Sun Oct 28 22:43:22 2012  reading all ideals from disk
Sun Oct 28 22:44:34 2012  memory use: 2589.1 MB
Sun Oct 28 22:45:01 2012  keeping 81610960 ideals with weight <= 200, target excess is 308877
Sun Oct 28 22:45:27 2012  commencing in-memory singleton removal
Sun Oct 28 22:45:53 2012  begin with 60249620 relations and 81610960 unique ideals
Sun Oct 28 22:48:26 2012  reduce to 11516 relations and 0 ideals in 13 passes
Sun Oct 28 22:48:26 2012  max relations containing the same ideal: 0

I've clearly misunderstood what 'reading all ideals from disk' meant - I thought it meant 'all ideals' as opposed to 'ideals above large_ideal_bound'.

jasonp · 2012-10-29, 11:26

It does mean 'all ideals selected to participate in the filtering', not 'all ideals'.

2012-10-29, 00:24	#3
jasonp Tribal Bullet Oct 2004 DCE₁₆ Posts	That's what I bet is happening. Either that or the large prime bound is much higher in the second case. Last fiddled with by jasonp on 2012-10-29 at 00:32

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Can you make mew understand the LLR Test?	a1call	Miscellaneous Math	9	2016-09-20 23:04
Don't understand this mfaktc behavior	Chuck	GPU Computing	2	2011-06-18 17:24
Finally I understand EVERYTHING!	Flatlander	Lounge	2	2010-10-10 14:30
Don't understand instructions for upgrading to v25	John S	Software	22	2008-11-10 21:20
I just don't understand this	kuratkull	Sierpinski/Riesel Base 5	2	2007-03-12 21:02

2012-10-28, 22:37	#2
henryzz Just call me Henry "David" Sep 2007 Cambridge (GMT/BST) 2×2,897 Posts	Have a lot of the ideals now got weights above 200 that hadn't before? Has the target excess shot up by 16M?

2012-10-29, 11:26	#5
jasonp Tribal Bullet Oct 2004 6716₈ Posts	It does mean 'all ideals selected to participate in the filtering', not 'all ideals'.