mersenneforum.org 3,748+ c204 Smaller-but-Needed
 Register FAQ Search Today's Posts Mark Forums Read

2021-11-09, 01:10   #232
charybdis

Apr 2020

11148 Posts

Quote:
 Originally Posted by EdH Wouldn't " -bkthresh1" be seen in the invocation for las, if it was used?
Yes it would, unless the siever automatically invokes it if it sees it doesn't have enough RAM. But I've never seen it do that so I presume it doesn't.

The same command uses the usual 5.2GB of memory for me. Is there any equivalent of "top" in Colab?

2021-11-09, 03:28   #233
EdH

"Ed Hall"
Dec 2009

25·131 Posts

Quote:
 Originally Posted by charybdis Yes it would, unless the siever automatically invokes it if it sees it doesn't have enough RAM. But I've never seen it do that so I presume it doesn't. The same command uses the usual 5.2GB of memory for me. Is there any equivalent of "top" in Colab?
Normally, the session only allows one terminal instance, but I can see if Chalsall's setup is still available, to get in from an ssh terminal while the session is running. I'm probably not going to be able to try that tomorrow.

2021-11-09, 07:24   #234
kruoli

"Oliver"
Sep 2017
Porta Westfalica, DE

839 Posts

Quote:
 Originally Posted by EdH […] the session only allows one terminal instance […]
Is tmux not available/possible on Colab?

Quote:
 Originally Posted by charybdis Looks an awful lot like it's averaging the first 1000 WUs rather than the last 1000...
Good catch, that is wrong indeed, I have changed it. It seems like I have changed too much in one go.

2021-11-09, 09:00   #235
henryzz
Just call me Henry

"David"
Sep 2007
Liverpool (GMT/BST)

3×5×397 Posts

Quote:
 Originally Posted by charybdis Also mystified by the low RAM usage on Colab. It is possible to get CADO to use less memory using -bkthresh1, but this comes at a huge cost in speed, and I don't think it ever gets triggered automatically. Are you sure you didn't catch the RAM usage during initialization when las hadn't started sieving yet? Also I see the average relations per WU figure has gone up from ~5000 to ~6000. Is somebody running clients with A=31?
My experience is that you can save a fair bit of memory using -bkthresh1 without affecting speed significantly. I think I even saw a very small speedup with the right value.

https://www.mersenneforum.org/showth...sh1#post567015
16 GB to 6 GB with a slight speedup.

Mileage may vary of course.

2021-11-09, 12:04   #236
charybdis

Apr 2020

22×3×72 Posts

Quote:
 Originally Posted by henryzz My experience is that you can save a fair bit of memory using -bkthresh1 without affecting speed significantly. I think I even saw a very small speedup with the right value.
Well well... I tried -bkthresh1 and discovered that the latest CADO has a bug! It was probably introduced during the recent improvements to las. I'll report it to the CADO team later.

Using an older revision, I couldn't seem to avoid losing speed with -bkthresh1 relative to the default. Generally larger values use less memory but sieve more slowly.

2021-11-09, 14:37   #237
EdH

"Ed Hall"
Dec 2009

106016 Posts

Quote:
 Originally Posted by kruoli Is tmux not available/possible on Colab?. . .
I use tmux all the time with all my machines and I didn't think of it! But, the Colab console is not a standard terminal, so I'm not sure if it will work in that manner. The ssh instance Chalsall provided did operate a standard terminal from a remote machine accessing the instance via ssh. I will not be able to try anything today, but perhaps tomorrow.

2021-11-09, 14:52   #238
henryzz
Just call me Henry

"David"
Sep 2007
Liverpool (GMT/BST)

3·5·397 Posts

Quote:
 Originally Posted by charybdis Well well... I tried -bkthresh1 and discovered that the latest CADO has a bug! It was probably introduced during the recent improvements to las. I'll report it to the CADO team later. Using an older revision, I couldn't seem to avoid losing speed with -bkthresh1 relative to the default. Generally larger values use less memory but sieve more slowly.

Too high -bkthresh certainly will slow it down. I think you want a fairly equal amount of work before and after the threshold. What values have you tried? Finding a good value may be difficult with different fb lims. I would suggest trying 1e8 or 5e7. Too small will have no effect on memory but shouldn't affect performance. I will experiment when I have time.

2021-11-09, 18:47   #239
charybdis

Apr 2020

24C16 Posts

Quote:
 Originally Posted by henryzz What values have you tried? Finding a good value may be difficult with different fb lims. I would suggest trying 1e8 or 5e7.
I tried 1e8, 5e7, 1e7 and 2e6. All were at least 20% slower than without -bkthresh1.

2021-11-09, 23:33   #240
henryzz
Just call me Henry

"David"
Sep 2007
Liverpool (GMT/BST)

3·5·397 Posts

Quote:
 Originally Posted by charybdis I tried 1e8, 5e7, 1e7 and 2e6. All were at least 20% slower than without -bkthresh1.
That's weird. My system takes 185s with 3e7 and 322s without bkthresh1

Code:
./las -poly 3_748plus.poly -q0 100000000 -A 30 -nq 20 -lim0 500000000 -lim1 300000000 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 3_748plus.afb -t 4 -stats-stderr -v -bkmult 1,1l:1.08598,1s:1.30264,2s:1.08933
My version of CADO is from May 2021.

It looks like people may need to try it on their own systems. I am not sure why our results are so different.
Still, 20% slower is better than not being able to run. If it allows increasing A or I then 20% slower may be worth it as well.

2021-11-09, 23:53   #241
charybdis

Apr 2020

22×3×72 Posts

Quote:
 Originally Posted by henryzz That's weird. My system takes 185s with 3e7 and 322s without bkthresh1 Code: ./las -poly 3_748plus.poly -q0 100000000 -A 30 -nq 20 -lim0 500000000 -lim1 300000000 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 3_748plus.afb -t 4 -stats-stderr -v -bkmult 1,1l:1.08598,1s:1.30264,2s:1.08933 My version of CADO is from May 2021. It looks like people may need to try it on their own systems. I am not sure why our results are so different. Still, 20% slower is better than not being able to run. If it allows increasing A or I then 20% slower may be worth it as well.
Strange. I get 136s with -bkthresh1 3e7 and 108s without. Adding in your -bkmult 1,1l:1.08598,1s:1.30264,2s:1.08933 changes this to 123s with -bkthresh1 and 112s without. I doubt my system is 3 times faster than yours, so something must be making yours run much more slowly than it should without -bkthresh1.

Still, -bkthresh1 is a useful option for expanding participation in big jobs, probably just as an --override though.

Last fiddled with by charybdis on 2021-11-09 at 23:54

 2021-11-10, 14:07 #242 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 25×131 Posts I couldn't get top or tmux to work in a familiar manner with Colab and it looks like Chalsall has taken down his ssh tunnel server, which isn't surprising, since I don't think anyone used it in more than a year.

 Similar Threads Thread Thread Starter Forum Replies Last Post fivemack Factoring 3 2017-09-19 08:52 skan YAFU 6 2013-02-26 13:57 akruppa Factoring 114 2012-08-20 14:01 fortega Data 2 2005-06-16 22:48 marc Factoring 6 2004-10-09 14:17

All times are UTC. The time now is 20:33.

Sat Jan 22 20:33:23 UTC 2022 up 183 days, 15:02, 0 users, load averages: 2.00, 2.07, 2.00