mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Cunningham Tables

Reply
 
Thread Tools
Old 2021-11-08, 13:42   #221
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

25×131 Posts
Default

Excellent! I was getting ready to post a three column version today, but no extra as you've done. I like the trimming. I've also been playing around with adding a unique clients count per day, but not very aggressively. I'm not really sure it would add anything. I see you changed the 400 to 1000 for comparison. I've been wondering if 400 might be better increased.
EdH is offline   Reply With Quote
Old 2021-11-08, 14:00   #222
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

839 Posts
Default

Quote:
Originally Posted by EdH View Post
I've also been playing around with adding a unique clients count per day, but not very aggressively. I'm not really sure it would add anything.
This would likely cause really weird results because I had some clients running with --single in a loop and not having --clientid set. So I have hundreds of "unique" clients.
kruoli is offline   Reply With Quote
Old 2021-11-08, 14:24   #223
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

101408 Posts
Default

Quote:
Originally Posted by kruoli View Post
This would likely cause really weird results because I had some clients running with --single in a loop and not having --clientid set. So I have hundreds of "unique" clients.
I had thought about that as well. My machines are all running --single, but I provide an ID. OTOH, all those unique IDs would show a lot of interest at that time!
EdH is offline   Reply With Quote
Old 2021-11-08, 17:20   #224
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

2·2,579 Posts
Default

Quote:
Originally Posted by EdH View Post
Excellent! I was getting ready to post a three column version today, but no extra as you've done. I like the trimming. I've also been playing around with adding a unique clients count per day, but not very aggressively. I'm not really sure it would add anything. I see you changed the 400 to 1000 for comparison. I've been wondering if 400 might be better increased.
When I requested 100 and 400, I had dropped a zero in my mental conversion from q-range to number of workunits.
Glad you changed 400 to 1000.
Comparing current yield to overall-job-yield tells us most of the relevant info about how yield is falling off, anyway.
VBCurtis is offline   Reply With Quote
Old 2021-11-08, 18:22   #225
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

25·131 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
When I requested 100 and 400, I had dropped a zero in my mental conversion from q-range to number of workunits.
Glad you changed 400 to 1000.
Comparing current yield to overall-job-yield tells us most of the relevant info about how yield is falling off, anyway.
Credit Oliver with the change. Should it be even greater?

Curiosity: I have a Colab instance running ATM:
Code:
INFO:root:Running /content/cado-nfs/build/e202773b5d6e/sieve/las
 -poly download/3_748plus.poly -q0 356442000 -A 30 -q1 356444000 -lim0 500000000 -lim1 300000000
 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 download/3_748plus.roots1.gz
 -out Colab.eFarm.work/3_748plus.356442000-356444000.gz -t 2 -adjust-strategy 2 -stats-stderr
INFO:root:[Mon Nov  8 16:36:57 2021] Subprocess has PID 12961
The resources show only 2.14GB of 12.69 available, in use. I didn't think RAM use was tied to threads, or at least not too closely. Something odd here?
EdH is offline   Reply With Quote
Old 2021-11-08, 21:08   #226
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

10100001001102 Posts
Default

Quote:
Originally Posted by EdH View Post
Credit Oliver with the change. Should it be even greater?
I didn't think RAM use was tied to threads, or at least not too closely. Something odd here?
I think 1000 and 10000 would be better than 400 and 1000; but it may be that yield just doesn't change much even over 10000 workunits, and just one "most recent yield" calculation has any value.

As for RAM: Seems odd to me. My guess is that CADO doesn't "see" all the available ram due to some colab-specific setting, and is running in minimal-ram configuration. My production runs have used 4,6,8,10 threads per client, and all have similar memory use.
VBCurtis is offline   Reply With Quote
Old 2021-11-08, 21:27   #227
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

839 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
I think 1000 and 10000 would be better than 400 and 1000 [...].
The values are now at 1000 and 25000. If this is too large I can change it back.
Quote:
Originally Posted by VBCurtis View Post
[...] all have similar memory use.
I saw a small increase of memory footprint when increasing the thread count. I also had a test with 16 threads that was slower than a single worker with 8 cores on a machine with 16 physical cores.
kruoli is offline   Reply With Quote
Old 2021-11-08, 23:50   #228
charybdis
 
charybdis's Avatar
 
Apr 2020

22·3·72 Posts
Default

Also mystified by the low RAM usage on Colab. It is possible to get CADO to use less memory using -bkthresh1, but this comes at a huge cost in speed, and I don't think it ever gets triggered automatically. Are you sure you didn't catch the RAM usage during initialization when las hadn't started sieving yet?

Also I see the average relations per WU figure has gone up from ~5000 to ~6000. Is somebody running clients with A=31?
charybdis is offline   Reply With Quote
Old 2021-11-09, 00:29   #229
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

2×2,579 Posts
Default

Quote:
Originally Posted by charybdis View Post
Also I see the average relations per WU figure has gone up from ~5000 to ~6000. Is somebody running clients with A=31?
Not guilty on A=31.

Did the yield jump right at Q=300M?
VBCurtis is offline   Reply With Quote
Old 2021-11-09, 00:58   #230
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

25×131 Posts
Default

The Colab instances are running rather slow at around 40 minutes per WU, but they are limited to two threads:
Code:
INFO:root:Running /content/cado-nfs/build/e202773b5d6e/sieve/las -poly download/3_748plus.poly -q0 362096000 -A 30 -q1 362098000 -lim0 500000000 -lim1 300000000 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 download/3_748plus.roots1.gz -out Colab.eFarm.work/3_748plus.362096000-362098000.gz -t 2 -adjust-strategy 2 -stats-stderr
INFO:root:[Tue Nov  9 00:01:07 2021] Subprocess has PID 18277
. . .
INFO:root:Running /content/cado-nfs/build/e202773b5d6e/sieve/las -poly download/3_748plus.poly -q0 362614000 -A 30 -q1 362616000 -lim0 500000000 -lim1 300000000 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 download/3_748plus.roots1.gz -out Colab.eFarm.work/3_748plus.362614000-362616000.gz -t 2 -adjust-strategy 2 -stats-stderr
INFO:root:[Tue Nov  9 00:40:25 2021] Subprocess has PID 18749
As you can see, las was started at 00:40:25 and I took the following check at 00:50:40:
Code:
Waiting to finish the current execution.
Python 3 Google Compute Engine backend

RAM: 3.16 GB/12.69 GB Disk: 46.70 GB/107.72 GB
I just did another check at 00:55:42 and it was identical.

Wouldn't " -bkthresh1" be seen in the invocation for las, if it was used?
EdH is offline   Reply With Quote
Old 2021-11-09, 01:01   #231
charybdis
 
charybdis's Avatar
 
Apr 2020

22·3·72 Posts
Default

Aha, I think I've got it. I noticed that the averages seemed to be stuck. And indeed:

Quote:
Originally Posted by kruoli View Post
Code:
lastarr=($(cat "$relationCountsLogFile")) <-- should be "tac"?

...

lastarrsize=${#lastarr[*]}
while [ $avcount -lt $lastarrsize ]
  do
    let avsum=${avsum}+${lastarr[$avcount]}
    let avcount=${avcount}+1
    if [ $avcount -eq $smallCount ]
      then
        let avsmall=${avsum}/$smallCount
    fi
    if [ $avcount -eq $bigCount ]
      then
        let avbig=${avsum}/$bigCount
    fi
  done
let avtotal=$avsum/$avcount
Looks an awful lot like it's averaging the first 1000 WUs rather than the last 1000...

Last fiddled with by charybdis on 2021-11-09 at 01:02
charybdis is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Using 16e on smaller numbers fivemack Factoring 3 2017-09-19 08:52
NFS on smaller numbers? skan YAFU 6 2013-02-26 13:57
Bernoulli(200) c204 akruppa Factoring 114 2012-08-20 14:01
checking smaller number fortega Data 2 2005-06-16 22:48
Factoring Smaller Numbers marc Factoring 6 2004-10-09 14:17

All times are UTC. The time now is 20:32.


Sat Jan 22 20:32:04 UTC 2022 up 183 days, 15:01, 0 users, load averages: 2.55, 2.16, 2.03

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

≠ ± ∓ ÷ × · − √ ‰ ⊗ ⊕ ⊖ ⊘ ⊙ ≤ ≥ ≦ ≧ ≨ ≩ ≺ ≻ ≼ ≽ ⊏ ⊐ ⊑ ⊒ ² ³ °
∠ ∟ ° ≅ ~ ‖ ⟂ ⫛
≡ ≜ ≈ ∝ ∞ ≪ ≫ ⌊⌋ ⌈⌉ ∘ ∏ ∐ ∑ ∧ ∨ ∩ ∪ ⨀ ⊕ ⊗ 𝖕 𝖖 𝖗 ⊲ ⊳
∅ ∖ ∁ ↦ ↣ ∩ ∪ ⊆ ⊂ ⊄ ⊊ ⊇ ⊃ ⊅ ⊋ ⊖ ∈ ∉ ∋ ∌ ℕ ℤ ℚ ℝ ℂ ℵ ℶ ℷ ℸ 𝓟
¬ ∨ ∧ ⊕ → ← ⇒ ⇐ ⇔ ∀ ∃ ∄ ∴ ∵ ⊤ ⊥ ⊢ ⊨ ⫤ ⊣ … ⋯ ⋮ ⋰ ⋱
∫ ∬ ∭ ∮ ∯ ∰ ∇ ∆ δ ∂ ℱ ℒ ℓ
𝛢𝛼 𝛣𝛽 𝛤𝛾 𝛥𝛿 𝛦𝜀𝜖 𝛧𝜁 𝛨𝜂 𝛩𝜃𝜗 𝛪𝜄 𝛫𝜅 𝛬𝜆 𝛭𝜇 𝛮𝜈 𝛯𝜉 𝛰𝜊 𝛱𝜋 𝛲𝜌 𝛴𝜎 𝛵𝜏 𝛶𝜐 𝛷𝜙𝜑 𝛸𝜒 𝛹𝜓 𝛺𝜔