Truncation of PrimeNet Summary Report

eepiccolo · 2003-03-12, 17:06

OK, try going to the PrimeNet summary report at http://www.mersenne.org/primenet/summary.txt. Chances are, you would see a report truncated around 18000000. The report has behaved like this for as long as I have been in the project (5 months), and only occasionally when you look at the summary will you get a full report. The reason I bring this up now is that the first-time test are in the 18,000,000 range now, whereas before, the truncated report was not such a big issue.
So, is there any way that the report can be fixed to behave properly and consistently?
Oh, and another issue is that the report claims to be updated hourly, but right now, it is 1705 UTC, and the report says it was lat updated at 1200 UTC.

Tim (eepiccolo)

jeff8765 · 2003-03-12, 23:38

The report used to update every hour and go to the full length within 15 minutes. Around 6 months ago it started taking about 6 hours and being complete only for about the last half hour. I don't know why it did that though.

NickGlover · 2003-03-13, 01:55

Quote:

Originally Posted by jeff8765

The report used to update every hour and go to the full length within 15 minutes. Around 6 months ago it started taking about 6 hours and being complete only for about the last half hour. I don't know why it did that though.

Also, at the same time that it started taking really long to generate, it started showing a bunch of lines like

[code:1]******** ******** 1[/code:1]

at the very end of the report. I'm guessing this issue is related to the really long generation time.

cheesehead · 2003-03-13, 04:53

Possible scenario: The report generator is failing to find, or to understand, some end-of-data indicator for the PrimeNet data base, and is continuing to process junk past the last valid data for quite a while. Once in a while the junk is close enough to valid data format for the program to interpret as an interval with bounds over 10^8, so it tries to spit out another report line. The language in which the generator is written handles integer field overflow by substituting asterisks instead of showing a truncated value or pushing the rest of the record to the right to accommodate outputting the entire overflowing value.

That behavior started only a few months ago because only then did data values exceed some limit below which the generator processed correctly.

Prediction: It will continue until someone fixes the report generator bug.

Advice: Until bug fix, set an alarm clock for about 5.5 hours past the time shown in the report heading to remind you to go back to see the complete report during the window of display completeness, which is from then until the end of that clock hour. That 5.5 hour interval will steadily increase over time, and the corresponding duration of the completeness window will decrease, until reaching 6 hours, at which time the completeness window will jump back to almost one hour then decrease again. In the absence of a bug fix, the interval will continue to 7, 8, 9 ... hours past report heading time, unless another program limitation or bug causes an abort at some point.

trif · 2003-03-13, 06:38

Several months before the report started having this multi-hour behavior, some members of Team Prime Rib asked George for some whopper exponents, up in the 79.2M range (what can I say, these guys are crazy, gotta love 'em, 2 months just to do the P-1). George put them in Primenet and let them pick them up through doing a manual connect. When that happened the report went from taking 5 minutes to complete to taking about 15 minutes. Since there were only a few added exponents, it was obvious that the report program was doing some kind of linear query on possibly every prime exponent between 0 and the upper limit of what was in Primenet, and thus having to compute and look up everything between the 33M's that were the highest and the 79M's that were the new highest, and this was delaying the report.

So perhaps something is corrupted and Primenet thinks the upper bound is somewhere in the several hundred millions or billions. If Primenet has to test every exponent for primality, that becomes more and more expensive the higher you get. Perhaps someone manually stuck some really big exponents in their worktodo and did a connect and this screwed Primenet up.

And just to give eepiccolo a teasing poke, it's been an issue for those of us who work factoring for quite some time. ;) I just deal by estimating that five and a half hours, as cheesehead details. So I get a full report four times a day.

eepiccolo · 2003-03-13, 13:21

Quote:

Originally Posted by trif

And just to give eepiccolo a teasing poke, it's been an issue for those of us who work factoring for quite some time. ;) I just deal by estimating that five and a half hours, as cheesehead details. So I get a full report four times a day.

Whoops

ops: . Sorry about that factorers (Is that a word? :( ). And thanks for the four report a day info; I was unaware of that.

norbert · 2003-03-13, 13:32

I have this line in my crontab:
[code:1]
50 * * * * cd /home/norbert/bin; ./crday
[/code:1]

where crday is a perl script that contains the following lines (among others):

[code:1]
$fn= "prime/status/summary";
$url="'http://mersenne.org/primenet/summary.txt'";
open FILE, "/usr/bin/lynx -source $url |";
@lines=<FILE>;
close FILE;

if (grep /33500000/,@lines)
{
open OUT,">$fn";
print OUT @lines;
close OUT;
}
[/code:1]

tha · 2003-03-26, 12:24

Quote:

Originally Posted by cheesehead

The language in which the generator is written handles integer field overflow by substituting asterisks instead of showing a truncated value or pushing the rest of the record to the right to accommodate outputting the entire overflowing value.

I noticed that the number of lines went up in the past weekend, after the long outage.

Also, this suggest that the server could be fixed with relative little work, right?

tha · 2003-03-26, 21:10

Given the fact that it is frequent down now, just after the sudden increase in these lines, suggests that the server breaking down once or twice a day will continue until the root cause has been eliminated.

YotN,

Henk.

NickGlover · 2003-03-26, 22:00

Quote:

Originally Posted by tha

I noticed that the number of lines went up in the past weekend, after the long outage.

Actually, the number of messed up lines has been steadily going down.

Some samples from my saved archive of summary.txt files:
31-Jan-03: 36
14-Feb-03: 33
21-Feb-03: 27
01-Mar-03: 18
14-Mar-03: 14
20-Mar-03: 13
21-Mar-03: 12
25-Mar-03: 12

I'm guessing the extremely high exponents are expiring, and once they all expire, hopefully the summary will start taking 20 minutes or less again.

tha · 2003-03-27, 08:43

Quote:

Originally Posted by NickGlover

Actually, the number of messed up lines has been steadily going down.

Hmm, yesterday, I was pretty sure a saw a screen full of them, today I can't find that many. The last line however stops halfway, the "1" is missing suggesting the report is not fully printed before a new starts.

2003-03-12, 17:06	#1
eepiccolo Dec 2002 Frederick County, MD 2×5×37 Posts	Truncation of PrimeNet Summary Report OK, try going to the PrimeNet summary report at http://www.mersenne.org/primenet/summary.txt. Chances are, you would see a report truncated around 18000000. The report has behaved like this for as long as I have been in the project (5 months), and only occasionally when you look at the summary will you get a full report. The reason I bring this up now is that the first-time test are in the 18,000,000 range now, whereas before, the truncated report was not such a big issue. So, is there any way that the report can be fixed to behave properly and consistently? Oh, and another issue is that the report claims to be updated hourly, but right now, it is 1705 UTC, and the report says it was lat updated at 1200 UTC. Tim (eepiccolo)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Machines Missing from Primenet Summary	edron1011	PrimeNet	2	2008-11-17 19:16
PrimeNet Report	ZFR	PrimeNet	4	2008-02-13 12:14
Asterisk in Primenet Account Report	jinydu	PrimeNet	5	2006-10-30 02:55
Core 2 Duo in primenet report	tha	PrimeNet	3	2006-08-06 22:19
Once again: How to report offline-work to primenet?	Unregistered	PrimeNet	19	2004-01-22 23:41

2003-03-12, 23:38	#2
jeff8765 Aug 2002 A Dyson Sphere 63₁₀ Posts	The report used to update every hour and go to the full length within 15 minutes. Around 6 months ago it started taking about 6 hours and being complete only for about the last half hour. I don't know why it did that though.

2003-03-13, 04:53	#4
cheesehead "Richard B. Woods" Aug 2002 Wisconsin USA 17014₈ Posts	Possible scenario: The report generator is failing to find, or to understand, some end-of-data indicator for the PrimeNet data base, and is continuing to process junk past the last valid data for quite a while. Once in a while the junk is close enough to valid data format for the program to interpret as an interval with bounds over 10^8, so it tries to spit out another report line. The language in which the generator is written handles integer field overflow by substituting asterisks instead of showing a truncated value or pushing the rest of the record to the right to accommodate outputting the entire overflowing value. That behavior started only a few months ago because only then did data values exceed some limit below which the generator processed correctly. Prediction: It will continue until someone fixes the report generator bug. Advice: Until bug fix, set an alarm clock for about 5.5 hours past the time shown in the report heading to remind you to go back to see the complete report during the window of display completeness, which is from then until the end of that clock hour. That 5.5 hour interval will steadily increase over time, and the corresponding duration of the completeness window will decrease, until reaching 6 hours, at which time the completeness window will jump back to almost one hour then decrease again. In the absence of a bug fix, the interval will continue to 7, 8, 9 ... hours past report heading time, unless another program limitation or bug causes an abort at some point.

2003-03-13, 06:38	#5
trif Aug 2002 2×101 Posts	Several months before the report started having this multi-hour behavior, some members of Team Prime Rib asked George for some whopper exponents, up in the 79.2M range (what can I say, these guys are crazy, gotta love 'em, 2 months just to do the P-1). George put them in Primenet and let them pick them up through doing a manual connect. When that happened the report went from taking 5 minutes to complete to taking about 15 minutes. Since there were only a few added exponents, it was obvious that the report program was doing some kind of linear query on possibly every prime exponent between 0 and the upper limit of what was in Primenet, and thus having to compute and look up everything between the 33M's that were the highest and the 79M's that were the new highest, and this was delaying the report. So perhaps something is corrupted and Primenet thinks the upper bound is somewhere in the several hundred millions or billions. If Primenet has to test every exponent for primality, that becomes more and more expensive the higher you get. Perhaps someone manually stuck some really big exponents in their worktodo and did a connect and this screwed Primenet up. And just to give eepiccolo a teasing poke, it's been an issue for those of us who work factoring for quite some time. ;) I just deal by estimating that five and a half hours, as cheesehead details. So I get a full report four times a day.

2003-03-13, 13:32	#7
norbert Aug 2002 21₁₀ Posts	I have this line in my crontab: [code:1] 50 * * * * cd /home/norbert/bin; ./crday [/code:1] where crday is a perl script that contains the following lines (among others): [code:1] $fn= "prime/status/summary"; $url="'http://mersenne.org/primenet/summary.txt'"; open FILE, "/usr/bin/lynx -source $url \|"; @lines=<FILE>; close FILE; if (grep /33500000/,@lines) { open OUT,">$fn"; print OUT @lines; close OUT; } [/code:1]

2003-03-26, 21:10	#9
tha Dec 2002 7·11² Posts	Given the fact that it is frequent down now, just after the sudden increase in these lines, suggests that the server breaking down once or twice a day will continue until the root cause has been eliminated. YotN, Henk.