Gigaflops rate greatly exceeds trend predicted 2 1/2 years ago
 

At the by-now rather obsolete page
[url]http://mersenne.org/ips/stats.html[/url]
a graph show a linear progression in Gigaflops for the GIMPS project.

The regression formula given is:
y = 1.2658 x - 57.418

Where y is the rate in Gigaflops
and x is the number of days since Nov 23 1997.

The graph shows the plot from Nov 23 1997 to Mar 9 2001 (1202 days).

If we extrapolate the regression line to Oct 19 2003 (2155 days), we get a prediction of 2670 Gigaflops.

In fact, though, the Primenet server is currently reporting 7698 Gigaflops, exceeding the prediction by almost a factor of 3.


I wonder what can explain this? I don't think SSE2 code for Pentium 4s or the arrival of large teams like Team_Prime_Rib, alone, can explain a factor of 3.

It would be interesting to see what happened to the graph after Nov 14 2001 when M39 was discovered.

It is mainly SSE2 and also the fact that modern cpu's have gotten very fast. Late 2000 I bought my first GHz cpu. Now I couldn't be bothered to run prime95 on it. My 2.4C @ 3.0 does the work of six 1.1 tbirds.

Team Prime Rib has about 600 GHz of power active now. This will increase as our woes with curtisc have resulted in a call for reinforcements - one of the benefits of being with a large group.

I don't think that "modern CPUs getting very fast" can explain a radical difference in the trends from 1997-2001 versus 2001-2003. Moore's law hasn't suddenly accelerated over the past 2 1/2 years. Then again Moore's law would predict exponential growth, so why does the data from 1997 to 2001 show only a straight-line trend?

SSE2 can't explain it. The speed gain in version 23 is "only" 30-40%, not a factor of 3.

A team like Team_Prime_Rib, despite its number one position, probably contributes less than (wild guess) 10% of the overall crunching power. So that wouldn't explain it either.

Team Prime Rib has contributed 3.7% of the total GIMPS work (sum CPU-years LL and factoring).

Accumulative contributions:
top 1 - 3.7%
top 10 - 12.5%
top 100 - 28%
top 1000 - 60%
top 10000 - 94%

(Data from a few weeks back.)

So user contributions is an exponential distribution as well, it's not just a 'top few' doing everything.

I won't attempt to guess at reasons for the increase, other than noting that exponential functions start out pretty much linear.

It'll be interesting to see how the graph goes after the $100k prize is claimed :)

-- Craig

True enough - as motivation factors go, it is hard to beat $100K :)

While v23 is only 30-40% better v21 was more than twice as fast as v20. In that respect v23 is about 3 times as fast - if not more - on a P4 than the version out in early 2001 was.

I see several factors:

1 CPUs and FSBs are getting faster; Moore's Law is exponential.
2 George continues to optimize for the latest chips.
3 Worldwide, more people now have PCs at home.
4 People who have a PC are more likely to own more than 1 PC.
5 A higher fraction of PC owners choose to run DC projects e.g. GIMPS.
6 DC projects are likely to speed up when the economy improves: more companies have more PCs to borg, and there are more employees to borg them!
7 A DC project enjoys a burst after publicity for that project: UD, Seti@Home, and others have seen that.
...

Some of these trends may saturate, so that you have a sigmoidal curve instead of an exponential:

8 Moore noted that speed will grow more slowly.
9 PC ownership will tend to saturate at some % of population.
10 Some PC owners will stop buying the latest and greatest because what they have meets their needs.
...

I suspect an additional factor may have been the completion of RC5-64 - when one of the largest (by participant count) DC projects ends, that leaves a LOT of folks (like me) looking for something new to do....

This is partly correct. Where I am, there are at least a dozen teams ... TPR grew out of the fallout of a stats debacle on another project, the last straw on the part of bad management. Within four months trial factoring crown was ours ... Of course this only applies to dc fanatics ... projects like seti get most of their power from casual users and with gimps there are only two teams to speak of ...

[QUOTE][i]Originally posted by QuintLeo [/i]
[B]I suspect an additional factor may have been the completion of RC5-64 - when one of the largest (by participant count) DC projects ends, that leaves a LOT of folks (like me) looking for something new to do.... [/B][/QUOTE]

Now that I have a bunch of old summary.txt files, we can take another look:

[url=http://opteron.mersenneforum.org/png/LLspeed.png][size=5]LL-testing[/size] P90 years/day (7 day average)[/url]

We see that the data follows one straight-line trend until late 2001, then it follows another straight-line trend with a much steeper slope.


For good measure, we can also look at:

[url=http://opteron.mersenneforum.org/png/TFspeed.png][size=5]Trial factoring[/size] P90 years/day (7 day average)[/url]

Here the data follows a consistent straight-line trend, with no inflection in 2001.

[QUOTE][i]Originally posted by mephisto [/i]
[B]True enough - as motivation factors go, it is hard to beat $100K :) [/B][/QUOTE]

The [url=http://www.eff.org/awards/prime-release1.html]EFF announced the $100K prize on March 31 1999[/url], and the first 10-million digit primes started being assigned by PrimeNet in September 1999.


However, that had no visible effect on the graph: it maintained the same straight line trajectory for another two years.

The change of slope in late 2001 seems to coincide with the arrival of Pentium 4s. See the [url=http://opteron.mersenneforum.org/png/machines.png]graph of machine types[/url] over time. It might also coincide with the [url=http://www.mersenne.org/13466917.htm]discovery of M39 on Nov 14 2001[/url] (exactly two years ago today).

[QUOTE][i]Originally posted by QuintLeo [/i]
[B]I suspect an additional factor may have been the completion of RC5-64 - when one of the largest (by participant count) DC projects ends, that leaves a LOT of folks (like me) looking for something new to do.... [/B][/QUOTE]

[url=http://www.distributed.net/pressroom/news-20020926.html]RC5-64 ended July 14 2002[/url]. Or actually, the winning key was not noticed until Aug 12 2002, and an announcement was made Sep 25 2002.

There was no noticeable change in the slope of the curve at those times (although there's a bit of a data gap at that point).

Maybe the bulk of new users taking the default work "that makes the most sense" took a dramatic increase in first time LL testing due to typical machine performance suddenly being above the threshold criteria for that task. Perhaps there would have been a slope increase in trial factoring too otherwise.

Is there any way of estimating whether the 'increased increase' stems mainly from

- more participants (suggesting publicity around the M39 discovery)
- more computers (suggesting more dedicated users or easier access to computing resources)
- or merely quicker computers (P4 and code improvements)?

Or even how much each of these factors have contributed?

BTW, thanks for very nice and clarifying graphs!
:grin:

See also the [url=http://www.mersenneforum.org/showthread.php?s=&threadid=1125]GIMPS losing popularity?[/url] thread, where Complex33 has posted highly detailed graphs (hourly data) of the LL-testing rate in P90 years (but only since May 2003).

As of December 5th, 2003:

The virtual machine's sustained throughput* is currently 9511 billion floating point operations per second (gigaflops), or 790.1 CPU years (Pentium 90Mhz) computing time per day. For the testing of Mersenne numbers, this is equivalent to 339 Cray T916 supercomputers, or 169.5 of Cray's most powerful T932 supercomputers, at peak power. As such, PrimeNet ranks among the most powerful computers in the world. (*Measured in calibrated P5 90Mhz, 32.98 MFLOP units: 25658999 FPO / 0.778s using 256k FFT.)

9.511 Teraflops!

[QUOTE][i]Originally posted by jinydu [/i]
[B]
9.511 Teraflops! [/B][/QUOTE]

Yes :) I wonder, if we reach 10 TFlops average when the first LL-results from the new participants trickle in... I bet on it!

Thereafter it's time to go for 1000 P90 CPU-years/day :grin: