mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > GPU Computing

Reply
 
Thread Tools
Old 2017-08-08, 04:20   #518
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009

195610 Posts
Default

Quote:
Originally Posted by James Heinrich View Post
It looks like a CudaPm1 result, but it's lacking the program identifier.
The manual results form is, on purpose, very particular about formatting. Do not edit the result lines before attempting to submit them.
Guilty! I was playing with a small sorting program and didn't realize it was truncating them. I ran another and formatted this one like the second. Problem solved.
storm5510 is offline   Reply With Quote
Old 2017-08-14, 14:27   #519
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

5,419 Posts
Default cudapm1 bug and wish lst update

Here is today's version of the list I am maintaining. As always, this is in appreciation of the authors' past contributions. Users may want to browse this for workarounds included in some of the descriptions, and for an awareness of some known pitfalls. Please respond with any comments, additions or suggestions you may have.
Attached Files
File Type: pdf cudapm1 bug and wish list.pdf (57.6 KB, 483 views)
kriesel is offline   Reply With Quote
Old 2017-08-20, 15:54   #520
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

5,419 Posts
Default short of memory in stage 2, repeating residual

Is this a known problem? It warns before starting stage 1 there may not be enough memory for stage 2 for an exponent near 300M (wanting about 3% more than the GPU has), goes ahead and completes stage 1 using ~670MB, reports a residual for stage 1, uses about 5/6 of the gpu's 1.5GB memory for stage 2, and despite the earlier memory warning, chugs along in stage 2, one relative prime at a time, reporting the final stage 1 residual with each. Iteration times appear to be normal.

CUDAPm1 v0.20
------- DEVICE 1 -------
name GeForce GTX 480
Compatibility 2.0
clockRate (MHz) 1401
memClockRate (MHz) 1848
totalGlobalMem 1610612736
totalConstMem 65536
l2CacheSize 786432
sharedMemPerBlock 49152
regsPerBlock 32768
warpSize 32
memPitch 2147483647
maxThreadsPerBlock 1024
maxThreadsPerMP 1536
multiProcessorCount 15
maxThreadsDim[3] 1024,1024,64
maxGridSize[3] 65535,65535,65535
textureAlignment 512
deviceOverlap 1

CUDA reports 1434M of 1536M GPU memory free.
Index 107
Using threads: norm1 256, mult 128, norm2 128.
Using up to 1584M GPU memory.
WARNING: There may not be enough GPU memory for stage 2!
Selected B1=2660000, B2=17955000, 5.04% chance of finding a factor
Starting stage 1 P-1, M299500177, B1 = 2660000, B2 = 17955000, fft length = 18432K
Doing 3837955 iterations

...

Iteration 3750000 M299500177, 0x16fc277b4c69b54a, n = 18432K, CUDAPm1 v0.20 err = 0.03320 (30:13 real, 36.2668 ms/iter, ETA 53:09)
Iteration 3800000 M299500177, 0xe97a5cb286fcf801, n = 18432K, CUDAPm1 v0.20 err = 0.03418 (30:13 real, 36.2698 ms/iter, ETA 22:56)
M299500177, 0x071ac99b54319724, n = 18432K, CUDAPm1 v0.20
Stage 1 complete, estimated total time = 38:41:27
Starting stage 1 gcd.
M299500177 Stage 1 found no factor (P-1, B1=2660000, B2=17955000, e=0, n=18432K CUDAPm1 v0.20)
Starting stage 2.
Using b1 = 2660000, b2 = 17955000, d = 2310, e = 2, nrp = 1
Zeros: 778308, Ones: 811452, Pairs: 143662
Processing 1 - 1 of 480 relative primes.
Inititalizing pass... done. transforms: 170, err = 0.03711, (3.02 real, 17.7483 ms/tran, ETA NA)
Transforms: 16700 M299500177, 0x071ac99b54319724, n = 18432K, CUDAPm1 v0.20 err = 0.03711 (5:16 real, 18.8891 ms/tran, ETA 42:23:55)

...

Processing 341 - 341 of 480 relative primes.
Inititalizing pass... done. transforms: 265, err = 0.02988, (5.16 real, 19.4604 ms/tran, ETA 12:27:36)
Transforms: 16664 M299500177, 0x071ac99b54319724, n = 18432K, CUDAPm1 v0.20 err = 0.03125 (5:15 real, 18.8980 ms/tran, ETA 12:22:20)
kriesel is offline   Reply With Quote
Old 2017-08-20, 18:17   #521
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009

22×3×163 Posts
Default

Quote:
Originally Posted by kriesel View Post
...Transforms: 16664 M299500177, 0x071ac99b54319724, n = 18432K, CUDAPm1 v0.20 err = 0.03125 (5:15 real, 18.8980 ms/tran, ETA 12:22:20)
Is there a particular reason for running an exponent this large?
storm5510 is offline   Reply With Quote
Old 2017-08-20, 18:27   #522
James Heinrich
 
James Heinrich's Avatar
 
"James Heinrich"
May 2004
ex-Northern Ontario

11×311 Posts
Default

Quote:
Originally Posted by storm5510 View Post
Is there a particular reason for running an exponent this large?
Especially when it already has a known 52-bit factor.
James Heinrich is offline   Reply With Quote
Old 2017-08-21, 06:23   #523
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

152B16 Posts
Default why

Quote:
Originally Posted by James Heinrich View Post
Especially when it already has a known 52-bit factor.
Thanks for asking. Yes it's a bit off the beaten path. That's the point of this run.

I started it as a joint test of my hardware and the software & its local configuration. Does it find the factor? (That's actually the technique the author of the software described using, for qualifying an installation, but years ago at lower exponents. Sometimes things go wrong at different fft lengths. Test exponents are selected for having a factor that should be found. You're right that that's the opposite of searching to find new factors to screen out LL test candidates.)

And such testing also can shed some light on the following, even if it fails the find-the-known-factor test. What is actual run-time as a function of exponent, so what's reasonable or unreasonable to run on given hardware? Does anything break at high P for CUDAPm1? What are the gpu memory requirements or default usage versus exponent and stage? What is the save file size versus p and stage? If it's memory limited, does the software handle too little gpu memory gracefully? Are there unknown or forgotten bugs that could be smoked out and dealt with before the wave of PrimeNet assignments hit the fft lengths that reveal them? Armies use scouts.

I had already run current-P-1-wavefront assignments, some double-check territory exponents assigned as LLDC that had only B1 done on them, and some current or recent wavefront LL tests assigned that had only B1 done IIRC, so had about half the data for a handy chart or two already, so why not get a few points elsewhere on the log plot?

It's an extension of some of the stuff I've been posting over at http://www.mersenneforum.org/showthr...t=22450&page=3 as I puzzle things out as a long time GIMPS participant (1996?) but new to gpu use for it.
If that sort of information is already available and assembled somewhere else, and it may well be, I'd love to know where. I've read a lot of threads and thread lists, and haven't found it yet. It's a big haystack. Maybe the future gpu-newbies will find the Available Software thread and find it useful. I would have.

I want to know the capabilities and limitations of the software, generally and in relation to the parameters of the models of gpu I have running (6) or on order (1). Understanding that will help deploy them in the most productive manner.

And finally, it's because it interests me, more than only doing one exponent after another in ascending order at the wavefront, on each gpu or cpu. I currently have a mix of mfaktc, cudapm1, cudalucas, and prime95 running on systems, which mostly are doing production work, cranking out a mix of ECM, LL, DC, TF, & P-1, but I enjoy looking into how things will be different later and what issues may turn up.
kriesel is offline   Reply With Quote
Old 2017-09-13, 21:26   #524
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

5,419 Posts
Default updated benchmark, memory requirements, limits, etc on GTX480

Note that for comparison, a GTX1070 can do M9100xxxx in about 6.5 hours. The GTX480 is limited by both run-time and 1.5GB video memory size.
Attached Files
File Type: pdf cudapm1-runtime scaling.pdf (35.4 KB, 117 views)
kriesel is offline   Reply With Quote
Old 2017-09-14, 02:09   #525
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009

7A416 Posts
Default

Quote:
Originally Posted by kriesel View Post
...cranking out a mix of ECM, LL, DC, TF, & P-1, but I enjoy looking into how things will be different later and what issues may turn up.
Off Topic:I ran multiple machines for a while. I found my utility bills, rather shocking, pardon the pun. It was a 1/3 increase over each billing cycle. So, one machine is used sparingly. Only the newest one runs constantly.
storm5510 is offline   Reply With Quote
Old 2017-11-12, 14:07   #526
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

5,419 Posts
Default Appearance of exponent limit on Quadro 2000

Has anyone else seen something similar? A GTX480 had no equivalent problem on the same 84M exponents. This is CUDAPm1 v0.20 on Windows 64-bit Vista.

After a few successful stage 1 and stage 2 p-1 runs of ~83.5M, each following exponent >84M runs through stage 1, but not through stage 1 gcd or stage 2, crashing the program instead.
Behavior is reproducible for exponents 84M+, including after program restarts, logouts, system restarts.

M83496143 found no factor (P-1, B1=685000, B2=12843750, e=2, n=4608K, aid=85D38BAC023FCFF8022AABA05F602C4C CUDAPm1 v0.20)
reported 11/1/17
M83496227 found no factor (P-1, B1=685000, B2=12843750, e=2, n=4608K, aid=A1656CF4111B3B15C4A71186811384FF CUDAPm1 v0.20)
reported 11/2/17
M83496247 found no factor (P-1, B1=685000, B2=12843750, e=2, n=4608K, aid=5F246BFB077E96AA450384EFEC8EC599 CUDAPm1 v0.20)
reported 11/3/17
M83496293 found no factor (P-1, B1=685000, B2=12843750, e=2, n=4608K, aid=725F9720C9179022C18CEA98F646F72E CUDAPm1 v0.20)
reported 11/4/17
M50001781 has a factor: 4392938042637898431087689 (P-1, B1=430000, B2=5000000, e=2, n=2688K CUDAPm1 v0.20)

All 5 exponents attempted above 84M failed:
PFactor=A3B66EB4FAAE78E8F283D5C96AD37A__,1,2,84228073,-1,76,2
PFactor=DC8BDAFB8D89D04B3B35742B11D9CE__,1,2,84228097,-1,76,2
PFactor=C996CF4EA78E42F9610D9789BE1666__,1,2,84228103,-1,76,2
and two more

A typical event log entry follows. From entry to entry, process id and application start time changes but other event data values do not.

Log Name: Application
Source: Application Error
Date: 11/4/2017 7:23:36 PM
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: eagle
Description:
Faulting application CUDAPm1_win64_20131118_CUDA_50.exe, version 0.0.0.0, time stamp 0x5285815f, faulting module CUDAPm1_win64_20131118_CUDA_50.exe, version 0.0.0.0, time stamp 0x5285815f, exception code 0xc0000005, fault offset 0x000000000000dd20, process id 0xd78, application start time 0x01d355cc5142bacb.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2017-11-05T00:23:36.000Z" />
<EventRecordID>256</EventRecordID>
<Channel>Application</Channel>
<Computer>eagle</Computer>
<Security />
</System>
<EventData>
<Data>CUDAPm1_win64_20131118_CUDA_50.exe</Data>
<Data>0.0.0.0</Data>
<Data>5285815f</Data>
<Data>CUDAPm1_win64_20131118_CUDA_50.exe</Data>
<Data>0.0.0.0</Data>
<Data>5285815f</Data>
<Data>c0000005</Data>
<Data>000000000000dd20</Data>
<Data>d78</Data>
<Data>01d355cc5142bacb</Data>
</EventData>
</Event>

Normal progression, 83M:
(end of stage 1)
Iteration 987000 M83496293, 0xf2fb4b229c8521b0, n = 4608K, CUDAPm1 v0.20 err = 0.16919 (0:37 real, 36.8380 ms/iter, ETA 0:39)
Iteration 988000 M83496293, 0x9ad528e521e85730, n = 4608K, CUDAPm1 v0.20 err = 0.16797 (0:37 real, 36.8401 ms/iter, ETA 0:03)
M83496293, 0x232eab21eaf81e92, n = 4608K, CUDAPm1 v0.20
Stage 1 complete, estimated total time = 10:10:44
Starting stage 1 gcd.
M83496293 Stage 1 found no factor (P-1, B1=685000, B2=12843750, e=2, n=4608K CUDAPm1 v0.20)
Starting stage 2.
Using b1 = 685000, b2 = 12843750, d = 2310, e = 2, nrp = 13
Zeros: 573917, Ones: 658723, Pairs: 125889
Processing 1 - 13 of 480 relative primes.
Inititalizing pass... done. transforms: 270, err = 0.16406, (5.09 real, 18.8644 ms/tran, ETA NA)
Transforms: 2106 M83496293, 0x52b341a257507f69, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:41 real, 19.4671 ms/tran, ETA 9:14:05)
Transforms: 2010 M83496293, 0x905f255bd35e844b, n = 4608K, CUDAPm1 v0.20 err = 0.17188 (0:39 real, 19.5838 ms/tran, ETA 9:15:02)
Transforms: 2014 M83496293, 0x673b942ac1fc4ae2, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:40 real, 19.5771 ms/tran, ETA 9:14:52)
...

Processing 469 - 480 of 480 relative primes.
Inititalizing pass... done. transforms: 357, err = 0.17090, (6.88 real, 19.2605 ms/tran, ETA 14:07)
Transforms: 2090 M83496293, 0x284e7914442300ef, n = 4608K, CUDAPm1 v0.20 err = 0.17090 (0:41 real, 19.4700 ms/tran, ETA 13:26)
Transforms: 2058 M83496293, 0xb1c240cc360984b8, n = 4608K, CUDAPm1 v0.20 err = 0.16797 (0:40 real, 19.5747 ms/tran, ETA 12:46)
Transforms: 2012 M83496293, 0xfa21edbaa82e8d9d, n = 4608K, CUDAPm1 v0.20 err = 0.16992 (0:40 real, 19.5721 ms/tran, ETA 12:07)
Transforms: 1958 M83496293, 0xfdc0e766f0aa5f44, n = 4608K, CUDAPm1 v0.20 err = 0.16992 (0:38 real, 19.5923 ms/tran, ETA 11:28)
Transforms: 1980 M83496293, 0xf808c66bf88da80d, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:39 real, 19.5757 ms/tran, ETA 10:50)
Transforms: 1998 M83496293, 0xed71c1b76d6c0757, n = 4608K, CUDAPm1 v0.20 err = 0.16602 (0:39 real, 19.5754 ms/tran, ETA 10:10)
Transforms: 1910 M83496293, 0x9587bca9e6a92d95, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:37 real, 19.5884 ms/tran, ETA 9:33)
Transforms: 1902 M83496293, 0xdd50dacef6b94028, n = 4608K, CUDAPm1 v0.20 err = 0.17383 (0:38 real, 19.5907 ms/tran, ETA 8:56)
Transforms: 1930 M83496293, 0x5c01c876ba23af0e, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:38 real, 19.6468 ms/tran, ETA 8:18)
Transforms: 1924 M83496293, 0x4967e5714a906dd8, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:37 real, 19.6022 ms/tran, ETA 7:40)
Transforms: 1914 M83496293, 0xb5338d4f9734dcbf, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:38 real, 19.5649 ms/tran, ETA 7:03)
Transforms: 1882 M83496293, 0xb3364da78f68767c, n = 4608K, CUDAPm1 v0.20 err = 0.17969 (0:37 real, 19.5884 ms/tran, ETA 6:26)
Transforms: 1916 M83496293, 0x63c6b998ac49a7a0, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:37 real, 19.5861 ms/tran, ETA 5:49)
Transforms: 1844 M83496293, 0x9b385d7b61a51d47, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:36 real, 19.5965 ms/tran, ETA 5:13)
Transforms: 1882 M83496293, 0xe0d8af2fcfffed20, n = 4608K, CUDAPm1 v0.20 err = 0.17188 (0:37 real, 19.5938 ms/tran, ETA 4:36)
Transforms: 1896 M83496293, 0x85a24d9c67bd9496, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:37 real, 19.5903 ms/tran, ETA 3:59)
Transforms: 1986 M83496293, 0x71a887caf40e5bb7, n = 4608K, CUDAPm1 v0.20 err = 0.17627 (0:39 real, 19.5874 ms/tran, ETA 3:20)
Transforms: 1978 M83496293, 0x65c7d9d6c70197bf, n = 4608K, CUDAPm1 v0.20 err = 0.16797 (0:39 real, 19.5815 ms/tran, ETA 2:41)
Transforms: 1986 M83496293, 0x8f7ecc43a94105ef, n = 4608K, CUDAPm1 v0.20 err = 0.16406 (0:39 real, 19.5769 ms/tran, ETA 2:02)
Transforms: 1950 M83496293, 0xaac5ccee0aafbde0, n = 4608K, CUDAPm1 v0.20 err = 0.16797 (0:38 real, 19.5877 ms/tran, ETA 1:24)
Transforms: 2036 M83496293, 0x34e6f17ecab893b1, n = 4608K, CUDAPm1 v0.20 err = 0.17188 (0:40 real, 19.5862 ms/tran, ETA 0:44)
Transforms: 2024 M83496293, 0x4b29a8a5677c72db, n = 4608K, CUDAPm1 v0.20 err = 0.17578 (0:40 real, 19.5816 ms/tran, ETA 0:04)

Stage 2 complete, 1710522 transforms, estimated total time = 9:18:00
Starting stage 2 gcd.
M83496293 Stage 2 found no factor (P-1, B1=685000, B2=12843750, e=2, n=4608K CUDAPm1 v0.20)

(results.txt entry made, worktodo modified, next exponent started)



Abnormal 84M exponent:
(end of stage 1 crashes before gcd, program restarted attempts to begin at stage 2 fail, stage 1 gcd message missing)
Iteration 994000 M84228073, 0xf6fe7d71235ae765, n = 4608K, CUDAPm1 v0.20 err = 0.21875 (0:37 real, 36.8486 ms/iter, ETA 0:55)
Iteration 995000 M84228073, 0xed35e0151d83c908, n = 4608K, CUDAPm1 v0.20 err = 0.22656 (0:36 real, 36.8537 ms/iter, ETA 0:19)
M84228073, 0xc840c55fb78fc6a2, n = 4608K, CUDAPm1 v0.20
Stage 1 complete, estimated total time = 10:15:26batch wrapper reports cudapm1 exited at Sat 11/04/2017 12:12:38.23
batch wrapper reports CUDAPm1 (re)launch at Sat 11/04/2017 12:12:39.17

(from here repeats except batch wrapper date/time stamps change, until worktodo file is manually modified to remove the stuck exponent)
CUDAPm1 v0.20
Warning: Couldn't parse ini file option UnusedMem; using default.
------- DEVICE 0 -------
name Quadro 2000
Compatibility 2.1
clockRate (MHz) 1251
memClockRate (MHz) 1304
totalGlobalMem 1073741824
totalConstMem 65536
l2CacheSize 262144
sharedMemPerBlock 49152
regsPerBlock 32768
warpSize 32
memPitch 2147483647
maxThreadsPerBlock 1024
maxThreadsPerMP 1536
multiProcessorCount 4
maxThreadsDim[3] 1024,1024,64
maxGridSize[3] 65535,65535,65535
textureAlignment 512
deviceOverlap 1

No Quadro 2000 fft.txt file found. Using default fft lengths.
For optimal fft selection, please run
./CUDAPm1 -cufftbench 1 8192 r
for some small r, 0 < r < 6 e.g.
CUDA reports 952M of 1024M GPU memory free.
No Quadro 2000 threads.txt file found. Using default thread sizes.
For optimal thread selection, please run
./CUDAPm1 -cufftbench 4608 4608 r
for some small r, 0 < r < 6 e.g.
Using threads: norm1 512, mult 128, norm2 128.
No stage 2 checkpoint.
Using up to 828M GPU memory.
Selected B1=690000, B2=12937500, 3.07% chance of finding a factor
Using B1 = 690000 from savefile.
Continuing stage 2 from a partial result of M84228073 fft length = 4608K
batch wrapper reports cudapm1 exited at Sat 11/04/2017 12:13:34.24
batch wrapper reports CUDAPm1 (re)launch at Sat 11/04/2017 12:13:36.14
kriesel is offline   Reply With Quote
Old 2017-11-12, 17:51   #527
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

10110111111102 Posts
Default

The 480 has 1.5x as much memory. I suspect that may be the issue.
henryzz is offline   Reply With Quote
Old 2017-11-13, 03:44   #528
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

5,419 Posts
Default

Quote:
Originally Posted by henryzz View Post
The 480 has 1.5x as much memory. I suspect that may be the issue.
Thanks for your reply. I considered that. I think it's too low an exponent for that to be the case. If you see I missed something, please explain.

Maybe the gcd uses much more memory than the rest of stage 1, but I was able to run up to about 290M on a GTX480 to completion, through stage 1 with gcd, and stage 2 with gcd. Observed stage 1 memory usage is rather linear with exponent in CUDAPM1, with regression fit 54.5MB+ 2.03 bytes times exponent value p, so I'd expect p=~84.2M to require only about 225MB in stage 1.

Stage 2 memory usage is impacted by both exponent and nrp selection; it picks nrp to fit within available memory up to an exponent where nrp=1, leaving at least about 200 MB of headroom on the GTX480 (presumably for the code to occupy). From these observations, and extrapolating downward in memory requirement from the two GTX480 runs with nrp=1 for p~250M and 290M, to 824MB required, I'd expect to be able to run up to p=~145M in a 1GB card.

For p=83.5M, the Quadro 2000 supported nrp=13. From the nrp=13 point on the GTX480, at p=120M, it was able to run over double the exponent.

The program log from the 83.5M and 84.2M runs says for stage 2 on the Quadro 2000,
Using up to 828M GPU memory.
The GTX480 says 1332MB for the same exponents.
But I've found that is just an expression of the available memory, not the amount reported by GPU-Z as in use during a stage.

The Quadro 2000 passed a maximum-feasible-size 38-block memory test. (38x25=950MB).
Attached Files
File Type: pdf cudapm1-runtime scaling.pdf (35.1 KB, 59 views)

Last fiddled with by kriesel on 2017-11-13 at 04:34
kriesel is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
mfaktc: a CUDA program for Mersenne prefactoring TheJudger GPU Computing 3497 2021-06-05 12:27
World's second-dumbest CUDA program fivemack Programming 112 2015-02-12 22:51
World's dumbest CUDA program? xilman Programming 1 2009-11-16 10:26
Factoring program need help Citrix Lone Mersenne Hunters 8 2005-09-16 02:31
Factoring program ET_ Programming 3 2003-11-25 02:57

All times are UTC. The time now is 07:00.


Mon Aug 2 07:00:26 UTC 2021 up 10 days, 1:29, 0 users, load averages: 2.21, 1.56, 1.30

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.