Raman's plans and questions

[jasonp]

In the US, if you work for an employer and use the employer's computers for other than work, overly paranoid employers can have you arrested. US universities are more lenient, but many are extremely nervous about being seen to 'tolerate hackers on their network', especially with the rash of lawsuits against filesharing students. Hopefully other countries are less uptight about that, is what I meant.

Perhaps if the staff lets you use the machines, they can set up a cron job to run the siever using 'nohup'. Then the siever binary can keep running even if you are logged out.

[Raman]

Actually, in fact, I am not bothered about disturbing the other people's work due to the execution of my program, because of the fact that, there are enough systems, and then the lab is less than half full most of the time.

What I am bothered about is that, is the running of my program at the night time. Because of continuously running the program, the system processors will get overheated and then damaged more sooner? I don't know whether this will really happen up so, but currently the systems are being turned off during the night time, thus, especially during the times of when the lightning is being possible.

[Raman]

Please give me up an estimate of about how many special-q's will be needed so for sieving thus
7,393+
7,396+
10,339+
6,335+
6,340+

[Random Poster]

Originally Posted by jasonp

Perhaps if the staff lets you use the machines, they can set up a cron job to run the siever using 'nohup'.

Or maybe they know better than that. nohup serves no useful purpose with a cron job.

[jasonp]

It should be clear to the sysadmins out there that I don't do this very much :)

[Raman]

Wrapped in CODE tags for readability. Also, failed to see hell in this message, honestly. This stage looks good. --SB

[Raman]

Wed Sep 09 18:22:12 2009 commencing linear algebra
Wed Sep 09 18:23:26 2009 read 5329870 cycles
Wed Sep 09 18:23:54 2009 cycles contain 14839450 unique relations
Wed Sep 09 19:54:28 2009 read 14839450 relations
Wed Sep 09 19:55:25 2009 using 32 quadratic characters above 536870168
Wed Sep 09 19:57:49 2009 building initial matrix
Wed Sep 09 20:11:02 2009 memory use: 1692.9 MB
Wed Sep 09 20:12:20 2009 read 5329870 cycles
Wed Sep 09 20:26:32 2009 matrix is 5329550 x 5329870 (1507.7 MB) with weight 492230814 (92.35/col)
Wed Sep 09 20:26:32 2009 sparse part has weight 357923420 (67.15/col)
Wed Sep 09 20:28:21 2009 filtering completed in 3 passes
Wed Sep 09 20:28:22 2009 matrix is 5316958 x 5317158 (1506.2 MB) with weight 491616260 (92.46/col)
Wed Sep 09 20:28:22 2009 sparse part has weight 357622666 (67.26/col)
Wed Sep 09 20:44:29 2009 read 5317158 cycles
Wed Sep 09 21:04:48 2009 matrix is 5316958 x 5317158 (1506.2 MB) with weight 491616260 (92.46/col)
Wed Sep 09 21:04:48 2009 sparse part has weight 357622666 (67.26/col)
Wed Sep 09 21:04:49 2009 saving the first 48 matrix rows for later
Wed Sep 09 21:04:52 2009 matrix is 5316910 x 5317158 (1440.8 MB) with weight 379374555 (71.35/col)
Wed Sep 09 21:04:52 2009 sparse part has weight 345795777 (65.03/col)
Wed Sep 09 21:04:52 2009 matrix includes 64 packed rows
Wed Sep 09 21:04:52 2009 using block size 65536 for processor cache size 4096 kB
failed to allocate 2041376 bytes

@jasonp: Is it not possible to design the linear algebra code such that all the post processing processes occur within the available memory itself? Every time it is happening like that only, or if newly redesigned, will the events such as the thrashing processes will occur up more frequently?

[10metreh]

Originally Posted by Raman

Msieve v. 1.38

Have you not upgraded to 1.42 yet, or do you not trust the new filtering routine?

[jasonp]

Originally Posted by Raman

@jasonp: Is it not possible to design the linear algebra code such that all the post processing processes occur within the available memory itself? Every time it is happening like that only, or if newly redesigned, will the events such as the thrashing processes will occur up more frequently?

If the linear algebra is going to run for a week, you do not want it swapping to disk at all, which is what would happen if the computations were made to fit in memory. Better to fail outright than to solve the matrix 100x slower.

Some memory could be saved if the matrix was stored on disk and then converted to packed form a block at a time, which is where a 32-bit OS is most likely to run out of virtual memory. I suspect that your version of windows does not do anything different with the /3GB switch turned on, otherwise you would be able to build the matrix; it is safe mode that turns off enough background stuff so that the LA can run. If you actually had 3GB of virtual memory the matrix would build, with a great deal of swapping, then the swapping would stop when the Lanczos iteration started.

[Batalov]

Raman, if this computer is under you control, then you may want to install something like the lightest Ubuntu you can find, get the linux msieve binary (you won't need anything else), and then proceed in 64-bit mode under Ubuntu -- using all memory (I presume this computer has more than 2Gb, the problem being that you just cannot quite access it from Windows).

[Raman]

I am trying up again under Windows (safe mode, /3GB switch) after updating up msieve to version 1.42, thus if that fails so again, I will try so within one of my college computers, which has Linux being installed within them up. Certainly, I think so that it is not feasible to install Linux within my desktop system, that has Windows because of the fact that to install so, I will have to format up everything within that system, and then create up new partitions for Linux and then Windows, to properly install them up, within that machine, only.