mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Factoring

Reply
 
Thread Tools
Old 2008-04-02, 14:25   #34
xilman
Bamboozled!
 
xilman's Avatar
 
"π’‰Ίπ’ŒŒπ’‡·π’†·π’€­"
May 2003
Down not across

2A2216 Posts
Default

Quote:
Originally Posted by bdodson View Post
Good thing, there's no ftp
(only sftp); automated ftp seems unlikely. -Bruce
Perhaps it's a local LeHigh situation, but automated sftp (or scp, which is essentially the same thing) is very easy to set up. I use it in a number of situations, mostly cronjobs which transfer gigabytes of files (PostgreSQL dumps mostly) to and from Harvard every week.

Let me know if you would like more details of how to set up scripted sftp transfers.


Paul
xilman is offline   Reply With Quote
Old 2008-04-02, 16:08   #35
bdodson
 
bdodson's Avatar
 
Jun 2005
lehigh.edu

210 Posts
Default

Quote:
Originally Posted by xilman View Post
Perhaps it's a local LeHigh situation, but automated sftp (or scp, which is essentially the same thing) is very easy to set up. I use it in a number of situations, mostly cronjobs which transfer gigabytes of files (PostgreSQL dumps mostly) to and from Harvard every week.

Let me know if you would like more details of how to set up scripted sftp transfers.

Paul
Sounds good; I'm not sure what Greg has in mind, data-wise (we
haven't gotten that far yet). On the 3+512 exchanges, there was
also a problem on Tom's side; the annon ftp wasn't accepting
sftp connections; or at least, I wasn't able to get one. Actually,
even if/when the new cluster gets ftp disabled, there's an old mail
server ...

I'm also interested in getting msieve assistance --- I ought to be
able to improve matters by removing duplicates, at least. If/when
I'm running complete sieving jobs (rather than partial ranges) I could
perhaps prepare a matrix to send off (as we used to do with Lehigh
factorizations finished on the MSR cluster for older nfs's, back in the
day); or perhaps the linux/quads here could be used? -Bruce
bdodson is offline   Reply With Quote
Old 2008-04-02, 16:18   #36
fivemack
(loop (#_fork))
 
fivemack's Avatar
 
Feb 2006
Cambridge, England

144308 Posts
Default

You've got enough resources there that I'd have thought you'd just run the msieve stage yourself. msieve is embarrassingly easy to run:

* put your number, unadorned, into a file worktodo.ini
* put 'N number' into a file msieve.dat, followed by the concatenation of all the relations
* write an msieve.fb file of the form
Code:
N number
A6 9
A0 1
R1 -1
R0 35917545547686059365808220080151141317043
* do 'msieve -v -nc1 2500000,2500000' to do the filtering, 'msieve -v -t4 -nc2' to do the linear algebra on four threads, 'msieve -v -nc3' to do the square roots, 'msieve -v -t4 -ncr' to restart the linear algebra if your computer crashed. Count on maybe 15MB of memory use per million relations in the linear algebra stage, and maybe 20MB in the other two - the 225Mrel job for 6+383 just barely failed to fit on a 4G machine for those two stages, so I just ran nc1 on an 8G machine, copied the directory to a reliable 4G machine that I could leave for a week, ran nc2, copied the directory back to the 8G machine and ran nc3.
fivemack is offline   Reply With Quote
Old 2008-04-02, 16:28   #37
jasonp
Tribal Bullet
 
jasonp's Avatar
 
Oct 2004

DD716 Posts
Default

Quote:
Originally Posted by fivemack View Post
* do 'msieve -v -nc1 2500000,2500000' to do the filtering
Note that the latest version has much improved automatic sizing of filtering bounds, so that '-nc1' without numerical arguments should do the same job with much less memory

Last fiddled with by jasonp on 2008-04-02 at 16:30
jasonp is offline   Reply With Quote
Old 2008-04-03, 02:35   #38
FactorEyes
 
FactorEyes's Avatar
 
Oct 2006
vomit_frame_pointer

23·32·5 Posts
Default

Quote:
Originally Posted by fivemack View Post
msieve is embarrassingly easy to run:
Ditto on the "embarrassing" part. It has made factoring easier, and made me dumber, because I've fallen out of the habit of thinking about how some of the parameters affect the process. (This is really my fault, and not Jason's, but msieve is a crutch -- a well-made one..)
FactorEyes is offline   Reply With Quote
Old 2008-04-04, 15:29   #39
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

7×503 Posts
Default

I'm not done with my current range, but I think I should get a new one on the books before fivemack takes off (assuming he won't be updating these pages while basking in the sun...)

Reserving 100-105

- ben.
bsquared is offline   Reply With Quote
Old 2008-04-04, 15:58   #40
fivemack
(loop (#_fork))
 
fivemack's Avatar
 
Feb 2006
Cambridge, England

23×11×73 Posts
Default

I'm not fleeing the country until Saturday 12 April - I was just hoping that I could get results from a one-week processing job before I left on a jetplane. So you may want a bigger reservation ...

I have posted to mersenneforum from a youth hostel in Serbia, but I don't have the ability to log into my compute farm from outside my house.
fivemack is offline   Reply With Quote
Old 2008-04-04, 16:07   #41
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

7·503 Posts
Default

Sorry, got dates mixed up.

I don't want to monopolize the ranges (in the hopes that many others will see there are fertile q left and reserve them...). If all goes well I should be nearly done with what I have before you leave, so I can grab a new reservation then.

Last fiddled with by bsquared on 2008-04-04 at 16:08
bsquared is offline   Reply With Quote
Old 2008-04-08, 14:15   #42
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

7·503 Posts
Default

Quote:
Originally Posted by bsquared View Post
I'm not done with my current range, but I think I should get a new one on the books before fivemack takes off (assuming he won't be updating these pages while basking in the sun...)

Reserving 100-105

- ben.
I'm on track to have this range done by saturday, so I'll go ahead and extend the reservation to 115. That will keep me busy for another couple weeks.
bsquared is offline   Reply With Quote
Old 2008-04-09, 17:39   #43
Andi47
 
Andi47's Avatar
 
Oct 2004
Austria

2·17·73 Posts
Default

40-41M finished and currently uploading.

Running two instances of 15e slows down the core 2 duo too much, so I will wait with reserving the next range until I have finished 41-42M.
Andi47 is offline   Reply With Quote
Old 2008-04-17, 07:04   #44
Andi47
 
Andi47's Avatar
 
Oct 2004
Austria

9B216 Posts
Default

41-42M is finished, I will upload the file today or tomorrow.

reserving 42-43M
Andi47 is offline   Reply With Quote
Reply

Thread Tools


All times are UTC. The time now is 15:39.


Fri Aug 6 15:39:01 UTC 2021 up 14 days, 10:08, 1 user, load averages: 2.73, 2.62, 2.73

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.