Raman's plans and questions - Page 16

10metreh · 2010-01-11, 13:25

Quote:

Originally Posted by Raman

But, unfortunately I haven't backed up the dependency file for 2,1778L before itself. So, should I have to re-run the linear algebra for 2,1778L starting up right from the scratch?

You need the relations and the cycles from the filtering to run the linear algebra, and you need both of those plus the dependencies for the square root, so if the dependencies have gone you will need to redo the linear algebra, if the cycles have gone you will need to redo the filtering, and if the relations have gone you will need to redo the sieving.

Raman · 2010-01-11, 14:56

What the hell is this case only?

Code:

 18:14:34   Msieve v. 1.43
 18:14:34   random seeds: ec00de47 2c3ba9c0
 18:14:34   factoring 1055412116234135999499545109262414853412059407974373221580633017431965879202266968949477759837803037992251938432194793281094868794155588894460590635766952454711354275120731783805628956609 (187 digits)
 18:14:37   no P-1/P+1/ECM available, skipping
 18:14:37   commencing number field sieve (187-digit input)
 18:14:37   R0: -100000000000000000000000000000000000000
 18:14:37   R1:  1
 18:14:37   A0:  100
 18:14:37   A1:  0
 18:14:37   A2:  0
 18:14:37   A3: -10
 18:14:37   A4:  0
 18:14:37   A5:  0
 18:14:37   A6:  1
 18:14:37   skew 1.00, size 3.933717e-11, alpha -0.126680, combined = 1.626930e-12
 18:14:37   
 18:14:37   commencing square root phase
 18:14:37   reading relations for dependency 1
 18:14:39   read 2505881 cycles
 18:14:53   cycles contain 6966576 unique relations
 18:17:55   read 6877268 relations
 18:18:31   algebraic side is not a square!
 18:18:31   reading relations for dependency 2
 18:18:34   read 2504879 cycles
 18:18:48   cycles contain 6964286 unique relations
 18:21:08   read 6874670 relations
 18:21:43   algebraic side is not a square!
 18:21:44   reading relations for dependency 3
 18:21:49   read 2504976 cycles
 18:22:02   cycles contain 6966768 unique relations
 18:24:22   read 6877214 relations
 18:24:58   algebraic side is not a square!
 18:24:59   reading relations for dependency 4
 18:25:04   read 2505365 cycles
 18:25:18   cycles contain 6966918 unique relations
 18:27:38   read 6877592 relations
 18:28:11   algebraic side is not a square!
 18:28:12   reading relations for dependency 5
 18:28:17   read 2507406 cycles
 18:28:31   cycles contain 6970614 unique relations
 18:30:50   read 6881332 relations
 18:31:24   algebraic side is not a square!
 18:31:25   reading relations for dependency 6
 18:31:29   read 2506041 cycles
 18:31:43   cycles contain 6970886 unique relations
 18:34:08   read 6881226 relations
 18:34:44   algebraic side is not a square!
 18:34:45   reading relations for dependency 7
 18:34:50   read 2505050 cycles
 18:35:03   cycles contain 6964834 unique relations
 18:37:22   read 6875600 relations
 18:38:00   algebraic side is not a square!
 18:38:00   reading relations for dependency 8
 18:38:05   read 2505494 cycles
 18:38:19   cycles contain 6964868 unique relations
 18:40:45   read 6875520 relations
 18:41:19   algebraic side is not a square!
 18:41:20   reading relations for dependency 9
 18:41:25   read 2503629 cycles
 18:41:39   cycles contain 6962938 unique relations
 18:43:58   read 6873348 relations
 18:44:31   algebraic side is not a square!
 18:44:32   reading relations for dependency 10
 18:44:37   read 2506567 cycles
 18:44:50   cycles contain 6967464 unique relations
 18:47:10   read 6877626 relations
 18:47:42   algebraic side is not a square!
 18:47:43   reading relations for dependency 11
 18:47:48   read 2505643 cycles
 18:48:02   cycles contain 6965812 unique relations
 18:50:21   read 6876588 relations
 18:50:56   algebraic side is not a square!
 18:50:57   reading relations for dependency 12
 18:51:02   read 2505364 cycles
 18:51:15   cycles contain 6967778 unique relations
 18:53:34   read 6878440 relations
 18:54:07   algebraic side is not a square!
 18:54:07   reading relations for dependency 13
 18:54:12   read 2505089 cycles
 18:54:26   cycles contain 6966058 unique relations
 18:56:52   read 6877044 relations
 18:57:29   algebraic side is not a square!
 18:57:29   reading relations for dependency 14
 18:57:34   read 2505441 cycles
 18:57:49   cycles contain 6967534 unique relations
 19:00:09   read 6878500 relations
 19:00:43   algebraic side is not a square!
 19:00:43   reading relations for dependency 15
 19:00:48   read 2504331 cycles
 19:01:02   cycles contain 6964386 unique relations
 19:03:28   read 6875222 relations
 19:04:03   algebraic side is not a square!
 19:04:04   reading relations for dependency 16
 19:04:09   read 2505926 cycles
 19:04:23   cycles contain 6967570 unique relations
 19:06:54   read 6878316 relations
 19:07:30   algebraic side is not a square!
 19:07:31   reading relations for dependency 17
 19:07:36   read 2505308 cycles
 19:07:49   cycles contain 6967200 unique relations
 19:10:12   read 6878072 relations
 19:10:45   algebraic side is not a square!
 19:10:45   reading relations for dependency 18
 19:10:50   read 2507033 cycles
 19:11:04   cycles contain 6972138 unique relations
 19:13:28   read 6882822 relations
 19:14:02   algebraic side is not a square!
 19:14:03   reading relations for dependency 19
 19:14:08   read 2504605 cycles
 19:14:21   cycles contain 6967812 unique relations
 19:16:49   read 6878262 relations
 19:17:27   algebraic side is not a square!
 19:17:28   reading relations for dependency 20
 19:17:33   read 2506409 cycles
 19:17:46   cycles contain 6969158 unique relations
 19:20:05   read 6879826 relations
 19:20:39   algebraic side is not a square!
 19:20:39   reading relations for dependency 21
 19:20:44   read 2503429 cycles
 19:20:58   cycles contain 6963208 unique relations
 19:23:25   read 6873490 relations
 19:24:00   algebraic side is not a square!
 19:24:00   reading relations for dependency 22
 19:24:05   read 2505961 cycles
 19:24:19   cycles contain 6966330 unique relations
 19:26:38   read 6877040 relations
 19:27:12   algebraic side is not a square!
 19:27:13   reading relations for dependency 23
 19:27:18   read 2506302 cycles
 19:27:31   cycles contain 6967486 unique relations
 19:29:54   read 6878076 relations
 19:30:30   algebraic side is not a square!
 19:30:30   reading relations for dependency 24
 19:30:36   read 2504967 cycles
 19:30:49   cycles contain 6962128 unique relations
 19:33:13   read 6872816 relations
 19:33:47   algebraic side is not a square!
 19:33:47   reading relations for dependency 25
 19:33:52   read 2504500 cycles
 19:34:05   cycles contain 6964042 unique relations
 19:36:34   read 6874400 relations
 19:37:07   algebraic side is not a square!
 19:37:07   reading relations for dependency 26
 19:37:12   read 2506055 cycles
 19:37:25   cycles contain 6967014 unique relations
 19:39:44   read 6877358 relations
 19:40:21   algebraic side is not a square!
 19:40:21   reading relations for dependency 27
 19:40:26   read 2503349 cycles
 19:40:39   cycles contain 6963252 unique relations
 19:43:35   read 6873844 relations
 19:44:09   algebraic side is not a square!
 19:44:09   reading relations for dependency 28
 19:44:14   read 2505773 cycles
 19:44:28   cycles contain 6970230 unique relations
 19:46:47   read 6880708 relations
 19:47:23   algebraic side is not a square!
 19:47:23   reading relations for dependency 29
 19:47:28   read 2506161 cycles
 19:47:42   cycles contain 6969898 unique relations
 19:50:01   read 6880418 relations
 19:50:33   algebraic side is not a square!
 19:50:34   reading relations for dependency 30
 19:50:39   read 2502921 cycles
 19:50:52   cycles contain 6962720 unique relations
 19:53:17   read 6872838 relations
 19:53:52   algebraic side is not a square!
 19:53:52   reading relations for dependency 31
 19:53:57   read 2504352 cycles
 19:54:10   cycles contain 6962044 unique relations
 19:56:29   read 6872630 relations
 19:57:03   algebraic side is not a square!
 19:57:03   reading relations for dependency 32
 19:57:08   read 2505919 cycles
 19:57:22   cycles contain 6968240 unique relations
 19:59:45   read 6878900 relations
 20:00:23   algebraic side is not a square!
 20:00:23   reading relations for dependency 33
 20:00:28   read 2503136 cycles
 20:00:41   cycles contain 6959476 unique relations
 20:03:02   read 6869882 relations
 20:03:42   algebraic side is not a square!
 20:03:42   reading relations for dependency 34
 20:03:47   read 2507580 cycles
 20:04:02   cycles contain 6971068 unique relations
 20:06:34   read 6881700 relations
 20:07:08   algebraic side is not a square!
 20:07:08   reading relations for dependency 35
 20:07:14   read 2505033 cycles
 20:07:27   cycles contain 6966046 unique relations
 20:09:50   read 6876658 relations
 20:10:25   algebraic side is not a square!
 20:10:26   reading relations for dependency 36
 20:10:31   read 2507291 cycles
 20:10:44   cycles contain 6970648 unique relations
 20:13:04   read 6881246 relations
 20:13:38   algebraic side is not a square!
 20:13:39   reading relations for dependency 37
 20:13:44   read 2504380 cycles
 20:13:58   cycles contain 6967106 unique relations
 20:16:17   read 6877480 relations
 20:16:50   algebraic side is not a square!
 20:16:50   reading relations for dependency 38
 20:16:56   read 2503745 cycles
 20:17:09   cycles contain 6964564 unique relations
 20:19:34   read 6875422 relations
 20:20:08   algebraic side is not a square!
 20:20:08   reading relations for dependency 39
 20:20:13   read 2504325 cycles
 20:20:26   cycles contain 6964616 unique relations
 20:22:48   read 6875156 relations
 20:23:22   algebraic side is not a square!
 20:23:22   reading relations for dependency 40
 20:23:27   read 0 cycles

I wasn't aware at all until it got terminated up because it was running up within an internal node within the cluster only!

PS: It seems that the dependency file gets corrupted up immediately after the square root phase has started up, right?

Should take up a back up copy safely of the dependency file immediately after the Linear Algebra stage only, and then that square root step has to be started up separately itself....

jasonp · 2010-01-12, 03:22

Quote:

Originally Posted by Raman

Why do you do all the latest modifications to msieve and then spoil up the previous code? I wish that I had written my own code to be devoid of these errors, being dependent upon others, but understanding the algorithm is too difficult, especially the notations given within the papers, so much optimizations needed... I don't have that patience for writing 1 man year of code at all...

Are you sure the relation file is exactly the same between your first machine and the current one? Can you run md5sum on the cycle files between the old machine and the new one? If the relation file is not the same as before, then neither the cycle file nor the dependencies can be expected to work with it. If the old and new machines have different endianness (i.e. new machine is a PowerPC) then old binary files will definitely not work.

And it's depressing that you are simultaneously accusing me of incompetence and asking for my help.

Raman · 2010-01-12, 03:35

Quote:

Originally Posted by jasonp

Are you sure the relation file is exactly the same between your first machine and the current one? Can you run md5sum on the cycle files between the old machine and the new one? If the relation file is not the same as before, then neither the cycle file nor the dependencies can be expected to work with it. If the old and new machines have different endianness (i.e. new machine is a PowerPC) then old binary files will definitely not work.

And it's depressing that you are simultaneously accusing me of incompetence and asking for my help.

What is md5sum? Is it a command in Linux? If so, I will check it out.

Few points have to be noted that:
If I have to re-run the linear algebra, then the new square root code should properly work out without segmentation fault once square root is continued after linear algebra.

Dependencies have to be backed up at the immediate end of square root stage. It seems that it gets altered afterwards.

Yes, it is indeed the same file only copied by using the scp command itself.

I will not be available to monitor the systems on Friday & Saturday. I am going to out of station to view the Annular Solar Eclipse that is approaching upon 15 January.

Raman · 2010-01-12, 11:15

There is difference in md5sum between msieve compiled in department system and msieve compiled at computing cluster system. Both are 64 bit Linux binary only. What does it mean? Will the old binary not work under the other system at all?

What do you get the md5sum with your msieve-1.43, 64 bit Linux binary?
357bc5e227119d001cf76d01c1987561 - compute cluster
acf42a2eb06c8e9f8cbf2890921f2d1c - department computer
cbd2952dda80dd67dff846b3b01bce19 - 32 bit Linux binary compiled on department

I assume that the relation files remain unchanged, and work with both binaries of msieve, right? It is not possible to run md5sum on relation or cycle files. They are too huge enough. But the same files copied should have same md5sum only. Will the relation file work with both copies of msieve?

I don't mind re-running the post processing for 2,1778L 7,320+ 10,339+ 10,351+
but the execution should be successful this time, with the factors.

What binary should I use to avoid crash during the square root stage? Will the msieve-1.43 binary that is compiled under compute cluster be successful?
What should be done to avoid corrupt of files after the job is being executed?

These are all long time executing jobs. I can't check out whether or not the program crashes quickly. 2,1778L takes 4 days to get completed up entirely.
The main problem with the compute cluster is that, the intermediate outputs are not known, only after the execution of jobs is over, the final output is shown.

smh · 2010-01-12, 11:20

Quote:

Originally Posted by Raman

I will be very angry, if you ignore this post, for this time only.

QUICK RESPONSE PLEASE.

The problem with forum is that I don't get quick responses at all.

He's done this before and i think he is crossing the line again. To me, he lost his little bit of credit that he build up over the last year. Time to ignore him (again)

Raman · 2010-01-12, 11:46

Quote:

Originally Posted by smh

He's done this before and i think he is crossing the line again. To me, he lost his little bit of credit that he build up over the last year. Time to ignore him (again)

Won't you get frustrated or annoyed if you go upto the final square root phase, and then it fails, when using binary to run jobs again destroys all the old files?

Especially for a very large job, if you don't know what to do at all?

Anyway, it is true that most people do not know about the matter at all, only a few people can be able to help me up.

R.D. Silverman · 2010-01-12, 12:27

Quote:

Originally Posted by Raman

rious:

Why do you do all the latest modifications to msieve and then spoil up the previous code? I wish that I had written my own code to be devoid of these errors, being dependent upon others, but understanding the algorithm is too difficult, especially the notations given within the papers, so much optimizations needed... I don't have that patience for writing 1 man year of code at all...

Raman. You really are a total asshole.

You are arrogant, despite not having produced anything of value yourself.
You have deprecated others for not publishing a result immediately.
You criticize others for bugs in code that is:

(1) Given to you freely. You did not pay for it or help in its development
in any way.
(2) A labor of love.
(3) Clearly intended as research code. It is not developed for commercial
purposes.

You want EVERYTHING handed to you.

GROW UP.

Congratulations. You just made my ignore list. Your posts are not
worth reading.

Raman · 2010-01-12, 13:02

I asked for speed of publication, just for curiousity. It was not intended to be serious. Though suggestion of project was serious. The numbers I have more knowledge about. Just suggestion, whether to take it or not is your wish. What is meaning of inappropriate? It means just off topic, partially related to topic itself. What is meaning of announce? Just telling everyone? Just people have to receive it. Putting in paper means announcement is done? No flame intended.

Pay? I hate that word. My family is facing serious financial problems, and is it feasible for buying some costly products from abroad? Products from USA are much more expensive than from the standards at India. Factoring numbers has been a good entertainment activity for me for the past two years. Instead of paying for it, and owning the product, I contribute some numbers for some project. Isn't that compensated? Who will get motivated to join up any distributed computing project, if everything is made for money? I will have more resources till June 2011, so I can expect more numbers to be done soon. Just tackling the crash in the square root phase and whether the processing of files is altered between two environments should be known.

If I had to write my own implementation, then it should be only useful for factoring small numbers. And that is not the current status for factoring at all. It took Mr. Jason Papadopoulos more than 2.5 years to patiently understand and then implement the code.

The code is really rigorous, and is not feasible to understand. So, I can't contribute to its development right now. And for understanding the algorithms, Number Field Sieve, most of the notations given in the book are not understandable. I need to start from graduate level books in mathematics and go step by step higher. For elliptic curve method, my implementation was damn slow. Many optimizations have been worked for years together. It will take years to understand. Please be patient. The FFT algorithm in the book is written in very odd idiom.

Just answer one question: That will do so. I will get the factors of 2,1778L 7,320+ 10,339+ 10,351+ within 4 days. Please be patient. I can do it. I have resources. I have 8 GB of RAM for Linear Algebra. I have got minimum 25 machines for sieving.

Question:
What binary file of msieve will avoid crash in square root phase? Will the original relations file from GGNFS merged together on old computer work properly with msieve on compute cluster? The post processing will take only 4 days, and I don't mind doing it again. Can I use the version of msieve-1.43 that is compiled under compute cluster? Will it work out properly? Sure? I will not know the output of compute cluster processing until the task is over, execution of batch job by using the 'qsub' command only.

You have no restrictions for delaying the factors by a few days, right? What can I do if it says "Algebraic side is not a perfect square!"?

henryzz · 2010-01-12, 13:15

To be on the safe side in your situation i would compile your binary on the system that will be using it. It should then work perfectly. People haven't been finding too many bugs lately in msieve.

Raman · 2010-01-12, 13:25

Quote:

Originally Posted by henryzz

To be on the safe side in your situation i would compile your binary on the system that will be using it. It should then work perfectly. People haven't been finding too many bugs lately in msieve.

Are you really sure enough? I have been actually stunned up by this post only.
I was keenly expecting up for the factors only yesterday morning itself.

--

The rural children of India, do not have as much as primary education as urban children of India
The urban people of India, do not have as much as advanced education as those people of USA

(The rural children of India face so much difficulties even with simple primary school level Mathematics)
(I find so much difficulty in understanding or implementing the advanced algorithms with so much of optimizations)

2010-01-12, 11:15	#170
Raman Noodles "Mr. Tuch" Dec 2007 Chennai, India 3·419 Posts	There is difference in md5sum between msieve compiled in department system and msieve compiled at computing cluster system. Both are 64 bit Linux binary only. What does it mean? Will the old binary not work under the other system at all? What do you get the md5sum with your msieve-1.43, 64 bit Linux binary? 357bc5e227119d001cf76d01c1987561 - compute cluster acf42a2eb06c8e9f8cbf2890921f2d1c - department computer cbd2952dda80dd67dff846b3b01bce19 - 32 bit Linux binary compiled on department I assume that the relation files remain unchanged, and work with both binaries of msieve, right? It is not possible to run md5sum on relation or cycle files. They are too huge enough. But the same files copied should have same md5sum only. Will the relation file work with both copies of msieve? I don't mind re-running the post processing for 2,1778L 7,320+ 10,339+ 10,351+ but the execution should be successful this time, with the factors. What binary should I use to avoid crash during the square root stage? Will the msieve-1.43 binary that is compiled under compute cluster be successful? What should be done to avoid corrupt of files after the job is being executed? These are all long time executing jobs. I can't check out whether or not the program crashes quickly. 2,1778L takes 4 days to get completed up entirely. The main problem with the compute cluster is that, the intermediate outputs are not known, only after the execution of jobs is over, the final output is shown. Last fiddled with by Raman on 2010-01-12 at 11:37

2010-01-12, 13:02	#174
Raman Noodles "Mr. Tuch" Dec 2007 Chennai, India 3·419 Posts	I asked for speed of publication, just for curiousity. It was not intended to be serious. Though suggestion of project was serious. The numbers I have more knowledge about. Just suggestion, whether to take it or not is your wish. What is meaning of inappropriate? It means just off topic, partially related to topic itself. What is meaning of announce? Just telling everyone? Just people have to receive it. Putting in paper means announcement is done? No flame intended. Pay? I hate that word. My family is facing serious financial problems, and is it feasible for buying some costly products from abroad? Products from USA are much more expensive than from the standards at India. Factoring numbers has been a good entertainment activity for me for the past two years. Instead of paying for it, and owning the product, I contribute some numbers for some project. Isn't that compensated? Who will get motivated to join up any distributed computing project, if everything is made for money? I will have more resources till June 2011, so I can expect more numbers to be done soon. Just tackling the crash in the square root phase and whether the processing of files is altered between two environments should be known. If I had to write my own implementation, then it should be only useful for factoring small numbers. And that is not the current status for factoring at all. It took Mr. Jason Papadopoulos more than 2.5 years to patiently understand and then implement the code. The code is really rigorous, and is not feasible to understand. So, I can't contribute to its development right now. And for understanding the algorithms, Number Field Sieve, most of the notations given in the book are not understandable. I need to start from graduate level books in mathematics and go step by step higher. For elliptic curve method, my implementation was damn slow. Many optimizations have been worked for years together. It will take years to understand. Please be patient. The FFT algorithm in the book is written in very odd idiom. Just answer one question: That will do so. I will get the factors of 2,1778L 7,320+ 10,339+ 10,351+ within 4 days. Please be patient. I can do it. I have resources. I have 8 GB of RAM for Linear Algebra. I have got minimum 25 machines for sieving. Question: What binary file of msieve will avoid crash in square root phase? Will the original relations file from GGNFS merged together on old computer work properly with msieve on compute cluster? The post processing will take only 4 days, and I don't mind doing it again. Can I use the version of msieve-1.43 that is compiled under compute cluster? Will it work out properly? Sure? I will not know the output of compute cluster processing until the task is over, execution of batch job by using the 'qsub' command only. You have no restrictions for delaying the factors by a few days, right? What can I do if it says "Algebraic side is not a perfect square!"? Last fiddled with by Raman on 2010-01-12 at 13:20

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
What are your CRUS plans?	rogue	Conjectures 'R Us	35	2013-11-09 09:03
Raman's stuff	Raman	Chess	8	2013-04-16 20:52
Further Plans	Kosmaj	Riesel Prime Search	6	2009-05-20 01:27
Further Plans	Kosmaj	Riesel Prime Search	6	2006-09-29 22:32
64 bit plans	pyrodave	Software	17	2004-06-05 12:27

2010-01-12, 13:15	#175
henryzz Just call me Henry "David" Sep 2007 Cambridge (GMT/BST) 13371₈ Posts	To be on the safe side in your situation i would compile your binary on the system that will be using it. It should then work perfectly. People haven't been finding too many bugs lately in msieve.