Reserved for MF - Sequence 4788 - Page 197

wombatman · 2013-10-22, 19:59

Ed, which version of MSieve are you using (and which OS are you on)? I'm trying to do postprocessing of a C160 and running into issues as well.

RichD · 2013-10-22, 20:17

Quote:

Originally Posted by EdH

Edit: It's later! Does it seem right that this should take a month?

If you don't have InfiniBand, running on a single host would be much faster.

SVN 923 is the latest stable version which includes the faster LA code.

EdH · 2013-10-23, 01:08

Since it says over 700 hours, I'm going to interrupt it and do some more trials. I noticed that this is 886M, but I don't recall changing anything that should have flagged the M. I'll update to 923, as well. Then I'll see what it looks like a couple of different ways. I just ran a c144 in 2.5 hours last week. Should a c166 take that much longer with my setup?

My OS is Debian 7 on both the host and slave machines. I boosted the RAM from 4G to 8G on the host. The host normally runs LA faster using only two of the four cores. It is a Core2 Quad (Vpro) Q9400, 2.66GHz. Unfortunately, to boost the RAM, I had to take a different machine off line and that machine is kind of needed in the next few days, so I'm going to have to figure something out or acquire some more RAM.

Thanks for your help. Time to go play...

RichD · 2013-10-23, 02:02

With SVN 923 you may be able to utilize all four cores. On my Core-i5 it ran faster using 3 cores before upgrading (around SVN 902?) then 4 cores were better.

I thought you were playing around with MPI as an academic exercise. Even Gigabit Ethernet is not fast enough for distributed LA. Keep to one machine for best performance. I'm guessing it shouldn't be more than 8-10 days with your setup. If you can get a tight matrix, maybe half that time.

BTW, rebuild the matrix and add "-nc target_density=110" to the command line. If it doesn't build a matrix, drop it by 5 to 105 and try again. This will build a denser matrix and hence, less iterations and faster BL.

Edit: The default target_density is 70. Increasing it only works if the number is over-sieved, which this number is. I'm thinking (gut feel) a TD in the range of 100-110 would be adequate.

EdH · 2013-10-23, 03:11

Quote:

Originally Posted by RichD

With SVN 923 you may be able to utilize all four cores. On my Core-i5 it ran faster using 3 cores before upgrading (around SVN 902?) then 4 cores were better.

I thought you were playing around with MPI as an academic exercise. Even Gigabit Ethernet is not fast enough for distributed LA. Keep to one machine for best performance. I'm guessing it shouldn't be more than 8-10 days with your setup. If you can get a tight matrix, maybe half that time.

BTW, rebuild the matrix and add "-nc target_density=110" to the command line. If it doesn't build a matrix, drop it by 5 to 105 and try again. This will build a denser matrix and hence, less iterations and faster BL.

Edit: The default target_density is 70. Increasing it only works if the number is over-sieved, which this number is. I'm thinking (gut feel) a TD in the range of 100-110 would be adequate.

I have restarted it with your suggested target_density.

My setup gave me better timing for using two machines over one for previous tests, but adding a third machine really trashed it. All of my setup is a home grown LAN, put together with "junked" machinery, mostly. All kinds of P4 and up, with a couple "others" tossed in.

Thanks for all...

RichD · 2013-10-23, 03:43

Quote:

Originally Posted by EdH

My setup gave me better timing for using two machines over one for previous tests, ...

Let me know how the new tests perform. I may go out and get me a Gigabit switch for my two i5s.

EdH · 2013-10-23, 13:32

Quote:

Originally Posted by RichD

Let me know how the new tests perform. I may go out and get me a Gigabit switch for my two i5s.

At this point, I'm back to one machine and all four threads and have an improvement:

Code:

linear algebra at 0.0%, ETA 302h36m

I might break in and see how two threads would run, since it did make a difference before. Since I started the LA without mpi, I can't try that without restarting LA. Will changing to two threads require a restart of LA. I suppose I may find out...

RichD · 2013-10-23, 14:25

Quote:

Originally Posted by EdH

I might break in and see how two threads would run, since it did make a difference before. Since I started the LA without mpi, I can't try that without restarting LA. Will changing to two threads require a restart of LA. I suppose I may find out...

Changing to 2 (oe 3) threads is no big deal. Just be sure to use "-ncr" to indicate a resume.

Edit: Did you get a smaller matrix?

EdH · 2013-10-23, 15:58

Unfortunately, I got rid of the previous log because it had hundreds of thousands of error -1/-11/-6 lines. For the current log, I simply removed all the error lines, but I was having trouble reloading the other due to several sets of those error lines. Here's the current matrix build section, but I don't know if it can answer any of what you ask:

Code:

Wed Oct 23 02:03:17 2013  building initial matrix
Wed Oct 23 02:16:16 2013  memory use: 4571.7 MB
Wed Oct 23 02:17:15 2013  read 10590337 cycles
Wed Oct 23 02:17:17 2013  matrix is 10589507 x 10590337 (4440.9 MB) with weight 1368844072 (129.25/col)
Wed Oct 23 02:17:17 2013  sparse part has weight 1037074145 (97.93/col)
Wed Oct 23 02:23:37 2013  filtering completed in 3 passes
Wed Oct 23 02:23:40 2013  matrix is 10573751 x 10573951 (4437.5 MB) with weight 1367700655 (129.35/col)
Wed Oct 23 02:23:40 2013  sparse part has weight 1036388696 (98.01/col)
Wed Oct 23 02:25:54 2013  matrix starts at (0, 0)
Wed Oct 23 02:25:57 2013  matrix is 10573751 x 10573951 (4437.5 MB) with weight 1367700655 (129.35/col)
Wed Oct 23 02:25:57 2013  sparse part has weight 1036388696 (98.01/col)
Wed Oct 23 02:25:57 2013  saving the first 48 matrix rows for later
Wed Oct 23 02:26:00 2013  matrix includes 64 packed rows
Wed Oct 23 02:26:01 2013  matrix is 10573703 x 10573951 (4299.0 MB) with weight 1146577764 (108.43/col)
Wed Oct 23 02:26:01 2013  sparse part has weight 1021217371 (96.58/col)
Wed Oct 23 02:26:01 2013  using block size 8192 and superblock size 294912 for processor cache size 3072 kB
Wed Oct 23 02:27:29 2013  commencing Lanczos iteration (4 threads)
Wed Oct 23 02:27:29 2013  memory use: 3563.9 MB
Wed Oct 23 02:30:11 2013  linear algebra at 0.0%, ETA 302h36m

Changing to 2 threads increased the ETA to 374h57m. Changing back to four has brought it back down to 299h 8m. So a definite advantage! I am running four 2G memory modules now, while I previously only had two 2G modules. Maybe that's the difference.

According to top, the machine is still using almost all of the 8G and ~50M of swap space. Is this a potential slow down?

I don't think I have the figures anymore, but memory tells me that one of my "tests" (possibly a c132) showed ~3.5 hours for one machine, ~2.5 hours when I added a second one via mpi, but almost 6 hours when I added a third via mpi. The first two machines were core2 quad and the third was an i7. If I simply swapped the second and third I had a similar, but slightly greater ETA, compared with the earlier two system run. I made the assumption that my weak LAN was the trouble, which leads to a question:

If I was to add a Gigabit switch to a leg of my current LAN, would that help at all with the machines connected directly to it, or would the entire LAN need upgrading? I recently received a sales flyer for a 5-port Gigabit switch for only ~$20.00. There are probably even better deals around.

Another question for any tech gurus: Much of my current LAN is composed of 4-wire cabling. Does this cause speed loss? Maybe that's where my trouble lies...

RichD · 2013-10-23, 17:51

I don't have any experience with MPI except what I've read on this forum.

Does your NIC cards support Gigabit? An easy test is to wire two of them together with a cross-over cable and eliminate any switches or routers. That would be the optimum configuration. I've read someplace the newer cards can detect straight Cat-5 or cross-over and adjust accordingly. Again, no experience in that area.

Perhaps you are getting better performance by only moving two threads of data across the LAN and that is not a bottleneck (yet).

RichD · 2013-10-23, 17:59

I was going to get the rest of the .dat file to run with you but noticed it is already removed from the server. I also see Greg is running this number so he'll have the results in a few days. I guess he got impatient and needed to free up some space on the server.

2013-10-23, 02:02	#2160
RichD Sep 2008 Kansas 3,389 Posts	With SVN 923 you may be able to utilize all four cores. On my Core-i5 it ran faster using 3 cores before upgrading (around SVN 902?) then 4 cores were better. I thought you were playing around with MPI as an academic exercise. Even Gigabit Ethernet is not fast enough for distributed LA. Keep to one machine for best performance. I'm guessing it shouldn't be more than 8-10 days with your setup. If you can get a tight matrix, maybe half that time. BTW, rebuild the matrix and add "-nc target_density=110" to the command line. If it doesn't build a matrix, drop it by 5 to 105 and try again. This will build a denser matrix and hence, less iterations and faster BL. Edit: The default target_density is 70. Increasing it only works if the number is over-sieved, which this number is. I'm thinking (gut feel) a TD in the range of 100-110 would be adequate. Last fiddled with by RichD on 2013-10-23 at 02:10

2013-10-23, 15:58	#2165
EdH "Ed Hall" Dec 2009 Adirondack Mtns 3838₁₀ Posts	Unfortunately, I got rid of the previous log because it had hundreds of thousands of error -1/-11/-6 lines. For the current log, I simply removed all the error lines, but I was having trouble reloading the other due to several sets of those error lines. Here's the current matrix build section, but I don't know if it can answer any of what you ask: Code: Wed Oct 23 02:03:17 2013 building initial matrix Wed Oct 23 02:16:16 2013 memory use: 4571.7 MB Wed Oct 23 02:17:15 2013 read 10590337 cycles Wed Oct 23 02:17:17 2013 matrix is 10589507 x 10590337 (4440.9 MB) with weight 1368844072 (129.25/col) Wed Oct 23 02:17:17 2013 sparse part has weight 1037074145 (97.93/col) Wed Oct 23 02:23:37 2013 filtering completed in 3 passes Wed Oct 23 02:23:40 2013 matrix is 10573751 x 10573951 (4437.5 MB) with weight 1367700655 (129.35/col) Wed Oct 23 02:23:40 2013 sparse part has weight 1036388696 (98.01/col) Wed Oct 23 02:25:54 2013 matrix starts at (0, 0) Wed Oct 23 02:25:57 2013 matrix is 10573751 x 10573951 (4437.5 MB) with weight 1367700655 (129.35/col) Wed Oct 23 02:25:57 2013 sparse part has weight 1036388696 (98.01/col) Wed Oct 23 02:25:57 2013 saving the first 48 matrix rows for later Wed Oct 23 02:26:00 2013 matrix includes 64 packed rows Wed Oct 23 02:26:01 2013 matrix is 10573703 x 10573951 (4299.0 MB) with weight 1146577764 (108.43/col) Wed Oct 23 02:26:01 2013 sparse part has weight 1021217371 (96.58/col) Wed Oct 23 02:26:01 2013 using block size 8192 and superblock size 294912 for processor cache size 3072 kB Wed Oct 23 02:27:29 2013 commencing Lanczos iteration (4 threads) Wed Oct 23 02:27:29 2013 memory use: 3563.9 MB Wed Oct 23 02:30:11 2013 linear algebra at 0.0%, ETA 302h36m Changing to 2 threads increased the ETA to 374h57m. Changing back to four has brought it back down to 299h 8m. So a definite advantage! I am running four 2G memory modules now, while I previously only had two 2G modules. Maybe that's the difference. According to top, the machine is still using almost all of the 8G and ~50M of swap space. Is this a potential slow down? I don't think I have the figures anymore, but memory tells me that one of my "tests" (possibly a c132) showed ~3.5 hours for one machine, ~2.5 hours when I added a second one via mpi, but almost 6 hours when I added a third via mpi. The first two machines were core2 quad and the third was an i7. If I simply swapped the second and third I had a similar, but slightly greater ETA, compared with the earlier two system run. I made the assumption that my weak LAN was the trouble, which leads to a question: If I was to add a Gigabit switch to a leg of my current LAN, would that help at all with the machines connected directly to it, or would the entire LAN need upgrading? I recently received a sales flyer for a 5-port Gigabit switch for only ~$20.00. There are probably even better deals around. Another question for any tech gurus: Much of my current LAN is composed of 4-wire cabling. Does this cause speed loss? Maybe that's where my trouble lies...

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Reserved for MF - Sequence 3366	RichD	Aliquot Sequences	470	2021-04-22 02:17
Reserved for MF - Sequence 3408	RichD	Aliquot Sequences	474	2021-03-07 20:28
Reserved for MF - Sequence 276	kar_bon	Aliquot Sequences	127	2020-12-17 10:05
Assignments are reserved but not showing up	prism019	GPU to 72	6	2020-09-21 22:11
80M to 64 bits ... but not really reserved	petrw1	Lone Mersenne Hunters	82	2010-01-11 01:57

2013-10-22, 19:59	#2157
wombatman I moo ablest echo power! May 2013 6F5₁₆ Posts	Ed, which version of MSieve are you using (and which OS are you on)? I'm trying to do postprocessing of a C160 and running into issues as well.

2013-10-23, 01:08	#2159
EdH "Ed Hall" Dec 2009 Adirondack Mtns 2×19×101 Posts	Since it says over 700 hours, I'm going to interrupt it and do some more trials. I noticed that this is 886M, but I don't recall changing anything that should have flagged the M. I'll update to 923, as well. Then I'll see what it looks like a couple of different ways. I just ran a c144 in 2.5 hours last week. Should a c166 take that much longer with my setup? My OS is Debian 7 on both the host and slave machines. I boosted the RAM from 4G to 8G on the host. The host normally runs LA faster using only two of the four cores. It is a Core2 Quad (Vpro) Q9400, 2.66GHz. Unfortunately, to boost the RAM, I had to take a different machine off line and that machine is kind of needed in the next few days, so I'm going to have to figure something out or acquire some more RAM. Thanks for your help. Time to go play...

2013-10-23, 17:51	#2166
RichD Sep 2008 Kansas 3,389 Posts	I don't have any experience with MPI except what I've read on this forum. Does your NIC cards support Gigabit? An easy test is to wire two of them together with a cross-over cable and eliminate any switches or routers. That would be the optimum configuration. I've read someplace the newer cards can detect straight Cat-5 or cross-over and adjust accordingly. Again, no experience in that area. Perhaps you are getting better performance by only moving two threads of data across the LAN and that is not a bottleneck (yet).

2013-10-23, 17:59	#2167
RichD Sep 2008 Kansas 3,389 Posts	I was going to get the rest of the .dat file to run with you but noticed it is already removed from the server. I also see Greg is running this number so he'll have the results in a few days. I guess he got impatient and needed to free up some space on the server.