mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Conjectures 'R Us (https://www.mersenneforum.org/forumdisplay.php?f=81)
-   -   the scripts thread (https://www.mersenneforum.org/showthread.php?t=12845)

Mini-Geek 2009-12-10 22:47

the scripts thread
 
If anybody has any useful scripts related to CRUS work, feel free to post them here. :smile:

Here's one I made: a super simple perl script to remove k's from a sieve file (e.g. if you just found some primes and need to eliminate them so you can sieve further). The input file can be just the k's, (e.g. "506304") the whole primes, (e.g. "506304*7^10150+1") or the sequences (e.g. "506304*7^n+1). Really anything, as long as it starts with the k and then, if it continues, the k is followed by a '*'. It's not terribly flexible, as the sieve file must always be in sr_b.abcd (e.g. "sr_7.abcd") and is written there when finished. Of course, you can change this with a simple modification to the script.
Maybe someday I'll make it read the base and type from the primes, if present. For now, I think it's great for its purpose. :smile:
[code]#!/usr/bin/perl
# usage:
# have a file with one k or prime per line
# have sr_b.abcd present, the srfile-readable sieve file to remove from
# have srfile present
# run 'perl this-files-name.pl b t prime-file' with b=the base and t=+ or - for Sierp or Riesel, and prime-file a list of k's, primes, or sequences
# examples: (assuming this file is named remove-ks.pl)
# e.g. 'perl remove-ks.pl 3 - pfgw.log' uses srfile to delete all k's with primes (found in pfgw.log) from sr_3.abcd

$b = $ARGV[0];
$t = $ARGV[1];
open(IN, $ARGV[2]);

while(<IN>)
{
$line = $_;
chomp($line);
@linearray = split(/\*/,$line);
system("./srfile -d \"@linearray[0]*$b\^n".$t."1\" -a sr_$b.abcd");
}

close(IN);
[/code]This was made on/for Windows and may or may not work on Linux. I added a ./ in front of srfile since that's the only reason I can think of that it wouldn't work on Linux, and it still works like that on Windows.
Just to give full credit where it's due: this was inspired by one of Max's scripts, which didn't work for me on Windows, and uses a couple bits of code from some of his perl scripts.

And, everybody probably already knows of this one, but there's a PFGW script to make starting bases very easy, created by various people. [URL]http://www.mersenneforum.org/showpost.php?p=198146&postcount=57[/URL]

rogue 2009-12-18 20:30

Search script
 
This is Gary's PFGW script for searching, with changes to output the number being tested as k*b^n+/-c instead of the decimal expansion. This is also output to pfgw.log and pfgw_prime.log in the same way. Note that PFGW will not verify that the string being output is the same as the number being tested.

Also note that script files cannot take advantage of fast modular reduction. This is another good reason to not test too a high value of n with this script.

[code]
SCRIPT
#----------------------------------------------------------------------------------------
# This Script is used to start new bases. This is a version of a highly plagarized
# script that creates 3 new files: 1 containing k's eliminated by trivial factors, 1
# containing k's eliminated due to MOB (multiple of the base), and 1 containing k's
# eliminated by Generallized Fermat #'s (GFN's) for even Sierp bases. The script now
# automatically calculates all k's that are relavent to these files.
#
# The original scripts were written by Micha (Michaf), Karsten (Kar_bon), and
# Ian (MyDogBuster). More recent modifications were made by yours truly (gd_barnes).
#
# The maximum base that the script can handle is 2*3*5*7*11*13*17=510510. Higher bases
# would need tweaking done to allow for more modulos for k's with trivial factors.
#----------------------------------------------------------------------------------------

#----------------------------------------------------------------------------------------
# The following 5 lines control what is tested and are the only lines that should be
# changed. Type is 1 for Sierp and -1 for Riesel. The rest should be self explanatory.
#----------------------------------------------------------------------------------------
DIM base, 1022
DIM type, 1
DIM min_k, 1
DIM max_k, 7
DIM max_n, 10000

OPENFILEAPP g_file,pl_GFN.txt
OPENFILEAPP k_file,pl_remain.txt
OPENFILEAPP m_file,pl_MOB.txt
OPENFILEAPP p_file,pl_prime.txt
OPENFILEAPP t_file,pl_trivial.txt

DIMS tmpstr
DIMS type_str
DIMS test_str

DIM k, min_k - 1
DIM gfn_k, 1
DIM n
DIM cofact, base - 1
DIM fact, cofact
DIM prev_fact, 1
DIM fact_cnt, 0
DIM facta, -1
DIM factb, -1
DIM factc, -1
DIM factd, -1
DIM facte, -1
DIM factf, -1

IF (type == 1) THEN SETS type_str,+ ELSE SETS type_str,-

#-----------------------------------------------------------------------------------------
# The next lines calculate unique prime factors for base - 1. These will be used
# as modulos to determine k's that contain trivial factors and can be eliminated.
#-----------------------------------------------------------------------------------------
LABEL factor_rtn
FACTORIZE cofact
IF (FACTORFOUND > 1) THEN SET fact, FACTORFOUND ELSE SET fact, cofact
IF (fact == prev_fact) THEN GOTO factor_rtnb

SET fact_cnt, fact_cnt + 1
IF (fact_cnt == 1) THEN SET facta, fact
IF (fact_cnt == 2) THEN SET factb, fact
IF (fact_cnt == 3) THEN SET factc, fact
IF (fact_cnt == 4) THEN SET factd, fact
IF (fact_cnt == 5) THEN SET facte, fact
IF (fact_cnt == 6) THEN SET factf, fact

LABEL factor_rtnb
IF (FACTORFOUND == 1) THEN GOTO next_k

SET cofact, cofact / fact
SET prev_fact, fact
GOTO factor_rtn

LABEL next_k
SET k, k + 1
IF (k > max_k) THEN GOTO END

#-----------------------------------------------------------------------------------------
# The next 5 lines exclude Generallized Fermat k's for even Sierp bases.
#-----------------------------------------------------------------------------------------
LABEL calc_gfn
IF (type == -1) THEN GOTO calc_trivial
IF (base % 2 == 1) THEN GOTO calc_trivial
IF (k < gfn_k) THEN GOTO calc_trivial
IF (k == gfn_k) THEN GOTO GFN ELSE SET gfn_k, gfn_k * base
GOTO calc_gfn

#-----------------------------------------------------------------------------------------
# The next lines check to see if the k contains trivial factors. If so, the k is
# bypassed. k's with trivial factors are not written out for odd k's on odd bases.
#-----------------------------------------------------------------------------------------
LABEL calc_trivial
IF (facta > 2) THEN GOTO calc_trivial_r
IF (k % 2 == 1) THEN GOTO next_k

LABEL calc_trivial_r
IF (type == 1) THEN GOTO calc_trivial_s
IF (k % facta == 1) THEN GOTO trivial_factor
IF (k % factb == 1) THEN GOTO trivial_factor
IF (k % factc == 1) THEN GOTO trivial_factor
IF (k % factd == 1) THEN GOTO trivial_factor
IF (k % facte == 1) THEN GOTO trivial_factor
IF (k % factf == 1) THEN GOTO trivial_factor
GOTO calc_mob

LABEL calc_trivial_s
IF (k % facta == facta - 1) THEN GOTO trivial_factor
IF (k % factb == factb - 1) THEN GOTO trivial_factor
IF (k % factc == factc - 1) THEN GOTO trivial_factor
IF (k % factd == factd - 1) THEN GOTO trivial_factor
IF (k % facte == facte - 1) THEN GOTO trivial_factor
IF (k % factf == factf - 1) THEN GOTO trivial_factor

#-----------------------------------------------------------------------------------------
# The next 4 lines are the test for MOB. If the k is evenly divisible by the base, then
# add the type (Riesel or Sierp). PRP test the answer. If it is NOT prime, it is a MOB
# and will be added to the MOB file. If it is prime, it continues as normal.
# This logic is correct.
#-----------------------------------------------------------------------------------------
LABEL calc_mob
IF (k % base > 0) THEN GOTO Do_n
SETS test_str,%d+%d;k;type_str
PRP k + type, test_str
IF !(ISPRIME) THEN GOTO MOB

LABEL Do_n
SET n, 0

LABEL next_n
SET n, n + 1
SETS test_str,%d*%d^%d%s1;k;base;n;type_str
PRP k * base ^ n + type, test_str

IF (ISPRIME) THEN GOTO Prime_found
IF (n < max_n) THEN GOTO next_n

SETS tmpstr,%d*%d^n%s1;k;base;type_str
WRITE k_file,tmpstr
GOTO next_k

LABEL GFN
SET gfn_k, gfn_k * base
SETS tmpstr,%d;k
WRITE g_file,tmpstr
GOTO next_k

LABEL MOB
SETS tmpstr,%d;k
WRITE m_file,tmpstr
GOTO next_k

LABEL Prime_found
SETS tmpstr,%d*%d^%d%s1;k;base;n;type_str
WRITE p_file,tmpstr
GOTO next_k

LABEL trivial_factor
SETS tmpstr,%d;k
WRITE t_file,tmpstr
GOTO next_k

LABEL END
END
[/code]

henryzz 2009-12-18 20:44

[quote=rogue;199244]Also note that script files cannot take advantage of fast modular reduction. This is another good reason to not test too a high value of n with this script.
[/quote]
I presume there isnt an easy fix for this as it could save some time on large conjecture bases.

rogue 2009-12-18 21:43

[QUOTE=henryzz;199246]I presume there isnt an easy fix for this as it could save some time on large conjecture bases.[/QUOTE]

I'm working on it.

rogue 2009-12-19 02:58

[QUOTE=rogue;199247]I'm working on it.[/QUOTE]

I should have a beta ready by tomorrow. There are a number of changes to script handling for the next release including:

PRP sets ISPRP (new) and ISPRIME, but only sets ISPRIME if the number is prime.

PRIMEP, PRIMEM, and PRIMEC will do primality tests (-tp, -tm, and -tc). They will set both ISPRP and ISPRIME if the number is prime.

PRP can now take a string expression as its sole argument. This prevents the decimal expansion and allows PFGW to take advantage of special modular reduction. PRIMEP, PRIMEM, and PRIMEC only take a single string argument.

Some of this is subject to change if I get too much push back from other script users.

Mini-Geek 2009-12-20 00:13

Copying this script here:[quote=Mini-Geek;199328]Like this one: :smile:[code]#!/usr/bin/perl
#usage: file k a b
#e.g. 'mod 589 216 0 3' looks in 589.txt, removes values with k=216 and n=0 mod 3, and writes to 589-out.txt

open(IN, $ARGV[0].'.txt');
open(OUT, '>' .$ARGV[0].'-out.txt');
$k = $ARGV[1];
$a = $ARGV[2];
$b = $ARGV[3];
$removecount = 0;

while(<IN>)
{
$line = $_;
chomp($line);
@linearray = split(/ /,$line);
$thisk = @linearray[0];
#if this isn't the right k, then we don't want to remove it, so print it out
if ($thisk != $k) {
print OUT $line."\n";
next;
}
$thisn = @linearray[1];
#if there is nothing after a space on this line (usually the first line of a NewPGen-format file), then we don't want to remove it, so print it out
if ($thisn == '') {
print OUT $line."\n";
next;
}
$thisa = $thisn % $b;

#if the modular value isn't right, then we don't want to remove it, so print it out
if ($thisa != $a) {
print OUT $line."\n";
next;
}

#if we've got here, then everything matches, and we remove it by not doing anything, we also increment a counter so we can see how many were removed
$removecount++;
}
print "removed $removecount line(s)";

close(IN);
close(OUT);[/code]To use it, (say it's named mod.pl and the NewPGen-format file is 589.txt) run 'perl mod.pl 589 216 0 3', and it will write everything except k=216 values whose n=0 mod 3 to 589-out.txt. It's had only minimal testing, but I don't see any reason for it not to work as long as you use a NewPGen format file.[/quote]

gd_barnes 2010-01-07 06:52

[quote=rogue;199244]This is Gary's PFGW script for searching, with changes to output the number being tested as k*b^n+/-c instead of the decimal expansion. This is also output to pfgw.log and pfgw_prime.log in the same way. Note that PFGW will not verify that the string being output is the same as the number being tested.
[/quote]

I have added this enhancement in the lastest version of the script, which also corrects a previously mentioned exclusion of GFNs problem. See the "starting new bases" thread.

gd_barnes 2010-01-07 08:05

[quote=rogue;199244]LABEL calc_mob
IF (k % base > 0) THEN GOTO Do_n
SETS test_str,%d+%d;k;type_str
PRP k + type, test_str
IF !(ISPRIME) THEN GOTO MOB
[/quote]


Mark,

The above code was causing a problem. The SETS code should be:
SETS test_str,%d[COLOR=red]%s1[/COLOR];k;type_str

That is with the "+" removed, "d" instead of "s" and a "1" at the end. I'm correcting it in the latest version of the script. With the correction, the output in pfgw-prime.log comes out as "12+1" and in pfgw.out as "12+1 factors prime!: 13". I assume that was your intent.

The funny thing is before when everything was displayed in decimal format, I didn't even realize that it was writing out the primes that pertain to MOB, which really isn't necessary.

Question: Is there a way to make it NOT write to those files for MOB? If not, it's not a big deal because the important thing is that these superflous primes don't show up in pl_prime.txt file. But it'd be nice to not have them in those places since the primes don't apply to the conjectures themselves.


Gary

Mini-Geek 2010-01-21 02:31

After [URL="http://www.mersenneforum.org/showthread.php?p=202628#post202628"]rogue suggested it[/URL], I've added the ability to remove k's from pl_remain.txt along with removing them from the sieve file. Here is an updated version of my remove-ks.pl script: (it takes the same command line arguments, and assumes the availability of grep and sed in addition to srfile)
[code]#!/usr/bin/perl
# usage:
# have pl_remain.txt present
# have a file with one k or prime per line
# have sr_b.abcd present, the srfile-readable sieve file to remove from
# have srfile, grep, and sed present
# run 'perl remove-ks.pl b t file' with b=the base and t=+ or - for Sierp or Riesel,
# e.g. 'perl remove-ks.pl 3 - pfgw.log' uses srfile to delete all k's with primes (found in pfgw.log) from sr_3.abcd and pl_remain.txt
# the input file can either have a single k per line, or a prime

$b = $ARGV[0];
$t = $ARGV[1];
open(IN, $ARGV[2]);

while(<IN>)
{
# get k
$line = $_;
chomp($line);
@linearray = split(/\*/,$line);

# remove k from sieve file
system("srfile -d \"@linearray[0]*$b\^n".$t."1\" -a sr_$b.abcd");

# remove k from pl_remain.txt
system("grep -v \"@linearray[0]\" pl_remain.txt > temp.txt");
system("sed '1d' temp.txt > pl_remain.txt");
system("del temp.txt");
}

close(IN);
[/code]If you only want it to do one or the other (sieve file or pl_remain.txt and not both), comment out the appropriate line(s) of code. They're clearly marked.

Mini-Geek 2010-01-21 13:33

I've changed it from using srfile and grep and doing it all one k at a time to building a list of k's, then running it all through egrep at once (I switched from grep to egrep since it had an option I needed: reading the search pattern from a file, and a useful one: the -h option to not print the file name). This makes it MUCH faster, which is significant for removing many k's at once. It also prints out some status information so you know what's going on.
It now asks for the prime file and sieve file instead of the base, type, and prime file. More instructions are in the comments at the top of the file.
Instead of wanting the sieve file to be ABCD format and in sr_{b}.abcd, it is expected to be in the PFGW/ABC/NewPGen format where the k is on each line, and the file to find it in is specified by the user.
[code]#!/usr/bin/perl
# remove-ks.pl 3.0
# usage:
# have pl_remain.txt present
# have a file with one k or prime per line
# have a sieve file present, in a format that has the k on every line, e.g. PFGW/NewPGen's "k n", and not ABCD
# have egrep present
# run 'perl remove-ks.pl primefile sievefile'
# e.g. 'perl remove-ks.pl pfgw.log work.txt' uses srfile to delete all k's with primes (found in pfgw.log) from work.txt and pl_remain.txt
# the prime file can have, per line: a single k, (e.g. "1234") a prime, (e.g. "1234*3^2345-1") or a sequence (e.g. "1234*3^n-1");
# really anything, as long as it starts with the k, and if it continues, the k is followed by a *
# (e.g. "1234", "1234*3^2345-1", "1234*3^n-1", and "1234*JUNKTEXT" are all valid and will remove k=1234)

open(IN, $ARGV[0]);
$sievefile = $ARGV[1];
$ks = "";

system("echo getting list of k's...");

while(<IN>)
{
# get ks
$line = $_;
chomp($line);
@linearray = split(/\*/,$line);
$ks .= @linearray[0] . '|';
}
# remove the final | character since we don't need or want it
chop($ks);

# write the list of k's, formatted as a regex search pattern, to a file
open(OUT, '>ks.txt');
print OUT $ks;
close(OUT);

system("echo done, removing k's from sieve file...");

# remove k's from sieve file
system("egrep -vhf ks.txt $sievefile > temp.txt");
system("del $sievefile");
system("ren temp.txt $sievefile");

system("echo done, removing k's from pl_remain.txt file...");

# remove k's from pl_remain.txt
system("egrep -vhf ks.txt pl_remain.txt > temp.txt");
system("del pl_remain.txt");
system("ren temp.txt pl_remain.txt");

# clean-up
system("del ks.txt");

system("echo done, exiting");

close(IN);
[/code]

rogue 2010-01-21 15:39

I had problems with your newest script. I also think that it might have problems WRT removing entries from the ABC file. For example if you have k=123 and the ABC file has n with n=%123%, then won't egrep also remove those lines?

What we really need is for srfile to be able to remove multiple sequences with a single run rather than having to run it once for each sequence.

mdettweiler 2010-01-21 16:27

[quote=rogue;202713]I had problems with your newest script. I also think that it might have problems WRT removing entries from the ABC file. For example if you have k=123 and the ABC file has n with n=%123%, then won't egrep also remove those lines?[/quote]
Would this be fixed by having the script remove, in the above example, lines matching the regular expression:
[code]'123 ' instead of '123'[/code]
Of course, while this limits the script to numbers with "123" at the end of their k-value, it still doesn't completely ensure that there's nothing else prior to the 123 in the k. Possibly it could check to verify that the 123 is at the beginning of the line--combined with the trailing space, that would do the trick.

Mini-Geek 2010-01-21 17:30

Sorry for not thinking about those sorts of problems. I made it with Riesel base 3 work in mind, where these problems wouldn't come up.
[quote=rogue;202713]I had problems with your newest script.[/quote]
If you want any help, you'll have to tell me the problems. :smile: (unless you meant the issue you said next)
[quote=rogue;202713]What we really need is for srfile to be able to remove multiple sequences with a single run rather than having to run it once for each sequence.[/quote]
Yeah, that'd be nice. If it could also print a list of the sequences in a file, then the need for something like my script here would be eliminated. (maybe a batch file to do everything you want could be used instead)
[quote=rogue;202713]I also think that it might have problems WRT removing entries from the ABC file. For example if you have k=123 and the ABC file has n with n=%123%, then won't egrep also remove those lines?[/quote][quote=mdettweiler;202717]Would this be fixed by having the script remove, in the above example, lines matching the regular expression:
[code]'123 ' instead of '123'[/code]Of course, while this limits the script to numbers with "123" at the end of their k-value, it still doesn't completely ensure that there's nothing else prior to the 123 in the k. Possibly it could check to verify that the 123 is at the beginning of the line--combined with the trailing space, that would do the trick.[/quote]
You're both right. But in pl_remain.txt the usual format is "123*..." so it would complicate things to try to match it to "k " (the k with a space after it).
There is another shorthand: ^ means the beginning of the line.
Replace the applicable line (line 26) with this to make it run a safer check for the k:[code] $ks .= '^' . @linearray[0] . '[^0-9]|';
[/code]Instead of simply a space, I made it "[^0-9]" (translated: a character that is not a numeral) so that it can also work with the * from pl_remain.txt or any other separator (excluding where each k is alone on its line).
This makes the whole, generated, pattern look like:[code]^805817398[^0-9]|^807426766[^0-9]|...[/code](translated: (the beginning of a line followed by the first k followed by a character that is not a numeral) or (...))
Hopefully this should satisfy everybody. :smile: It certainly seems to me that it would work with nearly all common file formats and work types.
Edit: I removed the "?" after the "[^0-9]" since it made it match some k's that shouldn't (e.g. the code for 123 would match 1234). The only downside is it doesn't support k-only files (where each line is a single k). This could probably be added with another ^-like code if anybody thinks it useful...

Mini-Geek 2010-01-22 02:34

Just so the current version is in one place, instead of having 3.0 and then a 'patch':
(added changelog)
[code]#!/usr/bin/perl
# remove-ks.pl 3.1.1
# usage:
# have pl_remain.txt present
# have a file with one k or prime per line
# have a sieve file present, in a format that has the k on every line, e.g. PFGW/NewPGen's "k n", and not ABCD
# have egrep present
# run 'perl remove-ks.pl primefile sievefile'
# e.g. 'perl remove-ks.pl pfgw.log work.txt' uses srfile to delete all k's with primes (found in pfgw.log) from work.txt and pl_remain.txt
# the prime file can have, per line: a single k, (e.g. "1234") a prime, (e.g. "1234*3^2345-1") or a sequence (e.g. "1234*3^n-1");
# really anything, as long as it starts with the k, and if it continues, the k is followed by a *
# (e.g. "1234", "1234*3^2345-1", "1234*3^n-1", and "1234*JUNKTEXT" are all valid and will remove k=1234)

# changelog:
# 3.1.1: removed the "?" after the "[^0-9]" since it made it match some k's that shouldn't (e.g. the code for 123 would match 1234)
# 3.1: made the search pattern start on a new line and be followed by something besides a numeral to stop many possible false positives
# 3.0: added version number; rewrote to get all the k's first, then use egrep to remove them all from both files in one shot; results in a huge speed improvement
# 2.0: (unnumbered originally) uses grep to remove k's from pl_remain.txt along with using srfile to remove them from the sieve file
# 1.0: (unnumbered originally) first release, used srfile to delete k's from a sieve file

open(IN, $ARGV[0]);
$sievefile = $ARGV[1];
$ks = "";

system("echo getting list of k's...");

while(<IN>)
{
# get ks
$line = $_;
chomp($line);
@linearray = split(/\*/,$line);
$ks .= '^' . @linearray[0] . '[^0-9]|';
}
# remove the final | character since we don't need or want it
chop($ks);

# write the list of k's, formatted as a regex search pattern, to a file
open(OUT, '>ks.txt');
print OUT $ks;
close(OUT);

system("echo done, removing k's from sieve file...");

# remove k's from sieve file
system("egrep -vhf ks.txt $sievefile > temp.txt");
system("del $sievefile");
system("ren temp.txt $sievefile");

system("echo done, removing k's from pl_remain.txt file...");

# remove k's from pl_remain.txt
system("egrep -vhf ks.txt pl_remain.txt > temp.txt");
system("del pl_remain.txt");
system("ren temp.txt pl_remain.txt");

# clean-up
system("del ks.txt");

system("echo done, exiting");

close(IN);
[/code]

Mini-Geek 2010-01-22 13:55

# 3.2: now supports command line arguments for what files to work on; also helpfully errors out if you don't give it sufficient input
How you give it input has changed. Now you do it with -p -s and -r command line arguments (for prime, sieve, and remaining, files, respectively). You must have -p and at least one of -s or -r. This should make it much more user-friendly. :smile:
[code]#!/usr/bin/perl
# remove-ks.pl 3.2
# usage:
# have a file with one k or prime per line (k's to remove)
# have at least one of:
# a file of remaining k's present (e.g. pl_remain.txt)
# a sieve file present, in a format that has the k on every line, e.g. PFGW/NewPGen's "k n", and not ABCD
#
# have egrep present
# run 'perl remove-ks.pl -p primefile -s sievefile -r remainingksfile' (-p mandatory, need at least one of -s and -r)
# e.g. 'perl remove-ks.pl -p pfgw.log -s work.txt -r pl_remain.txt' deletes all k's with primes (found in pfgw.log) from work.txt and pl_remain.txt
# or 'perl remove-ks.pl -p pfgw-prime.log -r pl_remain.txt' deletes all k's with primes (found in pfgw-prime.log) from pl_remain.txt
# the prime file can have, per line: a single k, (e.g. "1234") a prime, (e.g. "1234*3^2345-1") or a sequence (e.g. "1234*3^n-1");
# really anything, as long as it starts with the k, and if it continues, the k is followed by a *
# (e.g. "1234", "1234*3^2345-1", "1234*3^n-1", and "1234*JUNKTEXT" are all valid and will remove k=1234)

# changelog:
# 3.2: now supports command line arguments for what files to work on; also helpfully errors out if you don't give it sufficient input
# 3.1.1: removed the "?" after the "[^0-9]" since it made it match some k's that shouldn't (e.g. the code for 123 would match 1234)
# 3.1: made the search pattern start on a new line and be followed by something besides a numeral to stop many possible false positives
# 3.0: added version number; rewrote to get all the k's first, then use egrep to remove them all from both files in one shot; results in a huge speed improvement
# 2.0: (unnumbered originally) uses grep to remove k's from pl_remain.txt along with using srfile to remove them from the sieve file
# 1.0: (unnumbered originally) first release, used srfile to delete k's from a sieve file

use Getopt::Std;
getopt('spr');

if (length($opt_p) < 1) { # if no prime file was specified
print "must supply a file with primes (k's to remove), usage: -p filename.txt\n";
exit;
}
if (length($opt_s) < 1) {
$remsieve = 0;
} else {
$sievefile = $opt_s;
$remsieve = 1;
}
if (length($opt_r) < 1) {
$remremain = 0;
} else {
$remainfile = $opt_r;
$remremain = 1;
}
if ((!$remremain) && (!$remsieve)) { # if neither sieve nor remain file was specified
print "must supply at least one file to remove from, usage: -r pl_remain.txt or -s sievefile.txt\n";
exit;
}
$kstxt = "temp-ks.txt";
if ($remainfile == $kstxt) { # just in case the list of remaining k's is at temp-ks.txt, change our temp list
$kstxt = "temp-ks-alt.txt";
}


open(IN, $opt_p);

print "getting list of k's to remove from $opt_p...\n";

while(<IN>)
{
# get ks
$line = $_;
chomp($line);
@linearray = split(/\*/,$line);
$ks .= '^' . @linearray[0] . '[^0-9]|';
}
# remove the final | character since we don't need or want it
chop($ks);

# write the list of k's, formatted as a regex search pattern, to a file
open(OUT, ">$kstxt");
print OUT $ks;
close(OUT);

if ($remremain) {
print "done, removing k's from $remainfile...\n";

# remove k's from remaining k's file
system("egrep -vhf $kstxt $remainfile > temp.txt");
system("del $remainfile");
system("ren temp.txt $remainfile");
}

if ($remsieve) {
print "done, removing k's from $sievefile...\n";

# remove k's from sieve file
system("egrep -vhf $kstxt $sievefile > temp.txt");
system("del $sievefile");
system("ren temp.txt $sievefile");
}

# clean-up
system("del $kstxt");

print "done, exiting\n";

close(IN);
[/code]I also noticed that this statement was mistaken:[quote=Mini-Geek;202724]Edit: I removed the "?" after the "[^0-9]" since it made it match some k's that shouldn't (e.g. the code for 123 would match 1234). [B]The only downside is it doesn't support k-only files (where each line is a single k).[/B] This could probably be added with another ^-like code if anybody thinks it useful...[/quote]
It still works when each line is just a single k. So it works properly with every format I can think of where the k comes at the start of each line. :smile:
e.g. it properly recognizes every instance of 123 in this file:[code]123 456
1123 456
456 123
7456 789
1234 987
123
1234
123*3^n-1
1234*3^n-1
4123[/code]

rogue 2010-02-03 19:18

I have a a mods to srfile so that the -d argument can take an input file or an sequence. The input file would be a file such as pfgw.log, which has a sequence (or number) on each line. It will delete all candidates from the input file before saving. This should be a huge time saver if you have very large files from srsieve/sr2sieve. I've forwarded the changes to Geoff, but his focus has been elsewhere so it is unknown if and/or when it will make it into a release. If anyone is interested I can post the code changes here.

gd_barnes 2010-02-04 10:00

[quote=rogue;202713]I had problems with your newest script. I also think that it might have problems WRT removing entries from the ABC file. For example if you have k=123 and the ABC file has n with n=%123%, then won't egrep also remove those lines?

What we really need is for srfile to be able to remove multiple sequences with a single run rather than having to run it once for each sequence.[/quote]


You've used the acronym WRT numerous times. Please clarify what it means.

I googled it to no avail.

Batalov 2010-02-04 10:12

'with respect to'?

Xentar 2010-02-04 11:10

[url]http://www.acronymfinder.com/WRT.html[/url]
:smile:

gd_barnes 2010-02-04 11:15

Thanks for the info guys. That's a cool link Xentar. I've bookmarked the page for future confusing acronyms. :-)

Mini-Geek 2010-03-19 19:55

[quote=Batalov;208827]A script for algebraic elimination hints for pl_remain.txt
[ATTACH]4870[/ATTACH]

If it reports
[FONT=Fixedsys]* x^3 ...[/FONT]
then it means the whole k can be eliminated (for a hypothetical example 8*125^n+1) -- practice shows that even squares are occasionally missed.

If it says
[FONT=Fixedsys]1|3 46^3 184*529^n+1[/FONT]
then just eliminate n=1 (mod 3) from the ...b529_k184... file[/quote]
copied here :smile:

Mini-Geek 2010-03-19 20:20

[quote=Batalov;208827]A script for algebraic elimination hints for pl_remain.txt
[ATTACH]4870[/ATTACH]

If it reports
[FONT=Fixedsys]* x^3 ...[/FONT]
then it means the whole k can be eliminated (for a hypothetical example 8*125^n+1) -- practice shows that even squares are occasionally missed.

If it says
[FONT=Fixedsys]1|3 46^3 184*529^n+1[/FONT]
then just eliminate n=1 (mod 3) from the ...b529_k184... file[/quote]
A very cool and useful script, once I figured out how to use it. :smile:
For anybody else trying to figure it out, it works off of standard input or by one or two file names added as arguments (or at least that's what seems to be the case), and takes sequences in a form like 8*125^n+1 (one per line), as you'd find in pl_remain.txt after running the PFGW script.
So you can run "hiddenPowers.pl pl_remain.txt" or "hiddenPowers.pl" followed by entering one or more sequences on standard input.
I found a minor bug: when the file has no trailing newline and the final number has more than one elimination, it does not put a newline between the lines of the last sequence's eliminations, making it run together confusingly.
e.g. with this file:[code]2*18^n-1
64*177^n-1[/code](with no trailing newline) it outputs:[code]1|2 6^2 2*18^n-1
0|2 8^2 64*177^n-10|3 4^3 64*177^n-1[/code]Using a chomp on "$_" and then adding "\n"'s (newlines) where needed fixes the problem.

I thought a more human-readable output would be better, so I made it look like this (with my above bug fixed):[code]2*18^n-1 n=1 mod 2 factors due to 6^2
64*177^n-1 n=0 mod 2 factors due to 8^2
64*177^n-1 n=0 mod 3 factors due to 4^3
8*125^n-1 every n factors due to x^3[/code]If you, or anybody else, is interested, here's the code:
[code]#!/usr/bin/perl -w
# you can use this script on pl_remain.txt
# created by Batalov, modified by Mini-Geek
use Math::BigInt;

line: while(<>) {
chomp($_);
next unless /^(\d+)\*(\d+)\^n([+-])1/ && $1 && $2;
$k = Math::BigInt->new($1);
$a = Math::BigInt->new($2);
my @powers = ($3 eq '+') ? qw(3 5 7 11) : qw(2 3 5 7 11);

foreach $m (@powers) {
$b = $k->copy()->broot($m);
if ($b->copy()->bpow($m) == $k) {
if ($a->copy()->broot($m)->bpow($m) == $a) {
print "$_\tevery n factors due to x^$m\n";
next line;
}
print "$_\tn=0 mod $m factors due to $b^$m\n";
}
}
for ($n=1; $n<$powers[$#powers]; $n++) {
$a->bmul($k);
foreach $m (@powers) {
next if $n>=$m;
$b = $a->copy()->broot($m);
print "$_\tn=$n mod $m factors due to $b^$m\n" if ($b->copy()->bpow($m) == $a);
}
}
}
[/code]

Batalov 2010-03-19 21:13

Ah. The script needed Windows-proofing and usability testing.
It's the Windows-style line breaks that were confusing it.
Looks good! Thanks.

rogue 2010-03-19 21:30

How does one use this script to remove candidates from an ABC file?

Batalov 2010-03-19 22:04

Can't say anything about ABCD files, but for ABC I simply do
(e.g. if the hint script said "remove n= 1 mod 3") this:

[FONT=Fixedsys]head -1 myold.npg > mybetter.npg[/FONT]
[FONT=Fixedsys]tail +2 myold.npg | awk '$2%3!=1' >> mybetter.npg[/FONT]
[FONT=Fixedsys][/FONT]
I do that as early as possible, after the script is done, the "hiddenPowers pl_remain.txt" is run and initial srsieve runs are done. Some "[FONT=Fixedsys]mybetter.npg[/FONT]" (for individual [I]k[/I]'s) files will become empty after the above filtering.
...and then merge all npg files with sort +1n and awk into the ABC file...
Later .npg files can be sieved some more as needed, and re-merged, etc.

Mini-Geek 2010-03-19 23:30

[quote=rogue;208942]How does one use this script to remove candidates from an ABC file?[/quote]
I use mod.pl, which I made, not this hiddenPowers script (since it doesn't do it :smile:).
Here it is:
[URL]http://www.mersenneforum.org/showthread.php?p=199328#post199328[/URL]
An example usage would be: (doing this manually, where lines starting with ">" show a command that was run)[code]>hiddenPowers.pl pl_remain.txt
64*177^n-1 n=0 mod 2 factors due to 8^2
64*177^n-1 n=0 mod 3 factors due to 4^3
>mod.pl 177-file 64 0 2
>mod.pl 177-file-out 64 0 3
>ren 177-file-out-out.txt the-result.txt[/code](the last step was, of course, not strictly necessary, but I put it to clarify that the final result was in "177-file-out-out.txt")
It might be worthwhile to merge hiddenPowers.pl and mod.pl, or make a modification of hiddenPowers.pl that optionally runs mod.pl. If I have some free time over the weekend (I should), and feel like it, (I probably will) I'll at least look in to this.

Mini-Geek 2010-03-20 17:41

remove-ks.pl 3.2.1:
[LIST][*]improved help text[*]made it show full help whenever something's wrong (instead of specific to the area) so that just running the script will give you all the info[/LIST] [code]#!/usr/bin/perl
# remove-ks.pl 3.2.1
# usage:
# have a file with one k or prime per line (k's to remove)
# have at least one of:
# a file of remaining k's present (e.g. pl_remain.txt)
# a sieve file present, in a format that has the k on every line, e.g. PFGW/NewPGen's "k n", and not ABCD
#
# have egrep present
# run 'perl remove-ks.pl -p primefile -s sievefile -r remainingksfile' (-p mandatory, need at least one of -s and -r)
# e.g. 'perl remove-ks.pl -p pfgw.log -s work.txt -r pl_remain.txt' deletes all k's with primes (found in pfgw.log) from work.txt and pl_remain.txt
# or 'perl remove-ks.pl -p pfgw-prime.log -r pl_remain.txt' deletes all k's with primes (found in pfgw-prime.log) from pl_remain.txt
# the prime file can have, per line: a single k, (e.g. "1234") a prime, (e.g. "1234*3^2345-1") or a sequence (e.g. "1234*3^n-1");
# really anything, as long as it starts with the k, and if it continues, the k is followed by a *
# (e.g. "1234", "1234*3^2345-1", "1234*3^n-1", and "1234*JUNKTEXT" are all valid and will remove k=1234)

# changelog:
# 3.2.1: improved help text, made it show all info whenever something's wrong
# 3.2: now supports command line arguments for what files to work on; also helpfully errors out if you don't give it sufficient input
# 3.1.1: removed the "?" after the "[^0-9]" since it made it match some k's that shouldn't (e.g. the code for 123 would match 1234)
# 3.1: made the search pattern start on a new line and be followed by something besides a numeral to stop many possible false positives
# 3.0: added version number; rewrote to get all the k's first, then use egrep to remove them all from both files in one shot; results in a huge speed improvement
# 2.0: (unnumbered originally) uses grep to remove k's from pl_remain.txt along with using srfile to remove them from the sieve file
# 1.0: (unnumbered originally) first release, used srfile to delete k's from a sieve file

use Getopt::Std;
getopt('spr');

$help = "must supply a file with primes (k's to remove), -p filename.txt, and\nat least one file to remove from, -r pl_remain.txt or -s sievefile.txt\n\nUsage: remove-ks.pl -p PRIME_FILE [-r K_FILE] [-s SIEVE_FILE]\n remove-ks.pl -p pfgw.log -r pl_remain.txt -s sieve.txt";

if (length($opt_p) < 1) { # if no prime file was specified
print $help;
exit;
}
if (length($opt_s) < 1) {
$remsieve = 0;
} else {
$sievefile = $opt_s;
$remsieve = 1;
}
if (length($opt_r) < 1) {
$remremain = 0;
} else {
$remainfile = $opt_r;
$remremain = 1;
}
if ((!$remremain) && (!$remsieve)) { # if neither sieve nor remain file was specified
print $help;
exit;
}
$kstxt = "temp-ks.txt";
if ($remainfile == $kstxt) { # just in case the list of remaining k's is at temp-ks.txt, change our temp list
$kstxt = "temp-ks-alt.txt";
}


open(IN, $opt_p);

print "getting list of k's to remove from $opt_p...\n";

while(<IN>)
{
# get ks
$line = $_;
chomp($line);
@linearray = split(/\*/,$line);
$ks .= '^' . @linearray[0] . '[^0-9]|';
}
# remove the final | character since we don't need or want it
chop($ks);

# write the list of k's, formatted as a regex search pattern, to a file
open(OUT, ">$kstxt");
print OUT $ks;
close(OUT);

if ($remremain) {
print "done, removing k's from $remainfile...\n";

# remove k's from remaining k's file
system("egrep -vhf $kstxt $remainfile > temp.txt");
system("del $remainfile");
system("ren temp.txt $remainfile");
}

if ($remsieve) {
print "done, removing k's from $sievefile...\n";

# remove k's from sieve file
system("egrep -vhf $kstxt $sievefile > temp.txt");
system("del $sievefile");
system("ren temp.txt $sievefile");
}

# clean-up
system("del $kstxt");

print "done, exiting\n";

close(IN);
[/code]

henryzz 2010-03-20 17:50

would it be possible for your script to accept as input the output from batalov's script?

Mini-Geek 2010-03-20 18:45

1 Attachment(s)
[quote=henryzz;209037]would it be possible for your script to accept as input the output from batalov's script?[/quote]
I'm not sure which of my scripts you mean, but...
[quote=Mini-Geek;208955]It might be worthwhile to merge hiddenPowers.pl and mod.pl, or make a modification of hiddenPowers.pl that optionally runs mod.pl. If I have some free time over the weekend (I should), and feel like it, (I probably will) I'll at least look in to this.[/quote]
I've made it so hiddenPowers.pl will call mod.pl if you give it a sieve file to remove from. Both modified scripts are attached (mod.pl was only slightly modified, and is now improved for standalone use, and more easily compatible with use externally).
Don't give hiddenPowers.pl a sieve file without having mod.pl available. (it can still run alone as long as you only give it a pl_remain.txt file)

So all you have to do is run something like:
hiddenPowers.pl pl_remain.txt sievefile.txt

henryzz 2010-03-21 07:58

[quote=Mini-Geek;209045]I'm not sure which of my scripts you mean, but...

I've made it so hiddenPowers.pl will call mod.pl if you give it a sieve file to remove from. Both modified scripts are attached (mod.pl was only slightly modified, and is now improved for standalone use, and more easily compatible with use externally).
Don't give hiddenPowers.pl a sieve file without having mod.pl available. (it can still run alone as long as you only give it a pl_remain.txt file)

So all you have to do is run something like:
hiddenPowers.pl pl_remain.txt sievefile.txt[/quote]
Brilliant thats exactly what i was asking.:smile: Sorry to be unclear.

Mini-Geek 2010-03-21 12:37

I was wondering if anybody had written a script that will copy certain results (e.g. all base 168) from a PRPnet server's completed_tests.log file, then I realized grep could trivially do this. But I like being able to just plug in a number, so I made this little extract-base.pl script:[code]$base = $ARGV[0];
system 'grep "\*'.$base.'\^"'." completed_tests.log > completed_tests_$base.log";[/code]As you can see, it uses grep (so you'll need it available), must be run from the folder where completed_tests.log is, takes the base as its argument, and writes the output to completed_tests_$base.log).
Note that this looks for "*base^", so anything whose results don't appear in that form won't be picked up.
If grep is unavailable, this could be modified to use [URL="http://perldoc.perl.org/functions/grep.html"]Perl's grep function[/URL].

mdettweiler 2010-03-21 18:23

[quote=Mini-Geek;209100]I was wondering if anybody had written a script that will copy certain results (e.g. all base 168) from a PRPnet server's completed_tests.log file, then I realized grep could trivially do this. But I like being able to just plug in a number, so I made this little extract-base.pl script:[code]$base = $ARGV[0];
system 'grep "\*'.$base.'\^"'." completed_tests.log > completed_tests_$base.log";[/code]As you can see, it uses grep (so you'll need it available), must be run from the folder where completed_tests.log is, takes the base as its argument, and writes the output to completed_tests_$base.log).
Note that this looks for "*base^", so anything whose results don't appear in that form won't be picked up.
If grep is unavailable, this could be modified to use [URL="http://perldoc.perl.org/functions/grep.html"]Perl's grep function[/URL].[/quote]
Actually, a while back I wrote a rather more generalized version of this script as part of my results-processing suite. What it does is split the number on each result line of an LLRnet-formatted results file into its constituent parts (k, b, n, c) so that only results matching the given criteria will be printed to the output file:
[code]
#!/usr/bin/perl
print "Enter name of input file: ";
$inputFile = <STDIN>;
chomp($inputFile);
print "Enter name of output file: ";
$outputFile = <STDIN>;
chomp($outputFile);
open(INFILE, $inputFile);
open(OUTFILE, '>', $outputFile);
$usernameLine = <INFILE>;
$datetimeLine = <INFILE>;
while(<INFILE>) {
$justRead = $_;
($knpair, $foo) = split(/ is /, $justRead);
($k, $base_n_minus1) = split(/\*/, $knpair);
($base, $n_minus1) = split(/\^/, $base_n_minus1);
($n, $one) = split(/\-/, $n_minus1);
if([B]$n > 500000 and $n < 600001[/B]) {
print OUTFILE $usernameLine; print OUTFILE $datetimeLine; print OUTFILE $justRead;
}
$usernameLine = <INFILE>;
$datetimeLine = <INFILE>;
}
close(INFILE);
close(OUTFILE);
[/code]
I've highlighted the criteria line in bold. I suppose if I wanted to make it extra robust I could have it prompt for the criteria at the beginning, but for my own purposes it's easy enough to just edit the script (right now it's set for a Sierp. base 9 range that I last used it for).

While this deals strictly in LLRnet-formatted results (I always convert PRPnet results to LLRnet format before processing so I can use the same scripts), it could be modified to manual LLR format by removing all such occurrences of these lines:
[I]$usernameLine = <INFILE>;
$datetimeLine = <INFILE>;[/I]
And changing this line:
[I]print OUTFILE $usernameLine; print OUTFILE $datetimeLine; print OUTFILE $justRead;[/I]
to this:
[I]print OUTFILE $justRead;[/I]
I imagine it could be converted to PRPnet format without too much hassle.

Mini-Geek 2010-03-21 21:42

[quote=mdettweiler;209118]Actually, a while back I wrote a rather more generalized version of this script as part of my results-processing suite. What it does is split the number on each result line of an LLRnet-formatted results file into its constituent parts (k, b, n, c) so that only results matching the given criteria will be printed to the output file:
...
I imagine it could be converted to PRPnet format without too much hassle.[/quote]
Cool, cool. :smile: I don't think I'll be using this since right now I only need a simple task for PRPnet results and not a complex task for LLRnet-format results, but that's what this thread is for.
[quote=mdettweiler;209118][code] print OUTFILE $usernameLine; print OUTFILE $datetimeLine; print OUTFILE $justRead;[/code][/quote]
You can do the same thing, but with much less code and more readability, like this:
[code] print OUTFILE $usernameLine.$datetimeLine.$justRead;[/code]

mdettweiler 2010-03-21 22:41

[quote=Mini-Geek;209127]You can do the same thing, but with much less code and more readability, like this:
[code] print OUTFILE $usernameLine.$datetimeLine.$justRead;[/code][/quote]
Yeah, I wrote that script a while back, when I was still getting the hang of Perl. Needless to say, my more recent creations are a bit less messy. :smile: When I eventually get around to cleaning up my results-processing suite and integrating it into one software package for public release, I'll probably end up re-doing much of it.

Mini-Geek 2010-04-11 14:22

It's not in an entirely finished state (only works right with purely composite results; a prime will probably have a composite-like output but with PRIME as the residue), but here's a script I wrote to recreate (as best as possible with drawing only from the CandidateTest table; the only thing wrong is that the timestamps show the reservation date, not the finishing date) the completed_tests.log file from the DB. To use it, first export the wanted results (an example SQL query is given that does it for all results in the DB, but you might want to add other limitations e.g. "candidatename like '%*192^%'") to a tab-separated file (MySQL's default) containing, in this order:
CandidateName,TestID,PRPingProgram,Residue,EmailID,UserID,ClientID
Then, assuming the script is saved as recover-results.pl and the DB data is in DB-out.txt and you want the output in completed_tests.log, run this:
[code]perl recover-results.pl DB-out.txt > completed_tests.log[/code][code]# select CandidateName,TestID,PRPingProgram,Residue,EmailID,UserID,ClientID
# from CandidateTest where residue != 'null' order by TestID into outfile 'completed_tests.log';
use POSIX qw(strftime);

while (<>) {
chomp;
($num, $timestamp, $prog, $res, $email, $user, $client) = split(/\t/);
@time = gmtime($timestamp);
$timestr = strftime("%Y-%m-%d %H:%M:%S", @time);
print "[$timestr GMT] $num received by Email: $email User: $user Client: $client Program: $prog Residue: $res \n";
}[/code]

rogue 2010-04-11 16:10

[QUOTE=Mini-Geek;211385]It's not in an entirely finished state (only works right with purely composite results; a prime will probably have a composite-like output but with PRIME as the residue), but here's a script I wrote to recreate (as best as possible with drawing only from the CandidateTest table; the only thing wrong is that the timestamps show the reservation date, not the finishing date) the completed_tests.log file from the DB. To use it, first export the wanted results (an example SQL query is given that does it for all results in the DB, but you might want to add other limitations e.g. "candidatename like '%*192^%'") to a tab-separated file (MySQL's default) containing, in this order:
CandidateName,TestID,PRPingProgram,Residue,EmailID,UserID,ClientID
Then, assuming the script is saved as recover-results.pl and the DB data is in DB-out.txt and you want the output in completed_tests.log, run this:
[code]perl recover-results.pl DB-out.txt > completed_tests.log[/code][code]# select CandidateName,TestID,PRPingProgram,Residue,EmailID,UserID,ClientID
# from CandidateTest where residue != 'null' order by TestID into outfile 'completed_tests.log';
use POSIX qw(strftime);

while (<>) {
chomp;
($num, $timestamp, $prog, $res, $email, $user, $client) = split(/\t/);
@time = gmtime($timestamp);
$timestr = strftime("%Y-%m-%d %H:%M:%S", @time);
print "[$timestr GMT] $num received by Email: $email User: $user Client: $client Program: $prog Residue: $res \n";
}[/code][/QUOTE]

Is is possible for a test to have the residue of "PRP" instead of "PRIME". This would happen in the case that phrot was used (non-x86) or an older version or LLR was used without pfgw. If you join with the Candidate table, then the LastUpdateTime would be when the result was finished. You could then use the IsPRP and IsPrime columns instead of looking directly at the residue.

10metreh 2010-06-21 19:48

There is a bug in remove_ks.pl: when the prime file is large, egrep gives the error "Regular expression too big", and the script then wipes the sieve file and the file with ks remaining. This has caused me a big problem on Riesel base 51 because I didn't back up the file of ks remaining. The effort to solve that problem is being "coordinated" in the Bases 33-100 thread.

Mini-Geek 2010-10-13 13:49

[url]http://www.mersenneforum.org/showthread.php?p=233311#post233311[/url]
[QUOTE=Mini-Geek;233311]The odds of prime spreadsheet is very useful, but when the n (or, to a lesser extent, the k) varies greatly, (as is common in CRUS work, or over large areas of any work) it is hard to choose the right average n to get accurate results.
I've made a simple command line Java app that has the same function as the odds of prime spreadsheet, but instead of making you pick the average k and n, it reads each k/n pair and works off of that. It reports relevant numbers for primes and twin primes (not triplet or quadruplet).
It is attached as a .jar, along with the source (it's not commented, and it includes some other code unused here, but I figured better messy source than no source :smile:). Run it without any arguments (or with -h or whatever) to get help on how to use it ("java -jar calcPrimes.jar" will do it).
Note that it is pretty picky with the sieve depth and sieve file. The sieve depth parser is extremely simple: first replace "G" with 9 zeroes and "T" with 12 zeroes, then use Java's Long.parseLong (e.g. 1.5*10^12, 1.5T, and 15M are all invalid, while 1T, 1500G, and 15000000 are valid). And the sieve file must be in NewPGen format ("k n" on each line) with no header of any sort, just the k and n.
This has had very little testing, but I've checked it against the spreadsheet on one file, and the results seems to be accurate.[/QUOTE]
[URL="http://www.mersenneforum.org/attachment.php?attachmentid=5799&d=1286977423"]calcPrimes.zip[/URL]

Mini-Geek 2011-01-23 17:05

[QUOTE=Puzzle-Peter;248735]I am not sure about the "algebraic factors" which are mentioned in the software thread though.[/QUOTE]

See this post:
[url]http://www.mersenneforum.org/showthread.php?p=248690#post248690[/url]
And the script here (also linked up there):
[url]http://www.mersenneforum.org/showthread.php?p=209045#post209045[/url]

Puzzle-Peter 2011-01-23 17:08

[QUOTE=Mini-Geek;248737]See this post:
[URL]http://www.mersenneforum.org/showthread.php?p=248690#post248690[/URL]
And the script here (also linked up there):
[URL]http://www.mersenneforum.org/showthread.php?p=209045#post209045[/URL][/QUOTE]

Wow, that's a quick answer!
Thanks!

EDIT: is this made for linux? /usr/bin/perl does not sound like windows.

EDIT2: srsieve gives warnings for algebraic factors for k=324, 900, 1296, 4356, 5184, 5776, 9409, 11449, 14884. Does that mean I should remove these k's from the sieve file?

Mini-Geek 2011-01-23 18:38

[QUOTE=Puzzle-Peter;248738]EDIT: is this made for linux? /usr/bin/perl does not sound like windows.[/QUOTE]
It was made for Windows, but should also work on Linux with little to no modification. The /usr/bin/perl you mention is something that's usually at the start of all Perl files. If I'm not mistaken, that's for Unix systems to know what binary file to execute the script with. Windows has file association that can take care of that. Of course, you'll need some version of Perl installed to run the script.
[QUOTE=Puzzle-Peter;248738]EDIT2: srsieve gives warnings for algebraic factors for k=324, 900, 1296, 4356, 5184, 5776, 9409, 11449, 14884. Does that mean I should remove these k's from the sieve file?[/QUOTE]
I think MyDogBuster's answer is correct. I'll also note that the script will remove all numbers that need to be removed, which can include removing entire k's, so that's an easy no-confusing-math way to deal with this. :smile:

Puzzle-Peter 2011-01-23 19:25

I downloaded and installed strawberryperl for windows. I try to start the script with

hiddenPowers.pl pl_remain.txt sr_228.pfgw

and I get

Use of uninitialized value in open at C:\...\hiddenPowers.pl line 8
readline() on closed filehandle IN at C:\...\hiddenPowers.pl line 11

What's wrong?

Mini-Geek 2011-01-23 19:59

[QUOTE=Puzzle-Peter;248771]I downloaded and installed strawberryperl for windows. I try to start the script with

hiddenPowers.pl pl_remain.txt sr_228.pfgw

and I get

Use of uninitialized value in open at C:\...\hiddenPowers.pl line 8
readline() on closed filehandle IN at C:\...\hiddenPowers.pl line 11

What's wrong?[/QUOTE]

I'm not entirely sure what the problem is. Does pl_remain.txt exist in the folder you're in (and double check it's spelled/typed right, and isn't really .txt.txt or anything else)? If not, of course, that's your problem; if so, what is your Perl version? Maybe something changed that I need to fix for it to work with the newest versions. The version I've got is v5.10.0.
If a solution isn't apparent soon, you can just ignore it like gd_barnes said, since it doesn't make a huge difference, especially when working with small numbers.

Puzzle-Peter 2011-01-23 20:40

[QUOTE=Mini-Geek;248779]I'm not entirely sure what the problem is. Does pl_remain.txt exist in the folder you're in (and double check it's spelled/typed right, and isn't really .txt.txt or anything else)? If not, of course, that's your problem; if so, what is your Perl version? Maybe something changed that I need to fix for it to work with the newest versions. The version I've got is v5.10.0.
If a solution isn't apparent soon, you can just ignore it like gd_barnes said, since it doesn't make a huge difference, especially when working with small numbers.[/QUOTE]

I typed only the first characters and hit tab to auto-complete the file names, so they should be correct. I couldn't see any spelling errors either.
My perl version is 5.12.2.0 and I'm running WinXP 32bit.

EDIT: I just changed the second and third code line to

open(IN, "pl_remain.txt");
$file = "sr_228.pfgw";

and it started working, but every output line telling me which k is worked on is followed by "Removed 0 line(s)" and two "Syntax error" messages. Is abc a valid format for the script? I tried switching to NewPGen format using srfile -g, but I ended up with one file for each k...

MyDogBuster 2011-01-23 20:50

[QUOTE]I tried switching to NewPGen format using srfile -g, but I ended up with one file for each k... [/QUOTE]You need the -G not the -g. They are both NewPGen but -G gives you a PRP file and not individual files for each k

Puzzle-Peter 2011-01-23 20:56

[QUOTE=MyDogBuster;248794]You need the -G not the -g. They are both NewPGen but -G gives you a PRP file and not individual files for each k[/QUOTE]

Only one file now, but the syntax error messages are still there.

Mini-Geek 2011-01-24 00:26

[QUOTE=Puzzle-Peter;248791]My perl version is 5.12.2.0 and I'm running WinXP 32bit.[/QUOTE]
Pretty close to what I've got...I doubt it would be broken because of that.
[QUOTE=Puzzle-Peter;248791]EDIT: I just changed the second and third code line to

open(IN, "pl_remain.txt");
$file = "sr_228.pfgw";

and it started working, but every output line telling me which k is worked on is followed by "Removed 0 line(s)" and two "Syntax error" messages.[/QUOTE]
It looks like there's a problem with your Perl setup understanding how to read in arguments. Try looking at this and see if your .pl file association looks correct or not:
[url]http://www.perlmonks.org/?node_id=819994[/url]
In trying to test it all with Strawberry Perl to try to track down the bug, it seems there's a lot of weird things about it. Try installing [URL="http://www.activestate.com/activeperl"]ActivePerl[/URL] and running the script with that...
Are you running this from a command line or did you make a shortcut? Try doing it from an actual command prompt if you're not already. If you want to help debug this and none of this helped, try to see what you are and aren't getting as arguments, and see if you can get it to actually see what you're giving it.
Or you can just ignore it since it's probably not worth this much trouble just to remove a few numbers.
[QUOTE=Puzzle-Peter;248791]Is abc a valid format for the script? I tried switching to NewPGen format using srfile -g, but I ended up with one file for each k...[/QUOTE]
ABC should work fine, as long as it starts with k and n (which srfile -w does).

Puzzle-Peter 2011-01-29 07:24

I tried ActivePerl now and it works just like it should.

Thanks!

Mathew 2011-08-27 19:55

1 Attachment(s)
Hello,

Attached is a perl script which is intended for ranges/bases not started.

It does the following:
a) It creates the input file for a new_base
b) ./pfgw -f100 -l new_base
c) ./srsieve -a -n [n testing range] -N [n max sieving range] -P 100e6 -m 4e9 pl_remain.txt
d) ./sr2sieve -P [sieve range] -i sr_base.abcd
e) ./srfile -a -k factors sr_base.abcd
f) It gzips the entire folder

The attached Readme gives details of the commands needed. The _win version is for windows this has been tested against Strawberry Perl.

Note: This is intended for small k ranges. I have used it for k-ranges of 1M for base 7. I would recommend using no more than the range you would normally do using the starting bases script.

Suggestions/Improvements are welcome.

Thanks,

Mathew

henryzz 2012-04-03 10:43

1 Attachment(s)
I am near the end of a introductory probability course and thought I would fiddle with the odds of prime spreadsheet. I have added the probabilities of finding exactly 1, 2 or 3 primes, twins, triplets or quads.

I inputed the data from serp base 6 for n=510k-600k and there is:
36.768% chance of exactly 1 prime
17.770% chance of exactly 2 primes
5.725% chance of exactly 3 primes

Xentar 2012-05-29 19:53

Hello there,

I've got a small question, maybe someone can help me.
I have an input file with remaining k in the form
[code]634*19^n+1
1446*19^n+1
2526*19^n+1
...[/code]
This file was used by me to start sieving the range n=25k-100k.

And some primes in the form
[code]3706*19^25538+1
706894*19^25791+1
705766*19^25798+1
173056*19^25816+1
...[/code].
Now I'd like to start a new sieving, so I have to delete all the K, where a prime has been found, from the input file.
Can someone tell me how to do this easily? There are some hundred primes, so I would prefer a automatic solution. :smile:


Thank you,
Daniel

rogue 2012-05-29 20:19

[QUOTE=Xentar;300648]Hello there,

I've got a small question, maybe someone can help me.
I have an input file with remaining k in the form
[code]634*19^n+1
1446*19^n+1
2526*19^n+1
...[/code]
This file was used by me to start sieving the range n=25k-100k.

And some primes in the form
[code]3706*19^25538+1
706894*19^25791+1
705766*19^25798+1
173056*19^25816+1
...[/code].
Now I'd like to start a new sieving, so I have to delete all the K, where a prime has been found, from the input file.
Can someone tell me how to do this easily? There are some hundred primes, so I would prefer a automatic solution. :smile:


Thank you,
Daniel[/QUOTE]

Use srfile. Use the -d option with a file that contains a list of primes with one prime per line. It will ignore the value of n and remove the k.

Xentar 2012-05-29 21:43

Doh.. I am using this tool for some time now, but I don't see the most obvious things.

Thank you.


All times are UTC. The time now is 09:34.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.