the scripts thread

[TimSorbet]

If anybody has any useful scripts related to CRUS work, feel free to post them here.

Here's one I made: a super simple perl script to remove k's from a sieve file (e.g. if you just found some primes and need to eliminate them so you can sieve further). The input file can be just the k's, (e.g. "506304") the whole primes, (e.g. "506304*7^10150+1") or the sequences (e.g. "506304*7^n+1). Really anything, as long as it starts with the k and then, if it continues, the k is followed by a '*'. It's not terribly flexible, as the sieve file must always be in sr_b.abcd (e.g. "sr_7.abcd") and is written there when finished. Of course, you can change this with a simple modification to the script.
Maybe someday I'll make it read the base and type from the primes, if present. For now, I think it's great for its purpose.

#!/usr/bin/perl
# usage:
# have a file with one k or prime per line
# have sr_b.abcd present, the srfile-readable sieve file to remove from
# have srfile present
# run 'perl this-files-name.pl b t prime-file' with b=the base and t=+ or - for Sierp or Riesel, and prime-file a list of k's, primes, or sequences
# examples: (assuming this file is named remove-ks.pl)
# e.g. 'perl remove-ks.pl 3 - pfgw.log' uses srfile to delete all k's with primes (found in pfgw.log) from sr_3.abcd

$b = $ARGV[0];
$t = $ARGV[1];
open(IN, $ARGV[2]);

while(<IN>)
{
    $line = $_;
    chomp($line);
    @linearray = split(/\*/,$line);
    system("./srfile -d \"@linearray[0]*$b\^n".$t."1\" -a sr_$b.abcd");
}

close(IN);

This was made on/for Windows and may or may not work on Linux. I added a ./ in front of srfile since that's the only reason I can think of that it wouldn't work on Linux, and it still works like that on Windows.
Just to give full credit where it's due: this was inspired by one of Max's scripts, which didn't work for me on Windows, and uses a couple bits of code from some of his perl scripts.

And, everybody probably already knows of this one, but there's a PFGW script to make starting bases very easy, created by various people. http://www.mersenneforum.org/showpos...6&postcount=57

[rogue]

This is Gary's PFGW script for searching, with changes to output the number being tested as k*b^n+/-c instead of the decimal expansion. This is also output to pfgw.log and pfgw_prime.log in the same way. Note that PFGW will not verify that the string being output is the same as the number being tested.

Also note that script files cannot take advantage of fast modular reduction. This is another good reason to not test too a high value of n with this script.

SCRIPT
#----------------------------------------------------------------------------------------
# This Script is used to start new bases.  This is a version of a highly plagarized
# script that creates 3 new files: 1 containing k's eliminated by trivial factors, 1
# containing k's eliminated due to MOB (multiple of the base), and 1 containing k's
# eliminated by Generallized Fermat #'s (GFN's) for even Sierp bases.  The script now
# automatically calculates all k's that are relavent to these files.
#
# The original scripts were written by Micha (Michaf), Karsten (Kar_bon), and
# Ian (MyDogBuster).  More recent modifications were made by yours truly (gd_barnes).
#
# The maximum base that the script can handle is 2*3*5*7*11*13*17=510510.  Higher bases
# would need tweaking done to allow for more modulos for k's with trivial factors.
#----------------------------------------------------------------------------------------
                                                                                        
#----------------------------------------------------------------------------------------
# The following 5 lines control what is tested and are the only lines that should be
# changed.  Type is 1 for Sierp and -1 for Riesel.  The rest should be self explanatory.
#----------------------------------------------------------------------------------------
DIM base,  1022
DIM type,  1
DIM min_k, 1
DIM max_k, 7
DIM max_n, 10000

OPENFILEAPP g_file,pl_GFN.txt
OPENFILEAPP k_file,pl_remain.txt
OPENFILEAPP m_file,pl_MOB.txt
OPENFILEAPP p_file,pl_prime.txt
OPENFILEAPP t_file,pl_trivial.txt

DIMS tmpstr
DIMS type_str
DIMS test_str

DIM k, min_k - 1
DIM gfn_k, 1
DIM n
DIM cofact, base - 1
DIM fact, cofact
DIM prev_fact, 1
DIM fact_cnt, 0
DIM facta, -1
DIM factb, -1
DIM factc, -1
DIM factd, -1
DIM facte, -1
DIM factf, -1

IF (type == 1) THEN SETS type_str,+ ELSE SETS type_str,-

#-----------------------------------------------------------------------------------------
# The next lines calculate unique prime factors for base - 1.  These will be used
# as modulos to determine k's that contain trivial factors and can be eliminated.
#-----------------------------------------------------------------------------------------
LABEL factor_rtn
FACTORIZE cofact
IF (FACTORFOUND > 1) THEN SET fact, FACTORFOUND ELSE SET fact, cofact
IF (fact == prev_fact) THEN GOTO factor_rtnb

SET fact_cnt, fact_cnt + 1
IF (fact_cnt == 1) THEN SET facta, fact
IF (fact_cnt == 2) THEN SET factb, fact
IF (fact_cnt == 3) THEN SET factc, fact
IF (fact_cnt == 4) THEN SET factd, fact
IF (fact_cnt == 5) THEN SET facte, fact
IF (fact_cnt == 6) THEN SET factf, fact

LABEL factor_rtnb
IF (FACTORFOUND == 1) THEN GOTO next_k

SET cofact, cofact / fact
SET prev_fact, fact
GOTO factor_rtn

LABEL next_k
SET k, k + 1
IF (k > max_k) THEN GOTO END

#-----------------------------------------------------------------------------------------
# The next 5 lines exclude Generallized Fermat k's for even Sierp bases.
#-----------------------------------------------------------------------------------------
LABEL calc_gfn
IF (type == -1) THEN GOTO calc_trivial
IF (base % 2 == 1) THEN GOTO calc_trivial
IF (k < gfn_k) THEN GOTO calc_trivial
IF (k == gfn_k) THEN GOTO GFN ELSE SET gfn_k, gfn_k * base
GOTO calc_gfn

#-----------------------------------------------------------------------------------------
# The next lines check to see if the k contains trivial factors.  If so, the k is
# bypassed.  k's with trivial factors are not written out for odd k's on odd bases.
#-----------------------------------------------------------------------------------------
LABEL calc_trivial
IF (facta > 2) THEN GOTO calc_trivial_r
IF (k % 2 == 1) THEN GOTO next_k

LABEL calc_trivial_r
IF (type == 1) THEN GOTO calc_trivial_s
IF (k % facta == 1) THEN GOTO trivial_factor
IF (k % factb == 1) THEN GOTO trivial_factor
IF (k % factc == 1) THEN GOTO trivial_factor
IF (k % factd == 1) THEN GOTO trivial_factor
IF (k % facte == 1) THEN GOTO trivial_factor
IF (k % factf == 1) THEN GOTO trivial_factor
GOTO calc_mob

LABEL calc_trivial_s
IF (k % facta == facta - 1) THEN GOTO trivial_factor
IF (k % factb == factb - 1) THEN GOTO trivial_factor
IF (k % factc == factc - 1) THEN GOTO trivial_factor
IF (k % factd == factd - 1) THEN GOTO trivial_factor
IF (k % facte == facte - 1) THEN GOTO trivial_factor
IF (k % factf == factf - 1) THEN GOTO trivial_factor

#-----------------------------------------------------------------------------------------
# The next 4 lines are the test for MOB.  If the k is evenly divisible by the base, then 
# add the type (Riesel or Sierp).  PRP test the answer.  If it is NOT prime, it is a MOB
# and will be added to the MOB file.  If it is prime, it continues as normal.
# This logic is correct.
#-----------------------------------------------------------------------------------------
LABEL calc_mob
IF (k % base > 0) THEN GOTO Do_n
SETS test_str,%d+%d;k;type_str
PRP k + type, test_str
IF !(ISPRIME) THEN GOTO MOB

LABEL Do_n
SET n, 0

LABEL next_n
SET n, n + 1
SETS test_str,%d*%d^%d%s1;k;base;n;type_str
PRP k * base ^ n + type, test_str
 
IF (ISPRIME)   THEN GOTO Prime_found
IF (n < max_n) THEN GOTO next_n

SETS tmpstr,%d*%d^n%s1;k;base;type_str
WRITE k_file,tmpstr
GOTO next_k

LABEL GFN
SET gfn_k, gfn_k * base
SETS tmpstr,%d;k
WRITE g_file,tmpstr
GOTO next_k

LABEL MOB
SETS tmpstr,%d;k
WRITE m_file,tmpstr
GOTO next_k

LABEL Prime_found
SETS tmpstr,%d*%d^%d%s1;k;base;n;type_str
WRITE p_file,tmpstr
GOTO next_k

LABEL trivial_factor
SETS tmpstr,%d;k
WRITE t_file,tmpstr
GOTO next_k
 
LABEL END
END

[henryzz]

Originally Posted by rogue

Also note that script files cannot take advantage of fast modular reduction. This is another good reason to not test too a high value of n with this script.

I presume there isnt an easy fix for this as it could save some time on large conjecture bases.

[rogue]

Originally Posted by henryzz

I presume there isnt an easy fix for this as it could save some time on large conjecture bases.

I'm working on it.

[rogue]

Originally Posted by rogue

I'm working on it.

I should have a beta ready by tomorrow. There are a number of changes to script handling for the next release including:

PRP sets ISPRP (new) and ISPRIME, but only sets ISPRIME if the number is prime.

PRIMEP, PRIMEM, and PRIMEC will do primality tests (-tp, -tm, and -tc). They will set both ISPRP and ISPRIME if the number is prime.

PRP can now take a string expression as its sole argument. This prevents the decimal expansion and allows PFGW to take advantage of special modular reduction. PRIMEP, PRIMEM, and PRIMEC only take a single string argument.

Some of this is subject to change if I get too much push back from other script users.

[TimSorbet]

Copying this script here:

Originally Posted by Mini-Geek

Like this one:

#!/usr/bin/perl
#usage: file k a b
#e.g. 'mod 589 216 0 3' looks in 589.txt, removes values with k=216 and n=0 mod 3, and writes to 589-out.txt

open(IN, $ARGV[0].'.txt');
open(OUT, '>' .$ARGV[0].'-out.txt');
$k = $ARGV[1];
$a = $ARGV[2];
$b = $ARGV[3];
$removecount = 0;

while(<IN>)
{
    $line = $_;
    chomp($line);
    @linearray = split(/ /,$line);
    $thisk = @linearray[0];
    #if this isn't the right k, then we don't want to remove it, so print it out
    if ($thisk != $k) {
        print OUT $line."\n";
        next;
    }
    $thisn = @linearray[1];
    #if there is nothing after a space on this line (usually the first line of a NewPGen-format file), then we don't want to remove it, so print it out
    if ($thisn == '') {
        print OUT $line."\n";
        next;
    }
    $thisa = $thisn % $b;
    
    #if the modular value isn't right, then we don't want to remove it, so print it out
    if ($thisa != $a) {
        print OUT $line."\n";
        next;
    }
    
    #if we've got here, then everything matches, and we remove it by not doing anything, we also increment a counter so we can see how many were removed
    $removecount++;
}
print "removed $removecount line(s)";

close(IN);
close(OUT);

To use it, (say it's named mod.pl and the NewPGen-format file is 589.txt) run 'perl mod.pl 589 216 0 3', and it will write everything except k=216 values whose n=0 mod 3 to 589-out.txt. It's had only minimal testing, but I don't see any reason for it not to work as long as you use a NewPGen format file.

[gd_barnes]

Originally Posted by rogue

This is Gary's PFGW script for searching, with changes to output the number being tested as k*b^n+/-c instead of the decimal expansion. This is also output to pfgw.log and pfgw_prime.log in the same way. Note that PFGW will not verify that the string being output is the same as the number being tested.

I have added this enhancement in the lastest version of the script, which also corrects a previously mentioned exclusion of GFNs problem. See the "starting new bases" thread.

[gd_barnes]

Originally Posted by rogue

LABEL calc_mob
IF (k % base > 0) THEN GOTO Do_n
SETS test_str,%d+%d;k;type_str
PRP k + type, test_str
IF !(ISPRIME) THEN GOTO MOB

Mark,

The above code was causing a problem. The SETS code should be:
SETS test_str,%d%s1;k;type_str

That is with the "+" removed, "d" instead of "s" and a "1" at the end. I'm correcting it in the latest version of the script. With the correction, the output in pfgw-prime.log comes out as "12+1" and in pfgw.out as "12+1 factors prime!: 13". I assume that was your intent.

The funny thing is before when everything was displayed in decimal format, I didn't even realize that it was writing out the primes that pertain to MOB, which really isn't necessary.

Question: Is there a way to make it NOT write to those files for MOB? If not, it's not a big deal because the important thing is that these superflous primes don't show up in pl_prime.txt file. But it'd be nice to not have them in those places since the primes don't apply to the conjectures themselves.


Gary

[TimSorbet]

After rogue suggested it, I've added the ability to remove k's from pl_remain.txt along with removing them from the sieve file. Here is an updated version of my remove-ks.pl script: (it takes the same command line arguments, and assumes the availability of grep and sed in addition to srfile)

#!/usr/bin/perl
# usage:
# have pl_remain.txt present
# have a file with one k or prime per line
# have sr_b.abcd present, the srfile-readable sieve file to remove from
# have srfile, grep, and sed present
# run 'perl remove-ks.pl b t file' with b=the base and t=+ or - for Sierp or Riesel,
# e.g. 'perl remove-ks.pl 3 - pfgw.log' uses srfile to delete all k's with primes (found in pfgw.log) from sr_3.abcd and pl_remain.txt
# the input file can either have a single k per line, or a prime

$b = $ARGV[0];
$t = $ARGV[1];
open(IN, $ARGV[2]);

while(<IN>)
{
    # get k
    $line = $_;
    chomp($line);
    @linearray = split(/\*/,$line);
    
    # remove k from sieve file
    system("srfile -d \"@linearray[0]*$b\^n".$t."1\" -a sr_$b.abcd");
    
    # remove k from pl_remain.txt
    system("grep -v \"@linearray[0]\" pl_remain.txt > temp.txt");
    system("sed '1d' temp.txt > pl_remain.txt");
    system("del temp.txt");
}

close(IN);

If you only want it to do one or the other (sieve file or pl_remain.txt and not both), comment out the appropriate line(s) of code. They're clearly marked.

[TimSorbet]

I've changed it from using srfile and grep and doing it all one k at a time to building a list of k's, then running it all through egrep at once (I switched from grep to egrep since it had an option I needed: reading the search pattern from a file, and a useful one: the -h option to not print the file name). This makes it MUCH faster, which is significant for removing many k's at once. It also prints out some status information so you know what's going on.
It now asks for the prime file and sieve file instead of the base, type, and prime file. More instructions are in the comments at the top of the file.
Instead of wanting the sieve file to be ABCD format and in sr_{b}.abcd, it is expected to be in the PFGW/ABC/NewPGen format where the k is on each line, and the file to find it in is specified by the user.

#!/usr/bin/perl
# remove-ks.pl 3.0
# usage:
# have pl_remain.txt present
# have a file with one k or prime per line
# have a sieve file present, in a format that has the k on every line, e.g. PFGW/NewPGen's "k n", and not ABCD
# have egrep present
# run 'perl remove-ks.pl primefile sievefile'
# e.g. 'perl remove-ks.pl pfgw.log work.txt' uses srfile to delete all k's with primes (found in pfgw.log) from work.txt and pl_remain.txt
# the prime file can have, per line: a single k, (e.g. "1234") a prime, (e.g. "1234*3^2345-1") or a sequence (e.g. "1234*3^n-1");
# really anything, as long as it starts with the k, and if it continues, the k is followed by a *
# (e.g. "1234", "1234*3^2345-1", "1234*3^n-1", and "1234*JUNKTEXT" are all valid and will remove k=1234)

open(IN, $ARGV[0]);
$sievefile = $ARGV[1];
$ks = "";

system("echo getting list of k's...");

while(<IN>)
{
    # get ks
    $line = $_;
    chomp($line);
    @linearray = split(/\*/,$line);
    $ks .= @linearray[0] . '|';
}
# remove the final | character since we don't need or want it
chop($ks);

# write the list of k's, formatted as a regex search pattern, to a file
open(OUT, '>ks.txt');
print OUT $ks;
close(OUT);

system("echo done, removing k's from sieve file...");

# remove k's from sieve file
system("egrep -vhf ks.txt $sievefile > temp.txt");
system("del $sievefile");
system("ren temp.txt $sievefile");

system("echo done, removing k's from pl_remain.txt file...");

# remove k's from pl_remain.txt
system("egrep -vhf ks.txt pl_remain.txt > temp.txt");
system("del pl_remain.txt");
system("ren temp.txt pl_remain.txt");

# clean-up
system("del ks.txt");

system("echo done, exiting");

close(IN);

[rogue]

I had problems with your newest script. I also think that it might have problems WRT removing entries from the ABC file. For example if you have k=123 and the ABC file has n with n=%123%, then won't egrep also remove those lines?

What we really need is for srfile to be able to remove multiple sequences with a single run rather than having to run it once for each sequence.