the scripts thread

[Batalov]

Ah. The script needed Windows-proofing and usability testing.
It's the Windows-style line breaks that were confusing it.
Looks good! Thanks.

[rogue]

How does one use this script to remove candidates from an ABC file?

[Batalov]

Can't say anything about ABCD files, but for ABC I simply do
(e.g. if the hint script said "remove n= 1 mod 3") this:

head -1 myold.npg > mybetter.npg
tail +2 myold.npg | awk '$2%3!=1' >> mybetter.npg

I do that as early as possible, after the script is done, the "hiddenPowers pl_remain.txt" is run and initial srsieve runs are done. Some "mybetter.npg" (for individual k's) files will become empty after the above filtering.
...and then merge all npg files with sort +1n and awk into the ABC file...
Later .npg files can be sieved some more as needed, and re-merged, etc.

[Mini-Geek]

Originally Posted by rogue

How does one use this script to remove candidates from an ABC file?

I use mod.pl, which I made, not this hiddenPowers script (since it doesn't do it ).
Here it is:
http://www.mersenneforum.org/showthr...328#post199328
An example usage would be: (doing this manually, where lines starting with ">" show a command that was run)

>hiddenPowers.pl pl_remain.txt
64*177^n-1      n=0 mod 2 factors due to 8^2
64*177^n-1      n=0 mod 3 factors due to 4^3
>mod.pl 177-file 64 0 2
>mod.pl 177-file-out 64 0 3
>ren 177-file-out-out.txt the-result.txt

(the last step was, of course, not strictly necessary, but I put it to clarify that the final result was in "177-file-out-out.txt")
It might be worthwhile to merge hiddenPowers.pl and mod.pl, or make a modification of hiddenPowers.pl that optionally runs mod.pl. If I have some free time over the weekend (I should), and feel like it, (I probably will) I'll at least look in to this.

[Mini-Geek]

remove-ks.pl 3.2.1:

• improved help text
• made it show full help whenever something's wrong (instead of specific to the area) so that just running the script will give you all the info

#!/usr/bin/perl
# remove-ks.pl 3.2.1
# usage:
# have a file with one k or prime per line (k's to remove)
# have at least one of:
# a file of remaining k's present (e.g. pl_remain.txt)
# a sieve file present, in a format that has the k on every line, e.g. PFGW/NewPGen's "k n", and not ABCD
# 
# have egrep present
# run 'perl remove-ks.pl -p primefile -s sievefile -r remainingksfile' (-p mandatory, need at least one of -s and -r)
# e.g. 'perl remove-ks.pl -p pfgw.log -s work.txt -r pl_remain.txt' deletes all k's with primes (found in pfgw.log) from work.txt and pl_remain.txt
# or 'perl remove-ks.pl -p pfgw-prime.log -r pl_remain.txt' deletes all k's with primes (found in pfgw-prime.log) from pl_remain.txt
# the prime file can have, per line: a single k, (e.g. "1234") a prime, (e.g. "1234*3^2345-1") or a sequence (e.g. "1234*3^n-1");
# really anything, as long as it starts with the k, and if it continues, the k is followed by a *
# (e.g. "1234", "1234*3^2345-1", "1234*3^n-1", and "1234*JUNKTEXT" are all valid and will remove k=1234)

# changelog:
# 3.2.1: improved help text, made it show all info whenever something's wrong
# 3.2: now supports command line arguments for what files to work on; also helpfully errors out if you don't give it sufficient input
# 3.1.1: removed the "?" after the "[^0-9]" since it made it match some k's that shouldn't (e.g. the code for 123 would match 1234)
# 3.1: made the search pattern start on a new line and be followed by something besides a numeral to stop many possible false positives
# 3.0: added version number; rewrote to get all the k's first, then use egrep to remove them all from both files in one shot; results in a huge speed improvement
# 2.0: (unnumbered originally) uses grep to remove k's from pl_remain.txt along with using srfile to remove them from the sieve file
# 1.0: (unnumbered originally) first release, used srfile to delete k's from a sieve file

use Getopt::Std;
getopt('spr');

$help = "must supply a file with primes (k's to remove), -p filename.txt, and\nat least one file to remove from, -r pl_remain.txt or -s sievefile.txt\n\nUsage: remove-ks.pl -p PRIME_FILE [-r K_FILE] [-s SIEVE_FILE]\n       remove-ks.pl -p pfgw.log -r pl_remain.txt -s sieve.txt";

if (length($opt_p) < 1) { # if no prime file was specified
    print $help;
    exit;
}
if (length($opt_s) < 1) {
    $remsieve = 0;
} else {
    $sievefile = $opt_s;
    $remsieve = 1;
}
if (length($opt_r) < 1) {
    $remremain = 0;
} else {
    $remainfile = $opt_r;
    $remremain = 1;
}
if ((!$remremain) && (!$remsieve)) { # if neither sieve nor remain file was specified
    print $help;
    exit;
}
$kstxt = "temp-ks.txt";
if ($remainfile == $kstxt) { # just in case the list of remaining k's is at temp-ks.txt, change our temp list
    $kstxt = "temp-ks-alt.txt";
}


open(IN, $opt_p);

print "getting list of k's to remove from $opt_p...\n";

while(<IN>)
{
    # get ks
    $line = $_;
    chomp($line);
    @linearray = split(/\*/,$line);
    $ks .= '^' . @linearray[0] . '[^0-9]|';
}
# remove the final | character since we don't need or want it
chop($ks);

# write the list of k's, formatted as a regex search pattern, to a file
open(OUT, ">$kstxt");
print OUT $ks;
close(OUT);

if ($remremain) {
print "done, removing k's from $remainfile...\n";

# remove k's from remaining k's file
system("egrep -vhf $kstxt $remainfile > temp.txt");
system("del $remainfile");
system("ren temp.txt $remainfile");
}

if ($remsieve) {
print "done, removing k's from $sievefile...\n";

# remove k's from sieve file
system("egrep -vhf $kstxt $sievefile > temp.txt");
system("del $sievefile");
system("ren temp.txt $sievefile");
}

# clean-up
system("del $kstxt");

print "done, exiting\n";

close(IN);

[henryzz]

would it be possible for your script to accept as input the output from batalov's script?

[Mini-Geek]

Originally Posted by henryzz

would it be possible for your script to accept as input the output from batalov's script?

I'm not sure which of my scripts you mean, but...

Originally Posted by Mini-Geek

It might be worthwhile to merge hiddenPowers.pl and mod.pl, or make a modification of hiddenPowers.pl that optionally runs mod.pl. If I have some free time over the weekend (I should), and feel like it, (I probably will) I'll at least look in to this.

I've made it so hiddenPowers.pl will call mod.pl if you give it a sieve file to remove from. Both modified scripts are attached (mod.pl was only slightly modified, and is now improved for standalone use, and more easily compatible with use externally).
Don't give hiddenPowers.pl a sieve file without having mod.pl available. (it can still run alone as long as you only give it a pl_remain.txt file)

So all you have to do is run something like:
hiddenPowers.pl pl_remain.txt sievefile.txt

[henryzz]

Originally Posted by Mini-Geek

I'm not sure which of my scripts you mean, but...

I've made it so hiddenPowers.pl will call mod.pl if you give it a sieve file to remove from. Both modified scripts are attached (mod.pl was only slightly modified, and is now improved for standalone use, and more easily compatible with use externally).
Don't give hiddenPowers.pl a sieve file without having mod.pl available. (it can still run alone as long as you only give it a pl_remain.txt file)

So all you have to do is run something like:
hiddenPowers.pl pl_remain.txt sievefile.txt

Brilliant thats exactly what i was asking. Sorry to be unclear.

[Mini-Geek]

I was wondering if anybody had written a script that will copy certain results (e.g. all base 168) from a PRPnet server's completed_tests.log file, then I realized grep could trivially do this. But I like being able to just plug in a number, so I made this little extract-base.pl script:

$base = $ARGV[0];
system 'grep "\*'.$base.'\^"'." completed_tests.log > completed_tests_$base.log";

As you can see, it uses grep (so you'll need it available), must be run from the folder where completed_tests.log is, takes the base as its argument, and writes the output to completed_tests_$base.log).
Note that this looks for "*base^", so anything whose results don't appear in that form won't be picked up.
If grep is unavailable, this could be modified to use Perl's grep function.

[mdettweiler]

Originally Posted by Mini-Geek

I was wondering if anybody had written a script that will copy certain results (e.g. all base 168) from a PRPnet server's completed_tests.log file, then I realized grep could trivially do this. But I like being able to just plug in a number, so I made this little extract-base.pl script:

$base = $ARGV[0];
system 'grep "\*'.$base.'\^"'." completed_tests.log > completed_tests_$base.log";

As you can see, it uses grep (so you'll need it available), must be run from the folder where completed_tests.log is, takes the base as its argument, and writes the output to completed_tests_$base.log).
Note that this looks for "*base^", so anything whose results don't appear in that form won't be picked up.
If grep is unavailable, this could be modified to use Perl's grep function.

Actually, a while back I wrote a rather more generalized version of this script as part of my results-processing suite. What it does is split the number on each result line of an LLRnet-formatted results file into its constituent parts (k, b, n, c) so that only results matching the given criteria will be printed to the output file:

#!/usr/bin/perl
print "Enter name of input file: ";
$inputFile = <STDIN>;
chomp($inputFile);
print "Enter name of output file: ";
$outputFile = <STDIN>;
chomp($outputFile);
open(INFILE, $inputFile);
open(OUTFILE, '>', $outputFile);
$usernameLine = <INFILE>;
$datetimeLine = <INFILE>;
while(<INFILE>) {
 $justRead = $_;
 ($knpair, $foo) = split(/ is /, $justRead);
 ($k, $base_n_minus1) = split(/\*/, $knpair);
 ($base, $n_minus1) = split(/\^/, $base_n_minus1);
 ($n, $one) = split(/\-/, $n_minus1);
 if($n > 500000 and $n < 600001) {
  print OUTFILE $usernameLine; print OUTFILE $datetimeLine; print OUTFILE $justRead;
 }
 $usernameLine = <INFILE>;
 $datetimeLine = <INFILE>;
}
close(INFILE);
close(OUTFILE);

I've highlighted the criteria line in bold. I suppose if I wanted to make it extra robust I could have it prompt for the criteria at the beginning, but for my own purposes it's easy enough to just edit the script (right now it's set for a Sierp. base 9 range that I last used it for).

While this deals strictly in LLRnet-formatted results (I always convert PRPnet results to LLRnet format before processing so I can use the same scripts), it could be modified to manual LLR format by removing all such occurrences of these lines:
$usernameLine = <INFILE>;
$datetimeLine = <INFILE>;
And changing this line:
print OUTFILE $usernameLine; print OUTFILE $datetimeLine; print OUTFILE $justRead;
to this:
print OUTFILE $justRead;
I imagine it could be converted to PRPnet format without too much hassle.

[Mini-Geek]

Originally Posted by mdettweiler

Actually, a while back I wrote a rather more generalized version of this script as part of my results-processing suite. What it does is split the number on each result line of an LLRnet-formatted results file into its constituent parts (k, b, n, c) so that only results matching the given criteria will be printed to the output file:
...
I imagine it could be converted to PRPnet format without too much hassle.

Cool, cool. I don't think I'll be using this since right now I only need a simple task for PRPnet results and not a complex task for LLRnet-format results, but that's what this thread is for.

Originally Posted by mdettweiler

  print OUTFILE $usernameLine; print OUTFILE $datetimeLine; print  OUTFILE $justRead;

You can do the same thing, but with much less code and more readability, like this:

  print OUTFILE $usernameLine.$datetimeLine.$justRead;