mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Programming (https://www.mersenneforum.org/forumdisplay.php?f=29)
-   -   Script for split file (https://www.mersenneforum.org/showthread.php?t=27787)

pepi37 2022-05-10 11:11

Script for split file
 
I need script that will "ask" me number of workers and then split input file equally into worktodo.txt file ( and if possible add "Worker line") ( in this case 3 workers)


[QUOTE]example

input.txt
PRP=1,2,3
PRP=1,2,4
PRP=1,2,5


output

[Worker #1]
PRP=1,2,3
[Worker #2]
PRP=1,2,4
[Worker #3]
PRP=1,2,5
[/QUOTE]


Thanks

paulunderwood 2022-05-10 11:17

[QUOTE=pepi37;605586]I need script that will "ask" me number of workers and then split input file equally into worktodo.txt file ( and if possible add "Worker line") ( in this case 3 workers)
[/QUOTE]

Windows or Linux? What is your preferred (scripting) language?

kruoli 2022-05-10 13:38

In [C]bash[/C]:
[CODE]if [ $# -ne 3 ];
then
echo 'Usage: bash splitter.sh [in file] [out file] [number of workers]';
exit -1;
fi;
if ls "$1."* 1> /dev/null 2>&1;
then
echo 'Temporary files could not be created.';
exit -2;
fi;
range=$(seq 1 $3);
if [ $? -ne 0 ];
then
echo 'The worker count was not numerical.';
exit -3;
fi;

for w in $range;
do
echo "[Worker #$w]" >> "$1.tmp";
done;
cat "$1" >> "$1.tmp";
split -n r/$3 "$1.tmp" "$1.s";
cat "$1.s"* > "$2";
rm "$1."*;[/CODE]

pepi37 2022-05-10 13:45

[QUOTE=paulunderwood;605587]Windows or Linux? What is your preferred (scripting) language?[/QUOTE]

Whatever I can run under Linux 😀
Language, everyone you like it. I will adopt
Thanks

Uncwilly 2022-05-10 13:54

What about this tool? [url]https://www.mersenne.ca/balance.php[/url]

pepi37 2022-05-10 14:37

[QUOTE=Uncwilly;605597]What about this tool? [url]https://www.mersenne.ca/balance.php[/url][/QUOTE]

At first sight look perfect!

pepi37 2022-05-10 22:07

[QUOTE=Uncwilly;605597]What about this tool? [URL]https://www.mersenne.ca/balance.php[/URL][/QUOTE]
But doesnot work on PRP tests...

pepi37 2022-05-10 22:22

[QUOTE=kruoli;605595]In [C]bash[/C]:
[CODE]if [ $# -ne 3 ];
then
echo 'Usage: bash splitter.sh [in file] [out file] [number of workers]';
exit -1;
fi;
if ls "$1."* 1> /dev/null 2>&1;
then
echo 'Temporary files could not be created.';
exit -2;
fi;
range=$(seq 1 $3);
if [ $? -ne 0 ];
then
echo 'The worker count was not numerical.';
exit -3;
fi;

for w in $range;
do
echo "[Worker #$w]" >> "$1.tmp";
done;
cat "$1" >> "$1.tmp";
split -n r/$3 "$1.tmp" "$1.s";
cat "$1.s"* > "$2";
rm "$1."*;[/CODE][/QUOTE]


Work perfect.
Now I will see can I adopt it to Win
Thanks!

Uncwilly 2022-05-10 22:27

1 Attachment(s)
[QUOTE=pepi37;605627]But doesnot work on PRP tests...[/QUOTE]
I just tried it with some made up PRP lines and it recognized them, sorted them, and attempted to balance them.

Batalov 2022-05-10 22:53

bash
[CODE]N=3
for i in `seq 1 $N`; do
echo '[Worker #'$i']'; awk 'a++%'$N'=='$i'-1' input.txt
done > output.txt [/CODE]

paulunderwood 2022-05-11 00:37

Here is a perl script. usage: [C]perl myscript.pl input_file worker_count > output_file[/C]

[CODE]open(fh,"<",$ARGV[0]);
$count=0;
while(<fh>){$count++;}
close(fh);

open(fh,"<",$ARGV[0]);
$per_worker=int($count/$ARGV[1]);
$extras=$count%$ARGV[1];
$worker_blurb="[Worker \#";
$worker=0;
while(<fh>){
$line=$_;
$count++;
if($count>$per_worker){
$count=0;
print("\n");
$worker++;
print($worker_blurb.$worker."]\n");
if($extras==0){$per_worker--;}
$extras--;
}
print($line);
}
close(fh);
[/CODE]


All times are UTC. The time now is 13:10.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.