mersenneforum.org  

Go Back   mersenneforum.org > Extra Stuff > Programming

Reply
 
Thread Tools
Old 2017-09-11, 01:04   #1
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

2·32·197 Posts
Default Perl Reading of Called Process Output

First, it's entirely possible I'm misunderstanding the actual problem, but here's is my take:

I have more than one version of the Perl script that retrieves factordb composites and runs YAFU to factor them. None of my versions display the intermediate information that is normally shown by YAFU as it runs. To my limited knowledge base, it appears that Perl cannot process incoming text until a LF is received and YAFU uses a CR only so that subsequent lines overwrite instead of scrolling.

I was looking at the script and cannot fully understand how retrieved information is processed, but wondered if there might be a way to feed YAFU's output through to stdout without intermediate processing, possibly with getc?

The code I'm currently looking at includes the following section which runs YAFU. (I do not know who its author is, since I can't find it in the script.):
Code:
        open(YAFU, "./yafu \"factor($composite)\" |") or die "Couldn't start yafu to factor $composite $!";
        while (<YAFU>) {
                print "$_";
                chomp;
            if (/^[CP].*? = (\d+)/) {
              push( @results, $1 );
            print "*****\n";
          }
        }
        close(YAFU);
I could possibly write a C++ version of the script and make it work that way, but shouldn't Perl be able to do this?

Thanks!
EdH is offline   Reply With Quote
Old 2017-09-11, 02:14   #2
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

9,421 Posts
Default

Quote:
Originally Posted by EdH View Post
I could possibly write a C++ version of the script and make it work that way, but shouldn't Perl be able to do this?
Perl can certainly do what you want. I've only dropped onto my console for a moment (to check on the weather), but binmode(FH) is probably what you need.
chalsall is offline   Reply With Quote
Old 2017-09-11, 02:51   #3
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

67328 Posts
Default

Quote:
Originally Posted by chalsall View Post
Perl can certainly do what you want. I've only dropped onto my console for a moment (to check on the weather), but binmode(FH) is probably what you need.
Thanks. I had considered reading the data as binary input vs. text, but, although I wrote some scripts in Perl many years ago, it all looks foreign ATM. I have to go back and figure out what everything in the above snippet actually does.

If I have the concept of what I want to do figured out correctly, I'll expect to invoke YAFU and look a binary stream being returned, catch a certain number of bits/bytes and send them for display, treating them as text. Will Perl let me do this directly or will I have to do some kind of type conversion?

Off to study this, in hopes of not being sidetracked too soon...
EdH is offline   Reply With Quote
Old 2017-09-11, 19:30   #4
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

100100110011012 Posts
Default

Quote:
Originally Posted by EdH View Post
If I have the concept of what I want to do figured out correctly, I'll expect to invoke YAFU and look a binary stream being returned, catch a certain number of bits/bytes and send them for display, treating them as text. Will Perl let me do this directly or will I have to do some kind of type conversion?
Happy to try to help, but it would help to have a bit of context... What environment are you running in? Unix or Winblows? What version of Perl are you running?

If it is a simple matter of the lines overwriting each other in your above script, I might suggest you try changing your code to be:
Code:
        while (<YAFU>) {
            chomp;
            print "$_\n";
            if (/^[CP].*? = (\d+)/) {
              push( @results, $1 );
            print "*****\n";
          }
        }
...and see if that does what you want. If so, then you don't need to bother with binmode.
chalsall is offline   Reply With Quote
Old 2017-09-11, 19:41   #5
yoyo
 
yoyo's Avatar
 
Oct 2006
Berlin, Germany

593 Posts
Default

I think the script is based on my script https://www.rechenkraft.net/wiki/Ben...:Yoyo/factordb

The while loop is line oriented, so it sticks to the end of line indication. If no EOL is comming it doesn't continue to the next loop.
If you don't want this line handling, you have to use binmode and read buffers of bytes. But in this case you will have problems with the regexp inside the loop which checks for C or P at the beginning of the line, because you don't have this line mode anymore.
yoyo is offline   Reply With Quote
Old 2017-09-11, 22:46   #6
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

354610 Posts
Default

Quote:
Originally Posted by chalsall View Post
Happy to try to help, but it would help to have a bit of context... What environment are you running in? Unix or Winblows? What version of Perl are you running?

If it is a simple matter of the lines overwriting each other in your above script, I might suggest you try changing your code to be:
Code:
        while (<YAFU>) {
            chomp;
            print "$_\n";
            if (/^[CP].*? = (\d+)/) {
              push( @results, $1 );
            print "*****\n";
          }
        }
...and see if that does what you want. If so, then you don't need to bother with binmode.
The script is running on Ubuntu and Debian systems mainly with a Fedora box thrown in on occasion. Perl is mostly, if not all, "perl 5, version 22, subversion 1 (v5.22.1) built for x86_64-linux-gnu-thread-multi." Your above code simply adds a padding line between the others.

Quote:
Originally Posted by yoyo View Post
I think the script is based on my script https://www.rechenkraft.net/wiki/Ben...:Yoyo/factordb

The while loop is line oriented, so it sticks to the end of line indication. If no EOL is comming it doesn't continue to the next loop.
If you don't want this line handling, you have to use binmode and read buffers of bytes. But in this case you will have problems with the regexp inside the loop which checks for C or P at the beginning of the line, because you don't have this line mode anymore.
Ah, yes, it probably is yours. I had remembered ali.pl was but not this one, which was not named yafu.pl anywhere. I must have renamed it for some reason. I had thought I'd copy/pasted from a thread, maybe prior to you putting it up on your page?

Anyway, thanks for clarifying what I had thought was the issue. Whether I rewrite to use binmode or not will probably be based on how much study I'll need, but sometimes I use these whims to force myself to learn something new about a programming language. And, I had noticed the catch for C and P, and that I would still need to address that. The YAFU code must be sending at least a CR in order to overwrite. Shouldn't I be able to catch any CRs to signal a check for C or P, using their ASCII codes?
EdH is offline   Reply With Quote
Old 2017-09-11, 23:39   #7
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

9,421 Posts
Default

Quote:
Originally Posted by EdH View Post
Your above code simply adds a padding line between the others.
OK, one last thing I can suggest... If you are sure yafu is not buffering its output, and each line ends in a CR (instead of a CRLF) (which is a bit odd), try adding $/ = "\r"; to your script.
chalsall is offline   Reply With Quote
Old 2017-09-12, 00:38   #8
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

2×32×197 Posts
Default

Quote:
Originally Posted by chalsall View Post
OK, one last thing I can suggest... If you are sure yafu is not buffering its output, and each line ends in a CR (instead of a CRLF) (which is a bit odd), try adding $/ = "\r"; to your script.
Thanks. I haven't tried this yet, but I plan to soon.

However, you have given me a thought:

If I modify YAFU to send CRLF, then I should be able to strip the LF in the perl code...
EdH is offline   Reply With Quote
Old 2017-09-12, 02:23   #9
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

67328 Posts
Default

Quote:
Originally Posted by chalsall View Post
OK, one last thing I can suggest... If you are sure yafu is not buffering its output, and each line ends in a CR (instead of a CRLF) (which is a bit odd), try adding $/ = "\r"; to your script.
Tried it without success. YAFU is sending:
Code:
printf("\r");
fflush(stdout);
I can (and have) change(d) it to:
Code:
printf("\n");
fflush(stdout);
This does give me the expected stream of full lines being printed in yafu.pl. I've not however found a way to invoke printing the lines how I want in Perl. The CR appears to have transformed into an LF which is translated into a CRLF by Perl for printing. If I remove the LF, there is no signal to empty the buffer until it fills, even if I translate the LF back into a CR.

And, all of this is just to work with the ECM portion. I haven't even found out where to work in YAFU for the SIQS portion.

I'm beginning to think it might be easier to "exec" YAFU and harvest the factors from the log...
EdH is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
HI needing help on installing Perl/ntheory Trejack Information & Answers 5 2016-04-17 03:02
Factoring polynomials in perl chris2be8 Programming 12 2015-08-26 16:23
Here is a fun little game for Katydids called "Hypergraphia".Children enjoy Kathegetes Miscellaneous Math 35 2014-04-30 21:18
they used to be called programs... chappy Lounge 15 2012-08-11 21:02
Perl help with bigint kuratkull Programming 28 2007-04-30 14:14

All times are UTC. The time now is 03:32.

Thu Jan 28 03:32:17 UTC 2021 up 55 days, 23:43, 1 user, load averages: 3.48, 3.39, 3.37

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.