Perl Reading of Called Process Output

[EdH]

First, it's entirely possible I'm misunderstanding the actual problem, but here's is my take:

I have more than one version of the Perl script that retrieves factordb composites and runs YAFU to factor them. None of my versions display the intermediate information that is normally shown by YAFU as it runs. To my limited knowledge base, it appears that Perl cannot process incoming text until a LF is received and YAFU uses a CR only so that subsequent lines overwrite instead of scrolling.

I was looking at the script and cannot fully understand how retrieved information is processed, but wondered if there might be a way to feed YAFU's output through to stdout without intermediate processing, possibly with getc?

The code I'm currently looking at includes the following section which runs YAFU. (I do not know who its author is, since I can't find it in the script.):

        open(YAFU, "./yafu \"factor($composite)\" |") or die "Couldn't start yafu to factor $composite $!";
        while (<YAFU>) {
                print "$_";
                chomp;
            if (/^[CP].*? = (\d+)/) {
              push( @results, $1 );
            print "*****\n";
          }
        }
        close(YAFU);

I could possibly write a C++ version of the script and make it work that way, but shouldn't Perl be able to do this?

Thanks!

[chalsall]

Originally Posted by EdH

I could possibly write a C++ version of the script and make it work that way, but shouldn't Perl be able to do this?

Perl can certainly do what you want. I've only dropped onto my console for a moment (to check on the weather), but binmode(FH) is probably what you need.

[EdH]

Originally Posted by chalsall

Perl can certainly do what you want. I've only dropped onto my console for a moment (to check on the weather), but binmode(FH) is probably what you need.

Thanks. I had considered reading the data as binary input vs. text, but, although I wrote some scripts in Perl many years ago, it all looks foreign ATM. I have to go back and figure out what everything in the above snippet actually does.

If I have the concept of what I want to do figured out correctly, I'll expect to invoke YAFU and look a binary stream being returned, catch a certain number of bits/bytes and send them for display, treating them as text. Will Perl let me do this directly or will I have to do some kind of type conversion?

Off to study this, in hopes of not being sidetracked too soon...

[chalsall]

Originally Posted by EdH

If I have the concept of what I want to do figured out correctly, I'll expect to invoke YAFU and look a binary stream being returned, catch a certain number of bits/bytes and send them for display, treating them as text. Will Perl let me do this directly or will I have to do some kind of type conversion?

Happy to try to help, but it would help to have a bit of context... What environment are you running in? Unix or Winblows? What version of Perl are you running?

If it is a simple matter of the lines overwriting each other in your above script, I might suggest you try changing your code to be:

        while (<YAFU>) {
            chomp;
            print "$_\n";
            if (/^[CP].*? = (\d+)/) {
              push( @results, $1 );
            print "*****\n";
          }
        }

...and see if that does what you want. If so, then you don't need to bother with binmode.

[yoyo]

I think the script is based on my script https://www.rechenkraft.net/wiki/Ben...:Yoyo/factordb

The while loop is line oriented, so it sticks to the end of line indication. If no EOL is comming it doesn't continue to the next loop.
If you don't want this line handling, you have to use binmode and read buffers of bytes. But in this case you will have problems with the regexp inside the loop which checks for C or P at the beginning of the line, because you don't have this line mode anymore.

[EdH]

Originally Posted by chalsall

Happy to try to help, but it would help to have a bit of context... What environment are you running in? Unix or Winblows? What version of Perl are you running?

If it is a simple matter of the lines overwriting each other in your above script, I might suggest you try changing your code to be:

        while (<YAFU>) {
            chomp;
            print "$_\n";
            if (/^[CP].*? = (\d+)/) {
              push( @results, $1 );
            print "*****\n";
          }
        }

...and see if that does what you want. If so, then you don't need to bother with binmode.

The script is running on Ubuntu and Debian systems mainly with a Fedora box thrown in on occasion. Perl is mostly, if not all, "perl 5, version 22, subversion 1 (v5.22.1) built for x86_64-linux-gnu-thread-multi." Your above code simply adds a padding line between the others.

Originally Posted by yoyo

I think the script is based on my script https://www.rechenkraft.net/wiki/Ben...:Yoyo/factordb

The while loop is line oriented, so it sticks to the end of line indication. If no EOL is comming it doesn't continue to the next loop.
If you don't want this line handling, you have to use binmode and read buffers of bytes. But in this case you will have problems with the regexp inside the loop which checks for C or P at the beginning of the line, because you don't have this line mode anymore.

Ah, yes, it probably is yours. I had remembered ali.pl was but not this one, which was not named yafu.pl anywhere. I must have renamed it for some reason. I had thought I'd copy/pasted from a thread, maybe prior to you putting it up on your page?

Anyway, thanks for clarifying what I had thought was the issue. Whether I rewrite to use binmode or not will probably be based on how much study I'll need, but sometimes I use these whims to force myself to learn something new about a programming language. And, I had noticed the catch for C and P, and that I would still need to address that. The YAFU code must be sending at least a CR in order to overwrite. Shouldn't I be able to catch any CRs to signal a check for C or P, using their ASCII codes?

[chalsall]

Originally Posted by EdH

Your above code simply adds a padding line between the others.

OK, one last thing I can suggest... If you are sure yafu is not buffering its output, and each line ends in a CR (instead of a CRLF) (which is a bit odd), try adding $/ = "\r"; to your script.

[EdH]

Originally Posted by chalsall

OK, one last thing I can suggest... If you are sure yafu is not buffering its output, and each line ends in a CR (instead of a CRLF) (which is a bit odd), try adding $/ = "\r"; to your script.

Thanks. I haven't tried this yet, but I plan to soon.

However, you have given me a thought:

If I modify YAFU to send CRLF, then I should be able to strip the LF in the perl code...

[EdH]

Originally Posted by chalsall

OK, one last thing I can suggest... If you are sure yafu is not buffering its output, and each line ends in a CR (instead of a CRLF) (which is a bit odd), try adding $/ = "\r"; to your script.

Tried it without success. YAFU is sending:

printf("\r");
fflush(stdout);

I can (and have) change(d) it to:

printf("\n");
fflush(stdout);

This does give me the expected stream of full lines being printed in yafu.pl. I've not however found a way to invoke printing the lines how I want in Perl. The CR appears to have transformed into an LF which is translated into a CRLF by Perl for printing. If I remove the LF, there is no signal to empty the buffer until it fills, even if I translate the LF back into a CR.

And, all of this is just to work with the ECM portion. I haven't even found out where to work in YAFU for the SIQS portion.

I'm beginning to think it might be easier to "exec" YAFU and harvest the factors from the log...