Windows concurrent output to the screen and a file.

[petrw1]

... First sorry; my current assignment must be done in Windows ...

I have a perl scripts (consisting of dozens of sub modules) that runs for many hours.
It produces thousands of lines of output. Some is critical for future analysis;
some is just there to let me know it is still working. i.e. xx,000 rows processed so far.

I want to be able to monitor it occasionally at the screen but also have the output saved to a file for future review.

I'm ok either running it as perl stuff.pl
OR stuff.bat that calls stuff.pl

I have come across several "TEE" solutions.
1. TEE.bat for batch files
2. TEE.pl for perl scripts
3. Tee_Object within PowerShell
4. Start_Transcript within PowerShell

NONE of the above give me continuous and immediate output to both the screen AND the file.

The first 3 above will output to the screen somewhere between every 100 lines or whenever a perl sub-module ends. Some will write to the file in the same intervals; some even less frequent.

Number 4. writes to the Screen immediately but nothing appears in the output file until the entire run ends and I do a Stop_Transcript (could be 18 hours later).

ANY BETTER SOLUTIONS?

Thanks

[ATH]

Try Wintee:

http://www.softpedia.com/get/System/...s/Wintee.shtml
https://code.google.com/archive/p/wintee/

[GP2]

Windows Subsystem for Linux (WSL). It works very well. Just use a bash shell.

Or if you use PowerShell, there is a Tee-Object command, but I haven't used it myself.

[chris2be8]

*If* you are willing to update the perl script you could have it write both to the screen and a log file. factMsieve.pl does it as follows:

BEGIN {
  # Define LOGOPEN in this block so logwrite and logclose
  # can access it, but no other code can access it.
  my $LOGOPEN=0;        # Flag to say if the log is open

  sub logwrite($) {
  ###########################################################
  # Write a record to the log and print it to STDOUT.
  ###########################################################
    unless ($LOGOPEN) {
      open(LF, ">>$LOGFILE") or die "can't open $LOGFILE $!";
      $LOGOPEN=1;
    }
    my $msg= $_[0];
    chomp $msg; # it may or may not have \n at the end.
    my $time = localtime time;
    print LF $time,' ',$msg,"\n";
    print $msg,"\n";
    return;
  }

  sub logclose {
  ###########################################################
  # Close the log.
  ###########################################################
    if ($LOGOPEN) {
      close(LF) or warn "Can't close LF $!";
      $LOGOPEN=0;
    }
  }
}

Called as follows:

  logwrite("linecount read $count records from $_[0]");

But that is probably a lot of work updating the perl to call logwrite instead of print for everything you might want to reference later.

Chris

[petrw1]

Originally Posted by chris2be8

*If* you are willing to update the perl script you could have it write both to the screen and a log file. ...
Chris

It sort of does this already; my "write to log" module has a
PRINT "Text"
PRINT OUTPUT "Text"

However, that is a catch:

There are several cases where the output comes from statements that run a SQL file or a SQL-Server utility which outputs directly to the console and optionally to a log file too. I do NOT see an "append" option though so I can't have these write to the same main log.

P.S. Thanks to everyone who took the time to respond.

[xilman]

Originally Posted by petrw1

It sort of does this already; my "write to log" module has a
PRINT "Text"
PRINT OUTPUT "Text"

However, that is a catch:

There are several cases where the output comes from statements that run a SQL file or a SQL-Server utility which outputs directly to the console and optionally to a log file too. I do NOT see an "append" option though so I can't have these write to the same main log.

P.S. Thanks to everyone who took the time to respond.

I'm not sure I understand. Perl certainly can open an output stream in append mode. Something along the lines of

open (OUTPUT, " >> /path/to/log_file")

[petrw1]

Originally Posted by xilman

I'm not sure I understand. Perl certainly can open an output stream in append mode. Something along the lines of

open (OUTPUT, " >> /path/to/log_file")

Yes perl does this just fine....

It's the "-o Updates.log" below that is the issue:
$result = system("sqlcmd -d $DB -e -i Updates.sql -o Updates.log");

It does not have an "append" option.

And I don't suspect perl could then copy it to the main log while this perl script itself has it open for OUTPUT.

[LaurV]

Can't you print to a temp file, or string, and then use perl to append the file/string to the log? (or dos command "copy /b logfile+file", or so)

[BudgieJane]

Many moons ago I copied a bunch of unix-type .exe files into a subdirectory C:\Users\Jane\Documents\unix. It was then easy to keep an eye on what is happening by means of

C:\Users\Jane\Documents\unix\tail -f logfile

[M344587487]

Originally Posted by BudgieJane

Many moons ago I copied a bunch of unix-type .exe files into a subdirectory C:\Users\Jane\Documents\unix. It was then easy to keep an eye on what is happening by means of

C:\Users\Jane\Documents\unix\tail -f logfile

That's fine for log files that are updated quickly and purged, but if fwrite() or equivalent is buffered it isn't always real-time, tail can't see the data until it is written to disk. petrw1 also has a setup where not everything he wants saved is written to the log file. I believe tee or the windows equivalent is the optimal solution, but if you are also dealing with stderr it looks a little more complicated.

[chris2be8]

Originally Posted by petrw1

Yes perl does this just fine....

It's the "-o Updates.log" below that is the issue:
$result = system("sqlcmd -d $DB -e -i Updates.sql -o Updates.log");

It does not have an "append" option.

And I don't suspect perl could then copy it to the main log while this perl script itself has it open for OUTPUT.

Could you put something like this after the above call (or call a subroutine to do this if there are several calls):

open FROM, "<Updates.log" or die "Can't open Updates.log $!";
while(<FROM>) {
  print OUTPUT $_;
}
close FROM or warn "Can't close FROM $!";

NB. Untested code!

Chris