mersenneforum.org Windows concurrent output to the screen and a file.
 Register FAQ Search Today's Posts Mark Forums Read

 2018-04-20, 21:33 #1 petrw1 1976 Toyota Corona years forever!     "Wayne" Nov 2006 Saskatchewan, Canada 41·113 Posts Windows concurrent output to the screen and a file. ... First sorry; my current assignment must be done in Windows ... I have a perl scripts (consisting of dozens of sub modules) that runs for many hours. It produces thousands of lines of output. Some is critical for future analysis; some is just there to let me know it is still working. i.e. xx,000 rows processed so far. I want to be able to monitor it occasionally at the screen but also have the output saved to a file for future review. I'm ok either running it as perl stuff.pl OR stuff.bat that calls stuff.pl I have come across several "TEE" solutions. 1. TEE.bat for batch files 2. TEE.pl for perl scripts 3. Tee_Object within PowerShell 4. Start_Transcript within PowerShell NONE of the above give me continuous and immediate output to both the screen AND the file. The first 3 above will output to the screen somewhere between every 100 lines or whenever a perl sub-module ends. Some will write to the file in the same intervals; some even less frequent. Number 4. writes to the Screen immediately but nothing appears in the output file until the entire run ends and I do a Stop_Transcript (could be 18 hours later). ANY BETTER SOLUTIONS? Thanks
 2018-04-20, 21:57 #2 ATH Einyen     Dec 2003 Denmark 32×347 Posts
 2018-04-20, 23:04 #3 GP2     Sep 2003 1010000111012 Posts Windows Subsystem for Linux (WSL). It works very well. Just use a bash shell. Or if you use PowerShell, there is a Tee-Object command, but I haven't used it myself.
 2018-04-22, 15:28 #4 chris2be8     Sep 2009 37708 Posts *If* you are willing to update the perl script you could have it write both to the screen and a log file. factMsieve.pl does it as follows: Code: BEGIN { # Define LOGOPEN in this block so logwrite and logclose # can access it, but no other code can access it. my $LOGOPEN=0; # Flag to say if the log is open sub logwrite($) { ########################################################### # Write a record to the log and print it to STDOUT. ########################################################### unless ($LOGOPEN) { open(LF, ">>$LOGFILE") or die "can't open $LOGFILE$!"; $LOGOPEN=1; } my$msg= $_[0]; chomp$msg; # it may or may not have \n at the end. my $time = localtime time; print LF$time,' ',$msg,"\n"; print$msg,"\n"; return; } sub logclose { ########################################################### # Close the log. ########################################################### if ($LOGOPEN) { close(LF) or warn "Can't close LF$!"; $LOGOPEN=0; } } } Called as follows: Code:  logwrite("linecount read$count records from $_[0]"); But that is probably a lot of work updating the perl to call logwrite instead of print for everything you might want to reference later. Chris 2018-04-23, 18:30 #5 petrw1 1976 Toyota Corona years forever! "Wayne" Nov 2006 Saskatchewan, Canada 41×113 Posts Quote:  Originally Posted by chris2be8 *If* you are willing to update the perl script you could have it write both to the screen and a log file. ... Chris It sort of does this already; my "write to log" module has a PRINT "Text" PRINT OUTPUT "Text" However, that is a catch: There are several cases where the output comes from statements that run a SQL file or a SQL-Server utility which outputs directly to the console and optionally to a log file too. I do NOT see an "append" option though so I can't have these write to the same main log. P.S. Thanks to everyone who took the time to respond. 2018-04-23, 19:10 #6 xilman Bamboozled! "𒉺𒌌𒇷𒆷𒀭" May 2003 Down not across 11·971 Posts Quote:  Originally Posted by petrw1 It sort of does this already; my "write to log" module has a PRINT "Text" PRINT OUTPUT "Text" However, that is a catch: There are several cases where the output comes from statements that run a SQL file or a SQL-Server utility which outputs directly to the console and optionally to a log file too. I do NOT see an "append" option though so I can't have these write to the same main log. P.S. Thanks to everyone who took the time to respond. I'm not sure I understand. Perl certainly can open an output stream in append mode. Something along the lines of Code: open (OUTPUT, " >> /path/to/log_file") 2018-04-23, 19:33 #7 petrw1 1976 Toyota Corona years forever! "Wayne" Nov 2006 Saskatchewan, Canada 41×113 Posts Quote:  Originally Posted by xilman I'm not sure I understand. Perl certainly can open an output stream in append mode. Something along the lines of Code: open (OUTPUT, " >> /path/to/log_file") Yes perl does this just fine.... It's the "-o Updates.log" below that is the issue:$result = system("sqlcmd -d $DB -e -i Updates.sql -o Updates.log"); It does not have an "append" option. And I don't suspect perl could then copy it to the main log while this perl script itself has it open for OUTPUT.  2018-04-24, 06:01 #8 LaurV Romulan Interpreter Jun 2011 Thailand 25·5·59 Posts Can't you print to a temp file, or string, and then use perl to append the file/string to the log? (or dos command "copy /b logfile+file", or so) Last fiddled with by LaurV on 2018-04-24 at 06:03  2018-04-24, 06:08 #9 BudgieJane "Jane Sullivan" Jan 2011 Beckenham, UK 251 Posts Many moons ago I copied a bunch of unix-type .exe files into a subdirectory C:\Users\Jane\Documents\unix. It was then easy to keep an eye on what is happening by means of Code: C:\Users\Jane\Documents\unix\tail -f logfile 2018-04-24, 07:35 #10 M344587487 "Composite as Heck" Oct 2017 11000110102 Posts Quote:  Originally Posted by BudgieJane Many moons ago I copied a bunch of unix-type .exe files into a subdirectory C:\Users\Jane\Documents\unix. It was then easy to keep an eye on what is happening by means of Code: C:\Users\Jane\Documents\unix\tail -f logfile That's fine for log files that are updated quickly and purged, but if fwrite() or equivalent is buffered it isn't always real-time, tail can't see the data until it is written to disk. petrw1 also has a setup where not everything he wants saved is written to the log file. I believe tee or the windows equivalent is the optimal solution, but if you are also dealing with stderr it looks a little more complicated. 2018-04-25, 15:34 #11 chris2be8 Sep 2009 204010 Posts Quote:  Originally Posted by petrw1 Yes perl does this just fine.... It's the "-o Updates.log" below that is the issue:$result = system("sqlcmd -d $DB -e -i Updates.sql -o Updates.log"); It does not have an "append" option. And I don't suspect perl could then copy it to the main log while this perl script itself has it open for OUTPUT. Could you put something like this after the above call (or call a subroutine to do this if there are several calls): Code: open FROM, "<Updates.log" or die "Can't open Updates.log$!";
while(<FROM>) {
print OUTPUT $_; } close FROM or warn "Can't close FROM$!";
NB. Untested code!

Chris

 Similar Threads Thread Thread Starter Forum Replies Last Post pepi37 Linux 23 2015-09-28 18:23 wildrabbitt Hardware 5 2015-05-12 16:20 conere10 Software 7 2011-10-15 23:05 jasong Sierpinski/Riesel Base 5 1 2005-03-13 05:41 edorajh Software 8 2003-11-20 02:28

All times are UTC. The time now is 07:17.

Sat May 8 07:17:28 UTC 2021 up 30 days, 1:58, 0 users, load averages: 4.90, 4.09, 3.87