mersenneforum.org  

Go Back   mersenneforum.org > New To GIMPS? Start Here! > Homework Help

Reply
 
Thread Tools
Old 2018-04-20, 21:33   #1
petrw1
1976 Toyota Corona years forever!
 
petrw1's Avatar
 
"Wayne"
Nov 2006
Saskatchewan, Canada

23·569 Posts
Default Windows concurrent output to the screen and a file.

... First sorry; my current assignment must be done in Windows ...

I have a perl scripts (consisting of dozens of sub modules) that runs for many hours.
It produces thousands of lines of output. Some is critical for future analysis;
some is just there to let me know it is still working. i.e. xx,000 rows processed so far.

I want to be able to monitor it occasionally at the screen but also have the output saved to a file for future review.

I'm ok either running it as perl stuff.pl
OR stuff.bat that calls stuff.pl

I have come across several "TEE" solutions.
1. TEE.bat for batch files
2. TEE.pl for perl scripts
3. Tee_Object within PowerShell
4. Start_Transcript within PowerShell

NONE of the above give me continuous and immediate output to both the screen AND the file.

The first 3 above will output to the screen somewhere between every 100 lines or whenever a perl sub-module ends. Some will write to the file in the same intervals; some even less frequent.

Number 4. writes to the Screen immediately but nothing appears in the output file until the entire run ends and I do a Stop_Transcript (could be 18 hours later).

ANY BETTER SOLUTIONS?

Thanks
petrw1 is offline   Reply With Quote
Old 2018-04-20, 21:57   #2
ATH
Einyen
 
ATH's Avatar
 
Dec 2003
Denmark

1011110111012 Posts
Default

Try Wintee:

http://www.softpedia.com/get/System/...s/Wintee.shtml
https://code.google.com/archive/p/wintee/
ATH is offline   Reply With Quote
Old 2018-04-20, 23:04   #3
GP2
 
GP2's Avatar
 
Sep 2003

1010000101112 Posts
Default

Windows Subsystem for Linux (WSL). It works very well. Just use a bash shell.

Or if you use PowerShell, there is a Tee-Object command, but I haven't used it myself.
GP2 is offline   Reply With Quote
Old 2018-04-22, 15:28   #4
chris2be8
 
chris2be8's Avatar
 
Sep 2009

200310 Posts
Default

*If* you are willing to update the perl script you could have it write both to the screen and a log file. factMsieve.pl does it as follows:
Code:
BEGIN {
  # Define LOGOPEN in this block so logwrite and logclose
  # can access it, but no other code can access it.
  my $LOGOPEN=0;        # Flag to say if the log is open

  sub logwrite($) {
  ###########################################################
  # Write a record to the log and print it to STDOUT.
  ###########################################################
    unless ($LOGOPEN) {
      open(LF, ">>$LOGFILE") or die "can't open $LOGFILE $!";
      $LOGOPEN=1;
    }
    my $msg= $_[0];
    chomp $msg; # it may or may not have \n at the end.
    my $time = localtime time;
    print LF $time,' ',$msg,"\n";
    print $msg,"\n";
    return;
  }

  sub logclose {
  ###########################################################
  # Close the log.
  ###########################################################
    if ($LOGOPEN) {
      close(LF) or warn "Can't close LF $!";
      $LOGOPEN=0;
    }
  }
}
Called as follows:
Code:
  logwrite("linecount read $count records from $_[0]");
But that is probably a lot of work updating the perl to call logwrite instead of print for everything you might want to reference later.

Chris
chris2be8 is offline   Reply With Quote
Old 2018-04-23, 18:30   #5
petrw1
1976 Toyota Corona years forever!
 
petrw1's Avatar
 
"Wayne"
Nov 2006
Saskatchewan, Canada

455210 Posts
Thumbs up

Quote:
Originally Posted by chris2be8 View Post
*If* you are willing to update the perl script you could have it write both to the screen and a log file. ...
Chris
It sort of does this already; my "write to log" module has a
PRINT "Text"
PRINT OUTPUT "Text"

However, that is a catch:

There are several cases where the output comes from statements that run a SQL file or a SQL-Server utility which outputs directly to the console and optionally to a log file too. I do NOT see an "append" option though so I can't have these write to the same main log.

P.S. Thanks to everyone who took the time to respond.
petrw1 is offline   Reply With Quote
Old 2018-04-23, 19:10   #6
xilman
Bamboozled!
 
xilman's Avatar
 
"π’‰Ίπ’ŒŒπ’‡·π’†·π’€­"
May 2003
Down not across

101001010110112 Posts
Default

Quote:
Originally Posted by petrw1 View Post
It sort of does this already; my "write to log" module has a
PRINT "Text"
PRINT OUTPUT "Text"

However, that is a catch:

There are several cases where the output comes from statements that run a SQL file or a SQL-Server utility which outputs directly to the console and optionally to a log file too. I do NOT see an "append" option though so I can't have these write to the same main log.

P.S. Thanks to everyone who took the time to respond.
I'm not sure I understand. Perl certainly can open an output stream in append mode. Something along the lines of
Code:
open (OUTPUT, " >> /path/to/log_file")
xilman is offline   Reply With Quote
Old 2018-04-23, 19:33   #7
petrw1
1976 Toyota Corona years forever!
 
petrw1's Avatar
 
"Wayne"
Nov 2006
Saskatchewan, Canada

11C816 Posts
Default

Quote:
Originally Posted by xilman View Post
I'm not sure I understand. Perl certainly can open an output stream in append mode. Something along the lines of
Code:
open (OUTPUT, " >> /path/to/log_file")
Yes perl does this just fine....

It's the "-o Updates.log" below that is the issue:
$result = system("sqlcmd -d $DB -e -i Updates.sql -o Updates.log");

It does not have an "append" option.

And I don't suspect perl could then copy it to the main log while this perl script itself has it open for OUTPUT.
petrw1 is offline   Reply With Quote
Old 2018-04-24, 06:01   #8
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

23·19·61 Posts
Default

Can't you print to a temp file, or string, and then use perl to append the file/string to the log? (or dos command "copy /b logfile+file", or so)

Last fiddled with by LaurV on 2018-04-24 at 06:03
LaurV is offline   Reply With Quote
Old 2018-04-24, 06:08   #9
BudgieJane
 
BudgieJane's Avatar
 
"Jane Sullivan"
Jan 2011
Beckenham, UK

3608 Posts
Default

Many moons ago I copied a bunch of unix-type .exe files into a subdirectory C:\Users\Jane\Documents\unix. It was then easy to keep an eye on what is happening by means of

Code:
C:\Users\Jane\Documents\unix\tail -f logfile
BudgieJane is offline   Reply With Quote
Old 2018-04-24, 07:35   #10
M344587487
 
M344587487's Avatar
 
"Composite as Heck"
Oct 2017

761 Posts
Default

Quote:
Originally Posted by BudgieJane View Post
Many moons ago I copied a bunch of unix-type .exe files into a subdirectory C:\Users\Jane\Documents\unix. It was then easy to keep an eye on what is happening by means of

Code:
C:\Users\Jane\Documents\unix\tail -f logfile
That's fine for log files that are updated quickly and purged, but if fwrite() or equivalent is buffered it isn't always real-time, tail can't see the data until it is written to disk. petrw1 also has a setup where not everything he wants saved is written to the log file. I believe tee or the windows equivalent is the optimal solution, but if you are also dealing with stderr it looks a little more complicated.
M344587487 is offline   Reply With Quote
Old 2018-04-25, 15:34   #11
chris2be8
 
chris2be8's Avatar
 
Sep 2009

2,003 Posts
Default

Quote:
Originally Posted by petrw1 View Post
Yes perl does this just fine....

It's the "-o Updates.log" below that is the issue:
$result = system("sqlcmd -d $DB -e -i Updates.sql -o Updates.log");

It does not have an "append" option.

And I don't suspect perl could then copy it to the main log while this perl script itself has it open for OUTPUT.
Could you put something like this after the above call (or call a subroutine to do this if there are several calls):
Code:
open FROM, "<Updates.log" or die "Can't open Updates.log $!";
while(<FROM>) {
  print OUTPUT $_;
}
close FROM or warn "Can't close FROM $!";
NB. Untested code!

Chris
chris2be8 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
console output to txt file pepi37 Linux 23 2015-09-28 18:23
many monitor output leads to one ouput screen wildrabbitt Hardware 5 2015-05-12 16:20
Additional File Output for Visualization Project conere10 Software 7 2011-10-15 23:05
prp output file jasong Sierpinski/Riesel Base 5 1 2005-03-13 05:41
Iterations between results file output? edorajh Software 8 2003-11-20 02:28

All times are UTC. The time now is 22:29.

Sun Feb 28 22:29:23 UTC 2021 up 87 days, 18:40, 0 users, load averages: 2.21, 1.60, 1.49

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.