Wednesday, March 16, 2011

'tee' coprocess vs. 'tee' pipe in Korn shell

I had this piece of code which logged the output of a while loop to a file but also showed it on the screen:
while ...
do
...
done | tee logfile

A code change required to set a variable in the while loop and make it known after the loop had ended so I tried
while ...
do
   x=123
...
done | tee logfile
echo x=$x
but x was empty since in a piped sequence the while loop is considered to be a sub shell and thus it cannot set variables in the parent process.

There is a solution though in Korn shell using coprocesses.
# Start a coprocess to log input to logfile via tee 
# and report it also to the current tty
(tee logfile >/dev/tty)|&

# Send output of while loop to coprocess
while ...
do
   x=123
...
done >&p
echo x=$x
will report x correctly.

This works fine for my example where the script is run always in a terminal window.

If the script is run in the background or via cron its (terminal) output needs to be captured, anything else does not make sense, the script writer had an idea why things would be written to the terminal, surely not just for fun.

6 comments:

  1. Alternatively you could use process substitution:

    while ...
    do
    x=123
    ...
    done > >(tee logfile)
    echo x=$x

    This works correctly also when stdout happens to be not /dev/tty.

    ReplyDelete
    Replies
    1. This syntax does not work in my ksh
      done >>(tee logfile)

      Which shell is this (and which version)? And on which platform did you test this?

      Delete
    2. A bit more facts and research:

      I was testing this on Solaris 10 (I think originally I had encountered the issue on Solaris 9).
      The korn shell version is reported as
      Version M-11/16/88i
      which is basically ksh88

      Process substitution is supported as described and here is a simple example:
      cat /etc/hosts | tee >(wc)
      reports the file content and the counts in the last line

      #
      # Internet host table
      #
      ::1 localhost
      127.0.0.1 localhost
      10.0.2.15 unknown # Added by DHCP
      6 16 95

      But the syntax used in the anonymous comment does not work
      cat /etc/hosts > >(wc)
      ksh: syntax error: `(' unexpected

      Whereas both work fine when using 'bash' on Solaris 10, in particular the latter:
      cat /etc/hosts > >(wc)
      6 16 95

      First of all process substitution is not implemented in all shells and secondly it does not work equally in shells where it is implemented so another feature to be cautious about when using it, especially if one works in differing environments.

      Delete
  2. Big Data and Hadoop is an ecosystem of open source components that fundamentally changes the way enterprises store, process, and analyze data.
    hadoop training in bangalore

    ReplyDelete
  3. Harvard Business Review named data scientist the "sexiest job of the 21st century".This Data Science course will cover the whole data life cycle ranging from Data Acquisition and Data Storage using R-Hadoop concepts, Applying modelling through R programming using Machine learning algorithms and illustrate impeccable Data Visualization by leveraging on 'R' capabilities.With companies across industries striving to bring their research and analysis (R&A) departments up to speed, the demand for qualified data scientists is rising.
    data science training in bangalore

    ReplyDelete
  4. myTectra Amazon Web Services (AWS) certification training helps you to gain real time hands on experience on AWS. myTectra offers AWS training in Bangalore using classroom and AWS Online Training globally. AWS Training at myTectra delivered by the experienced professional who has atleast 4 years of relavent AWS experince and overall 8-15 years of IT experience. myTectra Offers AWS Training since 2013 and retained the positions of Top AWS Training Company in Bangalore and India.
    aws training in bangalore

    ReplyDelete