Wednesday, December 15, 2010

Echo multiline variables with and without quotes

When working with multiline variables in a Bourne or Korn shell there are certain subtleties to using quotes which - if used careless - can give quite different results.
A multiline variable is a variable which contains two or more lines of text.
I will discuss for a given multiline variable $T the differences between
  • echo $T
  • echo "$T"
  • printf $T
  • printf "$T"
  • printf "%s" $T
  • printf "%s" "$T"
If you know which of these commands show the same result you can skip this page.

Example: assume your /etc/hosts looks like this
# Internet host table
#       localhost      comp-nis    foo.Bar.COM loghost   foo.Bar.COM-ce1 # DO NOT MODIFY
and your variable $T should contain all lines with the string 'host' in it ie. lines 2, 4 and 6.
% T=`grep host /etc/hosts`
echo $T displays all 3 lines in one line
% echo $T
# Internet host table localhost foo.Bar.COM loghost

echo "$T" displays all 3 lines as 3 lines ie. truly showing the results of the grep command
% echo "$T"
# Internet host table       localhost    foo.Bar.COM loghost

Since the use of echo is discouraged these days one needs to look at printf.
printf $T dislays only the first word in $T which happens to be just # in our case.
% printf $T

printf "$T" shows all 3 lines but without the final newline
% printf "$T"
# Internet host table       localhost    foo.Bar.COM loghost%

printf "%s" $T shows all 3 lines in one line by removing all white space (blanks and newlines) i.e. also without the final newline
% printf "%s" $T

printf "%s" "$T" shows all 3 lines but without the final newline (same output as printf "$T")
% printf "%s" "$T"
# Internet host table       localhost    foo.Bar.COM loghost%
So out of all versions only echo "$T" did what one would expect and could be replaced by printf "$T\n" or printf "%s\n" "$T" ie. printf with a final newline.

The difference becomes important when you feed the contents of a variable to some command like U=`echo $T | some_cmd ...`

Assume you want to replace 'host' by 'HOST' in our example.
U=`echo "$T" | sed 's/host/HOST/'` is the right solution: 3 changed lines in $U
% echo "$U"
# Internet HOST table       localHOST    foo.Bar.COM logHOST
U=`echo $T | sed 's/host/HOST/'` will produce a one line output and only the first occurance of 'host' will be replaced
% echo "$U"
# Internet HOST table localhost foo.Bar.COM loghost
Sometimes though transferring a multiline output into a single line is what is needed so the difference in behaviour can be used to one's advantage.
Note that in csh there doesn't seem to be a way to preserve the multiline nature in a variable (at least I don't know of any).


  1. Harvard Business Review named data scientist the "sexiest job of the 21st century".This Data Science course will cover the whole data life cycle ranging from Data Acquisition and Data Storage using R-Hadoop concepts, Applying modelling through R programming using Machine learning algorithms and illustrate impeccable Data Visualization by leveraging on 'R' capabilities.With companies across industries striving to bring their research and analysis (R&A) departments up to speed, the demand for qualified data scientists is rising.

    data science training in bangalore

  2. myTectra Amazon Web Services (AWS) certification training helps you to gain real time hands on experience on AWS. myTectra offers AWS training in Bangalore using classroom and AWS Online Training globally. AWS Training at myTectra delivered by the experienced professional who has atleast 4 years of relavent AWS experince and overall 8-15 years of IT experience. myTectra Offers AWS Training since 2013 and retained the positions of Top AWS Training Company in Bangalore and India.

    aws training in bangalore

  3. Big Data and Hadoop is an ecosystem of open source components that fundamentally changes the way enterprises store, process, and analyze data.

    hadoop training in bangalore