Thursday, December 16, 2010

Find and -exec with backticks like `... {}`

In this article I'll explain the reason for the following rule.

Rule:
You can't use backticks in a find's exec.

First of all in order to have a working example I created a subdir with a couple of more subdirs in it and the following find statement lists all subdirs named 'dd'.

mkdir -p aa/dd aa/bb/dd aa/cc/dd
find aa -name dd
aa/bb/dd
aa/cc/dd
aa/dd

A slightly more complex use of find is something like

find aa -name dd -exec dirname {} \;
which lists all directories which contain a 'dd' subdir or file
aa
aa/bb
aa/cc

My naive knowledge of find led me to believe that an equivalent command is

find aa -name dd -exec echo `dirname {}` \;
Very wrong. Here is the result:
.
.
.
So obviously '{}' is not supplied to 'dirname' as expected and none of the directory names is printed,

A little googling in comp.unix.shell revealed that I hadn't understood how 'exec' is invoked.
You cannot use any shell-type substitution or redirection in the 'exec' clause since 'exec'
executes the 'exec()' system call.

A hacker's way of doing it is to invoke another shell in the 'exec' clause.
Since the shell does not know about the '{}' parameter it has to be supplied as a positional parameter in a rather tricky way:

find aa -name dd -exec /bin/sh -c 'echo `dirname {}`' \;
does not work.

find aa -name dd -exec /bin/sh -c 'echo `dirname $0`' {} \;
works as expected but note that '{}' is seen as argument $0.

And with csh instead of sh you'll have to change the arg number to $1:

find aa -name dd -exec /bin/csh -c 'echo `dirname $1`' {} \;

Originally this problem came up when I wanted to rename all subdirs of a certain name to something else so

find aa -name dd -exec mv {} `dirname {}`/zz \;
would have been nice and short but it doesn't work.
find aa -name dd -exec /bin/sh -c 'mv $0 `dirname $0`/zz' {} \;
does it now for me.

Solutions with 'xargs'

Here is an invocation with 'xargs' (note that '-i' provides individual args (as {}) rather than one long arg list):
find aa -name dd -exec dirname {} \; | xargs -i mv {}/dd {}/zz

The general problem is that I need a 'dirname' to get the parent dir and I need a 'mv' to rename the dir.
Putting all of this into the 'xargs' section also means that I have to invoke another shell e.g.

find aa -name dd | xargs -i /bin/sh -c 'mv $1 `dirname $1`/zz' {} {}
which is of similar complexity than the 'find-only' solution.

Simpler but one more in the pipeline:

find aa -name dd | xargs -i dirname {} | xargs -i mv {}/dd {}/zz

4 comments:

  1. Harvard Business Review named data scientist the "sexiest job of the 21st century".This Data Science course will cover the whole data life cycle ranging from Data Acquisition and Data Storage using R-Hadoop concepts, Applying modelling through R programming using Machine learning algorithms and illustrate impeccable Data Visualization by leveraging on 'R' capabilities.With companies across industries striving to bring their research and analysis (R&A) departments up to speed, the demand for qualified data scientists is rising.

    data science training in bangalore

    ReplyDelete
  2. myTectra Amazon Web Services (AWS) certification training helps you to gain real time hands on experience on AWS. myTectra offers AWS training in Bangalore using classroom and AWS Online Training globally. AWS Training at myTectra delivered by the experienced professional who has atleast 4 years of relavent AWS experince and overall 8-15 years of IT experience. myTectra Offers AWS Training since 2013 and retained the positions of Top AWS Training Company in Bangalore and India.

    aws training in bangalore

    ReplyDelete
  3. Big Data and Hadoop is an ecosystem of open source components that fundamentally changes the way enterprises store, process, and analyze data.

    hadoop training in bangalore

    ReplyDelete