Wednesday, February 20, 2019

Using the 'timeout' command

In Linux there is timeout command (usually in /usr/bin/timeout) which is very handy when you really want to restrict possibly long running commands. Here I want to show a couple of invocations to explain its usage.

First of all: timeout exits with error code 124 if the timeout has been reached.
Of particular interest is the handling of exit codes if commands are failing.

I am playing with various invocations of a timeout setting of 3 seconds and a sleep command of 5 seconds and vice versa.

Output Exit code
The timeout threshold is reached.
timeout 3 sleep 5
124
timeout 3 sh -c "sleep 5; exit 1"
124
timeout 3 sh -c "echo something; sleep 5; exit 1"
something124Failure
but part of the commands ran successfully creating output.
timeout 3 sh -c " sleep 5; echo something; exit 1"
124
The timeout threshold is not reached.
timeout 5 sleep 3
0Success.
The command finished before the timeout threshold.
timeout 3 sh -c "exit 1"
1Failure.
The command finished before the timeout threshold but with an error.
timeout 3 sh -c "echo something; exit 1"
something1Failure.
The commands ran and finished before the timeout threshold but with an error.

Here is an example that shows how a loop gets executed partially.

timeout 3 sh -c "for i in 1 2 3; do sleep 1; echo i=\$i;  done"

i=1
i=2

and exit code 124
In this case one needs to carefully continue and understand possible consequences of a partially executed command chain and how to recover from timeout errors.

Error handling could be done this way:

case $? in
0)
  # all ok
  ;;
124)
  # this was a timeout
  ;;
*)
  # some other error induced by the executed command(s)
  ;;
esac

Maybe this usage is also helpful sometimes: capture only the timeout exit code.

timeout 3 sh -c "echo something; sleep 5" || { [ $? -eq 124 ] && echo TIMEOUT; }
echo $?

will show output of the executed command, the timeout exit code check and an exit code 0

something
TIMEOUT
0
Interesting to see what happens when the executed command fails. The exit code of the failed command is still available to subsequent checks.
timeout 3 sh -c "echo something; exit 1" || { [ $? -eq 124 ] && echo TIMEOUT; }
echo $?

will show output of the executed command(s) and the exit code of the command chain

something
1

Thursday, February 14, 2019

Ansible tags with 'include'

When using the include directive things do not quite work as I would expect when I also add a tag to the include section.

Note: this was tested with Ansible 2.5.2 and python 2.7.15

Scenario

I split the playbook of the previous example into two:
The tasks (tagged in the same fashion as before) have been moved to a new file include_this.yml:
---

- name: Debug No Tag
  debug:
    msg: "No tag"

- name: Debug Tag A
  debug:
    msg: "Tag A"
  tags:
    - tagA

- name: Debug Tag B
  debug:
    msg: "Tag B"
  tags:
    - tagB

- name: Debug Tag A and B
  debug:
    msg: "Tag A and B"
  tags:
    - tagA
    - tagB
The playbook playbook.yml has been reduced to two tasks:
  • one debug task
  • one include task
I am testing two versions: the second one has an additional tag for the include directive.

Version 1Version 2
- name: Debug Playbook
  hosts: localhost
  tasks:
    - name: Debug No Tag in Playbook
      debug:
        msg: "No tag in playbook"
    - name: Include
      include: include_this.yml
- name: Debug Playbook
  hosts: localhost
  tasks:
    - name: Debug No Tag in Playbook
      debug:
        msg: "No tag in playbook"
    - name: Include
      include: include_this.yml
      tags:
        - tagA

The playbook is run as

ansible-playbook [--tags tagA] [--skip-tags tagB] playbook.yml

Results for version 1

If called with --tags or --skip-tags version 1 delivers the same results as described in my previous post when the include file was part of the playbook.
The additional 5th task in playbook.yml is executed in case of --skip-tags tagB or when no arguments are supplied.
no args
TASK [Debug No Tag in Playbook] 
TASK [Debug No Tag] 
TASK [Debug Tag A] 
TASK [Debug Tag B]
TASK [Debug Tag A and B]
--tags tagA
TASK [Debug Tag A] 
TASK [Debug Tag A and B]
--tags tagA --skip-tags tagB
TASK [Debug Tag A] 
--skip-tags tagB
TASK [Debug No Tag in Playbook]
TASK [Debug No Tag] 
TASK [Debug Tag A] 
Now I am reversing tagA and tagB in the call which leads to the expected results.
--tags tagB
TASK [Debug Tag B] 
TASK [Debug Tag A and B]
--tags tagB --skip-tags tagA
TASK [Debug Tag B] 

Results for version 2

no args
TASK [Debug No Tag in Playbook] 
TASK [Debug No Tag] 
TASK [Debug Tag A] 
TASK [Debug Tag B]
TASK [Debug Tag A and B]
Same as version 1.
All five tasks are being executed for both version of the playbook. This works as expected. No restrictions of any sort apply.
--tags tagA
TASK [Debug No Tag] 
TASK [Debug Tag A] 
TASK [Debug Tag B]
TASK [Debug Tag A and B]
All four task of the included files are being executed. No filtering by tagA for the included file takes place. This was a complete suprise to me. It seems that the tagging of the include step supersedes somehow the tags of the included file.
--tags tagA --skip-tags tagB
TASK [Debug No Tag]
TASK [Debug Tag A] 
The negative filter is applied to the result before and leaves two tasks for execution.
--skip-tags tagB
TASK [Debug No Tag in Playbook]
TASK [Debug No Tag] 
TASK [Debug Tag A] 
Same as version 1.
The two tasks with tagB are skipped and the other three are executed.
--tags tagB
TASK [Debug Tag B]
TASK [Debug Tag A and B]
Same as version 1.
The two tasks with tagB are executed and the tag setting in the include directive is not taken into account.
--tags tagB --skip-tags tagA
... nothing ...
Now here is a surprise. The --skip-tags tagA has skipped the whole include file and no task is being executed at all.

An explanation using sets

Let's look at it from a set perspective.
--tags tagA--skip-tags tagA
These two results are complementary and - if joined - build the complete set.
TASK [Debug No Tag] 
TASK [Debug Tag A] 
TASK [Debug Tag B]
TASK [Debug Tag A and B]
TASK [Debug No Tag in Playbook] 
Here the same complimentary sets for tagB.
--tags tagB--skip-tags tagB
TASK [Debug Tag B]
TASK [Debug Tag A and B]
TASK [Debug No Tag in Playbook]
TASK [Debug No Tag] 
TASK [Debug Tag A]  
Now any invocation of --skip-tags leads to an intersection with the respective sets.
Example: --tags tagB --skip-tags tagA: the sets have no task in common and thus nothing will be executed.

Conclusion

My idea was that a tag for the include directive would determine whether the include happens or not. Obviously wrong. It seems that it only determines which other tasks in the playbook.yml are executed i.e. it prohibits the untagged task from being run. The include takes place in any case (see the examples when using tagB either in --tags or --skip-tags) but in a strange way since only other tags than the supplied one are applied. I find this hard to remember or explain so my personal rule of mutual tag exclusion will be:
  • tagged include section => no tags in the included file
  • tags in the included file => no tag for the include section