Ansible - convert yaml file to json

Using the Ansible filters from_yaml and to_json it is easy to construct a task to convert a yaml file to JSON.

Here is the Ansible playbook yaml_to_json.yml (in a real world example the names of the files would probably be parameterized).

---

- hosts: localhost

  tasks:
    - shell: cat my.yaml
      register: result

    - set_fact:
        myvar: "{{ result.stdout | from_yaml | to_json }}"

    - copy:
        content: "{{ myvar }}"
        dest:    my.json

Here is an example of a yaml input file which contains some special characters, lists, strings, numbers etc.

---

xx:
 - a:
 - b:
     A: aadf7{fdfd"öög
     B: [ 'asdfadsf*5%@@@"masdf' , 123456 ]

yy:
 n:
   C: 'AAdf7{fdfd"öög'
   D: [ 'ASdfadsf*5%@@@"masdf' , 789.56 ]
 m:

When run via ansible-playbook yaml_to_json.yml the JSON output file below (run via jq .) is generated.

{
  "yy": {
    "m": null,
    "n": {
      "C": "AAdf7{fdfd\"\\u00f6\\u00f6g",
      "D": [
        "ASdfadsf*5%@@@\"masdf",
        789.56
      ]
    }
  },
  "xx": [
    {
      "a": null
    },
    {
      "b": {
        "A": "aadf7{fdfd\"\\u00f6\\u00f6g",
        "B": [
          "asdfadsf*5%@@@\"masdf",
          123456
        ]
      }
    }
  ]
}

Note how lists and dictionaries and special characters are put into the resp. JSON format.

Also note that I am using the Ansible copy module. A shell: echo "{{ myvar }}" > my.json does not work since it does not take care of the correct quote and special characters subsitutions.

Different 'exit' behaviour between bash and Korn shell in command pipelines

When running command pipelines it is sometimes convenient to break out of a sub process and exit a shell script.

There are differences though between different types of shell and I want to show this behaviour and its consequences , in particular how common expectations might be met or not, and also suggest solutions.

Here is my example. It is a simple script: a command pipeline consisting of a printf (printing two lines) followed by a while, wrapped by a starting and closing echo

echo Before pipeline
printf "text1\ntext2\n" | while read line ; do
  # Do something useful with 'line'
  echo $line
done
echo After pipeline

and the output - with whichever shell - is

Before pipeline
text1
text2
After pipeline

Now let's modify this script and add an exit into the while loop so that it looks like this

echo Before pipeline
printf "text1\ntext2\n" | while read line ; do
  # Do something useful with 'line'
  echo $line
  exit 1
done
echo After pipeline

As artificial as this example might look you can simply assume that there are more complex things happening in the while loop and under certain conditions one might want to exit the loop.

Here are the results for bash and Korn shell ksh

bash #!/bin/bash	ksh #!/bin/ksh
Before pipeline text1 After pipeline	Before pipeline text1
Exit code: 0	Exit code: 1
Common behaviour: both shells leave the loop and neither prints the second line 'text2'
bash continues with the commands after the while loop (and exits with the result of the last 'echo' command)	ksh exits the whole script

While the behaviour of ksh seems more natural (exit means exit everything) the bash behaviour can be explained when considering that the while loop is a sub shell with its own scope as if it would be a separate shell script. Exiting the loop means to return to the parent. This also means that the parent script can capture this exit code and thus the solution is a check after the loop.

echo Before pipeline
printf "text1\ntext2\n" | while read line ; do
  # Do something useful with 'line'
  echo $line
  exit 1
done
[ $? -ne 0 ] && echo ERROR && exit 2
echo After pipeline

The result for bash: exit code 2 and the output

Before pipeline
text1
ERROR

The exit code check line is meaningless for ksh since it will never be reached.

I've seen plenty of bash scripts with similar constructs where the author coded a quick 'exit' line into the while loop but forgetting to check the result, probably assuming the ksh behaviour.