Ansible role argument specification

It has been a while since our last hardcore technical post. So we decided to put steamy cloud posts aside for a moment and get down and dirty with one of the new features of Ansible Core 2.11: argument specification for Ansible roles.

We will start with a short description of how we can parameterize Ansible roles and what problems we can expect when using them. Next, we will give a high-level overview of the argument specification for Ansible roles, describe its benefits to Ansible playbook and role authors, and go over a simple example.

And for those of you who will stick with us to the bitter end, we will even throw in a short story about an argument specification development.

Sounds like an (in)sane plan? Good ;)

Prerequisites

If you would like to follow along, you will need to install the ansible-core Python package. To prevent making a mess out of your system, you should install Ansible Core into a new virtual environment:

$ python3 -m venv venv
$ . venv/bin/activate
(venv) $ pip install ansible-core

You will also need to have the Sensu Go Ansible Collection installed:

(venv) $ ansible-galaxy collection install "sensu.sensu_go:<1.11"

With the installation out of the way, we can start talking about Ansible roles and their arguments.

Role arguments

Strictly speaking, Ansible roles do not have arguments. But we can parameterize them with variables. Typically, Ansible playbooks look something like this:

---
- name: Install Sensu components
  hosts: all
  become: true

  tasks:
    - name: Install backend
      include_role:
        name: sensu.sensu_go.install
      vars:
        components: [ sensu-go-agents ]

Of course, we do not have to set the variables on the task itself. Variable values can come from a wide variety of sources, but we defined them close to the include task for the sake of simplicity.

The most common way of documenting Ansible role variables today is to describe them in a readme document. Why? Because Ansible Galaxy knows how to display it. The Apache Cassandra Ansible role contains a great example .

With the introduction of Ansible Collections, things became a bit more complex. Since Ansible Collections can contain more than one Ansible role, it is almost impossible to place all that information into a single readme file without making it obscenely long. For the Sensu Go Ansible Collection , we solved this issue by publishing a dedicated documentation site . Each Sensu Go role has a dedicated section there.

Documenting Ansible role variables does give all required information to Ansible playbook authors. However, there is still one major problem: we rely on playbook authors not to make typing and copy-paste mistakes. And to make matters even worse, such typos usually manifest themselves relatively late - when Ansible tries to access the variable value deep inside the Ansible role.

For example, if we run our Ansible playbook from the start of this section, we will see output similar to this:

PLAY [Install Sensu components] ******************************************

TASK [Gathering Facts] ***************************************************
ok: [default]

TASK [Install backend] ***************************************************

TASK [sensu.sensu_go.install : Prepare package repositories] *************
included: /home/tadej/.../tasks/repositories.yml for default

TASK [sensu.sensu_go.install : Prepare package repositories] *************
included: /home/tadej/.../tasks/dnf/prepare.yml for default

TASK [sensu.sensu_go.install : Include distro-specific vars (CentOS)] ****
ok: [default]

TASK [sensu.sensu_go.install : Add yum repository] ***********************
changed: [default]

TASK [sensu.sensu_go.install : Add yum source repository] ****************
changed: [default]

TASK [sensu.sensu_go.install : Install selected packages] ****************
included: /home/tadej/.../tasks/packages.yml for default

TASK [sensu.sensu_go.install : Install selected components (Linux)] ******
included: /home/tadej/.../tasks/dnf/install.yml for default

TASK [sensu.sensu_go.install : Install component] ************************
failed: [default] (item=sensu-go-agents) => {
  "ansible_loop_var": "item",
  "changed": false,
  "failures": ["No package sensu-go-agents available."],
  "item": "sensu-go-agents",
  "msg": "Failed to install some of the specified packages",
  "rc": 1,
  "results": []
}

PLAY RECAP ***************************************************************
default                    : ok=8    changed=2    unreachable=0    failed=1
                                        skipped=0    rescued=0    ignored=0   

As you can see, Ansible executed quite a few tasks before one of them failed. And the error message is not especially helpful here because it depends on the module that failed. (Can you spot the error in our Ansible playbook?)

One way of making sure all required variables are set and contain acceptable values is to add an ansible.builtin.assert task at the start of each task file in the Ansible role. This way, Ansible can error-out at the beginning of the Ansible role run.

Adding an assert task at the start of the task file has one major downside: Ansible role maintainers must keep assertions in sync with the documentation. But if we could get rid of that information duplication, we’d’ be golden. This is precisely what argument specification allows us to do.

Argument specification

Argument specification is, at its essence, machine-executable Ansible role documentation:

1. It serves as a source from which ansible-doc can produce reference documentation for an Ansible role.
2. ansible-playbook uses it to validate variable values before the Ansible role gets executed.

If we now update our Sensu Go Ansible Collection to the latest stable version, ansible-doc will produce the following output:

(venv) $ ansible-galaxy collection install "sensu.sensu_go:>=1.11"
(venv) $ ansible-doc --type role --list
sensu.sensu_go.agent   configure    Configure Sensu Go agent
sensu.sensu_go.agent   start        Start Sensu Go agent
sensu.sensu_go.agent   main         Install, configure, and start Sensu Go
sensu.sensu_go.backend configure    Configure Sensu Go backend
sensu.sensu_go.backend start        Start Sensu Go backend
sensu.sensu_go.backend main         Install, configure, and start Sensu Go
sensu.sensu_go.install repositories Enable Sensu Go repos
sensu.sensu_go.install packages     Install selected Sensu Go packages
sensu.sensu_go.install main         Enable Sensu Go repos and install

Role names should look familiar to anyone who used the Sensu Go Ansible Collection before. But why are there three lines for each Sensu Go Ansible role, and what do the values in the second column mean?

All Sensu Go Ansible roles are modular. Each of them has three entry points that ansible-doc lists in the second column. And if you never heard of entry points: they are task files that Ansible playbook authors can import individually from an Ansible role.

Most Ansible roles only have one entry point called main that Ansible executes by default when we include an Ansible role. But Ansible playbook authors can also select a different entry point by setting the tasks_from parameter to a non-default value.

We can get the documentation for the main entry point that we are using in our sample Ansible playbook by running the following command:

(venv) $ ansible-doc --type role --entry-point main sensu.sensu_go.install
> SENSU.SENSU_GO.INSTALL
  (/home/tadej/.ansible/collections/ansible_collections/sensu/sensu_go)

ENTRY POINT: main - Enable Sensu Go repos and install selected packages

        The main entry point just combines the repositories and
        packages entry points.

OPTIONS (= is mandatory):

- build
        Package build to install.
        Can be any valid build string such as `8290' or a special
        value latest.
        If the `version' variable is set to latest, this variable is
        ignored and the latest available build is installed.
        [Default: latest]
        type: str

- channel
        Repository channel that serves as a source of packages.
        Visit the packagecloud site to find all available channels.
        [Default: stable]
        type: str

- components
        List of components to install.
        (Choices: sensu-go-backend, sensu-go-agent, sensu-go-
        cli)[Default: ['sensu-go-backend', 'sensu-go-agent', 'sensu-
        go-cli']]
        elements: str
        type: list

- version
        Package version to install.
        Can be any valid version string such as `6.2.5' or special
        value `latest'.
        [Default: latest]
        type: str

The output of the ansible-doc command more or less replicates the documentation available on our documentation site .

If we now re-run the ansible-playbook command from before, we will also get a different output:

PLAY [Install Sensu components] ******************************************

TASK [Gathering Facts] ***************************************************
ok: [default]

TASK [Install backend] ***************************************************

TASK [sensu.sensu_go.install : Validating arguments against arg ] ********
fatal: [default]: FAILED! => {
  "argument_errors": [
    "value of components must be one or more of: sensu-go-backend,
     sensu-go-agent, sensu-go-cli. Got no match for: sensu-go-agents"
  ],
  ...
}

PLAY RECAP ***************************************************************
default                    : ok=1    changed=0    unreachable=0    failed=1
                                        skipped=0    rescued=0    ignored=0   

Because the sensu.sensu_go.install Ansible role in the latest Sensu Go Ansible Collection version has an argument specification, Ansible automatically inserted a validation task before it started executing tasks from the main entry point. And the error message is clear now: we entered an invalid component name. Awesome!

And now that we know how to use the argument specification let us take a stab at writing a simple one ourselves.

Writing argument specification

The argument specification we will dissect here will only have the main entry point with a single components argument. So just enough to cover the use case we were playing through the post. And without any further ado, here it is in all of its glory:

argument_specs:
  main:
    short_description: Enable Sensu Go repos and install selected packages
    description:
      - The main entry point just combines the repositories and packages
             entry points.
    options:
      components:
        description:
          - List of components to install.
        type: list
        elements: str
        choices:
          - sensu-go-backend
          - sensu-go-agent
          - sensu-go-cli
        default:
          - sensu-go-backend
          - sensu-go-agent
          - sensu-go-cli

The first six lines should be pretty self-explanatory: they attach some human-readable information to the main entry point. Ansible adds this information to the generated documentation.

The remainder of the specification is where the documentation meets validation. In our case, the components variable must conform to the following rules:

1. It must be a list of strings (type: list and elements: str).
2. It can only contain sensu-go-backend, sensu-go-agent, and sensu-go-cli strings.
3. By default, the variable will hold all valid package names.

Once we place the argument specification into the meta/argument_specs.yml file, Ansible will automatically use it to validate variables before executing an Ansible role.

For more information on writing argument specifications, we can visit the official documentation that contains more details and some additional samples. Or browse through the roles directory in the GitHub repository for the Sensu Go Ansible Collection.

And now for the best part of today’s post: storytime!

A tale where CI saves the day

We added an argument specification to all roles in the Sensu Go Ansible Collection right after the Ansible Core 2.11 saw its first stable release. Things went suspiciously smoothly: it took us about an hour to transform our documentation into argument specifications.

When we tested the new functionality locally using Ansible Core 2.11.0, all checks were green. So we polished things a bit and created a pull request. Then all hell broke loose .

We run our integration tests against all supported Ansible versions. And it turned out that the introduction of argument specification broke backward compatibility with Ansible 2.9 and Ansible Base 2.10, which is a no-go for a certified collection.

HINT: Integration tests are a necessity for certified collections, but using tools like Steampunk Spotter, which scan Ansible content for errors, and integrating it with GitLab, GitHub, Jenkins, and other CI tools will help you detect misconfigurations and sneaky human errors that occur during the development cycle. Learn more about Spotter.

A quick chat with the core developers confirmed that this is indeed a bug in Ansible Core. Fortunately for us, Ansible core developers are fantastic, and they prepared a fix for this in no time at all. The bugfix did not make it to the stable release yet, so this is why we are using the prerelease version of Ansible Core in this post.

What did we learn

If everything went according to plan, you now know more about Ansible role parameterization and how argument specification helps you make your automation setup more robust. And if you did not know about entry points before, well, now you do ;)

We also learned that the quality of your Ansible collection depends heavily on the quality of your test suite and that we should try to test new features before the latest stable version of Ansible is out. If you want to learn more about testing Ansible collections, we recommend watching the Intro to testing Ansible Collections webinar .

Also, if you need help creating or upgrading your Ansible content, feel free to contact us . We will do our best to help.

Cheers!