Managing Sensu Go 6 using Ansible

This post was originally published on the Sensu Go blog .

Earlier this year, we shared the certified Ansible Collection for Sensu Go , which makes it easy to automate your monitoring and achieve real-time visibility into auto-scaling infrastructure. Now that Sensu Go 6 has been released , we will share the latest updates on the Collection, including the management aspects of Sensu Go 6, with a focus on the structure of Ansible playbooks in the Sensu Go 6 world.

We will start with a short overview of the Sensu Go 5 management process. Next, we will take a long, hard look at the management process and identify areas with room for improvement. And for the grand finale, we will show how Sensu Go 6 addressed those weaknesses and what this means for our Ansible playbooks.

Managing Sensu Go 5

When it comes to Sensu Go, there are at least three different things we need to manage:

1. Sensu Go backends,
2. Sensu Go agents, and
3. Monitoring configuration.

For the sake of simplicity, we will assume that configuration for each component from the above list is in a separate playbook.

A typical Ansible playbook for backend management looks like this:

---
- name: Install, configure and run Sensu Go backend
  hosts: backends
  become: true
  tasks:
    - name: Install backend
      include_role:
        name: sensu.sensu_go.backend
      vars:
        version: 5.21.2

When we execute such a playbook, Ansible ensures that the sensu-backend is installed and properly configured on the backend host. Usually, we add a few other tasks to the Ansible playbook that configure the firewall at the minimum, but that kind of configuration is out of this post’s scope.

An Ansible playbook for agent management looks quite similar to the backend one. The configuration is different because we are configuring a different part of the observability pipeline. But other than that, things should feel familiar.

---
- name: Install, configure and run Sensu Go agent
  hosts: agents
  become: true
  tasks:
    - name: Install agent
      include_role:
        name: sensu.sensu_go.agent
      vars:
        version: 5.21.2
        agent_config:
          name: my-agent
          backend-url:
            - ws://backend.host:8081
          deregister: true
          keepalive-interval: 5
          keepalive-timeout: 10
          subscriptions:
            - linux

The last piece of the puzzle is the Ansible playbook that contains the Sensu Go monitoring configuration. This final Ansible playbook contains Sensu Go definitions of resources that we use in our monitoring pipeline (assets, checks, mutators, and handlers) and RBAC configuration (users, roles, and role bindings).

- name: Configure monitoring
  hosts: localhost
  tasks:
    - name: Create Sensu Go asset
      sensu.sensu_go.bonsai_asset:
        auth: &auth
          url: http://backend.host:8080
        name: sensu/monitoring-plugins
        version: 2.2.0-1

    - name: Create Sensu Go ntp check
      sensu.sensu_go.check:
        auth: *auth
        name: ntp
        runtime_assets: sensu/monitoring-plugins
        command: check_ntp_time -H time.nist.gov --warn 0.5 --critical 1.0
        output_metric_format: nagios_perfdata
        publish: true
        interval: 30
        timeout: 10
        subscriptions:
          - linux

So far, things look nice and tidy. But something is not right. Did you spot the problem? Let us explain.

Separation of concerns

What do we need to do if we would like to reconfigure the backend? Well, we update the Ansible playbook that contains the backend’s configuration, run ansible-playbook, and take a sip of tea or coffee while Ansible reconfigures the backend. And the same process works if we need to reconfigure the agent.

But updating our observability pipeline in Sensu Go 5 is a bit more complicated. Why? Because in specific scenarios, we might need to update both the agent and the monitoring configuration playbooks. One such change is updating subscriptions. Reassigning checks to different subscriptions and updating the entity’s subscription list means we must update two Ansible playbooks.

The root of this problem is the agent’s configuration that contains two sets of configuration options. The first set includes configuration options for the agent process itself (for example, backend URL and TLS configuration). The second set contains configuration options for an entity representing the agent (e.g., annotations, labels, and subscriptions). Splitting those two sets would solve our two-Ansible-playbooks problem.

And guess what? Sensu Go 6 did just that!

Configuring Sensu Go 6

The problem we describe in the previous section made it quite clear that the entity-related configuration must move from the agent to the monitoring configuration. In the world of Ansible playbooks, this boils down to:

1. Removing some configuration options from the agent Ansible playbook, and
2. Adding removed options to the monitoring configuration playbook in the form of a new entity Ansible task.

Once we make those changes, our agent and monitoring configuration playbooks look like this:

---
- name: Install, configure and run Sensu Go agent
  hosts: agents
  become: true
  tasks:
    - name: Install agent
      include_role:
        name: sensu.sensu_go.agent
      vars:
        version: 5.21.2
        agent_config:
          name: my-agent
          backend-url:
            - ws://backend.host:8081
          keepalive-interval: 5
          keepalive-timeout: 10

- name: Configure monitoring
  hosts: localhost
  tasks:
    - name: Add subscriptions to agent entity
      sensu.sensu_go.entity:
        auth: &auth
          url: http://:8080
        name: my-agent
        entity_class: agent
        deregister: true
        subscriptions:
          - linux

    - name: Create Sensu Go asset
      sensu.sensu_go.bonsai_asset:
        auth: *auth
        name: sensu/monitoring-plugins
        version: 2.2.0-1
    
    - name: Create Sensu Go ntp check
      sensu.sensu_go.check:
        auth: *auth
        name: ntp
        runtime_assets: sensu/monitoring-plugins
        command: check_ntp_time -H time.nist.gov --warn 0.5 --critical 1.0
        output_metric_format: nagios_perfdata
        publish: true
        interval: 30
        timeout: 10
        subscriptions:
          - linux

Now we do not have to edit two playbooks at the same time to get something done!

Parting words

We hope you will sleep better now that you know how to prepare your Ansible playbooks for Sensu Go 6. For more information, you can always visit the Sensu Go Ansible Collection documentation, or drop a question in Discourse, the Sensu Community Forum , or reach out to us .