November 19, 2020 - Words by Tadej Borovšak - 4 min read
This post was originally published on the Sensu Go blog.
Earlier this year, we shared the certified Ansible Collection for Sensu Go, which makes it easy to automate your monitoring and achieve real-time visibility into auto-scaling infrastructure. Now that Sensu Go 6 has been released, we will share the latest updates on the Collection, including the management aspects of Sensu Go 6, with a focus on the structure of Ansible playbooks in the Sensu Go 6 world.
We will start with a short overview of the Sensu Go 5 management process. Next, we will take a long, hard look at the management process and identify areas with room for improvement. And for the grand finale, we will show how Sensu Go 6 addressed those weaknesses and what this means for our Ansible playbooks.
Managing Sensu Go 5
When it comes to Sensu Go, there are at least three different things we need to manage:
- Sensu Go backends,
- Sensu Go agents, and
- Monitoring configuration.
For the sake of simplicity, we will assume that configuration for each component from the above list is in a separate playbook.
A typical Ansible playbook for backend management looks like this:
--- - name: Install, configure and run Sensu Go backend hosts: backends become: true tasks: - name: Install backend include_role: name: sensu.sensu_go.backend vars: version: 5.21.2
When we execute such a playbook, Ansible ensures that the sensu-backend is installed and properly configured on the backend host. Usually, we add a few other tasks to the Ansible playbook that configure the firewall at the minimum, but that kind of configuration is out of this post’s scope.
An Ansible playbook for agent management looks quite similar to the backend one. The configuration is different because we are configuring a different part of the observability pipeline. But other than that, things should feel familiar.
--- - name: Install, configure and run Sensu Go agent hosts: agents become: true tasks: - name: Install agent include_role: name: sensu.sensu_go.agent vars: version: 5.21.2 agent_config: name: my-agent backend-url: - ws://backend.host:8081 deregister: true keepalive-interval: 5 keepalive-timeout: 10 subscriptions: - linux
The last piece of the puzzle is the Ansible playbook that contains the Sensu Go monitoring configuration. This final Ansible playbook contains Sensu Go definitions of resources that we use in our monitoring pipeline (assets, checks, mutators, and handlers) and RBAC configuration (users, roles, and role bindings).
- name: Configure monitoring hosts: localhost tasks: - name: Create Sensu Go asset sensu.sensu_go.bonsai_asset: auth: &auth url: http://backend.host:8080 name: sensu/monitoring-plugins version: 2.2.0-1 - name: Create Sensu Go ntp check sensu.sensu_go.check: auth: *auth name: ntp runtime_assets: sensu/monitoring-plugins command: check_ntp_time -H time.nist.gov --warn 0.5 --critical 1.0 output_metric_format: nagios_perfdata publish: true interval: 30 timeout: 10 subscriptions: - linux
So far, things look nice and tidy. But something is not right. Did you spot the problem? Let us explain.
Separation of concerns
What do we need to do if we would like to reconfigure the backend? Well, we update the Ansible playbook that contains the backend’s configuration, run ansible-playbook, and take a sip of tea or coffee while Ansible reconfigures the backend. And the same process works if we need to reconfigure the agent.
But updating our observability pipeline in Sensu Go 5 is a bit more complicated. Why? Because in specific scenarios, we might need to update both the agent and the monitoring configuration playbooks. One such change is updating subscriptions. Reassigning checks to different subscriptions and updating the entity’s subscription list means we must update two Ansible playbooks.
The root of this problem is the agent’s configuration that contains two sets of configuration options. The first set includes configuration options for the agent process itself (for example, backend URL and TLS configuration). The second set contains configuration options for an entity representing the agent (e.g., annotations, labels, and subscriptions). Splitting those two sets would solve our two-Ansible-playbooks problem.
And guess what? Sensu Go 6 did just that!
Configuring Sensu Go 6
The problem we describe in the previous section made it quite clear that the entity-related configuration must move from the agent to the monitoring configuration. In the world of Ansible playbooks, this boils down to:
- Removing some configuration options from the agent Ansible playbook, and
- Adding removed options to the monitoring configuration playbook in the form of a new entity Ansible task.
Once we make those changes, our agent and monitoring configuration playbooks look like this:
--- - name: Install, configure and run Sensu Go agent hosts: agents become: true tasks: - name: Install agent include_role: name: sensu.sensu_go.agent vars: version: 5.21.2 agent_config: name: my-agent backend-url: - ws://backend.host:8081 keepalive-interval: 5 keepalive-timeout: 10 - name: Configure monitoring hosts: localhost tasks: - name: Add subscriptions to agent entity sensu.sensu_go.entity: auth: &auth url: http://:8080 name: my-agent entity_class: agent deregister: true subscriptions: - linux - name: Create Sensu Go asset sensu.sensu_go.bonsai_asset: auth: *auth name: sensu/monitoring-plugins version: 2.2.0-1 - name: Create Sensu Go ntp check sensu.sensu_go.check: auth: *auth name: ntp runtime_assets: sensu/monitoring-plugins command: check_ntp_time -H time.nist.gov --warn 0.5 --critical 1.0 output_metric_format: nagios_perfdata publish: true interval: 30 timeout: 10 subscriptions: - linux
Now we do not have to edit two playbooks at the same time to get something done!
We hope you will sleep better now that you know how to prepare your Ansible playbooks for Sensu Go 6. For more information, you can always visit the Sensu Go Ansible Collection documentation, or drop a question in Discourse, the Sensu Community Forum, or reach out to us.