Spotter Custom Policies 104: Accelerating Policy Development for Spotter with ChatGPT

In the evolving world of DevOps, ensuring Ansible playbooks adhere to security best practices and internal compliance rules is crucial. Steampunk Spotter provides a robust platform for validating these playbooks, including the ability to enforce custom policies written in Rego, which Sara already utilised and created a workflow to detect Personal Identifiable Information leakage detection . But Sara, has now discovered an even more efficient way to craft these policies — by leveraging GenAI (for example ChatGPT, Gemini or DeepSeek and ect.) to accelerate policy development.

Custom Policies and Their Role in Spotter

Steampunk Spotter allows organizations to write custom policies using Rego, the policy language of Open Policy Agent (OPA). These policies ensure that playbooks conform to internal security standards, preventing misconfigurations and enforcing best practices, more on this in our documentation .

Before integrating AI into her workflow, Sarah had to:

• Manually write Rego policies based on documentation.

• Test and debug policies iteratively.

• Ensure traceability and clarity in policy violations.

• Manually document her policies.

Now, with AI acting as a Rego policy assistant, Sarah has reduced her policy-writing time significantly and improved the accuracy of her checks.

Step 0: Prerequisites – Providing Clear Instructions

Before Sarah can generate effective policies, she ensures that the selected AI understands how Spotter processes policies. She does this by:

• Providing Spotter’s policy structure documentation, including the required JSON format and evaluation criteria.

• Sharing examples of existing Rego policies that align with Spotter’s validation requirements.

• Defining custom policy rules, such as input data conventions and correlation ID requirements.

• Explicitly specifying how errors should be formatted and reported for compliance tracking.

Instructions provided in this blog post

In this blog post, we simply copied the entire Custom Policies documentation available in our public library and provided it to GenAI. This ensured that it had complete context on how policies should be structured, validated, and applied within Steampunk Spotter.

General key information in the documentation:

• Package Declaration: How every policy begins with a package name, for example, package hello_world.

• Rule Structure: Each rule has a descriptive name, a return value (often named [result]), and a body where the policy logic is defined.

• Task Evaluation: Guidelines on accessing task arguments (e.g., input.tasks[_].task_args) and returning a result that includes a correlation ID (task.task_id) for traceability.

• Custom Messages and Error Reporting: Examples show how to use sprintf to construct input-specific messages, ensuring that violations are clearly communicated.

By preparing these detailed instructions, Sarah ensures that AI generates policies that seamlessly integrate with Spotter, minimizing rework and improving accuracy.

Step 1: Defining Policy Requirements

Sarah starts by outlining the compliance rules her playbooks must follow. For example, one of her latest requirements is ensuring that EC2 security groups do not allow unrestricted access to port 22.

Policy Requirements:

• Module to Check: amazon.aws.ec2_security_group

• Violation Condition: If from_port == 22 and cidr_ip == "0.0.0.0/0"

• Error Reporting: Provide a descriptive message with task.task_id for traceability.

Here is the example promt that Sarah used to generate a policy for her:

“Generate a Rego policy that checks if an Ansible task using amazon.aws.ec2_security_group has a rule where from_port is 22 and cidr_ip is 0.0.0.0/0. Include a correlation ID and an error message.”

She recived the following Rego policy:

package open_ssh_rule

OpenSshFromAnywhere[result] {
    task = input.tasks[_]
    sg_args = task.task_args["amazon.aws.ec2_security_group"]
    rule = sg_args.rules[_]

    rule.from_port == 22
    rule.cidr_ip == "0.0.0.0/0"

    result := {
        "check_type": "TASK",
        "correlation_id": task.task_id,
        "message": "Security group rule found that allows SSH from anywhere (port 22 / 0.0.0.0/0).",
    }
}

Step 2: Integrating and Testing the Policy

With GenAI providing a solid draft, Sarah integrates the policy into Spotter.

Import the custom policy:

spotter policies set policy.rego

More information on how to actualy manage and import policies in our first blog post of the series Spotter Custom Policies 101: A Beginner’s Guide to Tailored Automation Checks .

Check Example Playbook with Spotter:

spotter scan example_playbook.yml

- hosts: all
  connection: local
  gather_facts: false
  tasks:
    - name: example ec2 group
      amazon.aws.ec2_security_group:
        name: example
        description: an example EC2 group
        vpc_id: 12345
        rules:
          - proto: tcp
            from_port: 22
            to_port: 22
            cidr_ip: 0.0.0.0/0

Review Violations:

Spotter flags any playbook tasks violating the rule, ensuring misconfigurations are detected before deployment.

Playbook.yml:6:7: ERROR: [E2300] Security group rule found that allows SSH from anywhere (port 22 / 0.0.0.0/0).

Step 3: Refining Policies

After testing, Sarah refines her policies with AI-powered iterations:

• Clarify Error Messages: “Improve the message to specify alternative secure configurations.”

• Extend Policy Checks: “Ensure to_port is also 22 if from_port is 22.”

• Debug Policy Issues: “This rule isn’t triggering for some tasks. What might be wrong?”

By interacting with AI tools, Sarah iterates on her policies faster than ever, improving both clarity and effectiveness.

Step 4: Documenting and Sharing Policies

With AI, Sarah also automates documentation:

“Summarize this policy’s purpose, usage, and an example violation in a Markdown format. Include both a good and bad example to clearly illustrate correct and incorrect usage of the policy.”

Generated result:

# EC2 Security Group SSH Restriction Policy

## Purpose
This policy checks if an Ansible task is using the amazon.aws.ec2_security_group module to create or modify a security group rule where SSH (port 22) is allowed from anywhere (CIDR range 0.0.0.0/0). Allowing unrestricted SSH access can pose a security risk, so this policy helps identify such potentially unsafe configurations.

## Usage
- Upload to Spotter as a custom policy.
- Run `spotter scan` to detect violations.

## Example Violation

my_ansible_file.yml:10:3: ERROR: [E2303] Security group rule found that allows SSH from anywhere (port 22 / 0.0.0.0/0).

## Good Example
A compliant EC2 security group that restricts SSH access to a specific IP address:

- name: Create EC2 security group with restricted SSH access
  amazon.aws.ec2_security_group:
    name: my-secure-group
    rules:
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: 192.168.1.0/24

This configuration **passes** the policy check because it does not allow public access to port 22.

## Bad Example
An EC2 security group exposing port 22 to the public:

- name: Create EC2 security group with open SSH access
  amazon.aws.ec2_security_group:
    name: my-group
    rules:
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: 0.0.0.0/0

This configuration **fails** the policy check because it allows unrestricted public access to port 22.

By automating documentation, Sarah ensures better policy traceability and team adoption.

Conclusion

GenAI has transformed Sarah’s policy-writing workflow:

• Faster policy creation with AI-generated Rego templates.

• More accurate and flexible policies with iterative refinements.

• Streamlined testing and debugging by leveraging AI as a Rego assistant.

• Automated documentation for better transparency and collaboration.

With Steampunk Spotter seamlessly integrating these AI-powered policies, Sarah’s compliance process is now more efficient, accurate, and scalable than ever.

Secure Your Playbooks with Spotter Today!

Want to supercharge your Spotter policies like Sarah? Book a demo of Steampunk Spotter today!