Groups within Ansible are no mystery to frequent users and are a linchpin to almost all playbooks. Groups are fairly flexible and allow for sorting hosts into meaningful categories, specifying group-specific variables, and even nested groups. Groups are typically managed by the inventory, whether static or dynamic, and are used to classify which hosts will be targetted for a given play. Let’s take a look at a basic inventory with a group:

zk01 ansible_host=10.0.0.30
zk02 ansible_host=10.0.0.31

[zookeeper]
zk01
zk02

A simple playbook, making use of our grouping above, could resemble:

- name: manage the zookeeper servers
  hosts: zookeeper
  roles:
    - zookeeper

Only the hosts in the zookeeper group will get the zookeeper role. Pretty simple stuff.

Let’s take this example a bit further. Zookeeper has two mandatory config options that need to be set: ZOOKEEPER_MYID (node’s ID in the cluster) and ZOOKEEPER_SERVERS (a comma-separated list of server IPs to connect to). At first glance, it may be tempting to simply define these in the inventory:

zk01 ansible_host=10.0.0.30
zk02 ansible_host=10.0.0.31

[zookeeper]
zk01 ZOOKEEPER_MYID=1 ZOOKEEPER_SERVERS=10.0.0.30,10.0.0.31
zk02 ZOOKEEPER_MYID=2 ZOOKEEPER_SERVERS=10.0.0.30,10.0.0.31

This may satisfy our use case, but doesn’t scale and get’s us into a data management problem. Instead, let’s try to derive this information at run time, and let’s start with populating the ZOOKEEPER_SERVERS variable:

- name: manage the zookeeper servers
  hosts: zookeeper
  pre_tasks:
    - name: create zookeeper_ips group
      add_host: >
        hostname={{hostvars[item]['ansible_default_ipv4']['address']}}
        groups=zookeeper_ips
      with_items: play_hosts|sort
      run_once: true

    - name: set ZOOKEEPER_SERVERS
      set_fact: ZOOKEEPER_SERVERS={{groups['zookeeper_ips']|join(',')}}
  roles:
    - zookeeper

What we’ve done here is created an ad-hoc group called zookeeper_ips and populated it with the IPs of all the hosts in the zookeeper group. A few things to note, we use play_hosts instead of groups['zookeeper'] to avoid adding failed/downed hosts, and we make use of the add_host module to determine the IPs of the servers. We also make use of the sort filter to help our play be idempotent. Without the sort, we run the risk of the string changing values without any real reason other than the order changed. This could be especially bad when a change in config causes a restart of a process.

We can’t use something like a with_items loop in the set_fact play because we need to create a list before we can use the join filter. This may also seem like excess work when we could just use {{ groups['zookeeper'] | join(',') }}. By default, the groups['zookeeper'] variable will just return the hostnames of the servers. In our case, we have a requirement to use the IPs and not hostnames, so that option is out.

Let’s now move onto setting the ZOOKEEPER_MYID variable:

- name: manage the zookeeper servers
  hosts: zookeeper
  pre_tasks:
    - name: create zookeeper_ips group
      add_host: >
        hostname={{hostvars[item]['ansible_default_ipv4']['address']}}
        groups=zookeeper_ips
      with_items: play_hosts|sort

    - name: set ZOOKEEPER_SERVERS
      set_fact: ZOOKEEPER_SERVERS={{groups['zookeeper_ips']|join(',')}}

    - name: set ZOOKEEPER_MYID
      set_fact: ZOOKEEPER_MYID={{ item.0 + 1 }}
      when: item.1 == ansible_default_ipv4.address
      with_indexed_items: groups['zookeeper_ips']
  roles:
    - zookeeper

Using the same set_fact module, we are able to determine, dynamically, a unique ID for the zookeeper instance to use. The task starts by looping through the zookeeper_ips group, adding an index. The index is stored as item.0, and the value (in this case the IP of the host) is item.1. A when conditional checks to make sure the IP in the loop matches the host’s target IP. If the IPs do match, we set ZOOKEEPER_MYID to the index + 1. The conditional enforces that the host with the IP in the current iteration gets the ID with the proper index, otherwise every host would get the ID equal to the length of the zookeeper_ips array.

As you can see, ad-hoc groups in ansible can help make your plays more dynamic, reducing the load played on variable/data management. At this point, we could add another zookeeper host and the cluster would be automatically configured to support the additional host. One caveat to this approach is you need to ensure your roles/tasks support a dynamically adjusting cluster. Say we added another host, zk00 with IP 10.0.0.29, we could get into a situation where when the play runs, zk00 get’s the ID of 1 and the others each get bumped. This could cause problems if the cluster requires a lock on a specific ID, in which case, it may be useful to either adjust the logic or set the ID in hostvars.

In the next post, we’ll take a look at utilizing ad-hoc groups for better cluster management.

If you have questions or comments, send us a tweet at @thisendout and if you are interested in learning ansible, check out our training course on Udemy, Mastering Ansible (use coupon code TEOBLOG15 for a discount)!