Ansible Semaphore Configuration Automation

[cf-sewe]

As we apply GitOps for everything, we want to automate Ansible Semaphore configuration via Ansible.

Here is our current simple approach, based around ansible.builtin.uri.

• The playbook is created for my use case, so it may not be great for more generic uses.
• I am happy to incorporate any improvement ideas.
  To keep it simple, for now I probably wont consider implementing a Galaxy advanced role with Python.
• Schedules will come once API does not implement GET for schedules #1028 is fixed.

---
# Semaphore configuration by Ansible
# The role will check for existance of the Semaphore configuration objects and create them if they do not exist.

- hosts: docker
  become: true
  vars:
    docker_semaphore_admin_password: "SECRET"
    docker_semaphore_repository_name: "example-repo"
    docker_semaphore_repository_git_url: "git@github.com:acme/example-repo.git"
    docker_semaphore_repository_git_branch: "main"
    docker_semaphore_tasks:
      - name: "letsencrypt"
        description: "Let's Encrypt certificate renewal"
        playbook: "playbooks/scheduled/letsencrypt.yml"
        arguments: '["-C"]'
    docker_semaphore_schedules:

  tasks:
    - name: "Determine Semaphore Container IP Address"
      block:
        - name: "Semaphore | Determine container IP address"
          become: true
          changed_when: false
          check_mode: false
          ansible.builtin.command:
            cmd: "/usr/bin/docker container inspect -f '{% raw %}{{ .NetworkSettings.Networks.traefik_proxy.IPAddress }}{% endraw %}' semaphore"
          register: semaphore_container_ip_output
        - name: "Semaphore - Set fact for Semaphore Container IP Address"
          ansible.builtin.set_fact:
            semaphore_container_ip: "{{ semaphore_container_ip_output.stdout }}"
        - name: "Debug"
          ansible.builtin.debug:
            var: semaphore_container_ip
    - name: "Semaphore | Perform Login"
      changed_when: false
      check_mode: false
      ansible.builtin.uri:
        use_proxy: false
        url: "http://{{ semaphore_container_ip }}:3000/api/auth/login"
        method: POST
        body:
          auth: "admin"
          password: "{{ docker_semaphore_admin_password }}"
        body_format: json
        status_code: 204
        timeout: 5
      register: semaphore_login

    - name: "Semaphore | Configure Project"
      block:
        - name: "Semaphore | Determine Projects"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/projects"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: semaphore_projects
        - name: "Semaphore | Create Project"
          changed_when: "semaphore_project_created.status == 201"
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/projects"
            method: POST
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            body:
              name: "{{ inventory_dir | basename }}"
            body_format: json
            status_code: 201
            timeout: 5
          register: semaphore_project_created
          when:
            - "(inventory_dir | basename) not in (semaphore_projects.json | map(attribute='name'))"
        - name: "Semaphore | Set fact semaphore_project_id (existing)"
          ansible.builtin.set_fact:
            semaphore_project_id: "{{ semaphore_projects.json | selectattr('name', 'equalto', inventory_dir | basename) | map(attribute='id') | first }}"
          when:
            - "(inventory_dir | basename) in (semaphore_projects.json | map(attribute='name'))"
        - name: "Semaphore | Set fact semaphore_project_id (created)"
          ansible.builtin.set_fact:
            semaphore_project_id: "{{ semaphore_project_created.json.id }}"
          when:
            - "(inventory_dir | basename) not in (semaphore_projects.json | map(attribute='name'))"

    - name: "Semaphore | Configure Keystores"
      block:
        - name: "Semaphore | Determine Keystores"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/keys"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: "semaphore_keystores"
        # SSH key for ansible user
        - name: "Semaphore | Read Ansible SSH key from system"
          delegate_to: localhost
          become: false
          no_log: true
          ansible.builtin.slurp:
            path: "/home/ansible/.ssh/id_rsa"
          register: "ssh_key_ansible"
        - name: "Semaphore | Create SSH key for ansible"
          changed_when: "semaphore_key_ansible_created.status == 204"
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/keys"
            method: POST
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            # project_id must be integer
            body: >-
              {
                "project_id": {{ semaphore_project_id | int }},
                "name": "ansible-ssh-key",
                "type": "ssh",
                "ssh":
                {
                  "private_key": {{ ssh_key_ansible.content | b64decode | to_json }}
                }
              }
            body_format: json
            status_code: 204
            timeout: 5
          register: semaphore_key_ansible_created
          when:
            - "semaphore_keystores.json | selectattr('name', 'equalto', 'ansible-ssh-key') | length == 0"

        # GitHub deploy key
        - name: "Semaphore | Read ssh key from system"
          delegate_to: localhost
          become: false
          no_log: true
          ansible.builtin.slurp:
            path: "/home/ansible/.ssh/ops-deploy-key-ro-20220119.key"
          register: "ssh_key_github"
        - name: "Semaphore | Create SSH key for GitHub"
          changed_when: "semaphore_key_github_created.status == 204"
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/keys"
            method: POST
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            # project_id must be integer
            body: >-
              {
                "project_id": {{ semaphore_project_id | int }},
                "name": "github-ssh-key",
                "type": "ssh",
                "ssh":
                {
                  "private_key": {{ ssh_key_github.content | b64decode | to_json }}
                }
              }
            body_format: json
            status_code: 204
            timeout: 5
          register: semaphore_key_github_created
          when:
            - "semaphore_keystores.json | selectattr('name', 'equalto', 'github-ssh-key') | length == 0"
        # TODO vault key, how to access it from runtime? from vault?
        - name: "Semaphore | Determine Keystores (after creation)"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/keys"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: "semaphore_keystores"
        - name: "Semaphore | Set fact semaphore_ansible_ssh_key_id"
          ansible.builtin.set_fact:
            semaphore_ansible_ssh_key_id: "{{ semaphore_keystores.json | selectattr('name', 'equalto', 'ansible-ssh-key') | map(attribute='id') | first }}"
        - name: "Semaphore | Set fact semaphore_github_ssh_key_id"
          ansible.builtin.set_fact:
            semaphore_github_ssh_key_id: "{{ semaphore_keystores.json | selectattr('name', 'equalto', 'github-ssh-key') | map(attribute='id') | first }}"

    - name: "Semaphore | Configure Repository"
      block:
        - name: "Semaphore | Determine Repositories"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/repositories"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: semaphore_repositories
        - name: "Semaphore | Create Repository"
          changed_when: "semaphore_repository_created.status == 204"
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/repositories"
            method: POST
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            # project_id must be integer
            body: >-
              {
                "project_id": {{ semaphore_project_id | int }},
                "name": "{{ docker_semaphore_repository_name }}",
                "git_url": "{{ docker_semaphore_repository_git_url }}",
                "git_branch": "{{ docker_semaphore_repository_git_branch }}",
                "ssh_key_id": {{ semaphore_github_ssh_key_id | int }}
              }
            body_format: json
            status_code: 204
            timeout: 5
          register: semaphore_repository_created
          when:
            - "semaphore_repositories.json | selectattr('name', 'equalto', docker_semaphore_repository_name) | length == 0"
        - name: "Semaphore | Determine Repositories (after creation)"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/repositories"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: "semaphore_repositories"
        - name: "Semaphore | Set facts semaphore_repository_id"
          ansible.builtin.set_fact:
            semaphore_repository_id: "{{ semaphore_repositories.json \
              | selectattr('name', 'equalto', docker_semaphore_repository_name) \
              | map(attribute='id') | first }}"

    - name: "Semaphore | Configure Environment"
      block:
        - name: "Semaphore | Determine Environments"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/environment"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: semaphore_environments
        - name: "Semaphore | Create Environment"
          changed_when: "semaphore_environment_created.status == 204"
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/environment"
            method: POST
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            # project_id must be integer
            body: >-
              {
                "project_id": {{ semaphore_project_id | int }},
                "name": "default",
                "json": "{}"
              }
            body_format: json
            status_code: 204
            timeout: 5
          register: semaphore_environment_created
          when:
            - "semaphore_environments.json | selectattr('name', 'equalto', 'default') | length == 0"
        - name: "Semaphore | Determine Environments (after creation)"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/environment"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: "semaphore_environments"
        - name: "Semaphore | Set facts semaphore_repository_id"
          ansible.builtin.set_fact:
            semaphore_environment_id: "{{ semaphore_environments.json \
              | selectattr('name', 'equalto', 'default') \
              | map(attribute='id') | first }}"

    - name: "Semaphore | Configure Inventory"
      block:
        - name: "Semaphore | Determine Inventories"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/inventory"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: semaphore_inventories
        - name: "Semaphore | Create Inventory"
          changed_when: "semaphore_environment_created.status == 201"
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/inventory"
            method: POST
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            # project_id must be integer
            body: >-
              {
                "project_id": {{ semaphore_project_id | int }},
                "name": "{{ inventory_dir | basename }}",
                "ssh_key_id": {{ semaphore_ansible_ssh_key_id }},
                "type": "file",
                "inventory": "inventories/{{ inventory_dir | basename }}/hosts"
              }
            body_format: json
            status_code: 201
            timeout: 5
          register: semaphore_inventory_created
          when:
            - "semaphore_inventories.json | selectattr('name', 'equalto', inventory_dir | basename) | length == 0"
        - name: "Semaphore | Set fact semaphore_inventory_id (existing)"
          ansible.builtin.set_fact:
            semaphore_inventory_id: "{{ semaphore_inventories.json | selectattr('name', 'equalto', inventory_dir | basename) | map(attribute='id') | first }}"
          when:
            - "semaphore_inventories.json | selectattr('name', 'equalto', inventory_dir | basename) | length > 0"
        - name: "Semaphore | Set fact semaphore_inventory_id (created)"
          ansible.builtin.set_fact:
            semaphore_inventory_id: "{{ semaphore_inventory_created.json.id }}"
          when:
            - "semaphore_inventories.json | selectattr('name', 'equalto', inventory_dir | basename) | length == 0"

    - name: "Semaphore | Configure Task Templates"
      block:
        - name: "Semaphore | Determine Task Templates"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/templates"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: semaphore_task_templates
        - name: "Semaphore | Create Task Templates"
          # changed_when: "semaphore_task_created.status == 201"
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/templates"
            method: POST
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            # prevent coercion, see https://stackoverflow.com/a/70990069
            body: >-
              {{ {
                "project_id": semaphore_project_id |int,
                "repository_id": semaphore_repository_id |int,
                "inventory_id": semaphore_inventory_id |int,
                "environment_id": semaphore_environment_id |int,
                "name": item.name |string,
                "description": item.description |string,
                "playbook": item.playbook |string,
                "arguments": item.arguments |string
              } }}
            body_format: json
            status_code: 201
            timeout: 5
          register: semaphore_task_created
          when:
            - "semaphore_task_templates.json | selectattr('name', 'equalto', item.name) | length == 0"
          loop_control:
            label: "Task {{ item.name }}"
          loop: "{{ docker_semaphore_tasks }}"

    - name: "Semaphore | Configure Schedules"
      block:
        - name: "Semaphore | Determine Schedules"
          changed_when: false
          check_mode: false
          ansible.builtin.uri:
            use_proxy: false
            url: "http://{{ semaphore_container_ip }}:3000/api/project/{{ semaphore_project_id }}/schedules"
            method: GET
            headers:
              Cookie: "{{ semaphore_login.cookies_string }}"
            status_code: 200
            timeout: 5
          register: semaphore_schedules
        - debug:
            var: semaphore_schedules

    - name: "Semaphore | Perform Logout"
      changed_when: false
      check_mode: false
      ansible.builtin.uri:
        use_proxy: false
        url: "http://{{ semaphore_container_ip }}:3000/api/auth/logout"
        method: POST
        headers:
          Cookie: "{{ semaphore_login.cookies_string }}"
        status_code: 204
        timeout: 5

[bbaassssiiee]

Great idea, but it seems the API changed a bit meanwhile. I created a POC in a repo that also installs Semaphore binary as systemd service, with Postgres SSL and NGinx.

You can run in with Vagrant, or on localhost. Or on your own inventory.

https://github.com/playingfield/controller