How to specify zoneID while creating a domain using boto_route53.present

[nbari]

Description of Issue/Question

When using multiple private zones using the same domain name, how to define what zoneID to use?

Setup

buildbot_DNS_record:
  boto_route53.present:
    - name: buildbot.mydomain.internal
    - value: 10.10.0.6
    - zone: mydomain.internal
    - ttl: 3600
    - record_type: A
    - profile: aws_vpc_profile

Steps to Reproduce Issue

Having the domain mydomain.internal created multiple times and assigned to multiple VPC, while using boto_route53.present the record is added to the first zone found. wondering if there is a way to specify a ZONE ID.

Versions Report

Salt Version:
           Salt: 2016.3.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.11 (default, Jun  5 2016, 01:18:51)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.4

System Versions:
           dist:   
        machine: amd64
        release: 10.3-STABLE
         system: FreeBSD
        version: Not Installed

[Ch3LL]

@nbari jsut for clarification purposes are you stating you have one domain with multiple hosted zone ids. I'm not seeing anywhere in the code where to specify hosted zone ids.

[nbari]

In aws, private zones can be created and they can use the same domain. In my case I had the need to modify/add/update some records on one of does private zones, but notice that changes where applied to other zone using the same domain, I think they were applied to the first private zone created with the same domain.

To exemplify, suppose I have this domain salt.internal as a private zone and is in use on 3 different VPC:

                type      Hosted ZONE ID
salt.internal   private   ABC1
salt.internal   private   ABC2
salt.internal   private   ABC3

In this scenario, how to add a record for example to the Hosted ZONE ID ABC3?

I think currently is not posible but would be nice to specify the ZONE ID for this cases.

[Lomexian]

I've not got a similar setup but while playing I did try using the identifier which I could only get to work if I used a zoneid so maybe that's what it's for? The documentation doesn't make it clear.

buildbot_DNS_record:
  boto_route53.present:
    - name: buildbot.mydomain.internal
    - identifier: zoneid

[nbari]

Can't make it work, this is what I get:

 ID: dns.test
    Function: boto_route53.present
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/local/lib/python2.7/site-packages/salt/state.py", line 1723, in call
                  **cdata['kwargs'])
                File "/usr/local/lib/python2.7/site-packages/salt/loader.py", line 1650, in wrapper
                  return f(*args, **kwargs)
                File "/usr/local/lib/python2.7/site-packages/salt/states/boto_route53.py", line 197, in present
                  if identifier and identifier != record['identifier']:
              KeyError: 'identifier'

And this is the yaml:

  dns.test:                                                                                                                                                                                                                                                                     
    boto_route53.present:                                                                                                                                                                                                                                                       
      - value: 172.168.0.4                                                                                                                                                                                                                                                    
      - zone: test                                                                                                                                                                                                                                                              
      - ttl: 86400                                                                                                                                                                                                                                                              
      - record_type: A                                                                                                                                                                                                                                                          
      - profile: my-profile                                                                                                                                                                                                                                                     
      - identifier: ZYABCDEF  

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[kjohnston-cashstar]

Is there any chance of reopening this? I also have a similar use case for multiple hosted zones with the same name and don't see a way to make it work with salt.

[stale[bot]]

Thank you for updating this issue. It is no longer marked as stale.

[Ch3LL]

@kjohnston-cashstar if you set the identifier do you get hte same keyerror?

[kjohnston-cashstar]

@Ch3LL thanks for getting back so quickly! I didn't actually try that because, based on https://docs.saltstack.com/en/latest/ref/states/all/salt.states.boto_route53.html, it doesn't sound like identifier is used to specify a hosted zone id in salt.states.boto_route53.present. The documentation says it is used for "The unique identifier to use for this record."

I ended up rolling my own solution with boto3. If identifier is intended to be used to specify a hosted zone id, maybe that doc should be updated to clarify?

[Ch3LL]

ahh gotchya looks like you are right. i'll approve this as a feature request, unless @saltstack/team-boto sees something i'm missing here?

[japtain-cack]

So it looks like boto_route53 was not updated to use boto3, instead there is a separate boto3_route53 state. I suppose this is for backwards compatibility. Idk why anyone would be using boto vs boto3 these days, but I've seen crazier things.

Anyhow, you need to be using boto3_route53.rr_present(). Read about it in the docs here:
https://docs.saltproject.io/en/latest/ref/states/all/salt.states.boto3_route53.html#salt.states.boto3_route53.rr_present

We have a lot of pillars, so the below code still uses pillar names based on the original boto_route53 state to make migrating to the newer state easier. All I had to do is add a zone_id to my existing pillars that were already structured for boto_route53.

My state file looks like this:

{%- set records  = salt['pillar.get']('aws:route53:records', []) %}

include:
  - apps.boto3

{%- for record in records %}
  {%- set host_name = record.get('name') or grains['id'].split('.')[0] %}

  {%- if record['enabled'] %}
  # Sometimes primary_ipv4 isn't set. Assuming order places primary in the first item of the list of IPs
    {%- set value = record.get('value') or salt['grains.get']('primary_ipv4') or salt['grains.get']('ipv4')[0] %}

r53_rr_update_{{ host_name }}_{{ value }}:
  boto3_route53.rr_present:
    - name: {{ host_name }}.{{ record['zone'] }}
    - HostedZoneId: {{ record['zone_id'] }}
    - PrivateZone: {{ record.get('private_zone', False) }}
    - Type: {{ record.get('record_type', 'A') | upper }}
    - ResourceRecords:
      - {{ value }}
    - TTL: {{ record.get('ttl', 60) }}
    - require:
      - pip: boto3_install

  {%- else %}
r53_rr_remove_{{ host_name }}_{{ value }}:
  boto3_route53.rr_absent:
    - name: {{ host_name }}.{{ record['zone'] }}
    - HostedZoneId: {{ record['zone_id'] }}
    - Type: {{ record.get('record_type', 'A') | upper }}
    - PrivateZone: {{ record.get('private_zone', False) }}
    - require:
      - pip: boto3_install

  {% endif -%}

{% endfor -%}

Pillar example:

aws:
  route53:
    records:
      # Ensure the zone has a trailing '.'.
    - zone: vpc.internal.
      enabled: True
      zone_id: ZAS35500V098V0T
      # (optional) value: defaults to host IP
      # (optional) name: defaults to hostName.zone
      # (optional) record_type: defaults to 'A'
      # (optional) ttl: defaults to '60'
      # (optional) private_zone: defaults to 'False'

Hopefully this will be helpful for anyone with this issue that didn't realize there was a completely separate state.

[japtain-cack]

I would also recommend adding a notice/link to boto3_route53 in the boto_route53 docs. Nobody should be using that and it isn't clearly mentioned that there are two route53 states. Almost every book/doc on the internet points to boto_route53 when mentioning how to set up salt to manage aws DNS, so I think adding a note/link in the docs would be meaningful.