AWS VPC - subnets, routers, sg's and acl's

Hi All,

Love Ansible.

I have few questions as I’m creating a VPC with 800 subnets:

  1. Does anyone know if there is a way to get list of routers without having to parse them with shell aws describe? Would I have to construct similar lists for ACL’s?
  2. When using vpc mod Ansible is trying to re-create specified VPC as a new one. Attempt to update existing VPC tries to delete it first (state: present). How can I update existing VPC?

Peter

So far I found those to help.

Split out route table and subnet functionality from VPC module. #322
https://github.com/ansible/ansible-modules-core/pull/322/files

Add initial module for managing AWS VPC network ACLs #403
https://github.com/ansible/ansible-modules-core/pull/403/files

Another question arose. Ansible will remove resources that are not in VPC script. But if I have 2 environment setups in the same VPC? I run the 1st env (one set of subnets). then run the second one (second set of subnets), Ansible thinks the subnets have changed and tries to remove subnets from the 1st env. Any way to overwrite this behavior?

Interesting use-case. Perhaps it makes sense to add a purge_subnets: False option to that module? I’m the author of the first PR you linked to (#322). Would you mind leaving a comment on that pull request so I can keep track?

Thanks,
Robert

Hi Robert,

I now spent about 24h with your module for isolated subnets. Tomorrow I continue with routers module.
Works great with with_items iterator and variables in separate envs file. I can just copy-paste my list of subnets and consume in the main script across AZ’s in the region. Its definitely way to go forward and I hope Ansible approves your request to merge.

Important but not urgent: ability to update resource_tags. Ansible detects changes locally but not pushing changes. Can you confirm this is working on your side?

I will also attempt to reference created subnets by Name further in the main script. I’m trying to isolate all my variables into environment files so can’t have subnet-id nor cidr references in main script. For example when I’m associating routers, acls or instances. Can you confirm reference (to subnet) by name is possible?

Many Thanks,
Peter

Unfortunately reference to subnet by name is not possible because subnets cannot currently be tagged in AWS.

Thanks for pointing that out about resource_tags. That’s definitely a bug: my code searches for an exact tag match, which precludes providing tags to update.

There are ways around this (selecting nearest match, allowing only superset updates of tags, etc.), but I think that at a fundamental level, conflating resource_tags for both search and update is a broken idea.

I’ll make my PR’s behaviour match the original (search for subset match) as I intended, but I think that functionality needs to be looked at more closely. Perhaps remove searching by resource_tags entirely, which would more closely mimic other modules. (e.g., you search for files based on name, not its attributes).

Robert

The tag update regression’s now fixed in the pull request and should behave the same as the original module: It will attempt to match VPCs whose tags are a subset of the provided tags, and will add new tags as necessary. It is not possible to update or remove existing tags; the only way to do that would be to remove search functionality or have separate parameters for matching/updating tags.

As for your subnet-id question, what I’ve been doing is something roughly like this, if that helps:

  • name: Ensure VPC exists
    local_action:
    module: ec2_vpc
    state: present
    cidr_block: 10.0.0.0/16
    resource_tags:
    Name: ‘{{vpc_env}} VPC’
    VPCEnv: ‘{{vpc_env}}’
    region: ‘{{ec2_region}}’
    register: vpc_res

  • name: Ensure Internet Gateway exists
    local_action:
    module: ec2_vpc_igw
    state: present
    vpc_id: ‘{{vpc_res.vpc_id}}’
    region: ‘{{vpc_res.vpc.region}}’
    register: ‘igw’

  • name: Jumpbox subnet
    local_action:
    module: ec2_vpc_subnet
    vpc_id: ‘{{vpc_res.vpc_id}}’
    region: ‘{{vpc_res.vpc.region}}’
    resource_tags:
    Name: Jumpbox Subnet
    cidr: 10.0.0.0/28
    state: present
    register: jumpbox_subnet

  • name: Public subnet route table
    local_action:
    module: ec2_vpc_route_table
    vpc_id: ‘{{vpc_res.vpc_id}}’
    region: ‘{{vpc_res.vpc.region}}’
    resource_tags:
    Name: Public
    subnets:

  • ‘{{jumpbox_subnet.subnet_id}}’
    routes:

  • dest: 0.0.0.0/0
    gateway_id: ‘{{igw.gateway_id}}’
    register: public_route_table

  • name: Ensure only desired subnets and route tables exist
    local_action:
    module: ec2_vpc
    vpc_id: ‘{{vpc_res.vpc_id}}’
    region: ‘{{vpc_res.vpc.region}}’
    subnet_ids:

  • ‘{{jumpbox_subnet.subnet_id}}’
    route_table_ids:

  • ‘{{public_route_table.route_table_id}}’

Argh. Sorry for the line noise, I was thinking of IGWs which cannot be tagged. Subnets can be. However they is no current support for reference-by-tag in these changes. Not sure if that’s better to add to this PR versus add as a later enhancement? I haven’t heard any input on this yet from the core devs.

Robert

Hi Robert,

Subnet and Routers module is great and proper separation speeds up my work a lot.
I found a bug in Routers or omission rather. I need to be able to create Router with no entires (as AWS is always adding default LOCAL route for VPC) and that’s what missing in one of the cases.

`

  • name: Create/Update Route Table >> ElbRouteTable_KeyApi
    local_action:
    module: ec2_vpc_route_table
    vpc_id: “{{ vpc_id }}”
    region: “{{ ec2_region }}”
    resource_tags: { “Name”:“{{ env }}-ElbRouteTable_KeyApi” }
    subnets:
  • ‘{{ref_subnet_AccessDmzA.subnet_id}}’
  • ‘{{ref_subnet_AccessDmzB.subnet_id}}’
  • ‘{{ref_subnet_AccessDmzC.subnet_id}}’
    register: reg_ElbRouteTable_KeyApi

`

Would be really cool if you could allow this perhaps next time you do an update.

Working example below works great (example). Same thing requires about 100 lines in CloudFormation

`

  • name: Create/Update Route Table >> ElbRouteTable_Ui_CoreApi
    local_action:
    module: ec2_vpc_route_table
    vpc_id: “{{ vpc_id }}”
    region: “{{ ec2_region }}”
    resource_tags: { “Name”:“{{ env }}-ElbRouteTable_Ui_CoreApi” }
    subnets:
  • ‘{{ref_subnet_ElbUiA.subnet_id}}’
  • ‘{{ref_subnet_ElbUiB.subnet_id}}’
  • ‘{{ref_subnet_ElbUiC.subnet_id}}’
  • ‘{{ref_subnet_CoreApiA.subnet_id}}’
  • ‘{{ref_subnet_CoreApiB.subnet_id}}’
  • ‘{{ref_subnet_CoreApiC.subnet_id}}’
    routes:
  • dest: 10.0.0.0/8
    gateway_id: ‘{{vgw_id}}’
  • dest: 172.16.0.0/12
    gateway_id: ‘{{vgw_id}}’
  • dest: 192.168.0.0/16
    gateway_id: ‘{{vgw_id}}’
    register: reg_ElbRouteTable_Ui_CoreApi

`

Best,
Peter