optimize haproxy enable/disable backends

Archives Ansible Project
1

Hello -

We have apps that use haproxy as a lb for http/https and we use 8 haproxy sockets. Below I have included a playbook extract and a sample run where I take a single server in and out of the lbs. In this example the server is behind 2 lbs and has 2 backends. So when I run the haproxy module play to enable or disable the server there are 32 subtasks (see below).

I am wondering if there is a faster way of doing this.

I have pipelining=True in the ansible.cfg.

This example is relatively simple - I have other configs with 4 backends and 6 lbs so the deploys can get a bit bogged down.

thank you

Michael

playbook extract

---
  - name: put loadbalanced servers in and out of maintenance
    hosts: all
    remote_user: ansible_user
    become: yes
    roles:
      - baseinstall
    #  - "{{ app_role }}"
    serial: "25%"
  #  accelerate: True # this is deprecated

    vars:
      disable_app: False
      enable_app: False

    tasks:
      - name: set pre-timestamp
        set_fact:
          pretimestamp: "{{ lookup('pipe', 'date +%Y%m%d-%T') }}"

      - name: show hostname and prestamp
        debug:
          msg: "{{pretimestamp}} - {{ansible_hostname}}"

      - block:
        - name:  disable server in lb
          haproxy: 'state=disabled host="{{ ansible_hostname }}" backend={{ item[1] }} socket={{item[2]}}'
          delegate_to: "{{ item[0] }}"
          with_nested:
          - "{{loadbalancer_group}}"
          - "{{lbbackends}}"
          - "{{haproxysocks}}"
        when: loadbalanced and disable_app

      - block:
        - name:  enable server in lb
          haproxy: 'state=enabled host="{{ ansible_hostname }}" backend={{ item[1] }} socket={{item[2]}}'
          delegate_to: "{{ item[0] }}"
          with_nested:
          - "{{loadbalancer_group}}"
          - "{{lbbackends}}"
          - "{{haproxysocks}}"
        when: loadbalanced and enable_app

      - name: set post-timestamp
        set_fact:
          posttimestamp: "{{ lookup('pipe', 'date +%Y%m%d-%T') }}"

      - name: show stats
        debug:
          msg: "{{pretimestamp}} - {{posttimestamp}}"

output extract (not showing enable play)


### ansible-playbook /etc/ansible/ansible-modules/lb_app_maint.yml -s -e "loadbalanced=True" -e "disable_app=True" -e "enable_app=True" -l web-sin-1.example.com

PLAY [put loadbalanced servers in and out of maintenance] *********************************************************************************************************************************************************


...


TASK [disable server in lb] ***************************************************************************************************************************************************************************************
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock1'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock2'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock3'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock4'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock5'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock6'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock7'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapp', u'/var/run/haproxysock8'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock1'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock2'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock3'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock4'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock5'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock6'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock7'])
changed: [web-sin-1.example.com -> 10.64.141.111] => (item=[u'lb-sin-1.example.com', u'myapphttps', u'/var/run/haproxysock8'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock1'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock2'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock3'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock4'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock5'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock6'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock7'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapp', u'/var/run/haproxysock8'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock1'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock2'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock3'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock4'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock5'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock6'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock7'])
changed: [web-sin-1.example.com -> 10.64.141.107] => (item=[u'lb-sin-2.example.com', u'myapphttps', u'/var/run/haproxysock8'])

TASK [show stats] *************************************************************************************************************************************************************************************************
ok: [web-sin-1.example.com ] => {
    "changed": false,
    "msg": "20170419-16:38:46 - 20170419-16:39:28"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
web-sin-1.example.com         : ok=7    changed=2    unreachable=0    failed=0
2

Is there a better forum for questions like this? I spent a fair amount of time reading and googling before posting. I am really curious how to optimize tasks like this. Where shoudl I post this?

3

Maybe async can be what you are looking for.
https://docs.ansible.com/ansible/playbooks_async.html

4

Thank you - I considered this but the haproxy enable and disable steps can’t be asynchronous.

You made me think of what I want and cant do in Ansible -

Lets say an app group with 20 targets and 4 lbs - if i set serial to 25% and take 5 out at a time it would maybe be faster to generate a new ocnfig without those 5 targets and reload haproxy. I’ll have to test if that is less disruptive than the current way.