# ansible on 'brownfield' sites **URL:** \ **Category:** Ansible Project\ **Created:** [June 16, 2016, 12:15pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792 "2016-06-16T12:15:28Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Dick\_Davies](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Dick\_Davies](https://forum.ansible.com/u/Dick_Davies)\ **Post date:** [June 16, 2016, 12:15pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/1 "2016-06-16T12:15:28Z")
I've had great success using Ansible to build up multiple environments (devs, staging, prod) from kickstart to production, and I imagine that's how most folks here would use it. My current mission is to add some automation onto a lot of 'handbuilt' environments that have grown up from scripts. As you'd expect, each new environment has learned from the lessons of the previous one and the scripts have evolved. Unfortunately, that means each environment is different, sometimes radically - not just in scale of a given service but often in the presence or absence of it. There's a lot of commercial software deployed too, which results in the inevitable "we don't have enough licenses for $TECHNOLOGY to run it in all the dev. environments" and more drift. Basically "a maze of twisty environments, non alike" .. I'm making some headway in getting inventories written up to at least start to use adhoc tasks for some of the routine tasks. Next phase is "roads and sewers" - fairly straightforward services that are simple to setup but essential (e.g. NTP, SNMP, yum repos). I'm hoping to tackle some of the inconsistencies with liberal use of $inventory/group\_vars folders to enable/disable roles based on where I'm running. I had a look around but haven't seen much discussion about retrofitting Ansible to try to tame this kind of sprawl. Would be interested in how others have tackled the challenge - "nuke the site from orbit" and "run away screaming" excluded. ---
**Author:** ![Mark\_Janssen](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@Mark\_Janssen](https://forum.ansible.com/u/Mark_Janssen)\ **Post date:** [June 16, 2016, 2:37pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/2 "2016-06-16T14:37:29Z")
Mostly the way you seem to be doing it as well I guess… I have various customers where I’m implementing ansible on existing infra. I start with the basics, getting information on various systems and using group\_vars and host\_vars to enable/disable bits of configuration from ansible. Lots of running ansible with --check and --diff on small sets of hosts and trying to minimize (functional) diffs. Then slowly getting the systems all in line with eachother. Greenfields would be nice… but most customers aren’t up for it… but it’s fine either way. ---
**Author:** ![pixel\_fairy](https://avatars.discourse-cdn.com/v4/letter/p/6f9a4e/32.png) [@pixel\_fairy](https://forum.ansible.com/u/pixel_fairy)\ **Post date:** [June 16, 2016, 4:32pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/3 "2016-06-16T16:32:37Z")
git(or vcs of choice) is your friend! now that thats out of the way, we keep different inventory files for some stuff, and sometimes a vagrant file with its own inventory as “dev/test” for smaller or personal projects. one folder will have roles downloaded from galaxy. the others next to it have be for their own sections instead of one big ansible tree for everything. for example, all the routers and firewalls are in their own tree, so if you want to look at the changelog for those, its easy. so your tree could look like this. if you want to keep them all in one big repository, you’ll probably want git submodules. ansible ---
**Author:** ![pixel\_fairy](https://avatars.discourse-cdn.com/v4/letter/p/6f9a4e/32.png) [@pixel\_fairy](https://forum.ansible.com/u/pixel_fairy)\ **Post date:** [June 16, 2016, 4:39pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/4 "2016-06-16T16:39:21Z")
if by nuke it from orbit you mean build a new, streamlined environment and wipe out the old one, thats my favorite approach. ---
**Author:** ![Adam\_Morris](https://avatars.discourse-cdn.com/v4/letter/a/ecae2f/32.png) [@Adam\_Morris](https://forum.ansible.com/u/Adam_Morris)\ **Post date:** [June 17, 2016, 2:08am UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/5 "2016-06-17T02:08:52Z")
I am currently concentrating on building the new systems all the same. If nothing else happens as the old systems are life cycled the new systems will take over. If I get to the point where I have spare time I will start slowly fixing the old systems. One (or at least one batch) at a time. I don't intend to modify everything but if I can take over the management of the basic infrastructure then at least that will be uniform across all systems. DNS, NTP, monitoring, backups, SA accounts,... ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 17, 2016, 3:04pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/6 "2016-06-17T15:04:55Z")
I distrust ‘manual systems’, the first thing I normally do is reverse engineer each type of server and rebuild in an automated fashion (I did this way before Ansible, through bash/sh/ksh/perl/python scripting and other CM tools). The easy part is looking at installed packages and things configured in /etc, a bit harder is weeding out stuff that is not used anymore or obsolete. Then the ‘fun’ begins, find the /home/whatever/app.rb that is being executed from a detached session from when a dev logged in 2yrs ago and see how that app SHOULD run, its deps and requirements. Sometimes you need to change code as stuff is hardcoded and expectations are implied in subtle ways. Normally you can justify these in the process of adding high availability, scalability and/or security to an app. In the end I have a way to spin up any type of machine (app, web, db, etc) by running a script with a few options (last 3 jobs it was playbooks!). It is hard work, takes a long time and sometimes you have to overcome resistance by your coworkers rather than technical hurdles, but the end result is worth it. ---
**Author:** ![Adam\_Morris](https://avatars.discourse-cdn.com/v4/letter/a/ecae2f/32.png) [@Adam\_Morris](https://forum.ansible.com/u/Adam_Morris)\ **Post date:** [June 17, 2016, 4:27pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/7 "2016-06-17T16:27:19Z")
Resistance from co-workers is a big problem… Legacy systems is another one but can be dealt with… In some cases their are other issues though… But equally it is not always possible to replace manual installations. I am not an Oracle DBA and in the organisation I work for the DBAs are a separate team. While it might be possible for me to automate Oracle installation and configuration, I don’t know what to do, and I don’t think that they would let me. (Heck, they don’t listen when I tell them that asmlib is not needed, is a hack, and we should use the OS equivalent which is better supported) Some of our installs could be automated, but when you only install the app once on two machines and the vendor supplied instructions are several hundred pages, I am not going there. If the vendor wants to automate their install process then great, but in that case I am not. (The application, database, message bus, DNS entries, file systems, network settings, etc are all detailed. The application alone consists of several hundred daemons and has its own registry…) Finally I also have to deal with some systems which have been approved by a regulatory agency. Those ones cannot be changed without a lot more approval work. In those cases we are mostly at the mercy of the application vendor. ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 17, 2016, 4:36pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/8 "2016-06-17T16:36:32Z")
Not all environments are the same. I’ve been a DBA at times, other times I had reasonable DBAs, I’ve been QA at times, other times I had reasonable QA … etc … etc. When it comes to govt/regulation restrictions, then you really are SOL and just have to ‘live with it’. What I described above applies mostly to medium/small shops or (rare case) a big company that for some reason has been able to allow for a full revision of the systems, not just the machines, but standards and workflow. ---
**Author:** ![Adam\_Morris](https://avatars.discourse-cdn.com/v4/letter/a/ecae2f/32.png) [@Adam\_Morris](https://forum.ansible.com/u/Adam_Morris)\ **Post date:** [June 17, 2016, 4:58pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/9 "2016-06-17T16:58:00Z")
Indeed. I am talking about a large organisation (several hundred Unix/Linux servers alone, across five states, six major data centres) which has a lot of cruft, some very large, very complex applications, some federal regulation, and so on. Even in this situation though we can introduce standardised server builds for all new servers, roll the infrastructure pieces under configuration management on (most of) the existing servers, and so on. Even if the applications look different on every machine (and some of those will be rolled out with the same configuration management tool, increasing as the apps get lifecycled) the operating systems themselves will become more and more uniform. While it would be nice to be able to just rebuild everything it isn’t always possible. But starting with one or two changes and rolling those out, then moving on to the next one might seem like small victories but over time they all add up. When you know that you can log on to a server and have standard tools and configurations it feels much better than not knowing who hand crafted the artisanal server you now have to deal with and what their preferences are. Of course management likes the fact that I can roll out a new Linux server in 15 minutes compared to the old “two weeks” and they were all different. ---
**Author:** ![oravirt](https://avatars.discourse-cdn.com/v4/letter/o/3e96dc/32.png) [@oravirt](https://forum.ansible.com/u/oravirt)\ **Post date:** [June 17, 2016, 8:06pm UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/10 "2016-06-17T20:06:37Z")
A slight detour from the original question but, I’m an Oracle DBA and I use nothing but Ansible to manage my installations/configurations and I couldn’t be happier. It doesn’t matter if it’s a cluster (RAC) or stand alone systems, the [toolkit](https://github.com/oravirt/ansible-oracle)I wrote handles it. So, it can definitely be done. I recently changed jobs and I’m in the process of retro-fitting our (fairly large) Oracle environments to be managed by Ansible. Fortunately I haven’t had much resistance yet. Good luck! /M ---
**Author:** ![Dick\_Davies](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Dick\_Davies](https://forum.ansible.com/u/Dick_Davies)\ **Post date:** [June 19, 2016, 10:52am UTC](https://forum.ansible.com/t/ansible-on-brownfield-sites/21792/11 "2016-06-19T10:52:58Z")
Thanks all, glad i'm not on totally the wrong track 🙂 There's plenty of buy in from the new team, it's just deciding which sequence of bites we use to eat the elephant. Agree with earlier points that the key to this is knowing when to stop (Oracle team have actually got their act together, but they're happy using Oracle-specific management and I have no intention of changing that for changes sake). Another interesting point is there are at least 3 other CM systems in play here, all of which are doing a bit of the management roles (and yes that's as interesting as it sounds). I'm still finding servers I 'miss' because they aren't in the official inventory (I have a nasty feeling there are several official inventories \>\_\< ). I'm also expecting some talks with architects when it comes time to start decommissioning e.g. the puppet layer; the on call teams are always the ones who see the benefit but when you start to revisit the Agreed Solution you need to be careful 🙂 I'm wary of stepping on toes and coming across as a zealot, so I think it's time to meet up with a few teams and see who's keen to migrate first. With a bit of luck that'll get us some momentum. It's a redhat/satellite shop so the upcoming Ansible integration with Satellite is hopefully going to make this an easier sell (plus the change windows I've been putting in are waay smaller and management have started to notice that). Wish me luck, should be interesting 🙂