Hi,
I need to perform (in a loop) some tasks on host ‘A’ till something happens on host ‘B’. What is the ‘best practice’ of doing such things with ansible?
I saw loops over single task but this is not what I need (presumably), unless I write a shell script which perform all the tasks by connecting to different hosts and grabs the results. As the whole point of ansible (to me) is to pull all the ssh commands out of scripts (and get rid of these scripts too)
does that mean you’re running cross-host function that takes params from A and B and returns value on B? And you expect certain value on B before you move forward? I’m not an expert but it doesn’t seem to be Ansible domain. Maybe Fabric would be a better fit?
yep. exactly the case. at the moment this logic is implemented using .sh script… so looks like the best option is to mix .sh with ansible (for pure deployment tasks). Will take a look into Fabric … though I’d like to avoid introducing new tool for every simple task 
I would benefit by a more specific example so I can understand what you mean.
Ansible playbooks have not a mechanism for repeating a part of a playbook, but this feature could be useful for your user case. For example, you could write a series of tasks and add at the end of them a ‘goback’ task (by creating a ‘goback’ action plugin) that lets you go back at the first task of that group of tasks you want to repeat. That ‘goback’ task would of course have the appropriate conditional, which would normally be set by the repeating tasks.
You know what happens, when you use goto, right? https://xkcd.com/292/
There is no goback task in Ansible.
There will not be one.
(I’d still benefit from more detail from the original poster, btw)
Adding some details:
in terms of states I need not only have app ‘started’ on node A but also have it ‘warmed up’. as warming up a java process is a complex task the only way to put app in ‘warmed up’ state is to organize feedback loop between A and B and stop warmup runs (on B) as soon as number of compiled/recompiled methods by hotspot drops to certain level. At this point you’ll get some ‘confidence’ that app is warmed up and you can proceed further
ansible has a way of repeating single task but for some (may be good) reason there is no way of repeating ‘composite task’ (which is what roles are to my understanding)
Hello,
I haven’t tried it but maybe you can loop over an ‘include’ task ? have you tried it already ?
- include: warmup.yml
register: result
until: result.stdout.find("all systems go") != -1
retries: 5
delay: 10
"Adding some details:
in terms of states I need not only have app ‘started’ on node A but also have it ‘warmed up’. as warming up a java process is a complex task the only way to put app in ‘warmed up’ state is to organize feedback loop between A and B and stop warmup runs (on B) as soon as number of compiled/recompiled methods by hotspot drops to certain level. At this point you’ll get some ‘confidence’ that app is warmed up and you can proceed further
ansible has a way of repeating single task but for some (may be good) reason there is no way of repeating ‘composite task’ (which is what roles are to my understanding) "
This does not seem to be a solution to his problem. What he actually needs is a way to repeat a number of different, load generating tasks on host B until his java app on host A has been warmed up for good. It is that repetition that Ansible cannot currently handle.
“It is that repetition that Ansible cannot currently handle.”
I’m not sure that’s true.
Ansible has loops and things like with_sequence, as well as do_until loops.
Ansible can do this kind of loops, but only for a single task. It does not offer a way to repeat a number of different tasks as a group.
“Ansible can do this kind of loops, but only for a single task. It does not offer a way to repeat a number of different tasks as a group.”
Agreed, we really aren’t trying to create a programming language.
Do you have to be as careful with generating exactly a certain amount of load?
I would ordinarily think a script (called by ansible of course) could be created that generates a certain amount of load and that would be predictable enough to warm up most things.
It’s not about being that careful but rather about getting to predictable result as soon as possible.
As I mentioned warming up of java program is a complex thing and depends heavily on jvm version/ switches as well as load ‘patterns’. In my case everything could change… So to answer your question yes it’s possible to get guaranteed warm up … but the cost of such guarantee would be high (for my use case) - for sure app will be warmed up after half an hour of running load generators…
i’m fine with hybrid solution (i.e to run ansible from within a shell script then run some ‘imperative’ logic … just trying to pull as much ‘ssh’ out of scripts as possible)
btw not of immediate demand but what’s the best way to get ‘cached’ ansible facts from within shell script? i think it might be useful in certain circumstances
When a need to support a common task execution scenario arises and Ansible cannot handle it, I consider it a limitation. We are not trying to create a programming language, but we are all using Ansible to script task execution scenarios. I expect to be able to tell my machines “repeat that group of tasks until something becomes ready or reaches some limit”. This is speaking to my machines and telling them what to do, it’s not programming. I think Ansible can be improved on that aspect. Instead of writing a shell script, another option would be to write another ansible playbook with his load generation tasks and call it repeatedly in a local action in the “main” playbook (with an ‘until’ directive). That way, he will not have to abandon Ansible’s goodies (modules). Ansible inside Ansible!
I think you're picking at words a bit - it's programming computers to do
things. There are going to be cases where something like the script module
are going to apply.
Having ansible evolve entirely into a general purpose programming language
would make a very very bad programming language.
I'm ok if that's something we can't do, because it's going to be impossible
to do /everything/.
Ansible is already more flexible than any other config system out there,
because it's more stepwise, versus compiled-model based (even the coded
ones use DSLs still compile down).
Here are some ideas that might work for you:
bottom line - it sounds like your problem calls for more liberal use of technologies than just limiting yourself to one. Ansible is a good configuration management and orchestration system. Python is a programming language and Fabric is a framework written in it - Ansible has API you can call out to from within Fabric and get things done on a different level.
Ansible is powerful as a configuration system for bringing systems in a desired state and flexible for deploying software, but it could be more flexible as a task execution engine for things such as feedback-based repeated execution and bootstrapping of custom clustering environments (not using ready-made amazon’s or google’s or rackspace’s cloud engine products/services) based on complex nested data. This flexibility could come from increasing flexibility in creating task loops. It seems that you are not interested in going that direction with the excuse that more flexible loops are for programming languages. However, the fact that programming languages are very flexible in creating loops does not mean that other languages/systems (including Ansible) will automatically turn into programming languages if they become more flexible in that. Loops are natural. They are a very basic concept. They are everywhere in life, not only in programming, and people start learning about and using them from the very beginning. We have all played “Snakes and Ladders” as kids, which is exactly a loop of repeating steps until you reach a goal. So, having a ‘goback’ feature, for example, would be like being swallowed by a snake and going to repeat your previous steps :D. Nothing to worry about programming languages and other monsters…
“It seems that you are not interested in going that direction with the excuse that more flexible loops are for programming languages.”
These are project directions, not excuses, and we’re mincing at words here.
Just so this isn’t misinterpreted, Ansible already provides the ability to write custom iterators, we have things like “with_nested”, “with_sequence”, etc. It already can loop over single tasks quite fine,
So what you are asking about is the ability to loop over more than one set of tasks - I would not implement this as a “goback” even if we did this, it would be a higher level language construct, more akin to a block with control structures. Gotos are a gross construct nearly everywhere we are found.
In most cases it’s not needed - Modules already take care of most of the provisioning needs, for instance, “exact_count” on provisioning modules allow spinning up an arbitrary number of images, and then you can apply tasks to images using the host loop. Thus we can ALREADY do all of that cluster spinup, and do that every day.
While looping over multiple tasks in a common loop could be something Ansible does in the future, we have a lot of more important things to chase first in the PR queue before we would consider such complexity when so few things need it. So far the only example we have that we “can’t” do is “warm up group of servers A from load of B until servers A reach threshold X”, which strikes me as very arbitrary.
We’ve suggested a simple compromise of generating a set load, and that was ruled insufficient, or even calling out to a simple script - I see no point to going on about this infinitely given there are solutions. It’s a very niche scenario and I’m ok with not having a clear way to do EXACTLY that, when there are other ways to achieve the same ends, just not with the technical criteria stated about exactly how it should be implemented.
As long as there’s one possible way, that’s enough for me - we don’t have to implement exactly this proposed way, especially when we think it would reduce the readability of the Ansible playbook language and encourage more complicated playbooks when Ansible’s goal is encouraging simplicity and elegant playbook design.