During the Ansible Contributor Summit at CfgMgmtCamp and in a forum post, we discussed a lack of AI policy in Ansible and concluded that it should be developed, discussed, and accepted Ansible-wide ASAP.
Without an AI policy in place, contributors and maintainers don’t know how to act regarding AI, for example:
Is it OK to accept AI-assisted contributions?
Should AI usage be disclosed? If yes, how? Are there exceptions?
Is it OK to merge a pull request created solely by an AI bot or can we legitimately reject such pull requests because they don’t involve a human in the loop?
We tried to keep it as minimalist as possible: it sets a flexible and general basis, defining a minimal set of requirements.
Individual projects can have their own (even stricter) AI policies that do not contradict this AI policy (similar to what we have with our CoC in the Ansible community package collection inclusion requirements).
It’ll apply to:
Ansible GitHub organizations such as ansible, ansible-community, ansible-collections.
Official platforms such as this forum, Matrix channels, etc.
Third-party projects that are part of official Ansible-related distributions such as Ansible collection included in the Ansible community package.
Important practical implications:
The policy gives maintainers an option to reject contributions opened by AI bots as violating the policy without any other justifications.
It allows maintainers to use AI bots for maintenance purposes including creating PRs, but with mandatory human gating.
It requires AI-assistance disclosure from contributors in a specific format.
It requires AI-assisted contributions to comply with the CoC and the specific project’s license.
Proposed roadmap for this initiative:
Discuss and polish the AI policy text in the pull request
Get the policy approval from Red Hat legal
Start a poll here to accept the policy (merge the PR) or to work on it more UPDATE: let’s just get some approvals in the PR and merge it.
We are looking forward to hearing your thoughts on this initiative here in comments and in the PR.
Hey, I think the policy looks good, but I have some concerns with this section’s wording:
The use of AI tools MUST be explicitly disclosed by the author when a significant part of the contribution is taken from the AI tools output without significant changes. Grammar, spelling, and stylistic corrections do not require disclosure.
What counts as a significant part of the contribution, and how much is a significant change?
I’ve come up with a number of quantifiers, but they all have pretty significant tradeoffs.
Line count: How many lines is “significant”?
Line count as % of PR: Ditto ^. Plus, it makes no sense that a 1000 line quality PR should be allowed to include 100 lines of LLM code without disclosing while a 100 line quality PR generated by an LLM has to disclose.
Character count: Is changing variable names a significant change? What about renaming functions? What about refactoring some part of the code to a separate function? In this last case, the human impact on the number of characters is minimal with the fn declaration, editing some variable names to use fn inputs, and adding fn invocation(s) to the original code, but a refactor requires some understanding of the logical structure of the code. Is this a “significant change”?
Logic vs “typing”: One common use case for me is when writing an integration test. I’ll only give task names, and maybe a comment if I need some specificity. Then, I pass that to an LLM and it goes through and fills in all of the parameters. I wrote all of the logic, but an LLM did the actual typing of task blocks. Is this significant use? Probably, but the line between logic and typing is just as hazy as “significant”
I understand that this is meant to be a general policy, and applying a quantitative standard like my ideas is certainly worse than leaving it open, but “significant” seems vague and impossible to enforce. This is probably an intractable problem, but hey:
import this
...
Explicit is better than implicit
...
;)
Furthermore, I think that enforcing the format Assisted-by: model name, version, tool creates branding pressure and incentives that benefit AI companies more than the project. It feels like the “Sent from my Iphone” of the LLM company world. I would suggest including something like LLM-assisted-contribution as an alternative to the specific format to allow more room.
As a maintainer reviewing code, my behavior might change slightly for “LLM-assisted PR” vs “Human written PR”, but the specific model used makes no difference. I understand the rationale for tracking specific models (perhaps for analyzing contribution patterns over time), but I think the branding concerns and minimal review value outweigh this benefit.
In my view, if in doubt, disclose it. If you can’t be sure that a contribution is significant or not, it’s already a large enough that one should let others know about it.
