Ansible AI Policy?

Project Discussions
, ,
1

Should the Ansible project have a AI policy?

Some Free / Open Source projects have AI polices, for example Servo has:

AI contributions

Contributions must not include content generated by large language models or other probabilistic tools, including but not limited to Copilot or ChatGPT. This policy covers code, documentation, pull requests, issues, comments, and any other contributions to the Servo project.

For now, we’re taking a cautious approach to these tools due to their effects — both unknown and observed — on project health and maintenance burden. This field is evolving quickly, so we are open to revising this policy at a later date, given proposals for particular tools that mitigate these effects. Our rationale is as follows:

Maintainer burden: Reviewers depend on contributors to write and test their code before submitting it. We have found that these tools make it easy to generate large amounts of plausible-looking code that the contributor does not understand, is often untested, and does not function properly. This is a drain on the (already limited) time and energy of our reviewers.

Correctness and security: Even when code generated by AI tools does seem to function, there is no guarantee that it is correct, and no indication of what security implications it may have. A web browser engine is built to run in hostile execution environments, so all code must take into account potential security issues. Contributors play a large role in considering these issues when creating contributions, something that we cannot trust an AI tool to do.

Copyright issues: Publicly available models are trained on copyrighted content, both accidentally and intentionally, and their output often includes that content verbatim. Since the legality of this is uncertain, these contributions may violate the licenses of copyrighted works.

Ethical issues: AI tools require an unreasonable amount of energy and water to build and operate, their models are built with heavily exploited workers in unacceptable working conditions, and they are being used to undermine labor and justify layoffs. These are harms that we do not want to perpetuate, even if only indirectly.

Note that aside from generating code or other contributions, AI tools can sometimes answer questions you may have about Servo, but we’ve found that these answers are often incorrect or very misleading.

In general, do not assume that AI tools are a source of truth regarding how Servo works. Consider asking your questions on Zulip instead.

There has recently been a discussion about this on a GitHub issue.

Personally I’d support the Ansible Project adopting something like this, has / could / should this be discussed by the steering committee?

3 Likes
2

I tend to agree, but I’d focus more on the Copyright/Ethical issues as the most relevant. “Maintainer Burden” and “Quality & Security of code” are both fuzzy topics and are the most likely to evolve rapidly (and therefore, harder to keep a policy up to date on), while “this poses risk we’re unwilling to take” and “this does not align with our values” are much more concrete positions to have conversation around.

At the very least, I very much agree that this should be addressed by the steering committee, and some decision made.

2 Likes
3

I really like this! Regarding maintainer burden, I think this is a very real problem, even though we’ve been (as far as I can tell) lucky to haven’t gotten much (or any? not sure) of it. Some other projects, like curl, have been flooded with AI generated bug/security reports, which consume a considerable amount of resources from the maintainers. Also considering that many Ansible collections have too few and/or overworked maintainers, we should definitely discourage AI submissions of any kind (in particular bug reports or pull requests).

3 Likes
4

To be clear, I wasn’t down-playing either the contributor burden or the security/correctness issues. They just seem easier to argue with/harder to understand than the other points.

1 Like
5

I like the idea of having an official policy on AI contributions that Ansible ecosystem projects and collections can choose to adopt, but I don’t think this should be mandated. Whether or not we agree it, at the end of the day, it’s up to individual maintainers whether they want to accept AI contributions or not.

3 Likes
6

Good point @gotmax23! I think it depends on what is meant by “the Ansible project”. The whole Ansible ecosystem? I wouldn’t want to make an AI policy mandatory there.

I’m happy with deciding on an AI policy when it comes to, for example, ansible-build-data. Or work on an example policy that collections can adopt, although this would be voluntarily. Like the GitHub workflows we’re providing for collections.

1 Like
7

Hi folks,
I appreciate this being raised and the discussion so far. This was something I’ve been thinking about, but hadn’t gotten to posting. Some similarities with back hacktoberfests

Where AI spam exists

  • GitHub, especially ansible/ansible
  • This Forum

We have seen an uptick in AI spam in ansible/ansible (it’s likely elsewhere, though I’ve not noticed it).

I’ve seen cases on this Forum that people have reported as spam (thank you), and after looking at the posts (and their others) have been fairly sure they are AI generated, and while they “look nice”, they don’t make sense. So I’ve treated them the same as other spam.

Code of Conduct (CoC)

  1. As to the “where this applies”, we could define this to be where the Ansible Code of Conduct (CoC) applies. Which is all the repo under out GitHub Orgs, and in a lot of collection repos. As well as Meetups, Forum, Matrix, IRC, reddit’s r/ansible, etc
  2. Though the CoC should be high level, and likely isn’t the place for anything specific about LLM. We don’t want it to change too often.
  3. Extending the CoC to say something like “Read and follow the Contributor Guidelines”, could work
  4. We don’t currently have a unified Contributor Gudelines, though maybe we could have some common bits for this, The Forum, etc. Which would allow a projects Guidelines to say “See the main Contributor guidelines” and “here are the foobar project specific bits”
  5. Some CoC have had generic statements like “Do not claim the work of others as your own”

Wording

  1. If we think about what we are trying to achieve, which is “We want contributions that have has some thought put into them”, ie don’t blindly copy stuff from StackOverflow/AI/whatever.
  2. At a high level, if people are posting junk (from AI or direct from their brain), it’s disruptive and was waste of other people’s time, which to me is a CoC issue.

What other projects are doing.

3 Likes