Adopting an AI policy for sphinx-doc

[jdillard]

Given that there is more and more AI usage in terms of issues and PRs, it has been suggested by @timhoffm that it would be a good idea to adopt an official AI policy. So I figured I would start a discussion to get others input.

This isn't my wheel house, but I put together a list of examples below for inspiration to help get the discussion started.

Python ecosystem examples

Some example of what other Python projects have done regarding AI policies.

CPython

https://devguide.python.org/getting-started/ai-tools/

Quick Summary:

• Contributors are responsible for everything they submit.
• AI output must be reviewed carefully.
• Contributors should be able to explain their changes in their own words.
• Human judgment remains essential.

NumPy

A fairly comprehensive policy that covers more ground and worth looking at.

https://numpy.org/devdocs/dev/ai_policy.html

SymPy

Similar to NumPy, just kind of showcasing that there is somewhat of a convergance.

https://docs.sympy.org/dev/contributing/ai-generated-code-policy.html

Other examples

@melissawm put together this awesome repo https://github.com/melissawm/open-source-ai-contribution-policies

[timhoffm]

I think, just taking over numpy's policy is fine.

[jdillard]

I made a PR here #14473

[jdillard]

Not sure where to bring up misbehaving bots, but this bot closed it's own PR, then re-opened the same PR. I closed that PR, but if it re-opens it again, than the DDoS type behavior should be banned.

[jdillard]

I asked the owner of that bot to remove it from the org and they did so: #14482 (comment)

I don't have block permissions either it appears. I don't know if it is a huge problem just yet, most bots seem to have some mechanism to reach a human.

[jdillard]

@melissawm I thought about doing similar. I can create a ai-contribution label, what would we want the description of the label to be because I think that would help clarify when it should be used.

Going through PRs it isn't always entirely obvious it is an AI bot as sometimes their creators seem to try and personify them. Or in the case it is a human contribution (or done with a human's account), that it was in fact done with AI. Should it be a "at least some AI was used" label, or a "we believe AI was used" label (even though it wasn't disclosed), or should it be a "not following the AI policy" label?

[melissawm]

I think the only verifiable option is "not following the AI policy" label, but I'm sure @timhoffm has opinions as well if he wants to share 😅

[jdillard]

Now that the AI policy is in place, it may just be best to see how that plays out and handle the current PRs on a case-by-case basis.

[timhoffm]

I think the actual label text doesn't matter to much, because contributors will likely not care about labels. But if asked I would use "not following the AI policy" in the spelling "ai-policy-violation". That's pretty unambiguous. For unclear cases we'll have to do a short explanatory comment anyway (like "This looks AI generated, which would violate our AI policy").

[jdillard]

Closing this since we have a policy in place and a violation label. Thanks everyone!