pouët.net

Go to bottom

Coding assistants

category: code [glöplog]
tl;dr: We need a more nuanced GenAI debate that distinguishes between code and other content.

Generative AI is a contentious topic in the Demoscene, with most parties now restricting or outright banning its use in some or all compos.

Most discussions so far have been focused on graphics and, to some extent, music. The use of AI in coding (ranging from tab completion to full vibe coding) is often not touched upon or just implicitly assumed to be covered under the same umbrella.

I think we need to more explicitly distinguish AI-based coding assistents from other GenAI tools. Here are some reflections on how AI use in coding differs from e.g. graphics in ways that warrant separate treatment:

- Coding assistants mainly accelerate editing tasks, rather than provide creative input. In my experience, the code resulting from a tab completion or prompt is basically what I would have written anyway, just produced faster.

- The primary task that coding assistants do provide novel input for is using unfamiliar APIs. While this can take up a (frustratingly) substantial portion of many coding tasks, it is not my impression that this work is something that is particularly prized in the Demoscene.

- In contrast with graphics, generating code with AI is typically not viewed as stealing the work of specific individuals. You don't ask your coding assistant to produce code "in the style of John Carmack" or anything like that.

- The line between non-AI and AI coding assistance is blurry. Tab completion and other high-level editing features have gradually increased in sophistication over the years. Many demo coders have probably been using AI features for several years already without thinking that this might be something that contaminates their code with respect to compo rules.

- Many demos use third-party libraries under various permissive licenses. With no tradition (let alone requirement) for libraries to declare any use of AI, it can be difficult to guarantee that your demo is completely free of AI generated code.

- Coding assistants can be a very powerful assistive technology. If a person has a disability that makes it difficult to type fast on a keyboard or look at a screen for extended periods, AI code generation can be a game changer, enabling creative work that would otherwise be infeasible.

I am interested in hearing views on this, especially from compo organizers who are restricting AI use in competitions involving code.
added on the 2025-07-20 16:29:03 by Blueberry Blueberry
I don't really care if you used an AI assistant to speed up boilerplate code, or for the kind of stuff you'd previously go searching stackexchange for, but I'm not interested in watching a vibe coded demo.
added on the 2025-07-20 16:45:23 by xeron xeron
if one wants to use AI, just let him (or her). I just don't think it's a good idea*, though.

* learn assembly, learn C, learn higher level languages. understand what makes things "tick" from the ground up. that's the way, IMHO.
added on the 2025-07-20 16:50:14 by bsp bsp
Quote:
I don't really care if you used an AI assistant to speed up boilerplate code, or for the kind of stuff you'd previously go searching stackexchange for, but I'm not interested in watching a vibe coded demo.



</thread>
added on the 2025-07-20 17:44:06 by lynn lynn
Quote:
I don't really care if you used an AI assistant to speed up boilerplate code, or for the kind of stuff you'd previously go searching stackexchange for, but I'm not interested in watching a vibe coded demo.

So how would you distinguish those uses in compo rules?
added on the 2025-07-20 17:56:51 by Blueberry Blueberry
I don't know. I would hope people just understand the demoscene spirit and behave accordingly.

Maybe the rules should just say "please specify any days in which ai was used in your prod" and people will judge you as they see fit.
added on the 2025-07-20 18:03:11 by xeron xeron
*ways
added on the 2025-07-20 18:03:32 by xeron xeron
Sure, the rules could look like that. But that would be extremely lenient as current typical compo rules go. Consider the AI clause from the Evoke 2025 compo rules:
Quote:
(Generative) AI content will be disqualified as part of the preselection process. Usage of AI is prohibited. Entries containing AI content that pass preselection do so by accident, not endorsement, and are still violating the rules. For more details, see the rules of the individual compos.

Does this mean that if AI-driven tab completion is used for any part of the code, the demo is ineligible? That's the issue I am bringing up here.
added on the 2025-07-20 18:18:39 by Blueberry Blueberry
"Were all your thoughts pure while you were making this prod? Prove that you didn't have any nasty thoughts even for a second during production."

- Current AI tools are just tools. Even image generation is very much limited artistically and in possible ideas space.
- I found vibe coding to be extremely underwhelming for practically anything beyond generating test cases, and maybe some glue code.
- There's no technical way to enforce any "no AI" rules. Similar to client-side anticheat, trying to do it is fundamentally futile and just wastes effort that could be spent on something more useful.
- That said, badly used AI is immediately distinguishable as such. AI art that looks somewhat good requires a non-trivial amount of effort.
- In the end I expect crappy AI getting voted out, and reasonable one will be imbibed by the community and will be just yet another tool in the belt.
- "How dare you use 3dfx, did I learn all these rasterization optimization assembly tricks for nothing?"
- "How dare you using game engines for demos, did I learn all these rendering pipeline and shader optimization tricks and math for nothing?"
added on the 2025-07-20 18:37:56 by provod provod
And what does AI even mean. Or the "AI" that everybody hates, and should be banned (or atleast marked) in competitions? Is it neural nets? Backpropagation? Automatic optimization? LLMs? Datasets containing stolen information? Natural language interface? Chatgpt specifically?
Quote:

Does this mean that if AI-driven tab completion is used for any part of the code, the demo is ineligible? That's the issue I am bringing up here.


Blueberry nailed it.

Quote:

- Current AI tools are just tools. Even image generation is very much limited artistically and in possible ideas space.
- I found vibe coding to be extremely underwhelming for practically anything beyond generating test cases, and maybe some glue code.
- There's no technical way to enforce any "no AI" rules. Similar to client-side anticheat, trying to do it is fundamentally futile and just wastes effort that could be spent on something more useful.
- That said, badly used AI is immediately distinguishable as such. AI art that looks somewhat good requires a non-trivial amount of effort.
- In the end I expect crappy AI getting voted out, and reasonable one will be imbibed by the community and will be just yet another tool in the belt.
- "How dare you use 3dfx, did I learn all these rasterization optimization assembly tricks for nothing?"
- "How dare you using game engines for demos, did I learn all these rendering pipeline and shader optimization tricks and math for nothing?"


Couldn't agree more.
added on the 2025-07-20 18:46:57 by fra fra
Much needed thread!

I am disabling copilot nowadays for the any demo repo which is seriously annoying because I also only use the autocomplete feature. I am not using any of the interactive features or generate full sections with it.

Even at parties that are AI "friendly" you have to state on the beamslide which AI tools you were using. So even the simple autocomplete stuff would require me to state "Github Copilot" on the beamslide and then the AI shittery and downvoting starts.


I can still develop without AI autocomplete but it is a serious downer. I am getting mad all the time filling every stupid struct by hand etc. not sure if the situation will normalize.
added on the 2025-07-20 19:01:41 by mop2 mop2
Quote:
generating code with AI is typically not viewed as stealing the work of specific individuals.


Yes it is. By me at least. Having more victims (and all of them at once) does not make it less stealing.

Also, the usual arguments apply:
* LLMs generate plausible-looking code that has subtle bugs => maintenance effort increases
* LLMs do not model software design, which results in diffuse or interpolated designs. => maintenance effort increases, sometimes to an amount that keeps you from reaching your goal entirely
* The legal questions involved in vibe coding are still unanswered.
* In size coding, the boundaries between code and art are fluid, so the arguments from the art discussion apply as well IMHO
* Specifically vibe coding turns a "write code"-style task into a harder, less rewarding "review kinda dumb code"-style task. Unclear if only I enjoy that less than writing the real thing.
added on the 2025-07-20 19:32:00 by NR4 NR4
Nobody would even know if your code was generated. But things that it generates and how much effort to babysit it, one could have done the effort themselves anyways. It's not hard to make it generate a basic plasma effect, but do what with it? It looked like tutorial generic plasma when I generated it. If you want to do something more interesting you'd have to heavely build on that so it's not worth it.
added on the 2025-07-20 22:23:55 by Optimus Optimus
The arguments against AI are undeniably well-founded.

But what are we to do when some of the admitedly valuable contributors to the scene (ie. Blueberry or mop2 here) start asking where exactly we draw the line between LLMs and “smart” autocomplete?
added on the 2025-07-20 22:46:33 by fra fra
Quote:
But what are we to do when some of the admitedly valuable contributors to the scene (ie. Blueberry or mop2 here) start asking where exactly we draw the line between LLMs and “smart” autocomplete?


One possible option for drawing the line is prohibiting using any "scrambler" system that takes in works of others without asking their permission, and then produces results based on those works in any way.

Strictly formulated, that would prevent using probably all modern LLMs, regardless of whether one is cloud-proprietary, or self-hosted, and other generative things too. Unless you train your own LLM on farbrausch public repo or something, and make a demo about it.

That line, however, can be met with counter-arguments:
1. Is remixing legal? (kekw)
2. Legally speaking, IIUC quite recently there was a court decision that basically said that training using illegal stuff is fine, unless it can be de-scrambled and reproduced verbatim. Or something, correct me if I'm wrong.
3. Aren't we doing the same thing with our brains -- look at other people works, and then trying to reproduce it?
4. IIUC the strategy of all those AI shops was "move fast and break things + ask for forgiveness, not permission", i.e. do (arguably) illegal things and then entrench themselves into the fabric of society so deeply, that banning and reversing any of that becomes impractical. My opinion here is that they, unfortunately, succeeded so far, and Pandora's box is not closing any time soon.
5. Line between old-school autocomplete and poisoned LLMs will get blurrier and blurrier, with modern tools/IDEs getting more and more thing embedded and pre-installed, to the point where you're using it and not realizing it. We can't make everyone use vim or notepad exclusively for demos.
added on the 2025-07-20 23:09:03 by provod provod
Frankly, after the scene has grown up tolerating - if not actively embracing - remixes of everything from Rob Hubbard to Boris Vallejo to the Amen Break to Mezzoforte, I have to doubt the sincerity of anyone who suddenly decides that we now have to take an ultra-purist stance on LLM output that might theoretically contain traces of copyrighted code, just because it happens to support the current anti-AI narrative.

(and yes, I say that as someone who has regularly flipflopped between blatant copyright infringement and being an absolute stickler for it, mostly on the basis of "is this party small enough to slip under the radar")
added on the 2025-07-20 23:57:27 by gasman gasman
As per the @party compo rules:

"Any entry utilizing generative AI is restricted to the Wild category and must clearly state what tool was used, no exceptions. As per the previous rule, you must ensure you have the rights to release content created with this tool and to its training data."

So genAI is not banned in principle, and for the moment let's ignore the Wild compo restriction, the core here is to not use models or train on data to which you don't have rights to use. That is what @party believes in. You don't necessarily have to have produced all of the content in your production yourself, but you *should* absolutely have been granted the rights to use that content in your production, and realistically you should also credit those who provided that data in a README at the very least.

So for features that accelerate coding tasks, as lead organizer for @party I would ask that you *do not* use coding acceleration features from Github, etc. that train on lots of repositories without permission. But if there is some other tool that trains on only public domain code or where the original authors have explicitly granted permission, go for it.

There is also no way to check for this, so it's all completely by the honor system. And getting back to the whole Wild compo restriction thing: If you petition the orga team to allow use of a coding acceleration tool and we understand it's in good faith, we'd probably be OK with you entering it into a regular compo. I mean, if you wanted to cheat you would have just entered it without asking.

Finally, I expect that eventually after the hype dies down, some of these acceleration tools will become standard practice and nobody will care. Take a look at "context aware fill" in raster graphics applications. Based on other @party compo rules this should not be pertinent, but to parties with photo compos it's probably completely realistic to be OK with tools like that. And I imagine at some point boilerplate code generation will just be baked in, and potentially not even using a hueristic anymore but a classical algorithm.

But *right now* genAI is sucking all of the oxygen out of the room and you would not believe how many techbro chads have told me about some vibe coding they want to enter at @party and I just need rules to pioint at to immediately shut that down. The compo team has neither the time nor patience to deal with all of this when we have better uses of our time. No offense, but if you want to participate in the scene, at the very least we'd suggest you find a category in which you can be creative on your own, make some prods with your personal skills, and then maybe consider what some sort of generative tool could do for you a in a fair and ethical way, and bring it up to the compo team.
added on the 2025-07-21 00:01:32 by DrClaw DrClaw
To gasman's point, @party tries its best to not accept art rips and we do our best to encourage people to create original works.

And to Provod's point playing devil's advocate about "isn't this what we do as artists?":
The response to that is that artists take time, years of practice sometimes, to assimilate a few of the styles of others. They can't just be fed billions of artworks and build a model that spits that stuff out in a few hours. Human artists also tend to have big enough egos that they try to change things enough that it's "their version of Picasso" and not just "hey, I did a Picasso reproduction".
added on the 2025-07-21 00:10:02 by DrClaw DrClaw
Quote:
But what are we to do when

Mourn.
added on the 2025-07-21 12:08:41 by Gargaj Gargaj
This thread is about coding assistants, not AI-generated pictures or music.

It's undeniable that coding assistants speed up mundane tasks or enable easy use of a new API. As people have mentioned already, the copying and pasting of code has been a thing since forever. That has never been sufficient to make a good demo though, because the audience isn't inspecting the code during the compo. I think the main problem will be dealing with huge numbers of low-quality vibe-coded entries, especially in the PC demo category. I don't think coding assistants will raise the standard much, but they save the programmer's time for the more interesting work of solving new problems so perhaps high-quality releases will become more frequent.
added on the 2025-07-21 12:36:11 by fizzer fizzer
Using coding assistant is like going for a run on your e-scooter. You can do it, but it won't grow you muscles.

We are getting older, our brain needs to be properly exercised. Why not use standard pattern matching for API complete if you forget stuff? Those were actually even better, because they were *not* hallucinating.

Next thing you will need assistance to wipe your a**

Also as Gargaj said, those tools violate licenses blatantly.
added on the 2025-07-21 12:43:12 by tomkh tomkh
Quote:
Using coding assistant is like going for a run on your e-scooter. You can do it, but it won't grow you muscles.

It depends on how you use it. I generally find that spending less time on tedious typing allows me to spend a greater proportion of the coding time actually thinking about the code I write. And it shifts the mode of thinking towards creative problem solving (both on a technical and aesthetic level) away from boilerplate.
added on the 2025-07-21 13:04:11 by Blueberry Blueberry

login

Go to top