pouët.net

Go to bottom

On the AI benchmarking

category: residue [glöplog]
 
The real test has to be Commodore 64 optimization, when it suggests illegal opcodes and hardware hacks to bypass VIC-II limits or save a single raster interrupt, that's when i call it "intelligent".
added on the 2026-02-18 22:21:50 by rudi rudi
Even intelligence in humans is poorly defined, so good luck with that.
added on the 2026-02-19 00:26:22 by fizzer fizzer
Oh I call it “extremely intelligent” when it fails to understand that an upside down cup is in fact a cup turned upside down.
Quote:
https://youtube.com/shorts/3fYiLXVfPa4?si=x11C4qxSSPYzbEu3
added on the 2026-02-19 06:41:39 by 4gentE 4gentE
Additionally, I call myself “extremely intelligent” when I fail to mark a link as a link and mark it as a quote instead. Sorry.

https://youtube.com/shorts/3fYiLXVfPa4?si=x11C4qxSSPYzbEu3
added on the 2026-02-19 06:44:13 by 4gentE 4gentE
I think what we typically recognise as intelligence in a broader sense requires things like self-determination, with adaptable agendas and motivations. Possibly even feelings. Like fizzer wrote, it's a poorly understood concept, and before we understand exactly what we mean with "intelligence", it's pointless to try and construct machines that exhibit it.

It's like programming; if you don't understand the problem you are trying to solve, your code is just going to be gibberish.
added on the 2026-02-19 06:58:45 by Radiant Radiant
Quote:
Possibly even feelings.

...and a body. With expiration date. And knowledge/understanding of it having an expiration date. Or without that knowledge/understanding if we aim for animal "no consciousness" intelligence. But yeah. "What is intelligence" is a very VERY legitimate question. Problem today is that most of the time this otherwise legitimate question is being (ab)used by techbros and AI groupies that just want to muddy the water, lower the bar for "intelligence". You know, in the way the "free speech" is being (ab)used for years and years now as a temporary means to ultimately promote hate and repression. It's the zeitgeist.
added on the 2026-02-19 07:49:00 by 4gentE 4gentE
If it is trained on our conversations about illegal opcodes and hardware hacks, it will still grab the relevant stuff from the training data and give an output that resembles a deep understanding of these techniques. But I sense it's always an illusion.

I had moments I asked it some things just for test, maybe to explain me some oldschool code or write something simple for retro platforms in either C or assembly. There was output that surprised me and momentarily made me think "wow, it even mentioned that niche optimization trick I heard from other coders". But I know it just found it in training data, never discovered it from scratch.

Also, at other moments the hallucinations are so funny, z80 code, few lines will be instructions that basically don't exist, unless it got them from later version of z80. I saw a mul a,b but I think e-z80 has such a mul. But then I saw ld hl,de, add bc,bc xor de,de and I asked about it and it made excuses that "Oh no, that's just pseudo-code of certain assemblers!"
added on the 2026-02-19 10:18:28 by Optimus Optimus
You can pass for intelligent by just repeating things other people say.
or copy/pasting links that other people have already copied/pasted.

better looksmaxxxing nowadays…
There are multiple aspects of intelligence, and yes, AI (so far) is not good in all of them, the same way humans are not good at certain things (computations in general).

The big challenge moving forward is to reduce the amount of "shortcuts" AI is doing. It's a huge problem, as you never know how the model really arrives at the conclusion. Usual question is, is it really applying fundamental reasoning principles or just finds a frequently occuring pattern that may fail in unseen scenario. This was big problem early on when everything was trained on the "human brain output" rather than step by step reasoning (which you don't have data for available at large). But guess what...it is well known problem and it is going to improve (yay!).
added on the 2026-02-19 14:10:34 by tomkh tomkh
proof that you can be smart enough to understand that sort of crap but really bad at taking a social hint that your presence is a nuisance and your creations are mediocre.
kaneel: what you said is not particularly nice, is it. Maybe you should care more about your own creations.
added on the 2026-02-19 17:25:33 by tomkh tomkh
my own mediocrity does not make want to work for the commodification of everything that I like doing, I still wake up, train for things I like to do, I still write my own music, I still play instruments, I still create… it's hard, but I work on it. You thought work for the people who are commodifying the shit out of everything we love, you're working with the people who will send us back to the fields for the sake of productivity, you're human trash to me.

And I don't need to be nice, not to things like you.
Yup, so we have it. If you are remotely considering using AI for anything or maybe even doing research on AI related topics you are automatically "bad person", enemy of the people etc... I guess, I'm not surprised here.

This is like the next-level of polarization - left vs right, AI haters vs AI enthusiasts..we are all gonna kill each other one day, while the true enemy (people who truly benefit all of this) quietly stay behind the scenes doing completely immoral things (rape islands and such). Just great!
added on the 2026-02-19 18:13:01 by tomkh tomkh
Don't be a snowflake about it though, it's getting old fast.
"you're human trash to me" "Don't be a snowflake about it"

Whatever, dude.
added on the 2026-02-19 18:26:23 by tomkh tomkh
It feels like this has devolved into a "with us" or "against us" thing, but it should be possible to hold two thoughts at once. I think we can acknowledge it for what it is, a massive pattern-matching engine, without necessarily surrendering our "soul" or craft into it. To Optimus' point: sure, it might just be spitting back illegal opcodes it found in an old coodebase, but that doesn't mean it can't be a useful rubber duck for brainstorming.

It's like the difference between a tool and the intent behind. You can be a purist who loves the frit of manual assembly and hardware tricks, while also using AI to scan for tedious boiler plate. Using a tool doesn't mean you support the commodification of everything, it just means you're using the available tech to get to the fun part faster.
added on the 2026-02-19 22:11:29 by rudi rudi
rudi: thanks for trying, but it's over. I doubt there is any hope for civilized discussion here anymore.
added on the 2026-02-19 22:29:17 by tomkh tomkh

login

Go to top