State of the art tooling for 4k intro for the Web
category: code [glöplog]
Lastly, I am not liking the standard SVG+fetch() packing system. Depending on Cache-Control and other factors, it will actually require downloading the demo twice. I think this makes the idea of “very few bytes can do a lot of content” a bit harder to sell, because you have to make it about the "bytes used to store the demo on the server's disk" instead of "bytes you had to download to see watch the demo". Basically, when you open the Networking panel on your browser, the SVG method feels a bit like cheating.
So I explored other deliveries methods:
* Direct BR/GZ tranmission through HTTP, makes the demo watchable with a single query, potentially halving the number of bytes down the wire. It still does require a server, although if you happen to have the file on disk, it’s easy enough to just right click and decompress it with your OS’s default ZIP utility.
* A Base222 self-contained HTML, which is also playable with a single HTTP request, and can also run serverless by just drag-and-dropping the file to a browser if you have it on disk. It’s a bit bigger on the wire (and disk) since the encoding is not as efficient.
You can argue that unlike the SVG method which relies on Web APIs only, the BR/GZ method depend on HTPP since it does requires Content-Encoding gzip headers for the intro to be compressed. I'd argue the SVG method can only make fair claims about download sizes if there are proper Cache-Control headers anyways, so to me both play on a plain field really. Both require a server for delivery, but BR/GZ is guaranteed to send the true bytes once.
Both SVG and BR/GZ still require a server, though BR/GZ is a bit more friendly if you have the intro on disk and just want to watch it without launching a server. But the only true serverless solution is the Base222 one.
In summary:
With all this in mind, does you think size coding for the Web should be about "bytes on (the server's) disk" (SVG and Base222) or about "bytes to download" (BR/GZ and Base222) ???
So I explored other deliveries methods:
* Direct BR/GZ tranmission through HTTP, makes the demo watchable with a single query, potentially halving the number of bytes down the wire. It still does require a server, although if you happen to have the file on disk, it’s easy enough to just right click and decompress it with your OS’s default ZIP utility.
* A Base222 self-contained HTML, which is also playable with a single HTTP request, and can also run serverless by just drag-and-dropping the file to a browser if you have it on disk. It’s a bit bigger on the wire (and disk) since the encoding is not as efficient.
You can argue that unlike the SVG method which relies on Web APIs only, the BR/GZ method depend on HTPP since it does requires Content-Encoding gzip headers for the intro to be compressed. I'd argue the SVG method can only make fair claims about download sizes if there are proper Cache-Control headers anyways, so to me both play on a plain field really. Both require a server for delivery, but BR/GZ is guaranteed to send the true bytes once.
Both SVG and BR/GZ still require a server, though BR/GZ is a bit more friendly if you have the intro on disk and just want to watch it without launching a server. But the only true serverless solution is the Base222 one.
In summary:
Code:
Method On Disk On HTTP Page Source Serverless
======= ======= ======= =========== ==========
SVG 4987 9974 4987 No
BR/GZ 4865 4865 6548 No/Kindof
Base222 5284 5284 5284 YesWith all this in mind, does you think size coding for the Web should be about "bytes on (the server's) disk" (SVG and Base222) or about "bytes to download" (BR/GZ and Base222) ???
Nice!
Have you tried ECT instead of Zopfli ?
It gave better results in my recent tests.
For quick and dirty experimentation purpose on Elevated: Throwing the source code at Claude, instructed to use terser, roadroller, ECT, the SVG loader, .... came at 5,217 bytes at first. Iterating a bit, without sacrificing the audio or visuals, came down to 4,928 and 4,937 depending if the audio generated at first or live.
So yeah, in the 4.8-4.9 Kb ballpark, or ~1.2x the size of native, which matches pretty closely my experience in 1kb intros.
Have you tried ECT instead of Zopfli ?
It gave better results in my recent tests.
Quote:
😮 Wowzers!modified version of Roadroller that Blueberry created, which saved 200 bytes.
For quick and dirty experimentation purpose on Elevated: Throwing the source code at Claude, instructed to use terser, roadroller, ECT, the SVG loader, .... came at 5,217 bytes at first. Iterating a bit, without sacrificing the audio or visuals, came down to 4,928 and 4,937 depending if the audio generated at first or live.
So yeah, in the 4.8-4.9 Kb ballpark, or ~1.2x the size of native, which matches pretty closely my experience in 1kb intros.
About bytes on disk or served, the second request should be cached so zero bytes actually transfered.
The Web requires a server.
A few things work from file:// as a convenience, but the Web is the Web, so there should be no controversy for Web prods to need a server, even if only to serve the proper headers.
The Web requires a server.
A few things work from file:// as a convenience, but the Web is the Web, so there should be no controversy for Web prods to need a server, even if only to serve the proper headers.
I tried ECT, Zopfli and Turtle, but Zopfli worked best for me.
Damn, I could have a whole weeks of work :(
Quote:
For quick and dirty experimentation purpose on Elevated: Throwing the source code at Claude, instructed to use terser, roadroller, ECT, the SVG loader, .... came at 5,217 bytes at first. Iterating a bit, without sacrificing the audio or visuals, came down to 4,928 and 4,937 depending if the audio generated at first or live.
Damn, I could have a whole weeks of work :(
* I could have saved a whole week
Quote:
Ah! I only tried ECT recently, and just learnt about Turtle. But then I will definitely try the 3 of them next time I work with Deflate.I tried ECT, Zopfli and Turtle, but Zopfli worked best for me.
Quote:
I could have saved a whole week
"Saved".
Instead you learnt a ton in that week about how these tools work with each other.
FWIW, as plain html.br it goes down to 4,595 and 4,612 depending if the audio is generated first or live. That's roughly 1.125x from native.
Quote:
With all this in mind, does you think size coding for the Web should be about "bytes on (the server's) disk" (SVG and Base222) or about "bytes to download" (BR/GZ and Base222) ???
muhmac (and later used by Bercon, in Incendium) packed Brotli compressed intro into a Powershell script. When the script is ran, it acts as a tiny web server, opens a browser pointing to the local url, and serves the intro as a .br compressed stream. This gives an intro that can be executed locally; does not need wanky browser security settings; is <= 4k on disk; and uses Brotli for compression; all without requiring DecompressionStream('brotli').
Powershell bootstrap code costs bytes though:
https://github.com/Bercon/incendium/blob/16c14f83ce7ad2d4436bdb157a58793200530003/scripts/packager/compress.py#L15
I used Incendium as one of the test cases for the modified Roadroller. It came out slightly ahead of the PowerShell+brotli version, with 4057 against 4093.
To me the appeal of 'web' intros is that they're fully cross-platform, so having the Powershell dependency feels a bit antithetical. But I guess you could say that dropper would only be used for the compo, perhaps.
Okey, I'm officially done with the experiment. I landed at 4,760 for the html.br version and 4,813 for the SVG version.
Sadly, all my development was on Edge/Chrome/Firefox, and in the last minute I had to add ~150 bytes to make it work on Safari too, and to make it compute the music quickly enough on that browser (Safari's JS engine doesn't like complicated loops it seems). I can see a few places to shave another 25 bytes or so, but at this point the Safari performance issue is so fragile that I don't want to touch it anymore. Yet another reason to compute the sound on a shader/the GPU perhaps?
I also refused to remove the ~100 byets needed for a clean user experience (loading message, windows resizing support, legal sound after click, etc).
So yeah, similar to p01's experiment, I got that the Web:Windows ratio is ~1.15x for this particular intro. I'll upload it and add the link in the prod's page tomorrow. Thanks Blueberry for the help with Roadroller.
Sadly, all my development was on Edge/Chrome/Firefox, and in the last minute I had to add ~150 bytes to make it work on Safari too, and to make it compute the music quickly enough on that browser (Safari's JS engine doesn't like complicated loops it seems). I can see a few places to shave another 25 bytes or so, but at this point the Safari performance issue is so fragile that I don't want to touch it anymore. Yet another reason to compute the sound on a shader/the GPU perhaps?
I also refused to remove the ~100 byets needed for a clean user experience (loading message, windows resizing support, legal sound after click, etc).
So yeah, similar to p01's experiment, I got that the Web:Windows ratio is ~1.15x for this particular intro. I'll upload it and add the link in the prod's page tomorrow. Thanks Blueberry for the help with Roadroller.
Quote:
I'll upload it and add the link in the prod's page tomorrow.
Feel free to replace the online version link! 👍
Oh I didn't see that one. I think both can co-exist, the "4.6 kb web version", which is my faithful remaster of it, and then also your port there which is more of a 100% faithful/bit exact-to-the-original historical preservation effor (with a nicer UX on top)
(and I mean I didn't see the link already added to the prod, not the actual port)
Wait, did Mr.Doob publish his port of Elevated without mentor's, iq's and puryx' approval ? o_O
I follow the "it is better to ask for forgiveness than permission" approach. Unless I know the author is against that approach.
mrdoob messaged me when he released it, weeks ago. I'm sure he would have taken it down had I asked him. I'm generally against having to opt-out rather than opt-in into things (I don't appreciate the Internet Archive for that reason).
But hey, it's fine, it's just a 4k intro, and I actually do like what he has done. For me preservation as an actual running program has more magic than preservation as a youtube/mp4 video, at least for modern demos like Elevated that use high level APIs anyways rather than low level hardware tricks. You can feel the same algorithm and same maths as the original still running through your computer, pumpinb with life again, rather than havnig to watch a ghost of it through an mp4 that only imitates the original piece.
That being said, even though I appreciate mrdoob's version, I realized something was still missing. What makes Elevated (and many 4k intros) interesting is what it was able to do within limited space. So I started my exploratoin of 4k size coding for the Web and this thread.
The way I see it:
* Mine is an actual port to a new platform, with the same exact visuals, music and intent of the original. I fought for each byte of it, I revisited everyone of Mentor's compression tricks with joy and listened to Puryx's track hundreds of times again.
* mrdoob' is more of an emulation of the original intro, that preserves the content to the bit, subpixel and sound subsample, exactly. And so it's perfect for historical archival while still keeping the maths and magic alive in it instead of being just a recording. But because all the exactness/emulation aspect, it can't really be a 4k-ish intro anymore.
But hey, it's fine, it's just a 4k intro, and I actually do like what he has done. For me preservation as an actual running program has more magic than preservation as a youtube/mp4 video, at least for modern demos like Elevated that use high level APIs anyways rather than low level hardware tricks. You can feel the same algorithm and same maths as the original still running through your computer, pumpinb with life again, rather than havnig to watch a ghost of it through an mp4 that only imitates the original piece.
That being said, even though I appreciate mrdoob's version, I realized something was still missing. What makes Elevated (and many 4k intros) interesting is what it was able to do within limited space. So I started my exploratoin of 4k size coding for the Web and this thread.
The way I see it:
* Mine is an actual port to a new platform, with the same exact visuals, music and intent of the original. I fought for each byte of it, I revisited everyone of Mentor's compression tricks with joy and listened to Puryx's track hundreds of times again.
* mrdoob' is more of an emulation of the original intro, that preserves the content to the bit, subpixel and sound subsample, exactly. And so it's perfect for historical archival while still keeping the maths and magic alive in it instead of being just a recording. But because all the exactness/emulation aspect, it can't really be a 4k-ish intro anymore.
Quote:
I'm sure he would have taken it down had I asked him.
Yep!
Quote:
But hey, it's fine, it's just a 4k intro, and I actually do like what he has done. For me preservation as an actual running program has more magic than preservation as a youtube/mp4 video, at least for modern demos like Elevated that use high level APIs anyways rather than low level hardware tricks. You can feel the same algorithm and same maths as the original still running through your computer, pumpinb with life again, rather than havnig to watch a ghost of it through an mp4 that only imitates the original piece.
Beautifully put.
iq seemed surprised about the AI port being already linked to on Elevated's page, but all is well. 👍🏻
Quote:
I follow the "it is better to ask for forgiveness than permission" approach.
This checks out.
