pouët.net

Couple of quick questions, about making intros (4..64k):

1. What do people do to store scenes in shaders nowadays? Previously I've done something like:



but it'd be nice to have some less sucky approach. Anyone worked out something better?


2. I need a softsynth. Yes, yes, there are many, but... I'm writing for macOS, and I'm also doing stuff that means my exe has to be 64bit. All the common ones seem to be written in 32bit assembly, which is a problem (especially as I have minimal asm knowledge and can't fix that..)

Something in C and reasonably cross platform would be good. Ideally I'd like to target 4/8K, but 64K is an option. Any suggestions?

Regarding 1. what you described is essentially what I do (in shader code). It's probably not the best but has worked well enough for me, and allows more flexible editing (can edit without recompiling the exe). Also if you write it in a smart and consistent way Crinkler will compress the heck out of it anyway. Additionally, see this thread:https://www.pouet.net/topic.php?which=10820.

Also the two Rootkids 4k and 8k macOS intros are using 4klang, so it's at least possible to get it somehow working :)

It’s possible yes... in 32bit mode :) Which is what I need to avoid (mostly due to using Metal for rendering which is 64bit only)

1. I've done different camera-angles with just modulo: angle1=floor(mod(beats, 2.)), angle2=floor(mod(beats, 3.)) etc. With enough time and live shader editing you can get decent results. But proper direction seems to require if statements in one form or another, just like noby said.

2. You don't need a real softsynth if you've got a musician who is able to program compute shaders. In Pheromone the music is produced by a single shader.

1. I am doing timings of the intro scenes base on the value returned by Clinkster_GetPosition() function: http://www.pouet.net/topic.php?which=11181 I know, I know this approach (many if statements) sucks but it works for me;) I went through sources of few 4kb intros and found "direction array". It is very interesting:

Hope it helps:)

cce:
Seems we're working on similar stuff :) I did something similar in the last scene of partycoded.exe (the camera is just lerping through an array of fixed camera positions).

There's also a nice path tracing trick in there. If you have motion blur working, just increase the blur duration up to a couple of seconds just before and after a scene change and you'll get a crossfade ;)

You can also see what happens when you realise you've got no synth just before the party in that one :D

Doing the synth as a shader is interesting. I'd like to leave the GPU free (because my recent stuff is getting pretty heavy), but I guess this could just run as precalc.

flow: Interesting. Not seen that particular way to do it before. It's not a good fit for what i'm working on, but I might use elements of that.

"Where the dead things dwell" uses a beat-based approach, in which I define a base beat for each scene then a vec4 multitrack Rocket-kinda sequence (by hand, no Rocket involved).

Example of base beats for scenes:


Example of a sequenced track:

Re. #1, I used if statements, but in a BST automatically generated by a Python script :) . It was much faster than linear. This was for webgl 1 in ShaderToy (before the update), so I didn't have access to static data. E.g., https://bitbucket.org/inclinescene/tdf2017/src/68cdae5cc7bf1af195c28ef9eaacbf46be32c209/gen-gfx.py?at=master&fileviewer=file-view-default#gen-gfx.py-216

I do all the timing in the shader. Only the time is provided from outside, and the shader translates it to bars/beats by dividing with a suitable float.

We don't hardcode timings at all in our 64ks: shaders have uniform inputs (positions of things, boolean toggles, colors, basically whatever numbers we decide to not hardcode). At editing time, our tool displays UI elements for all of those and we can animate them with keyframes and such. The keyframe data is dumped by the tool as a file containing C code that is basically a huge array initialization. That file gets included when we build the intro executable, so the keyframe interpolation code ends up using the exact same input data to produce those uniforms as it was using when we were editing. In addition to the interpolation data, there's also information about the current shot, i.e. basically a number indicating which rendering function should be called at that time.

I think what cupe's describing is probably the more popular way to do 64k timing. The keyframing method probably makes it considerably easier (in my limited experience) to quickly create, iterate, and refine animations, with the only major disadvantage afaik being that you have to create tooling for it. Trackers like Rocket can help here as they provide an animation editor GUI and library you can include in your project, and it just works. I know Logicoma uses Rocket in their recent 64ks, pretty sure it's reasonably common in 4ks as well (correct me if I'm wrong).

cupe: hmm.... so very tempting to write a better editor after reading that :) (Current one is merely a buggy but very productive live shader editor, great for code, but no sliders even never mind key framing).

I'm wondering now just how much can be moved out of the shader. With "real" motion blur (since I'm path tracing stuff), everything gets interpolated, not just per frame but within it. And if say there's a cross fade (which is cheap with path tracing), that implies there's 2 camera paths (both interpolated), plus two sets of scene data (also interpolated) during a single frame...

Maybe, then, the best approach is something like this:

- The actual "drawScene()" function just uses a bunch of "if (scene==1) {} etc." to draw each scene
- The scene number is handled CPU side, but it's a floating point number. Integers = scene, fractions = crossfade between scenes.
- The rendering loop can just do something like:



That should draw a percentage of frames from scene n, plus the remainder from scene n+1. (Code not checked and probably wrong :) Which handles scene number plus crossfades.

Do take the time to integrate GNU Rocket into your framework. You will never look back.

Rocket parameters hook up quite naturally to shader inputs. If you query the shader for its inputs at each reload, you can even add parameters without restarting your intro.

Quick test of my 'crossfade' thing:



Yep :) That's fading between 2 scenes (the floor changes material and the sphere changes from mirror to glass). If you're rendering many times per frame, this is a nice and pretty cheap way to fade. (Just make sure to get the step() parameters in the right order ;)

Blueberry: will take a look, see how it might fit in, thanks.

flow
Didn't expect my direction code to find its way into a thread like this.

NOTE: I had a discussion about this with noby some time ago, and as he says up there, he's writing his direction code completely in the shaders. Tests seem to indicate this compresses very well. The direction array code seen above is, from the viewpoint of the compressor, essentially noise.

Reliable comparison would require one to do the same direction using both methods. I haven't had the patience.

Ok, so block of if(){} it is.

Any suggestions on the audio side, other than generating some waves in a shader? :)

psonice: oidos seems to have a mac 64bits VST version atleast, so i guess the source player code should also compile in 64bits. O_o haven't tested though.

Quite a few of them have 64bit compatible VSTs, but the actual replayer coder is usually 32bit x86 assembly. Makes sense if you're optimising for size for a 32bit exe (most common case for intros)

psonice: We also had the same issue with macOS API's being 64bit only, however we came around that by having a x86 exe running 4klang, then we fired up python and used PyObjC (part of macOS, a bit buggy but works for it purposes) to access the ObjC API's. I think you can bootstrap MTKView pretty easily with python and PyObjC, and basically do the same hack as we did.

If you want, you can get access to our source for inspiration.

Tapped: if you could write a couple paragraphs describing the process more clearly (with example code and links to stuff) that would be quite useful to include at http://in4k.github.io

Tapped: I'd very much like a look at that source, that sounds like a hack of pure beauty! Thanks! Please drop me a line at: psonice at me dot com.

Btw, I thought I had a different fix earlier. I noticed, if I build my exe as 32bit, it fails on MetalKit but *not* Metal. And, in fact, it's possible to build a 32bit exe using only Metal! Of course there's a catch: it won't detect any GPU /o\ :D

I've also looked at including the MSL shaders as both raw source strings and compiled libraries, the compiled library ends up bigger than the whole exe with source, so that's a no-go.

psenough: I could probably do something like that.

psonice: I've sent you a copy of the source to your email :)

Tapped: did you have to make any modifications to 4klang to get it running on macOS?

All my attempts so far have just resulted in segfaults, but you're proven it's at least somehow possible.