Synchronizing sound with graphics in 4k intros
category: code [glöplog]
Hi ! I'm pretty new to 4k and making my way through my first one (Windows intro).
I'm facing a sync issue between sound and graphics. The graphics is a GLSL shader and the music is a generated WAV buffer that I start playing with sndPlaySound (SND_ASYNC mode) just before the rendering loop (basically like in iq's 4k framework). The WAV has the same duration as the intro, and the rendering loop simply runs until timeGetTime() - startTime > introDuration.
Ideally, the audio and graphics should start roughly at the same time (without much of a difference at least). However, I notice a random delay (about 0.1s) between the moment the window/graphics start and the sound starts playing, resulting in wrong graphics/audio timing and the intro terminating a bit before the music finishes.
Does anyone know why it is happening and if/how it can be fixed ? Is there a correct way to ensure graphics/audio sync ?
Maybe my approach at 4k is completely outdated, haha...
Thanks !
I'm facing a sync issue between sound and graphics. The graphics is a GLSL shader and the music is a generated WAV buffer that I start playing with sndPlaySound (SND_ASYNC mode) just before the rendering loop (basically like in iq's 4k framework). The WAV has the same duration as the intro, and the rendering loop simply runs until timeGetTime() - startTime > introDuration.
Ideally, the audio and graphics should start roughly at the same time (without much of a difference at least). However, I notice a random delay (about 0.1s) between the moment the window/graphics start and the sound starts playing, resulting in wrong graphics/audio timing and the intro terminating a bit before the music finishes.
Does anyone know why it is happening and if/how it can be fixed ? Is there a correct way to ensure graphics/audio sync ?
Maybe my approach at 4k is completely outdated, haha...
Thanks !
sndPlaySound is cheap but not reliable, sadly. If you have a few more bytes, you should probably be better off with just a single-buffer DirectSound setup (that you can render directly into) and then using the GetCurrentPosition to tell where your play cursor is.
(Upside: you can also #ifdef _DEBUG yourself seeking in the intro.)
(Upside: you can also #ifdef _DEBUG yourself seeking in the intro.)
Thank you for the advice ! I'll try using DirectSound then.
Just out of curiosity, the Microsoft's documentation mentions WASAPI as a better alternative to DirectSound (which is apparently legacy). Have there been any attempt at using it, or does it add too much of an overhead for 4k compared to DirectSound ?
Just out of curiosity, the Microsoft's documentation mentions WASAPI as a better alternative to DirectSound (which is apparently legacy). Have there been any attempt at using it, or does it add too much of an overhead for 4k compared to DirectSound ?
You are probably using mmsystem.h, as it was used in Leviathan and IQ's framework. With some Windows update, this was of playing sound started to have this annoying delay. The delay even seems to be machine-specific. I worked so hard to figure out what the delay was on my machine (in the 4k intro Adam), but when the intro played on compo machine, the sync was off again.
Thus, don't use it. Instead, use DirectSound, as Gargaj suggested, with DSBCAPS_TRUEPLAYPOSITION to syncs land right on time. See here for a barebones examples and here how it was used with 4klang in an intro.
Thus, don't use it. Instead, use DirectSound, as Gargaj suggested, with DSBCAPS_TRUEPLAYPOSITION to syncs land right on time. See here for a barebones examples and here how it was used with 4klang in an intro.
Alright, thank you for the links !
Don't use timeGetTime(), it's not synced with music playback, and nobody guarantees that visuals and audio start at the same time even if the start commands are located close to each other.
If you use waveOutOpen() to play music, you ask the system "what time is it now in audio land", and it returns you the current time position. You do this with the waveOutGetPosition() function.
Compofiller Studio's Templates/4k_intro/exemain.asm contains a working example how you play stuff.
Loppu means end in Finnish. PROD_END_TIME is the end-time of your prod in samples.
If you use waveOutOpen() to play music, you ask the system "what time is it now in audio land", and it returns you the current time position. You do this with the waveOutGetPosition() function.
Compofiller Studio's Templates/4k_intro/exemain.asm contains a working example how you play stuff.
Code:
...
global MMTime
; MMTIME is a "union" struct that can be used for many different content types based on the first field
MMTime: dd 2 ; wType: TIME_SAMPLES = 2
MMTime_sample: dd 0 ; actual position value
db 0, 0, 0, 0 ; some room for other cases
...
extern _waveOutOpen@24
extern __imp__waveOutOpen@24
extern _waveOutPrepareHeader@12
extern __imp__waveOutPrepareHeader@12
extern _waveOutWrite@12
extern __imp__waveOutWrite@12
extern _waveOutGetPosition@12
extern __imp__waveOutGetPosition@12
section .code2 code align=1
; ---- GetPosition returns the current audio playback position ----
section .code3 code align=1
global GetPosition
GetPosition:
push 12 ; sizeof MMTIME struct
push MMTime
push DWORD [hWaveOut]
call [__imp__waveOutGetPosition@12]
mov eax, [MMTime_sample]
ret
...
call GetPosition ; <-- get the time to eax
; did we reach the end of the prod?
cmp eax, %[PROD_END_TIME]
jae loppu
Loppu means end in Finnish. PROD_END_TIME is the end-time of your prod in samples.
@yzi: you don't need WaveOutPrepareHeader@12 btw
Ok. I think I tried leaving out everything one by one, and left the waveOutPrepareHeader call in place for some reason.
I tried leaving out waveOutPrepareHeader() and then audio doesn't start. Maybe you could hard-code some stuff in the struct, and then some bytes gained by leaving out the function call would be lost by the added data values.
Have you tested this and how much does it save?
Have you tested this and how much does it save?
it boils down to this: https://github.com/vsariola/sointu/blob/master/examples/code/asm/386/asmplay.win32.asm#L37
I always use it this way in my intros and didn't notice anything breaking yet
I always use it this way in my intros and didn't notice anything breaking yet
it saves the symbol + 3 * (push instruction + operand) + call instruction. So overall maybe not that much, but any byte you can save there adds to what you can add content-wise
I tested it, it worked and saved 5 bytes in the final Crinklerized output.
To apply the change, edit exemain.asm and write 2 in the dwFlags field of WaveHDR:
And comment out the waveOutPrepareHeader call
Nice!
To apply the change, edit exemain.asm and write 2 in the dwFlags field of WaveHDR:
Code:
global WaveHDR
WaveHDR:
WaveHDR_lpData: dd rendered_audio_data
WaveHDR_dwBufferLength: dd (AUDIO_BUFFER_ALLOCATED_LENGTH * 4) ; audio buffer length in bytes
WaveHDR_dwBytesRecorded: dd 0
WaveHDR_dwUser: dd 0
WaveHDR_dwFlags: dd 2 ; <----- wave header is declared as prepared
WaveHDR_dwLoops: dd 0
WaveHDR_lpNext: dd 0
WaveHDR_reserved: dd 0
And comment out the waveOutPrepareHeader call
Code:
; push 32 ; sizeof(WaveHDR)
; push WaveHDR
; push DWORD [hWaveOut]
; call [__imp__waveOutPrepareHeader@12]
Nice!