whats up with 6 minutes precalc on demos these days?
category: general [glöplog]
loaderbars are overrated!
so a priest, a rabbi and a horse walk into a loaderbar...
well, basically.. by just loading everything into memory from hd and then process it in a smart way it doesnt really matter what format it is in, as long as its acceptable..
reading raw data should take more time than compressed data (wich uses a sensible compression for decodingtime).
however, mmapped way can be very cool on consoles (i think that was kb kindof was talking about anyways).. but sure, ill have a look what takes more time, just for fun. kb: got any stats from yours compared to traditional png loading?
reading raw data should take more time than compressed data (wich uses a sensible compression for decodingtime).
however, mmapped way can be very cool on consoles (i think that was kb kindof was talking about anyways).. but sure, ill have a look what takes more time, just for fun. kb: got any stats from yours compared to traditional png loading?
mega: I did raw reading tests on plain old win32, and mmapped beats standard methods (mmap < win32 file IO < C stdio).
There's not much difference between win32 and stdio, but mmapped basically beats the other two by a factor of 4x (for both random access to small portions of files, to reading whole file sequentially).
The other point is that usually (except for vector art :)) dds compressed textures are smaller than pngs (but larger than jpgs, true). Probably teh best option would be dxtn compress + zip/lzma/whatever compress them, then read whole thing via mmap, decompress directly into hardware surface.
There's not much difference between win32 and stdio, but mmapped basically beats the other two by a factor of 4x (for both random access to small portions of files, to reading whole file sequentially).
The other point is that usually (except for vector art :)) dds compressed textures are smaller than pngs (but larger than jpgs, true). Probably teh best option would be dxtn compress + zip/lzma/whatever compress them, then read whole thing via mmap, decompress directly into hardware surface.
you people are strange. i find loaderbars sexy. yeah, you heard me - sexy.
Finally, an important subject raised up again! Ps, I wondered exactly the same years ago. Now that we have the CPU power and can calculate realtime complex stuff at 60frames per second, why would it make much more for pregenerating a bunch of textures and models?
Than again I am not sure about the answer. I think I'll have to try coding a modern 3d accel demo and load a large ammount of data in my gfx card and then I will know for sure.. ;P
Than again I am not sure about the answer. I think I'll have to try coding a modern 3d accel demo and load a large ammount of data in my gfx card and then I will know for sure.. ;P
interesting point, kb. I'm not really up-to-date with pc 3d demo programming anymore, but i was under the impression, it would work on pc as on xbox they often load stuff without 3d getting choppy... well, i guess that's another reason why i gave up pc gaming some years ago and mostly play on consoles nowadays :)
NeARAZ:
Youre telling me its faster to read a big file via mmapping than with CreateFile under win32?
Then seriously... Something is FUCKED UP with CreateFile.
Youre telling me its faster to read a big file via mmapping than with CreateFile under win32?
Then seriously... Something is FUCKED UP with CreateFile.
I really wonder where the original bandwidths are going.
Modern harddisks can load at least 15MB/sec or even more.
Even AGP 4X is able to transfer 2GB/s of textures/geometry. Internal, the speed of transferring data is often much much faster, even if most games don't even use the full bandwidth! Many Demo's have a limitation to somewhat 16MB. Decrunching, even if its very complex, not more than like 6 or maximum 10 seconds. And when we see in the caphic card's producers info's about certain graphic cards on how many mega and gigatexels the cards can do without even using the CPU much.
It makes me really wonder what the average PC Demo does all the time, when the result is not even using new Techniques like Normalmapping, Pixelshaders and stuff like that.
In this case, Loading / Precalculaing is outdated now.....
Modern harddisks can load at least 15MB/sec or even more.
Even AGP 4X is able to transfer 2GB/s of textures/geometry. Internal, the speed of transferring data is often much much faster, even if most games don't even use the full bandwidth! Many Demo's have a limitation to somewhat 16MB. Decrunching, even if its very complex, not more than like 6 or maximum 10 seconds. And when we see in the caphic card's producers info's about certain graphic cards on how many mega and gigatexels the cards can do without even using the CPU much.
It makes me really wonder what the average PC Demo does all the time, when the result is not even using new Techniques like Normalmapping, Pixelshaders and stuff like that.
In this case, Loading / Precalculaing is outdated now.....
Quote:
you people are strange. i find loaderbars sexy. yeah, you heard me - sexy.
You are hereby sandwiched between furries and people who gorge in their own extrete.
Exin:
First of all, forget about theory about speed, demos and programming in general are about practicing it. And second of all, if you really calculate of the "speed", you will see its not so good numbers anymore, even in theory.
"It makes me really wonder what the average PC Demo does all the time, when the result is not even using new Techniques like Normalmapping, Pixelshaders and stuff like that."
Seriously, seen a demo without per-pixel lightning that won a compo the last three years?
First of all, forget about theory about speed, demos and programming in general are about practicing it. And second of all, if you really calculate of the "speed", you will see its not so good numbers anymore, even in theory.
"It makes me really wonder what the average PC Demo does all the time, when the result is not even using new Techniques like Normalmapping, Pixelshaders and stuff like that."
Seriously, seen a demo without per-pixel lightning that won a compo the last three years?
I don't know why, and I'm not OS bashing, but our demo "autumn dreams" took a long time to start in windows (I don't really remember but let's say 10 seconds) and started instantly in linux.
I don't know why exactly, but when you are using multiple os's, you don't really want to code OS specific code, at least as little as possible.
thec
I don't know why exactly, but when you are using multiple os's, you don't really want to code OS specific code, at least as little as possible.
thec
I wonder why nobody yelled "Tom Thumb" yet...
thec, well.. either you had fucked up compiler flags or otherwise made stupid stuff, or you used FILE * ... pick your bug ;)
turm: couz its an freesource animation not a demo.
Most probably people are using PNG for demo graphics, which is great for compression but takes a lot of cpu to decrunch (gotta love my g4@1.25 afterburning it's fans)... but yes back in the time people used to precalc tables on the free rastertime of the previous effect (cf. Sanity Operating System demos like WOC92, Arte or Roots)
PNG takes time to decrunch? Wtf have you been smoking?
I just timed about 6 seconds decrunching for a 4096x4096x8bpp (22meg filesize) image using standard OS-provided png decruncher in Mac OSX 10.3, running on g4@1.25ghz
i assume its libpng in osx.. but i dont know.. and you ONLY decoded it? NOT show it? since your system specs sounds dangerously similair to an old laptop from Apple with rather bad graphics drivers.
oh, and btw.. can you email the image? stefan@mac.scene.org
kb said it: optimized game loader code loads 50mb level code in 8secs... but wtf... there's that size limit thing :) so if you have a lot of texture data plain dxt is just not an option. and doing halfways decent dxt compression is really, really slow. so what you have to do is "decode jpg, encode dxt, upload". even worse for normal maps which you want to store as heightmaps (because they compress much better and compression artifacts are less), then nmap, normalize and then quantisize. the bandwidth of the hd transfer is really insignificant there. just a lot of cpu work.
same goes for geometry data. it can grow huge if you just store it raw. so you need various quantizisers and you want to calculate stuff that a game stores during load time (tangents, vertex splits, normals etc).
And well, this all adds up :)
One usually can get a good factor of 2-4 speedup by handoptimizing most of the loading routines... but then that is a LOT of work i'd rather spend to increase compression quality rather than speed.
Doing stuff in the background is not that good an idea either. What if you do not finnish in time with decrunching to show it? The only option is to have a "prepare for part 2" kind of thing, but i'd rather endure a longer loading bar than silly interuptions.
same goes for geometry data. it can grow huge if you just store it raw. so you need various quantizisers and you want to calculate stuff that a game stores during load time (tangents, vertex splits, normals etc).
And well, this all adds up :)
One usually can get a good factor of 2-4 speedup by handoptimizing most of the loading routines... but then that is a LOT of work i'd rather spend to increase compression quality rather than speed.
Doing stuff in the background is not that good an idea either. What if you do not finnish in time with decrunching to show it? The only option is to have a "prepare for part 2" kind of thing, but i'd rather endure a longer loading bar than silly interuptions.
without multitasking multiusersystems - loading in the background would rock.
megadelux, pls check your mail... I tried looking for it again on vterrain.org but no luck... the site is huge :)
waiting for a 10-minute loaderbar is always better than 10 minutes without a loaderbar imho.
I guess a lot of the loadingtime these days goes on computing mipmaps. gluBuild2DMipmaps f.e.
