Collada?
category: code [glöplog]
I'm looking for an mesh export format from that would be effective in a data path starting from maya/3ds max and ending up in our runtime engine. I need something quite general, including meshes, materials, skinning, IK-chains, materials.. the lot really.
I have been looking at the Collada format, but I am wondering about people's experiences with using exporters like openCollada to get from Maya to .dae. I am trying to avoid spending alot of time parsing, say, a collada file only to find out that the exporter plugin I just installed with Maya is absolute garbage.
What good formats and tools are out there? Any thoughts are welcome.
I have been looking at the Collada format, but I am wondering about people's experiences with using exporters like openCollada to get from Maya to .dae. I am trying to avoid spending alot of time parsing, say, a collada file only to find out that the exporter plugin I just installed with Maya is absolute garbage.
What good formats and tools are out there? Any thoughts are welcome.
I don't know about the Maya exporter but for 3dsmax we're switching a bit back and forth between the autodesk exporter and the opencollada exporter. I don't remember the details but they both have their problems (and are producing a slightly different format), but opencollada is usually preferred.
Not sure what else to use if you also need skinning etc (which we don't).. it's probably your best option after all then..
Not sure what else to use if you also need skinning etc (which we don't).. it's probably your best option after all then..
FBX.
Hyde: I can recommend COLLADA as a asset-interchange format (and pre-build storage), but you should consider using something else at run-time; COLLADA is both bloated storage-wise, and heavy to parse (compared to simpler alternatives). You'll get big distributions with long loading-times if you go down that path. I've used some simpler XML-formats for scene-description plus binary blobs for vertex/index buffers with relatively good results before.
i've had less problems with FBX compared to Collada
and some guy kindly provided a dx10 fbx viewer with sourcecode:
http://dougrogers.blogspot.com/2008/03/fbx-loader-for-directx.html
Reads and renders .fbx meshes, texture, material and animation
creates the frame hierarchy
processes layer data : materials, textures, texture coordinates
resamples animation into DirectX
vertex shader to render fbx skinning
creates meshes, animation controllers, animation keys, bones
uses the same interfaces and .x files
creates mesh attributes from the material and textures assigned to polygon
can render thumnails of the first animation frame. see below
can create multiple instances for animating one or more copies of a model
and some guy kindly provided a dx10 fbx viewer with sourcecode:
http://dougrogers.blogspot.com/2008/03/fbx-loader-for-directx.html
Reads and renders .fbx meshes, texture, material and animation
creates the frame hierarchy
processes layer data : materials, textures, texture coordinates
resamples animation into DirectX
vertex shader to render fbx skinning
creates meshes, animation controllers, animation keys, bones
uses the same interfaces and .x files
creates mesh attributes from the material and textures assigned to polygon
can render thumnails of the first animation frame. see below
can create multiple instances for animating one or more copies of a model
pantaloon: unfortunately the source is not available on google code anymore :( do you have a copy somewhere?
I would go with FBX instead of Collada.
kusma: that's also how we store 3d data internally (and have some exporters to) - xml file with the structure and pointers into a big binary blob for vertex/index buffers, both packed in a common .zip.
My experience with FBX is that it seems to be a superset of all autodesk products 3D save formats, so it's quite hard to support everything and even the autodesk products themselves don't - so much for interchange. Then I'd rather have a simpler format, requiring the built in exporter to do the transformations to that. FBX is btw not read directly, but (only?) through the FBX SDK.
But only working with the Maya FBX output is probably a lot easier.
My experience with FBX is that it seems to be a superset of all autodesk products 3D save formats, so it's quite hard to support everything and even the autodesk products themselves don't - so much for interchange. Then I'd rather have a simpler format, requiring the built in exporter to do the transformations to that. FBX is btw not read directly, but (only?) through the FBX SDK.
But only working with the Maya FBX output is probably a lot easier.
as the one whos been in charge of the collada exporters for max and maya in the ps3 sdk for the past few years I can speak with some experience..
collada is broken. they didnt specify it very well, so its incredibly wooly. it means that as an interchange format its bust.. the hope of any prog loading any collada file is remote at best when you go past basic data. apparently its officially not an interchange fmt tho anymore (god knows what it actually is)..
however, if you use and fix one exporter and write an importer for it, as we have, it works. we add stuff as needed and fix as needed. it means we deviate from spec a bit but hey - it works. we have max and maya exporters whoich both import well into our tools but interchange. - max to maya etc - is much more problematic.. under the hood those beasts are very different.
if you have access to them, the ps3 sdk exporters should be a lot more solid than the opencollada ones. as for the autodesk collada exporters - dont even bother. :)
collada is broken. they didnt specify it very well, so its incredibly wooly. it means that as an interchange format its bust.. the hope of any prog loading any collada file is remote at best when you go past basic data. apparently its officially not an interchange fmt tho anymore (god knows what it actually is)..
however, if you use and fix one exporter and write an importer for it, as we have, it works. we add stuff as needed and fix as needed. it means we deviate from spec a bit but hey - it works. we have max and maya exporters whoich both import well into our tools but interchange. - max to maya etc - is much more problematic.. under the hood those beasts are very different.
if you have access to them, the ps3 sdk exporters should be a lot more solid than the opencollada ones. as for the autodesk collada exporters - dont even bother. :)
Smash: It's time to relase parts of PS3 sdk as Open Source then. :)
thanks everyone. I was on my way in the direction that kusma mentions, aiming to use collada in the data path but not in the runtime itself. Now I am not so sure what to do. It would be great having time to write the software in both ends, but it would be even greater to not have to think about maintaining exporters at all..
I will take a look at FBX even though it is closed and non-xml and stuff, and do some comparing.
Thanks again.
I will take a look at FBX even though it is closed and non-xml and stuff, and do some comparing.
Thanks again.
does it really _have to_ be an interchangable format? i mean, i experimented quite a lot with coding mesh/material importers from numerous formats, but what really satisfied me in the end was to write my own import/export-plugin for max and lightwave (don't know if it's that more difficult for maya, suppose not).
because in the end what you as a coder really care about is data structure and loading time, and IMHO that's best defined and most efficient if you crafted the data format yourself and made it fit to your needs. there's just so much aspects (such as, for example, defining arbitrary material properties that are new/uncommon to well-established formats) that turn out to be a breeze when done in your own format compared to poking it into some general purpose one. also, at least for demos, it's nice to know that you're not wasting tons of megabytes on unnecessary meta-information you'll never use in the first place.
because in the end what you as a coder really care about is data structure and loading time, and IMHO that's best defined and most efficient if you crafted the data format yourself and made it fit to your needs. there's just so much aspects (such as, for example, defining arbitrary material properties that are new/uncommon to well-established formats) that turn out to be a breeze when done in your own format compared to poking it into some general purpose one. also, at least for demos, it's nice to know that you're not wasting tons of megabytes on unnecessary meta-information you'll never use in the first place.
Major players in the cg industry seem to be pushing alembic really hard. However I am not a coder, so I can't really judge if it's useful for demoscene purposes.
http://code.google.com/p/alembic/
"Alembic is designed to be very efficient with its use of memory and disk space. Much effort has been directed towards automatic and transparent mechanisms for data de-duplication and reading/writing as little data as possible in order to fully represent the data that has been given to it."
Doesn't sound too bad to me anyway, so what's your opinion on that format, people in-the-know?
http://code.google.com/p/alembic/
"Alembic is designed to be very efficient with its use of memory and disk space. Much effort has been directed towards automatic and transparent mechanisms for data de-duplication and reading/writing as little data as possible in order to fully represent the data that has been given to it."
Doesn't sound too bad to me anyway, so what's your opinion on that format, people in-the-know?
we use lightwave data exclusively in our demo tools and we load it natively - i.e. lwo/lws files directly. this is awesome for several reasons:
1. our artists (and .. me) use lw so it makes sense to support them as good as possible
2. we dont need an exporter. if we need a new feature we never need to change the exporter, reexport all the data - the data is always "just there", we just have to change the loader to find it. and we never, ever have exporter bugs. :)
3. we can tailor our engine to support lightwave closely. 3d programs are all pretty different when it comes to animations, rotations etc.. we dont have to be generic and support all of them, we just have to support the lw way. in fact our own node graph and animation code in demo tool works the same way for animation and transforms as lightwave. easy fit!
4. if we do need maya data, i wrote a nice maya to lightwave exporter which handles animation, nodes, skins etc. (if youre going to export to a format, why not use a preexisting one with good reference tools huh..?). i might actually release that one day if there is demand.
if you're working with an interchange format, the best way - or, the only way - is to make a tailored path from program a -> program b. generic virtually never works once you get beyond the basics of meshes.. when you get into animation, skins etc it gets nasty.
1. our artists (and .. me) use lw so it makes sense to support them as good as possible
2. we dont need an exporter. if we need a new feature we never need to change the exporter, reexport all the data - the data is always "just there", we just have to change the loader to find it. and we never, ever have exporter bugs. :)
3. we can tailor our engine to support lightwave closely. 3d programs are all pretty different when it comes to animations, rotations etc.. we dont have to be generic and support all of them, we just have to support the lw way. in fact our own node graph and animation code in demo tool works the same way for animation and transforms as lightwave. easy fit!
4. if we do need maya data, i wrote a nice maya to lightwave exporter which handles animation, nodes, skins etc. (if youre going to export to a format, why not use a preexisting one with good reference tools huh..?). i might actually release that one day if there is demand.
if you're working with an interchange format, the best way - or, the only way - is to make a tailored path from program a -> program b. generic virtually never works once you get beyond the basics of meshes.. when you get into animation, skins etc it gets nasty.
(btw, altho we load lwo/lws files natively we process them into binary data dumps for the final demo package for fast loading)
I use blender / cal3d. But the exporter was pretty hackish and I'm currently the maintainer of it (it seems). It's been removed from the standard blender dist. But it still works. I don't export animations or lighting, that needs to be tweaked closer to the code IMO. So it's not that horrible an exporter...
from what I've seen, collada is a pointless mess.
In principle I'd agree with smash, writing different readers of formats is better.
However: how much data in a lightwave file is padding/not used by the demo? (i.e. waste of space). I'm not one to talk though, since cal3d and obj are pretty wasteful formats :) although they compress well...
I would assume that the following can be regarded as pointless for a demo 2011, but personally I find it always amusing to try to optimize size a bit, even for a fullblown 30MB demo...
In principle I'd agree with smash, writing different readers of formats is better.
However: how much data in a lightwave file is padding/not used by the demo? (i.e. waste of space). I'm not one to talk though, since cal3d and obj are pretty wasteful formats :) although they compress well...
I would assume that the following can be regarded as pointless for a demo 2011, but personally I find it always amusing to try to optimize size a bit, even for a fullblown 30MB demo...
smash: ...which you in turn load three times? :)
@smash: that's exactly what i did when i started using lightwave as my main modelling tool.
For the reasons mentioned above i then progressed to my own exporter (which in the beginning indeed suffered from bugs/lack of downwards compatibility, but not anymore). plus, i included an immediate viewer for my own demo-engine prior to the actual export, and it made my modelling cycle a lot faster and efficient. basically, i change little things in the model, start my exporter plugin, look at the data presented with my demo-framework and correct errors _before_ actually exporting it. so that's another advantage of having your own code running within the modeler.
For the reasons mentioned above i then progressed to my own exporter (which in the beginning indeed suffered from bugs/lack of downwards compatibility, but not anymore). plus, i included an immediate viewer for my own demo-engine prior to the actual export, and it made my modelling cycle a lot faster and efficient. basically, i change little things in the model, start my exporter plugin, look at the data presented with my demo-framework and correct errors _before_ actually exporting it. so that's another advantage of having your own code running within the modeler.
i forgot to mention that when i started my exporter plugin, there was no support for edges or polygon connectivity in the lightwave format. this made loading the .lwo files last for an unnecessarily long time when i needed that information (for stencil shadowing, btw). so there's another reason for using my own format (i admit that it's now void).
now that i've worked with my own format for quite some time i got used to the work cycle, and every modeler i need additional support for means only another day of writing the lowlevel exporter layer which gathers the mesh/material/scene data.
now that i've worked with my own format for quite some time i got used to the work cycle, and every modeler i need additional support for means only another day of writing the lowlevel exporter layer which gathers the mesh/material/scene data.
jaw: yea, thats why we dump the vertex buffer data, structures etc for use in the final demo. faster load times. altho lightwave is quite cool, it generates a lot of data like normals etc - so at times our dumped data is larger than the lwo.
The only reason why i use FBX is coz it's generally easier within the Autodesk famility to use then the native SDK,
and i use FBX to output it to the final binary format that is used by runtime...
This made me look up an old hack i added for my last pc demo, and it is still there... gotta remove it some day.
Code:
UglyHackVertex* vertices = (UglyHackVertex*) m_data;
UglyHackVertexPacked* packedVertices = new (SCRATCHPAD) UglyHackVertexPacked[vertexCount];
for (uint32 vertexIndex = 0; vertexIndex < vertexCount; ++vertexIndex)
packedVertices[vertexIndex] = packVertex(vertices[vertexIndex]);
jaw: i tried import .3ds in blender, it seems like the keyframe data is missing (or am i wrong?). would be neat to import all the .3ds data.
yeah and about nobody does that because it's a bitch to get right.. dump 3ds, write your own converter or use a crappy existing one (clax) for it.
quick lightwave question: how does a typical lw scene look on disk? .lws and multiple .lwos plus reffed texture data or what? im still downloading it but i want to prepare some code to pull it out of my resource system already ;)
quick lightwave question: how does a typical lw scene look on disk? .lws and multiple .lwos plus reffed texture data or what? im still downloading it but i want to prepare some code to pull it out of my resource system already ;)
plek: that's how it looked like back when I worked with lwo, but that's 10 years ago :P