First, a little background: I have tried probably six different exporters for Maya to try to get models into a format that can even import correctly into XNA (I'm on Vista x64 w/ Maya 2k9 x64). Basic models work fine - anything moderately complex would blow up. I've been trying to find a solid workflow for about a year now. Last week I finally came across a few MEL (maya scripting) commands that would allow me to get the information I would need out of Maya to write a Custom Importer.

The process: I wrote a custom importer which allows me to keep my original Maya files in the project solution (huge plus -- I get source control now, automatic triangulation, plus I don't have to worry about exporting files anymore).  The importer launches a instance of the maya.exe file in batch mode (as a console application) and I redirect standard I/O into my importer. I then execute a series of MEL commands to get a list of meshes, joints, etc. with their underlying data (verts, normals, uvs, colors, etc.). Everything works beautifully, and I now have a solid, reliable way to get data from Maya into XNA. Side note: It takes much longer to open Maya than it does to execute my scripts -- anyway to get the build process to let me keep one "global" copy of Maya open that multiple importers can attach to?

As I've been adding features I came to the realization that I think I'm adding code that would normally be in a Content Processor in my Content Importer. I love the XNA development team -- but guys, your documentation skills are an epic fail. From how it seems, and I can be completely wrong here, is that the real role of a Content Importer is to extract all information from the source asset and put it into ContentNodes (and various subclasses). Then it would seem that the Content Processor (which it should be obvious I haven't written one yet) picks and chooses that information it is given from the Content Importer and serializes it into the run-time format.

If I am correct on that assessment, I have a lot of refactoring to do. How accurate is a Content Importer supposed to represent the data? Should I try to create a 1-1 mapping between a Maya Node and a ContentNode, if appropriate? As it is right now - I am "Flattening" the tree so all of my models are direct children of the root ContentNode. Should I be leaving all of the extra nodes in there and let the Content Processor decide how it wants to structure it?

As far as structuring the tree of ContentNodes - I have yet another question. I found on another post that BoneContent should be a direct child of the MeshContent using it. What happens if you have multiple MeshContents using the same skeleton - do I duplicate the skeleton and animation data - one for each MeshContent?

I realize there's about 10 questions in here but any feedback would be greatly appreciated. A simple summary of the roles of what (in detail please) the Content Importer and Content Processor should do and not do would be a great summary. Thanks.

KentDub:

wrote a custom importer which allows me to keep my original Maya files in the project solution (huge plus -- I get source control now, automatic triangulation, plus I don't have to worry about exporting files anymore).

That is just awesomely cool!

KentDub:

From how it seems, and I can be completely wrong here, is that the real role of a Content Importer is to extract all information from the source asset and put it into ContentNodes (and various subclasses). Then it would seem that the Content Processor (which it should be obvious I haven't written one yet) picks and chooses that information it is given from the Content Importer and serializes it into the run-time format.

Exactly right. The role of the importer is to pull data from the source tool into a standard, normalized format, so it removes any quirks that are specific to the source tool. Then the processor turns this normalized data model into whatever more specialized format the runtime game engine requires.

One way to think of this:

• Importer = specific to source modelling tool or file format
• Processor = specific to game engine

The benefit of separating the two is that as long as everyone uses the same content object model (NodeContent etc) in between them, this can make the same importer able to be used with many different game engines (for instance our Custom Model Class sample is able to use the same FbxImporter as our built-in Model class), and also the same game engine can be used with many different importers (for instance many of our samples use a mixture of both .X and FBX format models, but their processors don't need to care which format the input data came from).

Of course, this is just a principle, and you don't have to follow it. If you wanted, you could do all the work in the importer, have it output a game-ready object, and not need a processor at all. Or your importer could be a no-op that just returned the filename string it was given, and then you could have a processor that did all the work of reading the contents of that file. Either of those designs loses the benefit of being able to mix and match different importers and processors, but that's ok if you don't care about mixing and matching.

KentDub:

If I am correct on that assessment, I have a lot of refactoring to do. How accurate is a Content Importer supposed to represent the data? Should I try to create a 1-1 mapping between a Maya Node and a ContentNode, if appropriate? As it is right now - I am "Flattening" the tree so all of my models are direct children of the root ContentNode. Should I be leaving all of the extra nodes in there and let the Content Processor decide how it wants to structure it?

Our FBX and X importers preserve the node tree from the input model, but this is really up to you. Preserving the hierarchy is useful if you want to do hierarchical things with the data, especially animation where you may have jointed objects parented off of other jointed objects.

KentDub:

As far as structuring the tree of ContentNodes - I have yet another question. I found on another post that BoneContent should be a direct child of the MeshContent using it. What happens if you have multiple MeshContents using the same skeleton - do I duplicate the skeleton and animation data - one for each MeshContent?

You should typically have something like:

- NodeContent (scene root)
|-- MeshContent (first model)
|-- MeshContent (second model)
|-- MeshContent (third model)
|-- BoneContent (skeleton root)
     |-- BoneContent (head bone)
     |-- BoneContent (left arm)
          |-- BoneContent (left wrist)
etc.

Thanks for the quick response Shawn. That is the exact information I was looking for. I am planning on keeping the importer sperate from the processor - as I eventually will be creating several different content processors to import different data from Maya (Such as a typical Model, an AnimatedModel, or maybe even a Level).

I would make the importer extract as much data from Maya as you know how to put into Content nodes, and stop there. Retain hierarchy, additional vertex channels, etc.
Then, the Content Processor would do things like remove hierarchy, if you don't want it. The bonus with that is that you can easily apply that same hierarchy-removing processor to a .X or .FBX file you import. Similarly, you can import a hierarchical file (as needed for a skeleton, or an articulating assembly) from Maya as long as the importer retains hierarchy.

If there is no DLL version of Maya that you can load and P/Invoke in your processor, then I know of no way to accelerate the Maya starting. Instead, I would suggest bundling all the MEL script into a single exporter that just spits out a big file with all the data in it. Call that your "foo" file format. Then make your importer capable of importing "foo" files instead of "ma" files.

An alternative is to do a two-stage build: Keep adding the ma file, but when you process the file, store the output into a cache file. When your importer gets called, check whether the cache file is newer on disk than the ma file; if it is, just read the cache file (in whatever format the mel script would output). Then proceed to build the content nodes from there. That way, re-builds that happen because you change code won't require re-invoking Maya.

Thanks JWatte. I have the CacheImportedData attribute property set to true, and it couldn't work better.

I did find that Maya has a "command port" -- which is basically a listening socket that you can connect to and execute MEL commands on. I was thinking I could use that and startup maya in the background with a prebuild command and shut it down with a postbuild command. It's not nearly as clean in my opinion, but it would save quite a bit of time for a full rebuild of a project with a lot of Maya files in it. The major downside of doing this is if you already had Maya open it would have ownership of the default port, and the content importer would takeover your Maya window and not use its own.... Maybe there is an argument I can pass to Maya to get it to switch to a non-standard port.... If I went this route it would almost defeat the purpose of the CacheImportedData attribute property as Maya would be launched no matter what. Sorry to keep rambling on but ideas keep jumping into my head... I suppose I could even parse the Content project file to grab a list of Maya Files and compare the dates of them against the cache files XNA creates... That way I could determine if I should shutdown Maya or leave it running... (Basically use the sockets idea but without the pre/post build commands).... I'm starting to like that idea..

I can extract an amazing out of data about each node - in fact Maya creates almost 300 to 400 default attributes for a simple node. I was going to grab all of the current values and dump them into the opaque data collection, which would allow some pretty cool processing for any content processors that looked for the right keys (I believe that's the right place to put all of that). This would be essential if I was to allow importing of intermediate objects or invisible objects and allow the processor to decide what to do with them. This would also enable users to put custom attributes onto a node and get access to them in their content processor.

BTW: I have access to a bunch of data that I haven't seen an obvious use for (yet)... FaceNormals (Right now I'm just bringing in VertexFace normals), Bi-normals, etc.... Is it worth importing this? I think I might have seen something about tangents also, which if I remember correctly is necessary for normal mapping...

Thanks a lot, I really appreciate the guidance. Once I get this a little more stabilized I'll release the code for the community. I've been playing with XNA for about two years now and just now am I able to consistantly get models from Maya all the way to drawing them without issue.

Face normals are useful in specialized cases (like for collision geometry). However, if you add face normals to your vertex buffer, it will split many more vertices for normals, so the model will end up drawing less efficiently -- even if you don't end up using them in the shader.

Face normals are also easy to recalculate from the vertex positions if you ever need them for such things. Unlike vertex normals (which are a somewhat fuzzy concept that can be calculated in several different ways depending on what aesthetic result you are going for, and in some cases may even be manually tweaked by artists) face normals are a simple and well defined mathematical construct, so there isn't really any need to bother storing them.

Thank you all for your feedback, it has been invaluable.  I'm hard at work refractoring the code to accomidate the suggestions.

So far I have working:

• Can automatically triangulate meshes
• Can import a scene with multiple meshes
• Can import multiple color sets
• Can import multiple uv sets
• Can import skeleton data
• Can import weighting information
• Can import tangent data
• Can generate binormal data
• Can import basic material information
• Can import basic material texture maps
• Can import opaque data
• Generates a fair amount of warnings when it detects your Maya settings arn't exactly XNA friendly, but still tries to import, will be very useful for if you forget something or are newer to XNA and can't quite get the models in correctly

I'm still working on:

• Importing animation clips (Have the basics working, but it still needs a lot of work)
• Importing HLSL shaders

Performance is definatly an issue with complex or large scenes, but at this point I'm just extremely happy that everything is getting into the pipeline correctly. At some point I may try to find someone who can write a Maya plugin so I can have the mel scripts reproduced as compiled code. For now, I'm going to mark this thread as answered =)