Hey all, after rewriting my rendering and scene management code many times to support multiple render passes, etc, I ended up doing the following so far.  Am I on the right track here?  I still find it difficult to add new render techniques especially as multiple passes are required.

For example, right now I have a sperate render queue for each pass which seems kinda hackish. I'm in the process of supporting shadow maps right now.  And eventually I'm going to need to some sort of occlusion culling.  Any tips on doing a good renderer design/engineering and getting good performance?  Is it worth trying to batch up geometry in my render queues to lower the number of draw calls?  And I still not sure how I can manage lights in the renderer. (hard-coding lights so far).

During game logic pass:
If a game object wants to render something, it add a scene node to the scene graph (which has a parent-child relationship).  Node could be a transform node (doesn't render anything, just manipulates world matrix) or something visible like terrain or model or some effect.  Scene nodes persist from frame-to-frame, game objects are responsible for updating them during game logic pass.  Each scene node also belongs to a node within a quadtree that subdivides game world.

During render scene pass (per frame):
1. Walk down the scene graph and update the world matrix for each node. This also updates the scene node's bounding box (in world-space) and its quadtree node membership (if it moved outside the quadtree node into another node)

2. Walk down the quadtree and go into nodes that are within the current camera's viewing frustum, then for each scene node belonging to the quad tree node, check if the scene node's bounding box intersects the view frustum.  If it does, call the scene node's Render() method.

2a. In the scene node's Render() method, it takes a Geometry object that includes a vertex/index buffer, and a Material (wrapper over Effects that sets up the shader parameters and render state).  This Geometry object is enqueued into one or more priotity queues in the renderer.  The priority is a number that determines the order it'll be rendered, but I'm not really taking advantage of this yet.  I'm assuming I should prioritize geometry that's closer to the camera first?  I have separate queues for reflection and refraction map in addition to the main scene queue, and I'm sure I'll be adding more.

3. Render the reflection and refraction map queues to their own render targets.
3a. This includes setting a new render target, and emptying the geometry queue into a bunch of DrawIndexedPrimitives() calls.

4. Render the main scene geometry queue. Render target here is also off-screen in a post processor texture.

5.  Do the 2-D render pass (goes straight to frame buffer)
5a.  Renders the post processor render target as a sprite that fills the whole screen with optional shader effect.
5b.  Render any extra sprites (I keep a list of sprites separate from the scene graph) and debug info

Sorry for the long post, I'm hoping someone has an opinion on this design, if it sucks or I'm on the right track. Thanks.

Personally I'm not a fan of generic node-based scene graphics, I find they're more useful in 3d editors like Maya than in a game engine, but that's more a personal preference.

Overall your design is fine, and batching as much as possible is always a good thing.  There are a few things I'd change.

1) Update world matrices in your Update() - unless you copy the matrices from a game state object into your scene graph.  The Update/Draw separation is a good idea, and IMO simulation state and presentation state should be as separated as possible.

2) There's no need to re-add objects each frame.  Add objects to their respective queues when you create them.  Then when you traverse your quadtree simply update the object's Visible property. 

If you do need to re-arrange queues do it with a single Sort operation rather than a priority queue.  Normally the only reason to re-sort is if you want to sort Z-depth, but usually only particles/alpha passes need that.  Although sometimes Z-sorting can help with due to early Z-rejection.

3) You may want to generalize your passes so that a Pass object has a Begin/End where the per-pass data is set (Camera matrices, projection matrices, rendertargets etc).  This give a very flexible system, where you have ShadowPass, MainPass, UIPass, PostFXPass, etc..  Each pass would potentially update the Visibility of it's own queue, this is neccessary because objects that are visible in your shadow pass may not be visible in your main pass and vice versa.  (e.g. A tree casting shadow onto the ground, but the tree is off screen)

There's a pretty good series of articles here which deal with the same type of system you're planning:
http://www.beyond3d.com/content/articles/98/
http://www.beyond3d.com/content/articles/102/
http://www.beyond3d.com/content/articles/109

Thanks a lot for the input, I'll take a look at the links too.  I agree that the abstract scene graph approach may be sort of overengineering, it seemed like a good idea at the time.  But in reality my scene graph is very flat and if I needed to parent one object transform to another, I could probably just accomplish that more efficiently in the game logic.

Yes, in theory scene graphs sound like a nice way to organize data, lots of academic literature and 3d modeling tools use it.  However, like you've noted, a game scene is usually fairly flat.  There are complex node hierarchies but they are generally internal to model data.  I've found very little need to for any graph depth greater than "Character holding object", and I'd just manage that in the Character logic code.

I agree with fr3shme4t. I stopped using a general scenegraph around 2 years ago. Instantly a whole bunch of problems disappeared and now I just code the Lists and relationships as I need them.