Greetings All!

I'm currently in the process of design an implementation of a multi-threaded texture streaming system. It's for my quasi-open-world 2D side scroller game & I have a few concerns regarding how I hope it's going to work..

Currently I'm looking at a setup like this..

The main game runs on the main thread & on initialisation, I setup a second thread to run in the background as my texture streaming thread.. My spacial sub-division system (which segments the world into finite cells, each tentatively the dimensions of 3xscreenWidth by 3xScreenHeight & with 3 cells present in memory at any one time.. ) will run at the start of my update loop (on the main thread), check the camera position, direction of movement & velocity & then adjust the current loaded cells by doing the following:

(e.g. Player is moving to the right..)

- If the camera reaches the bounding edge of middle cell/left most edge of rightmost cell do the following;

• Compile a list of all the textures to load in for the next cell (from the LevelDefinition obj)
• Grab the cell ID & setup a LoadRequestDataStructure
• Push the LoadRequestDataStructure on the end of the stream load queue
• Create & setup the new cell, & set all the entities within it to render as & when their textures are available/loaded
  
• Compile a list of all the textures in the left most cell (the one we've moved far away from)
  
• Grab the cell ID & setup a DisposeRequestDataStructure
• Push the DisposeRequestDataStructure on the end of the stream dispose queue
• Destroy the old cell
• Rinse, Repeat
  

- From here the texture streaming thread will do as follows;

• Check the stream load queue for a LoadRequestDataStructure (if none then skip nest three points)
  
• If there is one then pop it off the queue
• Create a ContentManager & run a load ThreadTask with it to load in all the textures specific to the cell ID (check the LevelDefinition)
• Once loaded, send a LoadRequestCompleted callback to tell the main thread it's done
• Check the stream dispose queue for a DisposeRequestDataStructure (if none then sleep till next run..)
• If there is one then pop it off the queue
• find the ContentManager for the cell ID & dispose of it
• once done, send a DisposeRequestCompleted callback in case the main thread cares

So this is what I have so far however i'm concerned that there are going to be some issues relating to the implementation so I need some clarification on them:

- Is it ok to create & destroy multiple content managers in this way & will it affect performance?
- Is it ok to pass information this way between threads & will it affect performance?
- Is there anything I need to be aware of when dealing with accessing Texture2Ds created by a seperate thread's content manager?
- I believe ContentManagers support load redundancy (i.e. trying to load a texture thats already in memory will just assign it & not load it into memory again..) however if each cell has a seperate content manager then will this still be the case (across seperate contentmanager loads?)
- Are there any other 'gotchas' with the design that I need to be aware of before continuing to implement everything..?

Thanks guys!

---

My Game Blog: http://houseofsmash.blogspot.com
My Blog: http://watchhogstories.wordpress.com

Why are you creating multiple content managers? You shouldn't need to.

---

Jim Perry
Here's what I'm up to.

That approach sounds feasible to me. There are no problems handing objects between threads as long as you make sure you lock whatever data structures you are using to perform this exchange (ie. never have one thread adding to a list at the same time as a different thread could be using that list).

ContentManager will merge duplicate load requests, but only within a given ContentManager. If you create a separate manager per tile, you won't get any merging across tiles. You can customize this if you want some other behavior, though.

---

--
XNA Framework Developer
blog - homepage

Thanks for the replies..

Machaira:

Why are you creating multiple content managers? You shouldn't need to.

I was under the impression that the only way to dispose of textures in memory would be to call the dispose function on the content manager which loaded all the textures you wish to eliminate..

Is there a way in which I can just dispose of textures individually (which would make my life easier in that I'd only ever have to manage a single content manager..)

---

My Game Blog: http://houseofsmash.blogspot.com
My Blog: http://watchhogstories.wordpress.com

Shawn Hargreaves:

That approach sounds feasible to me. There are no problems handing objects between threads as long as you make sure you lock whatever data structures you are using to perform this exchange (ie. never have one thread adding to a list at the same time as a different thread could be using that list).

ContentManager will merge duplicate load requests, but only within a given ContentManager. If you create a separate manager per tile, you won't get any merging across tiles. You can customize this if you want some other behavior, though.

Awesome thanks for the headsup shawn!

I figures locking the queue before adding/removing items would be a necessity so as to aleviate race conditions & i've already tested loading a single file on a background thread successfully.. I just need to scale it up now to handle everything I want to handle..

---

My Game Blog: http://houseofsmash.blogspot.com
My Blog: http://watchhogstories.wordpress.com

newclyusinju:

Is there a way in which I can just dispose of textures individually (which would make my life easier in that I'd only ever have to manage a single content manager..)

Texture2D.Dispose()? I use it for all my on-the-fly generated textures (ie texture atlases) and never textures I load via the content manager, but I don't believe it'll cause any issues if you call it directly as long as you are sure to check for IsDisposed on textures that it can be called on.

---

vbGORE - open-source VB6 multiplayer ORPG engine

Spodi:

newclyusinju:

Is there a way in which I can just dispose of textures individually (which would make my life easier in that I'd only ever have to manage a single content manager..)

Texture2D.Dispose()? I use it for all my on-the-fly generated textures (ie texture atlases) and never textures I load via the content manager, but I don't believe it'll cause any issues if you call it directly as long as you are sure to check for IsDisposed on textures that it can be called on.

Awesome! I don't know why I missed that!

Thanks

---

My Game Blog: http://houseofsmash.blogspot.com
My Blog: http://watchhogstories.wordpress.com

Disposing a texture won't do what you want if you loaded it through the content manager.

Calling Dispose on an object just frees any native resources that it might be using, but does not reclaim the managed object memory. The actual memory can only ever be reclaimed once the garbage collector notices that nobody is referencing it any more.

If you load an asset through the content manager, it holds a reference to that asset, so the garbage collector can never reclaim the memory. Worse, if you then manually dispose that object, it will now be in an invalid state, but the content manager will still be holding a reference to the disposed memory, so if you later try to reload that same asset, you'll just get back a reference to this previous disposed instance, which won't do what you want.

If you load something through the content manager, you must unload it through the content manager as well, not just by manually disposing it. If you need to customize the rules for how things get unloaded, you should do that by extending the content manager to implement whatever custom lifespan management policy you might desire.

---

--
XNA Framework Developer
blog - homepage