XNA Framework

Re: Avoiding int.ToString() GC garbage

gamefool — Mon, 26 Jan 2009 09:05:00 GMT

Doh! The following will handle that boundary case:

    
                public static class StringBuilderExtensions  
        
                {  
        
                    public static StringBuilder AppendWithoutGarbage(this StringBuilder stringBuilder, int number)  
        
                    {  
        
                        if (number < 0)  
        
                        {  
        
                            stringBuilder.Append('-');  
        
                        }  
        
                        int index = stringBuilder.Length;  
        
                        do 
        
                        {  
        
                            stringBuilder.Insert(index, _digits, (number % 10) + 9, 1);  
        
                            number /= 10;  
        
                        }  
        
                        while (number != 0);  
        
                        return stringBuilder;  
        
                    }  
        
                    private static readonly char[] _digits = new char[]

                    {

                        '9', '8', '7', '6', '5', '4', '3', '2', '1', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9'

                    };  
        
                }

Re: Avoiding int.ToString() GC garbage

jwatte — Mon, 26 Jan 2009 06:15:59 GMT

Your function fails if I try to append the number -2147483648 :-)
It's the only legal "int" that it fails for, though.

Re: Avoiding int.ToString() GC garbage

gamefool — Mon, 26 Jan 2009 05:33:51 GMT

I decided to use CLR Profiler on my emulator and came across the same problem as described in this thread. I threw together a quick and dirty StringBuilder extension method that verifies it can avoid generating garbage:

    
                public static class StringBuilderExtensions  
        
                {  
        
                    public static StringBuilder AppendWithoutGarbage(this StringBuilder stringBuilder, int number)  
        
                    {  
        
                        if (number < 0)  
        
                        {  
        
                            stringBuilder.Append('-');  
        
                            number = Math.Abs(number);  
        
                        }  
        
                        int index = stringBuilder.Length;  
        
                        do 
        
                        {  
        
                            stringBuilder.Insert(index, _digits, number % 10, 1);  
        
                            number /= 10;  
        
                        }  
        
                        while (number > 0);  
        
                        return stringBuilder;  
        
                    }  
        
                    private static readonly char[] _digits = new char[] { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' };  
        
                }

For some reason the StringBuilder.Insert(Int32, Char) overload calls ToString() internally, but this isn't in the Compact Framework anyway, so I use StringBuilder.Insert(Int32, Char[], Int32, Int32). Obviously there's a performance vs garbage tradeoff ... so consider this a starting point that eliminates the garbage with the least effort.

Re: Avoiding int.ToString() GC garbage

nop — Wed, 24 Sep 2008 05:20:03 GMT

nop:
Come to think of it I should probably clean up the code and release it or something...

Well, it took me almost a month, but I finally found some time and untangled the font code from my hideous mess of a utility library and put it up on Google code. Sorry for the lack of sample project, etc, work is killing me right now and I haven't had much time to get everything I want to release in a reasonably good state. You can look at the FrameStats component if you want to get an idea of where to start with it.

Re: Avoiding int.ToString() GC garbage

The ZMan — Sat, 06 Sep 2008 16:18:24 GMT

Ego:

You can get nasty and unexpected side effects by making strings mutable via pointer manipulation or reflection. Beyond the uncertainty in your own code caused by strings being mutable, string interning may cause you to get unexpected results, and mutable strings are unreliable as keys in a Dictionary<> or similar.

Absolutly, my intention was to hide the whole thing inside another class so the implementation was totally hidden and the mutable strings themselves were never exposed. Its not perfect but it would cut down on the major issues....

Re: Avoiding int.ToString() GC garbage

Ego — Sat, 06 Sep 2008 15:16:10 GMT

You can get nasty and unexpected side effects by making strings mutable via pointer manipulation or reflection. Beyond the uncertainty in your own code caused by strings being mutable, string interning may cause you to get unexpected results, and mutable strings are unreliable as keys in a Dictionary<> or similar.

The additional of a StringBuilder overload to DrawString() etc. is great, but still not quite far enough if you're doing a lot of text manipulation. You can't use string.Format() or StringBuilder.AppendFormat() without generating garbage. You can't append individual integers, floats etc. via StringBuilder.Append() or .Insert() without generating garbage. If you want allocation-free string formatting you have to write it yourself, but even appending individual characters to a StringBuilder isn't guaranteed to happen without allocation. Moreover, that's one function call per character, and if you're dealing with large volumes of text at high frame rates, the whole thing can be rather unperformant.

Much as it's unpleasant and extreme, custom text output, formatting and word wrap based on custom memory buffers does avoid these problems. You don't break string itself, and you can get back to strings at any point. However, I still wouldn't advocate it except in extreme cases, as it's something of a pain to create.

Re: Avoiding int.ToString() GC garbage

Brandon Bloom — Sat, 06 Sep 2008 05:54:17 GMT

As suggested earlier, you could use a dictionary of int->string.

This technique is called memoization: http://en.wikipedia.org/wiki/Memoization

Dictionary healthMemos = new Dictionary();

string GetHealthString(int health)
{
string s;
if (!healthMemos.TryGetValue(health, out s))
{
s = "Health: " + health;
healthMemos[health] = s;
}
return s;
}

Re: Avoiding int.ToString() GC garbage

jwatte — Sat, 06 Sep 2008 01:47:36 GMT

That's a step forward, and perhaps enough. However, I wonder why there's no "DrawString(char[ ], int length, ...)" as that would be the easiest fall-back for many possible cases. If it were me, that's the method I would have implemented first, and then put the strings on top of that...

Re: Avoiding int.ToString() GC garbage

nop — Fri, 05 Sep 2008 22:22:57 GMT

The ZMan:
Do you use the standard font XNB files or did you have to write your own exporter too.

I rolled my own. Simple enough task really. Render character into bitmap. Trim away blank space. Pack characters into Alpha8 texture. I used interop to get at the kerning data, and stopped at calling GetCharABCWidthsFloat. Still not sure if I should go all the way and use GetKerningPairs. It hasn't really seemed necessary yet.

Most of the work was on the runtime side. When I create a font I pick a set of point sizes to render it at, and at runtime I specify font size as a fraction of the screen height and it picks the closest matching set of glyphs based on screen resolution and the available font sizes. This has been pretty useful on the XBOX because I can do things like scalling my font sizes in code by some global amount and the layout/render just works without having to have a _pc and _xbox (the same as _pc, but bigger) set of fonts. The final glyph layout stuff is dead simple but the actual line layout (word wrap, center text in rectangle, clip/don't clip, etc) is a loop of pure insanity, written with raw char pointers (since I wanted to be able to use it with either char[ ] or string data) and probably has a few bugs in it.

Re: Avoiding int.ToString() GC garbage

Shawn Hargreaves — Fri, 05 Sep 2008 22:03:38 GMT

In 3.0, you can pass a StringBuilder directly to SpriteBatch.DrawString.

You have to be careful to format into a StringBuilder without causing allocations, but this is possible if you tickle it just right, and easier than implementing your own system entirely from scratch.

Re: Avoiding int.ToString() GC garbage

The ZMan — Fri, 05 Sep 2008 21:40:03 GMT

nop:

The ZMan:
Strings are only immutable if you stay in safe code ;-)

Been there. Done that. Couldn't figure out how to set the length... The approach works fairly well, though I ended up formatting into a char[ ], and then unsafely copying into a preallocated string once the final length was known. I was basically at the point of setting up a caching system for all the different-length strings I was using when I decided to just bite the bullet and write my own font renderer (and sneak in a few extra features as well).

Off the top of my head the 1st 2 (4?) bytes are the length... but note that only enough memory is there for the initial string. So you need to make a nice long string to 'reserve' the memory. Then you can just fill it with any shorter string and reset the length.

I should tidy up the struct approach - worked pretty well in my PoC...

You are right about handling all the other stuff though. Do you use the standard font XNB files or did you have to write your own exporter too.

Re: Avoiding int.ToString() GC garbage

nop — Fri, 05 Sep 2008 21:08:06 GMT

The ZMan:
Strings are only immutable if you stay in safe code ;-)

Been there. Done that. Couldn't figure out how to set the length... The approach works fairly well, though I ended up formatting into a char[ ], and then unsafely copying into a preallocated string once the final length was known. I was basically at the point of setting up a caching system for all the different-length strings I was using when I decided to just bite the bullet and write my own font renderer (and sneak in a few extra features as well).

The ZMan:
Of course then you end up implementing your own number formating code but thats much easier than writing your own font rendering IMO

You end up doing both if you go the custom font route. Of course if you do the custom font rendering engine you can do things like rendering a font at multiple sizes and picking the closest one at runtime based on screen resolution and desired vertical height, or inserting formatting codes (just color changes in my case) into strings rather than breaking them up ahead of time and drawing them with multiple Measure/Draw calls (which can get ugly if you switch colors mid-word in oddly kerned fonts).

Re: Avoiding int.ToString() GC garbage

The ZMan — Fri, 05 Sep 2008 20:25:32 GMT

Strings are only immutable if you stay in safe code ;-) Its not too hard to work out how the string is laid out and modify it without creating garbage. Of course then you end up implementing your own number formating code but thats much easier than writing your own font rendering IMO

Another idea I got from Frank's talk at GameFest is to use a struct that overlays a byte[ with a string. Than you can modify the byte[ and pass the now modified string into the .Draw method. You don't even need unsafe code to do that. I did a proof of concept but I didn't do any serious coding.

Re: Avoiding int.ToString() GC garbage

reedake2 — Sun, 31 Aug 2008 04:32:45 GMT

sigh, sometime I miss the days of and free() and delete() lol.

Re: Avoiding int.ToString() GC garbage

Ego — Sat, 30 Aug 2008 10:06:47 GMT

Very well put Jon. I'd advocate options 1 and 2 where possible, but performance issues lead me down the dark, winding road of option 3.

The only way I've managed to avoid horrendous garbage issues when drawing large amounts of text (or small amounts of regularly-changing text; or a moderate amount of text which needs to word wrap in any sort of resizeable "window") is to discard the string data type as a means of holding displayed text, and implement my own font rendering based on efficient text buffers rather along the lines of MemoryStream (though cutting out that particular middle man). On the face of it it might seems ludicrous and backward to do so, but the performance gain has been more than quite noticable. It's a last resort, but it works.