Dapple 1.2 and NSString Leaks

Dapple 1.2

I haven’t been posting very frequently lately, mostly because I’ve been working very hard to get this Dapple 1.2 update out the door. I’m thrilled to announced that I submitted it this afternoon! Dapple and Dapple Lite 1.2 were both submitted today. If everything goes according to plan, they should be available around this time next week.

The update is quite exciting, in that it adds support for global high scores lists! Now you can see how your top score compares to other people around the world! With the update you’ll be able to see your score and the 9 people ranked directly above you, or look at the top overall scores. It’s already been exciting watching the testers’ scores roll in. I can’t wait to see how everyone does!

Dapple Lite also allows you to submit your scores online, but you’ll be using the same scoreboards as those with the full version. If you want to get near the top of the scoreboards, you’ll have to buy the full version. ;)

There are also a few user interface tweaks that I’ve made and also a few little fixes. Finally, there’s also a feature for a top secret project I’ve been involved in. You’ll have to wait for the update to find out what it is!

Leaderboards

Now I want to talk a little bit about the technology that I used for the high scores lists and then talk about a nasty memory leak I tracked down today before submitting. If you’re not interested in this kind of thing, I’ll forgive you if you want to stop reading now…

When I started thinking about implementing the leaderboards, I first looked a few existing technologies, hoping that I might be able to just “drop something in” and it would work. There are a few technologies out there that looked like the might be able to do that. Where they fell short, however, was in my ability to completely customize how I wanted to store the data and display it. Plus, I thought it would be fun to write my own system.

When I started reading up on the Google App Engine, I was convinced it would be easy to set up a server back-end using that, and I was right. Using the App Engine, I had the server-side working in about 3 days of work (and that included learning Python and how the App Engine works). It’s really easy to get things running with their system.

What took time, as usual, was the user interface. I had considered creating an in-game web view that would load a web page that had the high scores on it, but I decided (perhaps somewhat foolishly) that I didn’t want the player to have to leave the look and feel of the game. I decided to build it using my existing UI framework. This only caused a few problems, mostly around trying to find space to fit all the data. However, I also ran into some nasty string memory stuff in the code I wrote to parse the data coming from the server.

I encountered a memory leak this morning that I want to share with you, because it’s not immediately obvious why it’s a leak. When I get my data from the server, it comes in as a long string of data. I use NSString’s handy methods to chop it up and stuff the results into a dictionary, then I shove those dictionaries into a big array. Here’s a bit of the code that had the mem leak:

NSArray* recordData = [[records objectAtIndex:i]
                       componentsSeparatedByString:@","];
NSString* username =[[NSString alloc] initWithString:
                     [recordData objectAtIndex:0]];
...
NSMutableDictionary* dict = [NSMutableDictionary dictionary];
[dict setObject:username forKey:@"name"];
...
[mCachedGlobalHighScores addObject:dict];
[username release];

This causes a memory leak that is caused at first line of code. I was baffled by what was causing this. The memory leak seemed to be happening on the string inside [recordData objectAtIndex:0] but I couldn’t figure out why. I asked around on Twitter and many people pointed me to the method [myObj retainCount] that allows you to see how many retains have been made on an object. BunnyHero pointed out that you can run this from the GDB command prompt when stopped at a breakpoint: ‘print (int)[myObj retainCount]‘.

I set breakpoints on the line where username is allocated, and on the line after. When execution halted just before username was allocated, I checked the retain count on the object at index 0. It was 1. This was expected. However, right after username was allocated, it became 2! Why was allocating a new string using initWithString (which is supposed to copy the string and create a new object altogether) increasing the ref count?!

I think it had something to do with the fact that the NSArray is storing the values as type id, not NSString. When you do an initWithString, I think it was just copying the pointer, and doing a retain, so the new object allocated memory, but pointed to the same object. So, in order to fix that, I had to do this instead:

NSArray* recordData = [[records objectAtIndex:i]
                       componentsSeparatedByString:@","];
NSString* username =[[NSString alloc] initWithFormat:@"%@",
                     [recordData objectAtIndex:0]];

Now the memory leak no longer happened when recordData was created…it happened when username was created! But that’s ok, now at least the leak is happening at a more reasonable place.

So now I looked at where I was cleaning up my big array of cached data:

[mCachedGlobalHighScores removeAllObjects];

Reading the docs, removeAllObjects is supposed to call release on each removed object. I assumed that would do a deep release. Not the case. So I tried this instead:

for (int i = 0; i < [mCachedGlobalHighScores count]; i++)
{
    NSMutableDictionary* record =
        [mCachedGlobalHighScores objectAtIndex:i];
    [record removeAllObjects];
}
[mCachedGlobalHighScores removeAllObjects];

But that still didn’t work. The username string wasn’t been released! So finally, I did this:

for (int i = 0; i < [mCachedGlobalHighScores count]; i++)
{
    NSMutableDictionary* record =
        [mCachedGlobalHighScores objectAtIndex:i];
    NSString* name = [record objectForKey:@"name"];
    [name release];
    [record removeAllObjects];
}
[mCachedGlobalHighScores removeAllObjects];

And it works! You might be asking why I didn’t do an autorelease on the username NSString. I tried that. It worked most of the time. But if you flipped back and forth on the screen quickly (causing everything to delete and recreate quickly), I found that I could get into situations where sometimes that string wouldn’t release at all, causing a leak. I decided that if I managed the memory myself, at least I’d know what’s going on.

So, the lessons are:

1) If you are dealing with a string that’s stored as type id, make sure you’re careful about how you use it to init new strings.
2) When you alloc (or retain) you have to release. (Yes, I know that should be obvious by now, but sometimes you think a built-in method is releasing or retaining when it isn’t). :)

Well, apparently I’ve decided to make up for a lack of posts with one gigantic post. Look for more news on the Dapple 1.2 update when it gets approved. And keep your eyes and ears open for news on that secret project I may or may not have mentioned…

Owen

7 Responses to “Dapple 1.2 and NSString Leaks”

  1. Iain Delaney says:

    I think what you’re missing here is that adding an object to an array or dictionary sends that object a retain message. You shouldn’t need the string alloc-init at all.

    This should work:
    NSArray* recordData = [[records objectAtIndex:i]
    componentsSeparatedByString:@”,”];

    [dict setObject:[recordData objectAtIndex:0] forKey:@”name”];

    No allocs, no retains, no releases.
    It’s also important to note that the runtime saves memory whenever possible. So copying a string, if the string is immutable, just results in a new pointer and an incremented retain count. Check the values of your pointers to see if they are pointing at the same object.

  2. OG says:

    @Iain – That certainly looks simpler. I did realise that adding a string to the array resulted in a retain, which gets released when you remove the object from the array.

    However, it was when I tried to let everything work through the autorelease stuff that I ran into problems with sometimes things not being released (and hence a leak), which is why I put the explicit alloc in there. It’s entirely possible that I’m doing something else dumb with my memory stuff.

    This is why I don’t like smart pointers. If I have to manage my own memory, at least it forces me to understand what’s going on all the time. ;)

  3. bunnyhero says:

    how odd… to my eye, that looks like it shouldn’t leak! and from what i know of objective-C (which could certainly be wrong), it shouldn’t matter that the string is stored as an id rather than NSString*.

    also, i’ve read in several places (mostly forum posts, some from apple employees, iirc) that the retainCount method can’t necessarily be relied upon to tell you what’s really going on leakwise, because of implementation details. it might return what looks like a ‘wrong’ value, but not actually be leaking there.

    curiouser and curiouser… it seems like it’d be worth filing as a bug with apple if you can reproduce it in a small test app.

  4. Dominik says:

    I have to second what Iain says: Immutable Strings won’t be copied, but just retained due to efficiency.

    In addition it’s not quite clear to me why you think you need that loop for clearing your cache. The only thing could be that you might have some cyclic references in your cache. Otherwise I’d be really surprised that this won’t work. What makes you sure that this is indeed a leak?
    The second version leaks the username and that should be the reason why you have to loop through the array and release it manually.

    In addition ‘componentsSeparatedByString’ is a cool convenient method, but you can get into a whole lot of trouble, when the data you receive is not what you expect (NSScanner does a good job in reliably parsing strings). (Just think about somebody is spamming your online high score list with some data, which probably could easily crash your app).

    Why didn’t you consider using JSON, this would have taken the burden away from you having to parse the data yourself?

    Btw. it’s more efficient to do
    for(NSMutableDictionary* record in mCachedGlobalHighScores) { …

  5. Doug says:

    Owen, care to elaborate a little more on Google App Engine? Is it robust enough to handle the traffic your app might generate? Does Google have a cap on the number of transactions you can hit them with? I’ve been thinking of setting up a global space as well, but have been concerned about the backend servers and having to purchase server space.

    doug

  6. OG says:

    @Dominik: I was sure it was a leak because Leaks was telling me so. :)

    I wasn’t aware of NSScanner, but that’s definitely something I’ll look into for the next update. Right now I do a lot of error checking on the data coming in from the server. I hope it’s enough…

    As for JSON, I don’t know what it is, so that’s why I didn’t consider it. ;)

    @Doug – I don’t see myself having a huge load on the servers, but even so, the free quotas that Google App Engine provides are pretty impressive. There are full details here:

    http://code.google.com/appengine/docs/quotas.html

    Needless to say, I don’t think I’ll be exceeding 10,000,000 datastore API calls per day anytime soon. ;)

    As for robustness, it’s built on top of their server cloud, so your datastores are stored across multiple servers (if I understand it correctly). Which means, even if one fails, another server should be able to handle the request. The other nice thing about it is that if someone should happen to hack into the system, it’s not the same server as my website. :)

    O.

  7. Doug says:

    @Owen: Thanks! I will check it out.

    As far as JSON. I’ve been using it A LOT at work lately and I have to say I love it. Especially in my case where I submit a javascript array to the backend and if rejected, I can easy recreated the array on the client-side and allow correction.

Leave a Reply