Symbolicating crash logs

Introduction

This post is devoted to a frequent routine of many iOS/Cocoa developers: symbolicating crash logs from users. If you're not interested in details and just want The Solution you can go directly to the section named "symbolicatecrash".

The Solution to what?

Well, according to Apple's Technical Note TN2151 one can symbolicate crash logs just dropping them at Device Logs section in XCode's organizer window. All what they ask you for is a dSYM and .app file somewhere on your disk.

In practice, that's not so smooth. As for me, it never worked as supposed to. Well, it works fine for the latest debug build. But never for any of old AdHoc builds. All you can get from XCode is just symbolication of stack frames related to system libraries, but frequently that's not enough to realize what went wrong.

Symbolication crash logs manually

In the follow chapter I'll try to explain how one can symbolicate a crash log without XCode. Our main tools here will be atos, otool and dwarfdump.

A few words about what is a crash log. That's basically just a plain text file. It contains three sections in essence: headers, stack traces for all threads and binary images information. Headers contain basic info about device, crashed process, versions, date/time and crash reason.

Stack traces look like this:

...
10  YandexMaps                     0x00003a22 0x1000 + 10786
...

First, a frame number goes, then binary image name, then an absolute frame address, finally the same address, but represented as a sum of binary image load address and offset. The process of symbolication is exactly about converting that frame address into a readable symbol name.

The end of file contains a list of all binary images loaded. The list consists of lines like this:

0x1000 -   0x1bbfff +YandexMaps armv7  <257b985709d23dc690477e12bc17fa48> /var/mobile/Applications/97CB76B8-19C5-4266-938E-26669B5CCAD3/YandexMaps.app/YandexMaps

First, image's address range goes. Then image name, architecture, image's UUID and a path, from where the image was loaded. The UUID is actually the most important info here — that's the key to finding a corresponding .app/dSYM file.

dwarfdump

If configured properly, XCode creates a dSYM file together with every app build. dSYM files contains debug information (sadly, for your app only) in a widely used DWARF format. So dwarfdump is a command line tool for dumping info from such files.

First thing you do is finding a corresponding dSYM file for your crash log. The UUID described in the previous section is the key. You can get the UUID from a dSYM file with the following command:

dwarfdump --arch <arch> --uuid <dsymname>

If you store all dSYMs in a specific place, you use grep to find the necessary one:

dwarfdump --arch <arch> --uuid *.dSYM | grep <first-digits>

Finally when you find the dSYM, you can get information about a frame address:

dwarfdump --arch <arch> --lookup <address> <dsymname>

Usually dwarfdump is enough to extract information about all your app's stack frames — you just go line by line. Unfortunately, that doesn't work if you're curious about frames belonging to system libraries. That's where we need the next two tools.

otool and atos

Surprise: there are no dSYM files for system libraries in the SDK. Good news are that we can extract all necessary information directly from binary images. We could do that with our app as well, but usually it's much easier to store just a dSYM file instead of a pair of the dSYM and .app.

Exactly as with dSYM, first we need to make sure that we have a binary image with matching UUID. We can read image's UUID with otool:

otool -arch <arch> -l <binary-image>

The output is quite long but we're only interested in a section like this:

Load command 7
    cmd LC_UUID
cmdsize 24
   uuid FD032EF9-2890-39ED-957A-A99CA51F120C

Forgot to say: you can easily find all necessary binary images in /Developer/Platforms/iPhoneOS.platform/DeviceSupport/.

Now when we have the image we can use it to convert numeric addresses to symbols. That's exactly what atos does. You do that this way:

atos -arch <arch> -o <binary-image> -l <image-load-address> <list-of-interesting-addresses>

The output is pretty straightforward and looks like this:

-[__NSOperationInternal start] (in Foundation) + 652
-[NSOperation start] (in Foundation) + 16
-[NSRunLoop(NSRunLoop) runUntilDate:] (in Foundation) + 56

-l option allows us to use exactly the same address as in the crash log without any additional math.

symbolicatecrash

Originally symbolicatecrash is a script shipped with the SDK by Apple. But exactly as XCode it is too capricious. Thanks to @nskboy, there is a working fork of it. It needs only a dSYM file anywhere on disk (versus dSYM+app required by the original version) and also fixes few nasty bugs. You can get it from GitHub.

The usage is very easy:

./symbolicatecrash -o symbolicated.crash <original-crash-log>

It works stably for me and I'm pretty sure it will for you.

Why ignoring compiler warnings is bad

Imagine that you're observing the following bug: for some reason your method gets its arguments with values different from what it was called with. Then you learn that it's not a debugger's fault, because adding a couple of NSLog's gives the same result. Then you learn that it's not a compiler's bug too, because changing it to clang/llvm gives nothing too. WTF?!, you think now.

The make the situation more real, here's the code which can help to feel that:

// MyClass.h

@interface MyClass : NSObject {

}

- (void)printFloat2:(float)x;

@end

// MyClass.m

@implementation MyClass

- (void)printFloat:(float)x {

 NSLog(@"x = %f", x);

}

- (void)printFloat2:(float)x {

 NSLog(@"x = %f", x);

}

@end

// main.m

 

#import <Foundation/Foundation.h>

#import "MyClass.h"

int main(int argc, char *argv[]) {

 NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init]; 

MyClass * obj = [[MyClass alloc] init];

[obj printFloat:42.0];

[obj printFloat2:43.0];

[obj release];

[pool release];

return 0;

}

What you think will it print out? Well, here's the answer:

x = 0.000000

x = 43.000000

You see that the first line looks a bit wrong.

So, what's wrong? To tell the truth, I don't fully understand it and I would be glad if you could explain it. What is obviously not good it that we have a compiler warning at the appropriate line. -[MyClass printFloat:] isn't public so the compiler warns us that it can't find the method. Fixing that by declaring the method as public resolves the bug. Another good reason to fix all warnings in the code.

Here's what I discovered while trying to understand why does it behave so:

1. If we change argument type to double, everything works fine without declaring the method.

2. If we substitute obj with a proxy in order to dump NSInvocation, we see that methods have the same signature, but the argument is already different at the moment of forwardInvocation:.

It looks clear that the compiler somehow uses information from method declaration when storing the argument, but I don't understand why and how.

Obvious things

Yesterday I finally bought a copy of Rework (and have already read half of it till today) When I wrote about my buying on twitter, @yurabereza replied (it's my translation of his tweet): "I haven't read it yet, but I heard that it only consists of banalities :) I can advice some really good books". All I think about it doesn't fit into a short tweet, so I want to reply on his tweet here.

Let's put Rework aside because I'm not about it and let's talk about banalities. And clearnesses as well, since in this case they are the same. To my opinion, ignorance and lack of attention to banal and obvious things is an essential problem of many people. When they talk about these things they say that it's obvious and just move forward. But when it comes to the point, all these obvious things aren't done for some reason.

Here's sore example: meetings. Everybody knows that they're demotivating, that one needs to limit their time, that one have to have a clear plan in the beginning and discuss only concrete questions. However most of my last meetings were just horrible. But still, if I asked any of participants about those simple rules, he or she would say: "yes, sure, that's clear". It looks like just that "sure" lets their brain forget about the rules right in the begging of every meeting.

I don't see a reason to read "really good books" about I guess "non-banal things" until I can follow banal things for the beginning. (by the way, many of people around me don't follow even a half of ideas from Rework. I myself a year ago wasn't getting a half of them too. Fortunately, during last year in Yandex I learned a lot). I really believe that we should pay much MORE attention to things that seem obvious to us.

Discovering non-documented calls to your delegate object

It can happen that you provide a delegate object to some third-party code and want to know how that code uses your object. Here's a quick way to discover many new things about your delegate is used: just override respondsToSelector like this:

- (BOOL)respondsToSelector:(SEL)aSelector {
 NSLog(@"Was tested for %@", NSStringFromSelector(aSelector));
 return [super respondsToSelector:aSelector];
}

Maintaining application state between launches

Maintaining state between launches is an important job of any iPhone application. Unfortunately it's not trivial when your app has many views having dependencies among each other. And coming of iPhone OS 4.0 will not make your life easier since applications can be (and will be) terminated anyway.

So, when your application is relatively simple, saving its state is simple as well. You can do it this way:

• in the applicationWillTerminate: method you save your state into NSUserDefaults
  
• in viewDidLoad: or applicationDidFinishLaunching: (whichever is more convenient for you) you read the state from user defaults and apply it (say, set current tab of a UITabBarController)
  

After your app gets bigger these methods start to grow and become very messed, especially if you have a navigation controller with potentially deep navigation stack. UIViewController implements NSKeyedArchiver, however after I experimented with it, I realized that it codes/encodes pretty weirdly and moreover anyway there's no way to restore dependencies between view controllers (if you, say, passed an object from Nth view controller to (N+1)th and expect to have the same instance in both controllers).

Thus I want to describe the approach to saving application state we used in Yandex.Maps application.

First, we don't use NSUserDefaults in order to separate state from application preferences. We store the state in a separate file. Then, we declared three protocols very similar to NSCoding:

@protocol YMCoder

- (void)encodeObject:(id)object forKey:(NSString*)key;
- (void)encodeController:(id)vc forKey:(NSString*)key;

@end

@protocol YMDecoder

- (id)decodeObjectForKey:(NSString*)key;
- (void)decodeController:(id)vc forKey:(NSString*)key;

@end

@protocol YMPersistantController

- (void)decodeStateWithDecoder:(id)decoder;
- (void)encodeStateWithCoder:(id)coder;

@end
  

Now each controller (not necessarily UIViewController) implements YMPersistantController like this:

 - (void)encodeStateWithCoder:(id)coder {
 [coder encodeObject:[NSNumber numberWithBool:self.searchActive] forKey:@"searchActive"];
 [coder encodeController:self.searchController forKey:@"searchController"];
 }
 - (void)decodeStateWithDecoder:(id)decoder {
 self.searchActive = [[decoder decodeObjectForKey:@"searchActive"] boolValue];
 [decoder decodeController:self.searchController forKey:@"searchController"];
 if (self.searchActive) {
  [self activateSearch];
 }
 }

The controller doesn't care about when this methods are called (since they can be called a bit later after launch to visually decrease app launch time) or about naming conflicts (it can be guaranteed while coding/decoding). All it cares about is only its own state and dependent controllers.

Finally, we implemented a kind of NSKeyedArchiver, which takes a root controller, recursively encodes it from NSData or decodes an NSData given. Encoding/decoding is called from applicationWillTerminate:/applicationDidFinishLaunching: respectively.

This is it. Actually, I see nothing tricky or special in this approach, but it was the key to adding persistence to Maps application painlessly because existing code was already complex and hard to redesign.
In the end I want to point at completely different solution I saw in three20 code. Shortly speaking, the idea is that your view controllers have URLs like webpages. You store the URL like /themap/search/resultdetails?query=coffee&resultid=42 in user defaults and then restore from it. I find this approach pretty interesting, so you're thinking about persistence in your app take a look at three20's code, it may fit you better.

ASIHTTPRequest vs NSURLConnection

Introduction

Performing HTTP requests and fetching data via HTTP is a quite frequent task in iPhone applications. iPhone SDK provides three kinds of API to do that:

1. BSD sockets. Say hello to unix and old good socket(), read(), ... Of course, you can use it to do HTTP, but I bet nobody does it this way on iPhone.

2. CFNetwork. It's a Core Services framework for networking. According to ADC it's a

low-level, high-performance framework that gives you the ability to have detailed control over the protocol stack.

And according to ADC again

CFNetwork is based, both physically and theoretically, on BSD sockets.

Unfortunately, as every CF framework, it's a pure C framework so it's quite inconvenient to be used. Fortunately, there is a framework called as ASIHTTPRequest. It's very popular and frequently recommended on forums when somebody asks about how to download an URL.

ASIHTTPRequest is based on CFNetwork framework. It's important to say that ASI is synchronous by its nature. So if you want asynchronous URL fetching, you need to use NSOperationQueue (in last versions ASIHTTPRequest can do that itself so you just call [request startAsynchronously]).

3. NSURLConnection. A pretty Objective-C class which lets you do HTTP in just a couple of lines. According to ADC it's based on top of CFNetwork. In contrast to ASIHTTPRequest, NSURLConnection is asynchronous by design. But there is a method which lets you use it in a synchronous manner.

One of the biggest benefits of ASI over NS is that ASI provides a lot of useful functionality for building multipart POST requests, using compressed requests, authentication, proxies, ....

However the question bothering me all last week is: who is faster? Apparently, since NSURLConnection is based on top of CFNetwork, pure CFNetwork should be faster in all cases. But is ASIHTTPRequest faster than NSURLConnection?

The battle

In order to find out who is faster, I've performed some tests. There was 2 tests for each mode (synchronous and asynchronous) for each framework:

1. Download 64 files, 4 kb each. (256 kb in total)
   
2. Download one 256 kb file.
   

All files was stored on Amazon S3.

So, in total there was 4 tests for each framework. Since results may differ from run to run (internet is a quite unstable thing), each test was run 10 times and then the best download time was picked as the result.

ASIHTTPRequest was as of 7th Dec 2009 (a3b974c67699c85bbe89277e016d277c590ccaab).

Tests was done in the Simulator and on my iPhone 2G, both laptop and iPhone was connected to the same wi-fi network.

Here is the results (in seconds):

	Simulator	iPhone 2G
	ASI	NS	ASI	NS
Asyncronous
64 files	6.18	2.69	6.91	3.81
1 file	0.64	0.57	6.33	5.21
Synchronous
64 files	19.56	9.92	28.56	13.30
1 file	0.65	0.59	6.02	3.98

You can get the source code of the test project here. The repository contains detailed logs of the runs presented above.

Conclusion

So what do we see? In all tests ASIHTTPRequest is slower than NSURLConnection, approximately in 2 times. The difference is less when we download 1 big file, but is still sensible.

So in case you can make multipart POST requests yourself, choose NSURLConnection. :) (I'm kidding — of course, ASI provides a lot of other nice functionality, therefore if you don't care about performance much, ASI is a good choice)

Another interesting fact we can notice is that synchronous mode is quite expensive when we perform many requests.

And finally some obvious to everybody, but still important facts:

• Simulator is always faster then real iPhone. On fetching big files, up to 10 times. Moreover, you can notice that Simulator behave completely differently at all. So ALWAYS test your application performance on a real device.
  
• Fetching lots of files is always slower that fetching 1 single file. (NSURLConnection somehow managed to download 64 files faster than 1 file in asynchronous mode on iPhone. But if we check the logs, we see that in most cases downloading 64 files is still slower)
  

Easiest iPhone unit-testing ever using Google Toolbox for Mac

One of the reasons why I wasn't actually writing iPhone/Mac unit tests for a long time is that setting them up using SenTestingKit is a bit... tricky. Even when you know how to do it, it takes rather a lot of time.

Fortunately, there is Google Toolbox for Mac. It's a collection of different source code which Google uses in its own Mac/iPhone projects. And this toolbox has some nice stuff for unit-testing as well.

Here is a little cheat sheet which I use then I need to set up unit tests in my project:

1. Create "Tests" group in your project.
2. Of course, you don't need the whole toolbox. Therefore I have a minimalistic set of required source files. You can download it here. Then copy folder called as "UnitTesting" to (project)/Tests.
3. Create a new build target and say call it "Unit Tests". The target should have "Application" type.
4. Open the newly created target and add Run Script phase to the end
5. Set script contents to

"$SOURCE_ROOT/Classes/UnitTesting/RunIPhoneUnitTest.sh"

6. Ensure using finder, that the mentioned file above is actually present ($SOURCE_ROOT is your project's folder) and has executable bit set. Adjust this path in case of any different layout.
7. Add all .m source files from UnitTesting group to the target
8. Add all necessary code for your tests
9. Removed all source files from Tests group from you main target (since XCode automatically added them there)

That's it. Now you can write tests as follows:

MySuperTest.m:

#import "GTMSenTestCase.h"

@interface MySuperTest : GTMTestCase {
}
@end

@implementation MySuperTest

- (void)setUp
{
}

- (void)tearDown
{
}

- (void)test1
{
STAssertNotNil(obj, @"");
}

@end


Since GTM uses SenTestingKit you can use the same assertion macroses as with SenTest. Also please note, that you can have just an .m file. (it isn't related to GTM/SenTest, just a convenient practice since you rarely #import test files)

One of the biggest benefits of using GTM is that you have tests as application-type target. That means that you can debug tests just by running the appropriate target with enabled breakpoints. And of course, you can debug tests not only in the simulator, but on a real iPhone as well.

Видео-блоггинг (for russian-speaking readers)

Я бы хотел поэкспериментировать с новым жанром: видео-блоги. В основном из-за интереса к тому, что вообще выйдет.

К счастью, есть области (в основном это программирование), в которых у меня есть шанс рассказать что-то новое зрителю. Но, к сожалению, самому мне довольно сложно выбрать с чего начать. Потому что все темы выглядят одинаково очевидными и всем и так известными.

Поэтому я бы хотел попросить помощи у вас, уважаемые читатели. Если вам было бы интересно увидеть мини-лекцию на какую-либо тему в моем исполнении — дайте знать. Об областях моей экспертизы вы можете узнать из резюме.

Рассказывать я планирую на русском (на английском и так полно всего). Именно поэтому и эта запись вдруг на русском.

Cocolua: Lua + Objective-C

I do not love Objective-C. It makes me feel like I'm in the beginning of 90's. Lots of stupid syntax, square brackets everywhere, separate interfaces/implementations,... Okay, I feel you got me.

There are two "official" solutions for people like me: RubyCocoa and PythonCocoa. Hovewer there is a third solution which people rarely think about: Lua (and other embeddable languages).

This solution slightly differs from RubyCocoa/PythonCocoa. Lua is a very lightweight (but powerful) language which depends on C only. You don't need to pull many N-megabyte frameworks and libraries. Lua was created as an embeddable language: you add its interpreter's source code to your project and use is programmatically. Apart of many other things that means that you can use Lua even on the iPhone and no one will ever notice that.

Thanks to Tom McClean, Eric Wing and others, integrating Lua into your project is terrifically easy. Here is a little tutorial (you can download an exemplary project from http://github.com/fourdman/cocolua/tree/master):

1. Download lua's source code from lua.org: http://www.lua.org/ftp/lua-5.1.4.tar.gz
   
2. Download LuaCocoa from http://www.assembla.com/spaces/LuaCocoa/trac_mercurial_tool (you'll need mercurial):
   hg clone http://hg.assembla.com/LuaCocoa
   
3. Add/copy LuaObjCBridge.{h,m} to your project
   
4. Add/copy all Lua interpreter's *.{h,c} from src/ without lua.c and luac.c
   
5. Compile and ensure that there are no errors.
   

How to use it?

Launching a script


// create an interpreter
lua_State* interpreter = lua_objc_init();

// do stuff: prepare globals, etc

// run a script
luaL_dofile(interpreter, scriptPath);

// get the error status
char *luaError = (char *)lua_tostring(interpreter, -1);

// do stuff again: read globals, etc

// kill the interpreter
lua_close(interpreter);


Reading/writing global variables

There are two options here: property-list compatible values and just id instances.
In case you want to transfer property-list compatible values across the bridge you do as follows:


lua_objc_pushpropertylist(interpreter, value);
lua_setglobal(interpreter, "globalVar");


Shortly saying you put the value on top of Lua interpreter's stack and then assign that value to a global variable. Property-list compatible values (NSNumber, NSString, NSData, NSArray, NSDictionary) are special in that way that they're automatically transformed into appropriate Lua's native types: number, string, string, table, table.

In case you have an arbitrary object, you do as follows:


lua_objc_pushid(interpreter, object);
lua_setglobal(interpreter, "globalVar");


In this case on Lua's side you get a table corresponding to the transfered object. And, for instance, you can do:


globalVar:describe()


Calling Cocoa code from Lua


theClass = objc.class("NSObject");
theInstance = theClass:alloc():init();


For more info please read LuaCocoa's code or Lua's documentation on its C API.

Actually, there is another even more easier way to operate with Lua's interpreter: LuaCore. This framework provides an even more cool interface:


LCLua *lua = [LCLua readyLua];
[lua pushAsLuaString:@"this is the value of s" withName:@"s"];
[lua pushDictionaryAsTable:[NSDictionary dictionary] withName:@"d"];
[lua pushGlobalObject:self withName:@"myObject"];
[lua runBuffer:@"print(s)"];
[lua runBuffer:@"print(myObject:description())"];


Finally, here are some useful links on Lua and embedding it:

• Lua's home page
  
• Metalua: Lua with metaprogramming (for pervs)
  
• Original LuaObjCBridge project
  
• A simple example of how to use LuaObjCBridge, pretty old and without downloadable code. But you can download my implementation.
  
• ewing's branch of LuaObjCBridge, adds support for Objective-C 2.0 and contains important bug fixes
  
• Original LuaCore project
  
• ewing's branch of LuaCore
  
• Tutorial: "Embedding Lua in C: Using Lua from inside C."
  
• Tutorial: "Embedding Lua in 5 Minutes" by code monk
  

CrashKit

Today I have good news: we are launching a limited beta of CrashKit, our multi-language online crash reporting tool.

We've started building it a few weeks ago for our consulting project (written in Java). And CrashKit has already saved our asses a couple of times since then. You feel like a real guardian angel watching over your product and being able to react before your users get what's happened.

CrashKit already has all lively important features but also there are many terrific features on our roadmap: collection of usage statistics, two-way customer interaction, much better bug tracker integration, more notification options, branding and internationalization. And, of course, more supported languages and frameworks. If you are missing any particular feature or have anything else to say, please drop us a line.

CrashKit is free to use during the beta period, and will always be free for open-source and small personal projects.