In the last few months we've released a number of new API and integration features. Let's take a look at the philosophy behind these new features, the features themselves, and future directions.

Responsibly Open Data

In order that Studios products provide full value to your organization their data needs to be both open and accessible. Clients integrating with Mingle and Go are interested primarily in rich data and the legitimate ways in which they can interact with that data in order to realize their desired goals. The clients care little about the systems that host the data. They simply want the data.

We need look no further than the web to find all we need to provide well-designed, open access to our data. HTTP is the simple, scalable foundation. The principles of REST provide a discoverable means of interacting with a system's resources. Atom exposes changes to a system's data as an event stream. And as we want to expose the data in an open but responsible fashion, we look to OAuth as the emerging standard for authorizing access to this data.

The above web standards provide the foundation for what we call Responsibly Open Data. When your data is available in this fashion it can continually be used in new, interesting, and valuable ways never dreamed of when the systems were designed. We seek for all the Studios products to embody the principles of Responsibly Open Data. 

RESTful APIs

Both Mingle and Go offer RESTful APIs. These are not new features but are worth mentioning as they are the basis for most of the integrations one is likely to build. Both APIs are continually improving. The Mingle REST API provides  access to Mingle projects, cards, attachments, comments, murmurs, users, transitions and more. The Go REST API provides access to artifacts, properties, configuration, pipelines and agents.

Even Jim Webber admits that an API must value utility above being perfectly RESTful, so we balance our desire for purity with your ability to easily utilize the APIs and the data. In other words, while we try to be good REST citizens, e.g., a card's transitions are discoverable and executable via links provided in the card resource, we value the APIs usability more highly.

Many tools and libraries exist to support your writing clients against our REST APIs.  Still, we are looking to publish some product-specific client libraries this year.

Atom event feeds

While the REST APIs expose current, static state, Atom provides a nice mechanism for exposing changes to state, as they happen, as a poll-able event stream. This type of API makes possible synchronization integrations such as our new Mingle-JIRA connector. It becomes quite easy to propagate changes from Mingle and Go to your other systems as they happen. Go's build events can be broadcast to chat rooms. Mingle changes can move to program and portfolio tools.

Mingle's Event API exposes all changes to your project data: creation, deletion and editing of cards, SCM commits, taggings, attachments, etc.. Go has a similar feed API that provides stage completion events, as they happen, including both build status and links to relevant pipelines, jobs, artifacts, and triggering commits.

You might be wondering why we've exposed events streams as Atom feeds rather then some sort of "web hook" publish-subscribe mechanism. The answer is quite simple. This style allows the client itself to guarantee delivery of events, and in the correct order. Web hooks do not afford this level of robustness. Atom feeds also scale much more easily than web hooks. Jim, Ian, and Savas have a lot more to say on the topic of using Atom to expose event streams in their REST in Practice book.

While Atom is a widely used standard, with many existing consumer libraries for all the popular languages, it can still be fairly tedious to have to first write a polling client in order to build your integration. As we want you to be able to build valuable integrations rather than worry about the tedium of polling, we've published OSS libraries for consuming the feeds. I've written mingle_events to help consume the Mingle feed using Ruby. Pavan Sudarshan from the Go team has written a client for the Go API using Java. There is obviously room for more such libraries to provide greater language and platform coverage.

Event APIs open up another interesting type of integration on which I won't go into too much detail but don't want to ignore. Something that can easily be done with events is the playback of those events. Perhaps you want to write a little tool to perform a rich historical analysis of your Mingle activity. You could poll the events and play them back, updating  your metrics along the way, taking a snapshot of the metrics at the end of every day or week or month. This type of integration isn't necessarily obvious and deserves a post of its own, so look for that in the future.

OAuth 

The "responsible" piece of Responsibly Open Data means that we need to provide some means of authentication and authorization. Currently this means piggybacking on the existing auth mechanisms of Mingle and Go. However, we are in the process of introducing OAuth 2.0 to our applications as a more appropriate means of API auth.

OAuth has emerged as the web standard for allowing a user to authorize one service to provide his data to another service, e.g., sharing your Flickr stream on your Facebook wall. OAuth is important because it discourages the proliferation of passwords and limits the privileges you grant to a service to the bare minimum. If you are interested in a deeper understanding of OAuth you can watch some videos we made. If you're a Rails developer you might be interested in the OAuth Provider plugin we've published on Github.

Currently, OAuth is only available in our products in support of our OpenSocial based integrations (which I'll post about tomorrow) but expect it to roll out for API access this year.

Wrapping Up

We hope that by providing these APIs we've given you easy-to-use standards-based access to your data. It's your data, you should be able to do what you like with it. We look forward to hearing about your experience building integrations against our products, and even seeing some of them published here on the community site. And we know our APIs are not perfect, so by all means send feedback our way so that we can continue to improve.

(Thanks to Jim Webber and Ian Robinson for their thought leadership and support.)

Anand from the Go team has launched a new project called Go Feeds.  You can find details about the project including downloads, screencasts, and details about how to join and contribute over at Go Add-Ons and Extras.

Join us for a one-day conference focused on Android, iOS, and web-based mobile app development at WindyCityGo. Learn how to create better mobile apps and how to sell them effectively. Meet others who are just as passionate as you. Developers, designers, entrepreneurs, and investors all find value in WindyCityGo.

Register by March 21st & Save $80 

When you are building an iOS application, whether for internal or external consumption, you will undoubtedly have bugs and crashes. These crashes contain stack dumps, but if the application is built in release mode then those stack dumps will not contain references to the actual code. This article explain how to ‘symbolicate’ these traces, and how to make sure that the SYM files are archived for your builds.

We describe how this works for one of our own internal applications, called ‘Expenses’.

Archiving SYM files using Go

To analyze crash dumps you need both the Expenses.app and the SYM files associated with that build. Note that the SYM files cannot be regenerated, so you need to archive them along with the application itself. Every build is unique, so it is important that the SYM files are created at the same time as the application itself.

This is easy, since we build our application with Go. Since Go includes an artifact repository, storing the SYM files is simply a matter of telling Go to keep that artifact.

Here is the configuration for the relevant part of the pipeline:

<job name="package">
  <tasks>
    <rake target="tw_expenses:clean tw_expenses:in_house_release" />
  </tasks>
  <resources>
    <resource>dist</resource>
  </resources>
  <artifacts>
    <artifact src="dist" />
    <artifact src="build/Distribution-iphoneos/Expenses.app.dSYM" />
  </artifacts>
</job>

See Managing Artifacts and reports in the Go help files.

Finding crash dumps

Whenever you synchronize your iPhone or iPod Touch, all the crash logs are transferred to your computer.

Here are their locations:

• Mac OS X : ~/Library/Logs/CrashReporter/MobileDevice/
• Windows XP: C:\Documents and Settings\\Application Data\Apple computer\Logs\CrashReporter/
• Windows Vista: C:\Users\\AppData\Roaming\Apple computer\Logs\CrashReporter/MobileDevice/

Downloading the SYM files from Go

We use Go’s artifact repository to store our SYM files for each build. So to get the SYM files associated with our app we simply download that directory as a zip file:

$ wget http://<username>:<password>@<go-server>:8153/go/files/tw-iphone-expenses/58/dist-release/2/package/Expenses.app.dSYM.zip
$ unzip Expenses.app.dSYM.zip

Extracting the app from an ipa package

We don’t store the Expenses.app file, instead we just store the Expenses.ipa. Since this is simply a zip file it is easy to extract the app from it:

$ wget http://<username>:<password>@<go-server>:8153/go/files/tw-iphone-expenses/58/dist-release/2/package/dist/Expenses.ipa
$ unzip Expenses.ipa
Archive:  Expenses.ipa
   creating: Payload/
   creating: Payload/Expenses.app/
   creating: Payload/Expenses.app/_CodeSignature/
  inflating: Payload/Expenses.app/_CodeSignature/CodeResources  
... [more lines omitted for brevity]
$ mv Payload/Expenses.app .
$ rm -r Payload

Symbolicating

Symbolicating adds back the actual stack traces so that you can see where the app failed

$ /Developer/Platforms/iPhoneOS.platform/Developer/Library/PrivateFrameworks/DTDeviceKit.framework/Versions/A/Resources/symbolicatecrash Expenses_2011-02-15-164912_bfus-iPhone.crash Expenses.app.dSYM

Before:

Thread 0 Crashed:
0   libSystem.B.dylib               0x311de2d4 0x31165000 + 496340
1   libSystem.B.dylib               0x311de2c4 0x31165000 + 496324
2   libSystem.B.dylib               0x311de2b6 0x31165000 + 496310
3   libSystem.B.dylib               0x311f2d72 0x31165000 + 580978
4   Expenses                        0x00025480 0x1000 + 148608
5   Expenses                        0x00024c78 0x1000 + 146552
6   Foundation                      0x31d4d856 0x31cc4000 + 563286
7   Foundation                      0x31d50226 0x31cc4000 + 573990
8   Foundation                      0x31d4f8fa 0x31cc4000 + 571642
9   Expenses                        0x00024c4a 0x1000 + 146506
10  Expenses                        0x00003672 0x1000 + 9842
11  CoreFoundation                  0x3146ebb8 0x31430000 + 256952
12  Expenses                        0x000102a4 0x1000 + 62116
13  CoreFoundation                  0x3146ebb8 0x31430000 + 256952
14  Foundation                      0x31ce478e 0x31cc4000 + 133006
15  CoreFoundation                  0x314867d6 0x31430000 + 354262
16  CoreFoundation                  0x314585b0 0x31430000 + 165296
17  CoreFoundation                  0x31457e54 0x31430000 + 163412
18  CoreFoundation                  0x31457c80 0x31430000 + 162944
19  CoreFoundation                  0x31457b88 0x31430000 + 162696
20  GraphicsServices                0x35d664a4 0x35d62000 + 17572
21  GraphicsServices                0x35d66550 0x35d62000 + 17744
22  UIKit                           0x338d5322 0x3389e000 + 226082
23  UIKit                           0x338d2e8c 0x3389e000 + 216716
24  Expenses                        0x000041ce 0x1000 + 12750
25  Expenses                        0x0000289c 0x1000 + 6300

After:

Thread 0 Crashed:
0   libSystem.B.dylib               0x311de2d4 __kill + 8
1   libSystem.B.dylib               0x311de2c4 kill + 4
2   libSystem.B.dylib               0x311de2b6 raise + 10
3   libSystem.B.dylib               0x311f2d72 abort + 50
4   Expenses                        0x00025480 -[TWExpenseSheetParser processLine:] (TWExpenseSheetParser.m:238)
5   Expenses                        0x00024c78 __-[TWExpenseSheetParser updateFrom:]_block_invoke_1 (TWExpenseSheetParser.m:47)
6   Foundation                      0x31d4d856 __-[NSString enumerateLinesUsingBlock:]_block_invoke_1 + 6
7   Foundation                      0x31d50226 -[NSString enumerateSubstringsInRange:options:usingBlock:] + 1182
8   Foundation                      0x31d4f8fa -[NSString enumerateLinesUsingBlock:] + 78
9   Expenses                        0x00024c4a -[TWExpenseSheetParser updateFrom:] (TWExpenseSheetParser.m:48)
10  Expenses                        0x00003672 -[TWExpenseReportReader requestFinished:] (TWExpenseReportReader.m:224)
11  CoreFoundation                  0x3146ebb8 -[NSObject(NSObject) performSelector:withObject:] + 16
12  Expenses                        0x000102a4 -[ASIHTTPRequest requestFinished] (ASIHTTPRequest.m:1928)
13  CoreFoundation                  0x3146ebb8 -[NSObject(NSObject) performSelector:withObject:] + 16
14  Foundation                      0x31ce478e __NSThreadPerformPerform + 262
15  CoreFoundation                  0x314867d6 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 6
16  CoreFoundation                  0x314585b0 __CFRunLoopDoSources0 + 376
17  CoreFoundation                  0x31457e54 __CFRunLoopRun + 224
18  CoreFoundation                  0x31457c80 CFRunLoopRunSpecific + 224
19  CoreFoundation                  0x31457b88 CFRunLoopRunInMode + 52
20  GraphicsServices                0x35d664a4 GSEventRunModal + 108
21  GraphicsServices                0x35d66550 GSEventRun + 56
22  UIKit                           0x338d5322 -[UIApplication _run] + 406
23  UIKit                           0x338d2e8c UIApplicationMain + 664
24  Expenses                        0x000041ce main (TimeAndExpensesIPhone-main.m:13)
25  Expenses                        0x0000289c start + 32

References

• http://www.anoshkin.net/blog/2008/09/09/iphone-crash-logs/
• Go Agile Release Management

While reading Shane Richmond's post in today's telegraph called 'Google's Android developer event criticised', I came to think of a recent conversation that I had with one of our clients around the benefit of iOS to enterprises lies in that it is a closed system, thereby limiting the choices that enterprises have to make and allowing the toolset (albeit limited) to reach maturity quickly.

I'm looking for Android developers to speak up about what they see the pros and cons of the openness of the platform to be.  

iOS devs are welcome to participate in the discussion.

Cheers,
Adam