Designing a public API

Disclaimer: a bit over a month ago I've joined the API team at 7digital. This post was actually written before that and stayed in edit mode far too long. I've decided not to update it, but instead to publish it as it is with the intention of writing a follow-up once I have enough new interesting insights to share.

The greatest example of what comes from a good internal API that I know of is Amazon. If you're not familiar with the story the way Steve Yegge (now from Google) told it, I recommend you read the full version (mirrored, original was removed). It's a massive post that was meant for internal circulation but got public by mistake. There's also a good summary available (still a long read). Steve followed up with an explanation after Google PR division learnt that he released his internal memo in public. If you're looking for an abridged version, there's a good re-cap available at API Evangelist.

I'd recommend you read all of those, but preferably not right now (apart from the 2-minutes re-cap in the last link), as it would take a better part of an hour. A single sentence summary is: you won't get an API, a platform others can use, without using it first yourself, because a good API can't, unfortunately, be engineered, it has to grow.

So to start on the service-oriented route, you have to first take a look at how various existing services your company has interact with each other. I am sure you will find at least one core platform, even if it's not recognised as such, with existing internal consumers. It's probably a mess. Talk to anyone who worked with those projects (probably most programmers) and you'll have lots of cautionary tales how API can go wrong, especially when you try to plan it ahead and don't get everything right. And you won't, it's just not possible.

Some of the lessons I've learnt so far (I'm sure others can add more!):

1. You need to publish your API.

   Last team I was with did this, sort of - we had NuGet packages (it's a .Net API, not a web one, ok?). Still, those packages contain actual implementation, not only surface interfaces, so they are prone to breaking. And they expose much more than is actually needed/should be used, so a whole lot of code is frozen in place (see 2.).

2. You need a deprecation mechanism.

   Your API will change. Projects change, and API needs to reflect this. It's easy to add (see next point), but how do you remove? Consumers of the API don't update the definition packages, we've had cases where removing a call that was marked as [Obsolete] for over a year broke existing code.

3. You need to listen to feedback from consumers.

   Internal consumers are the best, because you can chat with them in person. That's the theory, at least, I've seen project teams not talk to each other and it becomes a huge problem. Because of problems/lacks in API, we had projects doing terrible things like reading straight from another database or even worse, modifying it. This won't (hopefully) happen with an external consumer, but if the other team prefers to much around in your DB instead of asking for an API endpoint they need, you don't have a working API.

4. Your API needs to perform.

   Part of the reason for problems mentioned in 3. is that our API was slow at times. There were no bulk read/update methods (crucial for performance when working with large sets of items), we had bulk notification in the form of NServiceBus queues but it had performance problems as well. If the API is not fast enough for what it's needed for, it won't be used, it's that simple.

5. You need to know how your API is used.

   This last point is probably the most important. You won't know what you can remove (see 2.) or what is too slow (see 4.) if you don't have some kind of usage/performance measurement. Talk to your Systems team, I'm sure they will be happy to suggest a monitoring tool they already use themselves (and they are the most important user of your reporting endpoints). For Windows Services, Performance Counters are a good start, most administrators should already be familiar with them. Make sure those reports are visible, set up automatic alarms for warning conditions (if it's critical it's already too late to act). Part of this is also having tests that mirror actual usage patterns (we had public interfaces that weren't referenced in tests at all) - if a public feature does not have automated test then forget about it, it could as well not exist. Well unless you consider tests "we have deleted an unused feature and a month later found out another project broke" (see 2.).

In summary, the shortest (although still long!) path to a useful public API is to use it internally. Consumers with a quick feedback cycle are required to create and maintain a service-oriented architecture, and there's no faster feedback than walking to your neighbour's desk.