August 06, 2017
I spec out and build APIs (web APIs or otherwise) often, both at work and for my personal projects. Best practices about RESTful design for web APIs are all over the place, and they should ideally be followed everywhere. Here are some resources to learn more about the RESTful API designing guidelines and also a good tutorial to build REST APIs.
I feel emphasizing too much on RESTful design is a bit of an overkill, especially when almost every API provider now provides its users with SDKs in various languages. Instead, as PMs, we should work towards building usable APIs following the best practices where possible, and not the other way round - building the APIs following the best practices and making them usable where possible.
Is the usability of APIs hyped? I don’t think so. The usability of the API heavily depends on the API design. Just like badly written code isn’t manageable, a badly designed API is unusable without putting in a lot of effort.
After a couple of people asked me how I build APIs & SDKs, I’ve decided to write about it here. Here is the gist:
Know your users & their use casesDefine the experienceTurn the defined experience into specsEat your own (mocked) dog foodThis may seem obvious, but you need to know who is going to use the APIs (a web developer? An experienced dev with a VOIP background?) so that you don’t have to spend a lot of time educating the users. You must also know the use case of the users (if there are hundreds of them, at least the most dominant use cases) so that you can build APIs which are aligned towards those use cases and not the opposite.
For example, if your users are gonna use your API from a browser, then you will have to provide a token-based authentication mechanism instead of something like HTTP Basic Auth which works just fine for server-to-server communication.
You should acknowledge that providing APIs and the SDKs is just delivering a value proposition to the end-user, and thus you need to be well aware of the experiences you want the users to undergo. It is always better (both for the users as well as the business) to call out the downsides of using the API so users know what is easy to build and what is tough.
Prof. A. K. Jain of IIM Ahmedabad (he taught us a course called Consumer Based Business Strategies) says that every product or service a business provides will be a part of some process at the consumer’s end. How good a product or service is will depend on how well it can help that process. The same is the case here with building usable APIs.
For example, let’s say you are building an API to search for flights between two given places. And assume your users are going to show these results to their customers. You could come up with multiple experiences for this scenario.
Good experience: providing all the possible flights with their prices in the same response
Bad experience: providing all the possible flights, but the user needs to query separately for the price. Note that this perfectly conforms to REST guidelines.
Depending on the things your users value and the capabilities of your business, you might sometimes have to choose the one that provides a bad experience (and you will have to call this out in your communication). Remember that attempting to deliver a good experience that never works is the worst experience of all.
After you decide on the experience (which is mostly high-level), you need to turn them into specs so that you can start delivering those experiences to the users. While this is pretty straight forward, you might miss out on specifying a few functional and nonfunctional requirements. I use the following questionnaire to help myself out while checking my spec before implementation.
Do you need to track the usage of this API endpoint?
For example, which endpoints are users hitting the most? In most cases, data reveals some really interesting patterns that you can leverage for your next API version. Also, if you have SDKs too, make sure they use user-agents. Your API logs will then log these user-agents for you to analyze later.
Is the endpoint naming consistent with whatever your business already has?
If you are starting from scratch, always use a plural noun. Use sub-resources to denote relations and information/entity hierarchy.
Did you define the HTTP verbs and the status codes that the endpoint supports?
Sometimes, a 404 is better to be sent than a 403 to avoid accidental leakage of information to unauthorized users. So, it is usually best to send a 404 for user-level resources or sub-resources and a 403 for base resources.
Did you define the success responses and error responses?
Almost every API has the response data in JSON, but you might want to support XML as well if you have enterprise customers, or your use case requires the data to be in XML. Did you also prepare a list of error codes that will be thrown by this endpoint - and how the users should resolve these errors?
Is the API scope too huge?
You might then want to split the spec into multiple logical items. Remember that you can always add scope easily to the API later, but cannot remove functionality once it is public with the same level of ease.
Do you expect this API endpoint to return a huge list?
In that case, you might want to define the pagination strategy (identifier-based offset like after=xyz&limit=10 or index-based offset like offset=10&limit=10), filters to be supported and the aliased endpoints for most-used filters.
Do you expect your users to use the API from a browser?
You might want to check on enabling CORS and having a token-based authentication.
Does your API endpoint return a timestamp?
If yes, define the format in which the timestamp will be returned (usually ISO 8601).
Do you expect the API calls to be authenticated?
If yes OAuth 2 is most preferred for most client-side authentication, Basic Auth is widely used for server-side authentication. Sometimes JWT can be used for client-side auth.
Do you expect your users to use the API from behind a GET-POST proxy?
Some enterprise proxies won’t allow HTTP verbs other than GET and POST. In that case, it is best if the API can allow an X-HTTP-Method-Override header in the request while sending a POST.
Do you need your API endpoint rate-limited?
It is a good idea to rate-limit your users as per your fair usage policy or acceptable usage policy. Github API is a very good example of this:
They use X-RateLimit-Limit header to specify the maximum number of requests users can make in a given period, X-RateLimit-Remaining to specify the number of requests remaining in the current window and X-RateLimit-Reset to specify the time at which the current window is reset (UTC epoch seconds)
Is the API endpoint versioned?
It is normal for the APIs to evolve as the customers and their products do. So it is better to have versioned APIs so that the APIs can be changed to the next version without breaking the existing API and thus without breaking the production code of your customers.
Do I need to provide the users with an SDK?
The answer is mostly yes if you are building for the general audience. If it is an internal API, an SDK might be overkilling it - you can delay building these SDKs if these aren’t public-facing. If you need the SDKs to be developed, then you will have to define the interfaces that are consistent across resources. Decide on the languages to be supported and prioritize them after talking to the consumers. Have a look at my spec checklist for SDKs at this stage to finalize your spec.
It is a good idea (might not be scalable though) to mock the APIs (I use Flask for this - I just need to define the routes and return a static JSON) to build a prototype of what the users would build on their end. Doing this will help us identify what is missing in the APIs that aren’t helping the users achieve their goals.
That’s a wrap! This is what worked for me so far while designing usable APIs and I hope it works for you too.