Consistent, hierarchical, and modular APIs
So, you have decided to build your enterprise application as loosely coupled micro-services, even thinking of making them serverless after reading my old article — Serverless Microservices and after coming across REST API, you have decided to build your application APIs conforming to REST architectural style. Good Job! Basically, you have adopted the microservices architecture, where each piece of your application performs a smaller set of services and runs independently of the other part of the application and operates in its own environment and stores its own data and communicates/interacts with other services via REST API. But, then you have multiple REST APIs in your services to take care of and provide to the customers, especially if you are opening your services to third-parties. So here comes the problem:
When you let your micro-service teams to adopt to REST, they all come up with their own standards and conventions of doing things. Eventually, there is pandemonium, when clients and customers are not able to follow the REST APIs, designed for your enterprise services as each API is designed in its own unique style. When you, as an architect or team lead go ahead to discuss this problem with your micro-services development team, you get the following response.
Also, as your team adds more REST services and glue them together, you start worrying more about their standardisation and presentation as you never had put any convention in place. Now, you try to realize the problem of standardizing your RESTful APIs and wished for better and sane practices from the start. After all, migrating all your clients and customers on the new standard version of APIs won’t be an easy task.
You look at the REST API specification from the first principles and start by understanding what all its underlying syntax is customisable and can be standardized.
API — An API is a set of definitions and protocols for building and integrating application software, referred to as a contract between an information provider and an information user — establishing the content required from the consumer (the call) and the content required by the producer (the response).
The advantage of using APIs is that as a resource consumer, you are decoupled from the producer and you don’t have to worry about how the resource is retrieved or where it comes from. It also helps an organization to share resources and information while maintaining security, control, and authentication — determining who gets access to what.
REST — REST stands for REpresentational State Transfer and was created by computer scientist Roy Fielding. REST is a set of architectural constraints, not a protocol or a standard. When a client request is made via a RESTful API, it transfers a representation of the state of the resource to the requester or endpoint in one of several formats via HTTP: JSON, HTML, XML, plain text etc.
In order for an API to be considered RESTful, it has to conform to these criteria:
A layered system that organizes each type of server (those responsible for security, load-balancing, etc.) involved the retrieval of requested information into hierarchies, invisible to the client.
Code-on-demand (optional): the ability to send executable code from the server to the client when requested, extending client functionality.
API developers can implement REST in a variety of ways, which sometimes leads to chaos, especially when the syntactic schema for REST across multiple API development teams are not aligned and standardised. So, in next sections, the evaluation criteria for evaluating and suggestions to standardize REST APIs is presented to evade this chaos.
The REST APIs should be holistically evaluated and improved based on the following criteria:
The API should define a hierarchy, where each node is either a collection or a resource.
● A collection contains a list of resources of the same type. For example, a device type has a collection of devices.
● A resource has some state and zero or more sub-resources. Each sub-resource can be either a simple resource or a collection. For example, a device resource has a singleton resource state (say, on or off) as well as a collection of changes (change log).
A specific use case, the singleton resource can be used when only a single instance of a resource exists within its parent resource (or within the API, if it has no parent).
Here is a suggestion for simple and consistent API hierarchy:
Collection : device-types
Resource: device-types/{dt-id}
Singleton Resource: device-types/{dt-id}/state-machine
Collection: device-types/{dt-id}/attributes
Resource: device-types/{dt-id}/attributes/{attribute-id}
Collection: device-types/{dt-id}/changes
Resource: device-types/{dt-id}/changes/{change-id}
Collection: device-types/{dt-id}/devices
Resource: device-types/{dt-id}/devices/{d-id}
Singleton Resource: device-types/{dt-id}/devices/{d-id}/state
Custom Method: device-types/{dt-id}/devices/{d-id}/state:transition
Collection: device-types/{dt-id}/devices/{d-id}/changes
Resource: device-types/{dt-id}/devices/{d-id}/changes/{change-id}s
Note that in the above, id can be string for name, number or even UUID based on agreed convention. Example:
[https://tenant.staging.saas.com/api/v1/device-types/house-alarm/devices/cbb96ec2-edae-47c4-87e9-86eb8b9c5ce4s](https://tenant.staging.saas.com/api/v1/device-types/house-alarm/devices/cbb96ec2-edae-47c4-87e9-86eb8b9c5ce4s)
The API should support standard methods for LCRUD (List, Create, Read, Update and delete) on the nodes in the API hierarchy.
The common HTTP methods used by most RESTful web APIs are:
The following table describes how to map standard methods to HTTP methods:
HTTP Mapping: GET <collection URL>
HTTP Request Body: NA
HTTP Response Body: Resource* list
HTTP Mapping: GET <resource URL>
HTTP Request Body: NA
HTTP Response Body: Resource*
HTTP Mapping: POST <collection URL>
HTTP Request Body: Resource
HTTP Response Body: Resource*
HTTP Mapping: PUT or PATCH <resource URL>
HTTP Request Body: Resource
HTTP Response Body: Resource*
HTTP Mapping: DELETE <resource URL>
HTTP Request Body: NA
HTTP Response Body: NA
Based on the requirements, some or all of the above API methods for the node hierarchy should be supported. Note that the * marked resource data will be encapsulated inside the response body format containing status, request and data.
Here are the differences between POST, PUT, and PATCH for their usage in REST:
PUT requests must be idempotent but POST and PATCH requests are not guaranteed to be idempotent. If a client submits the same PUT request multiple times, the results should always be the same (the same resource will be modified with the same set of values).
Custom methods refer to API methods besides the above 5 standard methods for functionality that cannot be easily expressed via standard methods. One of the custom functionality is the state transition of devices based on API requests. The corresponding API can be modelled either of the following ways:
Based on Stripe invoice workflow design: Use / to separate the custom verb. Note that this might confuse it with resource noun.
Based on Google Cloud API design: Use : instead of / to separate the custom verb from the resource name so as to support arbitrary paths.
In either of the above ways, the API should use HTTP POST verb since it has the most flexible semantics.
Resources may have the following standard fields:
Note that displayName, timeZone, regionCode, languageCode etc are useful, when you want to provide localizations in your API.
Collections may have also have standard fields like totalCount in metadata.
Collections List API may have the following standard query parameters (with alternate names):
The standard query parameters can be separated from custom query parameters by preceding them with $. Example:
[https://tenant.staging.saas.com/api/v1/device-types/house-alarm/devices?$orderBy=volume&owner=jaykmr&$format=json](https://tenant.staging.saas.com/api/v1/device-types/house-alarm/devices?$orderBy=volume&owner=jaykmr&$format=json)
Success & Errors across all the methods should be consistent, i.e. have same standard structure, for example:
{
"status":{
"code":"",
"description":"",
"additionalInfo":""
},
"request":{
"id":"",
"uri":"",
"queryString":"",
"body":""
},
"data":{
"meta":{
"totalCount":"",
...
},
"values":{
"id":"",
"url":"",
...
}
}
}
All the API should have a common response structure and this can be achieved by using a common response formatter in the code for resource methods. Note, in case of success, when no data is returned, the API response can either return empty list [] for collection or empty object {} for resource, while in case of error, can just return data as null to keep a consistent response schema across methods.
Here are my suggestions on the naming conventions without the intention of provoking tabs vs spaces kind of debate:
● Collection and Resource names should use unabbreviated plural form and kebab case.
● Field names and query parameters should use lowerCamel case.
● Enums should use Pascal case names.
● Custom Methods should use lowerCamel case names. (example: batchGet)
There are multiple good suggestions like Google API Naming Convention but this depends on the organization, however whatever the organization chooses and adopts, they should be aligned across all the teams and strictly adhered to.
List Pagination : All List methods over collections should support pagination using the standard fields, even if the response result set is small.
The API pagination can be supported in 2 ways:
The cursor next link makes the API really RESTful as the client can page through the collection simply by following these links (HATEOAS). No need to construct URLs manually on the client side. Moreover, the URL structure can simply be changed without breaking clients (evolvability).
Delete Response : Return Empty data response {} in hard delete while updated resource data response in soft delete. Return Null in failures and errors.
Enumeration and default value : 0 should be the start and default for enums like state singletons and their handling should be well documented.
Singleton resources : For example, the state machine of the resource (say, device type) as well as the state of the resource (say, device) should never support the Create and Delete method as the states (ON, OFF, RING etc) can be configured i.e. Updated but not Created or Deleted.
Request tracing and duplication : All requests should have a unique requestID, like a UUID, which the server will use to detect duplication and make sure the non-idempotent request like POST is only processed once. Also, requestID will help in distributed tracing and caching. The unique requestID should also be part of the response request section.
Request Validation : Methods with side-effects like Create, Update and Delete can have a standard boolean query parameter validate, which when set to true does not execute the request but only validates it. If valid, it returns the correct status code but current unchanged resource data response, else it returns the error status code.
[https://tenant.staging.saas.com/api/v1/device-types/house-alarm/devices/cbb96ec2-edae-47c4-87e9-86eb8b9c5ce4s/state:ring?$validate=true](https://tenant.staging.saas.com/api/v1/device-types/house-alarm/devices/cbb96ec2-edae-47c4-87e9-86eb8b9c5ce4s/state:ring?$validate=true)
For example, the above request will validate whether alarm can be put to ring or not.
HATEOAS (Hypertext as the Engine of Application State): Provide links for navigating through the API (especially, the resource url). For example,
{
"orderID":3,
"productID":2,
"quantity":4,
"orderValue":16.60,
"links":[
{
"rel":"customer",
"href":"https://adventure-works.com/customers/3",
"action":"GET",
"format":["text/xml","application/json"]
},
{
"rel":"customer",
"href":"https://adventure-works.com/customers/3",
"action":"PUT",
"format":["application/x-www-form-urlencoded"]
},
{
"rel":"self",
"href":"https://adventure-works.com/orders/3",
"action":"GET",
"format":["text/xml","application/json"]
},
{
"rel":"self",
"href":"https://adventure-works.com/orders/3",
"action":"PUT",
"format":["application/x-www-form-urlencoded"]
},
{
"rel":"self",
"href":"https://adventure-works.com/orders/3",
"action":"DELETE",
"format":[]
}]
}
Versioning : Versioning enables the client or the consumer to keep track of the changes, be it compatible or even, incompatible breaking changes so that it can make specific version call for consumption, which it can process.
The versioning can be supported in 3 ways:
URI versioning :
Query string versioning :
https://tenant.staging.saas.com/api/device-types?$version=v1
Header versioning :
GEThttps://tenant.staging.saas.com/api/device-types Custom-Header: api-version=v1
So, you have divided the monolithic applications into microservices, all integrated by loose coupling into separate micro-applications. Now is the time to revisit your APIs, make them standardized and raise the bar of your APIs before making it external for customers. The APIs should be consistent, hierarchical and modular. The separation of methods, standard fields and patterns from the collection & resource hierarchy will allow you to build resource agnostic re-usable abstractions, which can be implemented by the resource interfaces and deployed as services.
You should even, break the frontend into micro-frontends and serve them separately to make it a complete micro-application. Refer to my previous article — Demystifying micro-frontends for such micro-services-based backend to make a complete micro-application-based architecture.
Solve the problem once, use the solution everywhere!
Standardizing RESTful APIs was originally published in Technopreneurial Treatises on Medium, where people are continuing the conversation by highlighting and responding to this story.