API REST API Guideline

The Opencell REST API Guideline provides a set of design principles and practices that should be considered by developers when designing, implementing and exposing a RESTful API. To provide the best experience for developers consuming Opencell REST APIs, it's important for API developers to follow up the principles and practices defined herein, making these APIs intuitive and easy to use.

The goals of the guideline are:

• Define the overall principles that should be taken into account when designing REST APIs for Opencell

• Define common practices and patterns for all Opencell REST APIs such as:

  • Filtering

  • Pagination

  • Error Handling

  • Documentation

• Define common practices for documentation

Although the guideline is based on the REST architectural style, it does not try to strictly adhere to RESTful constraints. The principles covered by the guideline are based on practical experiences in the development of REST APIs in Opencell while still conforming with most of the constraints of REST architectures.

Principles

This section defines the general principles that should be considered by API developers, regardless of the practices defined through this document (reference https://github.com/keycloak/keycloak-community/blob/master/design/rest-api-guideline.md).

Level of Service Maturity

Based on Richardson Maturity Model, developers MUST adhere to the maturity Level 2, where interactions with the API endpoints are based on the usage of HTTP Verbs and status codes to communicate status and errors.

With this principle in mind, APIs should leverage the HTTP Verbs defined in this guideline and respect their meaning, idempotency as well as use HTTP Status Codes in responses to communicate the success and failures when processing a request. For more details about the supported HTTP Verbs and Status Codes, see their respective sections in this document.

By respecting this principle, we expect developers to provide more concise and consistent APIs with a uniform interface.

Resource Path

In opencell, we choose to stick with the following convention for resource paths:

/api/rest/{version}/{API_GROUP}

Opencell APIs usually operate in the context of an opencell instance, the resource path should be prefixed with /api/rest/{version} where version is the current api version.

Versioning

Versioning helps to iterate faster when the needed changes are identified. Moreover, versioning helps APIs to adapt to changes to their fields or restructure resource representations. In addition to that, versioning can help developers to offer non-stable, technology preview features as well as control the end-of-life of APIs.

Change in an API is inevitable as your knowledge and experience of a system improve. Managing the impact of this change can be quite a challenge when it threatens to break existing client integration.

The version refers to the stability and the state of an API. The version does not necessarily refers to a opencell release version number so that it can be mapped to different opencell release versions.

Here is an example of a resource path related with the group of functionalities provided by the Account Management REST API:

/api/rest/v1/accountManagement/sellers/{sellerCode}

The opencell API follows semantic versioning where there is a a major and a minor version. The format of the version is MAJOR.MINOR or vMAJOR_MINOR. For example, v2_0 is a major version, while v2_1 is a minor version.

However, unlike in traditional semantic versioning, Opencell APIs must not expose minor or patch version numbers. For example, Opencell APIs use v1, not v1.0, v1.1, or v1.4.2. From a user's perspective, major versions are updated in place, and users receive new functionality without migration.

When to Version

Major versions

Major release versions introduce some breaking, backwards incompatible changes. The version would end in zero with the format vX_0 where X is the major version number.

Each major version has a separate endpoint. This is an example of the URL where X is the major version number.

/api/rest/vX

APIs only need to be up-versioned major version when a breaking change is made. Examples of breaking changes include:

• completely change the API

• a change in the format of the response data for one or more calls

• a change in the request or response type (i.e. changing an integer to a float)

• removing any part or functionality of the API.

• providing non-stable or technology preview features

Breaking changes should always result in a change to the major version number for an API or content response type.

Minor versions

Minor versions only introduce backwards compatible changes. For a minor version, vMAJOR_MINOR would have a MINOR number greater than zero.

When a minor version is released, the endpoint that is already in use will automatically be updated. This will not cause your code to break. You can continue to use your existing API call or client libraries.

Minor versions include new features or updates that do not affect your existing code. If you want to use these new features, then upgrade to the newest version.

Non-breaking changes, such as adding new endpoints or new response parameters, do not require a change to the major version number.

We provide in the following possible situations that we can meet and propose workarounds, in order to keep backward compatibility on different versions of API :

• If there are changes in the request type, for instance, a change of field name, or of datatype of a field, we note the old fields as Depreciated, and probably remove them after a certain duration

• If there are fields that are still in the actual version of API and will be removed in the new version, these fields need to be noted as Depreciated, and probably remove them after a certain duration

• If there are new fields that will be added to the new version of API, new version will have to be changed to consider these fields and give them default values, to satisfy backward compatibility

Group

The API_GROUP refers to a set of one or more functionality that can be grouped together. The endpoints associated with a group can manage different resources as far as they are strictly related with the functionality represented by the group.

By including the group in the resource path, we aim to provide a clear and consistency view of the functionalities exposed by the different APIs, with more control over the groups of functionality (in terms of functional and non-functional aspects) as well as make more clear how we communicate the functional structure of these APIs through the documentation.

Here is an example of a resource path related with the group of functionalities provided by the Account Management REST API:

/api/rest/{version}/accountManagement/sellers/{sellerCode}

The path refers to the accountManagementgroup and any resource/operation related with user account management (in this case for seller with code sellerCode) can be accessed from that path.

Group names should be in camelCase and it can be a single word name or a compound name when we want to group the API into a logical group.

Extensions

To better accommodate and separate the core API from extensions built on top of the API, developers should consider the following path format for extensions:

/api/rest/extensions/{extension_id}/{version}/{API_GROUP}

This separation aims to avoid confusion when consuming stable and official opencell APIs so that clients can differentiate specific, vertical and project APIs from those provided from the opencell core. This will allow extensions to manage separately there API versionning.

Store Resources

A resource store provides operations for a given resource. For example:

/api/rest/{version}/accountManagement/sellers

The example above is representing a resource store path from where we can manage the sellers. Based on this path, clients can perform different operations based on the HTTP verbs or any controller resource defined to the store. Usually this type of resource is associated with CRUD operations for a specific resource.

When a resource fits under this category, the name should be a plural noun.

Controller Resources

Under some circumstances, the HTTP verbs are not enough to represent an operation or action that can be performed in a resource. In this case, the resource path may indicate a specific action to be taken when accessed.

POST /api/rest/{version}/accountManagement/subscriptions/{SUB_CODE}/activate

The example above shows the usage of activate to operate subscriptions in order to activate them.

When a resource fits under this category, the name should be a verb, using camel-case if necessary.

HTTP Verbs on Controller Resources

Controller resources should use the HTTP POST method to perform operations on a resource. There are no restrictions on the request body as long as the resource representation is respected if the action requires a request body.

Resource Representation

The standard format for resource representation is JSON or XML and the former is the preferred format. There is no rule for schemas or the semantics used by a resource representation, as long as they follow the JSON syntax properly.

However, it is important to take into account the following principles when defining resource representations:

• Keep it simple for clients, so that a representation can be easily built and processed when making requests/responses to the API

• Be aware of extensive representations as they can affect overall performance and usability, if possible review your API to see if resources can be decomposed into smaller ones

• Avoid deeply nested JSON objects

• Avoid using generic resource representations

User-friendly resource representation

In Opencell, API endpoints use a type of data object, called Data Transfer Objects (DTO), to interact directly with the user. DTO are objects that encapsulate data for transfer between client - server. The purpose of creating a DTO is to reduce the amount of unnecessary information that needs to be transferred, and also increase security.

Since DTO is used by the user to communicate with the API, they need to be designed to be as easy to understand and grasp as possible. Indeed, they contain only pieces of information which are necessary for the request. A design of DTO containing more or less information than needed will be considered as a bad design.

In the process of designing and developing a new API endpoint, this task will be entrusted to designers, who have a view of data at a high-level (user level), during API documentation, so that we can have a user-friendly DTO. The transformation between the DTO and the corresponding entity is always done later in the service layer.

Http Status Codes

The tables below defines the HTTP status code that clients should expect from APIs.

This document also defines which codes are expected to be used for each HTTP verb, please take a look at HTTP Verbs for more details.

Clients should consider the status codes herein defined in order to properly handle responses from APIs, recover from errors, and understand the reasoning behind the API responses.

Success Codes

Redirection Codes

Client Side Error Codes

Server Side Error Codes

HTTP Verbs

Based on the REST architectural style, HTTP Verbs should be used as the means to communicate to APIs the operations to be performed on their resources. While HTTP verbs cover by themselves the most common operations you might want to look the Controller Resource as an alternative on how to overcome any limitation imposed by the usage of HTTP verbs herein defined.

The table below defines the HTTP verbs you should consider when exposing operations through your API and the circumstances where they apply.

GET

HTTP GET should be used to obtain current state of a resource. It is usually related with operations on resources that return either a list or a single representation, or start some action that does not change the resource state.

Expected Status Codes

It is expected that APIs responding to HTTP GET requests respect the following HTTP status codes to communicate the result of the operation:

POST

HTTP POST should be used to create new resources. It is a non-safe and non-idempotent operation and clients should be aware of that.

Successful POST requests should always result in a 201 status code and empty response. The response should also contain a reference to the resource that was created:

HTTP/1.1 201 Created
Content-Length: 0
Location: https://opencell.com/api/rest/v2/accountManagement/subscriptions/MY_SUB

Expected Status Codes

It is expected that APIs responding to HTTP POST requests respect the following HTTP status codes to communicate the result of the operation:

PUT

HTTP PUT should be used to replace a resource state.

Successful PUT requests should always result in a 204 status code and empty response.

HTTP/1.1 204 No Content
Content-Length: 0

PATCH

HTTP PATCH should be used to partially update the resource state.

Successful PATCH requests should always result in a 204 status code and empty response.

HTTP/1.1 204 No Content
Content-Length: 0

Partial updates should follow the RFC7396 - Json Merge Patch standard.

The PATH operation should be driven by a specific Content-Type header as defined by RFC7396 and other patch strategies. By using the Content-Type as a selector, this guideline can support additional patch strategies in the future.

Note on RFC7396 - Json Merge Patch

The RFC7396 is the most simple PATH strategy, but it has some limitations. The limitations are basically around updating lists on a resource which will be entirely replaced by the patch.

Under certain circumstances, other patch strategies may be useful thus the constraint around the Content-Type header of PATH requests.

Expected Status Codes

It is expected that APIs responding to HTTP PUT requests respect the following HTTP status codes to communicate the result of the operation:

DELETE

HTTP DELETE should be used to remove a resource.

Successful DELETE requests should always result in a 204 status code and empty response.

HTTP/1.1 204 No Content
Content-Length: 0

Expected Status Codes

It is expected that APIs responding to HTTP DELETE requests respect the following HTTP status codes to communicate the result of the operation:

Practices

Error Handling

A good error handling mechanism helps to communicate to clients the reasons why a request could not be fulfilled and eventually recover from this errors.

When returning errors, developers should use a common HTTP Status Code and include a body using JSON as the format with the information why a request failed.

The error message should have the following format:

Here is examples of error responses :

HTTP/1.1 400 Bad Request
{
    "status": "FAIL",
    "errorCode": "invalid_request_exception",
    "message": "The field 'email' is mandatory"
}

HTTP/1.1 401 Unauthorized
{
    "status": "FAIL",
    "errorCode": "unauthorized_exception",
    "message": "Authentication credentials were missing or incorrect"
}

HTTP/1.1 403 VERB Forbidden
{
    "status": "FAIL",
    "errorCode": "verb_forbidden_exception",
    "message": "The request is understood, but it has been refused or access is not allowed"
}

HTTP/1.1 404 Not Found
{
    "status": "FAIL",
    "errorCode": "entity_does_not_exist_exception",
    "message": "Seller with code='sellerCode' does not exist"
}

HTTP/1.1 409 VERB Conflict
{
    "status": "FAIL",
    "errorCode": "verb_conflict_exception",
    "message": "{Any message which should help the user to resolve the conflict}"
}

HTTP/1.1 415 Unsupported Media Type
{
    "status": "FAIL",
    "errorCode": "unsupported_media_type_exception",
    "message": "The format of payload is not supported"
}

We use the error code 500 if the server encountered an unexpected condition that prevented it from fulfilling the request. Therefore, the cause of this can be anything …

HTTP/1.1 500 VERB Internal Server Error
{
    "status": "FAIL",
    "errorCode": "server_error_exception",
    "message": "{Any message which should help the user to understand the error which could
                be timeout}"
}

Error Codes

Error codes can be any string. They should be lowercase and use _ when the code has multiple words. However, developers should consider existing error codes before creating new ones.

When processing error responses, developers should look the error codes being used by other APIs and see if they can be reused.

Pagination

When dealing with large data sets, store resources should consider paginating the results when fetching resource sets.

In Opencell, pagination is based on three request parameters:

• offset

  The position of the first result to retrieve

• limit

  The maximum number of entries to retrieve

• sort

  The sort order of the list of entities returned. We use the sign -to sort in descending order

These parameters should be sent in the body of a GET request as follows:

/api/rest/{version}/users
{
    "offset": 1,
    "limit": 20,
    "sort": "-description"
}

The example above shows how to request the first twenty entries, from the offset 1, and the entries are sorted by the field description in descending order.

As a result, the server should respond as follows:

HTTP/1.1 200 OK
Connection: keep-alive
Cache-Control: no-cache
Content-Type: application/json
Content-Length: 961
Date: Wed, 24 May 2121 13:44:57 GMT

Link: <http:/{host}/api/rest/{version}/users>; rel="next"

As defined by RFC-5988, responses from the server should contain a Link header indicating whether or not there is a next page and, if there are more pages, a link that can be followed to fetch the next page. If the Link header is not present, the client can assume that there are no more pages to fetch.

The possible links and rel values are:

Rate Limiting

Opencell does not provide rate limiting capabilities and for such it is necessary to use a third-party tool such as a API Manager.

Filtering

Sometimes, we need more control over the content that will return from the server, so that only a subset of the data is available in the response. For example, a list user's groups or permissions.

The Opencell's REST API allows to search for users based on optional parameters. However, such implementation is limited to a particular context (e.g, collection of users).

Filtering should be implemented as a query parameter, based on what we already do today. To filter on a field, should be as simple as the following request:

/api/rest/{version}/billingCycles
{
    "likeCriteria description": "*periode*"
}

This request is used to retrieve all billingCycles in the database, whose description contains the string “periode”.

Multiple filters should be written as follows :

/api/rest/{version}/billingCycles
{
    "likeCriteria description": "*periode*",
    "fromRange id": 1
}

The above request would provide results based on two kinds of filter on field descriptionand id. Filtering should take into consideration the list of attributes supported by the API. The list of attributes supported by Opencell’s API is given as follows:

• fromRange

Filter on a numeric field whose value is between a lower bound and plus infinity

• toRange

Filter on a numeric field whose value is between minus infinity and an upper bound

• inList

Filter on a numeric field whose value is in a list

• likeCriteria

Filter on a string field whose value contains a concrete substring. The searching substring should be placed between quotes.

• ne

Filter on a field whose value is different from a concrete value

Asynchronous operations

In Opencell, several API endpoints can take a considerable duration to complete. In order to deal with these processes, we need to create an instant response and process the request in an asynchronous way.

Let take an example of the endpoint “execute“ in the Job API. We create the process by executing this request :

/api/rest/v1/jobs/execute
{
    "code": "APIv2PermissionsSyncJob"
}

Instead of letting the user wait until the operation finishes, we can return the following reply :

HTTP/1.1 202 (Accepted)
{
    "status": "ACCEPTED",
    "location": "http://{host}/api/rest/v1/jobs/jobReports/APIv2PermissionsSyncJob"
}

We remark that instead of returning a HTTP code 200, the server responds with 202, showing that the request has been well received and accepted, but still not fully processed.

The Location header contains the resources permitting to consult the actual status of the request. A user can retrieve this information by issuing the request:

/api/rest/v1/jobs/jobReports/APIv2PermissionsSyncJob

The reply can be formatted as follows, containing all pieces of information about actual status and executed moment, code of executed job instance, etc.

HTTP/1.1 200 (OK)

{
    "actionStatus": {
        "status": "SUCCESS"
    },
    "jobExecutionResultDto": {
        "id": 16,
        "jobInstanceId": -30,
        "startDate": "2021-06-01T10:20:20Z",
        "endDate": "2021-06-01T10:20:21Z",
        "nbItemsToProcess": 0,
        "nbItemsCorrectlyProcessed": 0,
        "nbItemsProcessedWithWarning": 0,
        "nbItemsProcessedWithError": 0,
        "done": false,
        "status": "RUNNING",
        "jobInstanceCode": "APIv2PermissionsSyncJob"
    }
}

HATEOAS

Hypermedia as the engine of application state (HATEOAS) is a constraint of a REST API, making it different from most of the remaining network architectures. This constraint requires that a resource’s state representation includes links to related resources, such as images, videos, text or another resource. Links are the threads that weave the Web together by allowing users to traverse information and applications in a meaningful and directed manner, like when we use a web browser to go through web pages by clicking the relevant hyperlinks to achieve a final objective. The presence, or absence, of a link on a page is an important part of the resource’s current state.

Let consider the following request:

/billing/invoicing/getPreInvoicingReport

Below given JSON response from the request :

HTTP/1.1 200 (OK)

{
    "billingCycleCode": "aCode",
    "billingAccountNumber": "12345",
    "invoiceDate": "2021-05-25T08:25:38",
    "links": [{ 
        "rel": "invoicing",
        "href": "/billing/invoicing/getPostInvoicingReport"
    }]
}

In this example, the response returned by the server contains hypermedia links to post invoicing, which can be traversed by the client to retrieve post invoicing reports.

Documentation

API documentation is a technical content document, explaining many instructions how we can use effectively an API and integrate with an API. It is a kind of manual containing information about return types, arguments, error handling, data fields. API documentation should used as a resource guiding API implementation. Therefore, it should be written by designers, not by developers, at the beginning of development cycle.

A well-written API documentation has many benefits, such as:

• Helps software developers develop in the most accurate way

• API will be easily maintained and upgraded

• Quickly identify bugs and issues at the beginning of development for quick discussion with the product team or designers

• Reduce the time it takes to learn how the API works for the user

• Leads to consensus between designers and developers on the data type, argument, and return type of each API

Nowadays, many tools have been developed in order to meet the need for API documentation, of which the most used is OpenAPISpecification (OAS).

Opencell has chosen this tool for API documentation, actually maintained by the product team and designers. Moreover, developers also use this tool to automatically generate the documentation for the API endpoints that have been developed, in order to compare with the documentation written by the designers.

Conception and development process of an API

As part of the development of a consistent and coherent API for Opencell, we have existing endpoints and new endpoints to be designed.

• For existing endpoints, their conception need to be reviewed in terms of request (data fields, data types), response returned by the server (data fields, data types) in a way that only necessary data is taken into consideration. The error handling needs also to be reviewed.

  • A workaround that we can consider is to generate automatically an API documentation (OpenAPI, Redoc, etc.) for all existing endpoints which will be then reviewed by the product team or designers. The badly designed endpoints need to be corrected by designers and eventually, re-implemented by developers

• For new endpoints, we can have two possible following approaches :

  • Code First Approach (CFA) : New endpoints are directly implemented, from which a human-readable documentation is generated (OpenAPI, Redoc, etc.). Then the generated documentation needs to be reviewed by the product team or designers. This approach has been adopted for a long time at Opencell

  • Design First Approach (DFA) : The initial requirements are converted to a human-readable documentation (OpenAPI, Redoc, etc.) by the product team or designers , from which the implementation will be built

API endpoints need to be tested and finally, ready to be used by customers. Both approaches have advantages and disadvantages.

The CFA is mostly used while developers do not have much time to deliver new endpoints, so that they start coding directly from the requirements. Another case that we can adopt this approach is when developing new internal endpoints which will be only used by internal teams. However, DFA should be taken into account when we would like to produce new endpoints which will be used by external customers or partners. In this case, a good conception step at the beginning will play a key role in customer satisfaction. Moreover, a well designed endpoints help customers quickly understand our API, reducing the time taken to integrate with our API

Examples

CRUD

The following example demonstrates how a Seller API would look and how CRUD operations would be provided:

References

Keycloak Rest API Guide

REST-API-Design-Rulebook.pdf