RESTful APIs enable you to develop any kind of web or api base mobile application having all possible CRUD (create, retrieve, update, delete) operations. REST guidelines suggest using a specific HTTP method on a specific type of call made to the server. Today, we'll cover below topics regarding the RESTFul apis,
REST API PRINCIPLES
REST API TYPES
IMPLEMENTATION LIBRARIES
REQUEST FORMAT
RESPONSE FORMAT
API VERSIONS
API DESIGN GUIDELINES
API SECURITY PRINCIPLES
API SECURITY GUIDELINES
API TESTING GUIDELINES
Let's deep dive in each of above topic,
1) REST API PRINCIPLES:
Stateless : The requests sent from a client to a server will contain all the required information to make the server understand the requests sent from the client. This can be either a part of URL, query-string parameters, body, or even headers. The URL is used to uniquely identify the resource and the body holds the state of the requesting resource. Once the server processes the request, a response is sent to the client through body, status or headers
Client-Server : The client-server architecture enables a uniform interface and separates clients from the servers. This enhances the portability across multiple platforms as well as the scalability of the server components.
Uniform Interface : To obtain the uniformity throughout the application, REST has the following four interface constraints:
Resource identification
Resource Manipulation using representations
Self-descriptive messages
Hypermedia as the engine of application state
Cacheable : In order to provide a better performance, the applications are often made cacheable. This is done by labelling the response from the server as cacheable or non-cacheable either implicitly or explicitly. If the response is defined as cacheable, then the client cache can reuse the response data for equivalent responses in the future.
Layered system : The layered system architecture allows an application to be more stable by limiting component behaviour. This type of architecture helps in enhancing the application’s security as components in each layer cannot interact beyond the next immediate layer they are in. Also, it enables load balancing and provides shared caches for promoting scalability.
Code on demand : This is an optional constraint and is used the least. It permits a clients code or applets to be downloaded and to be used within the application. In essence, it simplifies the clients by creating a smart application which doesn’t rely on its own code structure.
2) REST API TYPES :
All of us working with the technology of the web, do CRUD operations. When I say CRUD operations, I mean that we create a resource, read a resource, update a resource and delete a resource. Now, to do these actions, you can actually use the HTTP methods, which are nothing but the REST API Methods. Below are the types,
HTTP GET
HTTP POST
HTTP PUT
HTTP DELETE
HTTP PATCH
HTTP GET :
Use GET requests to retrieve resource representation/information only – and not to modify it in any way. As GET requests do not change the state of the resource, these are said to be safe methods. Additionally, GET APIs should be idempotent, which means that making multiple identical requests must produce the same result every time until another API (POST or PUT) has changed the state of the resource on the server.
If the Request-URI refers to a data-producing process, it is the produced data which shall be returned as the entity in the response and not the source text of the process, unless that text happens to be the output of the process.
For any given HTTP GET API, if the resource is found on the server, then it must return HTTP response code 200 (OK) – along with the response body, which is usually either XML or JSON content (due to their platform-independent nature).
In case resource is NOT found on server then it must return HTTP response code 404 (NOT FOUND). Similarly, if it is determined that GET request itself is not correctly formed then server will return HTTP response code 400 (BAD REQUEST).
Example request URIs
HTTP GET http://www.appdomain.com/users
HTTP GET http://www.appdomain.com/users?size=20&page=5
HTTP GET http://www.appdomain.com/users/123
HTTP GET http://www.appdomain.com/users/123/address
HTTP POST :
Use POST APIs to create new subordinate resources, e.g., a file is subordinate to a directory containing it or a row is subordinate to a database table. Talking strictly in terms of REST, POST methods are used to create a new resource into the collection of resources.
Ideally, if a resource has been created on the origin server, the response SHOULD be HTTP response code 201 (Created) and contain an entity which describes the status of the request and refers to the new resource, and a Location header.
Many times, the action performed by the POST method might not result in a resource that can be identified by a URI. In this case, either HTTP response code 200 (OK) or 204 (No Content) is the appropriate response status.
Responses to this method are not cacheable, unless the response includes appropriate Cache-Control or Expires header fields.
Please note that POST is neither safe nor idempotent, and invoking two identical POST requests will result in two different resources containing the same information (except resource ids).
Example request URIs
HTTP POST http://www.appdomain.com/users
HTTP POST http://www.appdomain.com/users/123/accounts
HTTP PUT :
Use PUT APIs primarily to update existing resource (if the resource does not exist, then API may decide to create a new resource or not). If a new resource has been created by the PUT API, the origin server MUST inform the user agent via the HTTP response code 201 (Created) response and if an existing resource is modified, either the 200 (OK) or 204 (No Content) response codes SHOULD be sent to indicate successful completion of the request.
If the request passes through a cache and the Request-URI identifies one or more currently cached entities, those entries SHOULD be treated as stale. Responses to this method are not cacheable.
The difference between the POST and PUT APIs can be observed in request URIs. POST requests are made on resource collections, whereas PUT requests are made on an individual resource.
Example request URIs
HTTP PUT http://www.appdomain.com/users/123
HTTP PUT http://www.appdomain.com/users/123/accounts/456
HTTP DELETE :
As the name applies, DELETE APIs are used to delete resources (identified by the Request-URI).
A successful response of DELETE requests SHOULD be HTTP response code 200 (OK) if the response includes an entity describing the status, 202 (Accepted) if the action has been queued, or 204 (No Content) if the action has been performed but the response does not include an entity.
DELETE operations are idempotent. If you DELETE a resource, it’s removed from the collection of resources. Repeatedly calling DELETE API on that resource will not change the outcome – however calling DELETE on a resource a second time will return a 404 (NOT FOUND) since it was already removed. Some may argue that it makes the DELETE method non-idempotent. It’s a matter of discussion and personal opinion.
If the request passes through a cache and the Request-URI identifies one or more currently cached entities, those entries SHOULD be treated as stale. Responses to this method are not cacheable.
Example request URIs
HTTP DELETE http://www.appdomain.com/users/123
HTTP DELETE http://www.appdomain.com/users/123/accounts/456
HTTP PATCH :
HTTP PATCH requests are to make partial update on a resource. If you see PUT requests also modify a resource entity so to make more clear – PATCH method is the correct choice for partially updating an existing resource and PUT should only be used if you’re replacing a resource in its entirety.
Please note that there are some challenges if you decide to use PATCH APIs in your application:
Support for PATCH in browsers, servers, and web application frameworks is not universal. IE8, PHP, Tomcat, Django, and lots of other software has missing or broken support for it.
Request payload of PATCH request is not straightforward as it is for PUT request. e.g.
HTTP GET /users/1
produces below response:
{id: 1, username: 'admin', email: 'email@example.org'}
A sample patch request to update the email will be like this:
HTTP PATCH /users/1
[
{ “op”: “replace”, “path”: “/email”, “value”: “new.email@example.org” }
]
There may be following possible operations are per the HTTP specification.
[
{ "op": "test", "path": "/a/b/c", "value": "foo" },
{ "op": "remove", "path": "/a/b/c" },
{ "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
{ "op": "replace", "path": "/a/b/c", "value": 42 },
{ "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
{ "op": "copy", "from": "/a/b/d", "path": "/a/b/e" }
]
PATCH method is not a replacement for the POST or PUT methods. It applies a delta (diff) rather than replacing the entire resource.
3) REST API IMPLEMENTATION LIBRARIES :
Let's checkout few RESTFul apis implementation libraries in PHP & Dotnet.
1. PHP : RestController Library / Slim framework
Rest Controller API Dev Tutorials:
https://code.tutsplus.com/tutorials/working-with-restful-services-in-codeigniter--net-8814
https://www.codeofaninja.com/2017/02/create-simple-rest-api-in-php.html
https://www.c-sharpcorner.com/UploadFile/528a80/create-rest-api-for-android-app-using-php-and-mysql/
https://www.simplifiedcoding.net/android-login-and-registration-tutorial/
Slim framework API Dev Tutorials:
2. DOTNET : Asp.net Core WebApi
4) API REQUEST FORMAT
HTTP POST :
API URL | HTTP_METHOD | HEADERS | QUARY_PARAM | BODY_PARAM |
POST | VersionNo : V.1.0 AuthToken : "" Content-Type: application/json | NA |
HTTP GET :
API URL | HTTP_METHOD | HEADERS | QUARY_PARAM | BODY_PARAM |
GET | VersionNo : V.1.0 AuthToken : "" Content-Type: application/json | id | NA |
HTTP PUT :
API URL | HTTP_METHOD | HEADERS | QUARY_PARAM | BODY_PARAM |
PUT | VersionNo : V.1.0 AuthToken : "" Content-Type: application/json | NA | { “id”: 24, “author”: “Viktor Farcic” } |
HTTP DELETE :
API URL | HTTP_METHOD | HEADERS | QUARY_PARAM | BODY_PARAM |
DELETE | VersionNo : V.1.0 AuthToken : "" Content-Type: application/json | NA | { “id”: 24 } |
HTTP PATCH :
API URL | HTTP_METHOD | HEADERS | QUARY_PARAM | BODY_PARAM |
PATCH | VersionNo : V.1.0 AuthToken : "" Content-Type: application/json | NA | { “userId” : 11, "first-name" : “MyName" } |
5) API RESPONSE FORMAT
HTTP RESPONSE CODES :
Here are some HTTP response codes, which are often used with REST:
200 OK
This response code indicates that the request was successful.
201 CREATED
This indicates the request was successful and a resource was created. It is used to confirm success of a PUT or POST request.
204 – NO CONTENT:
The resource was successfully deleted (no response body).
304 – NOT MODIFIED:
The date returned is cached data (data has not changed).
400 BAD REQUEST
The request was malformed. This happens especially with POST and PUT requests, when the data does not pass validation, or is in the wrong format.
401 UNATHOURISED
This error indicates that you need to perform authentication before accessing the resource.
403 – FORBIDDEN:
The server understood the request, but is refusing it or the access is not allowed.
404 NOT FOUND
This response indicates that the required resource could not be found. This is generally returned to all requests which point to a URL with no corresponding resource.
405 METHOD NOT ALLOWED
The HTTP method used is not supported for this resource.
409 CONFLICT
This indicates a conflict. For instance, you are using a PUT request to create the same resource twice.
500 INTERNAL SERVER ERROR
When all else fails; generally, a 500 response is used when processing fails due to unanticipated circumstances on the server side, which causes the server to error out.
CUSTOM RESPONSE CODES :
We can implement the custom response code the way we want. But the heavy customisation can leads to multiple uncertainty in using the apis. Therefore it’s necessary to restrict everyone to common standards. For any given HTTP API, server must return HTTP response code 200 (OK) – along with the response body. The api should provide custom response code “SUCCESS” & “FAIL” along with responseData for next relevant operation.
{
"responseCode" : "SUCCESS",
“responseData”: { }
}
Or
{
"responseCode" : “FAIL",
"message" : "Book with relevant id not found.”
}
6) API VERSIONS
When your APIs are being consumed by the world, upgrading the APIs with some breaking change would also lead to breaking the existing products or services using your APIs.
http://api.yourservice.com/v1/companies/34/employees
is a good example, which has the version number of the API in the path. If there is any major breaking update, we can name the new set of APIs as v2 or v1.x.x
Keeping version number in header is better practice for multiple reasons.
7) API DESIGN GUIDELINES
Naming Best Practices
Use nouns to represent resources
Use “singular” name to denote document resource archetype.
Use “plural” name to denote collection resource archetype.
Use “plural” name to denote store resource archetype.
Use “verb” to denote controller archetype.
2. The URL should only contain resources(nouns) not actions or verbs. For example, the API path …/addNewEmployee contains the action addNew along with the resource name Employee which is wrong. Never use CRUD function names in URIs
HTTP GET http://api.example.com/device-management/managed-devices //Get all devices
HTTP POST http://api.example.com/device-management/managed-devices //Create new Device
3. Use HTTP methods (GET, POST, DELETE, PUT, PATCH) to appropriate operations
4. HTTP status codes are bunch of standardised codes which has various explanations in various scenarios. The server should always return the right status code.
5. If the request body or response type is JSON then please follow camelCase to maintain the consistency
6. Use forward slash (/) to indicate hierarchical relationships
http://api.example.com/device-management/managed-devices
7. Do not use trailing forward slash (/) in URIs
http://api.example.com/device-management/managed-devices /*This is much better version*/
8. Use hyphens (-) to improve the readability of URIs
http://api.example.com/inventory-management/managed-entities/{id}/install-script-location //More readable
9. Do not use underscores ( _ )
http://api.example.com/inventory-management/managed-entities/{id}/install-script-location //More readable
http://api.example.com/inventory_management/managed_entities/{id}/install_script_location //More error prone
10. Use lowercase letters in URIs
http://api.example.org/my-folder/my-doc
11. Do not use file extensions
http://api.example.com/device-management/managed-devices.xml /*Do not use it*/
http://api.example.com/device-management/managed-devices /*This is correct URI*/
12. Searching, sorting, filtering and pagination actions are simply the query on one data set. There will be no new set of APIs to handle these actions. We need to append the query parameters with the GET method API. If adding many query params in GET methods makes the URI too long, the server may respond with 414 URI Too long HTTP status, in those cases params can also be passed in the request body of the POST method. Use query component to filter URI collection like,
http://api.example.com/device-management/managed-devices?region=USA
http://api.example.com/device-management/managed-devices?region=USA&brand=XYZ
8) REST SECURITY DESIGN PRINCIPLES
Least Privilege: An entity should only have the required set of permissions to perform the actions for which they are authorized, and no more. Permissions can be added as needed and should be revoked when no longer in use.
Fail-Safe Defaults: A user’s default access level to any resource in the system should be “denied” unless they’ve been granted a “permit” explicitly.
Economy of Mechanism: The design should be as simple as possible. All the component interfaces and the interactions between them should be simple enough to understand.
Complete Mediation: A system should validate access rights to all its resources to ensure that they’re allowed and should not rely on the cached permission matrix. If the access level to a given resource is being revoked, but that isn’t reflected in the permission matrix, it would violate the security.
Open Design: This principle highlights the importance of building a system in an open manner—with no secret, confidential algorithms.
Separation of Privilege: Granting permissions to an entity should not be purely based on a single condition, a combination of conditions based on the type of resource is a better idea.
Least Common Mechanism: It concerns the risk of sharing state among different components. If one can corrupt the shared state, it can then corrupt all the other components that depend on it.
Psychological Acceptability: It states that security mechanisms should not make the resource more difficult to access than if the security mechanisms were not present. In short, security should not make worse the user experience.
9) API SECURITY GUIDELINES
Always Use HTTPS : By always using SSL, the authentication credentials can be simplified to a randomly generated access token that is delivered in the username field of HTTP Basic Auth. It’s relatively simple to use, and you get a lot of security features for free.
Use Password Hash : Passwords must always be hashed to protect the system (or minimize the damage) even if it is compromised in some hacking attempts. There are many such hashing algorithms which can prove really effective for password security e.g. PBKDF2, bcrypt and scrypt algorithms.
Never expose information on URLs : Usernames, passwords, session tokens, and API keys should not appear in the URL, as this can be captured in web server logs, which makes them easily exploitable.
Consider OAuth : Though basic auth is good enough for most of the APIs and if implemented correctly, it’s secure as well – yet you may want to consider OAuth as well. The OAuth 2.0 authorization framework enables a third-party application to obtain limited access to an HTTP service
Consider Adding Timestamp in Request : Along with other request parameters, you may add a request timestamp as an HTTP custom header in API requests. The server will compare the current timestamp to the request timestamp and only accepts the request if it is within a reasonable time frame (1-2 minutes, perhaps). This will prevent very basic replay attacks from people who are trying to brute force your system without changing this timestamp.
Input Parameter Validation : Validate request parameters on the very first step, before it reaches to application logic. Put strong validation checks and reject the request immediately if validation fails.
Authorisation : It is also important to have whitelist permissible methods. Sensitive resource collections and privileged actions should be protected. The API key or session token should be sent as a body parameter or cookie to make sure that privileged actions or collections are efficiently protected from unauthorised use.
Authentication : It is important to be in a position to verify the authenticity of any calls made to one’s API. The ideal way would be to have a shared secret with all authorised users.
The simplest form of authentication is the username and password credentials one. Other types would include multi-factor authentication and token-based authentication.
9. Auditing : Once in a while, security related events could take place in an organisation. It is imperative that thorough auditing is conducted on the system.
This would involve writing audit logs both before and after the said event.
Token validation errors should also be logged in so as to ensure that attacks are detected. Log data should be sanitised beforehand for purposes of taking care of log injection attacks.
10. TLS Handshake: Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), are cryptographic protocols that provide communications security over a computer network.
11. Cryptography : When it comes to security, this is probably the most important of the guidelines when building a REST API. In order to secure the DATA, you have to consider the following:
Data in transit - The encryption of data transmitted to your API, this is mostly done on the infrastructure level like TLS and SSL support.
Data in storage - The data you store, especially sensitive data, must be encrypted or have restricted access (DB access, File Storage, etc..)
Message integrity - The integrity of the data transmitted from your API to storage or to the client should be kept. Except for TLS, you can also calculate checksums (when it comes to files or objects that are uploaded in parts) or hashes of certain data in order to verify that no one handled the data while it was transmitted.
12. DoS Attacks : In a Denial of Service (DoS) attack, the attacker usually sends excessive messages asking the network or server to authenticate requests that have invalid return addresses. DoS attacks can render a RESTful API into a non-functional state if the right security measures are not taken.
13) Anti-Farming : There are always several marketing-heavy websites that offer consumers the best deal on everything from flights to vehicles and even groceries.In case your API does not have an Authorization/Authentication mechanism, it might lead to miss-use of your API, loading the servers and the API itself making it less responsive to others.
10) API TESTING GUIDELINES
Test whether the REST API syntax followed design guidelines
Test whether the REST API followed standard request formats
Test whether the REST API followed standard response formats
Test whether the REST API performing the operation it made for
Test whether the REST API works by varying request data set
Test whether the REST API prone to basic security concerns
Test the inter-dependant apis together to understand impact of operations performed by apis
Verify content updated / edited / deleted by POST/ PUT/ DELETE / PATCH apis followed by calling relevant GET apis / directly checking update in server database.
Conclusion
The most important takeaways for designing high-quality REST APIs is to have consistency by following web standards and conventions. JSON, SSL/TLS, and HTTP status codes are all standard building blocks of the modern web.
Performance is also an important consideration. We can increase it by not returning too much data at once. Also, we can use caching so that we don’t have to query for data all the time.
Paths of endpoints should be consistent, we use nouns only since the HTTP methods indicate the action we want to take. Paths of nested resources should come after the path of the parent resource. They should tell us what we’re getting or manipulating without the need to read extra documentation to understand what it’s doing.