Creating a Mobile-Optimized REST API Using Oracle Service Bus – Part 1

Introduction

To build functional and performant mobile apps, the back-end data services need to be optimized for mobile consumption. RESTful web services using JSON as payload format are widely considered as the best architectural choice for integration between mobile apps and back-end systems. At the same time, most existing enterprise back-end systems provide a SOAP-based web service application programming interface (API) or proprietary file-based interfaces. In this article series we will discuss how Oracle Service Bus (OSB) 12c can be used to transform these enterprise system interfaces into a mobile-optimized REST-JSON API. This architecture layer is sometimes referred to as Mobile Oriented Architecture (MOA) or Mobile Service Oriented Architecture (MOSOA). A-Team has been working on a number of projects with OSB 12c to build this architecture layer. We will explain step-by-step how to build this layer, and we will  share tips, lessons learned and best practices we discovered along the way. In this first part we will discuss how to design the REST API.

Other parts published in this series:

  • Part 2: Implementation of GET resources
  • Part 3: Implementation of POST and PUT resources and use of query parameters
  • Part 4: Techniques for logging, debugging, troubleshooting and exception handling

Main Article

Design Considerations

Let’s start with the first challenge: how do you design an API that is truly optimized for mobile apps? A common pitfall is to start with the back-end web services, and take that back-end payload as a starting point. While that may limit the complexity of transformations you have to do in OSB 12c (you could even use the automated “Publish-As-REST” function on a SOAP business service) it leads to an API which is everything but optimized for mobile. This brings us to our first recommendation:

The REST API design should be driven by the mobile developer.

He (or she) is the only one who can combine all the requirements, information and knowledge required for a good design:

  • he designs and builds the various screens, knows the supported form factors and knows exactly which data should be retrieved for which screen.
  • he knows the requirements for working in offline mode, and knows how this can be supported and implemented using his mobile development tool set.
  • he is responsible for data caching strategies to optimize performance in both online and offline scenarios
  • he decides which read and write actions can be performed in a background thread not impacting the user-perceived performance.

To illustrate how the above aspects impact the design of the API, we will introduce the sample “human resources” app that we will use throughout this article series. Lets start with the three screen mockups our API should support:

mockups

A first design for the read (GET) resources can look like this

  • /departments: returns list of departments containing department number and name. A “quickSearch” query parameter might be added to support filtering if this cannot be implemented or is undesirable to perform on the device because of the size of the dataset.
  • /departments/{departmentId}: returns all department attributes for the department matching the {departmentId} path parameter and a sub list of all employees working in this department consisting of id, firstname and lastName attributes.
  • /departments/{departmentId}/employees/{employeeId}: returns all employee attributes for the employee matching the {employeeId} path parameter.

As you can see this design is driven by the screens. It allows for “on-demand” data loading, using lean resources that only send the absolutely neccessary set of data across the wire, minimizing payload size and maximizing performance. This design is clearly optimized for online usage of the application.  If the mobile developer has to support an offline usage scenario, he would need to do the following to prepare the app for offline usage, storing all data locally on the device:

  • Call the /departments resource
  • Loop over all the departments returned, and for each department call the  /departments/{departmentId} resource.
  • Loop over all employees returned, and for each employee call the /departments/{departmentId}/employees/{employeeId} resource

Needless to say that this is not a very efficient way of data loading for offline usage. It can easily result in hundreds of REST calls causing a delay of many minutes to prepare the app for offline usage. So, to support offline usage, it would be handy to add a query param “expandDetails” to the /departments resource which when set to “true” would return all department and employee data in one roundtrip.

Of course there are limits to the amount of data you can offload to your phone or tablet. You sure don’t want to store the complete enterprise back-end database on your phone!  So, in our sample, depending on the number of departments in the back-end database, we might need additional query parameters to allow the mobile user to select a specific subset of departments for offline usage.

At this point you might think, no worries, I only have to support an online usage scenario. Well, not too fast… A-team has learnt from our experiences in mobile engagements that more aggressive upfront data caching strategies might still be needed for various performance-related reasons:

  • The app might be used in parts of the world where network connectivity and network bandwidth is unreliable so users prefer to have a longer waiting time at app startup to prevent network hick ups while using the app.
  • The performance of the back-end API calls might turn-out to be too slow for “on-demand” data loading.
  • The REST-JSON transformations in service bus are typically very fast. However, the required JSON payload might require assembling data from various back-end data sources, seriously degrading performance when looping over result sets is needed to get additional lookup data.

Let’s clarify this last point with an example. Assume there is a back-end HR interface that returns all employee data except for the job title (only job code is returned). Another “lookup” interface returns the job details including the job title. In service bus, a loop over all employees is then needed, and for each employee a call to the jobs “lookup” interface is needed to add the job title to the payload. If the lookup call takes just one second, the performance loss can already be significant with tens of employees returned.  In such a situation you have two options: cache the job lookup data in service bus to prevent expensive calls for each employee, or modify the JSON payload to pass only the job id and do the job lookup on the mobile device. This last option would require an additional /jobs resource in your design that returns all job titles for you to cache on the device.

In summary: various user groups might have different data caching needs, and initial data caching strategies might need to be revisited for performance reasons.

We can distill an important lesson from the above:

Your REST API design should be flexible in terms of the data caching options it can support.

Documenting the Design

Developers building traditional enterprise system interfaces often follow a design-by-contract approach. In the XML-based web services world, this means the use of XML Schema’s (XSD’s) and Web Service Definition Language (WSDL) to formally specify the interfaces. However, mobile developers live in a different world, they just think in JSON payloads! While there are emerging standards in the REST-JSON world like RAML and SWAGGER, most mobile developers currently prefer to use sample JSON payloads to document the design. Sample payloads can be changed easily, reflecting the agile and flexible nature mobile developers are used to when working with JavaScript-based mobile frameworks like Angular or Ionic.

So, as we started off with recommending to have the mobile developer drive the design, we should facilitate him with a documentation format he is comfortable with: sample payloads, together with the resource path, path and query parameters, the HTTP method and a short description. Optionally, you can add security constraints. Here is the design of our HR sample REST-JSON API based on the above screens.

Resource Method Description Req/Resp Payload
/departments
?expandDetails=true/false
GET Department list (query param is optional) Sample
/departments/{departmentId} GET Department details and list of employees Sample
/departments/{departmentId}/employees/{employeeId} GET Employee details Sample
/departments/{departmentId}/employees POST Add new employee to department Sample
/departments/{departmentId}/employees/{employeeId} PUT Update employee Sample
/employees GET Employee list Sample
/jobs GET Job list Sample

The last two resources are needed to populate the manager and job drop down lists when updating employee data.

Another advantage of specifying and agreeing on the sample payloads upfront is the ability to start developing the mobile UI against a mock-up REST API while the service bus developer is busy implementing the real API. There are some great free tools on the marketplace that make it easy to create such a mock-up API. A-Team succesfully used a combination of NodeJs,Express and MongoDB to do this. With MongoDB you can read and write any JSON payload without specifying a schema upfront, which makes it very easy to change the payload formats as you go. While we had no prior knowledge of these technologies, we got a working mockup API in a few hours, after following this tutorial.

Implementing the Design

In part 2 of this article series we will start implementing this design using Oracle Service Bus 12c. We will integrate with a back-end interface that is provided in the form of an ADF BC SDO SOAP web service. In subsequent parts, all aspects of the service bus implementation will be discussed, including XML schema’s, use of pipeline pairs en routers, pipeline templates, error handling, logging and security. No prior knowledge of service bus will be assumed. Stay tuned!

Add Your Comment