Use-case

Your API's Service Request endpoint returns a payload that needs to be processed prior to responding to the API consumer.

The problem

Let's assume that we expect a JSON payload (body). You are likely to have a Groovy Policy in the API's Response pipeline in order to access the Service Request's payload. To do so you would instinctively use this code:-

context.getServiceResponse().getBody().asJSONObject()

This seems quite natural because context.ServiceResponse is a reference to the IncomingMessage interface and that's what we're conceptually dealing with.

However, this code could fail in a particular and perhaps not unusual circumstance.

HTTP and data compression

The HTTP standard allows for message compression - most commonly used is 'gzip'. In order for the server to even consider compressing a payload, the client needs to tell the server that it will accept compressed data. Even if the client indicates that compressed data are acceptable, the server doesn't necessarily execute any form of compression. 

If the client will accept compressed (gzip) messages it would set the Accept-Encoding header to a value of 'gzip'. [ This is commonly set to 'gzip, deflate' even though 'deflate' is somewhat outdated. More detail about this header can be found here ]

If the server does decide to compress the payload, it will inform the client by setting the Content-Encoding header to 'gzip' (in the response).

Hence, the client needs to take account of the Content-Encoding header when dealing with the reply.

Different servers will have varying strategies for determining if compression is worthwhile. Thus the client cannot assume one way or the other and needs to check the Content-Encoding header to be certain how to handle the response.

A real-life scenario

A client invokes a RESTful Service Endpoint with various query parameters that will, unsurprisingly, affect the response. The client also sets the Accept-Encoding header to 'gzip'. 

One particular set of query parameters results in a relatively small payload - say, less than 1k bytes. The server decides that there's no benefit in compressing the response and so does not return a Content-Encoding header. The client can therefore imply that it can take the payload 'as is'.

Now we invoke the same Service Endpoint but with different parameters that result in a payload >4k bytes. The server decides to compress and therefore returns a Content-Encoding header set to 'gzip'. The client observes this and knows that it needs to decompress the response.

The point of contriving this scenario is to emphasise that the same Service Endpoint but with different input parameters (whether they be query parameters or based on a POSTed payload) may or may not compress the response.

Why might this be a problem?

If we invoke the Service Endpoint in our contrived scenario and receive a compressed response, this line of code will generate an exception:-

context.getServiceResponse().getBody().asJSONObject()

This happens due to a peculiarity in APIPCS whereby:-

context.ServiceResponse.Body

...is actually a reference to the raw response.

The decompressed body should actually be accessed via:-

context.ApiResponse.Body

In fact, we can generalise this and say that accessing the response body from the IncomingMessage interface within the Response pipeline is never good practice.

Hard to find

Just to emphasise this last point, this blog post and explanation came from an unfortunate customer experience.

They needed to access the response body and only return a subset of the data to the API consumer. They found that sometimes it worked but on other occasions it would fail. It transpired that compressed data were defined as being acceptable. Then, when the Service Endpoint server had a payload response that warranted compression, the custom code would fail.

The fix

Of course, we now know that best practice is to refer to context.ApiResponse.Body rather than context.ServiceResponse.Body when utilising a Groovy Policy in the Response pipeline.

It's also worth mentioning that we can ensure that the server will not compress under any circumstances by setting Accept-Encoding to 'identity'.