Everything you Should Know about the API Platform Groovy Policy

Introduction

Developers using the API Platform Cloud Service often make use of the built-in policies that comes with the platform; to implement any logic that needs to be executed before delivering the actual message to the backend service. One common policy used is the Groovy policy, which allows API developers to write Groovy scripts that can be executed either within the request or the response pipeline. These scripts can be used to inspect and even modify the message content (payload, queries, headers) therefore a very powerful tool to have at their disposal.

Because Groovy is a programming language, there are no boundaries of what can be implemented other than what is supported by its specification. Moreover; Groovy will accept any code that is valid in Java, bringing even more powerfulness to your APIs. However, this power comes with a high price to be paid.

The biggest challenge with the Groovy policy is that you have no way to verify if the code written is correct until it is deployed into the gateways. During deployment; the Groovy script will be transformed into a Java class that uses the code written as part of its implementation. If something in the script is wrong, there will be compilation errors that will cause the deployment of the API to fail, as shown in Figure 1.

Failed_Deploy

Figure 1: API Platform Management Console showing an API deployment that had failed.

This whole situation can be quite frustrating if complex Groovy code is being used or if one API accesses multiple instances of the Groovy policy, each one carrying different scripts. Hours can be spent trying to troubleshoot the root-cause of the deployment issue, which is a situation that the Oracle A-Team has been experiencing quite often with different customers. Thus, this blog will discuss the most important details about the Groovy policy, its common pitfalls and which situations must be avoided to ensure a smooth API deployment.

The Basics: Groovy Policy Under-the-Hood

Given the simplicity of the Groovy policy UI, you may be thinking that its internal implementation is also quite simplistic. After all, how complex should be getting a script that contains Groovy code and executing it in a JVM right? the truth is that it is way more complex than what may appear, and properly understanding what happens under-the-hood may give you a better perspective about what to expect from this policy.

The API Platform Cloud Service follows an interesting design that clearly separates where the APIs are defined, and where they are executed. While the APIs are defined in the management portal available in the Oracle Cloud; they are effectively deployed and executed in the gateways that can be installed in a given Cloud provider (Oracle, AWS, Azure) or On-Premise. Periodically, the gateways keep pulling the metadata of the APIs to keep in-sync with the management portal. If new metadata is received, the APIs will be (re)deployed in the gateways – and along with it – any policies that had been defined during design-time will be deployed as well. The gateways uses a all-or-nothing deployment strategy; meaning that if some policy fails to deploy, the whole API deployment will also fail.

Although they are part of a larger deployment transaction; each policy that is used within the API has its own deployment scope, which includes the steps of instantiating the action that holds the policy runtime implementation, and its configuration based on the settings defined for the policy during design-time.

In this context; the term action here refers to the DAF (Dynamic Application Framework) action from the OCSG (Oracle Communication Services Gatekeeper) platform, which is the technology used for the gateways. Actions are registered once and reused multiple times across different APIs, and they are the underlying implementation of the policies.

When an action is instantiated it receives the settings from the policy, that had been set by the user(s) during design-time. These settings are used to customize the policy behavior in runtime. But before being able to execute anything; the action must first set the stage for it, and that happens during the instantiation phase. Although most actions have a considerably simple instantiation phase, this is not the case for the action behind the Groovy policy.

For each Groovy policy deployed, its underlying action will create a custom classloader that will be used to load a Java class built from the script code. This classloader has several customization’s in place, which aims to ensure that no malicious code is loaded in the JVM. Internally, a source-code for a Java class will be generated, and this source-code includes the script code set for the policy. The source-code for this class will be compiled and the resulting bytecodes will be loaded in the JVM by the classloader. This generated class implements an interface called GroovyScriptLet, which contains a method called execute(), as shown in Figure 2.

GroovyScriptlet

Figure 2: Definition of the GroovyScriptlet interface.

The code written in the Groovy policy is used to implement the execute() method. Because of this, any typos or mistakes written in the script will surface as compilation errors. So in nutshell, be aware of the most important concept about the Groovy policy: although Groovy is a script-based language, the written code will be compiled like any other Java class. Figure 3 shows a high-level overview about how things work behind the scenes during the instantiation of an action.

ActionInstantiation

Figure 3: Action instantiation for the Groovy policy.

If compiled successfully; then the generated class will be loaded by the custom classloader in the JVM and an object of that class will be created, and it will be kept inside the action. In runtime, the action will delegate any request to this object, which in turn will execute the code defined on its execute() method.

Now that you are up-to-speed about how the Groovy policy works, we can start discussing situations that you might want to avoid to ensure that your APIs are correctly deployed. The following sections will cover situations that should be avoided, as well as best practices that may shorten the time spent with payload handling using Groovy.

Type Declarations and Import Statements

As discussed in the previous section; when a Groovy policy is used, a Java class is generated and this class uses the code written in the policy as the implementation for its execute() method. Thus, it’s important to understand that you have to write code that is acceptable within a Java method. To illustrate this situation, let’s discuss how type declarations should work. Figure 4 shows a set of examples of invalid and valid type declarations.

ClassDeclarationsWithinMethod

Figure 4: Examples of invalid and valid type declarations.

As illustrated in Figure 4, conventional type declarations are not possible in the Groovy policy. The only exception is when you declare a type using anonymous classes. In this case, you are declaring a type that is supposed to exist only during the lifecycle of the method and therefore, the declaration is permitted.

You might be aware that the gateway technology used by the API Platform runs on top of the JDK 1.8. Therefore, any piece of code that is supposed to work with that version of the JDK would run without problems, and that also includes Lambdas expressions. This would be particularly interesting if you want to reduce code complexity by replacing all the anonymous classes with Lambdas. However, this is not going to work within the Groovy policy since the “->” operator is not recognized by the Groovy compiler.

Import statements should also be used carefully. Differently from regular Groovy scripts where you can import types in the beginning of the script and refer to them throughout the code, you should perform type imports on-demand, as shown in Figure 5.

ImportStmts

Figure 5: Examples of invalid and valid type declarations.

Fortunately, there is a considerable amount of types in Groovy that are imported by default and won’t require explicit import. However, there are a few restrictions that must be taken into consideration with the Groovy policy, which are going to be discussed in the next section.

Restricted Packages, Classes and Methods

When the underlying action of the Groovy policy is instantiated, it creates a custom classloader that will be responsible to load and instantiate the Java class that contains the code written in the policy. As pointed out previously, this custom classloader has several customization’s in place. This section will discuss one of these customization’s, and how it may affect the deployment process of the Groovy policy.

The custom classloader uses something called expression checker. The expression checker validates which packages, classes and methods are mentioned in your code. The validation is implemented by verifying the expressions written in your code against a black-list. If an expression matches with some item contained in the black-list, then the expression checker returns false and the code is not allowed to be used. This happens in the compilation phase, which means that if some prohibited expression is used, the compilation will fail and the deployment of the policy will be aborted. Therefore, it is important to know which packages, classes and methods cannot be used within your Groovy scripts. Table 1 shows the black-list used by the Groovy policy.

Table

Table 1: List of unauthorized expressions for the Groovy policy.

The list shown in Table 1 clearly contains elements, that if allowed to be used, could cause some serious damage to the underlying gateway health. It would open the possibility to execute code that could potentially exhaust the compute resources (CPU, Memory, Disk and Network) available for its JVM.

Painless JSON and XML Payload Handling

This section is going to detail some best practices that you can use to speed up your development time while working with JSON or XML payloads. Often API developers find themselves in need of ways to read and parse payloads, whether if it is in JSON or XML. For this situation, the Groovy language provides two classes that everybody should know: JsonSlurper and XmlSlurper.

Let’s start with one example of XML parsing: Imagine that you have an API that receives SOAP messages from the backend service. You need to parse that message and retrieve only a couple fields to construct the API response in the JSON format. Listing 1 shows a sample XML payload that contains a SOAP message.

<?xml version='1.0' encoding='UTF-8'?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <soap:Body>
      <GetSunSetRiseTimeResponse xmlns="http://www.webserviceX.NET/">
         <GetSunSetRiseTimeResult>
            <Latitude>34</Latitude>
            <Longitude>13</Longitude>
            <SunSetTime>8.3878288</SunSetTime>
            <SunRiseTime>7.05658245</SunRiseTime>
         </GetSunSetRiseTimeResult>
      </GetSunSetRiseTimeResponse>
   </soap:Body>
</soap:Envelope>

Listing 1: Sample XML payload containing a SOAP message.

If you want to read the fields “SunSetTime” and “SunRiseTime” from that payload, you can use the following Groovy code:

def soapPayload = context.getServiceResponse().getBody().asString()
def soapObject = new XmlSlurper().parseText(soapPayload)

def sunSetTime = soapObject['Body']['GetSunSetRiseTimeResponse']['GetSunSetRiseTimeResult']['SunSetTime']
def sunRiseTime = soapObject['Body']['GetSunSetRiseTimeResponse']['GetSunSetRiseTimeResult']['SunRiseTime']

As you can see in the code, you can navigate through the payload without any concern about schemas and namespaces. The only thing that you should keep in mind is that you need to transverse the payload starting from the root, since the internal structure is a tree. Alternatively, you can also read data using XPath-like expressions (called GPathResult’s) as shown here.

To produce a JSON payload with the two fields read before, you can use the JsonBuilder class:

def builder = new JsonBuilder()
def result = builder sunSetTime: sunSetTime.toString(), sunRiseTime: sunRiseTime.toString()
println JsonOutput.prettyString(builder.toString())

That will produce the following JSON payload:

{
    "sunSetTime": "8.3878288",
    "sunRiseTime": "7.05658245"
}

The JsonBuilder class provides a notation-free style of building JSON payloads, which in conjunction with Groovy makes the developer’s life way easier and productive. Moreover, its runtime overhead is pretty minimal, since its implementation caches the values and reuse them efficiently.

Parsing JSON payloads is also very simple using the JsonSlurper class. Let’s see how simple that is by using the sample JSON payload shown in Listing 2.

{
  "name": "DeadPool",
  "alterEgo": {
    "firstName": "Wade",
    "lastName": "Wilson"
  },
  "aliases": ["Merc with a Mouth", "Wildcard", "Weapon XI"],
  "speacies": "Human Mutate"
}

Listing 2: Sample JSON payload.

As you can see; it is a fairly complicated JSON payload, and the only fields we want to read are the “lastName” and the first entry of the “aliases” field. To retrieve these fields, you can use the following Groovy code:

def jsonPayload = context.getApiRequest().getBody().asString()
def jsonObject = new JsonSlurper().parseText(jsonPayload)

println "lastName = " + jsonObject.alterEgo.lastName
println "firstAlias = " + jsonObject.aliases[0]

Once the JsonSlurper class parses the JSON payload, it creates an intelligent object capable of reading its fields using a graph notation. That means that you can navigate through the object to read its contents as if it had been fully defined as a formal class within the code.

Summary

This blog have shown the internal details about the API Platform Groovy policy, and which situations to avoid to ensure proper deployment of APIs. The blog also had shown utility classes that can be used to parse JSON and XML payloads.

Add Your Comment