Despite the full feature sets and capabilities that Oracle builds into their software-as-a-service (SaaS) cloud applications, there are still going to be occasional customers with business requirements that cannot be satisfied solely with a single SaaS subscription. In these cases, it is possible to extend SaaS products by building integrations with other SaaS products, platform-as-a-service (PaaS) components, and/or on-premise applications. These extensions can take several forms depending on business needs; UI mashups, batch data imports and exports, ATOM feeds, and incoming and outgoing web service integrations are all potential strategies. Often a combination of these integration approaches is the optimal solution to fulfill customers’ business requirements that fall outside the realm of standard SaaS product features.
This blog post explores one small but essential piece of many integrations: getting transport level security (TLS) set up correctly for web services integration components. The vast majority of SaaS extensions which utilize web services, both REST and SOAP, communicate over secure HTTP(s) server-to-server channels. Depending on direction (incoming or outgoing), a SaaS application can act as either the client or the server in these dialogs, while the extension target acts in the complementary role. At a minimum, setting up these secure web service connections successfully requires agreement between the client and the server on a common TLS protocol version, and related cipher suite, which both parties can support. What is known as the SSL handshake process initiates the session between client and server; the result, if all goes well, is an agreed-upon version of TLS/SSL and related cipher suite to use for the duration of the secure communication session. Best practice dictates that the most recent TLS protocol version, along with strongest cipher suite, be the target level of implementation.
A number of inter-related components and settings have an effect on how this handshake process shakes out. Systematically sorting out these components and settings in a checklist-type format should help integration developers identify and resolve potential issues as they set up web service connections. Moreover, if persistent attempts with connection setup continue to fail, the checklist should help pinpoint the cause of the failure. Obviously, the focal points of working through the checklist should be the integration components over which customers have control; the assumption is that Oracle has already set up the SaaS and PaaS modules optimally, providing support for the latest iteration of TLS security protocols. Again, it is probable due to ever-evolving security hardening policies that more vulnerable TLS protocol versions and support for weaker cipher suites is blocked for Oracle SaaS endpoints. Any necessary adjustments that surface as a result of working through the checklist should be directed to pieces of the integration infrastructure managed by the customer.
Fortunately there is a manageable set of pieces to consider when working through the TLS puzzle:
Working through this checklist, item by item, should result in a web services connection definition that is both stable and secure. If there is an issue with any of these components, developers should be able to identify them quickly and resolve problems.
SSL/TLS connections always begin with a negotiation between the two communications participants that sets parameters on how the rest of the session will be handled. This negotiation is known as the "handshake", and, in its simplest "happy path"-only sequence, it consists of several steps (dictated by IETF RFC standards), listed here, somewhat personified to provide some clarity (client is the initiator of the session, and server is the communications target):
Understanding this process, especially in the beginning stages where TLS version and cipher suite negotiations take place, will assist in isolating and correcting issues in setting up secure connections, if they arise.
The first step in building a web services integration component is getting the lay of the land from a network perspective. Taking an inventory of network infrastructure components that a web service integration will touch – firewalls, load balancers, reverse proxies, and VPN’s, along with the roles of these components – is an aid in targeting the components that need the most attention. One example should suffice here to underscore why this step is worth a time investment. Assume that an Oracle cloud customer is attempting to set up an outbound web services connection from Oracle Sales Cloud to their on-premise legacy CRM system. Because there is no VPN tunnel, the Internet will be used between Oracle Public Cloud (OPC) and the customer’s endpoint, making a secure HTTP(s) connection an absolute necessity. In supporting this type of connection, it can make a big difference if SSL is terminated at the customer’s CRM system or if SSL is terminated at a load balancer or reverse proxy server that stands in front of the targeted legacy CRM system. Knowing the termination endpoints will reveal where SSL certificates are installed in the customer’s network infrastructure, and where debugging would need to be focused if problems with connection setup arise. Without this knowledge, it is all too easy to go off track and end up at a dead end.
In parallel with getting to know the network infrastructure, an essential point of reference is knowing what versions of the TLS protocol are currently being supported by Oracle’s installed SaaS and PaaS certificates. What is important to understand is that these certificates contain embedded information that control how the client and server partners mutually decide on which version of the TLS protocol should be used for secure communications.
Knowing, for example, that oraclecloud.com certificates (at the time of this post) support TLSv1.2 and later, without support for TLSv1.1 and TLSv1.0, makes it far simpler to determine what version(s) of the protocol other certificates in the integration mix will be required to support. Fortunately, there is a very easy way to determine version levels. Qualys, a leading international provider of security solutions for the enterprise, maintains a public service website, https://www.ssllabs.com/ssltest, where it is possible to submit a hostname and subject it to a thorough analysis of what TLS protocols are supported, along with reporting on a wealth of related information pertaining to the SSL certificate for the host. What is most important to glean from the report, in addition to the supported protocol version(s), is the list of supported cipher suites for each protocol. This data will be needed for the cipher suites checklist item, discussed below.
Customer-managed certificates for endpoints which are going to become part of the integration should also be sent through the ssllabs.com reporting procedure. As an initial sanity check, comparing supported TLS version protocols and supported cipher suites for all of the hosts/certificates that are pieces of the integration will reveal whether there is enough common ground for a successful handshake to occur. If there is common support, is this sufficient to guarantee success? Regrettably, no. There are other settings and components which need to be examined (see below). Having certificates in place that support a certain TLS version on either side of a HTTPS session is necessary, but not sufficient, for a successful handshake.
While on the subject of certificates, it is important to point out that it will be necessary to import the third-party certificate chain into the SaaS application’s trust keystore, or vice versa, depending on direction (outbound or inbound) of the web service. Not completing this task, or doing it incorrectly, will normally result in a Java CertificateException or SSLHandshakeException at some point in the SSL handshake process.
In an earlier blog post, Transport Level Security (TLS) and Java, I discussed the relationship between Java version and supported default TLS protocol version. The blog post also provided details on how to change default protocol version support by using JVM startup parameters. Again, making sure the server’s Java version supports a specific TLS version, either as a default or by setting JVM startup parameters, is necessary but not sufficient to guarantee a successful handshake.
The ssllabs.com certificate audit report includes information on what cipher suites are supported for each version of the TLS protocol. These cipher mechanisms are listed from most to least preferred from a server perspective, which means, in other words, that during the handshake process, the session server will propose using the most preferred cipher suite, and will proceed down the list until the session initiator responds, accepting the proposed cipher suite. If there are no supported cipher suites in common between the session participants, then there will be no successful handshake. Again, this is a necessary, but not sufficient, piece to enable successful handshakes.
Depending on which cipher suites are being supported by a SaaS client or server, it may be necessary to augment a customer-managed server's Java version with an advanced set of cryptography libraries. These Java Cryptography Extension (JCE) libraries are available for newer versions of Java and, in the case of newer versions of Java, are downloadable from Oracle technetwork URL's. For example, the JCE extensions for Java 8 are here. Getting the JCE extension libraries for older versions of Java may require a My Oracle Support login.
Note that the role of the cipher suite in the handshake process is going to change in the proposed RFC draft specifications for TLS 1.3.
A less-commonly used method for controlling supported TLS versions is to edit the java.security file in the JVM’s ../jre/lib/security folder. It is possible to have any connections controlled by the JVM to reject TLS protocols of designated versions. For example:
Jdk.tls.disabled Algorithms= SSLv2Hello, SSLv3, TLSv1, TLSv1.1
This effectively prevents handshakes with any versions lower than TLSv1.2. Note that this works with Java 7 update 75 and later.
Similarly, the jdk.certpath.disabledAlgorithms key in the same file can be used to prevent a server from using cipher suites that may be considered vulnerable in light of all of the recent SSL exploits that have surfaced. A common example:
jdk.certpath.disabledAlgorithms=MD2, DSA, RSA keySize < 2048
Here, MD2, DSA, and RSA algorithms using a key size less than 2048 bytes would not be supported, no matter what is included in the certificate for the endpoint.
For more information, refer to the Oracle documentation here: http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html#DisabledAlgorithms
If an HTTP(s) endpoint is being served from WebLogic application server, there are several other options for controlling available and blocked TLS protocol versions. The "weblogic.security.SSL.protocolVersion" and/or "weblogic.security.SSL.minimumProtocolVersion" startup parameters are available in recent versions of the WebLogic server. Originally implemented to address the SSL V3.0 "Poodle" vulnerability, these parameters have evolved along with default support for specific TLS protocol versions with updated versions of WebLogic server. For a complete discussion of these settings with respect to versions of WebLogic from 10.3.1 through 12.2.x, see this Oracle "My Oracle Support" note: How to Change SSL/TLS Protocols in Oracle Weblogic Server - Disable SSL 2.0/3.0 and Enable TLS 1.x Options (Doc ID 2162789.1).
These settings behave like the Java version defaults or restrictions set in the java.security file: they may block TLS and older SSL protocol versions from being supported in a secure session that may have been acceptable to support if a server certificate was encoded for supporting same.
Assume for the moment that despite all best efforts, two endpoints, one of them an Oracle SaaS application endpoint, cannot seem to establish a secure TLS session while setting up a web service connection. The most productive path forward is to utilize this checklist to isolate the element or elements that may be affecting how the SSL handshake process is working (or failing in this context). Starting at the top, examine certificate support for TLS and cipher suites first, then move to state of the Java JVM's, and finally, if applicable, check WebLogic server startup parameters. The elements in the checklist are ordered purposefully so that the greatest potential for problems is first, with lessening probabilities with descending checklist items.
Secure Sockets Layer (SSL) and Transport Level Security (TLS) are moving targets. Unfortunately it isn’t a case of setting up a web service integration once and then forgetting about it. With evolution of these protocols being driven by new security exploits and discoveries of holes or gaps in the protocols’ security designs and algorithms, it is necessary for software to evolve along with the security protocols. One recent example was the upgrade of Fusion Applications from Release 11 to Release 12, where coverage of TLSv1.0 was dropped due to the potential for allowing exploits. Integration targets that only had support for TLSv1.0 and not anything newer could handshake successfully with FA Rel 11, but not with Rel 12.
Oracle will announce well ahead of time when there are plans to change a SaaS product’s support of security protocols. Preparing for a change in TLS protocol support may include upgrading certificates, upgrading Java version, and adding cipher suite libraries. Implementing some or all of these updates may take quite a bit of planning and time to deploy at the enterprise level, especially because all of these components are security-related, and the by-product of not setting something 100% correctly can be disastrous. There should be plenty of time budgeted for adequate testing of whatever upgrades need to be implemented. Therefore, the sooner the better for these activities to begin well ahead of a SaaS upgrade.