Simple Node.js backend app using Oracle Cloud Infrastructure(OCI) cloud native services

June 8, 2020 | 9 minute read

Venugopal Naik

Text Size 100%:

－＋

Thanks to Matt Vander Vliet for reviewing and suggesting improvements.

In this post we will create a simple backend node.js application on Oracle Container Engine for Kubernetes(OKE) which connects to OCI Autonomous transaction processing(ATP) instance created via OCI service broker.
We will use oracledb node.js package and oracle instant client for connecting to ATP. We will also create a docker container for DB app built using node.js and store the container image on Oracle Container Registry(OCIR). And finally, expose the deployment (by creating a load balancer service) and test the functionality.

Note:

This procedure does not demonstrate a complete list of cloud native services offered on OCI . Would recommend reading MuShop demo which has more advanced use cases of cloud native services on OCI.

Most steps discussed here are manual. The idea is to understand the steps in detail. The node.js app that we create uses the in-built libraries to listen and serve the http requests. This document tries to explore on how different OCI cloud native services fit together in creating application.

Pre-requisites

Create a OKE cluster

High level overview of the steps

Create OKE cluster
Deploy service catalog and OCI service broker
Provision ATP service instance using service broker
Using a temp pod, Manually bootstrap ATP instance (create a schema and table)
Build the application docker image and push to OCIR . This container will run the node.js service.
Create a kubernetes deployment using this image.
Expose the pod (Create service)
Test

Architecture

Service Broker Concepts

Service catalog and Service broker

Service Catalog is an API that enables applications running in Kubernetes clusters to easily use external managed software offerings, such as a datastore service offered by a cloud provider. Service Catalog uses the Open service broker API to communicate with service brokers, acting as an intermediary for the Kubernetes API Server to negotiate the initial provisioning and retrieve the credentials necessary for the application to use a managed service.
The architecture is well explained here: https://kubernetes.io/docs/concepts/extend-kubernetes/service-catalog/

The OCI Service Broker is an open source implementation of Open service broker API Spec for OCI services.
Would like to highlight two resources that service catalog offers and we would be creating them:

ServiceInstance: When a new ServiceInstance resource is created, the Service Catalog controller connects to the appropriate service broker and instruct it to provision the service instance.

ServiceBinding: Access credentials to a ServiceInstance. These are created by cluster operators who want their applications to make use of a ServiceInstance. Upon creation, the Service Catalog controller creates a Kubernetes Secret containing connection details and credentials for the Service Instance, which can be mounted into Pods.

Architecture summary

Now that we know the concepts, let's re-visit the architecture diagram above which makes it a bit more clear on the overall procedure.

Step-1: Create OKE cluster

Follow the instructions at: https://www.oracle.com/webfolder/technetwork/tutorials/obe/oci/oke-full/index.html

Step-2: Deploy service catalog and OCI service broker

We will not be going through the installation of service catalog and service broker in this document to avoid duplication, the installation instructions are provided here Once you follow the above guide and install, verify the service broker status is ready.

$ kubectl get clusterservicebrokers
NAME                 URL                                      STATUS   AGE
oci-service-broker   http://oci-service-broker.default:8080   Ready    8d

The clusterservicebroker holds the api endpoint information of your service broker. This is a resource within service catalog.

Step-3 Provision ATP service instance using service broker

High level steps:

Create a secret
Create ATP serviceinstance
Create ATP servicebinding

Create a file named atp-secret.yaml with the following details:

#
# Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
#
apiVersion: v1
kind: Secret
metadata:
  name: atp-secret
data:
  # {"password":"s123456789S@"}
  password: eyJwYXNzd29yZCI6InMxMjM0NTY3ODlTQCJ9
  # {"walletPassword":"Welcome_123"}
  walletPassword: eyJ3YWxsZXRQYXNzd29yZCI6IldlbGNvbWVfMTIzIn0K

Create a file named atp-instance.yaml with the following details:

#
# Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
#
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceInstance
metadata:
  name: demodb
spec:
  clusterServiceClassExternalName: atp-service
  clusterServicePlanExternalName: standard
  parameters:
    name: demodb
    compartmentId: "ocid1.compartment.oc1..xxxx"
    dbName: demodb
    cpuCount: 1
    storageSizeTBs: 1
    licenseType: NEW
    autoScaling: false
    freeFormTags:
      testtag: demodb
  parametersFrom:
    - secretKeyRef:
        name: atp-secret
        key: password

Create a file named atp-binding.yaml with the following details:

#
# Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
#
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceBinding
metadata:
  name: atp-demo-binding
spec:
  instanceRef:
    name: demodb
  parametersFrom:
    - secretKeyRef:
        name: atp-secret
        key: walletPassword

Apply the config:

$ kubectl apply -f atp-secret.yaml 
$ kubectl apply -f atp-instance.yaml 
$ kubectl apply -f atp-binding.yaml

Ensure that service instance is in Ready state:

192-168-1-8:db-app vsnaik$ kubectl get serviceinstance
NAME         CLASS                                      PLAN       STATUS   AGE
demodb       ClusterServiceClass/atp-service            standard   Ready    2d5h

atp-demo-binding secret automatically gets created which has the wallet details (tnsnames.ora, cwallet.sso etc.) of ATP database we created. we are now ready to use it in our application.

$ kubectl get secrets
NAME                                             TYPE                                  DATA   AGE
atp-demo-binding                                 Opaque                                9      37h
atp-demo-credentials                             Opaque                                4      42h
atp-secret                                       Opaque                                2      2d14h

To get more details, perform the below:
$ kubectl get secrets atp-demo-binding  -o yaml

Step-4: Bootstrapping DB manually

We can also automate this process, but we are doing it the hard way to understand what goes behind the scenes. Create a test-db pod and mount the atp-demo-binding secret at /tmp/wallet. As we noted previously atp-demo-binding has the wallet details for ATP service running on OCI.

Create a file named test-db.yaml with the following details:
Note: We are also mounting atp-secret at /tmp/wallet2 to fetch the ADMIN password.

apiVersion: v1
kind: Pod
metadata:
    name: test-pod
    labels:
      app: test
spec:
    containers:
    - name: test-pod
      image: oraclelinux:7-slim
      command: ["/bin/sh","-c"]
      args:
      - sleep 10000s
      volumeMounts:
      - name: wallet-raw
        mountPath: /tmp/wallet
        readOnly: false
      - name: atp-secret
        mountPath: /tmp/wallet2
        readOnly: false
    volumes:
    - name: wallet-raw
      secret:
        secretName: atp-demo-binding
    - name: atp-secret
      secret:
        secretName: atp-secret

Apply the config:

$ kubectl apply -f test-db.yaml

Login to test-pod and bootstrap the DB manually:
Note: there is currently a bug in the service broker that double encodes the secrets, so it's currently necessary to decode them. This process can also be automated by using an initContainer which we will demonstrate later.

$ kubectl exec -it test-pod -- bash

# Decode wallet

bash# mkdir -p /wallet
bash# for i in `ls -1 /tmp/wallet | grep -v user_name`; do cat /tmp/wallet/$i  | base64 --decode > /wallet/$i; done; ls -l /wallet/*

# Install oracle-instantclient

bash# yum -y upgrade
bash# yum -y update
bash# yum -y install oracle-release-el7
bash# yum-config-manager --enable ol7_oracle_instantclient
bash# yum -y install oracle-instantclient19.5-sqlplus
bash# yum -y install oracle-instantclient19.5-tools
bash# yum install -y jq

# copy the wallet files to sql client path

bash# mkdir -p /usr/lib/oracle/19.5/client64/lib/network/admin/
bash# cp /wallet/* /usr/lib/oracle/19.5/client64/lib/network/admin/

bash# export PATH=/usr/lib/oracle/19.5/client64/bin/:$PATH

bash# ATP_PW=`cat /tmp/wallet2/password | jq -r ".password"`
bash# ATP_UNAME=`cat /tmp/wallet/user_name`

bash# sqlplus ${ATP_UNAME}/\"${ATP_PW}\"@demodb_tp

SQL*Plus: Release 19.0.0.0.0 - Production on Wed May 13 01:02:00 2020
Version 19.5.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Last Successful login time: Wed May 13 2020 01:01:21 +00:00

Connected to:
Oracle Database 18c Enterprise Edition Release 18.0.0.0.0 - Production
Version 18.4.0.0.0

SQL>

Once you have successfully connected to DB, lets go ahead and create a user schema and tables required:

CREATE USER test_user IDENTIFIED BY default_Password1;

GRANT CREATE SESSION TO test_user;
GRANT UNLIMITED TABLESPACE TO test_user;
CREATE ROLE test_role;
GRANT test_role TO test_user;

CREATE TABLE test_user.products (
brand VARCHAR2(20),
title VARCHAR2(40),
description VARCHAR2(500)
);

GRANT SELECT, INSERT, UPDATE, DELETE ON test_user.products TO test_role;

INSERT INTO test_user.products VALUES ('Nike','Tech Fleece Joggers','Jogger pants for men');
INSERT INTO test_user.products VALUES ('Apple','iwatch','Apple series 5 iwatch');
INSERT INTO test_user.products VALUES ('Sony','Bravia','Sony Bravia Television');

SQL> select brand,title from test_user.products;

BRAND		     TITLE
-------------------- ----------------------------------------
Nike		     Tech Fleece Joggers
Apple		     iwatch
Sony		     Bravia

SQL> exit

To exit out of the pod:

bash# exit

Cleanup: Delete the bootstrap pod.

$ kubectl delete pod test-pod

Step-5: Building a Node.js app

Once the DB is initialized and ready, time to create our db-app docker image. lets automate this process to get a different perspective.
This is a very basic node.js file, which uses oracledb package to connect to database. There are more advanced examples, refer the link in references section.

Create a project folder named db-app and create the following files:
Create a file named dbconfig.js and add the following details:

module.exports = {
    user : process.env.OADB_USER ,
    password : process.env.OADB_PW,
    connectString : process.env.OADB_SERVICE,
    externalAuth : process.env.NODE_OADB_EXTERNALAUTH ? true : false
    };

Create a file named db.js and add the following details:

const http = require('http')
const oracledb = require('oracledb');
const dbconfig = require('./dbconfig.js');

let connection;

const server = http.createServer(async (req, res) => {
   //console.log(req.url)
   res.setHeader('Content-Type','text/html');
   
   if (req.url === '/home'){
    res.write("In home page, Welcome...")
    res.end()
   } 
   
   if (req.url === '/products') {
    console.log("In products page...")
  
    try {
    
      connection =  await oracledb.getConnection(dbconfig);
      const result =  await connection.execute(
      `SELECT brand, title, description
       FROM products`
    );

    //console.log(result.rows);
    res.write('The results fetched from DB are :' + '

');
    for(let results in result.rows){
        const [brand,title,description] = result.rows[results];
        console.log(brand,title,description);
        res.write('Brand: '+ brand + ' Title: '+ title +' Description: '+ description);
        res.write('
')
    }
    
    res.end()
  
  } catch (err) {
    console.error(err);
  } finally {
    if (connection) {
    try {
      await connection.close();
    } catch (err) {
      console.error(err);
      }
     }
    }
}
});

server.listen(process.env.PORT);

function terminate(){
  console.log("\nTerminating");
    if (connection) {
      try {
        connection.close();
      } catch (err) {
        console.error(err);
        }
    }
    process.exit(0);
}
process
  .on('SIGTERM',terminate)
  .on('SIGINT', terminate)

Create a file named package.json and add the following details:

{
    "name": "db-app",
    "version": "1.1.0",
    "description": "A Simple DB application",
    "main": "db.js",
    "dependencies": {
        "oracledb": "^4.2.0"
    }
}

Create a Dockerfile, In which we are trying to create a container with oracle-instantclient19.5, node.js and all its dependencies. Application will be listening on 8080.
Create a file named Dockerfile and add the following details:

FROM oraclelinux:7-slim as db-app

 RUN yum -y upgrade && \
  yum -y update && \
  yum -y install oracle-release-el7 && \
  yum-config-manager --enable ol7_oracle_instantclient && \
  yum -y install oracle-instantclient19.5-sqlplus && \
  yum -y install oracle-instantclient19.5-tools && \
  yum -y install oracle-nodejs-release-el7 oracle-release-el7 && \
  yum install -y nodejs 

RUN mkdir -p /home/node/app
WORKDIR /home/node/app
COPY * ./
RUN npm install

ENV NODE_ENV "production"
ENV PORT 8080
ENV ORACLE_HOME /usr/lib/oracle/19.5/
RUN export PATH=$ORACLE_HOME:$PATH
RUN export PATH=/usr/lib/oracle/19.5/client64/bin/:$PATH
ENV LD_LIBRARY_PATH /usr/lib/oracle/19.5/
EXPOSE 8080

CMD [ "node", "db.js" ]

Build the image as below:

$ docker build -t syd.ocir.io/your_tenancy_namespace/db-app:latest .

To push an image to OCIR, you need to login to the registry(using your user id and auth token), steps are mentioned here

$ docker login syd.ocir.io
$ docker push syd.ocir.io/your_tenancy_namespace/db-app:latest

Note: Here we are using sydney region (syd.ocir.io), it can be any region of your choice.

Moving on, The DB POD we create next would also need access to schema user and password created in the previous step. We also specify the service name to connect to (refering to the tnsnames.ora)

$ kubectl create secret generic atp-demo-credentials --from-literal=oadb_service=demodb_tp --from-literal=oadb_user='test_user' --from-literal=oadb_pw='default_Password1'

To pull an image from OCIR registry, we also need to create docker-registry secret which will be used in our deployment spec:

$ kubectl create secret docker-registry secret-name --docker-server=region-key.ocir.io --docker-username='tenancy-namespace/oci-username' --docker-password='oci-auth-token' --docker-email='email-address

Step-6: Finally, create Application Deployment

Create a file named db-app.yaml and add the following details:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: db-app
spec:
  replicas: 1
  selector:
    matchLabels:
      name: db-app
  template:
    metadata:
      labels:
        name: db-app
    spec:
        initContainers:
          - name: decode-wallet
            image: oraclelinux:7-slim
            command: ["/bin/sh","-c"]
            args: 
            - for i in `ls -1 /tmp/wallet | grep -v user_name`; do cat /tmp/wallet/$i  | base64 --decode > /wallet/$i; done; ls -l /wallet/*;sleep 10s;
            volumeMounts:
            - name: wallet-raw
              mountPath: /tmp/wallet
            - name: wallet
              mountPath: /wallet
        containers:
          - name: db-app
            image: syd.ocir.io/ociateam/db-app:latest
            imagePullPolicy: Always
            env:
            - name: OADB_USER
              valueFrom:
                secretKeyRef:
                  name: atp-demo-credentials
                  key: oadb_user
            - name: OADB_PW
              valueFrom:
                secretKeyRef:
                  name: atp-demo-credentials
                  key: oadb_pw
            - name: OADB_SERVICE
              valueFrom:
                secretKeyRef:
                  name: atp-demo-credentials
                  key: oadb_service        
            volumeMounts:
              - name: wallet
                mountPath: /usr/lib/oracle/19.5/client64/lib/network/admin/
        volumes:
          - name: wallet-raw
            secret:
              secretName: atp-demo-binding
          - name: wallet
            emptyDir: {}
        imagePullSecrets:
          - name: ocirsecret

Apply the configuration:

$ kubectl apply -f db-app.yaml

As you can see we are automating the wallet decoding with the help of "initContainers". This is necessary to overcome the double encoding bug with service broker.

Step-7: Create the service

kubectl expose deployment db-app --port=80 --target-port=8080 --type=LoadBalancer

Note: A new load Balancer with public IP will get created on OCI with --type=LoadBalancer. You can also use the --type as ClusterIP to avoid external access.

Step-8: Testing

Cleanup

$ kubectl delete svc

Note: This deletes your load balancer created on OCI

$ kubectl delete deployment db-app

$ kubectl delete servicebinding atp-demo-binding
Note: The above deletes your atp-demo-binding secrets as well.

$ kubectl delete serviceinstance atp-instance
Note: This deletes the ATP database on OCI

$ kubectl delete secret atp-secret
$ kubectl delete secret atp-demo-credentials
$ kubectl delete secret ocirsecret

Note: This document does not cover clean up of service broker deployed during the start of this procedure, refer https://github.com/oracle/oci-service-broker for additional information.

References/Credits

https://github.com/oracle-quickstart/oci-cloudnative
https://slides.mushop.ateam.cloud
https://github.com/oracle/node-oracledb
https://github.com/oracle/oci-service-broker