Overview

Updated on Jun 1st 2020.

The purpose of this documentation is to help configuring FortiGate-VM in HA mode on Oracle Cloud Infrastructure (OCI). Basic OCI and FortiGate-VM experience is recommended.

This configuration was validated using FortiGate-VM version 6.2.3.

For more details on how to use FortiGate-VM in HA, please visit the official Fortinet website. Documentation for HA or manual deployment can be found at https://docs2.fortinet.com/vm/oci/FortiGate-VM/6.2/oci-cookbook/6.2.0/138784/ha-for-FortiGate-VM-on-oci.

Caveats and Limitations

FortiGate-VM on OCI Marketplace

We recommend using the latest FortiGate-VM version with support for SR-IOV.

The license model for FortiGate-VM on OCI changed with version 6.0.2. The number of vCPUs indicated by the license does not restrict the FortiGate-VM from working on a compute shapes with a greater number of VCPUs. The rest of the VCPUs are not used.

https://docs2.fortinet.com/vm/oci/FortiGate-VM/6.0/about-FortiGate-VM-for-oci/6.0.0/43335/models

This new license model works only on version 6. An active FortiGate-VM license can work on both version 5 and 6.

A trial or permanent license needs to be used to be able to configure FortiGate-VM using GUI.

For installing FortiGate-VM on OCI read this page https://www.ateam-oracle.com/fortinet-FortiGate-VM-oci-installation.

Note: Minimum 4 VNICs are required for each instance to be able to configure Fortigate in HA. Minimum shape is vm.standard2.4 and up.

Step by step

Step 1. Solution description.

Step 2. Configure Fortinet Fabric Connectors.

Step 3. Configure FortiGate-VM in HA mode

Step 4. Troubleshooting

Step 1. Solution description.

FortiGate-VM for OCI supports active/passive high availability (HA) configuration with FortiGate-VM-native unicast HA synchronization between the primary and secondary nodes. When the FortiGate-VM detects a failure, the passive firewall instance becomes active and uses OCI API calls to configure its interfaces/ports.

Fortinet uses a solution called Fabric Connectors to interact with the OCI API. This will allow FortiGate-VM to read data and modify different objects in OCI. More details can be found at Fortinet official website https://www.fortinet.com/solutions/enterprise-midsize-business/fabric-connectors.html.

This solution uses the Active-Passive model. One or more Floating IP address (secondary IP addresses in OCI) will be moved from one FortiGate-VM to another in case of failover using the Fortinet Fabric Connector and the OCI API.

Keep in mind that each OCI VM vNIC (primary or secondaries) has a primary IP address. This primary IP address cannot be moved between VMs. For each vNIC (primary or secondaries) you can assign one or multiple secondary IP addresses. Only secondary IP addresses can be moved between FortiGate-VMs.

More details about OCI secondary IP addresses can be found at https://docs.cloud.oracle.com/iaas/Content/Network/Tasks/managingIPaddresses.htm.

Step 2. Configure Fortinet Fabric Connectors.

Before starting, keep in mind that all requests towards the OCI API from Fabric Connectors are done using the FortiGate Management interface. The Management interface, by default, is port1 on FortiGate-VM. This port uses by default DHCP and has a primary interface assigned by default by OCI. All other interfaces (except the primary interface) on OCI will not offer DHCP. You need to manually assign IP address for each additional FortiGate-VM port.

Port1 (Management interface) is used to access FortiGate-VM over SSH or GUI. For HA purposes, a second port (doesn't have to be port2) needs to be used to synchronize each the FortiGate-VMs and to send heartbeats (keep-alive packets). It is required to use different subnets in OCI for trust, untrust, HA and management traffic.

To start configuring the Fabric Connectors we need to select "Security Fabric" ==> "Fabric Connectors" and "Create New" button.

This will show a list with different Providers.

Select "Oracle Cloud Infrastructure (OCI)" from "Public SDN" list. The configuration page for a "New Fabric Connector" will be displayed.

"Use metadata IAM" option will change how the Fabric Connector will authenticate to OCI API. 

OCI uses two different methods for API authentication. 

One authentication method is using a native OCI feature called "Dynamic groups" or by using "API Signing Keys".

More details can be found in the official OCI documentation:

• "Dynamic groups" - https://docs.cloud.oracle.com/iaas/Content/Identity/Tasks/managingdynamicgroups.htm
• "API Signing Keys" - https://docs.cloud.oracle.com/iaas/Content/Identity/Concepts/usercredentials.htm

Configuration using "Use metadata IAM" option (Dynamic Groups).

Configuration without using "Use metadata IAM" option (API Signing Keys).

We do recommend using the "Use metadata IAM" (Dynamic Groups) option. This will provide more security and easy configuration.

We will not cover how to create instance principals. For more details, please check https://blogs.oracle.com/cloud-infrastructure/announcing-instance-principals-for-identity-and-access-management.

Here is the recommended policy to be used with Dynamic Groups:

Allow dynamic-group < dynamic-group-name> to use virtual-network-family in compartment <compartment>

Allow dynamic-group < dynamic-group-name > to use instance-family in compartment <compartment>

After configuring the Fabric Connector, the "OCI Connector" will be displayed in the "Fabric Connectors" page. The "OCI Connector" has the following options:

•  ==> Enable or disable the Fabric Connector.
•  ==> Refresh the status of the Fabric Connector.
•  ==> The Fabric Connector has successfully connected to the OCI API. This will change to color red  in case of not being able to connect to the OCI API.

Step 3. Configure FortiGate-VM in HA mode

Before starting, remember that you need an HA port (doesn't have to be port2) to synchronize each FortiGate-VM and to send heartbeats (keep-alive packets).

To add additional vNICs in OCI, select "Compute" ==> "Instance" ==> "Instance Details" ==> "Attached vNICs". Here you need to create a new vNIC that will be part of a specific subnet/VCN. To match a certain vNIC in OCI to FortiGate-VM, please use the MAC address as an identifier. In case of a miss-match, restart the FortiGate-VM instance.

To add more secondary IP addresses on OCI, select the vNIC name from the above "Attached vNICs". Select "IP Addresses" and click on "Assign Private IP address" to create a new secondary IP address. Keep in mind that all vNICs will have by default a primary IP address and it can’t be moved, only the secondary IP address is moved between the VMs. Only assign secondary IP addresses to the master VM, do not add to the slave VM. There is no need to assign secondary IP address to the primary vNIC (port1) and to the HA vNIC.

Now we need to configure each new OCI vNIC attached to the FortiGate-VM instance. Click on the square next to the status column header and click the gear icon, then select MAC Address, this will show the MAC address which you can use to match with the vNIC on the Oracle Console in case of doubt.

Select the new interface and click on "Edit" button. On the "Edit Interface" page we need to add the following details:

• "Alias" ==> A name for the selected interface.
• "Addressing mode" ==> Select "Manual". Here we will add the IP address defined in OCI.
• "IP/Network Mask" ==> IP address assigned by OCI.
• "Secondary IP Addresses" ==> This is optional. In case you need more than one IP address, enable this option and add all the other OCI secondary IP addresses for a specific vNIC.

We can use primary OCI IP addresses for Management and HA ports. 

For the other FortiGate-VM interfaces that will need to move IP addresses from one FortiGate-VM node to another, use secondary OCI IP addresses only as the primary address on the FortiGate configuration console. Do not turn on Secondary IP address and configure it as such.

Example FortiGate-VM interface configuration:

Configure the data ports on the FortiGate-VM only on the Master VM. On the slave VM configure only the HA port. There is no need to configure the other ports or policies as once HA is enabled the configuration will sync between the two VMs.

Save settings by clicking on OK button.

Now we need to configure FortiGate-VM in HA mode. To do this, we need to go to "System" ==> "HA". Here we need to add the following details:

• "Mode" ==> Select "Active-Passive" mode.
• "Device priority" ==> By default value is 100. A bigger value will be used for the Master node.
• "Group name" ==> A name to describe the cluster group.
• "Password" ==> Used to protect the link between the two FortiGate-VMs participating in HA.
• "Session pickup" ==> Synchronizes the primary unit's TCP session table to all cluster units.
• "Heartbeat interfaces" ==> Interface dedicated for heartbeat. Select the HA port
• "Interface" ==> Select the Management interface used to connect to the OCI API.
• "Gateway" ==> Add the gateway used for the Management interface.
• "Peer IP" ==> IP address from the second (slave) FortiGate-VM.

Remember to permit traffic for HA between FortiGate-VMs. By default, when creating a VCN in OCI, a default security list is created. This security list will not permit traffic between the same subnet.

Also keep in mind that when FortiGate-VM sends API requests towards OCI, it will use the FortiGate-VM Management interface and traffic will be on port 443. A proper security list for egress traffic needs to exist. By default, when creating a VCN in OCI, a default egress security is created, permitting all traffic outside (stateful firewall rule).

This FortiGate-VM Management interface will need to be able to reach the OCI API. This can be done by using an Internet Gateway or NAT Gateway or Service Gateway. Here is a list with all OCI API Endpoints https://docs.cloud.oracle.com/en-us/iaas/api/#/en/iaas/20160918/. 

The same setup needs to be done on the second (slave) FortiGate-VM participating in HA. Remember to use a lower priority, and for the Peer IP use the IP of the Master FortiGate-VM.

After finishing the configuration, the 2 FortiGate-VMs should be in sync and the one with the highest priority will be the Master. A successful sync state is represented by   symbol.

Step 4. Troubleshooting

========================================

You can display diagnose commands by running this command (without double quotes):

"diagnose test application ocid"

1. show HA stats

2. SDN api test

3. HA api test

4. filter list test

99. restart

You can verify that the following diagnose command works for the ocid daemon:

On Active FortiGate-VM:

"diag test application ocid 1"

ocid stats:

master: 1

On Standby FortiGate-MV:

"diag test application ocid 1"

ocid stats:

master: 0

========================================

To start debug mode, run the following command (without double quotes)::

"diagnose debug application ocid -1"

The command above will show all the VNICs and the IP addresses (primary and secondaries) on the VM if the fabric connector along with the Oracle Console settings are correctly configured and the management interface can reach the OCI API. This will show on the master FortiGate-VM. The slave will not show this info.

========================================

To stop debug mode, run the following command:

"diagnose debug application ocid 0"

========================================

For more troubleshooting commands, please visit the official Fortinet website http://docs.fortinet.com/vm/oci/fortigate/6.2/oci-cookbook/6.2.0/502638/troubleshooting.