Load Sharing using BGP over IPSec or FastConnect using multiple Providers or FC Partners

November 3, 2020 | 4 minute read
Andrei Stoian
Master Principal Cloud Architect | North America Cloud Engineering
Text Size 100%:

During past weeks I have received many questions regarding if load sharing can be performed when using BGP over IPSec when more than one Service Provider is used or when we have in place two FastConnect Virtual Circuits through two different FC Partners (https://www.oracle.com/cloud/networking/fastconnect-providers.html#northamerica).


In this blog post, we will analyze the possibility to have the load sharing in place in the absence of BGP ECMP (https://tools.ietf.org/id/draft-lapukhov-bgp-ecmp-considerations-01.html). As you know, Oracle is in the process of introducing the BGP ECMP feature. Until then, let's explore how we can achieve the load sharing only by manipulating the BGP attributes.

The BGP routing protocol is choosing the best available route to a destination by enforcing the BGP Best Path Algorithm (https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria-10) when receiving the same IP prefix from two different BGP neighbors, it does not have a default load balancing mechanism like other Dynamic Routing Protocols. Only the best route (after computing the BGP Best Path Algorithm) will be inserted in the routing table and used for packet forwarding to the destination.

In our example we will use a vendor-agnostic CPE with BGP over IPSec, in my lab this is a Linux VM running Libreswan for the IPSec module and Quagga for the dynamic routing module running BGP. The configuration used to achieve the load balancing can be applied to any CPE supporting the BGP routing protocol. The same approach is available if we are using FastConnect over multiple Partners because the BGP configuration to achieve this sort of load balancing is performed only on the CPE.

Network Topology

 

We will focus on the traffic leaving or entering the On-Premise Network to/from OCI.

Outgoing traffic: On-Premise Network to OCI

In the above diagram we have two paths to OCI from the On-Premise network:

1. The blue path will be considered a primary path for outgoing traffic (On-Premise -> OCI) for 10.0.0.0/29 and a secondary path for 10.0.0.8/29;

2. The red path will be considered a primary path for outgoing traffic (On-Premise -> OCI) for 10.0.0.8/29 and a secondary path for 10.0.0.0/29;

Incoming traffic: OCI to On-Premise Network

When OCI is sending the traffic to On-Premise Network, the traffic will be sent to destinations: 172.16.0.0/24, 172.16.1.0/24, 172.16.2.0/24, 172.16.3.0/24.

1. The blue path will be considered a primary path for incoming traffic to On-Premise Network for 172.16.0.0/24 and 172.16.1.0/24 plus a secondary path for 172.16.2.0/24 and 172.16.3.0/24;

2. The red path will be considered a primary path for incoming traffic to On-Premise Network for 172.16.2.0/24 and 172.16.3.0/24 plus a secondary path for 172.16.0.0/24 and 172.16.1.0/24;

On-Premise CPE BGP configuration

First, we will confirm both BGP OCI neighbors entered the Established state as per BGP Finite State Machine (https://www.ciscopress.com/articles/article.asp?p=2756480&seqNum=4):

libreswan-quagga# sh ip bgp neighbors 192.168.1.2
BGP neighbor is 192.168.1.2, remote AS 31898, local AS 64555, external link
  BGP version 4, remote router ID 192.168.1.2
  BGP state = Established, up for 01:09:30

libreswan-quagga# sh ip bgp neighbors 192.168.1.6
BGP neighbor is 192.168.1.6, remote AS 31898, local AS 64555, external link
  BGP version 4, remote router ID 192.168.1.6
  BGP state = Established, up for 01:10:09

Outgoing traffic: On-Premise Network to OCI

1. Define the IP Prefix-List matching the OCI announced prefixes 10.0.0.0/29 and 10.0.0.0/8:

ip prefix-list outgoing_tunnel1 seq 5 permit 10.0.0.0/29
ip prefix-list outgoing_tunnel2 seq 5 permit 10.0.0.8/29

2. Define the route-map to change the BGP Local_Preference (https://ipwithease.com/bgp-local-preference-attribute/) attribute to locally prefer one path vs another:

!
route-map outgoing_tunnel1 permit 1
 match ip address prefix-list outgoing_tunnel1
 set local-preference 1000
!
route-map outgoing_tunnel1 permit 2
 match ip address prefix-list outgoing_tunnel2
 set local-preference 400
!

route-map outgoing_tunnel2 permit 1
 match ip address prefix-list outgoing_tunnel2
 set local-preference 500
!
route-map outgoing_tunnel2 permit 2
 match ip address prefix-list outgoing_tunnel1
 set local-preference 900

3. Appy the route-maps for each BGP neighbor using the IN direction and restart the BGP sessions for the configuration to take effect:

 neighbor 192.168.1.2 route-map outgoing_tunnel1 in

 neighbor 192.168.1.6 route-map outgoing_tunnel2 in

 libreswan-quagga# clear ip bgp 192.168.1.2

 libreswan-quagga# clear ip bgp 192.168.1.6

4. Verify if we have the desired load balancing per destination using the correct path:

           

We can see that the best and preferred route to 10.0.0.0/29 is through 192.168.1.2 and the best and preferred route to 10.0.0.8/29 is through 192.168.1.6. As we can see the backup path is available as well.

Incoming traffic: OCI to On-Premise Network

1. Define the IP Prefix-List matching the On-Premise announced prefixes to OCI via BGP: 172.16.0.0/24, 172.16.1.0/24, 172.16.2.0/24, 172.16.3.0/24:

ip prefix-list incoming_tunnel1 seq 5 permit 172.16.0.0/24
ip prefix-list incoming_tunnel1 seq 10 permit 172.16.1.0/24
ip prefix-list incoming_tunnel2 seq 5 permit 172.16.2.0/24
ip prefix-list incoming_tunnel2 seq 10 permit 172.16.3.0/24

2. Define the route-maps to change the BGP AS_PATH (https://www.noction.com/blog/as-path-and-as-path-prepending) attribute to inform the BGP peers (OCI) to prefer one path vs another when sending the traffic to On-Premise Network from OCI. The AS_PATH prepend will have effect on the BGP peer side and will influence how the traffic is sent out from the peer.

!
route-map incoming_tunnel1 permit 1
 match ip address prefix-list incoming_tunnel1
!
route-map incoming_tunnel1 permit 2
 match ip address prefix-list incoming_tunnel2
 set as-path prepend 64555 64555
!
route-map incoming_tunnel2 permit 1
 match ip address prefix-list incoming_tunnel2
!
route-map incoming_tunnel2 permit 2
 match ip address prefix-list incoming_tunnel1
 set as-path prepend 64555 64555
!

3. Appy the route-maps for each BGP neighbor using the OUT direction and restart the BGP sessions for the configuration to take effect:

neighbor 192.168.1.2 route-map incoming_tunnel1 out

neighbor 192.168.1.6 route-map incoming_tunnel2 out

 libreswan-quagga# clear ip bgp 192.168.1.2

 libreswan-quagga# clear ip bgp 192.168.1.6

4. Verify if OCI is load balancing the traffic when sending the traffic back to On-Premise Network:

 

This is based on half-half load sharing using BGP. When BGP ECMP will be available the above configuration will not be nedded at all. Until then, give it a try.

Andrei Stoian

Master Principal Cloud Architect | North America Cloud Engineering


Previous Post

NetSuite Integration Series: Part 2: Keep data in Sync between Oracle CX Sales (Engagement Cloud) and NetSuite

Naveen Nahata | 13 min read

Next Post


BICC Extract Logs Monitoring on OCI Native

Ulrich Janke | 23 min read