During past weeks I have received many questions regarding if load sharing can be performed when using BGP over IPSec when more than one Service Provider is used or when we have in place two FastConnect Virtual Circuits through two different FC Partners (https://www.oracle.com/cloud/networking/fastconnect-providers.html#northamerica).
In this blog post, we will analyze the possibility to have the load sharing in place in the absence of BGP ECMP (https://tools.ietf.org/id/draft-lapukhov-bgp-ecmp-considerations-01.html). As you know, Oracle is in the process of introducing the BGP ECMP feature. Until then, let's explore how we can achieve the load sharing only by manipulating the BGP attributes.
The BGP routing protocol is choosing the best available route to a destination by enforcing the BGP Best Path Algorithm (https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria-10) when receiving the same IP prefix from two different BGP neighbors, it does not have a default load balancing mechanism like other Dynamic Routing Protocols. Only the best route (after computing the BGP Best Path Algorithm) will be inserted in the routing table and used for packet forwarding to the destination.
In our example we will use a vendor-agnostic CPE with BGP over IPSec, in my lab this is a Linux VM running Libreswan for the IPSec module and Quagga for the dynamic routing module running BGP. The configuration used to achieve the load balancing can be applied to any CPE supporting the BGP routing protocol. The same approach is available if we are using FastConnect over multiple Partners because the BGP configuration to achieve this sort of load balancing is performed only on the CPE.
We will focus on the traffic leaving or entering the On-Premise Network to/from OCI.
In the above diagram we have two paths to OCI from the On-Premise network:
1. The blue path will be considered a primary path for outgoing traffic (On-Premise -> OCI) for 10.0.0.0/29 and a secondary path for 10.0.0.8/29;
2. The red path will be considered a primary path for outgoing traffic (On-Premise -> OCI) for 10.0.0.8/29 and a secondary path for 10.0.0.0/29;
When OCI is sending the traffic to On-Premise Network, the traffic will be sent to destinations: 172.16.0.0/24, 172.16.1.0/24, 172.16.2.0/24, 172.16.3.0/24.
1. The blue path will be considered a primary path for incoming traffic to On-Premise Network for 172.16.0.0/24 and 172.16.1.0/24 plus a secondary path for 172.16.2.0/24 and 172.16.3.0/24;
2. The red path will be considered a primary path for incoming traffic to On-Premise Network for 172.16.2.0/24 and 172.16.3.0/24 plus a secondary path for 172.16.0.0/24 and 172.16.1.0/24;
First, we will confirm both BGP OCI neighbors entered the Established state as per BGP Finite State Machine (https://www.ciscopress.com/articles/article.asp?p=2756480&seqNum=4):
libreswan-quagga# sh ip bgp neighbors 192.168.1.2
BGP neighbor is 192.168.1.2, remote AS 31898, local AS 64555, external link
BGP version 4, remote router ID 192.168.1.2
BGP state = Established, up for 01:09:30
libreswan-quagga# sh ip bgp neighbors 192.168.1.6
BGP neighbor is 192.168.1.6, remote AS 31898, local AS 64555, external link
BGP version 4, remote router ID 192.168.1.6
BGP state = Established, up for 01:10:09
1. Define the IP Prefix-List matching the OCI announced prefixes 10.0.0.0/29 and 10.0.0.0/8:
ip prefix-list outgoing_tunnel1 seq 5 permit 10.0.0.0/29
ip prefix-list outgoing_tunnel2 seq 5 permit 10.0.0.8/29
2. Define the route-map to change the BGP Local_Preference (https://ipwithease.com/bgp-local-preference-attribute/) attribute to locally prefer one path vs another:
!
route-map outgoing_tunnel1 permit 1
match ip address prefix-list outgoing_tunnel1
set local-preference 1000
!
route-map outgoing_tunnel1 permit 2
match ip address prefix-list outgoing_tunnel2
set local-preference 400
!
route-map outgoing_tunnel2 permit 1
match ip address prefix-list outgoing_tunnel2
set local-preference 500
!
route-map outgoing_tunnel2 permit 2
match ip address prefix-list outgoing_tunnel1
set local-preference 900
3. Appy the route-maps for each BGP neighbor using the IN direction and restart the BGP sessions for the configuration to take effect:
neighbor 192.168.1.2 route-map outgoing_tunnel1 in
neighbor 192.168.1.6 route-map outgoing_tunnel2 in
libreswan-quagga# clear ip bgp 192.168.1.2
libreswan-quagga# clear ip bgp 192.168.1.6
4. Verify if we have the desired load balancing per destination using the correct path:
We can see that the best and preferred route to 10.0.0.0/29 is through 192.168.1.2 and the best and preferred route to 10.0.0.8/29 is through 192.168.1.6. As we can see the backup path is available as well.
1. Define the IP Prefix-List matching the On-Premise announced prefixes to OCI via BGP: 172.16.0.0/24, 172.16.1.0/24, 172.16.2.0/24, 172.16.3.0/24:
ip prefix-list incoming_tunnel1 seq 5 permit 172.16.0.0/24
ip prefix-list incoming_tunnel1 seq 10 permit 172.16.1.0/24
ip prefix-list incoming_tunnel2 seq 5 permit 172.16.2.0/24
ip prefix-list incoming_tunnel2 seq 10 permit 172.16.3.0/24
2. Define the route-maps to change the BGP AS_PATH (https://www.noction.com/blog/as-path-and-as-path-prepending) attribute to inform the BGP peers (OCI) to prefer one path vs another when sending the traffic to On-Premise Network from OCI. The AS_PATH prepend will have effect on the BGP peer side and will influence how the traffic is sent out from the peer.
!
route-map incoming_tunnel1 permit 1
match ip address prefix-list incoming_tunnel1
!
route-map incoming_tunnel1 permit 2
match ip address prefix-list incoming_tunnel2
set as-path prepend 64555 64555
!
route-map incoming_tunnel2 permit 1
match ip address prefix-list incoming_tunnel2
!
route-map incoming_tunnel2 permit 2
match ip address prefix-list incoming_tunnel1
set as-path prepend 64555 64555
!
3. Appy the route-maps for each BGP neighbor using the OUT direction and restart the BGP sessions for the configuration to take effect:
neighbor 192.168.1.2 route-map incoming_tunnel1 out
neighbor 192.168.1.6 route-map incoming_tunnel2 out
libreswan-quagga# clear ip bgp 192.168.1.2
libreswan-quagga# clear ip bgp 192.168.1.6
4. Verify if OCI is load balancing the traffic when sending the traffic back to On-Premise Network:
This is based on half-half load sharing using BGP. When BGP ECMP will be available the above configuration will not be nedded at all. Until then, give it a try.
Previous Post