Best Practices from Oracle Development's A‑Team

OCI DRGv2 and Route Conflict

Andrei Stoian
Principal Solutions Architect | A-Team - Cloud Solution Architects

In this new blog post, we will discuss one of the very hot topics related to OCI DRGv2, the route conflict. The route conflict simply means the route or routes that are used by a particular DRG route table to forward the traffic when the same route is learned from multiple sources. We will not discuss ECMP since this subject has been fully covered in this blog post: https://www.ateam-oracle.com/equal-cost-multi-path-ecmp-routing-on-oci-drgv2.

We will analyze different use cases based on the route preference implemented, together with clear examples. The route conflict can be consulted in detail at this link: https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingDRGs.htm section "Route Conflicts".

As we know, we can have two types of routes in a DRGv2 route table, static and dynamic routes. In a nutshell, the Route Conflict is resolved using the following algorithm:

1. Static routes always have a higher preference than dynamic routes, the static routes will always have an empty BGP AS_PATH;

2. Conflicts between dynamic routes are resolved by first analyzing the route's BGP AS_PATH, the shortest AS_PATH always wins:

2.1 Routes with a route source of VCN always have an empty BGP AS_PATH;

2.2 Routes with a route source of IPSEC_TUNNEL or VIRTUAL_CIRCUIT will have BGP AS_PATH information;

3. Otherwise, the attachment type that imported the route is evaluated according to the following priority based on the attachment type: VCN > VC > IPSec > RPC;

4.  If conflicting routes are imported from attachments of the same type, the conflict is resolved differently depending on the attachment type:

4.1 VCN Attachments: If identical CIDRs are imported from two VCN attachments, only one is selected;

4.2 VIRTUAL_CIRCUIT Attachments: 

- If multiple routes with the same CIDR and different BGP AS_PATH lengths are imported into a DRG route table, the route with the lowest AS_PATH length is selected;

- If the BGP AS_PATH is the same and  ECMP is enabled, all routes will be added to the route table for a maximum supported ECMP route of eight;

          - If the BGP AS_PATH is the same and ECMP is disabled, one route is chosen;

4.3 IPSEC_TUNNEL Attachments: similar to VIRTUAL_CIRCUIT Attachments;

4.4 RPC attachments: If identical CIDRs are imported from two RPC attachments, one route is chosen;

Since some of the above points are quite easy to grasp, we will focus on some use cases to prove what is listed above is right.

Networking topology

We will focus on the subnet received from multiple sources. We will analyze how the algorithm listed above picks the and use it for traffic forwarding when it is received from multiple sources and inserted in a DRG route table. We will use the DRG route table attached to the VCN 1 from the above networking topology.

Use Case 1: Prefer the static inserted route over dynamic routes for the same subnet received

The DRG route table attached to VCN 1 uses a Route Distribution with the following Statements:

The DRG route table for VCN 1 imports from BGP over IPSec Tunnel 1 attachment, from RPC attachment, and from VCN 2 attachment.

On another hand, the route table for VCN 1 attachment includes a static route to with the next-hop attachment name being the VCN 3:

Let's have a look inside the DRG route table for VCN 1 attachment after all the above are configured:

As we expect, we have four entries for The static route is marked as Active, the rest of the three dynamic routes received on different attachments are marked as Conflict routes. This confirms that the static route has a higher preference over dynamic routes.

Use Case 2: Prefer the VCN route from the dynamic routes for the same subnet

Let's remove the static route from the DRG route table and check if the route using the VCN as a next-hop attachment becomes the Active one:

The static route disappeared from the route table and according to the algorithm listed above, the VCN route becomes the Active one.

Use Case 3: Prefer the BGP over IPSec route over the RPC received route for

To prove this use case we will use the same BGP AS_PATH for announced to OCI from CPE 1 (Ashburn) and CPE 2 (Phoenix). That being said, the DRG from the Ashburn region will receive the subnet with only one BGP AS in the AS_PATH from the customer side (as per the networking topology listed above). Remember that using BGP over IPSec at the receiving routes from the CPE side, OCI is prepending one more AS to the AS_PATH received from the CPE for a total of 2 AS in the BGP AS_PATH in our example. This action is documented at this link: https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingDRGs.htm.

The DRG from the Ashburn region will receive the same over the RPC. In the Phoenix region, we are receiving the subnet with the same BGP AS_PATH. Over RPC no AS prepending happens, so the DRGv2 from Ashburn will receive the subnet with the same BGP AS_PATH.

Let's check the route table for VCN 1 attachment after removing the VCN route:

We can clearly see that the IPSec Tunnel next-hop attachment route is marked as Active due to equal BGP AS_PATH length and IPSec route being more prefered than the RPC route.

Use Case 4: Prefer the RPC route over BGP over IPSec received route for

This use case sounds strange, right? In use case 3 we just proved that BGP over IPSec is more prefered than the RPC. Is this possible at all? Yes, it is possible. Remember, the first deciding factor for the dynamic routes is the BGP AS_PATH.

Let's prepend one time the customer BGP ASN on CPE 1 for the prefix received over the BGP on IPSec Tunnel 1 in Ashburn, for a final BGP AS_PATH of three (two from the customer side including the prepended one and one automatically added by OCI):

app-server1# sh ip prefix-list  10
BGP: ip prefix-list 10: 1 entries
   seq 5 permit

app-server1# sh route-map prepend
route-map prepend, permit, sequence 1
  Match clauses:
    ip address prefix-list 10
  Set clauses:
    as-path prepend 64555
  Call clause:
    Exit routemap

Now, let's verify the route table for VCN 1 attachment:

With a very simple configuration on CPE manipulating the BGP AS_PATH, OCI is sending the traffic to On-Premise for over the RPC and not directly via the associated IPSec Tunnel in Ashburn.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha