A routing scenario, defining separate path for inspected vs. non-inspected traffic

What we are going to analyze in this blog post is a real production network and interesting routing use case I received from one of my customers. It is very interesting mainly because it is highlighting the DRG's powerful capabilities to separate the user traffic very close to what VRFs in an On-premises network does.

More and more complex routing use cases are being resolved by the DRG routing engine and we can truly say that it is a great routing tool that our customers can use to accommodate the most complex routing scenarios.

Less words, more facts, let's go straight into the use case which sounds in the following way:

- the networking topology is using two OCI regions using the RPC for region to region communication in an Active/DR scenario;

- the Active region is using a number of VCNs in different compartments representing different type of workloads;

- the DR region is a copy of the Active region;

- each region has a security VCN where the inspection firewalls are located;

- when the traffic is between the same type of workload from one region to another over the RPC, the traffic should not be inspected by any of the firewalls;

- when the traffic is between different type of workloads from one region to another over the RPC, the traffic should be inspected by the firewalls from both regions, essentially the traffic should be seen by the firewalls from both region;

The Networking Design proposed:

Based on the above described use case:

- Application 1 from Region 1 to Region 2 Application 1  (and vice-versa) should not be inspected by the firewalls;

- Application 1 from Region 1 to Region 2 Application 2 (and vice-versa) should be inspected by the firewalls from both regions;

Case 1 - Application 1 from Region 1 to Region 2 Application 1  (and vice-versa) should not be inspected by the firewalls

The design is using two RPCs between the two regions, one called rpc-bypass-firewall marked with green and another one called rpc-firewall-inspection marked with red. The rpc-bypass-firewall will be used for the traffic that should not be inspected as the name implies. We will discuss about rpc-firewall-inspection later.

The rpc-bypass-firewall DRG RPC attachment route tabels in both regions will dynamically import all the Application DRG VCNs attached (using the Route Distribution Import associated with the RPC attachment Route Table). The IP prefixes received over rpc-bypass-firewall will not be dynamically imported in any DRG attachment route table.

The Route Distribution Import will have the following configuration:

That being said, let's analyze how Region 1 Application 1 DRG VCN attachment route table looks like:

Let's do the same for Region 2 Application 1 DRG VCN attchment route table:

The static routes for the traffic that should not be inspected will directly send it over the defined RPC.

Let's verify the results with a traceroute from both regions Application 1 VMs:

The traceroute between the two Application 1 VMs in different regions does not pass through the firewalls and the tcpdump on the firewalls confirms that no traffic is received. Our scope is accomplished!

This solution offers a great scalability. Once new VCNs are added and attached to the DRG in each and every region that needs to comply with this routing posture, just a static route to the peer destination is needed in the DRG VCN attachment route. Just as in the example above. No changes are neccessary on the RPC.

Case 2 - Application 1 from Region 1 to Region 2 Application 2 (and vice-versa) should be inspected by the firewalls from both regions

Now comes into play our second RPC connection called rpc-firewall-inspection.

rpc-firewall-inspection DRG RPC attachment route table in both regions will contain a default static route with the Security VCN as a next-hop attachment:

Then, the DRG VCN route table will direct the traffic to the Firewalls IP address for inspection (just the normal OCI Transit Routing):

Region 1:

Region 2:

The DRG Security VCNs attachment route tables in both regions will dynamically import the IP prefixes received over rpc-firewall-inspection:

Besides of the default route imported using the rpc-firewall-inspection as next-hop, the DRG Security VCNs attachment route tables will dynamically import the region Application VCNs.

The Application DRG VCNs attachement route tables (in both regions) besides the specific static route configured for accomplishing Case 1, will contain a default route with the Security VCN as a next-hop:

After the above configuration was applied, let's verify if the traffic from Region 1 Application 1 to Region 2 Application 2 is passing through the firewalls from both regions:

10.0.10.2 is the IP address assigned to Region 1 Firewall while the 10.101.0.5 is for Region 2 Firewall. As we can see from the traceroute output and from the packet capture, the traffic between Region 1 Application 1 to  Region 2 Application 2 (and vice-versa) is passing through the firewalls in both regions. The second use case is accomplished.

As you guessed, once you add more and more VCNs that needs to comply with the routing posture defined for Case 2, you only need to attach the VCNs to the DRG and create a default route in the VCN attachment route table with the Security VCN as a next-hop, so simple! No changes needs to be performed on any of the RPCs.

At the end, is there any way to acomplish both use cases with just one RPC connection? Let me know your answer! Enjoy it!