Shark Attack on InfiniBand

*Reposted with permission from Oracle’s Networking Blog and Neeraj Gupta

So this is not for the faint heart. I am going to give you a roller coaster ride going at 40 Gb/s. ;) Used some shark teeth and cut the InfiniBand packets apart to show you some screenshots here.

Just kidding, dont worry. This post is about how to capture InfiniBand packets and then view them in latest Wireshark. Mellanox has generously released a nice and powerful utility called ibdump. You can visit their website and get a copy till supplies last. So hurry up !

I suggest saving the dums on tmpfs and transfer elsewhere later. Use the –mem-mode for precise and fast captures.

Screenshots Gallery

MAD packet with NodeDescription

shark-attack-on-ib-1IPv4 ping (ICMP) on IPoIB interface. You should notice that we are using connected RC mode for underlying InfiniBand. Also notice the timestamps of ICMP and IB packets.

shark-attack-on-ib-2I am still working on these for various analysis. So, please return here for more screenshots later. By the way, the wireshark interface with InfiniBand captures is fairly new so you may see a few erroneous reports especially when you turn on payload parsing. But its still a lot of fun. :)

*About the author: Neeraj Gupta joined Oracle as part of Sun Microsystems aqusition, where he spent last 11 years specializing in InfiniBand, Ethernet, Security, HA and Telecom Computing Platforms. Prior to joining Sun, Neeraj spent 5 years in Telecom industry focusing on Internet Services and GSM Cellular Networks. Currently Neeraj is part of Oracle’s Engineered Systems team focusing on Networking and Maximum Availability Architecture.

Add Your Comment