X

Best Practices from Oracle Development's A‑Team

Using Automatic Network Resource in Solaris Zone on Exalogic

Leo Yuen
Cloud Solutions Architect

This article will show you how to use Automatic Network (anet) resource to configure IPoIB and EoIB bonded interfaces in a Solaris zone on an Exalogic machine running Solaris 11.1.

Let's start with a basic configuration of a zone with no network interface configured:

zonecfg:zoss01> info zonename: zoss01 zonepath: /zones/zoss01 brand: solaris autoboot: false bootargs: file-mac-profile: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: fs-allowed: rootzpool:         storage: iscsi://192.168.10.15/luname.naa.600144F0B6FB373A0000531D230A0001

The name of the zone is "zoss01",  it is a Solaris 11 native zone with brand set to "solaris" and it is running over iSCSI. This zone is configured with exclusive-ip that is the default value in Solaris 11.1, please refer to the documentation on the differences between exclusive-ip and shared-ip mode. Anet resource is available in  exclusive-ip mode only.

The following steps illustrate how to add IPoIB network interfaces to the zone using anet resource:

root@el01cn01:~# zonecfg -z zoss01 zonecfg:zoss01> add anet zonecfg:zoss01:anet> set linkname=bond0_0 zonecfg:zoss01:anet> set lower-link=ibp0 zonecfg:zoss01:anet> set pkey=0xffff zonecfg:zoss01:anet> end zonecfg:zoss01> add anet zonecfg:zoss01:anet> set linkname=bond0_1 zonecfg:zoss01:anet> set lower-link=ibp1 zonecfg:zoss01:anet> set pkey=0xffff zonecfg:zoss01:anet> end zonecfg:zoss01> exit

where linkname is the name of the interface that will be appeared in the zone, you may give it a different name; pkey is the partition key where 0xffff is the default partition key;  lower-link should be set to the corresponding physical link as indicated by the output of dladm command:

root@el01cn01:~# dladm LINK                CLASS     MTU    STATE    OVER igb0                phys      1500   up       -- igb2                phys      1500   unknown  -- igb1                phys      1500   unknown  -- igb3                phys      1500   unknown  -- usbecm0             phys      1500   up       -- ibp1                phys      65520  up       -- eoib0               phys      1500   up       -- eoib1               phys      1500   up       -- ibp0                phys      65520  up       -- bond0_0             part      65520  up       ibp0 bond0_1             part      65520  up       ibp1 vnic3066_0          vnic      1500   up       eoib0 vnic3066_1          vnic      1500   up       eoib1

IMPORTANT: please verify the zone configuration to make sure that the property "link-protection" is *not* specified, if it was set to any value (even if it was set to the default value "mac-nospoof"),  use clear command in zonecfg to unset this property.

The following steps illustrate how to add EoIB network interfaces to the zone using anet resource:

root@el01cn01:~# zonecfg -z zoss01 zonecfg:zoss01> add anet zonecfg:zoss01:anet> set linkname=bond1_0 zonecfg:zoss01:anet> set lower-link=eoib0 zonecfg:zoss01:anet> set vlan-id=3066 zonecfg:zoss01:anet> end zonecfg:zoss01> add anet zonecfg:zoss01:anet> set linkname=bond1_1 zonecfg:zoss01:anet> set lower-link=eoib1 zonecfg:zoss01:anet> set vlan-id=3066 zonecfg:zoss01:anet> end zonecfg:zoss01> exit

where linkname is the name of the interface that will be appeared in the zone, you may give it a different name; vlan-id is the vlan ID of the EoIB network, in this example, it is 3066, if there is no VLAN, this property is not required to set;  lower-link should be set to the corresponding physical link as indicated by the output of dladm command:

root@el01cn01:~# dladm LINK                CLASS     MTU    STATE    OVER igb0                phys      1500   up       -- igb2                phys      1500   unknown  -- igb1                phys      1500   unknown  -- igb3                phys      1500   unknown  -- usbecm0             phys      1500   up       -- ibp1                phys      65520  up       -- eoib0               phys      1500   up       -- eoib1               phys      1500   up       -- ibp0                phys      65520  up       -- bond0_0             part      65520  up       ibp0 bond0_1             part      65520  up       ibp1 vnic3066_0          vnic      1500   up       eoib0 vnic3066_1          vnic      1500   up       eoib1

When the zone is up and running, here is what the dladm command will return inside the zone:

root@zoss01:~# dladm LINK                CLASS     MTU    STATE    OVER bond0_0             part      65520  up       ? bond0_1             part      65520  up       ? bond1_0             vnic      1500   up       ? bond1_1             vnic      1500   up       ?

The interfaces can now be used to configure bond0 and bond1.

Here is an example of how to configure bond0:

root@zoss01:~# ipadm create-ip bond0_0 root@zoss01:~# ipadm create-ip bond0_1 root@zoss01:~# ipadm create-ipmp bond0 root@zoss01:~# ipadm add-ipmp -i bond0_0 -i bond0_1 bond0 root@zoss01:~# ipadm set-ifprop -p standby=on -m ip bond0_1 root@zoss01:~# ipadm create-addr -T static -a 192.168.10.111/24 bond0/v4

Here is an example of how to configure bond1:

root@zoss01:~# ipadm create-ip bond1_0 root@zoss01:~# ipadm create-ip bond1_1 root@zoss01:~# ipadm create-ipmp bond1 root@zoss01:~# ipadm add-ipmp -i bond1_0 -i bond1_1 bond1 root@zoss01:~# ipadm set-ifprop -p standby=on -m ip bond1_1 root@zoss01:~# ipadm create-addr -T static -a 192.168.99.111/24 bond1/v4

Here is an example of what ipadm command will return after configuration:

root@zoss01:~# ipadm NAME              CLASS/TYPE STATE        UNDER      ADDR bond0             ipmp       ok           --         --    bond0/v4       static     ok           --         192.168.10.111/24 bond1             ipmp       ok           --         --    bond1/v4       static     ok           --         192.168.99.111/24 bond0_0           ip         ok           bond0      -- bond0_1           ip         ok           bond0      -- bond1_0           ip         ok           bond1      -- bond1_1           ip         ok           bond1      -- lo0               loopback   ok           --         --    lo0/v4         static     ok           --         127.0.0.1/8    lo0/v6         static     ok           --         ::1/128

By using anet resource, we can use the same link-name for different zones, in our example above, the underlying interfaces for bond0 are called bond0_0 and bond0_1, the same name can be used across different zones, system administrator may find it easier to manage and simpler to automate administrative tasks.

From the global zone point of view, zone-specific network interfaces will show up only when a zone is up and running, this can be illustrated by running dladm command in the global zone:

root@el01cn01:~# dladm LINK                CLASS     MTU    STATE    OVER igb0                phys      1500   up       -- igb2                phys      1500   unknown  -- igb1                phys      1500   unknown  -- igb3                phys      1500   unknown  -- usbecm0             phys      1500   up       -- ibp1                phys      65520  up       -- eoib0               phys      1500   up       -- eoib1               phys      1500   up       -- ibp0                phys      65520  up       -- bond0_0             part      65520  up       ibp0 bond0_1             part      65520  up       ibp1 vnic3066_0          vnic      1500   up       eoib0 vnic3066_1          vnic      1500   up       eoib1 zoss01/bond0_0      part      65520  up       ibp0 zoss01/bond0_1      part      65520  up       ibp1 zoss01/bond1_0      vnic      1500   up       eoib0 zoss01/bond1_1      vnic      1500   up       eoib1

The last four rows will only appear when zoss01 is running and they clearly indicate which zone they belong to. Without using anet resources, network interfaces have to be created explicitly in the global zone and they will appear in the command output no matter the zone is running or not.

 

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha