Starting with Fedora 16 the openvswitch user space tools and the required kernel modules were included in the Fedora distribution. I spent some time on these packages and here an introduction and some information on how to use these components.
Openvswitch is an alternative to the bridge module which has been part of the kernel, since the 2.4 series. The openvswitch has quite a few enhancements compared to the standard bridge module. These are described in detail at openvswitch.org. One of these enhancements is the support for the OpenFlow protocol.
Conceptually,a switch’s functions can be broken down into two pieces: Control plane, data plane. The control plane is the core intelligence of the switch which handles discovery, routing, path computation, communication with other switches etc.The control plane creates a forwarding/flow table which the data plane uses to process the incoming packets at line speed. OpenFlow is a protocol which lets you offload the control plane of all the switches to a central controller and lets a central software define the behavior of the network (also called Software Defined Networks).
This setup is in contrast to the current infrastructures with proprietary switches using proprietary tools/protocols for management. OpenFlow being an open standard allows switches from different vendors to be managed in a standard way without tying the infrastructure down with any proprietary tools.
In Fedora, the implementation of the openvswitch is broken down to two pieces, the openvswitch kernel module (Data Plane) and some user space tools (Control Plane). Since the incoming data Packets have to be processed as fast as possible, the data plane of the openvswitch was pushed to kernel space.
In order to use the openvswitch for networking VMs, the openvswitch service should be started and the command line tools have to be used to properly configure the openvswitch (will refer as vswitch here on) based bridge. The vswitch in Fedora can work in two modes: standalone, OpenFlow modes.
This article will primarily concentrate on how to bring up a VM connected toopenvswitch’s bridge in both Standalone & OpenFlow modes. The VirtualMachine will be attached to a TAP device, which will be connected to the openvswitch’s bridge and traffic will be forwarded to the physical network via the physical NIC attached to the bridge.
Before you go any further, install the openvswitch package with:
# yum install openvswitch
Bridging in Standalone Mode
As the name suggests, in this mode the vswitch’s bridge handles all the switching/forwarding functionality by itself. To bring up the vswitch’s bridgestart the openvswitch service with
# service openvswitch start
Use the standard tunctl command for creating a tap device:
# tunctl –t tap0 –u root
Use the following commands to stop the NetworkManager service, bring down the em1 interface and flush all the ip addresses (if any) on the em1 interface (em1 being the NIC which will be used by the VMs to access the physical network)
# service NetworkManager stop
# ip add flush em1
# ifconfig down em1
Run the following commands to create a vswitch-based bridge by the name ovs-switch :
# ovs-vsctl add-br ovs-switch
# ovs-vsctl add-port ovs-switch tap0
# ovs-vsctl add-port ovs-switch em1
# ifconfig tap0 promisc up
# ifconfig em1 promisc up
These commands create a new bridge and add a tap device and local network port (em1) to the bridge. Setting these ports in promiscuous mode will let the traffic from the VM to be forwarded to host’s physical network and vice versa. Set an IP address on the ovs-switch interface with DHCP as follows:
# dhclient ovs-switch
Now you can start your Virtual Machine with a command similar to the following:
# /usr/bin/qemu-kvm -smp 2,cores=2 -net tap,ifname=tap0,script=no -net nic,model=rtl8139,macaddr=52:54:00:45:67:30 -m 2048M ~/multiqueue/fedora.img
The virt-manager doesn’t yet support openvswitch. So there are currently no options available to start a VM using vswitch from virt-manager.
For running openvswitch in OpenFlow mode most of the commands from the above section have to be used. So, the commands are copied below to be comprehensive:
# yum install openvswitch
# service openvswitch start
# service NetworkManager stop
# ovs-vsctl add-br of-switch
# ovs-vsctl add-port of-switch tap0
# ovs-vsctl add-port of-switch em1
# ifconfig em1 promisc up
NOTE: the bridge setup in OpenFlow mode will be referred to as of-switch here on.
Before you go any further, an OpenFlow controller should be up and running. The moment the bridge comes up it needs to talk to an OpenFlow Controller to make any forwarding decisions. Quite a few controllers available in the form of both software and hardware. The POX controller (software) will be used in this article. Download the sources of POX from https://github.com/noxrepo/pox with the following command:
# git clone https://github.com/noxrepo/pox
After downloading the sources the controller can be run as follows:
# ./pox.py forwarding.l2_learning
The above command will start POX with OpenFlow enabled and uses a pluginforwarding.l2_learning on top. This plugin checks the incoming packets andautomatically creates a learning table for the bridge. Based on yourinfrastructure requirements you can customize the available plugins or createnew ones. You can start the POX controller without any plugins (as ./pox.py), but the controller will not know what to do with an incoming traffic. The user will have to manually setup the flows in the bridge in this scenario.
The above command will make pox listen on all network interfaces. If pox has to listen on a particular interface and a particular port, you can start pox with
#./pox.py OpenFlow.of_01 --address=192.168.4.197 --port=6666 forwarding.l2_learning
The openvswitch doesn’t support the hybrid openflow mode, where some of the traffic is processed by the built-in switch logic of openvswitch and some of the traffic is processed with the help of an OpenFlow Controller. The openvswitch has a failure mode of operation though. If the switch loses its connection with the Controller, it can fall back to using its built-in logic. This fall back behavior can be configured with the following commands:
# ovs-vsctl set-fail-mode ovs-switch standalone
The above command will make the bridge fallback to the standalone mode if the connection to the controller fails at some point.
#ovs-vsctl set-fail-mode ovs-switch secure
After setting the fallback mode to secure, if the connection to the OF controller fails, the switch will not be able to forward any packets at all.
Consider the scenario where the OF controller is running on a remote host and the vswitch’s bridge (of-switch) is brought up in secure mode. If dhclient in run on the of-switch the switch has no idea how to forward the DHCP traffic as it cannot talk to a remote Controller without an IP address. So you either need to set the of-switch interface’s fall back mode to standalone mode OR assign astatic ip and setup up a static route so that the bridge can access the remotecontroller. Once the bridge establishes a connection to the remote controller,the traffic will be forwarded by the bridge based on the plugins setup in theOF controller.
Now that the bridge and the OF controller are running, the controller information can be assigned to the bridge with the following command:
# ovs-vsctl set-controller of-switch tcp:0.0.0.0:6633
NOTE: with the above command, the bridge is connecting to an OF controller on the localhost which is listening on port 6633
Since the bridge and the controller are setup, you can power on the VM now (as shown in the above qemu command) and get the traffic directed by the OF controller. Whenever a new packet comes in and the bridge has no information on how to forward it, the bridge will send the packets header (with L2, L3 and some L4 information) to the controller. The controller will respond with an OF command telling the bridge how to forward traffic similar to that packet. The flows thusly assigned to the bridge can be listed with the following command:
# ovs-ofctl dump-flows of-switch
An example of one such response from the controller is shown below. A flow entry primarily contains a set of field=value entries and action entry. The field=value entries are used to identify the incoming packet and the actions tells the bridge with what to do with the matching traffic.
cookie=0x0, duration=14.604s, table=0, n_packets=61, n_bytes=7418, idle_timeout=10, hard_timeout=30,tcp, vlan_tci=0x0000, dl_src=78:2b:cb:4b:db:c5, dl_dst=00:21:9b:8e:36:62, nw_src=192.168.7.189, nw_dst=192.168.1.150, nw_tos=0, tp_src=22, tp_dst=60221actions=output:1
The above flow should be self-explanatory. If the traffic comes in from src mac address 78:2b:cb:4b:db:c5, destination mac address 00:21:9b:8e:36:62, traffic is tcp traffic, src ip=192.168.7.189, dest ip=192.168.1.150, TCP source port 22, tcp destination port 60221 forward the packet to port 1 (actions:1).
The ports configured in the bridge can be seen with the following command
# ovs-ofctl show ovs
OFPT_FEATURES_REPLY (xid=0x1): ver:0x1,dpid:0000782bcb4bdbc5
features: capabilities:0xc7, actions:0xfff
current: 1GB-FD COPPER AUTO_NEG
advertised:10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-HD 1GB-FD COPPER AUTO_NEG AUTO_PAUSE
supported: 10MB-HD 10MB-FD100MB-HD 100MB-FD 1GB-HD 1GB-FD COPPER AUTO_NEG
current: 10MB-FD COPPER
OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normalmiss_send_len=0
According to the above mentioned flow, the traffic has to be forwarded port 1 which is the em1 interface of the host.
If necessary some custom flows can be assigned to the bridge manually, following are some examples of the same:
# ovs-ofctl add-flow of-switch"in_port=LOCAL,table=0,idle_timeout=60,ip,hard_timeout=60,vlan_tci=0x0000, dl_src=78:2b:cb:4b:db:c5,dl_dst=00:09:8a:02:80:c0, nw_proto=1,nw_dst=192.168.1.100, n w_src=192.168.7.189,actions=drop"
The above rule makes the bridge drop all ICMP traffic (nw_proto=1) from 192.168.7.189 to 192.168.1.100.
As shown in the above add-flow example, the values idle_timeout and hard_timeout are defined for all the flows. These values tell the bridge how long this rule has to be used by the bridge. The idle_timeout causes the related flow to be deleted after a defined number of seconds of inactivity. The hard_timeout will delete the flow after the defined number of seconds regardless of the activity. If a static rule has to be setup on the bridge, the idle_timeout and hard_timeout have to be set to 0(zero).
This concludes my investigation for now. Hopefully this article will help more users start playing with the OpenFlow technology.
Disclaimer:Above article is for informational purposes only. The content is from myinterpretation of the technologies/terminologies mentioned. This information is provided as is and may contain some typographical errors and/or technical inaccuracies.