Network Filters

The above example indicates that the traffic of type ip will be asscociated with the chain 'ipv4' and the rule will have priority 500. If for example another filter is referenced whose traffic of type ip is also associated with the chain 'ipv4' then that filter's rules will be ordered relative to the priority 500 of the shown rule.

A rule may contain a single rule for filtering of traffic. The above example shows that traffic of type ip is to be filtered.

Supported protocols

The following sections enumerate the list of protocols that are supported by the network filtering subsystem. The type of traffic a rule is supposed to filter on is provided in the rule node as a nested node. Depending on the traffic type a rule is filtering, the attributes are different. The above example showed the single attribute srcipaddr that is valid inside the ip traffic filtering node. The following sections show what attributes are valid and what type of data they are expecting. The following datatypes are available:

• UINT8 : 8 bit integer; range 0-255
• UINT16: 16 bit integer; range 0-65535
• MAC_ADDR: MAC adrress in dotted decimal format, i.e., 00:11:22:33:44:55
• MAC_MASK: MAC address mask in MAC address format, i.e., FF:FF:FF:FC:00:00
• IP_ADDR: IP address in dotted decimal format, i.e., 10.1.2.3
• IP_MASK: IP address mask in either dotted decimal format (255.255.248.0) or CIDR mask (0-32)
• IPV6_ADDR: IPv6 address in numbers format, i.e., FFFF::1
• IPV6_MASK: IPv6 mask in numbers format (FFFF:FFFF:FC00::) or CIDR mask (0-128)
• STRING: A string
• BOOLEAN: 'true', 'yes', '1' or 'false', 'no', '0'

Every attribute except for those of type IP_MASK or IPV6_MASK can be negated using the match attribute with value no. Multiple negated attributes may be grouped together. The following XML fragment shows such an example using abstract attributes.

[...]
  <rule action='drop' direction='in'>
    <protocol match='no' attribute1='value1' attribute2='value2'/>
    <protocol attribute3='value3'/>
  </rule>
[...]

Rules perform a logical AND evaluation on all values of the given protocol attributes. Thus, if a single attribute's value does not match the one given in the rule, the whole rule will be skipped during evaluation. Therefore, in the above example incoming traffic will only be dropped if the protocol property attribute1 does not match value1 AND the protocol property attribute2 does not match value2 AND the protocol property attribute3 matches value3.

MAC (Ethernet)

Protocol ID: mac
Note: Rules of this type should go into the root chain.

Valid Strings for protocolid are: arp, rarp, ipv4, ipv6

[...]
<mac match='no' srcmacaddr='$MAC'/>
[...]

ARP/RARP

Protocol ID: arp or rarp
Note: Rules of this type should either go into the root or arp/rarp chain.

Valid strings for the Opcode field are: Request, Reply, Request_Reverse, Reply_Reverse, DRARP_Request, DRARP_Reply, DRARP_Error, InARP_Request, ARP_NAK

IPv4

Protocol ID: ip Note: Rules of this type should either go into the root or ipv4 chain.

Valid strings for protocol are: tcp, udp, udplite, esp, ah, icmp, igmp, sctp

IPv6

Protocol ID: ipv6 Note: Rules of this type should either go into the root or ipv6 chain.

Valid strings for protocol are: tcp, udp, udplite, esp, ah, icmpv6, sctp

TCP/UDP/SCTP

Protocol ID: tcp, udp, sctp
Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root.

ICMP

Protocol ID: icmp
Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root.

IGMP, ESP, AH, UDPLITE, 'ALL'

Protocol ID: igmp, esp, ah, udplite, all
Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root.

TCP/UDP/SCTP over IPV6

Protocol ID: tcp-ipv6, udp-ipv6, sctp-ipv6
Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root.

ICMPv6

Protocol ID: icmpv6
Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root.

IGMP, ESP, AH, UDPLITE, 'ALL' over IPv6

Protocol ID: igmp-ipv6, esp-ipv6, ah-ipv6, udplite-ipv6, all-ipv6
Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root.

Advanced Filter Configuration Topics

The following sections discuss advanced filter configuration topics.

Connection tracking

The network filtering subsystem (on Linux) makes use of the connection tracking support of iptables. This helps in enforcing the directionality of network traffic (state match) as well as counting and limiting the number of simultaneous connections towards a VM. As an example, if a VM has TCP port 8080 open as a server, clients may connect to the VM on port 8080. Connection tracking and enforcement of directionality then prevents the VM from initiating a connection from (TCP client) port 8080 to the host back to a remote host. More importantly, tracking helps to prevent remote attackers from establishing a connection back to a VM. For example, if the user inside the VM established a connection to port 80 on an attacker site, then the attacker will not be able to initiate a connection from TCP port 80 back towards the VM. By default the connection state match that enables connection tracking and then enforcement of directionality of traffic is turned on.
The following shows an example XML fragement where this feature has been turned off for incoming connections to TCP port 12345.

   [...]
    <rule direction='in' action='accept' statematch='false'>
      <tcp dstportstart='12345'/>
    </rule>
   [...]

This now allows incoming traffic to TCP port 12345, but would also enable the initiation from (client) TCP port 12345 within the VM, which may or may not be desirable.

Limiting Number of Connections

To limit the number of connections a VM may establish, a rule must be provided that sets a limit of connections for a given type of traffic. If for example a VM is supposed to be allowed to only ping one other IP address at a time and is supposed to have only one active incoming ssh connection at a time, the following XML fragment can be used to achieve this.

  [...]
  <rule action='drop' direction='in' priority='400'>
    <tcp connlimit-above='1'/>
  </rule>
  <rule action='accept' direction='in' priority='500'>
    <tcp dstportstart='22'/>
  </rule>
  <rule action='drop' direction='out' priority='400'>
    <icmp connlimit-above='1'/>
  </rule>
  <rule action='accept' direction='out' priority='500'>
    <icmp/>
  </rule>
  <rule action='accept' direction='out' priority='500'>
    <udp dstportstart='53'/>
  </rule>
  <rule action='drop' direction='inout' priority='1000'>
    <all/>
  </rule>
  [...]

Note that the rule for the limit has to logically appear before the rule for accepting the traffic.
An additional rule for letting DNS traffic to port 22 go out the VM has been added to avoid ssh sessions not getting established for reasons related to DNS lookup failures by the ssh daemon. Leaving this rule out may otherwise lead to fun-filled debugging joy (symptom: ssh client seems to hang while trying to connect).

Lot of care must be taken with timeouts related to tracking of traffic. An ICMP ping that the user may have terminated inside the VM may have a long timeout in the host's connection tracking system and therefore not allow another ICMP ping to go through for a while. Therefore, the timeouts have to be tuned in the host's sysfs, i.e.,

  echo 3 > /proc/sys/net/netfilter/nf_conntrack_icmp_timeout

sets the ICMP connection tracking timeout to 3 seconds. The effect of this is that once one ping is terminated, another one can start after 3 seconds.
Further, we want to point out that a client that for whatever reason has not properly closed a TCP connection may cause a connection to be held open for a longer period of time, depending to what timeout the TCP established state timeout has been set to on the host. Also, idle connections may time out in the connection tracking system but can be reactivated once packets are exchanged. However, a newly initiated connection may force an idle connection into TCP backoff if the number of allowed connections is set to a too low limit, the new connection is established and hits (not exceeds) the limit of allowed connections and for example a key is pressed on the old ssh session, which now has become unresponsive due to its traffic being dropped. Therefore, the limit of connections should be rather high so that fluctuations in new TCP connections don't cause odd traffic behavior in relaton to idle connections.

Command line tools

The libvirt command line tool virsh has been extended with life-cycle support for network filters. All commands related to the network filtering subsystem start with the prefix nwfilter. The following commands are available:

• nwfilter-list : list UUIDs and names of all network filters
• nwfilter-define : define a new network filter or update an existing one
• nwfilter-undefine : delete a network filter given its name; it must not be currently in use
• nwfilter-dumpxml : display a network filter given its name
• nwfilter-edit : edit a network filter given its name

Pre-existing network filters

The following is a list of example network filters that are automatically installed with libvirt.

Note that most of the above filters are only building blocks and require a combination with other filters to provide useful network traffic filtering. The most useful one in the above list is the clean-traffic filter. This filter itself can for example be combined with the no-ip-multicast filter to prevent virtual machines from sending IP multicast traffic on top of the prevention of packet spoofing.

Writing your own filters

Since libvirt only provides a couple of example networking filters, you may consider writing your own. When planning on doing so there are a couple of things you may need to know regarding the network filtering subsystem and how it works internally. Certainly you also have to know and understand the protocols very well that you want to be filtering on so that no further traffic than what you want can pass and that in fact the traffic you want to allow does pass.

The network filtering subsystem is currently only available on Linux hosts and only works for Qemu and KVM type of virtual machines. On Linux it builds upon the support for ebtables, iptables and ip6tables and makes use of their features. From the above list of supported protocols the following ones are implemented using ebtables:

• mac
• arp, rarp
• ip
• ipv6

All other protocols over IPv4 are supported using iptables, those over IPv6 are implemented using ip6tables.

On a Linux host, all traffic filtering instantiated by libvirt's network filter subsystem first passes through the filtering support implemented by ebtables and only then through iptables or ip6tables filters. If a filter tree has rules with the protocols mac, arp, rarp, ip, or ipv6 ebtables rules will automatically be instantiated.
The role of the chain attribute in the network filter XML is that internally a new user-defined ebtables table is created that then for example receives all arp traffic coming from or going to a virtual machine, if the chain arp has been specified. Further, a rule is generated in an interface's root chain that directs all ipv4 traffic into the user-defined chain. Therefore, all ARP traffic rules should then be placed into filters specifying this chain. This type of branching into user-defined tables is only supported with filtering on the ebtables layer.
As an example, it is possible to filter on UDP traffic by source and destination ports using the ip protocol filter and specifying attributes for the protocol, source and destination IP addresses and ports of UDP packets that are to be accepted. This allows early filtering of UDP traffic with ebtables. However, once an IP or IPv6 packet, such as a UDP packet, has passed the ebtables layer and there is at least one rule in a filter tree that instantiates iptables or ip6tables rules, a rule to let the UDP packet pass will also be necessary to be provided for those filtering layers. This can be achieved with a rule containing an approriate udp or udp-ipv6 traffic filtering node.

Example custom filter

As an example we want to now build a filter that fulfills the following list of requirements:

• prevents a VM's interface from MAC, IP and ARP spoofing
• opens only TCP ports 22 and 80 of a VM's interface
• allows the VM to send ping traffic from an interface but not let the VM be pinged on the interface
• allows the VM to do DNS lookups (UDP towards port 53)

The requirement to prevent spoofing is fulfilled by the existing clean-traffic network filter, thus we will reference this filter from our custom filter.
To enable traffic for TCP ports 22 and 80 we will add 2 rules to enable this type of traffic. To allow the VM to send ping traffic we will add a rule for ICMP traffic. For simplicity reasons we allow general ICMP traffic to be initated from the VM, not just ICMP echo request and response messages. To then disallow all other traffic to reach or be initated by the VM we will then need to add a rule that drops all other traffic. Assuming our VM is called test and the interface we want to associate our filter with is called eth0, we name our filter test-eth0. The result of these considerations is the following network filter XML:

<filter name='test-eth0'>
  <!-- reference the clean traffic filter to prevent
       MAC, IP and ARP spoofing. By not providing
       and IP address parameter, libvirt will detect the
       IP address the VM is using. -->
  <filterref filter='clean-traffic'/>

  <!-- enable TCP ports 22 (ssh) and 80 (http) to be reachable -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='22'/>
  </rule>

  <rule action='accept' direction='in'>
    <tcp dstportstart='80'/>
  </rule>

  <!-- enable general ICMP traffic to be initiated by the VM;
       this includes ping traffic -->
  <rule action='accept' direction='out'>
    <icmp/>
  </rule>

  <!-- enable outgoing DNS lookups using UDP -->
  <rule action='accept' direction='out'>
    <udp dstportstart='53'/>
  </rule>

  <!-- drop all other traffic -->
  <rule action='drop' direction='inout'>
    <all/>
  </rule>

</filter>

Note that none of the rules in the above XML contain the IP address of the VM as either source or destination address, yet the filtering of the traffic works correctly. The reason is that the evaluation of the rules internally happens on a per-interface basis and the rules are evaluated based on the knowledge about which (tap) interface has sent or will receive the packet rather than what their source or destination IP address may be.

An XML fragment for a possible network interface description inside the domain XML of the test VM could then look like this:

   [...]
    <interface type='bridge'>
      <source bridge='mybridge'/>
      <filterref filter='test-eth0'/>
    </interface>
   [...]

To more strictly control the ICMP traffic and enforce that only ICMP echo requests can be sent from the VM and only ICMP echo responses be received by the VM, the above ICMP rule can be replaced with the following two rules:

  <!-- enable outgoing ICMP echo requests-->
  <rule action='accept' direction='out'>
    <icmp type='8'/>
  </rule>

  <!-- enable incoming ICMP echo replies-->
  <rule action='accept' direction='in'>
    <icmp type='0'/>
  </rule>

Second example custom filter

In this example we now want to build a similar filter as in the example above, but extend the list of requirements with an ftp server located inside the VM. Further, we will be using features that have been added in version 0.8.5. The requirements for this filter are:

• prevents a VM's interface from MAC, IP and ARP spoofing
• opens only TCP ports 22 and 80 of a VM's interface
• allows the VM to send ping traffic from an interface but not let the VM be pinged on the interface
• allows the VM to do DNS lookups (UDP towards port 53)
• enable an ftp server (in active mode) to be run inside the VM

The additional requirement of allowing an ftp server to be run inside the VM maps into the requirement of allowing port 21 to be reachable for ftp control traffic as well as enabling the VM to establish an outgoing tcp connection originating from the VM's TCP port 20 back to the ftp client (ftp active mode). There are several ways of how this filter can be written and we present 2 solutions.

The 1st solution makes use of the state attribute of the TCP protocol that gives us a hook into the connection tracking framework of the Linux host. For the VM-initiated ftp data connection (ftp active mode) we use the RELATED state that allows us to detect that the VM-initiated ftp data connection is a consequence of ( or 'has a relationship with' ) an existing ftp control connection, thus we want to allow it to let packets pass the firewall. The RELATED state, however, is only valid for the very first packet of the outgoing TCP connection for the ftp data path. Afterwards, the state to compare against is ESTABLISHED, which then applies equally to the incoming and outgoing direction. All this is related to the ftp data traffic originating from TCP port 20 of the VM. This then leads to the following solution (since 0.8.5 (Qemu, KVM, UML)):

<filter name='test-eth0'>
  <!-- reference the clean traffic filter to prevent
       MAC, IP and ARP spoofing. By not providing
       and IP address parameter, libvirt will detect the
       IP address the VM is using. -->
  <filterref filter='clean-traffic'/>

  <!-- enable TCP port 21 (ftp-control) to be reachable -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='21'/>
  </rule>

  <!-- enable TCP port 20 for VM-initiated ftp data connection
       related to an existing ftp control connection -->
  <rule action='accept' direction='out'>
    <tcp srcportstart='20' state='RELATED,ESTABLISHED'/>
  </rule>

  <!-- accept all packets from client on the ftp data connection -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='20' state='ESTABLISHED'/>
  </rule>

  <!-- enable TCP ports 22 (ssh) and 80 (http) to be reachable -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='22'/>
  </rule>

  <rule action='accept' direction='in'>
    <tcp dstportstart='80'/>
  </rule>

  <!-- enable general ICMP traffic to be initiated by the VM;
       this includes ping traffic -->
  <rule action='accept' direction='out'>
    <icmp/>
  </rule>

  <!-- enable outgoing DNS lookups using UDP -->
  <rule action='accept' direction='out'>
    <udp dstportstart='53'/>
  </rule>

  <!-- drop all other traffic -->
  <rule action='drop' direction='inout'>
    <all/>
  </rule>

</filter>

Before trying out a filter using the RELATED state, you have to make sure that the approriate connection tracking module has been loaded into the host's kernel. Depending on the version of the kernel, you must run either one of the following two commands before the ftp connection with the VM is established.

    modprobe nf_conntrack_ftp   # where available  or

    modprobe ip_conntrack_ftp   # if above is not available

If other protocols than ftp are to be used in conjunction with the RELATED state, their corresponding module must be loaded. Modules exist at least for the protocols ftp, tftp, irc, sip, sctp, and amanda.

The 2nd solution makes uses the state flags of connections more than the previous solution did. In this solution we take advantage of the fact that the NEW state of a connection is valid when the very first packet of a traffic flow is seen. Subsequently, if the very first packet of a flow is accepted, the flow becomes a connection and enters the ESTABLISHED state. This allows us to write a general rule for allowing packets of ESTABLISHED connections to reach the VM or be sent by the VM. We write specific rules for the very first packets identified by the NEW state and for which ports they are acceptable. All packets for ports that are not explicitly accepted will be dropped and therefore the connection will not go into the ESTABLISHED state and any subsequent packets be dropped.

<filter name='test-eth0'>
  <!-- reference the clean traffic filter to prevent
       MAC, IP and ARP spoofing. By not providing
       and IP address parameter, libvirt will detect the
       IP address the VM is using. -->
  <filterref filter='clean-traffic'/>

  <!-- let the packets of all previously accepted connections reach the VM -->
  <rule action='accept' direction='in'>
    <all state='ESTABLISHED'/>
  </rule>

  <!-- let the packets of all previously accepted and related connections be sent from the VM -->
  <rule action='accept' direction='out'>
    <all state='ESTABLISHED,RELATED'/>
  </rule>

  <!-- enable traffic towards port 21 (ftp), 22 (ssh) and 80 (http) -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='21' dstportend='22' state='NEW'/>
  </rule>

  <rule action='accept' direction='in'>
    <tcp dstportstart='80' state='NEW'/>
  </rule>

  <!-- enable general ICMP traffic to be initiated by the VM;
       this includes ping traffic -->
  <rule action='accept' direction='out'>
    <icmp state='NEW'/>
  </rule>

  <!-- enable outgoing DNS lookups using UDP -->
  <rule action='accept' direction='out'>
    <udp dstportstart='53' state='NEW'/>
  </rule>

  <!-- drop all other traffic -->
  <rule action='drop' direction='inout'>
    <all/>
  </rule>

</filter>

Limitations

The following sections list (current) limitations of the network filtering subsystem.

IP Address Detection

In case a network filter references the variable IP and no variable was defined in any higher layer references to the filter, IP address detection will automatically be started when the filter is to be instantiated (VM start, interface hotplug event). Only IPv4 addresses can be detected and only a single IP address legitimately in use by a VM on a single interface will be detected. In case a VM was to use multiple IP address on a single interface (IP aliasing), the IP addresses would have to be provided explicitly either in the network filter itself or as variables used in attributes' values. These variables must then be defined in a higher level reference to the filter and each assigned the value of the IP address that the VM is expected to be using. Different IP addresses in use by multiple interfaces of a VM (one IP address each) will be independently detected.

Once a VM's IP address has been detected, its IP network traffic may be locked to that address, if for example IP address spoofing is prevented by one of its filters. In that case the user of the VM will not be able to change the IP address on the interface inside the VM, which would be considered IP address spoofing.

In case a VM is resumed after suspension or migrated, IP address detection will be restarted.

VM Migration

VM migration is only supported if the whole filter tree that is referenced by a virtual machine's top level filter is also available on the target host. The network filter clean-traffic for example should be available on all libvirt installations of version 0.8.1 or later and thus enable migration of VMs that for example reference this filter. All other custom filters must be migrated using higher layer software. It is outside the scope of libvirt to ensure that referenced filters on the source system are equivalent to those on the target system and vice versa.

Migration must occur between libvirt insallations of version 0.8.1 or later in order not to lose the network traffic filters associated with an interface.