Security groups in cloudstack

Security groups in Cloudstack:

Cloudstack basic zone network uses the security groups for isolation.
The security group rules are taken from the cloudstack 4.2

Security groups isolation is implemented in the hypervisor. Security groups in cloudstack are supported in the below hypervisors.

1. Xenserver
2. KVM

How security groups works:
While creating an instance you can add the instance into one or more security groups. What this makes is that later if you configure security group ingress/egress rules into any of the SG then the rule get applied to this VM.

Security groups default behaviour:

1. By default VM  egress traffic is allowed.
2. By default VM ingress traffic is bloked.
3. With out ingress rule one guest VM can NOT talk to another guest VM.

How security groups implemented using iptables:

iptables configuration for the SG in the hypervisor. The below example rules are taken from the xenserver with cloudstack version 4.2.

iptables FORWARD Chain rules:

All the VM traffic goes through the FORWARD chain of the iptables. This chain has first rule BRIDGE-FIREWALL a user defined chain and the traffic is passed to this chain.

The BRIDGE-FIREWALL chain has rules for all the user VMs and system VMs. This chain has two rules for each VM.
1. Ingress - The traffic which is out from physical dev and into the VIF of the VM
2. Egress -  The traffic which is into the physical dev and out from the VIF of the VM

User VM specific rules are in <vm-name>-def chain.

When ingress/egress rules is configured in SG, the rules goes into all VMs ingress/egress chains which are in this security group.

VM Ingress chain name:  <vm-name>
VM egress chain name: <vm-name>-eg
See the below image for the vm i-2-3-VM

Cloudstack allows the packets to/from the VM which having src/dst as its vm ip address. The spoofing packets from the VM are blocked by the SG.

There is ipset chain  for each vm. It contains the ip addresses of the VM. It is used for matching the ip address in iptables.
In case of multiple ip address to nic then vm ipset contains more than one ip.

Log file for each VM:

cloudstack stores vm specific information in a file. cloudstack compares this information and run time information and uses this to reapply rules. The file has below information.

vmName, vmID, vmIP, domID, signature, seqno, vmMac

Security groups ebtables configuration:

ebtables-save output:

#ebtables-save

# Generated by ebtables-save v1.0 on Mon Sep  2 05:09:00 UTC 2013

*filter

:INPUT ACCEPT

:FORWARD ACCEPT

:OUTPUT ACCEPT

:DEFAULT_EBTABLES ACCEPT

:i-2-3-VM ACCEPT

-A FORWARD -j DEFAULT_EBTABLES

-A FORWARD -i vif7.0 -j i-2-3-VM

-A FORWARD -o vif7.0 -j i-2-3-VM

-A DEFAULT_EBTABLES -p IPv4 --ip-dst 255.255.255.255 --ip-proto udp --ip-dport 67 -j ACCEPT

-A DEFAULT_EBTABLES -p IPv4 --ip-dst 255.255.255.255 --ip-proto udp --ip-dport 68 -j ACCEPT

-A DEFAULT_EBTABLES -p ARP --arp-op Request -j ACCEPT

-A DEFAULT_EBTABLES -p ARP --arp-op Reply -j ACCEPT

-A DEFAULT_EBTABLES -p IPv4 -d Broadcast -j DROP

-A DEFAULT_EBTABLES -p IPv4 -d Multicast -j DROP

-A DEFAULT_EBTABLES -p IPv4 --ip-dst 255.255.255.255 -j DROP

-A DEFAULT_EBTABLES -p IPv4 --ip-dst 224.0.0.0/4 -j DROP

-A DEFAULT_EBTABLES -p IPv4 -j RETURN

-A DEFAULT_EBTABLES -p IPv6 -j DROP

-A DEFAULT_EBTABLES -p 802_1Q -j DROP

-A DEFAULT_EBTABLES -j DROP

-A i-2-3-VM -s ! 6:f8:c8:0:0:9 -i vif7.0 -j DROP

-A i-2-3-VM -p IPv4 -i vif7.0 --ip-proto udp --ip-dport 68 -j DROP

-A i-2-3-VM -p IPv4 -o vif7.0 --ip-proto udp --ip-dport 67 -j DROP

Security groups arptables configuration:

output of 'arptables -L'

Chain INPUT (policy ACCEPT 3788 packets, 106K bytes)

Chain OUTPUT (policy ACCEPT 1 packets, 28 bytes)

Chain FORWARD (policy ACCEPT 22968 packets, 643K bytes)

-j i-2-3-VM -i vif7.0 -o any , pcnt=22 -- bcnt=616 

-j i-2-3-VM -i any -o vif7.0 , pcnt=3804 -- bcnt=107K 

Chain i-2-3-VM (2 references)

-j ACCEPT -i vif7.0 -o any -s 10.147.41.238 --src-mac 06:f8:c8:00:00:09 --opcode Reply , pcnt=15 -- bcnt=420 

-j RETURN -i vif7.0 -o any -s 10.147.41.238 --src-mac 06:f8:c8:00:00:09 --opcode Request , pcnt=7 -- bcnt=196 

-j ACCEPT -i any -o vif7.0 -d 10.147.41.238 --opcode Request , pcnt=1 -- bcnt=28 

-j ACCEPT -i any -o vif7.0 -d 10.147.41.238 --dst-mac 06:f8:c8:00:00:09 --opcode Reply , pcnt=1 -- bcnt=28 

-j DROP -i any -o any , pcnt=3801 -- bcnt=106K 

Rules reference on xenserver

A xenserver is added in basic zone and deployed one VM i-2-3 in that zone.

In this link Xenserver-sg-rules you can find example 

1. iptables security group rules configured on the host.
2. arptable security group rules
3. ebtables security group rules