Wednesday, October 2, 2013

Adding Juniper SRX external firewall in Cloudstack

                        Configure Juniper SRX in cloudstack


Before adding SRX into cloudstack there are two steps:

1. Preconfigure the SRX
2. Add SRX into cloudstack

Preconfigure SRX


Below explained physical device external firewall SRX configuration. This configuration has to done by the admin. This configuration is not part of the cloudstack orchestration job.

1.  Configure the system 

command to configure xnm-clear-text :
#set system services xnm-clear-text

Configure the name server:
set system name-server 10.103.128.16


set system time-zone Asia/Calcutta
set system root-authentication encrypted-password "$1$ucpHjRfH$dNkhOuzKXJxrpAtewvTu.1"
set system name-server 10.103.128.16
set system services ssh
set system services telnet
set system services xnm-clear-text
set system services dns
set system services web-management http interface vlan.0
set system services web-management http interface fe-0/0/0.0
set system services web-management https system-generated-certificate
set system services web-management https interface vlan.0


System services configuration:



2.  Configure the interfaces

Choose three interfaces in the srx device. Example took fe-0/0/0, fe-0/0/1 and fe-0/0/4

Three interfaces needs to be configure.
1. Management interface    - fe-0/0/0
2. Private/Guest interface   - fe-0/0/1
3. Public Interface              - fe-0/0/4.52 where 52 is the public VLAN

Commands to configure interfaces:


set interfaces fe-0/0/0 description "Management Interface"
set interfaces fe-0/0/0 unit 0 family inet address 10.147.40.3/23
set interfaces fe-0/0/1 description "Private network"
set interfaces fe-0/0/1 vlan-tagging
set interfaces fe-0/0/4 description "Public Network"
set interfaces fe-0/0/4 vlan-tagging
set interfaces fe-0/0/4 unit 52 vlan-id 52
set interfaces fe-0/0/4 unit 52 family inet sampling input
set interfaces fe-0/0/4 unit 52 family inet sampling output
set interfaces fe-0/0/4 unit 52 family inet address 10.147.52.3/24


Interface config:



3. Create zones

3.1 Create trust zone for guest network
3.2 Create untrust zone for public network
3.3 Add management interface trust zone and public interface to untrust zone.

Commands to Configuration zones:

root# show security zones | display set
set security zones security-zone trust host-inbound-traffic system-services all
set security zones security-zone trust interfaces fe-0/0/0.0
set security zones security-zone untrust host-inbound-traffic system-services ssh
set security zones security-zone untrust host-inbound-traffic system-services ping
set security zones security-zone untrust interfaces fe-0/0/4.52

Zones Config:


4. Allow traffic from trust zone to trust zone

Commands to configure security policy:

set security policies from-zone trust to-zone trust policy accept-all match source-address any
set security policies from-zone trust to-zone trust policy accept-all match destination-address any
set security policies from-zone trust to-zone trust policy accept-all match application any
set security policies from-zone trust to-zone trust policy accept-all then permit

5.  Add the specific routes to differentiate private and public traffic.

In the below routes 10.147.40.0, 10.146.0.0, 10.147.59.0 are the management network cidrs.
There is default route to public network gateway.

Command to configure routes:

set routing-options static route 10.147.40.0/23 next-hop 10.147.40.1
set routing-options static route 10.147.40.0/23 install
set routing-options static route 10.146.0.0/24 next-hop 10.147.40.1
set routing-options static route 10.146.0.0/24 install
set routing-options static route 0.0.0.0/0 next-hop 10.147.52.1
set routing-options static route 0.0.0.0/0 install
set routing-options static route 10.147.59.0/24 next-hop 10.147.40.1
set routing-options static route 10.147.59.0/24 install


Routes config:



Add SRX into cloudstack


1. After configuring the SRX into cloudstack make sure that the SRX is management ip address is reachable from the management server.

2.  Add the SRX in cloudstack.



3. Enable the SRX.

4. Create network offering to use the physical external firewall device SRX.
Dhcp: Virtual Router
Dns:    Virtual Router
Firewall: Juniper SRX
Source NAT: Juniper SRX
Static NAT: Juniper SRX
Port Forwarding: Juniper SRX






5. Deploy VM using the offering created for SRX.


Configure source NAT rule on SRX

In network offering selected 'Supported Source NAT type': per zone, So you need to configure source
nat manually on the SRX.

This configuration is required for VM to reach public network

Commands to configure source NAT:


set security nat source pool 10-147-52-3 address 10.147.52.3/32
set security nat source rule-set trust from zone trust
set security nat source rule-set trust to zone untrust
set security nat source rule-set trust rule demo match source-address 10.0.32.0/20
set security nat source rule-set trust rule demo then source-nat pool 10-147-52-3

Source NAT configuration:


Accessing public network after configuring Source NAT rule on SRX:



Cloudstack Static NAT rule on aquired ip of srxnetwork:




Set Firewall rule to on the public ip of srx network:


Accessing VM using the public ip and above configured rules:



Cloudstack Configuration files:

The config file included the pre configuration,  config after vm deployment and config after static nat, fw configuration

https://www.dropbox.com/sh/4zhgdpj7q0rc2d8/5g-cNyWA1i

Friday, September 20, 2013

How to find VIF corresponding to VM in xenserver


The below logs are taken from the cloudstack configured xenserver.
In cloudstack Advanced zone with isolated network is created and deployed VM in isolated network.

List of VMs running in xenserver:


[root@Rack1Pod1Host22 ~]# xe vm-list  power-state=running params=dom-id,name-label,uuid
uuid ( RO)          : 20238d97-f5cb-9de7-5605-771f2052b051
    name-label ( RW): s-1-VM
        dom-id ( RO): 4


uuid ( RO)          : 44b0e85e-00b4-73c0-6405-eacff81b8bbe
    name-label ( RW): i-2-3-VM
        dom-id ( RO): 7


uuid ( RO)          : 64abc60a-feeb-5dc6-7a1a-18b2b60ac75b
    name-label ( RW): r-4-VM
        dom-id ( RO): 6


uuid ( RO)          : 80f42fd0-0a78-a971-af67-640c96ce44fd
    name-label ( RW): v-2-VM
        dom-id ( RO): 2


uuid ( RO)          : c2e2ed54-9594-49f4-80aa-df341f998d6b
    name-label ( RW): Control domain on host: Rack1Pod1Host22
        dom-id ( RO): 0


VIF (virtual interface) of a VM in xenserver

-- vif# corresponds to VM i-2-3-VM is vif7.0 where 7 is the domainID
The VIFs belongs VM will have VIFx.y where x is domain ID and y starts from 0

#ifconfig
vif7.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:38 errors:0 dropped:7 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:1098 (1.0 KiB)  TX bytes:12734 (12.4 KiB)

The v-2-VM VM is having dom-id 2 and its interfaces are shown below ifconfig output with vif2.0, vif2.1, vif2.2


    name-label ( RW): v-2-VM
        dom-id ( RO): 2


ifconfig output after deploying VM

[root@xen6 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr D4:AE:52:BC:E6:51
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8705992 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8915911 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1427874321 (1.3 GiB)  TX bytes:997557971 (951.3 MiB)
          Interrupt:16 Memory:c0000000-c0012800

eth1      Link encap:Ethernet  HWaddr D4:AE:52:BC:E6:52
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:17 Memory:c2000000-c2012800

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:97703 errors:0 dropped:0 overruns:0 frame:0
          TX packets:97703 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:129495503 (123.4 MiB)  TX bytes:129495503 (123.4 MiB)

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:44 errors:0 dropped:36 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:0 (0.0 b)  TX bytes:2168 (2.1 KiB)

vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:12 errors:0 dropped:0 overruns:0 frame:0
          TX packets:64 errors:0 dropped:6 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:780 (780.0 b)  TX bytes:3228 (3.1 KiB)

vif2.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:2430 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13823 errors:0 dropped:38 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:249373 (243.5 KiB)  TX bytes:967543 (944.8 KiB)

vif2.2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:216 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11581 errors:0 dropped:37 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:13860 (13.5 KiB)  TX bytes:714949 (698.1 KiB)

vif4.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:9 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:740 (740.0 b)  TX bytes:1554 (1.5 KiB)

vif4.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:1717 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12634 errors:0 dropped:33 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:236632 (231.0 KiB)  TX bytes:957375 (934.9 KiB)

vif4.2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:278 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11292 errors:0 dropped:33 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:17840 (17.4 KiB)  TX bytes:699899 (683.4 KiB)

vif4.3    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:5 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10990 errors:0 dropped:60 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:408 (408.0 b)  TX bytes:675113 (659.2 KiB)

vif6.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:301 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10772 errors:0 dropped:36 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:20852 (20.3 KiB)  TX bytes:669426 (653.7 KiB)

vif6.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:125 errors:0 dropped:0 overruns:0 frame:0
          TX packets:115 errors:0 dropped:2 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:20664 (20.1 KiB)  TX bytes:19922 (19.4 KiB)

vif7.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40 errors:0 dropped:7 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:1098 (1.0 KiB)  TX bytes:13418 (13.1 KiB)

xapi0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet addr:169.254.0.1  Bcast:169.254.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:163 errors:0 dropped:0 overruns:0 frame:0
          TX packets:209 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:23216 (22.6 KiB)  TX bytes:24726 (24.1 KiB)

xenbr0    Link encap:Ethernet  HWaddr D4:AE:52:BC:E6:51
          inet addr:10.147.40.22  Bcast:10.147.41.255  Mask:255.255.254.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8697445 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7091372 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1270037854 (1.1 GiB)  TX bytes:838818223 (799.9 MiB)

xenbr1    Link encap:Ethernet  HWaddr D4:AE:52:BC:E6:52
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:468 (468.0 b)

[root@xen6 ~]#

bridges information in xenserver which is added to cloudstack 

[root@xen6 ~]# brctl show
bridge name bridge id STP enabled interfaces
xapi0 8000.feffffffffff no vif0.0
vif2.0
vif4.0
vif6.1
xenbr0 8000.d4ae52bce651 no eth0
vif2.1
vif2.2
vif4.1
vif4.2
vif4.3
vif6.0
vif7.0
xenbr1 8000.d4ae52bce652 no eth1


Thursday, September 5, 2013

Security groups in cloudstack

Security groups in Cloudstack:

Cloudstack basic zone network uses the security groups for isolation.
The security group rules are taken from the cloudstack 4.2

Security groups isolation is implemented in the hypervisor. Security groups in cloudstack are supported in the below hypervisors.

1. Xenserver
2. KVM

How security groups works:
While creating an instance you can add the instance into one or more security groups. What this makes is that later if you configure security group ingress/egress rules into any of the SG then the rule get applied to this VM.

Security groups default behaviour:

  1. By default VM  egress traffic is allowed.
  2. By default VM ingress traffic is bloked.
  3. With out ingress rule one guest VM can NOT talk to another guest VM.


How security groups implemented using iptables:

iptables configuration for the SG in the hypervisor. The below example rules are taken from the xenserver with cloudstack version 4.2.

iptables FORWARD Chain rules:

All the VM traffic goes through the FORWARD chain of the iptables. This chain has first rule BRIDGE-FIREWALL a user defined chain and the traffic is passed to this chain.


The BRIDGE-FIREWALL chain has rules for all the user VMs and system VMs. This chain has two rules for each VM.
1. Ingress - The traffic which is out from physical dev and into the VIF of the VM
2. Egress -  The traffic which is into the physical dev and out from the VIF of the VM

User VM specific rules are in <vm-name>-def chain.




When ingress/egress rules is configured in SG, the rules goes into all VMs ingress/egress chains which are in this security group.

VM Ingress chain name:  <vm-name>
VM egress chain name: <vm-name>-eg
See the below image for the vm i-2-3-VM

Cloudstack allows the packets to/from the VM which having src/dst as its vm ip address. The spoofing packets from the VM are blocked by the SG.




There is ipset chain  for each vm. It contains the ip addresses of the VM. It is used for matching the ip address in iptables.
In case of multiple ip address to nic then vm ipset contains more than one ip.


Log file for each VM:

cloudstack stores vm specific information in a file. cloudstack compares this information and run time information and uses this to reapply rules. The file has below information.

vmName, vmID, vmIP, domID, signature, seqno, vmMac

Security groups ebtables configuration:

ebtables-save output:

#ebtables-save
# Generated by ebtables-save v1.0 on Mon Sep  2 05:09:00 UTC 2013
*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
:DEFAULT_EBTABLES ACCEPT
:i-2-3-VM ACCEPT
-A FORWARD -j DEFAULT_EBTABLES
-A FORWARD -i vif7.0 -j i-2-3-VM
-A FORWARD -o vif7.0 -j i-2-3-VM
-A DEFAULT_EBTABLES -p IPv4 --ip-dst 255.255.255.255 --ip-proto udp --ip-dport 67 -j ACCEPT
-A DEFAULT_EBTABLES -p IPv4 --ip-dst 255.255.255.255 --ip-proto udp --ip-dport 68 -j ACCEPT
-A DEFAULT_EBTABLES -p ARP --arp-op Request -j ACCEPT
-A DEFAULT_EBTABLES -p ARP --arp-op Reply -j ACCEPT
-A DEFAULT_EBTABLES -p IPv4 -d Broadcast -j DROP
-A DEFAULT_EBTABLES -p IPv4 -d Multicast -j DROP
-A DEFAULT_EBTABLES -p IPv4 --ip-dst 255.255.255.255 -j DROP
-A DEFAULT_EBTABLES -p IPv4 --ip-dst 224.0.0.0/4 -j DROP
-A DEFAULT_EBTABLES -p IPv4 -j RETURN
-A DEFAULT_EBTABLES -p IPv6 -j DROP
-A DEFAULT_EBTABLES -p 802_1Q -j DROP
-A DEFAULT_EBTABLES -j DROP
-A i-2-3-VM -s ! 6:f8:c8:0:0:9 -i vif7.0 -j DROP
-A i-2-3-VM -p IPv4 -i vif7.0 --ip-proto udp --ip-dport 68 -j DROP
-A i-2-3-VM -p IPv4 -o vif7.0 --ip-proto udp --ip-dport 67 -j DROP

Security groups arptables configuration:

output of 'arptables -L'

Chain INPUT (policy ACCEPT 3788 packets, 106K bytes)

Chain OUTPUT (policy ACCEPT 1 packets, 28 bytes)

Chain FORWARD (policy ACCEPT 22968 packets, 643K bytes)
-j i-2-3-VM -i vif7.0 -o any , pcnt=22 -- bcnt=616 
-j i-2-3-VM -i any -o vif7.0 , pcnt=3804 -- bcnt=107K 

Chain i-2-3-VM (2 references)
-j ACCEPT -i vif7.0 -o any -s 10.147.41.238 --src-mac 06:f8:c8:00:00:09 --opcode Reply , pcnt=15 -- bcnt=420 
-j RETURN -i vif7.0 -o any -s 10.147.41.238 --src-mac 06:f8:c8:00:00:09 --opcode Request , pcnt=7 -- bcnt=196 
-j ACCEPT -i any -o vif7.0 -d 10.147.41.238 --opcode Request , pcnt=1 -- bcnt=28 
-j ACCEPT -i any -o vif7.0 -d 10.147.41.238 --dst-mac 06:f8:c8:00:00:09 --opcode Reply , pcnt=1 -- bcnt=28 
-j DROP -i any -o any , pcnt=3801 -- bcnt=106K 

Rules reference on xenserver

A xenserver is added in basic zone and deployed one VM i-2-3 in that zone.

In this link Xenserver-sg-rules you can find example 
  1. iptables security group rules configured on the host.
  2. arptable security group rules
  3. ebtables security group rules




Wednesday, September 4, 2013

Running the cloudstack in simulator

Running the cloudstack in simulator:


1. If you are new to cloudstack checkout out cloudstack code using the below link.

    #git clone https://git-wip-us.apache.org/repos/asf/cloudstack.git


2. To set up Cloudstack development environment refer the below link

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+CloudStack+Development+Environment

3. If you already have cloudstack, build the cloudstack using the below commands (If you are in ubuntu build in root prompt)

# mvn -Pdeveloper -Dsimulator clean install

# mvn -Pdeveloper -pl developer -Ddeploydb
# mvn -Pdeveloper -pl developer -Ddeploydb-simulator

start the cloudstack MS:

#mvn -pl client jetty:run -Dsimulator

4. Unset the MAVEN_OPTS
#unset MAVEN_OPTS

5. Set up zone using the simulator.
#mvn -Pdeveloper,marvin.setup -Dmarvin.config=setup/dev/advanced.cfg -pl :cloud-marvin integration-test