Single Node OpenStack Startup

Following records the main steps for starting up the single node Openstack environment.

Ubuntu Setup and Configuration

Setup

Use virt-manager, create a new virtual machine, install the system from ubuntu-12.04.3-server-amd64.iso, allocate 2 CPU and 4096 Memory, allocate the 80GB disk.
Create disk via:

Trusty@pc119:~/Code/Virt-Manager/SingleNode> qemu-img create -f qcow2 SingleNode.qcow2 80G
Formatting 'SingleNode.qcow2', fmt=qcow2 size=85899345920 encryption=off cluster_size=65536 lazy_refcounts=off

Configure the networking, bridge, then beging installing.
IP address set to 192.168.188.190/255.255.255.0, gateway first we set it to 192.168.188.1, later we will change it, because it’s a pseudo-addresses.

Configuration

First save a snapshot, shutdown the running virtual machine, then you could clone it for back up.
Configure the network in virtual machine via:

$ sudo ip route add default gw 192.168.0.xxx
$ sudo ip route add 192.168.0.0/24 dev eth0

SSH fast connect:

$ sudo vi /etc/ssh/sshd_config
UseDNS no
$ sudo service ssh restart

Now you could ssh to the virtual machine and update corresponding configurations.

In fact the snapshot could be made here.
Transfer the installtion media to virtual machine:

$ scp ./contrail-install-packages_1.21-74\~havana_all.deb Trusty@192.168.188.190:/home/Trusty/

Install OpenContrail

Set the root password:

$ sudo bash
root@SingleNode:~# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully

Install the deb file via:

root@SingleNode:~# ls
contrail-install-packages_1.21-74~havana_all.deb
root@SingleNode:~# dpkg -i contrail-install-packages_1.21-74~havana_all.deb 

Now setup the whole packages.

root@SingleNode:/etc/apt# cd /opt/contrail/contrail_packages/
root@SingleNode:/opt/contrail/contrail_packages# ls
contrail_debs.tgz  preferences  setup.sh  sources.list  VERSION
root@SingleNode:/opt/contrail/contrail_packages# ./setup.sh

Copy the example testbed file to testbed.py:

root@SingleNode:/opt/contrail/utils# cp fabfile/testbeds/testbed_singlebox_example.py fabfile/testbeds/testbed.py

Edit this file to, notice the + means we did modifications :

from fabric.api import env

#Management ip addresses of hosts in the cluster
+++ host1 = 'root@127.0.0.1'

#External routers if any
#for eg. 
#ext_routers = [('mx1', '10.204.216.253')]
ext_routers = []

#Autonomous system number
router_asn = 64512

#Host from which the fab commands are triggered to install and provision
+++ host_build = 'root@127.0.0.1'

#Role definition of the hosts.
env.roledefs = {
    'all': [host1],
    'cfgm': [host1],
    'openstack': [host1],
    'control': [host1],
    'compute': [host1],
    'collector': [host1],
    'webui': [host1],
    'database': [host1],
    'build': [host_build],
    'storage-master': [host1],
    'storage-compute': [host1],
}

#Openstack admin password
env.openstack_admin_password = 'secret123'

#Hostnames
env.hostnames = {
+    'all': ['SingleNode']
}

+++ env.password = 'rootpass'
#Passwords of each host
env.passwords = {
+++     host1: 'rootpass',

+++     host_build: 'rootpass',
}

#For reimage purpose
env.ostypes = {
+++    host1:'ubuntu',
}

Now use fab to install :

root@SingleNode:/opt/contrail/utils# fab -c fabrc install_contrail

This will take some minutes.
Then setup all:

# cd /opt/congrail/utils
# fab setup_all

After installation the machine will be reboot.
A snapshot will be created when reboot done, this will be the cloned initial startpoint for OpenContrail.

Start

Create a simple web server via python:

python -m SimpleHTTPServer 8000

Then Create the image in OpenStack via:

Install OpenContrail On Virt-Manager

Recently I am studying OpenContrail(based on OpenStack), so following includes the steps for me to setup a single-node emulation environment, the host machine is ArchLinux, which is one of my favorite Linux Distribution, and using nested virtualization for setting up a basic run-time environment.

libvirt

It seems the most convinient way for using nested virtualization is for using libvirt, so following is for setting up the virtualization environment.
First create libvirt user group and add current user into this group:

# groupadd libvirt
# gpasswd -a Trusty libvirt

Then uncomment following lines in /etc/libvirt/libvirtd.conf:

$ vim /etc/libvirt/libvirtd.conf
unix_sock_group = "libvirt"
unix_sock_ro_perms = "0777"
unix_sock_rw_perms = "0770"
auth_unix_ro = "none"
auth_unix_rw = "none"

Then start and enable the libvirtd.service via following commands:

$ sudo systemctl start libvirtd.service
$ sudo systemctl enable libvirtd.service

From here you could run virt-manager for using virt manager.

Installtion of OpenContrail

First you should download the deb file from juniper, then create a 4G Memory, at least 20G disk space virtual machine, notice your cpu selection should enable nested virtualization.
In Configuration, select “Copy Host CPU Configuration”, then your host machine’s CPU configuration will be delivered to the virtualized machine.
The installation steps are:

$ sudo bash
# passwd
# dpkg -i contrail-install-packages_1.21-74~havana_all.deb
# cd /opt/contrail/contrail_packages
# ./setup.sh
# cd /opt/contrail/
# cp utils/fabfile/testbeds/testbed_singlebox_example.py utils/fabfile/testbeds/testbed.py
# cd /opt/contrail/utils
# fab -c fabrc install_contrail
# fab setup_all

The installation will automatically restart.

OpenStack and OpenContrail

The access webpage are located at:
OpenStack:
http://YourIpAddress/horizon
OpenContrail:
https://YourIpAddress:8143
Simply open a browser and view this.

Nested

To check the nested virtualization, do following operations:

Trusty@Ubuntu1404:~$ cat /sys/module/kvm_intel/parameters/nested 
Y
Trusty@Ubuntu1404:~$ systool -m kvm_intel -v   | grep -i nested
    nested              = "Y"

Switch Back To Arch

Since I Moved the city and changed the job, my laptop had to be returned to company, so I bought another HP 8460P which is almost the same as my old one. But this one got i7 4 core CPU and more powerful.
First I installed Ubuntu and start playing virtualization on it, after successfully installed OpenStack and OpenContrail I started to think change back to my archLinux, so following is the steps.

grub

Use update-grub2 in Install Ubuntu, it will automatically scan all of the disks, cause I installed the archLinux in the second harddisk, it will add a new item in menu.lst, use this item we could reached ArchLinux.

Libvirt

Enable the libvirtd.service via:

sudo systemctl enable libvirtd.service

Then restart the system.

Nested Virtualization

Manually we could do via:

# modprobe -r kvm_intel
# modprobe kvm_intel nested=1

Then check it via:

$ systool -m kvm_intel -v | grep nested
    nested              = "Y"

Add it to automatically loaded kernel module:

$ cat /etc/modprobe.d/modprobe.conf
options kvm_intel nested=1

Then next time it boots up, will enter the nested mode.

Didn’t solved, fuck!!!!!!!

Mount LVM Partitions

The steps is listed as following:

[Trusty@~]$ sudo vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "DashUbuntu-vg" using metadata type lvm2
[Trusty@~]$ sudo vgchange -ay DashUbuntu-vg
  2 logical volume(s) in volume group "DashUbuntu-vg" now active
[Trusty@~]$ sudo lvscan
  ACTIVE            '/dev/DashUbuntu-vg/root' [690.48 GiB] inherit
  ACTIVE            '/dev/DashUbuntu-vg/swap_1' [7.91 GiB] inherit
[Trusty@~]$ sudo mount /dev/DashUbuntu-vg/root /mnt
[Trusty@~]$ ls /mnt
bin   etc         initrd.img.old  lost+found  opt   run   sys  var
boot  home        lib             media       proc  sbin  tmp  vmlinuz
dev   initrd.img  lib64           mnt         root  srv   usr  vmlinuz.old

Libvirt Network Configuration

Since I want to use bridge network in libvirt, while the standard bridge networking is not available for me to use, so following tips is for creating and managing the networking.

Enable Bridge

In host machine(OpenSuse), create bridge via:

$ sudo brctl addbr br0
$ sudo brctl addif br0 eth0 

This will add eth0 to bridge 0, while in startup file of OpenSuse we will do following:

# pwd
/etc/sysconfig/network
# cat ifcfg-eth0 
BOOTPROTO='static'
STARTMODE='ifplugd'
IFPLUGD_PRIORITY='1'
NAME='RTL8111/8168B PCI Express Gigabit Ethernet controller'
USERCONTROL='no' 
# cat ifcfg-br0 
BOOTPROTO='static'
STARTMODE='auto'
IPADDR='192.168.0.xxx/24'
NAME='Bridge For Virtualization'
BRIDGE='yes'
BRIDGE_PORTS='eth0'
BRIDGE_FORWARDDELAY='0'

So next time reboot you will see br0 hold the address of 192.168.0.xxx, this is the bridge port we have.

Virtual Machine Configuration

First list all of the virtual machine we have

$ sudo virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     Ubuntu12043                    running
 -     OpenContrail                   shut off

Edit the one we want to edit via:

$ sudo virsh edit Ubuntu12043
    <interface type='bridge'>
      <mac address='24:42:53:21:52:49'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

So next time when you boot-up the machine, it will have the physical network which connecting to br0.

Configure route

In virtual machine, give the fixed ip address via:

$ sudo ifconfig eth0 192.168.188.188
$ sudo ip route add 192.168.0.0/24 dev eth0
$ sudo route add default gw 192.168.0.xxx
$ sudo route -n
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface                                                                 
0.0.0.0         192.168.0.119   0.0.0.0         UG    0      0        0 eth0                                                                  
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0                                                                  
192.168.188.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0   

We want make it permanent via:
First make the permanent ip address:

# cat /etc/network/interfaces 
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.188.188
        netmask 255.255.255.0

Second we want to configure the route:

$ tail  /etc/rc.local
ip route add 192.168.0.0/24 dev eth0
route add default gw 192.168.0.xxx
echo "nameserver 114.114.114.114">>/etc/resolv.conf

This will set the route and the default gateway of your device, also the nameserver will be set.
But while in the host machine, you will add following iptables rules:

# ip route add 192.168.188.0/24 dev br0
# iptables -t nat -A POSTROUTING -s 192.168.188.0/24 -j SNAT --to 192.168.0.xxx
# iptables -A FORWARD -s 192.168.188.188 -j ACCEPT

The iptables rules should be saved via iptables-save, while the route rules should be added into following file:

$  cat /etc/rc.d/after.local 
ip route add 192.168.188.0/24 dev br0                

Now reboot the host machine, you route configuration will be saved.

Libvirt On OpenSuse

In order to try OpenContrail, I installed it on Virtualbox based, but it got failed, I think maybe it’s because virtualbox’s nested virtualization is not OK. So that’s why I have to try libvirt.

Installation

Install the virt-manager related software via:

$ sudo zypper install kvm libvirt libvirt-python qemu virt-manager
$ sudo zypper in patterns-openSUSE-kvm_server

Now if you directly call virt-manager you will got the following hint:

 error: authentication failed: Authorization requires authentication but no agent is available.
    error: failed to connect to the hypervisor

Or if you run virsh -c qemu:///system you will got the same hint message.
Solution is we have to enable the polkit service, add following script:

$ sudo vim /usr/bin/polkitexec
#! /bin/bash

prg=$1

[ "$prg" ] || exec echo "missing argument"
[ -x "$prg" ] || which $prg &>/dev/null || exec echo "program not found: $1"

shift
args=$*

[ `id -u` -eq 0 ] && exec $prg $args

uname -m | grep -q 64 && lib=/usr/lib64 || lib=/usr/lib

if [ "$DESKTOP_SESSION" = "kde" ]; then 
        pga=polkit-kde-authentication-agent-1
        lpga=/usr/$lib/kde4/libexec/$pka
else
        pga=polkit-gnome-authentication-agent-1
        lpga=/usr/lib/$pga
fi

case $DESKTOP_SESSION in
        gnome|gnome-shell|cinnamon*) exec $prg $args ;;
        *) ps nc -C $pga &>/dev/null && exec $prg $args || { [ ! -x $lpga ] && exec echo "program not found: $lpga" || $lpga & $prg $args ;} ;;
esac
$ sudo chmod 777 /sur/bin/polkitexec
$ vim ~/.zshrc
# Wrap virt-manager with polkitexec. 
alias virt-manager='polkitexec virt-manager'

Now the virt-manager could be run correctly.

Trouble-Shooting

If you met following error, add root to your kvm.conf is OK.

Could not access KVM kernel module: Permission denied
failed to initialize KVM: Permission denied
No accelerator found!

First change the priviledge:

#chown root:kvm /dev/kvm

Then modify the /etc/libvirt/qemu.conf:

#user="root"
user="root"
#group="root"
group="root"

Restart the libvirtd and restart the virt-manager:

#service libvirtd restart

Now continue to install OpenContrail.