I know a lot of folks are using the StackOps script thingy to install OpenStack. I’ve been installing it (quite a bit) lately just from packages, and it’s not all that difficult, so I thought I’d write up the details on how to do that. A lot of this is exactly what’s encoded into Chef recipes and Puppet modules out there – so if you’re looking to run with something already made, there’s plenty of options.

These instructions are assuming you’re starting with an Ubuntu based system – either 10.10 or 11.04. I haven’t tried it as yet with 11.10.

First things first, I recommend you make sure you have the latest bits of everything:

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get autoremove

Then we need to add the release “PPA” so that your system can grab the packages for Openstack:

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:openstack-release/2011.3
sudo apt-get update

Now we get into the details. I’m going to drive out the instructions that will start with a single host, but are set up to add additional virtualization hosts as you need. I’m writing this assuming you’re working in a small network, and setting it up for FlatDHCP networking. Choosing the networking strategy and IP address space to use is actually one of the trickier parts of doing a reasonable install. For just testing something out in a test lab, this setup will work reasonable well – the only thing to really note is that this *will* install a DHCP server to provide IP addresses to the virtual instances, so if you have another DHCP server handing out addresses, you might need to get into the details and change some of these settings.

Installing the packages:

OpenStack relies on using MySQL as a data repository for information about the openstack configuration, so we’ll need to set up a MySQL server. Normally when you install the packages for MySQL, it’ll ask you about configuring a root password and such. We can make that hands-off by pre-answering some of those questions. To do this, make a file named “/tmp/mysql_preseed.txt” and put in it the following:

mysql-server-5.1 mysql-server/root_password password openstack
mysql-server-5.1 mysql-server/root_password_again password openstack
mysql-server-5.1 mysql-server/start_on_boot boolean true

Then we can get into the commands to install the packages:

cat /tmp/mysql_preseed.txt | debconf-set-selections
apt-get install mysql-server python-mysqldb
apt-get install rabbitmq-server
# ^^ pre-reqs for running controller nova instance
apt-get install euca2ools unzip
# ^^ for accessing nova through EC2 APIs
apt-get install nova-volume nova-vncproxy nova-api nova-ajax-console-proxy
apt-get install nova-doc nova-scheduler nova-objectstore
apt-get install nova-network nova-compute
apt-get install glance

That’s got all the packages installed onto your local system! Now we just need to configure it up and initialize some information (that’s the bit about networks, etc).

Before I get into changing configs, let me explain what I’ll be setting up. In this example, my internal “network” is 172.17.0.0/24 – and I have a dedicated IP address for this host that is 172.17.0.133. The virtual machines will be in their own network space (10.0.0.0 to 10.0.0.254), and (at this point) not visible from the local network, but will be able to access the local network through their virtualization hosts. The machine I’m using also only has a single NIC (eth0), which is fine for a little test bed, but not likely what you want to do in any sort of real setup.

/etc/nova/nova.conf

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--flagfile=/etc/nova/nova-compute.conf
--verbose
#
--sql_connection=mysql://novadbuser:novaDBsekret@172.17.0.133/nova
#
--network_manager=nova.network.manager.FlatDHCPManager
--flat_network_bridge=br100
--flat_injected=False
--flat_interface=eth0
--public_interface=eth0
#
--vncproxy_url=http://172.17.0.133:6080
--daemonize=1
--rabbit_host=172.17.0.133
--osapi_host=172.17.0.133
--ec2_host=172.17.0.133
--image_service=nova.image.glance.GlanceImageService
--glance_api_servers=172.17.0.133:9292
--use_syslog

Now you might have noticed the MySQL connection string in there. We need to set up that user and password in MySQL to do what needs to be done. I also change the MySQL configuration so that remote systems can connect to MySQL. It’s not needed on a single host, but if you ever want to have more than one compute host, you need to make this change. In /etc/mysql/my.conf, find the line:

bind-address = 127.0.0.1

and change it to

bind-address 0.0.0.0

Now lets make the user in Mysql:

mysql -popenstack
CREATE USER 'novadbuser' IDENTIFIED BY 'novaDBsekret';
GRANT ALL PRIVILEGES ON *.* TO 'novadbuser'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;

And set up the database:

mysql -popenstack -e 'CREATE DATABASE nova;'
nova-manage db sync

If that last command gives you any trouble, then we likely don’t have the MySQL system configured correctly – the user can’t access the tables or something. Check in the logs for MySQL to get a sense of what might have gone wrong.

At this point, it’s time to configure up the internals of openstack – create projects, networks, etc.
We’ll start by creating an admin user:

# create admin user called "cloudroot"
nova-manage user admin --name=cloudroot --secret=sekret

This should respond with something like:

export EC2_ACCESS_KEY=sekret
export EC2_SECRET_KEY=653f3fad-df22-449b-9e6a-ea6c81e32621

You can scratch that down, but we’ll be getting that same information again later and using it, so don’t worry too much about it.

Now we create a project:

# create project "cloudproject" with project mgr: "cloudroot"
nova-manage project create --project=cloudproject --user=cloudroot

And finally, a network configuration for those internal IP addresses:

nova-manage network create private \
    --fixed_range_v4=10.0.0.0/24 \
    --num_networks=1 \
    --network_size=256 \
    --bridge=br100 \
    --bridge_interface=eth0 \
    --multi_host=T
# gateway assumed at 10.0.0.1
# broadcast assumed at 10.0.0.255

Now I’m using the multi-host flag, which is new in the Diablo release. This makes each compute node it’s own networking host for the purposes of allowing the VM’s you spin up to access your network or the internet.

At this point, you’re system should be up and running, all systems operational. Let me walk you through the command steps to actually kick up a little test VM though. These commands are all meant to be done as a local user (not root!)

sudo nova-manage project zipfile cloudproject cloudroot /tmp/nova.zip
unzip -o /tmp/nova.zip -d ~/creds
cat creds/novarc >> ~/.bashrc
source creds/novarc
#
euca-add-keypair mykey > mykey.priv
chmod 600 mykey.priv
#
image="ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz"
wget http://smoser.brickies.net/ubuntu/ttylinux-uec/$image
uec-publish-tarball $image mybucket
#
wget http://uec-images.ubuntu.com/releases/10.04/release/ubuntu-10.04-server-uec-amd64.tar.gz
uec-publish-tarball ubuntu-10.04-server-uec-amd64.tar.gz mybucket
...OUTPUT...
Thu Aug 18 14:02:20 PDT 2011: ====== extracting image ======
Warning: no ramdisk found, assuming '--ramdisk none'
kernel : lucid-server-uec-amd64-vmlinuz-virtual
ramdisk: none
image  : lucid-server-uec-amd64.img
Thu Aug 18 14:02:29 PDT 2011: ====== bundle/upload kernel ======
Thu Aug 18 14:02:34 PDT 2011: ====== bundle/upload image ======
Thu Aug 18 14:03:12 PDT 2011: ====== done ======
emi="ami-00000002"; eri="none"; eki="aki-00000001";
...OUTPUT...

And running the instances:

euca-run-instances ami-00000002 -k mykey -t m1.large
...OUTPUT...
RESERVATION r-1jj2a80v  cloudproject    default
INSTANCE    i-00000001  ami-00000002            scheduling  mykey (cloudproject, None)  0       m1.tiny2011-08-18T21:06:03Z unknown zone    aki-00000001    ami-00000000
...OUTPUT...
#
euca-describe-instances 
...OUTPUT...
RESERVATION r-1jj2a80v  cloudproject    default
INSTANCE    i-00000001  ami-00000002    10.0.0.2    10.0.0.2    building    mykey (cloudproject, SIX)   0   m1.tiny 2011-08-18T21:06:03Z    nova    aki-00000001    ami-00000000
...OUTPUT...
#
euca-describe-instances 
...OUTPUT...
RESERVATION r-1jj2a80v  cloudproject    default
INSTANCE    i-00000001  ami-00000002    10.0.0.2    10.0.0.2    running mykey (cloudproject, SIX)   0       m1.tiny 2011-08-18T21:06:03Z    nova    aki-00000001    ami-00000000
...OUTPUT...
#
euca-authorize -P tcp -p 22 default
ssh -i mykey.priv root@10.0.0.2

To add on additional hosts to support more VMs, you only need to install a few of the packages:

apt-get install nova-compute nova-network nova-api

You do need that exact same /etc/nova/nova.conf file though.

Note:
The default install of Glance expects the images that you’ve loaded up to be available on the local file system for every compute node at /var/lib/glance. Either NFS mount this directory from a central machine, or replicate the files underneath it to all your “compute hosts” when you upload a new image to be used in the virtual machines.

Also, the metadata URL needed for UEC images (169.154.169.154) may need help getting forwarded when running on a system with a single NIC. Two potential solutions: A) run nova-api on each of the compute nodes (quick and dirty) or B) specify the –ec2_dmz_host=$HOSTIP, and potentially invoke the command ip link set dev br100 promisc on to turn on promiscuous mode (per https://answers.launchpad.net/nova/+question/152528).

In the openstack project, we’re nearing the “Diablo Milestone”. To a large respect, it’s the fourth release of OpenStack. Even as we close down on the tail end of this release, there has been and is a huge amount of movement in the project.

We have Quantum and Glance shifting to use Github as a repository, new processes (that would be using Gerrit) wrapped around GitHub to allow the project to have a “gated trunk” methodology, and lots of “motion” within the various projects. It’s pretty easy to see on Github, a little harder with launchpad (I just don’t have the tools handy to create the pretty graphs) – you can see the impact graphs for swift, keystone, glance, quantum, and openstack-dashbaord to see what I mean.

The shifting to Gerrit hasn’t been without it’s trials, but is coming along pretty well now. I really wish the GitHub folks had been a bit more amenable to putting in a field that external folks could use to store metadata about a pull request. Several folks from the OpenStack project (including myself) reached out to them about this, all rebuffed (nicely, but still). In fact, one of the suggestions I got back from the github’r support was “Why don’t you set up Gerrit?”

With the changes in core repository, lots of dependencies are shifting as well. Dashboard was broken a bit this week we kicked things around to get the dependencies to match the new locations, I think we’ve got all those pieces worked around now (pull request outstanding for openstack-dashboard). The other piece that really shifted and broke with these changes were the install scripts that we’ve been using to build and work on a developer’s environment. The cloudbuilder team at Rackspace recently created a whole new setup that works very nicely, so I think we’re going to drop our older scripts (based on over-extended versions of Vish’s excellent nova.sh script) and move to using their new “stack.sh“. (And yeah, of course we’ll want to mess with it ourselves, so I’ve forked it…)