I’m a huge fan of CloudEnvy, so I’ve created a public github repository with the envy configuration and setup scripts to be able to set up a VM and completely build out all the existing documentation in roughly 20-25 minutes.

First, we install cloudenvy. It’s a python module, so it’s really easy to install with pip. My recommended installation process:

pip install -U cloudenvy

If you’re working on a mac laptop (like I do), you may need to use

sudo pip install -U cloudenvy

Once cloudenvy is installed, you need to set up the credentials to your handy-dandy local OpenStack cloud (y’all have one of those, don’t you?). For cloudenvy, you create a file in your home directory named .cloudenvy akin to this:

cloudenvy:
  clouds:
    cloud01:
      os_username: username
      os_password: password
      os_tenant_name: tenant_name
      os_auth_url: http://keystone.example.com:5000/v2.0/

Obviously, put in the proper values for your cloud.

Now you just need to clone the doctools envyfile setup, switch to that directory, and kick off Envy!

git clone https://github.com/heckj/envyfile-openstack-docs.git
cd envyfile-openstack-docs
envy up

20-25 minutes later, you’ll have a virtual machine running with all the tooling installed, run-through, and the output generated for all the documentation in the openstack manuals repository. The envyfile puts all this into your virtual machine at

~/src/openstack-manuals

To get there, you can use the command envy ssh to connect to the machine and do what you need.

For more on the how-to with contributing to OpenStack documentation, check out the wiki page https://wiki.openstack.org/wiki/Documentation/HowTo.

Years later in college, I learned that the window trinket was a classic science experiment regarding photons having mass. I saw one on think-geek some time ago, and got one for my house:

Looking at what glanceclient did, and had to do, I got really annoyed and wanted the client to have a much easier to use interface. At the same time, I was also digging around trying to allow the keystoneclient CLI to accept and use an override for the endpoint from the command line. Turns out the various mechanations to make the original client setup work with a system with two distinct URL endpoints was quite a mess under the covers, and that mess just propagated through to anyone trying to use the library.

We just landed some new code updates with keystoneclient to make it much easier to use. So this little article is intended to be a quick guide to using the python keystoneclient library and some of it’s new features. While we’re getting v3 API support installed, we’re still very actively using v2 apis, so we’ll use v2 API examples throughout.

The first is just getting a client object established.


>>> from keystoneclient.v2_0 import client
>>> help(client)


We’ve expanded the documentation extensively to make it easier to use the library. The base client is still working from httplib2 – I didn’t rage-change it into the requests library (although it was damned close).

There’s a couple of common things that you’ll want to do with initializing the client. The first is to authorize the client with the bootstrapping pieces so you can use it to configure keystone. In general, I’m sort of expecting this to be done mostly from the CLI, but you can also do it from the python code directly. To use this setup, you’ll need to initialize the client with two pieces of data:

• token
• endpoint

Token is what you’ll have configured in your keystone.conf file under admin_token and endpoint is the URL to your keystone service. If you were using devstack, it would be http://localhost:35357/v2.0

A bit of example code (making up the admin_token)

from keystone client.v2_0 import client

adminclient = client.Client(token='9fc31e32f61e78f114a40999fbf594c2',
                            endpoint='http://localhost:35357/v2.0')

Now at this point, you’ll have an instance of the client, and can start interacting with all the internal structures in keystone. adminclient.tenants.list() for example.

You may have spotted the authenticate() method on the client. If you’re using the token/endpoint setup, you do not want to call this method. When you’re using the admin_token setup, you don’t
have a full authorization token as retrieved from keystone, you’re short-cutting the system. This mode is really only intended to be used to bootstrap in projects, users, etc. Once you’ve done that, you’re better using the username/password setup with the client.

To do that, you minimally need to know the username, the password, and the “public” endpoint of Keystone. With the v2 API, the public and administrative endpoints are separate. With devstack, the example public API endpoint is http://localhost:5000/v2.0.

A bit of an example:

from keystoneclient.v2_0 import client
kc = client.Client(username='heckj', password='e2112EFFd3ff',
                   auth_url='http://localhost:5000/v2.0')

At this point, the client has been initialized, and as a default it will immediately attempt to authenticate() to the endpoint, so it already has some authorization data. With the updated keystoneclient library, this authorization info is stashed into an attribute “auth_ref”. You can check out the code in more detail – the class is keystoneclient.access.AccessInfo, and this represents the token that we retrieved after calling authenticate() to auth against keystone.

With only providing a username and password, the token is really only useful for about two things – getting a list of clients that this user can authorize to (getting a ‘scoped’ token – where the token represents authorization to a project), and then retrieving that token.

>>> kc.username
'heckj'
>>> kc.auth_ref
{u'token': {u'expires': u'2012-11-12T23:28:58Z', u'id': u'97913f8839634946afab2897ac19908d'}, u'serviceCatalog': {}, u'user': {u'username': u'heckj', u'roles_links': [], u'id': u'c8d112a0932a454097dfba0f3b598bdc', u'roles': [], u'name': u'heckj'}}
>>> kc.auth_ref.scoped
False
>>> kc.tenants.list()
[<Tenant {u'id': u'7dbf826d086c4580a28cf860a6d13046', u'enabled': True, u'description': u'', u'name': u'heckj-project'}>]
>>> kc.authenticate(tenant_name='heckj-project')
True
>>> kc.auth_ref.scoped
True
>>> kc.auth_ref
{u'token': {u'expires': u'2012-11-12T23:37:10Z', u'id': u'6d811d7c39034813b6cab2ad083cdf3e', u'tenant': {u'id': u'7dbf826d086c4580a28cf860a6d13046', u'enabled': True, u'description': u'', u'name': u'heckj-project'}}, u'serviceCatalog': [{u'endpoints_links': [], u'endpoints': [{u'adminURL': u'http://localhost:8776/v1/7dbf826d086c4580a28cf860a6d13046', u'region': u'RegionOne', u'internalURL': u'http://localhost:8776/v1/7dbf826d086c4580a28cf860a6d13046', u'publicURL': u'http://localhost:8776/v1/7dbf826d086c4580a28cf860a6d13046'}], u'type': u'volume', u'name': u'Volume Service'}, {u'endpoints_links': [], u'endpoints': [{u'adminURL': u'http://localhost:9292/v1', u'region': u'RegionOne', u'internalURL': u'http://localhost:9292/v1', u'publicURL': u'http://localhost:9292/v1'}], u'type': u'image', u'name': u'Image Service'}, {u'endpoints_links': [], u'endpoints': [{u'adminURL': u'http://localhost:8774/v2/7dbf826d086c4580a28cf860a6d13046', u'region': u'RegionOne', u'internalURL': u'http://localhost:8774/v2/7dbf826d086c4580a28cf860a6d13046', u'publicURL': u'http://localhost:8774/v2/7dbf826d086c4580a28cf860a6d13046'}], u'type': u'compute', u'name': u'Compute Service'}, {u'endpoints_links': [], u'endpoints': [{u'adminURL': u'http://localhost:8773/services/Admin', u'region': u'RegionOne', u'internalURL': u'http://localhost:8773/services/Cloud', u'publicURL': u'http://localhost:8773/services/Cloud'}], u'type': u'ec2', u'name': u'EC2 Service'}, {u'endpoints_links': [], u'endpoints': [{u'adminURL': u'http://localhost:35357/v2.0', u'region': u'RegionOne', u'internalURL': u'http://localhost:5000/v2.0', u'publicURL': u'http://localhost:5000/v2.0'}], u'type': u'identity', u'name': u'Identity Service'}], u'user': {u'username': u'heckj', u'roles_links': [], u'id': u'c8d112a0932a454097dfba0f3b598bdc', u'roles': [{u'name': u'Member'}], u'name': u'heckj'}, u'metadata': {u'is_admin': 0, u'roles': [u'08ccc339c0074a548104b9050bdf9492']}}

You might have noticed that you can now call authenticate() on the client and just pass in values that are missing from previous authenticate() calls, or you can switch them out entirely. You can change the username, password, project, etc – anything that you’d otherwise normally initialize with the client to do what you need.

Here is an overview of what’s looking to happen in Keystone over the grizzly release cycle.

From the summit, we had the state of the project slides, which might be of interest: http://www.slideshare.net/ccjoe/oct-2012-state-of-project-keystone

Since then, we’ve been working on fleshing out more details around those initial discussions, and we’ve been correlating who’s working on what to get an overview of what’s coming up for Keystone. If you’re into reading raw notes, take a look at https://etherpad.openstack.org/keystone-grizzly-plans.

For those looking for more of a tl;dr:

grizzly-1 plans:
* merging in V3 API work – “tech preview”
https://blueprints.launchpad.net/keystone/+spec/implement-v3-core-api

* move auth_token middleware to keystoneclient repo
https://blueprints.launchpad.net/keystone/+spec/authtoken-to-keystoneclient-repo

* AD LDAP extensions
https://blueprints.launchpad.net/keystone/+spec/ad-ldap-identity-backend

* enabling policy & RBAC access for V3 API
https://blueprints.launchpad.net/keystone/+spec/rbac-keystone-api

grizzly-2 plans:
* pre-authenticated token
https://blueprints.launchpad.net/keystone/+spec/pre-auth

* plugable authentication handlers
https://blueprints.launchpad.net/keystone/+spec/pluggable-identity-authentication-handlers

* consolidated policy documentation/recommendations
https://blueprints.launchpad.net/keystone/+spec/document-deployment-suggestions-policy

* PKI future work
https://blueprints.launchpad.net/keystone/+spec/delegation
– starting into delegation, signing of tokens
– annotations on signing for authorization

grizzly-3 plans:
* delegation
https://blueprints.launchpad.net/keystone/+spec/delegation

* multifactor authN
https://blueprints.launchpad.net/keystone/+spec/multi-factor-authn

Much of the work and desires around Delegation has yet to be fully defined and nailed down, and relies on a lot of additions in making PKI based tokens a stable, solid, default mechanism. I’m sure there will be some redirection once we get a few weeks down the road and see what’s happening with the V3 API rollout and PKI token extensions to support delegation, pre-auth, and so forth.

CloudEnvy (https://github.com/cloudenvy/cloudenvy) is what started that cascade. Brian Waldon started it some time ago as a script that emulated the ease of spinning up VM’s with vagrant, except wrapped over the OpenStack clients. I always wanted to like Vagrant, but it never really synced for me. I think mostly because I was in never-ending kernel panic hell with virtualbox. CloudEnvy is a different story.

I think the most interesting illustration of CloudEnvy is using it to spin up an instance in a cloud, and then run devstack in that instance.

CloudEnvy relies on a starter of two files – one that’s specific to the project (DevStack in this case): Envyfile, and one for your personal cloud configuration (~/.cloudenvy).

Here’s my .cloudenvy file (with the hostname and password redacted):

And the Envyfile I use with devstack:

You’ll notice that the Envyfile references a script I named “cloudenvy-setup.sh” – this is the basic script that cloudenvy uploads to the instance it creates to automatically provision things up. You could easily replace this with Puppet, Chef, or whatever it is you like to configure VMs in your world.

Here’s what I’m doing:

(all three of these files are in the gist https://gist.github.com/3969250)

The Envyfile also refers to a image_name. I’m using a stock UEC precise image that I uploaded to our instance of OpenStack. Pretty shortly, CloudEnvy will be replacing “image_name” with just “image”, and they recommend that you use an image ID (guaranteed uniqueness) over a name. For my immediate use, the name works pretty well.

Once this is all in place:


envy up


Creates the instance, assigns it a floating IP address, SSH’s into the instance, uploads the provision script, and starts cranking on the provisioning. 763 seconds later, a fully operational devstack in an instance running on OpenStack.


envy ssh


Gets you in, lets you do what you want.


envy list


Shows you the instance(s) you have running.

There’s more, a lot more, but hopefully this is sufficient to get you started.

The design sessions are definitely less about actually hacking code than they were a year ago, offset though with the increasing diversity of backgrounds and interests participating in the sessions. The core team developers joined me through-out Thursday and drove most of the discussions, with fantastic input from David Chadwick, Khaja, Ryan Lane. There were way more people in the sessions than that, but to me these three represent a set of fresh inputs from folks with a deep academic background, previous experience building identity systems, and active operator points of view. They and the the previous contributors provided tremendous feedback, asked great questions, and set the stage for a lot of interesting ideas.

This year all the project technical leads gave a “state of the project” overview, but we did that on Tuesday – so like John Griffith (the project technical lead for cinder), I was doing the ‘state of the project’ routine prior to getting the feedback and doing the brainstorming in the sessions. The slides from that presentation are online at http://www.slideshare.net/ccjoe/oct-2012-state-of-project-keystone if you’re interested. The coordinators all video-taped those segments, as I understand it, they should be appearing on the OpenStack channel in Youtube in the next couple of days.

There was also a very active session led by Gabriel Hurley seeking to drive more continuity into the OpenStack APIs, and a matching session by Doug Hellman and Dean Troyer for the OpenStack CLIs. The continued focus on bringing in new ideas while keeping the interfaces consistent and clear is a great sign for the project overall, and I was pleased to see a large number of like minded folks wanting to continue to move things forward in those areas.

This was also the first summit under the auspices of the OpenStack Foundation – they all met for an extended period of time early in the conference, and the Technical Committee managed to pull of a first all-in-person meeting over dinner and scattered conversation Tuesday evening.

And not at all related to any core projects or overall effort: check out the very creative riff on the OpenStack theme Dope’n'Stack, (video on YouTube as well). Gabriel and Erik were working their tails off prior to and during the conference to pull this off, culminating in a great presentation Wednesday evening at Piston’s party. (I was disappointed that Gabriel lost the mohawk for the summit, but he said he was sick of wearing it after three weeks).

We rolled RC1 for Keystone Folsom release out the door this past week, and at this point I think all the projects have an initial release candidate out the door. The original release date is 5 days away, and it’s looking pretty good for hitting it. If you want a quick overview of what’s coming in this release, I’d recommend a look at Emilien Macchi’s Folsom overview, which is a pretty nice high level summary.

While we’ve been busy nailing down bugs and wrapping this release together, the OpenStack Foundation has finally come into form. As the Keystone project technical lead, I’m on the OpenStack technical committee – picture and title, but I haven’t written a bio yet. (Sorry Lauren). I find it really quite difficult to write a bio at myself. Regardless, it’s great to finally see this moving into a foundation external to any single corporate interest. That’s not to say it’s all in the land of milk and cookies, there’s just a lot of people with all slightly different interests jumping into the pool to push this little rowboat in different directions.

One thing that we did early out of excellent foresight was keeping the direction of the core projects democratically oriented by the contributors to those projects. The people that show up to write, update, and support the code are the ones that are ultimately making the decisions on what features get implemented, and when. Lots of folks talk about what could be, but it’s the contributors that make it happen.

A perfect example of this is Adam Young, who in the past 6 months drove the implementation of PKI based tokens in Keystone. I’m not even sure I’ve met Adam face to face, but I definitely know him – and he’s been a fantastic contributor, and in the Folsom development cycle was promoted to the core Keystone team.

View from the ISS at Night from Knate Myers on Vimeo.

So then I caught word about one of my longest running obsessions/hobbies/collections/etc – Traveller. Mark Miller is hosting a crowd-sourcing setup on KickStarter to fund the release of Traveller 5. I saw it, was tempted, and then just a day or so ago decided to dive in and help back it. I have pieces of most every edition of traveller that ever came out, a huge collection of little black books, 8.5×11 paperbacks and a few hardcovers. Side games, spinoffs, and other do-dads. “Back in the day”, I was actively involved in a mailing list run on BITNET that we just called the “TML”. From there, and other places, I collected a fairly large archive of fan works and related pieces, a few bits of software, all sorts of things. I published them in one of my first web-based sites. I don’t recall what I called it early on, but now I just call it the “Missouri Archive” and even though I haven’t updated it or really touched the collection in 10 years, I keep it online with the help of a few friends. It’s still sticking around out there at http://traveller.mu.org/.

I don’t even want to think about how long that game has managed to capture my attention, how many days I spent day-dreaming about space operas, designing “starships” for the game, or generally just reading published games or stories related to it. I’ve enjoyed that setting and that game longer than any other I can image (far longer D&D as it turns out), so I’m very happy to help fund Traveller 5 and look forward to it shipping, and more evenings of reading and daydreaming.

I do still hope for some video or compute games that capture some of Traveller. TravellerAR was a long shot – of course I spent my money it too, but they’ve had to close up shop and I never really got to anything with it that I found fun or good (it crashed quite a bit). Sooner or later, I guess.

To say it’s been a busy six months since the last post is really quite an understatement. What we released in the past month, and the work that went in to getting us there, is nothing short of phenomenal.

The OpenStack Essex release is out, and I’m still sort of catching my breath from that. Just six withs prior to the release I was elected as the PTL for Keystone. I had no idea how much additional work it was to wrangle the bugs, approvals, and the other sundry details that go into getting the release out the door. I rather wish there was a “new PTL user’s manual” for the process. It was a lot of learning, most of it last-minute and ad-hoc around our project management, release process and associated the mechanics. I spent a lot of time trying to figure out how to wrangle Launchpad into some semblance of useful tracking, etc. (To be fair, it was mostly a matter of learning what Launchpad could and couldn’t do, and figuring out the conventions that were already in place).

The past month was terrifically busy with getting OpenStack deployed and tested – running in small environments (devstack), and large. I’ve been working from the Ubuntu distributions myself, and the whole gang that has been packaging OpenStack (Chuck, Adam, Kiall, and others – inside Canonical and out) with the downstream distributions has done a tremendous job wrapping together the changes and making it into a deployable release from packages.

A much better release:

Where I was, er, more than disappointed with the Diablo, I’m significantly happier with the Essex release. Keystone (the project for which I am now the PTL) didn’t advance this release cycle so much as retrench and prepare for advancement this next release cycle. Horizon made huge leaps and bounds, which I think is fantastic as the face of OpenStack to many users. Nova and Swift advanced, and the integration work and definition going into making Quantum (and Melange) a reality has been terrific.

I think if I were to pull out a star of the show for the Essex release, then I’ve really got to point to the combined work for the crew formulating and keeping DevStack solid, and the CI team that integrated it into our development and review process so that we had a minimum of guaranteed interoperability. Where I was screaming into the wind during the last milestone of Diablo for continued breakages between the components, the integration of devstack has demanded that changes be rolled in with though to an overall use case and interop. With 200+ developers all kicking this ball down the field, the guaranteed CI has been the piece that has done us the most good.

A little about Keystone:

Keystone, as we retrenched and simplified the codebase, also has a significant advancement in integration testing. We replaced the entire codebase based on a series of integration tests that verified and guaranteed API compatibility with the Diablo and trunk releases as well as the client while we made that switch. The underlying code base is now significantly simplified, and the internal architecture of that service now wrapped around some core “internal service” concepts to allow us to have drivers that cleanly back-end into external systems to support identity and authorization.

I’ve received a number of questions about “why the API still sucks” six months later after Diablo. There’s still no obvious means for a user to “log out” (invalidate their own token), or change their own password (assuming the back-end were to support it). In switching out the underlying code base and architecture we needed to keep the API stable so that we could make sure we had the internals correct. With the internals switched out, it is now time to revisit the API and take the lessons we’ve learned from 6+ months of using the v2.0 API and improve upon it.

Looking towards the summit:

While I’m much, much happier with the Essex release, there are still plenty of places for improvement – many that I remember from the Diablo summit, and some new things that I’d love to see some focus on for the upcoming six months.

While Horizon has given us a dramatically improved user experience for OpenStack, there’s much more that we could do there – both with a web based UI, and with our command line interfaces to OpenStack. One thing that could use some explicit attention is the proliferation of “clients” needed to interact with OpenStack. As projects are splintering off Nova into their own domains, we have a number of new command line clients (nova, keystone, glance, quantum) – and they don’t all act the same. There is some great work on driving them to consistency, but I wonder if we shouldn’t bag all the individual clients and roll them together into one “OpenStack” client that is consistent in how it handles command line options, what the “commands” look like, and generally how you interact with them.

I hate to admit this, but even as the PTL of Keystone, I ran into a brick wall of old docs referring to using the command of “nova-manage project list” to see the projects, when I knew darn well and good they were all in the keystone system, and to see a list of them I should use the command “keystone tenant-list“. Not so hot, huh? Quantum and the nova-network components are going to really come into their own in this release, so we’ll have more shattering of the user experience from a deployer’s point of view unless we do something to be specific about tying it together.

There are more changes to be made in Keystone that I’m paying a lot of attention to – some of which are honestly going to require buy-in from the entire community to make happen. Where we place the boundaries of role based access, and how we deal with trust and information sharing about identity between the projects is probably going to need some changes that will ripple all the way down into the API’s of nova, swift, glance, quantum, melange, etc. Getting the brainstorming around those concepts and desired features is what I primarily want to accomplish at the Design Summit, with follow on in the milestone timeframe thereafter to bring the new features into existence and integrate them across OpenStack. My ideal goal is to get all the heavy features implemented early in the Folsom release cycle so that they’re solid and nailed down for the later milestones, and we can tweak and fix what may be broken long before any release points.

About that CloudStack thing:

Seems like in the last week, everybody (or at least the pundits) got their knickers in a twist about Citrix finally being more open about their dual-interest in CloudStack and OpenStack. Regardless of Gartner pundit sensationalistic babbling, the release of CloudStack as an Apache licensed project is nothing but goodness for the whole community. The Apache license makes me feel more comfortable with reviewing their code and looking at how they attacked the same issues and problems OpenStack has, and I expect they’ll look closely at how OpenStack has done the same. I applaud any company trying to build an open-source community, and I’m looking forward to seeing what Citrix does to actually do that community building. If it gains ground and folks contribute to CloudStack, we’re all better off – it’s more ideas hitting the street and becoming reality.

Finally, for anyone that didn’t think that the user-facing EC2 API was the defacto API, wake the F* up. It is reasonable, solid, and nobody is going to be turning it off any time soon. It also doesn’t allow/enable some things that a lot of cloud administrators would like to do – so it shouldn’t surprise anyone that CloudStack has their “api” and OpenStack has their “api”. The APIs are another place where we can learn from each other, as we add some value beyond what the bookseller down street might want to impose and expose.

If you want a little pundit action from this source, watch the API’s of both CloudStack and OpenStack (not just the EC2 compatibility layer) over the next six months. I think you’ll be seeing some very interesting steps forward.

One thing that was intended as new, and I think worked well, was the splitting of the conference and design summit. It really made it a lot easier to be involved in the technical details and not have to choose between that and listening to folks talk to what they’re Openstack experiences have been, governance models and choices, and so forth. As an active contributor to (and general jack-of-all-trades with) a number of the OpenStack projects, it’s hard enough to balance my focus between the technical sessions.

There was also a ton of news related to OpenStack that is just fantastic for the community. Rackspace is starting to form out a foundation to run OpenStack long term, and the governance round-table at the conference made it clear that pretty much everyone agreed to a thoughtful, careful process to set it up to really make a long term run with OpenStack. There was notable consensus on ideas and vision of how to set up governance, including taking some long and deep looks at how other successful open source foundations have been set up, and looking to those foundations to learn what’s been successful, and what they might have done differently.

There was also the entrance of Hewlett Packard very publicly actively into the OpenStack world. Back in September, they said they’d be there, and with the conference they have committed hardware and time to continuous integration efforts. They were notably involved in the design summit as well, hosting a couple of sessions and getting actively involved in others. And I think most pleasing to me, they are actively submitting code contributions to the OpenStack core projects. I think it’s a little odd that they’re all coming in as “HP Nova Contributors“, but I’m glad to see it.

With HP running their cloud offering on OpenStack, they’re getting deeper in the details of the code. Dell’s been focusing their efforts on the amazing OpenStack deployment tool: Crowbar, and it seems clear to me that HP is taking their interest quite a bit deeper into the core projects.

I won’t even do justice the the individual project roadmaps from the summit – I think that the most of the project technical leads are likely recovering this weekend from an intensely focused week. In the next couple of weeks, we’ll see them hit the mailing list – and from what I saw from the sessions in the conference, there’s going to be some hard choices coming up on where to expend resources. Every one of the projects has more that they want to do than they will be able to smoothly accomplish in the next 6 months. We’ll see the details hitting the blueprints in the very near future I’m sure. Suffice to say that if you’ve been thinking about getting involved in OpenStack, now is very definitely the time to jump in – there’s a ton to do, and a lot of really interesting problems to solve in getting it done.

These instructions are assuming you’re starting with an Ubuntu based system – either 10.10 or 11.04. I haven’t tried it as yet with 11.10.

First things first, I recommend you make sure you have the latest bits of everything:

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get autoremove

Then we need to add the release “PPA” so that your system can grab the packages for Openstack:

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:openstack-release/2011.3
sudo apt-get update

Now we get into the details. I’m going to drive out the instructions that will start with a single host, but are set up to add additional virtualization hosts as you need. I’m writing this assuming you’re working in a small network, and setting it up for FlatDHCP networking. Choosing the networking strategy and IP address space to use is actually one of the trickier parts of doing a reasonable install. For just testing something out in a test lab, this setup will work reasonable well – the only thing to really note is that this *will* install a DHCP server to provide IP addresses to the virtual instances, so if you have another DHCP server handing out addresses, you might need to get into the details and change some of these settings.

Installing the packages:

OpenStack relies on using MySQL as a data repository for information about the openstack configuration, so we’ll need to set up a MySQL server. Normally when you install the packages for MySQL, it’ll ask you about configuring a root password and such. We can make that hands-off by pre-answering some of those questions. To do this, make a file named “/tmp/mysql_preseed.txt” and put in it the following:

mysql-server-5.1 mysql-server/root_password password openstack
mysql-server-5.1 mysql-server/root_password_again password openstack
mysql-server-5.1 mysql-server/start_on_boot boolean true

Then we can get into the commands to install the packages:

cat /tmp/mysql_preseed.txt | debconf-set-selections
apt-get install mysql-server python-mysqldb
apt-get install rabbitmq-server
# ^^ pre-reqs for running controller nova instance
apt-get install euca2ools unzip
# ^^ for accessing nova through EC2 APIs
apt-get install nova-volume nova-vncproxy nova-api nova-ajax-console-proxy
apt-get install nova-doc nova-scheduler nova-objectstore
apt-get install nova-network nova-compute
apt-get install glance

That’s got all the packages installed onto your local system! Now we just need to configure it up and initialize some information (that’s the bit about networks, etc).

Before I get into changing configs, let me explain what I’ll be setting up. In this example, my internal “network” is 172.17.0.0/24 – and I have a dedicated IP address for this host that is 172.17.0.133. The virtual machines will be in their own network space (10.0.0.0 to 10.0.0.254), and (at this point) not visible from the local network, but will be able to access the local network through their virtualization hosts. The machine I’m using also only has a single NIC (eth0), which is fine for a little test bed, but not likely what you want to do in any sort of real setup.

/etc/nova/nova.conf

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--flagfile=/etc/nova/nova-compute.conf
--verbose
#
--sql_connection=mysql://novadbuser:novaDBsekret@172.17.0.133/nova
#
--network_manager=nova.network.manager.FlatDHCPManager
--flat_network_bridge=br100
--flat_injected=False
--flat_interface=eth0
--public_interface=eth0
#
--vncproxy_url=http://172.17.0.133:6080
--daemonize=1
--rabbit_host=172.17.0.133
--osapi_host=172.17.0.133
--ec2_host=172.17.0.133
--image_service=nova.image.glance.GlanceImageService
--glance_api_servers=172.17.0.133:9292
--use_syslog

Now you might have noticed the MySQL connection string in there. We need to set up that user and password in MySQL to do what needs to be done. I also change the MySQL configuration so that remote systems can connect to MySQL. It’s not needed on a single host, but if you ever want to have more than one compute host, you need to make this change. In /etc/mysql/my.conf, find the line:

bind-address = 127.0.0.1

and change it to

bind-address 0.0.0.0

Now lets make the user in Mysql:

mysql -popenstack
CREATE USER 'novadbuser' IDENTIFIED BY 'novaDBsekret';
GRANT ALL PRIVILEGES ON *.* TO 'novadbuser'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;

And set up the database:

mysql -popenstack -e 'CREATE DATABASE nova;'
nova-manage db sync

If that last command gives you any trouble, then we likely don’t have the MySQL system configured correctly – the user can’t access the tables or something. Check in the logs for MySQL to get a sense of what might have gone wrong.

At this point, it’s time to configure up the internals of openstack – create projects, networks, etc.
We’ll start by creating an admin user:

# create admin user called "cloudroot"
nova-manage user admin --name=cloudroot --secret=sekret

This should respond with something like:

export EC2_ACCESS_KEY=sekret
export EC2_SECRET_KEY=653f3fad-df22-449b-9e6a-ea6c81e32621

You can scratch that down, but we’ll be getting that same information again later and using it, so don’t worry too much about it.

Now we create a project:

# create project "cloudproject" with project mgr: "cloudroot"
nova-manage project create --project=cloudproject --user=cloudroot

And finally, a network configuration for those internal IP addresses:

nova-manage network create private \
    --fixed_range_v4=10.0.0.0/24 \
    --num_networks=1 \
    --network_size=256 \
    --bridge=br100 \
    --bridge_interface=eth0 \
    --multi_host=T
# gateway assumed at 10.0.0.1
# broadcast assumed at 10.0.0.255

Now I’m using the multi-host flag, which is new in the Diablo release. This makes each compute node it’s own networking host for the purposes of allowing the VM’s you spin up to access your network or the internet.

At this point, you’re system should be up and running, all systems operational. Let me walk you through the command steps to actually kick up a little test VM though. These commands are all meant to be done as a local user (not root!)

sudo nova-manage project zipfile cloudproject cloudroot /tmp/nova.zip
unzip -o /tmp/nova.zip -d ~/creds
cat creds/novarc >> ~/.bashrc
source creds/novarc
#
euca-add-keypair mykey > mykey.priv
chmod 600 mykey.priv
#
image="ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz"
wget http://smoser.brickies.net/ubuntu/ttylinux-uec/$image
uec-publish-tarball $image mybucket
#
wget http://uec-images.ubuntu.com/releases/10.04/release/ubuntu-10.04-server-uec-amd64.tar.gz
uec-publish-tarball ubuntu-10.04-server-uec-amd64.tar.gz mybucket
...OUTPUT...
Thu Aug 18 14:02:20 PDT 2011: ====== extracting image ======
Warning: no ramdisk found, assuming '--ramdisk none'
kernel : lucid-server-uec-amd64-vmlinuz-virtual
ramdisk: none
image  : lucid-server-uec-amd64.img
Thu Aug 18 14:02:29 PDT 2011: ====== bundle/upload kernel ======
Thu Aug 18 14:02:34 PDT 2011: ====== bundle/upload image ======
Thu Aug 18 14:03:12 PDT 2011: ====== done ======
emi="ami-00000002"; eri="none"; eki="aki-00000001";
...OUTPUT...

And running the instances:

euca-run-instances ami-00000002 -k mykey -t m1.large
...OUTPUT...
RESERVATION r-1jj2a80v  cloudproject    default
INSTANCE    i-00000001  ami-00000002            scheduling  mykey (cloudproject, None)  0       m1.tiny2011-08-18T21:06:03Z unknown zone    aki-00000001    ami-00000000
...OUTPUT...
#
euca-describe-instances 
...OUTPUT...
RESERVATION r-1jj2a80v  cloudproject    default
INSTANCE    i-00000001  ami-00000002    10.0.0.2    10.0.0.2    building    mykey (cloudproject, SIX)   0   m1.tiny 2011-08-18T21:06:03Z    nova    aki-00000001    ami-00000000
...OUTPUT...
#
euca-describe-instances 
...OUTPUT...
RESERVATION r-1jj2a80v  cloudproject    default
INSTANCE    i-00000001  ami-00000002    10.0.0.2    10.0.0.2    running mykey (cloudproject, SIX)   0       m1.tiny 2011-08-18T21:06:03Z    nova    aki-00000001    ami-00000000
...OUTPUT...
#
euca-authorize -P tcp -p 22 default
ssh -i mykey.priv root@10.0.0.2

To add on additional hosts to support more VMs, you only need to install a few of the packages:

apt-get install nova-compute nova-network nova-api

You do need that exact same /etc/nova/nova.conf file though.

Note:
The default install of Glance expects the images that you’ve loaded up to be available on the local file system for every compute node at /var/lib/glance. Either NFS mount this directory from a central machine, or replicate the files underneath it to all your “compute hosts” when you upload a new image to be used in the virtual machines.

Also, the metadata URL needed for UEC images (169.154.169.154) may need help getting forwarded when running on a system with a single NIC. Two potential solutions: A) run nova-api on each of the compute nodes (quick and dirty) or B) specify the –ec2_dmz_host=$HOSTIP, and potentially invoke the command ip link set dev br100 promisc on to turn on promiscuous mode (per https://answers.launchpad.net/nova/+question/152528).

If you’re into OpenStack, come join us on September 28th to celebrate the Diablo release with other stackers in Seattle.

HP Cloud Services has graciously offered up their offices at 701 Pike St, Suite 1100, Seattle WA to host a meetup.

If you’re planning on coming, please stop by the Meetup link at http://www.meetup.com/OpenStack-Seattle/events/33922932/ and RSVP for us so we can get a sense of who might be wandering by chat and say hello!

We have Quantum and Glance shifting to use Github as a repository, new processes (that would be using Gerrit) wrapped around GitHub to allow the project to have a “gated trunk” methodology, and lots of “motion” within the various projects. It’s pretty easy to see on Github, a little harder with launchpad (I just don’t have the tools handy to create the pretty graphs) – you can see the impact graphs for swift, keystone, glance, quantum, and openstack-dashbaord to see what I mean.

The shifting to Gerrit hasn’t been without it’s trials, but is coming along pretty well now. I really wish the GitHub folks had been a bit more amenable to putting in a field that external folks could use to store metadata about a pull request. Several folks from the OpenStack project (including myself) reached out to them about this, all rebuffed (nicely, but still). In fact, one of the suggestions I got back from the github’r support was “Why don’t you set up Gerrit?”

With the changes in core repository, lots of dependencies are shifting as well. Dashboard was broken a bit this week we kicked things around to get the dependencies to match the new locations, I think we’ve got all those pieces worked around now (pull request outstanding for openstack-dashboard). The other piece that really shifted and broke with these changes were the install scripts that we’ve been using to build and work on a developer’s environment. The cloudbuilder team at Rackspace recently created a whole new setup that works very nicely, so I think we’re going to drop our older scripts (based on over-extended versions of Vish’s excellent nova.sh script) and move to using their new “stack.sh“. (And yeah, of course we’ll want to mess with it ourselves, so I’ve forked it…)

That I primarily work in two modes:

• “gettin’ shit done and stay the fuck outa my way”

Which is the rude way of saying I’m hammering out basic frameworks, making sure the theories of how something will go together work, mostly ignoring error conditions and error checking, and testing is manual and just as I need it. I sketch out structures, build blocks on each other, and hopefully end up with my arch/bridge/sofware component/etc – whatever construct I’m making.

• “making it correct”

In which I get totally OCD about understanding how something works – even something I’ve just created, down to creating tests to cover the logic, defining the failure conditions, and picking “just the right names” for objects, method calls, and general analogies.

The first shows more apparent progress – really short term progress – very quickly. But if that’s all you do, then I’m deep in the bug/muck in not very long at all. I’ve worked with a lot of people who don’t toggle between these modes – or I think even recognize them. I think that’s a bit of a tragedy, and something I encourage people to look at introspectively when they’re working for me.

Another realization:

I have far, far less patience with executives and managers than I do with technical folks/line workers.

The explanation into that is a bit more lengthy:

Being a good manager is very, very difficult. (Related reading that I highly recommend: Managing Humans and Behind Closed Doors. Being a good executive (director or whatever up) is even harder. Doing either of those jobs without resorting to being a dick – keeping an objective and fair eye out is damn near impossible. The dirty truth (that most folks in management positions realize pretty darn quickly) is that you get more money and tend to make more progress if you ARE a dick/backstabber/amoral sonuvabitch. Much of the thinking at an executive level is about capital, progress, and rewards – where anything you control is capital, what you output is progress, and what you get from all this is rewards. If you want to know in a general sense why an executive is doing anything, find out what their reward structure is. The track from there is usually abundantly clear.

Oh – and as for that ‘progress’ thing, that’s really naive of me to assert. Here’s the cynical me:

I’ve seen instances where output is status quo because that makes larger business sense: you want to keep some segment of the operation running solidly. Those are usually the pieces of the business that bore the shit out of me, and ironically the ones that are “looking for creative people” – usually to do mind-numbing work.

I’ve also seen directors having the output of one group to just fuck with other groups – not playing to win, but playing to have someone else loose. Why? Because that makes it easier for other resources you control appear to be making progress (even if it’s not the case). I’ve almost always seen this at the director and above level; managers just don’t have enough resources under their control to assign some of them to screwing with other folks, and it’s very, very rare that you’re overtly rewarded for screwing over the other guy. Okay, maybe not in Microsoft if you’re working for the “office” or “windows” franchise components. There’s another company (that I worked for directly – I haven’t worked at Microsoft) in which this pattern is abundant.

The end result of this cynicism (or realism, if you prefer) is that I have little patience with people whom I perceive as not “trying to make things better”. A bit idealistic of me, I know – a place where my idealism doesn’t match at all with reality. A little quote from a favorite movie (Secondhand Lions):

Hub: Sometimes the things that may or may not be true are the things a man needs to believe in the most. That people are basically good; that honor, courage, and virtue mean everything; that power and money, money and power mean nothing; that good always triumphs over evil; and I want you to remember this, that love… true love never dies. You remember that, boy. You remember that. Doesn’t matter if it’s true or not. You see, a man should believe in those things, because those are the things worth believing in.

But there is something about this game that just really annoys the crap out of me. The music theme is a (very!) thinnly veiled rip off from the Pirates of the Carribean theme. Its not the same, but its so close as to be nearly identical in my head. And that theme has been SO overworked in my hearing that I just want it to go away. I get that the harebrained folks were wanting to (I think, positively) riff on the genre, and that’s reasonably cool – but I’d really hoped for music that enhanced the game, rather than a piece which, probably because of my over exposure to it, pisses me off rather than adds. The game itself was pretty new and interesting for the gameplay – I wish they’d taken the trouble to make the music something of their own rather than relying on a “common theme” that most people will recognize to try and garner some emotion towards their game.

The artwork is a bit overdone for my tastes as well, but maybe it appeals to folks a bit more into steampunk than I am. It’s a fun, goofy genre – the artwork in the game reflects that.

The only other downside is that the game has a tendency to crash on me as well – usually right when you’re loading up a scenario to play it. I’m guessing they got a bit overexcited with the memory allocation or something, as restarting the iPad resolves it – makes me think they’re hitting upper limits on memory pressure, and just not dealing well when the iPad OS says “shrink or die!” under the covers. Their FAQ even tells you to do restart the iPad (or reinstall), which is an incredibly lame response. I hope they put a little more effort into tuning the memory use to avoid that crap in the future.

The game is Hnefatafl – not quite an ancestor of chess, but somewhat related. The board is 11×11, and the game is unusual in that it’s an asymmetric game. It’s harder to win as one player than the other.

I first learned about this game through a book (the Mabinogion) that had a reference to a game called Gwyddbwyll (from ‘the dream of rhonabwy’ – see how this crazy stuff all ties together?). Nobody really knows the details of gwyddbwyll, but there’s some reference to say its related to other games from that era – Irish and Welsh, all of which seem to be the asymmetric “save the king” tafl games.

Right now I’m building this on iPad and iPhone, but I’m not sure if I’ll release it under the iPhone – only because the squares are so damned small on that interface. I have a few ideas for how I want to make the interface function, and once I get beyond that I’m going to aim to make it a turn-based multiplayer game using Apple’s GameKit (just an excuse to use the new stuff, course). A multi-player only game seemed kind of limiting though, so I purchased a book on game AI (Artificial Intelligence for games) so that I could make a reasonable single-player game. It’s a darned good book – so we’ll see how I do implementing some of those algorithms in Objective-C. I bought the whole book for Chapter 8 (Board Games), and don’t regret a penny of the purchase.

We have unit testing up and running, and like most projects a few of those tests are more “functional” or “integration” tests than actually unit tests, but really the project’s been pretty darn good about keeping those separate. As it turns out, the Junit framework is truly awesome for fast, quick tests – but when you start to step into functional testing you want just a touch more linearity to the testing. The unit test frameworks that I’ve run into (nosetests, unittest2) don’t have much in their mechanisms that allow you to set up longer, repeated sequences or flows, so it was time to look around for what else might work there. (sidenote: If you know of some mechanisms that do longer, sequential tests and are based around unittest2 or nose, I’d love it hear about it! I’ll freely admit to not having searched every corner there…)

I’ve been watching and reading quite a bit on BDD and the cucumber framework for testing applications, so I thought it was time to try it out. Of course, being more of a python geek than a ruby geek, I’d heard about Lettuce, which is either a straight up port, or incredibly headily inspired by Cucumber.

Getting started with it was incredibly easy. Everything we’re doing is working from a virtualenv (which I highly recommend for development!), so I just added lettuce to the pip-requires list and it was pretty much ready to go.

In setting up lettuce, I probably spent as much time knocking together a quick “functional_tests.py” script as anything – making it responsible for the sequence of testing. The whole point there was to make a single script that could be run, returning an exit code that meant success or failure for the whole set, so it could be plugged into our local instance of Jenkins.

The core of that fabric script is really just simple fabric commands to set up, clean, and push code to a remote node. I ended up adding a script to our local repository to start up services on the far-end for functional testing – testsetup.sh – that runs the various pieces under “screen” – which makes it super easy to see what’s happened when something goes awry.

You’ll notice at the very end of that functional test script, I invoke “lettuce” on the local machine. That takes care of running the actual test sequences. The documentation for lettuce is pretty reasonable, so I recommend walking through that tutorial for a quick understanding. The super-basic gists is that lettuce uses a DSL that you basically create as you go along, and ends up being rather highly customized to your testing setup. You put together files in a “features” directory, and they use “steps.py” in that same directory to do what’s needed. The pattern that I’ve been using (and which seemed to be implied) was to use the “world” object in lettuce as a sort of black-board to get and store data as you walk through the steps in lettuce. There’s deeper integration into Django based projects (my example was using a little Flask based application run with gunicorn).

The key to setting up the functional testing was taking advantage of the file “terrain.py“, which lettuce uses to do global setup, and setup/teardown around steps, scenarios, features, or the whole shebang. Turns out I had several options for how to reach out and walk through the remote application, so I chose to use twill, although I could clearly have used the local test_client() with Flask, or even the web_driver setup from Selenium to drive Firefox.

One of the nicer things that I found while walking through setting this up was that I could write up a feature file and run lettuce, and it gave me really pleasant “this isn’t implemented yet” output that I could use to expand steps.py. For example, I created “example.feature” in the features directory:

and then ran “lettuce” and got the following output:

As you can see, it’s basically giving you the code to add to steps.py to make this work. As I went, I found myself refactoring steps.py to use some of lettuce more advanced mechanisms – regular expressions and grouping steps together in sequences. And to make sure I don’t leave you completely in the dark about setting up and using steps.py, here’s a snippet from my growing file.

This morning, I found a link that adds a new dimension into place: Raspberry Pi. Wikipedia has a good description of the details of the device, and there’s a BBC interview that shows it and gets into a nice overview.

It’s the price point of this device with the capabilities that changes the game a bit. It’s a computer the size of a tin of mints, with a processor capable of easily running higher level languages – python, ruby, java, whatever. There’s a lot out there now (Arduino) that is a roughly similar cost and form factor, but programming it requires working in C or C-like languages and some tools to get it all together.

Raspberry Pi BBC Interview:

While the interview and foundation talk to making low-cost computers available to kids, I think this kind of device has a much broader potential in personal computing. The tech specs of this device make it very similar to the computing power available in the iPad or iPhones. And while the first generation of this device is missing some things I’d like to see default (a networking port), it’s easily added with a USB hub. The device is also incredibly low power – so I could imagine a shoe-box sized container of these running a 10x the amount of compute, and generating less heat (using less power) than your average dell desktop.

For an always-on application, I want something that’s not drawing a lot of power and could conceivably run unattended in the closet all day and night. This enables it at a new, lower price point (~$25 compared to a Hawkboard SBC at ~$90). Now that’s cheaper (for the moment), than running an instance up on a cloud server…

As I got into the game, noodling on the design and how the interaction would work, I came to the realization that I really would like to have a reasonable single player game as well, and that I didn’t have a clue about how to do board-game AI to make a good game experience.

After looking around through a pile of game AI books, I came across “Artificial Intelligence for Games, Second Edition“. Heavy book, hardback – some $80 if you pick it up outside of Amazon’s discount mechanisms. It looked good after a glance-through and minimal reading, so I took it home to dig in depth. I’m really glad I did!

This book does an excellent job of describing the AI algorithms in pseudo good and good ole english, as opposed to many which seem to be littered with mostly un-parse-able fragments of C++. They reference a number of modern games, but rather than catalog “these used this type of AI, the others didn’t” (which I saw in a number of books), they took some time to explain the tradeoffs in choosing the AI mechanism in the game, even if they didn’t have the “why” of the game designer to pull from.

I liked it enough to put up a review on Amazon, and hey – I’m writing this too. If you’re looking for a good overview of AI algorithms and how they work, I recommend the book.