Setting Up VirtualBMC (IPMI) for TripleO

TripleO and IPMI via VBMC

When testing TripleO in a virtual environment using instack, I wanted Ironic to control the virtual machines using IPMI instead of the default pxe_ssh driver.

The setup I used is quite straight forward: one baremetal machine where I installed Centos 7.2, then proceeded setting up the environment using TripleO documentation. At the time of writing I have been testing both with Mitaka and Newton.

The assumption is that you have already a working undercloud VM setup and installed the undercloud software on it (openstack undercloud install).
In addition you have already imported the necessary images (overcloud-full, ironic-python-agent etc). You are now ready to import the instackenv.json defining your “baremetal” virtual machines into Ironic.

On the host (the hypervisor — the baremetal machine where you are running libvirt to host instack and the “baremetal” virtual machines) you will install and configure VirtualBMC.

 

On the Hypervisor

VirtualBMC can best be installed via pip install. You can install pip via yum. You could however choose to clone it and install it manually from git.

Note that you might have to (yum) install libvirt-devel and some other packages for pip to install virtualbmc successfully:

 

In addition, I like controlling iptables using firewalld. If you still don’t have it installed you can install and enable it using yum:

The idea is that the host has one IP (by default 192.168.122.1) and is accessible from instack VM. IPMI uses by default port 623. So for example, when you run a ipmitool command, it will default to this port. You can however provide the -p flag and specify different ports. We need to have a different port per VM because the IP is similar (it is the host’s IP 192.168.122.1).

Depending on the amount of virtual machines you want to have controlled via vbmc, allow access to the necessary ports so that the undercloud VM can reach the vbmc daemons. In my case I had 5 overcloud virtual machines (baremetalbrbm_0..4). This is how I chose to have the ports assigned:

baremetalbrbm_0 gets default IPMI port 623
baremetalbrbm_1 port 624
baremetalbrbm_2 port 625
baremetalbrbm_3 port 626
baremetalbrbm_4 port 627

We allow these ports in the firewall:

Now we can start adding our virtual machines (“domains”) to vbmc:

Yey! We have them added. Now all we have to do is start them up:

Let’s check they are up and listening:

 

On the Instack VM

Now we can log in to instack vm and check the connectivity using ipmitool:

Our instack.json should look like this: (note that I added the “name”: “bm[0-4]” attribute because it makes it easier to run commands later on without having to copy/paste entire UUID)

Now we can import our instackenv.json (openstack baremetal import instackenv.json).

And yes, you noticed correctly — we still haven’t defined the ports per node. The problem I faced was that I could not add a “pm_port” in the instackenv.json as it does not expect that value. I had to configure this after the nodes have been imported into Ironic:

This has to be done to all the nodes excluding node bm0 (it is not necessary because it is listening on the default IPMI port 623).
That’s it! Now you can run the introspection and proceed with deployment.

One caveat is that I did not manage to run bulk introspection because Ironic did not like the fact it had to boot up multiple nodes with the same IP address (192.168.122.1). It doesn’t know we are “hacking” in VirtualBMC on a hypervisor. Of course, in a real baremetal environment it would be very bad to have same IP’s on the same network …

Edit: there’s already work to move away from pxe_ssh and use pxe_ipmi with virtualbmc by default: https://blueprints.launchpad.net/tripleo/+spec/switch-to-virtualbmc

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.