Running CTS lab on OpenStack hosts

Note: This was originally written in 2012-2013 with minimal changes since, and is unlikely to be up to date.

If we ever get the OpenStack networking sorted out to the point where Corosync can use it, this documents the process of getting a Pacemaker cluster set up for testing with CTS.

Prep
Install python-novaclient

yum install -y python-novaclient

Export your OpenStack credentials

export OS_REGION_NAME=...   export OS_TENANT_NAME=...    export OS_AUTH_URL=...    export OS_USERNAME=...    export OS_PASSWORD=...

export IMAGE_USER=fedora

Allocate 5 floating IPs. For the purposes of the setup instructions (and probably your sanity), they need to be consecutive and to remain sane, should ideally start with a multiple of 10. Below we will assume 10.16.16.60-64

for n in `seq 1 5`; do nova floating-ip-create; done

Create some variables based on the IP addresses nova created for you:

export IP_BASE=10.16.16.60

and a function for calculating offsets

function nth_ipaddr { echo $IP_BASE | awk -F. -v offset=$1 '{ printf "%s.%s.%s.%s\n", $1, $2, $3, $4 + offset }' }

function ip_net { echo $IP_BASE | awk -F. '{ printf "%s.%s.%s.*\n", $1, $2, $3 }' }

Upload a public key that we can use to log into the images we create. I created one especially for cluster testing and left it without a password.

nova keypair-add --pub-key ~/.ssh/cluster Cluster

Make sure it gets used when connecting to the CTS lab exerciser

cat << EOF >> ~/.ssh/config Host cts-exerciser \`echo $IP_BASE | awk -F. '{ printf "%s.%s.%s.*", \$1, \$2, \$3 }'\` User	   	    root IdentityFile      ~/.ssh/cluster UserKnownHostsFile ~/.ssh/known.openstack EOF

Punch a hole in the firewall for SSH access and ping

nova secgroup-add-rule default tcp 23 23 10.0.0.0/8 nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0

Add the CTS lab exerciser to /etc/hosts

cat << EOF >> /etc/hosts `nth_ipaddr 0` cts-exerciser EOF

Create helper scripts on a local host

cat << END > ./exerciser.sh

echo export OS_REGION_NAME=$OS_REGION_NAME >> ~/.bashrc echo export OS_TENANT_NAME=$OS_TENANT_NAME >> ~/.bashrc echo export OS_AUTH_URL=$OS_AUTH_URL >> ~/.bashrc echo export OS_USERNAME=$OS_USERNAME >> ~/.bashrc echo export OS_PASSWORD=$OS_PASSWORD >> ~/.bashrc

function nth_ipaddr { echo $IP_BASE | awk -F. -v offset=\$1 '{ printf "%s.%s.%s.%s\n", \$1, \$2, \$3, \$4 + offset }' }

yum install -y python-novaclient git screen pdsh pdsh-mod-dshgroup

git clone --depth 0 git://github.com/beekhof/fence_openstack.git ln -s /root/fence_openstack/fence_openstack /sbin

mkdir -p /root/.dsh/group/ echo export cluster_name=openstack >> ~/.bashrc

rm -f /root/.dsh/group/openstack for n in `seq 1 4`; do   	echo "cluster-\$n" >> /root/.dsh/group/openstack echo \`nth_ipaddr \$n\` cluster-\$n >> /etc/hosts done

cat << EOF >> /root/.ssh/config Host cts-exerciser \`echo $IP_BASE | awk -F. '{ printf "%s.%s.%s.*", \$1, \$2, \$3 }'\` User      root IdentityFile ~/.ssh/cluster EOF

END

Some images do not allow root to log in by default and insist on a 'fedora' user. Create a script to disable this "feature":

cat << EOF > fix-guest.sh   #!/bin/bash # Re-allow root to log in   sudo sed -i s/.*ssh-/ssh-/ /root/.ssh/authorized_keys EOF

CTS lab exerciser (Fedora-18)
Create and update the lab exerciser

nova boot --poll --image "Fedora 18" --key_name Cluster --flavor m1.tiny cts-exerciser nova add-floating-ip cts-exerciser `nth_ipaddr 0`

If your image does not allow root to log in by default, disable this "feature" with the script we created earlier:

scp fix-guest.sh $IMAGE_USER@cts-exerciser: ssh -l $IMAGE_USER -t cts-exerciser -- bash ./fix-guest.sh

Now we can set up the CTS lab exerciser with the script we created earlier:

scp ~/.ssh/cluster cts-exerciser:.ssh/id_rsa scp exerciser.sh cts-exerciser: ssh cts-exerciser -- bash ./exerciser.sh

Create the Guests
First create the guests

for n in `seq 1 4`; do      nova boot --poll --image "Fedora 18" --key_name Cluster --flavor m1.tiny cluster-$n; nova add-floating-ip cluster-$n `nth_ipaddr $n` done

Then wait for everything to settle

sleep 10

Fix the Guests
If your image does not allow root to log in by default, disable this "feature" with the script we created earlier:

for n in `seq 1 4`; do      scp fix-guest.sh $IMAGE_USER@`nth_ipaddr $n`: ssh -l $IMAGE_USER -t `nth_ipaddr $n` -- bash ./fix-guest.sh; done

Prep
Switch to the CTS lab exerciser

ssh cts-exerciser

Clone Pacemaker for the latest version of CTS:

git clone --depth 0 git://github.com/ClusterLabs/pacemaker.git echo 'export PATH=$PATH:/root/pacemaker/extra:/root/pacemaker/cts' >> ~/.bashrc echo alias c=\'cluster-helper\' >> ~/.bashrc . ~/.bashrc

Now set up CTS to run from the local source tree

cts local-init

Configure a cluster (this will install all needed packages and configure corosync on the guests in the $cluster_name group)

cluster-init -g openstack --yes --unicast --hosts fedora-18

Run
cd pacemaker cts clean run --stonith openstack