Syncing Eucalyptus configuration with csync2


With Eucalyptus HA its very important that the configuration files between HA pairs is the same.  In the event of failover you want to be sure that the configuration of the new enabled system is exactly as it should be; mirroring the now disabled/notready system.  By not doing this you run the risk of Bad Things™.

To achieve consistency between configuration files, we can use a useful tool originating from LINBIT called csync2.  For those familiar with SUSE Linux HA, csync2 is the recommended method to keeping files in sync across cluster nodes.

Assumptions

I’m going to assume that Eucalyptus 3.X is already installed and you have all the Eucalyptus components paired on two separate hosts, on top of RHEL6.

Install csync2

The csync2 utility is part of the EPEL repository, which you will have configured during the Eucalyptus installation.  Install on both of your hosts with a simple yum command:

# yum -y install csync2

It’ll pull in xinetd, so always check other xinetd services in /etc/xinetd.d/ to make sure unwanted ones are disabled (typically with a “disable = yes” line).

Preparation

Before continuing, it’s probably a good idea to ensure that you have DNS configured and all hostnames are resolvable.  This isn’t mandatory but I’m going to assume this is configured, if not you’ll need to ensure /etc/hosts is up to date and that hostnames are set correctly, or use some of the csync options like the -N flag.  It should be noted that csync2 requires the fully-qualified domain name and will match hostnames per the output of the “hostname” command.

Configure csync2

Next, configure csync2 by first generating a pre-shared key to be used for authentication between your Eucalyptus systems:

# csync2 -k /etc/csync2/eucalyptus.key

Next, edit the csync2 configuration file on one of your systems and configure a basic group for your two hosts.  Remove the existing example entries and start with the following (remove my comments):

group eucalyptus {
 host euca-ha-1;   <- "euca-ha-1" should be replaced with the resolvable hostname of your system
 host euca-ha-2;
 key     /etc/csync2/eucalyptus.key;  <- specifies the pre-shared key for authentication
 include /etc/csync2/csync2.cfg;  <- always include the csync2 configuration file itself
 include /etc/eucalyptus/eucalyptus.conf; <- our primary configuration file
 include /etc/eucalyptus/drbd.conf; <- our drbd resource file
 include /etc/drbd.conf; <- the global drbd configuration file
 }

Prepare to sync!

With the csync2 configuration file built, copy this and the pre-shared key to the other host:

# scp /etc/csync2/csync2.cfg /etc/csync2/eucalyptus.key euca-ha-2:/etc/csync2/

Then chkconfig the services and start xinetd on both hosts:

# chkconfig csync2 on && chkconfig xinetd on
# service xinetd restart

You’ll need to make sure tcp port 30865 is opened in iptables on both hosts.

Csync!

Now you are good to sync, you can perform an initial dry-run sync with verbose output as follows.  Note that this won’t change any files but will propose the changes in its output.

# csync2 -xvd

If all looks good, go ahead and do the initial sync:

# csync2 -xv

Check your second host and you should see the configuration files synced over.

Schedule the sync

Now you can add a cron entry to schedule a periodic sync, if you wish.  Edit /etc/cron.d/csync2, perhaps consider a sync attempt every 10 minutes:

*/10 * * * * root csync2 -x

Advanced configuration

If you have Eucalyptus components spread across more than two hosts (i.e. Walrus, SC and CC on separate machines), consider using groups. Groups allow you to specify different sync tasks for different collections of hosts.  This is useful with Eucalyptus where standalone Walruses would have drbd configuration files but standalone CLCs would not, it would enable you to sync files based on these host roles.  The example below shows a possible configuration.  You’d probably want to sync the configuration file and key across all systems but then use the group functionality to split components.

group all {
 host euca-clc1;
 host euca-clc2;
 host euca-walrus1;
 host euca-walrus2;
 key /etc/csync2/eucalyptus.key;
 include /etc/csync2/csync2.cfg;
 }
group walruses {
 host euca-walrus1;
 host euca-walrus2;
 key /etc/csync2/eucalyptus.key;
 include /etc/eucalyptus/eucalyptus.conf;
 include /etc/eucalyptus/drbd.conf;
 include /etc/drbd.conf;
 }
group clcs {
 host euca-clc1;
 host euca-clc2;
 key /etc/csync2/eucalyptus.key;
 include /etc/eucalyptus/eucalyptus.conf;
 }

For walrus syncing, you’d have the following in a cron on one of the walruses:

*/10 * * * * root csync2 -x -G walruses

On a CLC:

*/10 * * * * csync2 -x -G clcs

Then the following entry on a “master” system to sync across all hosts:

*/5 * * * * csync2 -x -G all
Advertisements

One thought on “Syncing Eucalyptus configuration with csync2

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s