How to configure ESXi to shutdown using an APC SmartUPS (with lamw scripts)

VERSION 7 Published

Created on: Feb 12, 2009 9:50 AM by Joseph Holland - Last Modified:  Mar 13, 2009 5:36 AM by Joseph Holland

Introduction

 

This tutorial will show you how to configure a VMWare ESXi install to shutdown using an APC SmartUPS. This proven practice came about because our company was sending a Dell PowerEdge T300 server with ESXi and a APC UPS to one of our sales offices. Because it would just be this server and the UPS there, we needed a way for ESXi to shutdown the guest operating systems and then itself gracefully.

 

This document assumes that you have a VMWare ESXi install with VIMA running on it and an APC SmartUPS with a network management card installed.

 

This document includes scripts from the VMWare Communities user “lamw.” His scripts are available at http://communities.vmware.com/docs/DOC-9531 Lamw's scripts do not eed the enabling of the unsupported SSH option within ESXi.

Outline

 

  1. Configure the SmartUPS

  2. Install apcupsd on our VIMA virtual machine

  3. Put lamw's scripts onto the VIMA virtual machine and configure

  4. Test

 

How to configure ESXi to shutdown using an APC SmartUPS

 

apcups.png

 

1. Configure the SmartUPS

 

Now we need to add the VIMA virtual machine's IP address into the allowed hosts for the PowerChute Network Shutdown on the UPS.

 

Open the UPS's Network Management Card's web interface. Login with the username and password of “apc.” Click on the UPS name in the top left.

 

apc1.png

 

Click on the PowerChute link in the left panel now and add the VIMA server's IP address to the list of allowed hosts. Click "Add."

 

apc2.png

 

2. Install apcupsd on our VIMA virtual machine

 

Now we will install the latest version of apcupsd from www.apcupsd.com. The latest version as of writing is 3.14.5. Download the x86_64 version made for el5 (link).

 

Logon to your VIMA VM and run the following to install apcupsd.

 

sudo rpm -ivh apcupsd-3.14.5-1.el5.x86_64.rpm

 

Now we need to edit the apcupsd.conf file. Make sure to change to the IP address to the
IP of the UPS.

 

sudo rm /etc/apcupsd/apcupsd.conf

sudo nano /etc/apcupsd/apcupsd.conf

 

## apcupsd.conf v1.1 ##
UPSCABLE ether
UPSTYPE pcnet
LOCKFILE /var/lock
DEVICE UPS_IPADDRESS:apc:admin user phrase
UPSCLASS standalone
UPSMODE disable

 

Save and close.


Now we will have to disable the iptables firewall running on VIMA.

 

sudo chkconfig iptables off

sudo service iptables save 

sudo service iptables stop

 

Now start the apcupsd daemon

 

sudo /etc/init.d/apcupsd start

 

Check is it running

 

ps ax|grep apc

 

5332 ? Ssl 0:00 /sbin/apcupsd -f /etc/apcupsd/apcupsd.conf

 

Wait a few minutes and you should see:

 

Broadcast message from root (Thu Mar 12 03:40:28 2009):

+
Communications restored with UPS s-vima+

 

Run "apcaccess" to confirm connection to the UPS.

 

apcaccess

 

This command should give you back something like this if it connected successfully.

 

+APC : 001,051,1224
DATE : Thu Jan 08 11:32:21 GMT 2009
HOSTNAME : s-vima
RELEASE : 3.14.4
VERSION : 3.14.4 (18 May 2008) red hat
UPSNAME : S-VIMA
CABLE : Ethernet Link
MODEL : PCNET UPS Driver
UPSMODE : Stand Alone
STARTTIME: Thu Jan 08 11:32:14 GMT 2009
STATUS :
LINEV : 231.8 Volts
LOADPCT : 5.2 Percent Load Capacity
BCHARGE : 100.0 Percent
TIMELEFT : 62.0 Minutes
MBATTCHG : 10 Percent
MINTIMEL : 5 Minutes
MAXTIME : 0 Seconds
MAXLINEV : 234.7 Volts
MINLINEV : 233.2 Volts
OUTPUTV : 000.0 Volts
SENSE : High
DWAKE : 000 Seconds
DSHUTD : 090 Seconds
DLOWBATT : 05 Minutes
LOTRANS : 208.0 Volts
HITRANS : 253.0 Volts
RETPCT : 015.0 Percent
ITEMP : 29.7 C Internal
ALARMDEL : No alarm
BATTV : 27.5 Volts
LINEFREQ : 50.0 Hz
LASTXFER : Automatic or explicit self test
NUMXFERS : 0
TONBATT : 0 seconds
CUMONBATT: 0 seconds
XOFFBATT : N/A
SELFTEST : NO
STESTI : 336
STATFLAG : 0x07000000 Status Flag
REG1 : 0x40 Register 1
REG2 : 0x00 Register 2
REG3 : 0x00 Register 3
MANDATE : 12/13/03
SERIALNO : XXXXXXXXXX
BATTDATE : 12/13/03
NOMOUTV : 230 Volts
NOMBATTV : 24.0 Volts
EXTBATTS : 0
FIRMWARE : 600.3.I
APCMODEL : Smart-UPS 1000
END APC : Thu Jan 08 11:33:55 GMT 2009

 

 

3. Put lamw's scripts onto the VIMA virtual machine and configure

 

These steps are taken from http://communities.vmware.com/docs/DOC-9531 by lamw.


Download ghettoShutdown.pl and upsVIShutdown.pl from lamw's page. Put them into “/home/vi-admin”

 

Set executable.

 

chmod 755 /home/vi-admin/ghettoShutdown.pl

chmod 755 /home/vi-admin/upsVIShutdown.pl

 

Edit the following in “upsVIShutdown.pl”


Insert your ESX/ESXi host(s), use the hostname that you used to add to VIMA managment interface, if you're unsure run:

 

sudo vifp listserver


Remember to make sure the host that is running your VIMA VM is listed as the last entry.

 

my @hosts = ("esxi.host1","esxi.host2”);


Modify the logoutput of the shutdown process if you like, default will go to “/tmp/upsShutdown.log”

 

my $log_output = "/tmp/upsShutdown.log";

 

Insert the displayName of VIMA or VM that is monitoring your UPS device (case sensitive). Very important step else you could end up shutting down the VM that is executing this script, we wouldn't want that.

 

my $ups_vm_name = "S-VIMA";


Insert the number of seconds to delay after a guestOS has initiated to be shutdown, this will vary depending on the type of applications the VM(s) may be running, tweak this value as needed. Note, the shutdownVM() is a non-blocking function, if you set the delay to be too short the host could be powered down before the VM(s) have completed powering off (default 15secs).

 

my $sec_to_sleep = 15;


Now we are going to tweak the apccontrol file to change what is done when the UPS issues a shutdown command.

 

nano /etc/apcupsd/apccontrol


Edit the doshutdown section like so, change the IP address to the ESXi host:

 

doshutdown)
  echo "UPS ${2} initiated Shutdown Sequence" | ${WALL}
  /home/vi-admin/upsVIShutdown.pl
  ${SHUTDOWN} -h now "apcupsd UPS ${2} initiated shutdown"
;;


Save and close the file.

 

4. Test

 

If you go to the "Control" option on the left menu of the APC SmartUPS Network Management Card interface you can run a test to verify everything works. Change the
"Action" to "Turn UPS Off." Make sure that the ESXi server IS NOT being powered off of the UPS (just for testing).

 

apc3.png

 

Click "Next" and then "Intiate Control Action."

 

apc4.png

 

You should get this message broadcast on the VIMA server you installed apcupsd on.

 

Broadcast Message from root@s-vima (somewhere) at 11:50 ...
Remote Shutdown. Beginning Shutdown Sequence.
Broadcast Message from root@s-esximon (somewhere) at 11:50 ...

UPS s-vima initiated Shutdown Sequence

 

That's it, done.

 

 

Resources

 

Author

 

Joseph Holland, Kepak Group.

joseph.holland@kepak.com

 

Reviewers

 

William Lam (lamw), Innuendo_, llyubenov, Duncan Epping and Steve Chambers.

 

Disclaimer

 

You use this proven practice at your discretion. VMware and the author do not guarantee any results from the use of this proven practice. This proven practice is provided on an as-is basis and is for demonstration purposes only.

 

Average User Rating
(1 rating)




Mar 12, 2009 4:45 AM Click to view Joseph Holland's profile Joseph Holland says:

Thanks to William Lam (lamw), Innuendo_, llyubenov, Duncan Epping and Steve Chambers.

 

Mar 13, 2009 6:58 AM Click to view William Lam's profile William Lam says: in response to: Joseph Holland

Thanks for putting this detailed document together, great work.

May 12, 2009 2:58 PM Click to view Brian Bustin's profile Brian Bustin says:

I am always leery of disabling a firewall. Instead of disabling the firewall, I added two rules to the top of the INPUT chain in iptables. I didn't add it into the redhat chain that was already thee in case it gets auto generated. This seems to work so far. I have not yet tested a UPS-initiated shutdown. I did not add rules for SNMP or SNMPTRAPS because they seemed to already be there.

 

Here are the commands that I used. I hope this helps.

 

sudo iptables -I INPUT 1 -p TCP --dport apcupsd -m state --state NEW -j ACCEPT

sudo iptables -I INPUT 2 -p UDP --dport apc-3052 -m state --state NEW -j ACCEPT

sudo /etc/init.d/iptables save

May 13, 2009 1:34 AM Click to view Joseph Holland's profile Joseph Holland says:

Thanks Brian. I tried adding rules to VIMA's firewall to allow the UPS to talk with apcupsd, but I could not get it to work properly. If I get a chance to test this (may take a while as I do not have my spare UPS for testing anymore) or you can confirm that the UPS can communicate properly with the apcupsd daemon running on the VIMA VM I will add this into the document.

 

There are details on testing this here:

http://www.apcupsd.com/manual/Testing_Apcupsd.html#SECTION000145000000000000000

 

 

Joseph.

May 18, 2009 1:10 PM Click to view Brian Bustin's profile Brian Bustin says: in response to: Joseph Holland

We performed a simulated power outage test of our entire infrastructure as hurricane season is imminent. The test consisted of making sure that our backups were completed and then unplugging both of our UPS from building power.

 

The VMs running on ESXi as well as the ESXi hosts themselves shut down properly.

 

It looks like the firewall rules work, but I would encourage someone else to test as well just to double-check.

May 20, 2009 6:34 AM Click to view guidobras's profile guidobras says:

Hi, I'm using ESXi U4 and version 4 of this document (shutdown via ssh with a script running on ESXi host).

I'm testing the script connecting via ssh to the ESXi U4 host and manually typing in the relevant commands.

Unfortunately when I issue the final command to physically shutdown the host (/sbin/poweroff) it refuses to do it (Poweroff: No.)

To make it work I have to force the shutdown with /sbin/poweroff -f (which causes a non graceful shutdown)

is there a workaround to this? Or am I doing something wrong?

 

These are the commands I've issued and related messages:

 

~ # /sbin/shutdown.sh

Running sfcbd-watchdog stop

Stopping sfcbd

Running wsmand stop

Stopping openwsmand

watchdog-openwsmand: Terminating watchdog with PID 1557

Running sfcbd stop

This operation is not supported.

Please use /etc/init.d/sfcbd-watchdog start

Running slpd stop

Stopping slpd

Running hostd stop

watchdog-hostd: Terminating watchdog with PID 1475

Running ntpd stop

Stopping ntpd

config implicitly loaded

Saving current state in /bootbank

Clock updated.

Time: 09:22:18 Date: 05/20/2009 UTC

 

~ #/sbin/poweroff

poweroff: No.

May 25, 2009 2:45 AM Click to view guidobras's profile guidobras says: in response to: guidobras

Solved!

It was the "sleep 30", too short to allow my VM to complete a graceful shutdown

I used a longer sleep and all is OK now

Aug 6, 2009 3:37 AM Click to view FixR's profile FixR says:

Thanks for the very usefull scripts and example!

One question: How can I add a script for remote shutting down my storage? I think it must be done before shutting down the vMA server. Where can i integrate it?

 

Thanks in advance.

Aug 11, 2009 3:57 PM Click to view nycvelo's profile nycvelo says: in response to: Joseph Holland

Thanks for this helpful howto.

 

What version of apcupsd is right for ESX 3.5 hosts? I thought ESX (non-i) was based on something older than RHEL5 (and is i386 and not x86_64) but just checking...

 

Thanks again.

Aug 12, 2009 3:26 AM Click to view Joseph Holland's profile Joseph Holland says: in response to: nycvelo

The apcupsd is not installed on the ESXi host, it's installed on VIMA/VMA which is based on RHEL5 and x86_64.

Aug 12, 2009 6:00 PM Click to view nycvelo's profile nycvelo says: in response to: Joseph Holland

Thanks, Joseph! I did make one exception from your howto: Because I already have another machine connected to the monitored UPS, I installed and configued the VMA box as a client to this monitor box. Here's the relevant stuff from apcupsd.conf:

 

UPSCABLE ether

UPSTYPE net

DEVICE myapcupsdmonitor.examplecom:3551

 

I've tested this setup on other (physical) Linux boxes and it works fine.

 

I also added '/home/vi-admin/upsVIShutdown.pl' to apccontrol as per your howto.

 

So here's my question: With my VMA box as a client, will that perl script still shutdown each guest VM or will I need apcupsd instances running on each VM?

 

I should mention that the guest VMs are a mix of Windows, FreeBSD and OpenBSD. The Windows guest machines will gracefully shut down when hitting stop from the ESX VI client; the BSD boxes don't, so I need to run shutdown on them before powering off from ESX VI.

 

Thanks again!

 

Aug 17, 2009 1:31 AM Click to view FixR's profile FixR says:

How can I add a script for remote shutting down my storage? I think it must be done before shutting down the vMA server. Where can i integrate it?

 

Thanks in advance.

Aug 17, 2009 2:55 AM Click to view Joseph Holland's profile Joseph Holland says: in response to: FixR

@FixR - I have sent you a PM about this.

 

Anyway what OS does your storage server run, is it as easy as running a shutdown command via SSH? If so I think the best place for you to put this script/command is after the "/home/vi-admin/upsVIShutdown.pl" line in the "apcupsd.conf" file. See below:

 

doshutdown)

echo "UPS ${2} initiated Shutdown Sequence" | ${WALL}

/home/vi-admin/upsVIShutdown.pl

/path/to/script/shutdown_storage.sh

${SHUTDOWN} -h now "apcupsd UPS ${2} initiated shutdown"

;;

 

 

The above "should" work, I'm not 100% though. Give it a shot and let me know how you get on.

 

 

Joseph.

More Like This

  • Retrieving data ...

More by Joseph Holland