Applications : Resource: Using NFS as a data store. : Comments

RE: Resource: Using NFS as a data store.

lamontda — Fri, 12 Sep 2008 09:17:29 GMT

Note that NetApp best-practice documentation used to state that the advanced setting NFS.LockDisable should be set to 1 if NetApp snapshots are to be used, but surprise surprise disabling locking causes locking to be disabled, i.e. it is possible (and this happened to me!) for virtual machines to be started on multiple hosts concurrently. Current advice from VMware Support is to leave NFS.LockDisable at its default 0, though what effect this has on NetApp snapshots, I've still to establish.

RE: Resource: Using NFS as a data store.

schambers — Fri, 12 Sep 2008 09:42:16 GMT

Hey Tony, I concur that folks should follow the VMware Support advice and leave the defaults - are you going to check out the snapshots with this in place?

RE: Resource: Using NFS as a data store.

lamontda — Fri, 12 Sep 2008 10:40:24 GMT

Hi Steve, the following NetApp Technical Support Bulletin dated September 11th explains the situation nicely but also raises a few questions of its own: "TSB-0805-02: Virtual Machines using NFS storage pauses when using VMware Snapshots." A snip from the bulletin says:"

Summary:

Note: This is an update to the TSB issued on this subject on May 5, 2008 and contains changes to the workaround and solution sections as a result of new information provided by VMware.

Virtual machines utilizing NFS data stores can experience an extended period where I/Os are suspended when using VMware ESX Server snapshots (VMsnaps).

This behavior has been identified as bug SR 195302591 by VMware (NetApp Bug ID 264618), and NetApp has published a best practice recommendation in TR-3428. This TSB is intended to provide some additional information about potential issues & further expands on best practice recommendations.

Problem Description:

VMware ESX Server uses a locking mechanism to prevent an ESX Server host from simultaneously accessing the data files being used by another ESX Server host. When using NFS, ESX Server uses lock files to prevent multiple virtual machines from accessing the same set of ‘.vmdk’ files. This results in extended periods of suspended I/O for these virtual machines.

Please note that this TSB will be updated after more information from VMware is available.

Workaround:

NetApp believes that there are several best practice steps that help to dramatically reduce the exposure to this ESX Server issue with NFS data stores.

If this suspension of I/O is experienced, NetApp’s current best practice recommendations include all of the following:

1. Deploy the ESX Service Console ports on virtual switches with redundant NICs and links.

2. Deploy each Service Console port on the same virtual switch as any VMkernel port.

3. Deploy the ESX VMkernel Network Redundancy with redundant links.

4. Ensure that the Isolation response option is set to Power Off for each virtual machine using NFS storage.

NOTE: The Isolation response option default setting is Power Off in ESX Server for all protocols, and the recommended setting from both VMware and NetApp. However, in some FC configurations (not limited to NetApp FC storage) this setting may have been changed to Power On. Therefore, close attention should be paid to this setting in ESX Server configurations that may have initially used FC data stores and then used with NFS data stores.

5. Spanning tree protocol should be disabled on any network port using VMkernel connections.

Important Note: The previous version of this TSB recommended changing the default setting of NFS.LockDisable from 0 to 1. This is incorrect. DO NOT change the value to 1. If the value had previously changed to 1, reset it to its default of 0."

RE: Resource: Using NFS as a data store.

ukbrown — Thu, 25 Sep 2008 12:06:13 GMT

Had the same issue as you, as this was the reply from Vmware. It looks like the patch below will enable as good performance as if locking were disabled.

Vmware support reply below

You have some problems of consistency because I think you turn on the NFS.LockDisable to 1 means that you disable locks when accessing a disk in order to improve the performance. Just check it on "advanced setting" on your virtual center.

It was recommended to do that to improve the performance with NetApp but you risks of data inconsistency like in your case.

Just take a look to this document p.13 at the end of the page.

http://www.netapp.com/us/library/technical-reports/tr-3428.html

If you want to remediate to that just put 0 instead of 1 to NFS.LockDisable and put the patch for the performance (ESX350-200808401BG patch apply to ESX 3.5 update 1 or 2 only).