VIOPS: Proven Practices for Deploying and Managing VMware: Message List - Using NFS as a data store.

Re: Using NFS as a data store.

miriad@gmail.com — Tue, 12 May 2009 16:34:43 GMT

Here are some of my notes from our POC of RCU 2.0.1

While you make a golden datastore that holds your templates, the filer does an NDMP copy of the .vmdk files to a temporary directory on the destination datastore. It doesn't flexclone off of the original gold image.
If the template doesn't reside on the same filer, if will do a copy of the remote vmdk over IP using vCenter. NDMP is much faster, so it seems best to keep all the datastores attached the same filer IP address
Linux guest VMs don't seem to like LVM again. We've patched to the most up to date version of ESX, and all the guests have the most recent tools, but it's not working at the moment. The same template does do guest customization correctly using a standard deploy from template in vCenter. So for now I will remake the template SLES 10 guest without LVM and try again.
Make sure your template vmdks have aligned partitions or RCU will complain. You can grab mbralign here for NetApp customers: http://now.netapp.com/NOW/download/tools/mbralign/
Each time you do a rapid clone, RCU copies the vmdk from the golden datastore, regardless of if the template data already exists on the destination datastore or not. This causes another full sized copy to be made to the destination datastore, so be sure to keep sis enabled and scheduled on the destination datastore (RCU turns on SIS when it creates a datastore, assuming you're licensed for it).
It may be better to keep a cloned guest as a "datastore local" template if you know it's the most recent copy of the template. It is considerably faster to deploy this way, but you have to keep track of which datastores have the most recent template, which I could see might be cumbersome.

I'm looking forward to see what the plugin provides in vSphere! Hopefully it includes this functionality.

Re: Using NFS as a data store.

paulmon — Wed, 06 May 2009 19:31:16 GMT

We're going to be playing with RCU in our lab shortly.

As for best practices, followed the Netapp docs and anything else we could get our hands on.

Re: Using NFS as a data store.

miriad@gmail.com — Wed, 06 May 2009 19:09:36 GMT

Our shop uses 100% NFS to almost 50 ESX servers and a few labmanager environments with no issues (at least due to NFS). In terms of best practices, particularly if you're going to run NetApp filers, is to follow the info in the NetApp doc found here:

TR-3428

You will find a wealth of information in it. As a side note, if you can get your filers up to Ontap 7.3.1p1 or higher, you can run the new RCU 2.0.1 to have a precursor to some of the functionality that the vStorage interface will have in vSphere. If you're unfamiliar with RCU, you can take a look at this youtube video.. We're about to begin testing it tomorrow

Keep in mind you still have to follow all the normal best practices for a guest VM, including file system alignment, as well as setting certain things like SCSI timeout values on your guest operating system. You can look here for more details on that.

Re: Using NFS as a data store.

paulmon — Sat, 02 May 2009 18:59:02 GMT

Interesting thread and very topical for what we're currently going through. We're a big Netapp shop and deployed our virtual infrastructure on Netapp NFS. After experiencing some stability issues and odd VMotion oddities we were told by VMWare to move swap to iSCSI - Not something that I wanted to do. Interestingly Paul's post was back in '08 and it seems that VMWare support is still telling customers to do this, they recommended this about 5-6 weeks ago and I've been fighting the recommendation ever since.

Re: Using NFS as a data store.

dsulli29 — Tue, 20 Jan 2009 22:41:14 GMT

I am using NFS datastores on ESX 3.5, we moved away from iSCSI because of the storage overhead to snapshot LUNS on NetAPP, my volumes had to be double the lun size... Using NFS I keep my volumes 25-50% free space and I dedupe them with NetApp ASIS, which is a huge space savings.

We replicate these volumes to an offsite NetApp SAN... I am considering moving the vm guest swap files to a seperate dedicated NFS datastore that I will not replicate. I am looking to conserve the bandwith used by replication and other than the page files, there is not that much that changes in the VM guest. So I figure for DR, I don't need the page file.

40 VMs x 1GB page file = 40GB of data to replicate at every replication interval... that is hurting us.

Thoughts on this? Can anyone offer a reason I should not do this? I know it's best practice to store the swap file with the vmdk on NFS, but I really am not sure why. Does what I want to do make sense? Will it cause me any problems?

Thanks

Re: Using NFS as a data store.

anl — Tue, 25 Nov 2008 16:14:10 GMT

Thanks, Paul - clearing that up makes my day.

Re: Using NFS as a data store.

pmanning@vmware.com — Mon, 24 Nov 2008 17:01:37 GMT

The current best practice for NFS is to not seperate the VM swap space from the VMhome directory on a NFS datastore. The reason for the originial recommendation was just good old fashioned conservitiveness. The thought was that if the IP traffic slowed the response time for swap space access that it could have a significant impact on the performance of the VM. However, the performance conern was not an issue that made this step of placing swap on antoher storage device a needed step.

Further, it turns out that it is a more simple solution to address the concern of performance degredation is to not over subscribe the memory of the VM. However the performance of the access to NFS storage compared to FC storage for swap space is not considered to be a significant enough delta that would make the separation of VM swap from the VMhome worth it.

So even though the KB article 1004082 hasreference to separating them, it is not longer considered best practice. And that KB article is slated to be updated to be consistent with our current best practice of keeping swap in the VMhome Directory for VMs on NFS datastores.

Re: Using NFS as a data store.

schambers — Sun, 23 Nov 2008 02:07:40 GMT

Hey Andy, or should I call you Lazarus?

This might be a language thing. I've ping'd Paul to see if we can get clarification.

Re: Using NFS as a data store.

anl — Fri, 21 Nov 2008 15:29:30 GMT

I don't mean to drag a dead thread back to life, but:

-Is it truly a best practice to have vmswap on NFS? The only document in the VMware KB found when searching for "NFS swap" (1004082) suggests otherwise.

If so:

-What was the basis for the previous recommendation not to have vmswap on NFS?

-And what is the basis for the change in recommendations?

Thanks!

Re: Using NFS as a data store.

schambers — Tue, 09 Sep 2008 13:21:36 GMT

Thanks, Paul! Everyone, Paul is our Storage Guru in VMware, so sage advice indeed!

To make sure we don't lose this valuable advice, I've created a document Using NFS as a data store.

Anyone can edit and add to this document - just click Submit for approval when you're done and I'll approve it for publishing.

Why do this? Good advice gets lost in threads, so let's make sure that doesn't happen to us on VIOPS. Also, this site also behaves like a wiki - so we can all add our thoughts, ideas and practices in an equitable manner.

Is this a good idea? Violently for / against?