Hi Björn,
no problem. Yeah in your situation it sounds like a good idea to investigate putting VM swap on local SSD disk. Investigate the different options i mentioned in my previous reply and make sure you got enough bandwidth when you do vMotion within one site and between site if you put some data in the swap file. Just perform vMotion operations when using local SSD drives to hold the VM swap file and when the VM swap file is located on your NetApp storage.

regards,
Magnus

Hello Magnus and thank you for your thorough reply!

Ah right, dedup is recommended as well.

Since we currently only have a NetApp MetroCluster as our only storage with all aggregates replicated, it will still be fairly expensive disk. And the dedicated swap store has not done much for our pretty low deduplication levels (on other hands, we have a lot large databases and other stuff messing up dedup stats). I will investigate putting VM swap on local SSDs, but we also have a need for a vFRC for some workloads.

Thanks.

Hi Björn,
well in this case it is a recommendation from the storage vendor to use a separate datastore for the VM swap files. It has to do with both backup and deduplication.

The VM swap file is actually created, in its full size, when you power on the VM so that is a parameter you need to include when you calculate the storage required for your environment. So from a vSphere perspective the files are created but different storage vendors manage zeroes differently.

In a stretched cluster scenario you will configure your system to have enough capacity from a vSphere HA perspective to manage one site failure but this is not the same as it will never be any overcommitment of RAM.
Remember, vSphere HA, is not about performance. It’s about guaranteeing VMs to be powered on.

If VM swap has happen (based on physical RAM constraints during day 2 operation, VM boot storm or misconfiguration) the swap file will not be cleared meaning data will no be moved into physical RAM until the VM needs the actual data.
This means you might need to swap in memory form the VM swap file but it’s not available since the RAM in the swap file was swapped out a long time ago.

You might consider using e.g. Swap to host cache aka swap to SSD or putting VM swap files on local SSDs instead of shared storage area when the storage vendor recommends a split between VM swap file and the rest of the VM files.
The connection between the ESXi hosts when performing vMotion will come into play in such a configuration and also the size (RAM) of the VM.

If you are absolutely sure that the swap file will never be used, maybe by setting 100% RAM reservation per VM, this will not be an issue but i have only implemented a very few environments using this concepts.

I think this particular setup with dedicated swap in metro cluster is interesting to discuss. All the recommendations and documentation I’ve seen is not very clear, at least not to me. Would love to hear your view on it:

Question:
Why should there be a dedicated datastore for VM swap in a vSphere Metro Cluster based infrastructure?

Why:
The primary reason (and only reason for vMSC) I was advised and read is that swap can reside on a datastore that is not backed up and/or replicated. Therefore you can save the amount of provisioned VM memory in disk space = save money.

So if we back a few steps: To my understanding is that the VM swap file is not taking any disk space as long as the host memory is NOT over committed. True? At least, I have never seen a that any .vswp files takes any actual disk space at all in our environment. And if the VM does swap, there is obviously not a good user experience.

My point:
As long as you have enough RAM in your ESXi the swap file will never take any real disk space. That makes the recommendation of a dedicated datastore for swap in a VMware vMSC not a valid one. I can’t see that a company invests in a stretched storage solution but have insufficient memory in their hosts. It will just complicate your environment a bit more and takes a bit of management.

Do you agree? Am I missing something here?

Thanks

Björn

Hi Nicolas, yes totally agree regarding the datastore setup and that is what i suggested in the post. Haven’t tested the VM restart, just verified what happens when the datastore become unavailable. Don’t know if the VM will be restarted when the datastore is offline. As far as i know the ESXi host should fail back to VM home directory when the specific swap datastore don’t have any free space left. Hopefully it fails back the VM home directory when the swap directory is unavailable as well.

//Magnus