I have had numerous questions over the 18 months since i joined Nutanix about my home lab/working environment setup and this blog post will describe the setup and what i have been using my environment for. To give you a short summary it is built on a Nutanix Block and a Nutanix technology partner Arista sponsored 10 Gbps switch.
Before i get into any details the below drawing will give you a pretty good idea of what we are looking at.

Nutanix HCI

Since i started at Nutanix i have had the privilege of having a Nutanix block available at home or now it is actually placed in a datacenter close to where i live. The Nutanix block is a NX-3360, including 3 NX-3060 Nutanix nodes with the following specification

• 2 x Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
• 256 GB RAM
• 2 x 400 GB SSD
• 4 x 1TB HDD
• 2 x 1 Gbps NICs
• 2 x 10 Gbps NICs
• 1 x IPMI
  

The Nutanix block is rebuilt quite often, on average once a month, to allow for testing on all Nutanix supported hypervisors:

• Nutanix Acropolis Hypervisor (AHV)
• VMware ESXi
• Microsoft HyperV

The block is used for running workloads meaning virtual servers and or virtual desktops for testing purposes. The capacity and performance in the block is usually more than enough and really offers a good way of doing the right job.

Management Server

You all know the need for static infrastructure that can live for months or even years. Since i’m rebuilding my Nutanix block quite often i also have a standalone management servers. It’s a Dell PowerEdge T110 II server with the below specification where i run ESXi and a couple of management VMs:

• 1 x Intel (R) Xeon(R) CPU E3-1270 V2 @ 3.50 GHz
• 32 GB RAM
• 2 x 2 TB Disk
  

Based on limited mount of RAM i have enabled all RAM sharing features available in ESXi e.g. Transparent Memory Page Sharing, small memory pages and so on.

Network

To get the best performance you definitely should use the Nutanix Node 10 Gbps NICs but 10 Gbps switches are quite expensive for a home lab setup. I’m very happy to let you know that Arista in Swedenlet me borrow a 10 Gbps Arista 7124S switch my home lab. This is well worth a big shout out since from the moment it was installed i have been able to do testing and tasks that were just not possible before i had the switch.

I’m using the Arista switch only for the Nutanix infrastructure and then connect the switch with a Netgear 1 Gbps switch for connection to the rest of the world. This is a perfect setup for me since it is easy to manage and gives me the the performance i need to do the tests i want to do. This means i got the following network setup:

• Arista 7124S – This is used for 10 Gbps connections between the Nutanix Nodes so i can perform real world testing scenarios.
  
  • For the moment i’m only using 7 of the 24 ports and they are connected to the following devices:
    • Management Port 1 – 1 Gbps connected to the Netgear switch
    • Port 4 – 1 Gbps to the Netgear switch to provide access to and from the Nutanix environment.
    • Port 5 – 10 Gbps to Nutanix Node A eth2
    • Port 6 – 10 Gbps to Nutanix Node B eth2
    • Port 7 – 10 Gbps to Nutanix Node C eth2
    • Port 15 – 10 Gbps to Nutanix Node A eth3
    • Port 16 – 10 Gbps to Nutanix Node B eth3
    • Port 17 – 10 Gbps to Nutanix Node C eth3
• Netgear
  • Provide LAN functionality
  • Provide wireless LAN functionality
  • Connect the LAN to the 10 Gbps switch
  • Connect the Dell server

As you might already know Nutanix works on a 1 Gbps network infrastructure with a limited number of Nutanix nodes but of course performance is not as good as with 10 Gbps. Before Arista decided to let me use one of their switches i used the 1 Gbps infrastructure available and it worked good from a functionality perspective including VLAN and other stuff.
However, now i can do the functionality, availability and performance test i need. The built-in Nutanix diagnostics test basically does I/O and throughput running the diagnostics tests available from within Nutanix, the Controller Virtual Machine (CVM) including:

• Nutanix uses 8 outstanding IO operations and 4KB IO size for Random read and write tests.
• Nutanix uses 1 MB I/O size when running throughput

For the above tests the most obvious improvements using the Arista 10 Gbps switch compared to the 1 Gbps is Sequential writes that gets 5-7 times better throughput.

Running network throughput tests using iperf gets around 10 times the performance which is expected. Using only one of the 10 Gbps NIC on two different Nutanix Nodes and running iperf between the two VMs i get 16 Gbps. In this case its almost a 50/50 split between send and receive traffic meaning 8 Gbps each direction. This performance is consistent meaning i can have the test going for 1 or 60 minutes and get same performance. This is really good and really important for my testing.

The commands i ran were:

• On target VM
  • iperf -s
• On sender VM
  • iperf -c 10.10.100.206 -d -t 60  -P 8

Result came back:

• [SUM] 8.15 Gbits/sec
• [SUM] 7.91 Gbits/sec

I have also looked into the Arista 7150 model which includes among other things the Arista Networks’ Latency Analyzer (LANZ) which allows network administrators to gain real-time visibility into congestion conditions as

Software

For the moment i’m running the following software versions in my lab

• ESXi 6.0 on the Dell Server
• Acropolis Base Software version 4.6 on the Nutanix Xtreme Computing Platform (XCP). Available for Nutanix employees only at this point in time.
• Acropolis Hypervisor version 20151208 on the Nutanix Nodes. Same here, only available for Nutanix employees at this point in time.

Acropolis Hypervisor (AHV)

Just a short note about the hypervisor i run for most of the time in my home lab. The entire concept of the Nutanix AHV is that it is stateless and that don’t require any configuration. The only configuration i might change for customers and in my home lab during testing is the networking part since not all companies runs the same setup.

I’ll discuss the AHV load balancing options in another blog post.