Saturday, June 21, 2008

Recommended Network setup for VMWare ESX

Here's what I recommend for Network setup for VMWare ESX. ESX Servers should have a minimum of 6 network ports, preferably 8. As you can see from the screen shot below I like to configure 3 nics for my VM Network and my Service Console. I put these on the same vSwitch and have the ports configured for vlan trunking (in a 6 port setup I would give this vSwitch 2 nics). I then setup 2 nics for vMotion traffic (this is preferably on a separate network or vlan). I also like to use NFS for my Virtual Machines and so I setup another 3 nics for NFS Traffic. I keep this on a private vlan that has no gateway and have my NetApp NAS on the same private network. This keeps my NFS traffic secure and tight.

Another thing to note here is that each of these vSwitch configuration contains a physical network port located on a different network card and each of those are routed to a different physical switch. The idea here is we get the maximum amount of availability and reliability of the network. You can also run a esxcfg-vswitch -l command from your ESX console to view your configuration.


You will also notice I have a vSwitch created with no Physical Adapters in it. I setup this with a vLan ID of 1 and then either add a VM with DNS or manually give all the machines in the group an ip in the same range such as 192.168.0.5 / 255.255.255.0. With this setup, you have to remember to keep all the VM's you want to talk with each other on the same ESX Server so they can talk to each other.

This is a great way to take a copy of a Domain Controller and a few Client systems and put them in a test bubble without having to change anything on the DC besides the networking. I use this setup to test changes to my Domains or web servers without touching the production systems until I've thoroughly tested.

6 comments:

  1. I like your setup.
    So your 3 NICs that run to your NAS are obviously teamed and all active... Is that 1 x 3gb Path or 3 x 1 Gb path?

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Hi, it's a 3 x 1 Gb path. What I'm doing is I have 3 NFS mounts on my ESX Boxes and I load balance VM's across the NFS Mounts as well. At this point I haven't run into any storage bandwidth issues with this setup. My NAS unit is a NetApp 3040 with 4 nics teamed in an LACP config.

    More info on my Switch Config / vSwitch configs here: VMWare ESX Network Link Aggregation

    ReplyDelete
  4. what do you do to actually create the lacp trunk ?
    just creating a vswitch with nics ?

    thank you

    ReplyDelete
  5. You also need to set the failover policy on your vSwitch to "Route based on ip Hash". This will provided the quickest failover when a link or switch port goes down.

    ReplyDelete
  6. This whole information is absolutely useful and interesting. i like this blog because is easily understandable, and that is invaluable to the readers. I wanted to buy viagra online, and i saw this blog. I think this information will be useful for me specially because i have had some inconvenient with my network.

    ReplyDelete