VMWare ESX VM Network Aggregation / EtherChannel / LACP

I recently went through a series of tests with our VMWare ESX 3.5 environment to test link aggregation and failover. We tested a variety of Link aggregation methods including LACP, PgaP, and standard etherchannel while experimenting with different settings on the vSwitch.

Here's what the lab consisted of: a Dell R900 with 8 physical nics (3 used for the test), a Cisco 4507 Switch with 2 Gig high speed blades, 2 laptops and 3 Virtual Machines. This would also work on 2 cisco 3750's connected with a stackwise cable. We setup a virtual switch on the ESX Server that was just for Virtual Machine networks with 3 pnics. Our ports are set for trunk mode so we can have multiple vlan's on our VM's. We setup a ping from each of the Virtual Machines to one of the laptops with a command like: "ping 10.0.0.100 -t -w 500" and setup the same from the 2 laptops back to the virtual machines. This way we could see how many packets we lost on each setting change when unplugging a cable from the switch or taking a blade offline.

After quite a bit of testing we found Standard Etherchannel to work the best. With a standard etherchannel setup we would loose between 1 and 3 packets (at the faster retry time) if a network cable or switch blade was brought offline. In my opinion, this was an acceptable behavior, although I would like to see an LACP aggregation running.

Here's what our final configuration looked like:

VMWare ESX vSwitch Configuration: (under vSwitch Properties->Nic Teaming)
Load Balancing: Route based on IP Hash
Network Failover Detection: Link Status Only
(this could be beacon probing depending on your network)
Notify Switches: Yes
Failback: Yes




Switch Config (Cisco 4507 or 3750)
# Set Switch load balance to IP
port-channel load-balance src-dst-ip

# Add port 1/2 to group
interface GigabitEthernet1/2
switchport mode trunk
channel-group 1 mode on

# Add port 2/1 to group
interface GigabitEthernet2/1
switchport mode trunk
channel-group 1 mode on

# Add port 2/2 to group
interface GigabitEthernet2/2
switchport mode trunk
channel-group 1 mode on

# Setup Port Channel Group
interface Port-channel1
switchport switchport mode trunk
spanning-tree portfast trunk

Active Directory Install - Server 2008

Here are my basic steps for an Active Directory Installation using Server 2008. This is of course after a clean install of Windows 2008 Server and running Windows updates. I also like to turn off IPv6 in the networking and create a changelog.txt file in the all users -> startup folder.

Step 1. Configure Network
--- Start configureNics.bat ---
REM *** Configure IP Address
netsh interface ip set address name="Local Area Connection" static 10.0.0.10 255.255.255.0 10.0.0.1 1

REM *** Configure DNS Server (Point to Domain Controller)
netsh interface ip set dns "Local Area Connection" static 10.0.0.10

REM *** Configure WINS Server
netsh interface ip set wins "Local Area Connection" static 10.0.0.9
--- end configureNics.bat ---


Step 2. Rename Server
I then rename the Server to the name of my DC, usually somthing like DC01 or IT-DC01 as I don't like to rename domain controllers after the domain has been created.
--- Start renamecomputer.bat ---
@ECHO OFF
REM - Matt Brown 2008
REM ---------------------------------------------------
REM Rename Domain Controller
REM ---------------------------------------------------
ECHO

ECHO Please set your new computer name:
SET /P newpcname=[New Computer Name]
ECHO Renaming computer from %computername% to %newpcname%
netdom.exe renamecomputer %computername% /newname:%newpcname% /FORCE /VERBOSE
--- END renamecomputer.bat ---

Step 3. Prep Domain Controller
--- START prepdc.bat ---
ECHO *** Install .NET Framework
ServerManagerCmd -i NET-Framework-Core

ECHO *** Install Local and Remote Administration Tools
ServerManagerCmd -i RSAT-ADDS
--- END prepdc.bat ---

Reboot Server

Step 4. Prep Domain Controller Part 2
--- START prepdc-part2.bat ---
ECHO *** Install Local and Remote Administration Tools
ServerManagerCmd -i RSAT-ADDC
ServerManagerCmd -i RSAT-ADLDS
ServerManagerCmd -i RSAT-DNS-Server
ServerManagerCmd -i RSAT-WINS
ServerManagerCmd -i GPMC
ServerManagerCmd -i PowerShell

ECHO *** Install DNS Role
ServerManagerCmd -i DNS
--- END prepdc-part2.bat ---

Step 5. Install DC
--- START InstallDC.bat (run from c:\)---
ECHO *** Install Active Directory Domain Services Role
ServerManagerCmd -i ADDS-Domain-Controller
DCPromo /Answer:"C:\ad_setup.txt"
--- END InstallDC.bat ---

--- START ad_setup.txt ---
[DCInstall]
; New forest promotion
ReplicaOrNewDomain=Domain
NewDomain=Forest
NewDomainDNSName=corp.com
ForestLevel=2
DomainNetbiosName=CORP
DomainLevel=2
InstallDNS=Yes
ConfirmGc=Yes
Sitename=MainSite-001
CreateDNSDelegation=No
DatabasePath="C:\Windows\NTDS"
LogPath="C:\Windows\NTDS"
SYSVOLPath="C:\Windows\SYSVOL"
; Set SafeModeAdminPassword to the correct value prior to using the unattend file
SafeModeAdminPassword=
; Run-time flags (optional)
; RebootOnCompletion=Yes
--- END ad_setup.txt ---

Reboot Server, you now have a functioning Domain Controller.

Create Active Directory Users with Powershell

Here's a quick little script I wrote to create users in Active Directory using Powershell and the Quest extensions for AD.

---- Start Script CreateUsers.ps1 ----
### -----------------------------------------------------------------
### Written by Matt Brown - 12:13 PM 9/17/2008
###
### Powershell script requires a text file with the following fields
### Name,sAMAccountName,First,Last
### Brad,Bradley.J.Pitt,Brad,Pitt
###
### Requires Quest Powershell extenstions for AD
### -----------------------------------------------------------------

# Open the File of User Names and Put it in the Pipeline
Import-Csv "NewAccounts.txt" |

# Loop Through the CSV File, creating accounts
Foreach {
# Set Vars
$StrName = $_.Name
$StrSAMAccountName = $_.sAMAccountName
$StrFirst = $_.First
$StrLast = $_.Last

# Send vars to screen (1 line below)
write-Host "Creating User Account: $StrSAMAccountName - $StrName - $StrFirst - $StrLast"

# Create Account (1 line below)
New-QADUser -ParentContainer "OU=NewAccounts,dc=mydomain,dc=com" -Name $StrName -FirstName $StrFirst -LastName $StrLast -SamAccountName $StrSAMAccountName -DisplayName $StrName -Description "Training Account." -UserPassword 'P@ssword' -UserPrincipalName "$StrSAMAccountName@domain.edu" | Enable-QADUser
}
---- End Script CreateUsers.ps1 ----

And below is the text file used to create the accounts. Run this using the Quest Powershell extensions for AD and you'll be good to go.

---- Start NewAccounts.txt ----

Name,sAMAccountName,First,Last
Brad,BradleyJamesPitt,Brad,Pitt

---- End NewAccounts.txt ----