The Windows Server Performance team have
done a really interesting post on how to optimize network performance
inside of virtual machines by increasing the size of the VMBus buffers
used by our network adapters.
You can found it there
http://blogs.technet.com/winserverperformance/archive/2010/02/02/increase-vmbus-buffer-sizes-to-increase-network-throughput-to-guest-vms.aspx
Under load, the default buffer size used the by the virtual
switch may provide inadequate buffer and result in packet loss. We
recommend increasing the VM bus receive buffer from 1Mb to 2Mb.
Traffic jams happen every day, all across the world. Too many
vehicles competing for the same stretch of road, gated by flow control
devices like stop signs and traffic lights, conspire to ensnare drivers
in a vicious web of metal and plastic and cell phones. In the
technology world, networking traffic is notoriously plagued by traffic
jams, resulting in all sorts of havoc, including delayed web pages,
slow email downloads, robotic VOIP and choppy YouTube videos. (Oh, the
humanity!)
Virtualized networking can be complicated, what with the root and
child partitions relaying packets across the VM bus to reach the
physical NIC. The VM bus, anticipating contention, uses buffers to
queue data while the recipient VM is swapped out or otherwise not
keeping up with the traffic. The default buffer size for WS08 R2 is
1Mb, which provides 655 packet buffers (1,600 bytes per buffer).
The hypervisor, meanwhile, calculates a scheduling interval, or
quantum, derived from the system’s interrupt rate. The hypervisor
attempts to ensure every VM has a chance to run within that interval,
at which time the VM wakes up and does whatever processing it needs to
do (including reading packets from the VM bus). At very low interrupt
rates, that quantum can be nearly 10ms.
Whereas the native system handles on the order of 260,000 packets/second, virtualized systems, in some scenarios,
can—in the worst case scenario—begin seeing packet loss under traffic
loads as low as 65,500 packets/second. This isn’t an inherent tax
incurred by virtualizing or a design limit; rather, it’s the result of
specific characteristics of server load requiring more VM bus buffer
capacity. If the logical processors hosting the guest partitions are
receiving very few hardware interrupts, then scheduling quantum grows
larger, approaching 10ms. The longer scheduling quantum results in
longer idle periods between VM execution slices. If the VM is going to
spend almost 10ms asleep, then the VM bus’ packet buffers must be able
to hold 10ms worth of data. As the idle time for a VM approaches 10ms,
the maximum sustainable networking speed can be calculated as:
655 default packet buffers / ~10ms idle interval = maximum 65,500 packets / second
We can increase throughput, though, by increasing the amount of
memory allocated to the buffers. How much should it be increased? On
paper, 4Mb is the maximum useful size; a 4Mb buffer provides about 2600
buffers, which can handle 10ms’ worth of data flowing at approximately
260,000 packets per second (the max rate sustainable by native
systems). In reality, depending on the workload, the VM’s swapped-out
time probably doesn’t approach the maximum 10ms quantum. Therefore,
depending on how frugal you want to be with memory, increasing to 2 Mb
is probably adequate for most scenarios. If you’re living large in the
land of RAM, lighting your cigars by burning 4Gb memory sticks, then go
for broke, cranking the buffers up to 4Mb.
The buffers are allocated from the guest partition’s memory and
updating the buffer size requires, per each guest VM, adding two
registry keys. To increase the buffer size, we first need the GUID and
index associated with the network adapter. In the guest VM, open the Device Manager, expand Network Adapters , right click Microsoft Virtual Machine Bus Network Adapter and choose Properties (if
you have an a driver marked “(emulated)”, you should take a detour to
install Integration Services from the VM’s Action menu, then add a new
synthetic network driver through the VM setup. See http://technet.microsoft.com/en-us/library/cc732470(WS.10).aspx , step 3 for instructions).
On the Network Adapter Properties dialog, select the Details tab. Select Driver Key in the Property pull-down menu as shown in figure 1 (click the images to see a version that’s actually readable):
Record the GUID\index found in the Value box, as shown in figure 1, above. Open regedit and navigate to: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{GUID}\{index} as shown in figure 2:
Right click the index number and create two new DWORD values, entitled ReceiveBufferSize and SendBufferSize (see
figure 3). These values measure the memory allocated to buffers in 1Kb
units. So, 0×400 equates to 1,024Kb buffer space (the default, 640
buffers). In this example, we’ve doubled the buffer size to 0×800, or
2,048Kb of memory, as shown in figure 3:
Your
workloads and networking traffic may not need increased buffers;
however, these days, 4Mb of RAM isn’t a tremendous amount of memory to
invest as an insurance policy against packet loss. Now, if only I could
increase a few buffers and alleviate congestion on my daily commute!
Tom Basham
Virtualization Performance PM, Windows Fundamentals Team