Demonstrations were shown of application launch times on a Macbook Air, and comparisons made to ultrabooks (frequently SSD based) with a lot of analysis of the amount of IO these operations created. The net of the conversations was that the usual 10-20 IOPS we talk about for VDI sizing really won't give a comparable experience.
I found this message to be very interesting. There is a lot that goes into an overall user experience with storage performance being only one facet. The point was made though that if we talk about using VDI or RDS environments to to support BYO type solutions the experience of what we delivery to the users will always be compared to what their own device (the one the user bought) delivers.
With this in mind I set out to determine how to decide if a virtual desktop was capable of delivering as good (or better) of a user experience as what people were buying today.
The first problem was settling on a benchmark. How to best evaluate the experience a system is able to deliver? One that I could compare across many systems and wouldn't raise eyebrows as I tried to collect it. I chose the windows experience index (found in Windows 7 and 8.0 - but gone in 8.1) primarily for convenience but also because it rates each subsystem which makes it easier to decide what portions of a desktop are faster or slower than others.
With this decision made I went out and visited my local sources of consumer (BYO) laptops. After explaining myself to the staff at Best Buy and Costco I surveyed 35 laptops and ultrabooks available off the shelf. I also tossed in my own workstation class laptop and SSD based Macbook Pro (running fusion) into the mix. It's important to note that Windows 7 experience scores range from 1.0 to 7.9, and Windows 8.0 scores range from 1.0 to 9.9. This means that any score over 7.9 reflects a Win8 machine and that any scores of 7.9 probably need to be evaluated as "maxing the scale" on a Win7 system. As a note on the topic the price range of the surveyed systems ran from a low of $230 to a high of $1100.
Here's what I found -
- The "average" of the systems I sampled provided a base score (lowest subscore) of 5.2, with most systems falling between 4.4 and 6.0.
- Processor scores averaged 6.7 with most falling between 5.6 and 7.8.
- Memory scores averaged 6.9 with most falling between 5.9 and 7.9.
- Business Graphics scores average 5.4 with most in a range of 4.6 to 6.2.
- Gaming graphics averaged 6.4 with most in a very tight range of 6.1 to 6.6.
- Disk IO scores averaged 6.0 with most falling into a range of 5.6 to 6.3. This one is actually very interesting in that nearly every system sampled reported a score of 5.9, with a single exception reporting 8.1. Which strongly suggests that a "typical" laptop hard disk produces a 5.9.
- Systems with a base score of 5.9 were typically constrained by their disk system, where those with lower scores typically had graphics as their lowest performing statistic.
Now it was time to go off the lab and see what kind of scores my virtual desktop VMs could provide.
In fairness for disclosure, my lab is running vSphere 5.1 with both XenDesktop 7.1 and View 5.2 (5.3 dropped after my research was complete). I have not looked at how a different hypervisor impacts these scores and obviously different CPU memory and HW manufactures will play in as well. The virtual desktops in play were all running on the same Intel E5 2.0 Ghz 6-core phyical host connected a variety of storage platforms. Notable participants are an iSCSI array with 24 10K drives. A single MLC SSD drive; and an Atlantis ILIO diskless appliance. This system also has an nVidia K2 card configured to support both GPU Passthrough (vDGA) and GPU Sharing (vSGA) under vSphere.
So what kind of scores did I get from the virtual desktops (knowing that these are VM based scores and do not take into account anything associated with the display remoting protocols).
Base scores range from 2.0 to 7.2 depending on the configuration of the desktop. So potentially from quite a bit lower than the typical laptop to better than most.
Looking at the subscores -
- Disk benchmarks for the VM's ranged from 5.9 to 7.9 (on Windows 7) - never worse than a laptop.
- Gaming graphics range from 2.5 to 7.9 - often worse than a laptop but also potentially better.
- software rendered 3D by vSphere or the Citrix VDA turned in scores from 2.5 to 3.1.
- shared hardware 3D (vSGA) turned in scores from 5.8 to 6.0 (slightly below the laptops).
- dedicated hardware 3d (vDGA) turned in scores of 7.9 (well above the laptops).
- Business graphics range from 2.0 to 7.9.
- software rendered 3D by vSphere turned in scores at 2.9 and 3.0.
- software rendered 3D by Citrix's VDA turned in scores at 2.0.
- shared hardware 3D (vSGA) turned in scores of 5.9 and 6.0 (better than an average laptop).
- dedicated hardware 3d (vDGA) turned in scores of 7.9 (well above the laptops).
- Memory scores for the VM's range from 5.5 to 7.9.
- Processor scores ranged from 5.5 to 7.2 (remember I'm on a 2.0 Ghz core here).
However - there are some important considerations here. It's also quite possible for a VM to be considerably lower performing than the average laptop. Most noteably on the graphics scores where the worst laptop turned in a 4.3 an the worst VM turning in a 2.0. Even the best software rendered graphics only turn in a 3.0.
So, if you want to give a "better than physical" or even an "as good as physical" experience you can do it on SSD, or an Atlantis ILIO, or even on spinning disk, but you cannot do it without a hardware GPU. Your disk choices will be more about scaling and how many desktops can share the subsystem before performance degrades. CPU and memory scores will be about the choices you make in architecture and speed. If your users are buying 2.8Ghz cores, you should be too.
So my final conclusion - your user's laptops (and phones, and tablets) all have hardware GPU's; your virtual desktops should have them too.
