NVIDIA GRID vGPU: 512MB profiles, Win 10, framebuffer – new support article

I’ve been a bit quiet on the blogging front over the last month or so – a long vacation coupled with a slight role change – but also working on some internal support documentation around 512MB profiles which is now published.

As ever the NVIDIA Enterprise Support KB articles are living documents and you should check the main document here: http://nvidia.custhelp.com/app/answers/detail/a_id/4238/ at the time of reading for up-to-date NVIDIA sactioned advice.

However I thought it would be nice to highlight the availability and reproduce the article as it stands today. The search facility on the KB system is reasonably good and it’s always worth checking for new articles and searching for answers: http://nvidia.custhelp.com/app/home/.

The latest article gathers together some support and technical advise around 512MB including some additional advice on the use of small profiles with Win 10, as well as links to VMware and Citrix best practice on workign with Win 10.

The KB article as published is below, this was the work of a number of people within NVIDIA but also incorporated feedback and work with our NGCA (NVIDIA GRID Community Advisors), especially Rasmus Raun-Nielsen whose customer review and perspective were particularly helpful.

NVIDIA GRID vGPU: Memory exhaustion can occur with vGPU profiles that have 512 Mbytes or less of framebuffer

Symptom/Error

Symptoms and errors that may occur:

  • When playing video content (full screen 1080p) in a browser the session hangs and session reconnect fails on 512MB profiles.
  • This issue typically occurs when multiple display heads are used with Citrix XenDesktop or VMware Horizon on a Windows 10 guest VM.
    • When this error occurs, the NVIDIA host driver reports Xid error 31 and Xid error 43 in XenServer’s /var/log/messages file.
    • Or on VMware When this error occurs, the NVIDIA host driver reports Xid error 31 and Xid error 43 in the VMware vSphere log file vmware.log in the guest VM’s storage directory.

Root Cause

There is a known issue associated with changes in the way recent Microsoft Operating systems handle and allow access to overprovisioning messages and errors. NVIDIA is working with Microsoft closely to resolve these issues ongoing. Users with correctly provisioned systems should not encounter issues. As such users need to take care to ensure there is sufficient frame buffer to support their uses.

512MB is a very small framebuffer and as such users should be aware that the multiple demands made in a virtualized environment can lead to memory exhaustion. Uses that place demand on the framebuffer:

  • Use of more recent Microsoft OSs that place more demand on the framebuffer e.g. using Windows 10 rather than Windows 7. Windows 10 demands far more resources
  • Use of multiple monitors
  • Use of higher resolution monitors
  • Use of the framebuffer for hardware protocol encode (NVENC) – to reduce the probability of users encountering issues NVENC has been disabled for 512MB in the GRID 4.0 (August 2016) release for protocols such as Blast Extreme (VMware) and Citrix HDX/ICA.
  • Frame buffer intensive applications

Documentation

This issue is documented in the driver release notes for the GRID 4.0 (August 2016) release. Customers are advised to always read the known and resolved issues lists contained within the driver release notes for each release for their hypervisor (links below), for Citrix XenServer the release notes state:

Memory exhaustion can occur with vGPU profiles that have 512 Mbytes or less of framebuffer. This issue typically occurs when multiple display heads are used with Citrix XenDesktop or VMware Horizon on a Windows 10 guest VM. When this error occurs, the NVIDIA host driver reports Xid error 31 and Xid error 43 in XenServer’s /var/log/messages file.

The following vGPU profiles have 512 Mbytes or less of frame buffer:

  • Tesla M6-0B, M6-0Q
  • Tesla M10-0B, M10-0Q
  • Tesla M60-0B, M60-0Q
  • GRID K100, K120Q
  • GRID K200, K220Q

GRID Release notes for Citrix XenServer: http://us.download.nvidia.com/Windows/Quadro_Certified/GRID/369.17/XenServer-6.5/367.43-369.17-nvidia-grid-vgpu-release-notes.pdf

GRID Release notes for VMware ESXi: http://us.download.nvidia.com/Windows/Quadro_Certified/GRID/369.17/ESXi-6.0/367.43-369.17-nvidia-grid-vgpu-release-notes.pdf

Verifying framebuffer usage

Users can follow the advice in this article on monitoring NVIDIA GRID framebuffer usage  (http://nvidia.custhelp.com/app/answers/detail/a_id/4108/~/monitoring-the-framebuffer-for-nvidia-grid-vgpu-and-gpu-passthrough) to assist correctly size their environment with respect to framebuffer usage to avoid memory exhaustion. The advice is also of use to assess whether issues users are encountering are actually caused by memory exhaustion.

Consequences

To reduce the consequence of users encountering issues NVIDIA have disabled NVENC on 512MB profiles with the GRID 4.0 (August 2016 release) to minimize the risk of users encountering the Memory exhaustion. Application GPU acceleration remains fully supported and available with all profiles including 512MB. NVENC support from both Citrix and VMware is a recent new feature and as such the majority of users on older versions should encounter no change in functionality.

Workarounds and Solutions

Windows 10

Microsoft Windows 10 has significantly increased the demands upon graphical resources such as GPU framebuffer above older OS releases, as well as on other non-graphical system resources. As such both Citrix and VMware have published tools and configuration advice as to how users can reduce resources. Customers using Windows 10 are encouraged to consider following advice from virtualization vendors.

  • VMware: VMware have provided an OS optimization tool for Horizon View which can make and apply optimization recommendations for Windows 10 and other OSs. Users of Citrix/other virtualisations stacks may find this tool useful for the recommendations made even if they cannot then use the automated configuration tools. The tool can be found here: https://labs.vmware.com/flings/vmware-os-optimization-tool
  • Citrix: Citrix consultant Daniel Feller has published a number of articles on Windows 10 best practice and configuration many of which will also be relevant to VMware / other virtualization stacks. See: https://virtualfeller.com/?s=windows+10+optimization

Some users will find that a 512MB is inappropriate for their Windows 10 workload and that a 1GB profile is more appropriate.

Support

NVIDIA customers with support who believe they are encountering issues as a result of frame buffer memory exhaustion should raise a support case with NVIDIA Enterprise Support via https://nvidia-esp.custhelp.com and can reference issue #200130864.

Applicable products

NVIDIA GRID vGPU

GRID GPUs including M60, M6, M10, K1, K2

VMware Horizon and ESXi

Citrix XenDesktop and XenServer

Users are most likely to encounter this issue if using:

  • heavy graphical or video workloads
  • recent more graphically intensive Microsoft OSs e.g. Windows 10 rather than Windows 7
  • small framebuffers e.g. 512MB
  • Remoting protocols leveraging NVIDIA NVENC hardware encode e.g. recent versions of Citrix HDX/ICA or VMware Blast Extreme

Disclaimers

This Web site contains links to Web sites and third-party tools controlled by parties other than NVIDIA. NVIDIA is not responsible for and does not endorse or accept any responsibility for the contents or use of these third party Web sites or tools. NVIDIA is providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement by NVIDIA of the linked Web site. It is your responsibility to take precautions to ensure that whatever tools or information you select for your use is free of viruses or other items of a destructive nature.

NVIDIA GRID: More info on vApps and VPC/vWS Licensing

lukeblog
Check out Luke Wignall’s blog on NVIDIA GRID licensing and other GRID topics!

I wrote a blog on RDSH (including XenApp) licensing and the options available with NVIDIA GRID vGPU and GPU-passthrough a few weeks ago, which you can read – here (including support for multi-monitor and resolutions). Since then my colleague Luke has added some more information in a blog where he outlines various case studies including many on vApps, which is worth a read here:

Luke answers how many licenses and what type you will need for various use cases, answering questions such as:

  • Q: I am deploying Citrix XenDesktop for 5000 global users, using two data centers, to meet a follow the sun productivity goal.  The data centers are also backup sites to each other.  I expect at most 1200 users at each of our three regional areas to be on during their workday, connecting to their closest data center, but there is some overlap (people working late or starting early) so I am architecting with a buffer for a total of 1500 virtual desktops.  I need to be able to run all users from either data center of one should go down. My users are all engineers and their apps require Quadro.
  • Q:  I am deploying virtual desktops but using XenApp to do so, and am looking for improved end user experience, for 1000 users.  At any given time I expect no more than 850 users to be connected.  I have no other desktop delivery method.
  • Q:  I chose to run XenApp on a bare metal host, so no hypervisor (I would question the decision to forgo the flexibility and manageability of virtualization), delivering three Microsoft Office applications so .  I have 500 users but expect no more than 350 of them to be connected at any given time.  I have no Virtual desktops for these users.
  • Q:  I have 250 engineers using CATIA and similar apps, they must have Quadro drivers, but usually only 200 of them are working at any given time.  I also have 1000 knowledge workers that range from sales to support, their apps do not need Quadro but perform much better with GPU (=happy users), of those I typically see 800 actively on their desktops.  I am deploying VMware Horizon.  We have a set of web apps that all 1250 employees use for time keeping, expenses, and safety training, these I am delivering with XenApp.

 

There is a lot of information on GRID licensing in our knowledge base – just search on “GRID licensing” on our KB home page here:

Highlights include:

Licensing Documentation:

Of course one of the best references is the official licensing guides on the GRID resources page (under deployment guides) here: http://www.nvidia.com/object/grid-enterprise-resources.html. In particular these two are useful:

 

Questions

Any questions – ask below or on the support NVIDIA GRID forums at https://gridforums.nvidia.com

NVIDIA GRID – Citrix offers multi-monitor NVENC hardware encode for both Linux and Windows

A few weeks ago I wrote a KB article outlining the support on various virtualization stacks for hardware encode support.using NVIDIA NVENC functionality.  I’m now in the process of having the KB article updated as today Citrix announced with XenDesktop/XenApp 7.11 that they have added support for Windows alongside their existing Linux support.

A few vendors already offered hardware encode for both Linux and Windows such as NICE DVC and HP RGS, however prior to this release from Citrix:

  • Citrix only supported Linux
  • VMware only supported Windows (and then only single monitor)

This sees Citrix now offer multi-monitor support on both Linux and Windows. I expect VMware will now be under some pressure to match.

Many customers use both Windows and Linux so this parity makes life much easier for the system administrator.

VDI can use GPUs in many ways:

  • To accelerate the applications
  • For the protocol encoding and decoding e.g H.264 e.g. NVENC
  • Allowing protocols access to the frame buffer directly

So solutions without NVENC also do still benefit from GPU availability.

You can read about the HDX/XenDesktop 7.11 enhancements here:  https://docs.citrix.com/en-us/xenapp-and-xendesktop/7-11/hdx/gpu-acceleration-desktop.html

There are also some other big leaps in HDX such as the addition of hardware H.264 4:4:4 encode (removes the artefacts of standard H.264 4:2:0 but at a cost of a bit more bandwidth).

Nicely any Citrix Receiver that also supports H.264 decoding can be used with NVENC hardware encoding (H.264 4:2:0). Which should ease users trialing and adopting.

The protocol war rages on!

NVIDIA GRID – RDSH licensing (including XenApp)

I’ve had a few questions about what licensing is needed under the GRID 2.0 and up software licensing for the M60/M10/M6 GPUs for RDSH solutions such as XenApp. I think the confusion arises because it’s possible to use a number of GPU/vGPU different profiles for a server OS VM. The key point is to remember that the licensing is always per user.

Continue reading NVIDIA GRID – RDSH licensing (including XenApp)

NVIDIA GRID: Citrix HDX adds support for Relative and Absolute Mouse Modes to Linux Receiver

Update: 14th September 2016 – Receiver For Windows 4.5 released today now adds support for Windows in addition to Linux!

Just a quick blog to highlight the availability on the Citrix HDX/ICA protocol of a feature enabling Relative Mouse mode. This is a particularly interesting for many NVIDIA GRID vGPU and graphical users as it enables better behavior of certain gaming like applications, particularly those favoured in federal simulations (battle and flight-simulators) e.g Bohemia Simulations VBS 2 & 3. Before on certain application without using an addition gamepad device the mouse could behave strangely ending up with the user pointing their barrel at the ground or sky. Continue reading NVIDIA GRID: Citrix HDX adds support for Relative and Absolute Mouse Modes to Linux Receiver

When to start your tweets with a “@”! Why some people add a dot e.g. “.@rhbbse” to tweets

dottweet.pngThe quirk of twitter

To avoid personal chatter spreading and swamping others twitter put in a clever yet obscure feature to ensure those tweets you want to broadcast spread and one-on-one chatter dies a quick death in the twitterverse.

If you send a tweet starting with a handle i.e. an @ is the first character:

  • “@rhbbse you are awesome”

It will get seen in news streams by me and anyone who follows BOTH of us, “mutual friends”

If you want the world to know though you have to avoid the @ e.g.

By adding a “.” in front or some other text e.g. “Hey world” the tweet will go to all your followers and not just those that we share.

This also applies when retweeting others’ tweets which is why you often see a “.” added.

References

You can read more about this quirk here:

http://www.adrianjock.com/dot-in-front-of-twitter-handle/