HDX and H.264 Artefacts in Citrix XenDesktop/XenApp

Desktops are made of pixels

Images and desktops that make your screen consist of pixels. Your desktop in the datacenter or on your physical PC can be represented as a bitmap, where all the pixels are represented by 4 values:

• Red
• Green
• Blue
• Alpha (this is transparency)

Each of these values in HDX or recent operating systems is represented by (most commonly) 8 bits, hence why people talk about “32 bit colour”. The Red, Blue, Green, Alpha values are often called RGBA format. Since each colour is 8 bits, each colour has 256 shades, so we can multiply 256 for red, times 256 for green, times 256 for blue and get millions of colours, (256 x 256 x 256 = 16,777,216). Millions of colours are pretty much what’s accepted for a monitor’s colours to look “true” to the human eye.

Representing a desktop or picture as a raw bitmap is expensive but that bitmap does contain all of the original information, although a lot of it essentially redundant to the human eye. There are many compression algorithms available to reduce the volume of data in your desktop with acceptable visual quality for the human eye. I recently blogged about how JPEG compression can do this.

Another very common compression mechanism is H.264 compression.

 

An overview of H.264

H.264 was developed essentially as a codec optimised for the compression of video. This means that like JPEG it also works best on photographic video like data. H.264 is also a delta technology, a base frame is transmitted and subsequently deltas consisting of only the changes to each subsequent frame transmitted. This makes it low on bandwidth but of course there is also a cost to working out the changes between frames particularly within the encoding.

 

YUV and colour sub-sampling

RGB is one format for representing colour used in bitmaps. H.264 is based on a different format, called YUV. RGB does not exactly map to YUV and so H.264 is fundamental and theoretically lossy, and it is impossible to go back to the exact RGB representation in all situations.

In order to reduce bandwidth and data transmission further H.264 is commonly used in a format known as YUV 4:2:0 – this means that in one dimension 4 bits of colour data are represented by 4 bits, in one dimension by 2 and in the other no information is transmitted. This is chromatic (colour) sub-sampling. This relies on the fact that in a video the pixels next to each other usually have some relation to the next and are closely correlated, particularly when dealing with moving images (video again) this provides a good visual experience for the user. Currently most GPU implementations of H.264 are YUV 4:2:0, although many GPU vendors do have some level of or plans for 4:4:4 support.

The worst scenario for most types of compression involving chromatic sub-sampling is sharply contrasting neighbouring pixels and small scale structures with which the human eye is familiar…. i.e. text. Other types of data that suffer particularly are near parallel lines in CAD wireframe drawings.

 

H.264 Supercodec+lossless text

The sharp-eyed amongst you will have noticed that in Citrix XenDesktop and HDX, the standard VDA uses a codec known as the supecodec+lossless text. Desktops and traditional applications such as Microsoft office (Excel, Word, etc) are heavy on text and fairly low on video. As such there is some extra magic on top of Citrix’s H.264 codec that also transmits text at a higher quality or lossless. Again, nothing is free and there is a computational cost, mainly in CPU but a tiny bit in bandwidth too.

Pure H.264 YUV 4:2:0 Artefacts

In my previous blog on JPEG, I described how to recognise Fourier-like sampling artefacts associated with excessive compression. H.264 4:2:0 artefacts are different though as they are dominated by chromatic sub-sampling i.e. sharp edges between highly-contrasting neighbouring pixels are slightly smudged. Coloured lines on a black background are slightly dimmed (the black has crept into the lines and the colour of the lines leaks into the black background). Additionally near parallel edges on CAD wireframe models become slightly less well defined.

Original Lossless Bitmap:

H.264 YUV 4:2:0 encoding of the same CAD part:

You can exaggerate and highlight the effects of pure H.264 YUV 4:2:0 on text and lines, when using an encoder based on that technology) by selecting a block of cells in an application like Excel and colouring the cells in Red, the increase in contrast between the text and the background makes the artefacts more perceivable for most people.

JPEG compression can also use and suffer from chromatic sub-sampling but it’s usually the Fourier-like spatial artefacts that dominate (see my blog, here, for a fuller description and examples).

One easy way to visually tell which mode is being used is to load up the Windows 7 default background wallpaper in a session, and look at the artefacts on the red/blue boundary of the flag. H.264 compression introduces tell-tail visible artefacts that look like very clear-crisp aliasing (vertical or horizontal lines) on the red-blue boundary. JPEG tends to introduce more of a smudging effect to that border. Similarly gradients tend to become visable as a mosaic-like effect (these are the H264 Macroblocks becoming visible to the naked eye).

How to work out what encoder you are using and submitting good quality bug reports with Citrix technologies

In my work at Citrix I sometimes see support calls that describe symptoms and the settings the user _thinks_ they have applied. Often this information is incomplete or vague and can slow down a solution or lead to a user implementing a workaround that avoids the issue but doesn’t necessarily leave the user on the best possible configuration. The graphical modes and codecs used depend upon:

• The receiver versions and capabilities and settings on the receiver
• The hardware of the client end-point (the PC or the tablet the user is using)
• The policies set for the server
• The current network conditions – HDX will adapt when bandwidth is constrained

To work out what encoder and graphics mode you are using and to check that the policies you have set have been correctly applied (this eliminates faults such as race conditions when policies are set, checks that there are no bespoke (unsupported) registry keys interfering with the policies being fully applied, user conditions. You should check your system using the advice in:

• Citrix Support Article: CTX200370
• If you are submitting an issue you should always collect and submit the wmic results from the command line or the XML report from the export functionality of HDX Monitor (as detailed here).

Legacy mode and Thinwire Plus technologies from Citrix use JPEG and Bitmap technologies, so if you are seeing JPEG artefacts then you will not be using H.264 mode.

Folklore and Workarounds

I’ve occasionally seen some strange workarounds with H.264 technologies. Problems with missing text or resolution quality are usually indicative of a configuration issue, I see this occasionally in double hop scenarios (where an app is published and decoded within a desktop then re-encoded to before going to the user’s final end-client device).

I’ve seen users fiddle with the registry keys to change the encoder type, something Citrix doesn’t recommend; in particular the encoder key within the registry HKLM\Software\Citrix\Graphics. Since this has become semi-common knowledge I thought it best to clear up exactly what is being done is this key is changed. It may prove a useful debug tool, but isn’t something I’d recommend for production systems.

• Encoder = 2 is Pure H.264 (YUV 4:2:0). As with most vendors this is H.264 4:2:0 format, it’s designed for a balance of quality and bandwidth primarily on video and high-bandwidth CAD parts (not much text). This is used by the HDX 3D Pro VDA.
• Encoder = 1 is H.264+lossless text. This is used by default by the XenDesktop standard VDA and XenApp VDA.
• Encoder = 0 forces you to use Compatibility mode

I’ve highlighted some Excel cells in red to emphasise the effects of different encoders:

Here is encoder=2, what you will get by default on the HDX 3D Pro VDA, or any pure H.264 YUV 4:2:0 implementations from any vendor:

Next here is encoder=1, where lossless text has been implemented. In this case a few artefacts remain on the lines between cells.

Hardware encode

I often get asked about hardware encode for H.264 to reduce the CPU cost by offloading it to the GPU, for pure H.264 this would be a no-brainer. However to ensure high quality visuals on text, it gets a lot harder as currently GPU vendors and hardware often only support YUV 4:2:0 encoding.

Desktop users are very perceptive to text and ensuring the text is transmitted to a high enough quality is the hard part of the problem. Hardware YUV 4:4:4 is coming through from many vendors but it does have higher bandwidth demands than YUV 4:2:0, never-the-less I’d expect to see more options from GPU and remoting vendors leveraging YUV 4:4:4 and in the future H.265 too. YUV 4:4:4 vs. YUV 4:2:0 can often double the bandwidth requirement.

Visually lossless and Build-to-visually-lossless

In XenDesktop 7.6, with the release of the windows 4.2 Receiver in Q4 2014, Citrix introduced a software YUV 4:4:4 implementation. Its uses are fairly niche given the other options available (e.g. Supercodec+lossless text) but it has enabled some architectural futures. If you are interested in the visual quality differences between YUV 4:2:0 and YUV 4:4:4 it might be worth having a play with these graphical modes via the policies.

How to configure Visual Lossless?

Visual Lossless and Visual Build lossless can be configured via group policies from DDC only.

Steps:

1. Go to policies and search for “Allow visual Lossless compression”. Set this policy to “Enabled”
2. Search for policy “Visual Quality” and set this policy to “Always Lossless” for Visual Lossless or set “Build to lossless” for Visual build to lossless
3. Make sure you have 14.2 Windows Receiver or higher on the client
4. Inside ICA session launch HDX monitor and go to “Graphics – Thinwire Advanced” and check for below to verify if the policy has applied.
5. Visual Lossless: Encoder should be set to “DeepCompressionEncoder” and under WMI tab, Policy_VisualQuality should be set to “Lossless”
6. Visual Build to Lossless: Encoder should be set to “DeepCompressionEncoder” and under WMI tab, Policy_VisualQuality should be set to “Visual build to Lossless”