20Oct/15

CryptoAPI NG Certificate Support for Lync Server

Lync, Skype4B, ,

Simply put, Lync does not support certificates that are issued using the Cryptography API: Next Generation providers.  At the time of this writing, Lync 2010 and Lync 2013 only support certificates that are issued using legacy Cryptography API providers.  To determine if your certificate was issued with CryptoAPI:NG support, use these quick instructions:

Using certutil.exe, you can examine the information about the certificate.

Certutil.exe –v –store my “certificateserialnumber”

A lot of data will be returned, but you need concern yourself with only a single piece of that information.  If you do a search and find Microsoft Software Key Storage Provider, then your certificate has been issued using the new CryptoAPI:NG provider and won’t work with Lync.

CertUtilOutput

If you say to yourself, “So what!?  I’m going to use the certificate anyway!”…  Attempting to assign a certificate that is used using the CryptoAPI:NG providers within Lync will result in an error and the certificate cannot be used:

LyncError-BufferTooSmall
An error occurred: “System.Security.Crpytography.Cryptographic.Exception” “The buffer supplied to a function was too small.”

Personal Thoughts

Having done a number of Lync deployments, I had not run into this particular issue yet and was a bit puzzled at why this was the first time I had seen it.  It took some time and lab work, but was able to consistently reproduce it and determine what was occurring.  Does this issue constitute the end of the world?  No, of course not.  Does it mean Lync is insecure and broken?  Absolutely not.  Read on to calm yourself.

Background

Lync relies heavily on certificates and anyone familiar with it knows that.  Microsoft has taken extensive measures in documenting the certificate requirements for Lync:

http://technet.microsoft.com/en-us/library/gg195673(v=ocs.14).aspx

http://technet.microsoft.com/en-us/library/gg195752(v=ocs.14).aspx

http://technet.microsoft.com/en-us/library/gg195796(v=ocs.14).aspx

http://technet.microsoft.com/en-us/library/gg425950(v=ocs.14).aspx

http://technet.microsoft.com/en-us/library/hh202161(v=ocs.14).aspx

What is not well documented publically is that Lync doesn’t include support for CryptoAPI:NG.  I could not find a single reference anywhere that denotes this in TechNet.  If anyone out there can find this information, please let me know and I will update this post to reflect that.

What is CryptoAPI:NG?

As with anything in IT, things advance.  CryptoAPI:NG is simply that – an advancement of the cryptographic stack that was introduced in Windows Vista and Windows Server 2008.  For details on this, check out this Technet article that gives great details into what advancements are included and this Technet article that describes applications that are currently supported.  There are two important things to remember here:

  1. CryptoAPI:NG was introduced with Windows Vista and Windows Server 2008.  If you have Windows XP and Windows Server 2003, you need not be concerned because those OS’s don’t support it.
  2. While OS support might have been added, applications must explicitly opt-in for support to use the new CryptoAPI:NG features.

For every Lync installation this means that there is the potential to see this issue.  Lync will be installed on Server 2008 or newer, but the chances of seeing this heavily depend on your internal PKI configuration and how certificate templates are configured.

Does my PKI support CryptoAPI:NG?

First of all, only Server 2008 Certification Authorities or newer support CryptoAPI:NG.  If you only have Server 2003 Certification Authorities then you need not worry as you won’t ever see this issue.

Second, the default templates are not configured for CryptoAPI:NG support so you will never see this issue using the default Web Server template.

Third, if you are using custom certificate templates then there is a possibility the template has been configured to require the new CryptoAPI:NG providers.

To determine if the certificate template is configured for CryptoAPI:NG support, use these quick instructions:

On your Certification Authority, open the Certification Authority MMC.

Right-click Certificate Templates and click Manage

Right-click the applicable template and click Properties

Click on the Cryptography tab

If the following settings are checked, then CryptoAPI:NG is configured for the template:

CryptoAPINGSettingsSample
Provider Category – Key Storage Provider
Requests must use one of the following providers
Providers – Microsoft Software Key Storage Provider

How would I get a certificate with CryptoAPI:NG?

Answering this question is the all-important one and turns out to be relatively simple:

If you request certificates using the Lync Certificate Request Wizard or the Lync Management Shell, you won’t.

In all my testing I could never get a certificate using CryptoAPI:NG when using the wizard or shell, even if the CA template required CryptoAPI:NG.  When using the wizard or shell, the certificate always came back with the Microsoft RSA SChannel Cryptographic Provider, which is legacy CryptoAPI.  While it seems odd that the CA would issue a certificate with a provider it isn’t configured for, it does make sense that the certificate wizard and shell would not request it because it is up to the Application to opt-in for CryptoAPI:NG functionality.  Since Lync doesn’t support it, the wizard and shell use the legacy CryptoAPI functionality and everything works.  The only time I could successfully get a certificate using CryptoAPI:NG was by manually requesting the certificate through the Certificates MMC console on the computer.  In that scenario there are no restrictions by an application on what should be used, so the MMC console adheres to what is specified in the certificate template.

Note:  I have seen some reports that public CA’s have issued CryptoAPI:NG certificates, but I cannot verify whether those certificates were requested using the Lync tools.  I also did not test that scenario for this article.

I’ve got a CryptoAPI:NG certificate…now what?

Assume you have a CryptoAPI:NG certificate and you simply cannot request a new one.  It turns out it is possible to use that certificate with a little workaround:

Export the CryptoAPI:NG certificate (with private key) from the Lync Server machine and import it onto a Windows XP machine.  Then export the certificate (with private key) from the XP machine and import it onto your Lync Server machine.  This export process on the XP machine will convert the certificate into using the legacy CryptoAPI provider and will result in the certificate being able to be used on the Lync Server machine.  This MSDN article describes why this conversion occurs.

Again, assume you cannot use the Lync certificate wizard to request certificates and you don’t have an XP machine available to convert the certificate – maybe the organization requires a dedicated team to issue the requests for you and they always use the Certificates MMC.  Getting things working in that case requires the certificate template used to be configured to not require CryptoAPI:NG.  This could be accomplished in two ways:

Option 1

Use the default Web Server template

Option 2

Change the custom CA template so that CryptoAPI:NG is not required

CryptoLegacySettingsSample
Provider Category – Legacy Cryptographic Service Provider
Requests must use one of the following providers
Microsoft RSA SChannel Cryptographic Provider
Microsoft DH SChannel Cryptographic Provider

Summary

Again, to sum it all up:  Lync does not currently support CryptoAPI:NG certificates.  It is entirely possible that a Cumulative Update or the next version of Lync will include support, but for now make sure your certificates are usable by not requiring CryptoAPI:NG!

15Oct/15

Managing the Skype UI in Lync 2013 and Skype4B

Lync, Office365, Skype4B, ,

With Microsoft officially announcing that they will be upgrading Office365 to utilize the Skype for Business back-end, administrators will need to begin to take actions to prepare themselves and their users for the impact of this update.

Note: Since Skype for Business (hereafter, S4B) hasn’t been released to GA yet, this information is still pre-release and subject to change!

A few important things you should begin planning for:

Skype for Business will be provided as an update package to existing Lync 2013 clients

S4B will still remain “lync.exe” from an executable perspective and maintain the same major version number as Lync 2013. This greatly helps admins because Windows QoS policies should not need to be re-tooled and application whitelists will not need to be updated. Microsoft has not yet set a release date on the client update but an official announcement is likely to come soon.

Can I use Lync 2013 with a S4B Server?

The simple answer to this is “Yes!”. Lync 2013 clients will absolutely work when your user account is homed to a S4B pool. Remember that any new features of a S4B pool will not be presented to your user account until you update your client software from the Lync 2013 UI.

How do I control the UI presented to users?

This is a multi-faceted answer but largely boils down to two major points:

  1. If your Lync 2013 client has the latest S4B client update and your user account is homed on a S4B pool, upon first sign-in your client will automatically switch to the new S4B UI.
  2. If your Lync 2013 client has the latest S4B client update and your user account is homed on a S4B pool, you can override the automatic UI behavior by setting the EnableSkypeUI parameter within the Client Policies.

The EnableSkypeUI parameter, when set to $FALSE, ensures that the Lync 2013 UI is always used by any clients connecting to a S4B pool. This parameter is the only method you can use to ensure that the new Skype UI is not presented to users and can be controlled in a targeted fashion to help organizations manage a staged rollout of the new UI. I’ve included a table below that describes the various different combinations of clients, servers, and resulting client UI:

How does this effect Lync Online users?

Microsoft exerts total control over all policies and pools within Lync Online and have begun notifying customers that pending S4B upgrades will be coming within the next 90 days. Some organizations may not be ready to begin rolling out the new S4B UI but because Microsoft controls the pool upgrade process within Office365, there are limited options in controlling the client UI. Lync Online customers cannot customize Client Policies and all current Lync Online policies have a value of NULL for the EnableSkypeUI parameter. With the EnableSkypeUI parameter being NULL, clients will invoke the new UI if they have obtained the latest client update. At the current time there is no other recourse for Lync Online customers to prevent the Skype UI from being displayed, other than restricting the rollout of the latest client updates. I do believe that Microsoft will begin publishing additional client policies to allow organizations to disable the Skype UI, but customers will need to keep examining available client policies within Lync Online to discover which policies will be available:

Get-CsClientPolicy | Select Identity,EnableSkypeUI

What else should I know?

Microsoft continues to update TechNet with information regarding the upcoming Office365 updates. I strongly urge customers to examine the TechNet website for additional information and as always, I’ll update this post (or create additional posts) to reflect new changes as they are announced!

4/1/2015 Update

Microsoft has officially announced that two Client Policies will be available for customers to control the rollout of the Skype UI within Office365:

Tag:ClientPolicyEnableSkypeUI
Tag:ClientPolicyDisableSkypeUI

I strongly urge customers to examine these policies as they may not contain the same settings, such as DisableSaveIM, as the Client Policy you may be using!

9/9/2015 Update

Microsoft now has the following client policies available for customers to control the rollout of the Skype UI within Office365:

Global
Tag:ClientPolicyDefaultPhotoDisableSkypeUI
Tag:ClientPolicyDisableSkypeUI
Tag:ClientPolicyEnableSkypeUI
Tag:ClientPolicyNoIMURLDisableSkypeUI
Tag:ClientPolicyNoIMURLPhotoDisableSkypeUI
Tag:ClientPolicyNoSaveIMNoArchivingDisableSkypeUI
Tag:ClientPolicyNoSaveIMNoArchivingNoIMURLDisableSkypeUI
Tag:ClientPolicyNoSaveIMNoArchivingNoIMURLPhotoDisableSkypeUI
Tag:ClientPolicyNoSaveIMNoArchivingPhotoDisableSkypeUI

10/15/2015 Update

With Office 2016 being released there are some considerable changes to this EnableSkypeUI behavior.  You can no longer switch back and forth between the legacy Lync UI and the Skype UI when the Office 2016 suite is used.  If you are running the Office 2013 suite and April 2015 CU, you can still switch back and forth, but Office 2016 users will notice that the Skype UI will never change.  It is now a hard-coded setting and is enforced regardless of server version.

The Skype4B client, as a part of Office 2016, does still retain the “lync.exe” executable name although the version number of the software has been bumped to 16.0.

30Sep/15

Controlling Skype4B Application Sharing Bandwidth

Lync, Office365, Skype4B, , ,

In a previous blog post I talked about how administrators and architects should place more emphasis on planning for application sharing bandwidth in their Skype4B deployments. Armed with that information, the next logical progression of this blog series continues the focus on application sharing and discusses the available methods within Skype4B to manage and control the bandwidth requirements for application sharing.

General Methods of Controlling Bandwidth

Speaking broadly, there are typically two methods of controlling any sort of application bandwidth across enterprise networks. Both methods are not mutually exclusive and can be used in concert with one another, but it is largely up to network engineers and the application engineers to work together to find the best solution for your environment.

Control the traffic at the network

For most of the network engineers out there, this is the “preferred method”. Like controlling traffic on highways via “normal lanes” and “HOV lanes”, network traffic is separated, classified and handled in a manner that is configured by the network engineers to give preferential treatment (and bandwidth) to high priority traffic while giving normal treatment to non-priority traffic. This is most generally referred to as Quality of Service (QoS). QoS is typically seen in two forms:

  1. Differentiated Services – This is a QoS designation split into two layers of the OSI stack
    1. Class of Service (CoS) – This is a Data Link Layer classification method whereby a 3-bit CoS field is included in Ethernet headers in a 802.1Q VLAN
    2. Differentiated Services Code Point (DSCP) – This is a Network Layer classification method whereby a 6-bit DSCP value is included in the 8-bit Differentiated Services field within the IP headers of traffic
      1. This is typically the more common QoS model and the preferred model today.
  2. Integrated Services – This is a QoS designation where the traffic specification part (TSPEC) and request specification part (RSPEC) help to define and classify the traffic into unique flows. This is not a common QoS model still in deployment.

In either case above, network engineers can control, on a per-hop basis, how each classification of traffic is treated along the entire network path. While flexible and powerful, this method requires engineers to know all the various types of traffic on the network and to classify it accordingly (and ensure it is classified across all devices), which can be an arduous task and result in traffic being incorrectly classified. In addition to the work required, as more and more applications move to SaaS available across the Internet, QoS is lost as the traffic leaves the corporate network and moves across the Internet which restricts QoS from being available from end-to-end.

Control the traffic through the application

For most of the application engineers out there, this is the “preferred method”. Think of this as the “honor system” where some type of built-in application configuration tells the application to limit itself to X bandwidth when utilizing the network. While undoubtedly easy to deploy, this method has no awareness of other traffic around it and no integration with network devices that send/receive traffic, which results in a more limited “one-size-fits-all” approach.

How does Skype4B fit in?

In almost all enterprise deployments, architects and engineers desire to identify the traffic produced by Skype4B and fit that traffic within the available enterprise network configurations. Skype4B offers both of the configurations above and can be configured for one or both to suit the needs of the enterprise network.

Limit Application Sharing Bandwidth Within On-Premises Skype4B Conference Policies

This method is by far the easiest option to limiting application sharing bandwidth within Skype4B. The only built-in method to control application sharing bandwidth is via the Conferencing Policies within Skype4B. By default, the Global policy has no functional restriction on bitrate and is set to 50000.

Image 407

Get-CsConferencingPolicy | select Identity,AppSharingBitRateKb | fl

If you decide to limit bandwidth, two approaches could be taken:

  1. Alter the Global policy – while this approach is easiest, it will impact all users that don’t have a site or user policy assigned to them. As a result, you may limit users to an artificially low bandwidth in certain sites that have sufficient bandwidth.
  2. Create a Site or User policy – this approach allows flexibility in deployment by tailoring bandwidth requirements to a per-site or a per-user basis. As a result, you can limit users in a low bandwidth site to a low bitrate while allowing users in high bandwidth sites to a higher bitrate.
Image 410

Set-CsConferencingPolicy –Identity Global –AppSharingBitRateKb 1000

Image 411

New-CsConferencingPolicy –Identity TestPolicy –AppSharingBitRateKb 1000

Limits of this approach

While the second option above is my preferred option for handling bandwidth natively within Skype4B, it does come with limits. The biggest current limitation is that the AppSharingBitRateKb parameter is handled per-user and only is applicable to the presenter. Where this becomes a challenge is with branch sites that have much lower bandwidth than other larger branch sites with high bandwidth.

AppShareKB-HighBWtoLowBW

In the figure above, when the user in Site B goes to share their desktop with the user in Site A, the effective limitation is based on the user in Site B. With Site A limited to 1.5 Mbps, that could result in 67% of available bandwidth being consumed for a single desktop share. When you add in the possibility of multiple users sharing desktops between the two sites, oversubscription of the 1.5 Mbps circuit becomes a very real risk. Also of note is that this AppSharingBitRateKb behavior is applied both in P2P calls and within multiparty calls.

Impact of this approach with Lync for Mac

Through several of my deployments I’ve noticed there seems to be a significant difference with RDP performance on Mac OS clients, especially the Retina display models, when the AppSharingBitRateKb setting is reduced to a low value. Interestingly enough, the low value doesn’t seem to impact Windows-based clients but users on Lync 2011 for Mac would report that application sharing was largely unusable. In scenarios where there are Lync for Mac clients I would strongly suggest only using the Optimal numbers referenced in my previous post for the AppSharingBitRateKb setting.

Limit Application Sharing Bandwidth Within Skype4B Online Conferencing Policies

Sadly, this is a bit of a bait-and-switch option because it really isn’t an option at all. Microsoft pre-creates and manages all conferencing policies within Skype4B Online and as a result you cannot create new policies or edit existing ones. As a result of this any user accounts that are homed online are restricted to using the AppSharingBitRateKb value Microsoft has defined, which is currently 50000. For all intents and purposes, the application sharing bandwidth is not restricted for Skype4B Online deployments so architects and engineers should carefully plan your network ingress/egress points to ensure sufficient bandwidth is available. If bandwidth restrictions are required in Skype4B Online deployments, you must begin to examine QoS restrictions for each modality.

Limit Application Sharing Bandwidth for On-Premises Skype4B Deployments through QoS

There are a number of articles on the Internet that already talk about this overall process but it boils down to telling the Skype4B client to utilize certain port ranges for each modality and then configuring client/network devices to treat each modality in a certain way by dedicating queues (loosely equal to bandwidth) for each network hop. The pre-requisite to all of this is that individual port ranges must be specified first. The overall process of doing so consists of:

  1. Configure port ranges for clients – Remember that each port range should be unique and not overlap. Additionally, you should have 20 ports per modality.
  2. Configure port ranges for the MCUs – Remember that each port range should be unique and not overlap. Since MCUs handle modalities from multiple clients be very careful reducing the port numbers to a small number of ports per modality. Best practices should be observed here and stick as closely to what Microsoft puts on TechNet.
  3. Configure port ranges for Mediation Servers – Remember that the audio port range should be unique and not overlap any other server port ranges, such as video or application sharing. Since the Mediation Server handles all traffic to/from IP-PBXs and the PSTN, be very careful reducing the port numbers. Best practices should be observed here and stick as closely to what Microsoft puts on TechNet.

Once port ranges are defined, the clients and servers will begin utilizing those ranges for each modality when communicating on the network. With individual port ranges in use, architects can then begin to classify the traffic using QoS using two main methods:

  1. Mark DSCP utilizing Group Policy based QoS policies on Windows clients and servers – This method is typically easiest but requires that access-layer switches trust DSCP markings coming from the endpoints. If switches aren’t configured for mls qos trust dscp, then DSCP markings will be stripped and classification is lost.
  2. Mark DSCP utilizing class-map policies based off the port ranges per modality – This method requires more work to configure every access-layer switch across the enterprise to mark DSCP based on either source or destination port ranges, but is sometimes the only available option if network engineers choose not to trust client/server DSCP markings.

A typical DSCP/CoS classification of traffic is listed below:

Modality DSCP Value CoS Value
Audio 46 6
Video 34 4
SIP Signaling 24 3
Application Sharing 18 2
File Transfer 10 1

Once you have calculated the expected application sharing bandwidth through the Lync Bandwidth Calculator for each site in your environment, network engineers can configure QoS queues appropriately to handle the expected traffic.

Limit Application Sharing Bandwidth for Online Skype4B Deployments through QoS

This method is similar to on-premises deployments but is limited because administrators cannot define the port ranges utilized for Online Skype4B deployments. Microsoft has the following port ranges pre-created for clients that are homed Online:

Source Port Protocol Usage
50000-50019 UDP Audio
50020-50039 UDP Video
50040-50059 TCP Application Sharing and File Transfer

Architects still have the option to utilize Group Policy based QoS policies or class-map policies, but those have two major limitations:

  1. Having QoS in place mostly benefits P2P traffic – In this scenario the application sharing never leaves your corporate network so it can be controlled end-to-end.
  2. Skype4B Meetings and multi-party communications lose QoS upon egressing to the Internet – In this scenario you can maintain QoS up to the point where the traffic leaves your network. This can be helpful on your network, especially for endpoints that may need to traverse the corporate WAN before egressing to the Internet, but you are still at the mercy of traffic patterns on the Internet for a great deal of the hops and thus cannot guarantee QoS end-to-end.

Despite not being able to maintain QoS end-to-end in an Online deployment, you can still classify the traffic into discrete queues to ensure the bandwidth is allocated for application sharing which in turn ensures that application sharing bandwidth doesn’t impact other traffic on your network. Without reinventing the wheel, check out this blog post for information on configuring QoS for Online deployments.

The best option: utilize both application limits and QoS

For every on-premises deployment out there you should absolutely be coordinating application limiting in concert with QoS. In doing so you gain the following benefits:

  1. Predictable application BW limits – by configuring the AppSharingBitRateKb parameter you ensure that a maximum amount of bandwidth will be requested by Skype4B clients
  2. Predictable QoS queue utilization – by having expected bandwidth from the Skype4B team network engineers can map expected bandwidth consumption to available queues.
  3. Better management through improved communication between network and application teams – this one may not seem obvious but the more the two teams talk the less chance there is of misconfiguration and the better the solution is for the end user.
    1. If queues become saturated, should the application reduce bandwidth (and potentially harm the end user performance) or should queues be increased or potentially both options?
    2. If network bandwidth increases at a site, are queues going to be increased and if so, how will that impact the configuration within the application?
    3. Based on usage patterns of existing sites, how should network engineers plan for bandwidth for new sites that come online?

Note: There’s a number of other options here I’ve left out, but in every case the benefits are helpful to both the network team AND the application team.

For Online-only deployments, you should still utilize QoS even though you are limited in configuring the application. Despite granular application control not being available, network engineers can still classify traffic and give a roughly-predictable utilization for each queue of traffic (application sharing included).

But what about Call Admission Control?

An astute reader would point out that Skype has Call Admission Control that can be used to restrict traffic as well, and you would be correct to say that additional control may be available. Call Admission Control is only available for the following configurations:

  1. On-premises deployments – CAC is currently not available for users that are homed Online
  2. Audio/Video modalities – CAC is currently available for audio and video modalities only. Application sharing is not a supported CAC modality today.

If CAC is available for your deployment, configure it in tandem with your QoS configuration. In fact, CAC is my preferred approach to bandwidth limits for audio and video modalities because it allows flexibility per-site AND includes reporting. Conferencing Policy configurations of audio and video are applicable to each user regardless of where the user physically resides on the network, which can result in a user using more bandwidth than may be available if the user is travelling between sites. Since application sharing currently isn’t supported in CAC today, architects need to utilize Conference Policies for the foreseeable future.

25Sep/15

Skype4B App Share – What in the World Are You Doing to My Network?

Lync, Office365, Skype4B, ,

For many of the Skype for Business and Lync readers out there, you may think to yourself, “Hey, I’ve seen a similar title like that before…”, and you would be absolutely correct. During Lync Conference 2014, Lync MVP Jeff Schertz gave a fantastic presentation about “Video – What in the World Are You Doing to My Network” in which he gave a deep-dive into the impact of Lync’s new H.264 SVC video codec and how that impacts network bandwidth across the enterprise. While it is absolutely accurate that video can stress enterprise networks, the often forgotten (and sometimes neglected) truth is that app share traffic in Lync/Skype4B has a far greater impact (in my opinion) to impact overall bandwidth figures. What follows is my attempt not to reduce video planning but to place an equal (and maybe higher) importance on planning for application sharing bandwidth in Lync/Skype4B deployments.

Foundational Concepts of Application Sharing

Application Sharing has existed, in one form or another, since the Live Communications Server days and has received updates and/or changes with each iteration of the product (OCS, OCS R2, Lync 2010, Lync 2013). Some of those largest changes include:

  • Migrating from the NetMeeting protocol (H.323 based) in LCS 2005 to RDP (ITU T.120 based) within OCS 2007 R2 (also Lync 2010/2013 and Skype for Business).
  • Including formal meeting desktop sharing natively within Lync 2010 instead of relying on the LiveMeeting application.
  • Adding a high performance P2P desktop sharing in Lync 2013.

As stated above, application sharing in the current iterations of the Microsoft UC stack are based off the Remote Desktop Protocol. IT administrators across the globe utilize RDP every day for connecting to servers and workstations and are, as a result, very familiar with the overall capabilities offered. At its core, RDP has the following characteristics:

Generally speaking, the RDP protocol is fast, flexible and sufficient to deliver screen sharing across a wide variety of environments. Lync/Skype4B architects should exercise caution however, as there are significant differences in how RDP works and how Lync/Skype4B utilize RDP.

Foundational Concepts of RDP within Lync/Skype4B

While RDP is the foundation of application sharing used within Lync/Skype4B, there are differences in how the UC clients utilize RDP vs how RDP may be available through applications like Remote Desktop Connection.

Lync/Skype4B utilizes application sharing over TCP only

  • RDP is encapsulated into TCP-based RTP packets.
  • TCP is a connection-based protocol.
    • Every single TCP conversation on a network involves a three-way handshake. This handshake is utilized to ensure the remote endpoint is ready to receive data and ultimately delays the start of data transfer at the expense of guaranteed readiness.
  • TCP guarantees data delivery. If data is lost on transmission, the remote endpoint reports back to the sender that data needs to be re-sent – This behavior is great for data but is not ideal for real-time applications.
    • Due to the guaranteed delivery mechanisms of TCP, data is generally sent in “chunks” instead of being sent as a stream (in UDP). This behavior can lead to additional latency and slower performance.
  • UC TCP communications can be multiplexed over a single port.
    • This allows Passive (Viewer) and Active (Presenter) to be done over a single port instead of requiring individual, unique ports in UDP.
  • UC TCP communications can be firewall-friendly by utilizing RTP over TCP port 443 when direct connections between hosts aren’t available.
    • Despite this “friendliness”, beware of application layer firewalls that may try to intercept/manipulate traffic and cause failures in application sharing.

RDP performance isn’t “adjustable”

Almost every IT administrator has seen it within the normal Windows RDP Connection application – a tab called “Experience”. This configuration allows you to tailor the connection for the bandwidth available on your connection to best optimize performance of the RDP session. Lync/Skype4B, however, doesn’t get to take advantage of those configuration settings and instead is limited to utilizing the following protocols/configurations:

  • MS-RDPBCGR
    • RDP encryption must be turned off.
    • RDP Bulk Data Compression must be turned off for data between Viewer and the MCU.
    • RDP Bulk Data Compression must be turned on for data between the Sharer and the MCU.
  • MS-RDPEMC

So how does this impact my bandwidth estimations?

Microsoft (and many others) go to great lengths at describing bandwidth required for Lync/Skype4B but mainly restrict that description for audio/video related modalities:

If you search for information on application sharing bandwidth, you’ll likely come up with very few pieces of information. That information is actually hidden within two pieces of official Microsoft documentation:

  1. Lync 2010 and 2013 Bandwidth Calculator
  2. Network Planning, Monitoring, and Troubleshooting Lync Server

In both pieces of documentation, Microsoft explicitly outlines bandwidth estimations for application sharing traffic:

Table 1 – Application Sharing Bandwidth Estimations

Screen Size Acceptable Optimal
1280×800 384 Kbps 1.5 Mbps
1440×900 512 Kbps 2 Mbps
1680×1050 768 Kbps 2.75 Mbps
1920×1200 1 Mbps 3.5 Mbps

If you look at the info above, a few points can be made…

RDP bandwidth is variable in nature and can have a wide range of bandwidth requirements

There’s no denying that a static, non-moving screen won’t require much bandwidth, but movement of applications and refreshing of screens could require a dizzying 300% (or more) increase in bandwidth requirements. Don’t always assume the lower value is a hard-and-fast-rule – it’s really more an average towards the left side of the Bell Curve. Bandwidth above the average can (and will) occur.

Bandwidth requirements are most largely coupled to screen resolution

The higher the resolution, the more bandwidth required. Simple as that.

Screen resolutions continue to increase (4K or 8K screens, anyone?) and so too will bandwidth requirements

As users continue to receive laptops and/or monitors with 1080p (or better) resolutions, the likelihood of the lower resolutions being used lessens with each generation of products. It is difficult to argue that most users don’t at least have a 1680×1050 resolution today and even more difficult to argue that users won’t have higher resolutions in the future.

What makes it “Acceptable” vs “Optimal”?

Sadly, I cannot give an official answer on this but I can say through various rounds of testing these values that the “Acceptable” numbers provide a decent user experience whilst maintaining lower bandwidth. If you go much lower than those numbers, expect users to complain of slow refresh rates, jumpy application sharing, and/or wholly unusable application sharing sessions. If you want smooth application sharing, then you should plan for the “Optimal” numbers above.

What about that High Performance application sharing you talked about?

The bandwidth numbers above are all based off the basic frame rate of 2.5 fps that is default for all versions of Lync and Skype for Business. When Microsoft added the high performance application sharing capabilities they added the ability to allow a frame rate of up to 10 fps by enabling the appropriate policy configurations. This configuration allows very smooth and fluid sharing sessions but be prepared to pay for it – in my testing I regularly see up to 10 Mbps of bandwidth when the monitors are 1080p resolution.

How is Application Sharing different than Video?

While application sharing is similar to video, there are distinct differences in how the two modalities are implemented within Lync/Skype4B and thus two very different bandwidth patterns begin to emerge. The most obvious difference between the two is that different codecs are utilized, RDP vs H.264 SVC, which does have an impact on overall bandwidth numbers. Despite different codecs being a factor, the most significant difference is due to the intelligence the UC clients have when it comes to video.

P2P video bandwidth is directly proportional to the size of the call window on your screen. P2P application sharing bandwidth is directly proportional to the resolution of the sharer’s monitor.

By itself this may seem like a trivial difference but because Lync/Skype4B only requests sufficient bandwidth to fill the size of the video window, you end up very large differences in bandwidth consumption when the video window isn’t significantly altered and is left at default. For example, assume two users have 1080p monitors and initiate a video call to one another… In that scenario when users accept the video call the window starts out at a default size that will consume roughly 400-500 Kbps per video stream (per Table 2).

Table 2 – P2P Video Bandwidth Figures

Window Size Average Bit Rate Maximum Bit Rate
Default 115 Kbps 499 Kbps
Resized 596 Kbps 814 Kbps
Maximized 1727 Kbps 2768 Kbps
Full Screen 2888 Kbps 4415 Kbps

Assuming the users never resize the video window, initiating an application share during that call will result in an additional 1 Mbps to fulfill 1080p resolution requirements and will vary from 1-3 Mbps during normal application sharing usage such as minimizing/maximizing windows on the actively shared screen (per Table 1). When you begin to compare the two modalities it becomes very obvious how application sharing bandwidth can easily surpass video bandwidth.

Multiparty video bandwidth is directly proportional to the number of users in the conference AND whether application sharing is utilized. Multiparty application sharing bandwidth is directly proportional to the resolution of sharer’s monitor.

Thanks to Jeff Schertz’s presentation it is possible to see that bandwidth changes significantly as more users are added to a video conference, and in most cases the bandwidth drops. The bandwidth reduction is, again, the result of the clients only requesting sufficient bandwidth to fill the size of the video window and due to the fact that each additional user must occupy the same screen real estate as the rest of the video streams which results in lower resolution streams being utilized.

Table 3 – Multiparty Video Bandwidth Figures

Conference Size Average Bit Rate Maximum Bit Rate
2 2128 Kbps 4063 Kbps
3 4050 Kbps 5890 Kbps
4 1304 Kbps 2860 Kbps
5 1224 Kbps 2699 Kbps
6 1565 Kbps 3017 Kbps

When application sharing is added to a conference, the video bandwidth numbers again change due to the changing proportions of screen real estate for each modality. When application sharing takes over the Gallery View video streams are moved to the top of the window and shrink due to the increased presence of the application share.

Figure 1 – Example of Gallery View with Application Sharing

Lync2013GalleryView

In most scenarios involving application sharing within multiparty video conferences, you’ll see video resolutions and frame rates begin to change which results in video bandwidth taking a nosedive.

Table 5 – Multiparty Gallery View Video w/Application Sharing Bandwidth Figures

Conference Size Average Bit Rate
2 250 Kbps
3 375 Kbps
4 500 Kbps
5 625 Kbps
6 750 Kbps

Note: I’m current conducting further testing on these numbers. Please consider these preliminary for the time being.

Assuming again that 1080p resolutions are available to end users, initiating an application share during that call will result in an additional 1 Mbps to fulfill 1080p resolution requirements and will vary from 1-3 Mbps during normal application sharing usage such as minimizing/maximizing windows on the actively shared screen (per Table 1). In certain conference sizes you can have application sharing dwarfing the video bandwidth requirements by 300-400%.

Video usage is a very subjective use case in most organizations. Application Sharing is far more common for most organizations.

In many organizations video simply isn’t that common. Sometimes that is because of technical limitations (such as perceived lack of bandwidth) whilst in other cases it is due to a user culture that simply doesn’t want to utilize video. More often than not I see organizations broadly adopting application sharing whereas video remains a very small deployment. This may change over time, especially with the new generation of Millennials entering the workspace, but the hard truth is that application sharing is used far more frequently in organizations and video is an afterthought or edge case.

Show us the math!

Where all of this becomes apparent is when you start computing numbers within the Lync Bandwidth Calculator. The calculator works off of the “Acceptable” numbers in the table above and includes complex calculations for how users handle video when in a multi-view configuration.

Peer-To-Peer

Assume that you have two branch sites and a separate data center site. Users in each site regularly exchange P2P traffic including audio/video/application sharing. Also assume that these users are huge fans of Lync/Skype4B and that 5% of users in each site are utilizing the features concurrently. Lastly assume that there’s about 300 users in each site, which results in the numbers below:

Site Peak Audio BW (Mbps) Peak Video BW (Mbps) Peak AppShare BW (Mbps)
Site 1 0.51 10.04 17.00
Site 2 0.51 10.04 17.00

61.7% of all bandwidth required is for application sharing! Additionally, that’s only using the Acceptable bandwidth calculation of 1 Mbps for the 1920×1200 resolution monitors the users have! If you start computing numbers based on the Optimal bandwidth calculation it becomes even more lopsided with the Application Share bandwidth tripling.

Conferences

Assuming the same scenario above, the numbers will change significantly when conferences are involved. Much of this deals with the fact that the Lync client and AVMCU actually use less bandwidth for video as each additional user is added to the conference and even less bandwidth when application sharing is invoked.

Assuming the same 300 user count at each site we end up with the numbers below:

Site Peak Audio BW (Mbps) Peak Video BW (Mbps) Peak AppShare BW (Mbps)
Site 1 0.42 5.45 17.00
Site 2 0.42 5.45 17.00

74.3% of all bandwidth required is for application sharing! Additionally, that’s only using the Acceptable bandwidth calculation of 1 Mbps for the 1920×1200 resolution monitors the users have!  The large sucking sound on your network – it’s application sharing, not video.

You’ve proven your point, so now what?

I can’t overstate the importance of properly planning for application sharing bandwidth in addition to the rest of the bandwidth calculations required for Lync/Skype4B. As a result of this it’s important to begin asking the following questions:

  • What’s the most common resolution of monitors deployed within your organization?
  • Do we have sufficient bandwidth to support sharing with that resolution?
  • Do we intend on utilizing Quality of Service with application sharing?
  • How does our user culture view video vs application sharing?

Once you begin to answer the questions above you can begin to properly plan for bandwidth for application sharing. Many customers I work with are often surprised at the numbers because they are so focused on the fear about bandwidth requirements for video, but in reality video bandwidth is likely to be a lesser concern when compared to application sharing bandwidth. In a future article we’ll discuss methods you can utilize to help control application sharing bandwidth for both on-premises and Office365 deployments. Stay tuned!

28Apr/15

Configuring Server Discovery Addresses in Lync/Skype4B Clients for Office 365

Lync, Office365, Skype4B,

Since the days of OCS there has been a method to manually configure Microsoft’s UC clients with a specific server address for internal & external sign-in capabilities.  If the clients were not hard-coded to utilize a certain server FQDN, the UC clients would begin an automatic server discovery process to auto-magically find where it should attempt to register.  If you’re looking for information on how that works, I would suggest starting here:

Office Communicator Sign-In and Discovery

Lync 2010 DNS Requirements for Automatic Client Sign-In

Lync 2013 DNS Requirements for Autodiscover

When you’re in a Hybrid scenario and you want to bypass the on-premises automatic sign-in process (reasons are myriad why you would potentially need to do this – a separate discussion for another time), you can hard-code the server discovery address for both the Lync desktop clients and the Lync mobility clients:

Lync 2010/2013 Desktop

Internal server name:  sipdir.online.lync.com:443
External server name:  sipdir.online.lync.com:443

Lync for Mac

Internal server name:  sipdir.online.lync.com:443
External server name:  sipdir.online.lync.com:443

Skype for Business Desktop

Internal server name:  sipdir.online.lync.com:443
External server name:  sipdir.online.lync.com:443

Lync 2010/2013 Mobile

Internal discovery address:  webdir.online.lync.com/Autodiscover/AutodiscoverService.svc/root
External discovery address:  webdir.online.lync.com/Autodiscover/AutodiscoverService.svc/root

Skype for Business Mobile

Internal discovery address:  webdir.online.lync.com/Autodiscover/AutodiscoverService.svc/root
External discovery address:  webdir.online.lync.com/Autodiscover/AutodiscoverService.svc/root

The “:443” is required for the desktop clients to force registration (and all subsequent communications) to flow over the firewall friendly TCP 443 port.  If you forget, the desktop clients will attempt to utilize TCP 5061, which will result in a sign-in failure.  For the mobility clients, the entire URL string is required.

Again a word of caution here that manually configuring these entries will bypass any automatic client sign-in detection.  If you configure these settings for your Office365-homed account and then migrate back on-premises, your client will fail to sign-in.  Please make sure you understand the ramifications of manually configuring these settings!

31Mar/15

Uncovering SIP trunk High Availability issues through Lync CQM

Interop, Lync, Skype4B,

High availability is crucial for any critical application utilized by an enterprise.  High availability can refer to different things when you ask different IT professionals, but the core of the definition is that a service should be fault tolerant and able to withstand failures without administrator intervention.  Where this becomes a “grey area” is when you start considering how HA crosses sites/geographic boundaries, but we’ll save that conversation for a later date.

When it comes to Lync/Skype4B there are two configurations for High Availability (remember that Microsoft considers HA within a single site or data center only):

I generally recommend DNS load balancing for SIP as it is considerably easier to configure and troubleshoot.  DNS load balancing can be used for Mediation Server pools as well, but unless a PSTN gateway is configured to utilize DNS LB you end up not having HA at all.  With a recent Lync health check I was actually able to identify this lack of HA through the Call Quality Methodology data.

Call Quality Methodology

If you’ve never run this in your Lync environment, then you should.  Here’s the documentation on it.  Now – go forth and exercise your Lync CQM-fu!

Analyzing the CQM Data

The CQM scorecard outputs a bunch of useful data, but I wanted to examine things a little further so I took a look at the Plant_2_Mediation_Gateway CSV file.  This file contains all the streams between a Mediation Server and the trunks associated with the Gateways.

Plant_2_Mediation_Gateway_Clean

Simply looking at the table itself can provide some insight but where the data becomes extra useful is creating a new Pivot Table out of it.  Simply select the data source, create your Pivot Table and voila!

Plant_2_Mediation_Gateway__Pivot_Clean

Once I had the Pivot Table, it took me only a few seconds to identify an issue.  Go ahead and search for it, I’ll wait…  Still don’t see it?  Ok, I’ll help…

Plant_2_Mediation_Gateway__Pivot_Marked

Remember that for each Gateway to connect with a Mediation Server, it utilizes a Trunk in the Lync Topology.  If a Gateway is correctly configured and is utilizing load balancing for calls sent to Lync (either DNS based or potentially IP address based), we should see the trunk equally utilized across all servers in the mediation pool.  In the environment I was examining, calls from a certain Gateway were limited to a single server within the mediation pool.  Effectively the CQM data showed that HA was likely not configured.

Moral of the story

After checking with the customer on this, we were able to 100% corroborate that the SIP provider had only configured a single IP address on their SIP trunks to the Lync mediation pools.  If either of the Lync Front End servers would have gone down, all inbound calls from that provider would have begun failing.  Thankfully, the Lync Front End servers had not gone down, but this allowed us to identify and proactively remediate an issue before it resulted in an outage.

You may have thought that CQM was for call/network stats only, but it has proven to be much, MUCH more useful.  Kudos to Jens Trier Rasmussen and colleagues for this super-helpful tool!

 

29Mar/15

Office 365 Policies for Skype and Lync

Office365, Skype4B, , , ,

7/1/2018 Update – Spreadsheet updated with latest Office365 configuration for Skype for Business Online Policies.  Updated article to include information regarding new policy cmdlet access.

2/22/2016 Update – Spreadsheet updated with latest Office365 configuration for Skype for Business Online Policies

8/17/2016 Update – Spreadsheet updated with latest Office365 configuration for Skype for Business Online Policies

Some of the common questions companies ask when they begin a Skype/Lync Hybrid deployment (or even a Office 365 only deployment) are “How can I customize the environment?” or “What features/capabilities can I restrict for my users?”.  While the questions themselves are logical and seem simple enough, the answer is actually a much more complex discussion that often doesn’t boil down to a simple answer of “X, Y and Z”.

The Dirty Details

Microsoft exterts total control over the policy options available within Office 365.  What this means is that administrators can apply policies within their tenant but they cannot create/edit/delete policies.  This often comes as a surprise to administrators who are used to being able to customize everything when the service is hosted on-prem, but this is the reality of the Office 365 shared infrastructure.  Generally speaking, you’ll be able to run Grant-Cs cmdlets but no New-Cs or Set-Cs cmdlets.  For the complete list of cmdlets available, please see this TechNet article.

In Q1 of CY2017, Microsoft added the ability for customers to create custom policies within their tenants.  This was a huge addition to the service and definitely allows more flexibility in configuration.  While you can create new policies or in some cases modify existing policies, not all parameters are editable by customers.  Some parameters are restricted and Microsoft doesn’t tell you which can be changed.  If you attempt to change a parameter that is not available, you’ll get the ugly red text and have to move on.  Either way, the best practice suggestion is to use the default policies if possible and only create custom policies in the event a business or technical requirement needs to be met that requires the creation of custom policies.

The Proverbial Straw

Where this becomes an issue is that the policies available within Office 365 are not known to administrators prior to Microsoft enabling the tenant.  This results in administrators often going in “blind” and having little to no time to investigate the options available for the service.  It also causes problems for Hybrid scenarios because an organization’s on-prem policies may not exactly match up with the available policy options within Office 365.  In the latter scenario you may find that you have two differing sets of policies depending upon whether your user is homed online or homed on-premises.  While Microsoft does a pretty great job of publicly documenting each of the cmdlets and the various cmdlet parameters, they have no public resources that define what policies are available for the users of the Office 365 Skype/Lync service.

The Fix

Available above is a simple Excel spreadsheet that will help customers discern and track what policies are available for External Access, Conferencing, Client, Mobility, and External Communications.  The spreadsheet is filtered so all it takes is flipping the switches on each of the policy options to begin determining if the various options you desire are available in an existing policy.

Please note that this spreadsheet is only a snapshot in time.  Microsoft will continue to update the available policies within Office 365 so I will update the spreadsheet from time-to-time in reflection of those updates.

For the conferencing policies in the spreadsheet, please remember that you must first determine which policies your tenant has access to by utilizing the following cmdlet:

Get-CsConferencingPolicy -ApplicableTo [email protected]

Your tenant will not have access to all 200+ policies and will only have access to a subset.

HTH!