16Dec/15

A Detailed Look at Skype4B PSTN Conferencing in Office365

December 16, 2015Office365, Skype4BLicensing, PSTN Conferencing, ScriptingTrevor Miller

Update 3/2/2016 – Added new information on countries where PSTN conferencing license purchase is now available.

On December 1, 2015 Microsoft officially added PSTN conferencing as a native feature for Skype for Business Online customers. For a number of years PSTN conferencing was offered through Audio Conferencing Provider integration in Office365 but this marks the first time that Microsoft has included this feature natively. There are significant differences in how the ACP provider integration works and how the Microsoft native provider works, so be sure to check out those differences to make sure your business and technical requirements are met. For most customers, however, the Microsoft Bridge (aka Microsoft PSTN Conferencing Provider) will provide the needed features and do it for a lesser cost than other services in the market.

Licensing

Licensing is always one of those difficult conversations but people always ask… I won’t provide cost breakdowns here, as Microsoft may provide you different cost structures based on your existing Enterprise Agreements, but what I will provide is some background on how the PSTN conferencing feature fits in with the overall Office365 licensing structure for Skype for Business.

License Bundle Info

Option 1 – Purchase E5 licenses for your end users

The PSTN conferencing feature is included within the E5 suite purchase, so all you need to do is purchase the E5 suite, assign the E5 suite to your users and assign the PSTN Conferencing add-in license to your users. Users now have access to Skype for Business Online, PSTN Conferencing and any additional features available through the E5 suite.

Option 2 – Purchase PSTN Conferencing Add-In licenses for your E3 and E1 users

If you have existing E1 or E3 suite purchases, you would need to purchase only the PSTN Conferencing Add-in license for any users that will be hosting PSTN conferences. Users now have access to Skype for Business Online, PSTN Conferencing and any additional features available through the E3 or E1 suites.

Option 3 – Purchase PSTN Conferencing Add-In license for your Skype for Business Online Standalone Plan

This option will likely be less common, but if you have a Skype for Business Online Standalone Plan, you can purchase the PSTN Conferencing Add-In license for those users. An important distinction is that the Standalone Plan must be Plan 2; Plan 3 is not supported.

Licensing Availability

Not all tenant regions can purchase the PSTN Conferencing license. Microsoft will continue to add tenant regions during CY 2016, but you will need to check and see if your tenant region is currently available. Current regions include:

Country/region

Austria

Belgium

Brazil

Bulgaria

Canada

Croatia

Czech Republic

Denmark

Estonia

Finland

France

Germany

Greece

Ireland

Italy

Japan

Luxembourg

Netherlands

Norway

Poland

Portugal

Romania

Singapore

Slovakia

Slovenia

South Africa

South Korea

Spain

Sweden

Switzerland

United Kingdom

United States

PowerShell Licensing

Many folks out there handle licensing of Office365 accounts through PowerShell scripts and the structure of the PSTN Conferencing license is a huge improvement to what is offered for the configuration of licensing for E1 or E3 or E5 suite components. While assigning the base Skype for Business Online license is done through the E-suite license, the PSTN Conferencing license is actually a separate SKU within your tenant. As a result, licensing AccountSkuIDs look like this:

AccountSkuID
------------
UCVNEXT:STANDARDPACK
UCVNEXT:ENTERPRISEPACK
UCVNEXT:MCOMEETADV

The MCOMEETADV AccountSkuID is the license for the PSTN Conferencing Add-In. Assigning this AccountSkuID to a single user account within Office365 is a simple PowerShell command:

Set-MsolUserLicense -UserPrincipalName [email protected] -AddLicenses "UCVNEXT:MCOMEETADV"

For bulk adds, you can utilize a CSV file containing UPNs and assign licenses to your users en-masse:

Set-MsolUserLicense -UserPrincipalName $_.userprincipalname -AddLicenses "UCVNEXT:MCOMEETADV"

Remember that all users receiving the MCOMEETADV AccountSkuID license must have the Skype for Business Online license previously applied to the account. Trying to assign the MCOMEETADV license will result in failure if the account does not already have the Skype for Business Online license.

As an aside…if you aren’t handling licensing through Powershell…you should be. Stop assigning licenses through the Office365 Admin Portal and automate!

Local Number Availability

One of the major differences between the Microsoft Bridge and the ACP Bridges are that the range of local number availability for Microsoft is a bit smaller. This is a temporary limitation that will expand as Microsoft adds their reach into other countries across the globe, so be sure to check what countries are currently available. Current regions include:

Country/region
Argentina
Australia
Austria
Belgium
Brazil
Canada
Chile
China
Columbia
Costa Rica
Czech Republic
Denmark
Estonia
Finland
France
Germany
Hong Kong
Hungary
India
Ireland
Israel
Italy
Japan
Mexico
Netherlands
New Zealand
Norway
Philippines
Poland
Portugal
Romania
Russia
Singapore
Slovak Republic
Slovenia
South Africa
South Korea
Spain
Sweden
Switzerland
Taiwan
Thailand
Turkey
United Kingdom (UK)
United States (U.S.)

For any of the regions above, you can assign a local number Microsoft provides within that region as the default PSTN Conferencing number for a user account. The number you assign will then be added to the user’s Skype for Business meeting invites automatically:

PowerShell Configuration

There are several PowerShell cmdlets that will give you additional information about PSTN Conferencing configuration within your tenant. Some important ones:

Get-CsOnlineDialInConferencingServiceNumber

Set-CsOnlineDialinConferencingBridge

Set-CsOnlineDialInConferencingUser

A few samples of these in action:
Obtaining Dial-In Numbers

Get-CsOnlineDialInConferencingServiceNumber | Select-Object City,Number

Setting Default Tenant Dial-In Number

Set-CsOnlineDialinConferencingBridge -Name "Conference Bridge" -DefaultServiceNumber 17739228035

Assigning Dial-In Number to User Accounts

Set-CsOnlineDialInConferencingUser -Identity [email protected] -ServiceNumber 17739228035 -BridgeName "Conference Bridge"

Current Feature Limitations

While the PSTN Conferencing Service is fairly robust, there are some limitations that may impact rollouts and usage of the service. Microsoft maintains an authoritative list of those limitations but a few notable ones are included below.

Limited DTMF Support

At current, only two DTMF commands are supported:

*6 to mute/un-mute
*1 for help

This is a significant reduction when you compare DTMF commands that are available for on-premises deployments, but commands will continue to be added within CY 2016.

No Private Conference ID Support

Each meeting you create will have the same PSTN conference ID.

This can become an issue if you have back-to-back meetings and users start to join the second call early. You could potentially lock the meeting so that new users can’t join but this is a change from on-premises deployments where you can have unique conference IDs automatically created for each meeting request you create. Microsoft has indicated that private conference IDs are expected to be available in early CY 2016, so this limitation will hopefully be short lived.

No Toll-Free Support

Toll-free numbers are not currently available for PSTN Conferencing in Office365. This may be a concern for some customers that utilize toll-free calling with their current PSTN conferencing provider but I wouldn’t expect too many customers having a problem with this. Microsoft is looking at adding toll-free numbers to PSTN Conferencing, but limited information is available with how that may change either the pricing structure or feature configuration.

Maximum User Support

All Conference Policies within Skype for Business Online maintain a limit of 250 users within a conference and this then limits the total number of PSTN users that can join a Skype4B Online meeting. For customers that have large meetings with more than 250 users, this will be a potential problem with no current solution. Skype Meeting Broadcast may be an alternative here, but since Meeting Broadcast doesn’t provide PSTN integration it may not be a true alternative. Be sure to properly gauge your meeting sizes to ensure that you don’t leave users unable to join due to maxing out the number of users in a conference.

Limited Reporting

Office365 reports are somewhat limited in capabilities for Skype for Business and may not provide all types of reports required. Some organizations may required advanced reporting capabilities around cost center bill-back or detailed conference roster reports that currently aren’t available within Office365. Since Microsoft provides a RESTful API access to gather detailed reporting data via Excel, organizations may be able to build reports to fill in the gaps but I anticipate that not all requirements will be able to be met.

30Nov/15

A look at the new Cloud features for Skype4B in Office365

November 30, 2015Interop, Lync, Office365, Skype4BCloud PBX, Meeting Broadcast, PSTN Calling, PSTN ConferencingTrevor Miller

Beginning December 1, 2015, Microsoft will be officially rolling out a number of new features within Office365 that significantly enhance the Skype for Business workloads. These features have been in preview form for a few months now, but December 1 marks the date where it becomes generally available (GA) for all tenants within Office365, across the entire globe. What’s new, you ask? Let’s take a quick look…

Cloud PSTN Conferencing

For years, on-premises Skype4B/Lync deployments could integrate a PSTN dial-in number to access meetings generated by Skype4B/Lync Server. When Office365 initially came to the scene, PSTN dial-in number integration was only offered only through third-party Audio Conferencing Providers such as AT&T, PGi, or Intercall. Those ACP providers offered a wide breadth of local numbers but the feature set was limited, required separate ACP administration, and required additional costs above and beyond your existing Microsoft licensing costs.

Starting today, Microsoft now offers the dial-in service natively and best of all, Microsoft is making it very attractive from a licensing perspective. 42 local dial-in numbers are currently available, spanning APAC, EMEA, LATAM, and NOAM, with more regions and countries coming online during CY 2016. If you were looking for a reason to ditch your current dial-in provider and save on your TCO for conferencing, you should definitely take a look at what Microsoft is offering.

Skype Meeting Broadcast

For years, on-premises Skype4B/Lync deployments were limited to hosting meetings of up to 250 users in a single meeting. Large Meeting support was added within Lync Server 2013 to allow up to 1,000 users in a single on-premises meeting, but that support came with certain limitations & restrictions that often made it not feasible to implement. Office365 meeting support has always been limited to 250 users in a single meeting…until now.

Skype Meeting Broadcast is a component of Office365, Skype for Business Online, and Azure Media Services that lets you organize and host meetings of up to 10,000 attendees. Skype Meeting Broadcast is primarily targeted for town-hall style events where there is a one-to-many presentation that may include audio, video, recording, Bing Pulse, Yammer, and PowerPoint deck sharing. The service is offered digitally only, meaning you can only access the event through an HTML5 capable device – PSTN dial-in is currently not offered. Skype Meeting Broadcast is free, which means you could easily enable this feature for your Office365 tenant and replace other broadcast services with Microsoft’s free offering!

Cloud PBX

Cloud PBX is actually a compilation of several topologies, one of which that has existed as far back as Lync Server 2010. The differences in the various topologies isn’t always clearly communicated, but the basic tenant of Cloud PBX is that instead of using on-premises Skype4B/Lync servers for call control, you can utilize Office365 servers for call control. The differences really boil down to whether you already have Skype4B/Lync servers on-premises and where you desire the PSTN access to come from. I’ve tried to boil down the different options below:

Option 1 – You Have On-Premises Skype4B/Lync Servers and Want to Keep PSTN On-Premises

If you have existing servers, you can utilize Cloud PBX by using a feature called Cloud PBX with On-Premises PSTN Connectivity (aka – Hybrid Voice). What this means is that you can move users to Skype4B Online, and still allow those users to access the PSTN through your existing on-premises Skype4B/Lync Server deployment and whatever IP-PBX or ITSP your deployment may be already connected to. From the user’s perspective, nothing really changes (including your telephone number), but from an administrator perspective, you’ve offloaded users from your on-premises Skype4B/Lync Servers to Office365. There are some feature limitations here so be sure to check out what is available and what is not to see if this solution is a good fit for your users.

Option 2 – You Have On-Premises Skype4B/Lync Servers and Want to Move PSTN to the Cloud

If you have existing servers and you want to deprecate your on-premises PBX environment, you can utilize Cloud PBX by using a feature called Cloud PBX with PSTN Calling. What this means is that you can move users to Skype4B Online and allow those users to access the PSTN through Microsoft via Office365. PSTN Calling requires you to port your PSTN numbers from your existing on-premises provider, such as AT&T, Sprint, Verizon, etc, to Microsoft or request new numbers from Microsoft and assign those new numbers to your cloud users. The whole porting process is easily captured within the Office365 portal and can be centrally managed as such. Administrators will rejoice in that you’ve offloaded users from your on-premises Skype4B/Lync Servers to Office365 and offloaded PSTN access from your on-premises infrastructure as well, but at the end of the day you still maintain the same seamless integration and communication between your on-premises users and your cloud users. There are some feature limitations here so be sure to check out what is available and what is not to see if this solution is a good fit for your users and your environment.

Option 3 – You Have No On-Premises Skype4B/Lync Servers and Want to Keep PSTN On-Premises

Prior to a month ago, this wasn’t even an option, but the recently announced Cloud Connector Edition (aka – MinTop, or Minimum Topology) allows customers the ability to integrate on-premises IP-PBXs or ITSPs with their Cloud PBX users. While Microsoft will say “no on-premises server deployment”, the reality is that this solution implements on-premises servers, just in a special and unique way. This particular solution is still very new and has several feature restrictions that don’t make it viable for all organizations. Based on what I’ve seen thus far, there is a very small set of organizations that may benefit from this approach, but due to the current technical restrictions of CCE, it may be a while before this becomes a truly viable solution.

Option 4 – You Have No On-Premises Skype4B/Lync Servers and Want to Move PSTN to the Cloud

If you don’t have existing servers and you want to deprecate your on-premises PBX environment, you can utilize Cloud PBX by using a feature called Cloud PBX with PSTN Calling. This particular option is essentially the same as Option #2 above – port your numbers or obtain new ones from Microsoft. Your reliance in this scenario is solely on Microsoft itself which will provide Skype4B services along with PSTN access, as there are no on-premises integrations required for this approach. There are some feature limitations here so be sure to check out what is available and what is not to see if this solution is a good fit for your users and your environment.

Wrapping it Up

These new features significantly enhance the collaboration capabilities offered within Skype for Business Online and make Office365 attractive for a much larger audience. If you were holding off on the cloud prior to now, these features may be the tipping point that gets you to start thinking “cloud first” instead of “on-premises first”. All of these features will be enhanced over the coming months, so start thinking about your strategy to utilize these services to not only save yourself money, but improve collaboration across your organizations.

22Nov/15

Important Windows Server 2012 IIS ARR Hotfix for Lync Reverse Proxy Deployments

November 22, 2015Lync, Office365, Skype4BMobility, Patching, Reverse ProxyTrevor Miller

With Forefront TMG discontinued and Forefront UAG not supported for all Lync scenarios (hint, hint: mobility), customers will be increasingly looking at deploying ARR to serve as their reverse proxy solution for Lync. The good news is that ARR is fully capable and supported for the reverse proxy role; the bad news is that ARR is still relatively new for this scenario and not all issues are well known or easy to uncover. While working on a Lync 2013 deployment with a customer I ran into an interesting ARR issue specific to mobility clients. All other mobile clients worked but when iOS users attempted to sign-in via their Lync 2013 mobile client they received a “We can’t connect to the server…” error and were unable to successfully sign-in.

To verify I didn’t have a mobility misconfiguration in the environment I used the Lync Connectivity Analyzer to test things out. The results from LCA showed that the minimum requirements passed, which should indicate the ability to utilize mobile devices:

Completed tests for the Mobility (MCX) service.

Your deployment meets the minimum requirements for Lync mobile apps.

Additionally, I was able to successfully browse to all the required external Lync URLs via Internet Explorer using an external PC so I knew that the ARR URL rewrite rules were successfully configured. Despite everything passing and appearing to be externally available, iOS users continued to be unable to sign-in via the Lync 2013 mobile client, so I had to dig a bit deeper.

My first additional examination was to look at the IIS logs on the ARR server to verify there were no unusual HTTP responses or errors during the client login sequence. When looking at the IIS logs I could see that requests were sometimes receiving an HTTP 502 response (Bad Gateway). Based on the user agent header in the ARR logs those 502 requests seemed to correspond with the iOS Lync 2013 client, so I went sleuthing on the Internet for some assistance. My Bing-Fu must have been strong that day because in my searching I was able to come across a TechNet post that closely matched the issue I was seeing. Even more helpful was that the post contained information for a hotfix for Server 2012 ARR – KB2732764:

Hotfix for Microsoft Application Request Routing Version 2.5 for IIS7 (KB2732764) (X64)

This hotfix addresses a problem in Application Request Routing Version 2.5.  When ARR is installed on Windows Server 2012 it fails to proxy Windows authentication.

After downloading the hotfix and installing it on the Windows Server 2012 ARR server, iOS clients were able to connect successfully! Eventually this hotfix might be automatically downloaded and installed when you run through the ARR installer (via the IIS Web Platform installer), but for now make sure you include this particular hotfix in your ARR builds to avoid any issues with Lync 2013 mobility clients when using ARR for your reverse proxy.

18Nov/15

SIP 404 Not Found Error with Skype4B, Exchange, and Cisco Call Manager

November 18, 2015Exchange, Interop, Lync, Office365, Skype4BCisco, SIP Trunking, UMTrevor Miller

This is one of those “long overdue” posts that inevitably comes back to rear its ugly head from time to time. If you are deploying Lync Server, Skype for Business Server, or Exchange Unified Messaging with Cisco Unified Communications Manager as the IP-PBX, then read on.

So What’s the Fuss?

In the Microsoft UC world, everything is defined as fully qualified domain names (FQDNs), such as server01.corp.domain.com. In a Lync\Skype topology, everything from pools to web services to trusted applications to SQL servers to file shares to…you get it…it’s all defined as an FQDN. Exchange UM is a bit of a two-headed beast in that you really only need FQDN when you have a SIP-based UM dial plan, for the other dial plan types there isn’t a requirement. Where this can wreak havoc is when you tie these two in with CUCM in specific configuration instances that result in SIP 404 Not Found call failures between the Microsoft UC stack and CUCM. Those specific instances are defined below.

Lync\Skype4B

Within Topology Builder, if you define a PSTN gateway with an FQDN and that gateway is a Cisco UCM publisher or subscriber, you may see call failures from Lync\Skype4B to Cisco UCM.

Exchange Online Unified Messaging

Within Exchange Online, you must define an IP Gateway as the FQDN of your Session Border Controller’s external DNS name. For call flows inbound to Office365 you likely will not see issues, but for any REFERs or outbound calls from Office365 to CUCM you will see call failures.

Understanding Root Cause

The larger root cause is the difference in how the Microsoft stack wants FQDN within SIP communications and how Cisco defaults to utilizing IP addresses within SIP communications. When a Microsoft UC component sends a call outbound to CUCM with an FQDN in the SIP headers, it results in Call Manager performing additional checks on the FQDN to be sure that it “owns” the FQDNs and domains being utilized. If Cisco isn’t configured to recognize the FQDNs and domains that may be in use, it ultimately responds with a SIP 404 Not Found.

A sample 404 failure for Exchange Online below:

REFER sip:[email protected]:5061;transport=tls SIP/2.0
Via: SIP/2.0/TLS 207.46.198.124:5061;branch=z9hG4bKvvkpi32068ohrnv0d6e1sl0000g00.1
FROM: <sip:[email protected]>;tag=abe3165fe3
TO: <sip:XXXXXXXXXXXXXXXX@noam-us-tn-sbcext.widgets.com>;tag=1c1791622239
CSEQ: 1 REFER
CALL-ID: [email protected]
MAX-FORWARDS: 69
CONTACT: <sip:207.46.198.124:5061;ms-opaque=9a2ce5bced0e440e;transport=tls>;automata;text;audio;video;image
CONTENT-LENGTH: 0
EXPIRES: 600
REFER-TO: <sip:81226511@noam-us-tn-sbcext.widgets.com;user=phone;phone-context=dialstring>
REFERRED-BY: <sip:[email protected]>;ms-referee-uri="sip:[email protected]"
USER-AGENT: RTCC/5.0.0.0 MSExchangeUM/15.00.0918.000
P-ASSERTED-IDENTITY: <sip:[email protected]>

NOTIFY sip:172.18.33.73:5060;transport=tcp SIP/2.0
Via: SIP/2.0/TCP 10.131.26.115:5060;branch=z9hG4bK4921365f8681
From: <sip:[email protected]>;tag=95235~22aa7d2d-0b7a-48e4-8665-567787f9955d-33592509
To: <sip:[email protected]>;tag=1c1785660217
Call-ID: [email protected]
CSeq: 103 NOTIFY
Max-Forwards: 70
Date: Tue, 15 Apr 2014 02:09:59 GMT
User-Agent: Cisco-CUCM9.1
Event: refer
Subscription-State: terminated
Contact: <sip:[email protected]:5060;transport=tcp>
P-Asserted-Identity: <sip:[email protected]>
Content-Type: message/sipfrag;version=2.0
Content-Length: 23
SIP/2.0 404 Not Found

Workaround for Lync/Skype4B

The easiest workaround is to simply utilize IP addresses as the PSTN gateway name instead of using DNS names:

If you have already defined the PSTN gateway as an FQDN and utilized it in Voice Routes, you’ll need to add a new PSTN gateway to topology first. Then update your Voice Routes to add the new PSTN gateway and remove the original PSTN gateway. Then remove the original PSTN gateway from your topology.

Note: Using IP addresses, while easiest, may not be future proof as Microsoft changes and adapts Skype for Business. Additionally, utilizing IP addresses as PSTN gateway names impacts the ability to utilize SIP/TLS and SRTP with approved gateways in certain scenarios because you can’t issue a TLS certificate when the subject name is an IP address. Caveat emptor!

Workaround for Exchange Online

There really is no workaround. Exchange Online uses FQDN through and through and the only way would be to re-write SIP headers using your Session Border Controller so that the SBC sends IP addresses to Cisco whilst leaving FQDN on the call legs to Office365.

Global Workaround

To handle both scenarios without changing any existing Microsoft UC configurations, you can simply update the Cisco UCM configuration and bask gleefully in the fact that you can now use FQDN within SIP messaging.

In Cisco UCM, go to System>Parameters

Find the Clusterwide Domain Configuration section

In Organization Top Level Domain, enter your top level DNS domain – eg, widgets.com

In Cluster Fully Qualified Domain Name, enter your top level DNS domain or use a wildcard to specify your top level DNS domain plus any child domains – eg, *.widgets.com

Save the Changes

Restart Call Manager Cluster Services

Note: Restarting CUCM Cluster Services is production impacting. Be sure to have appropriate change requests and outage windows defined.

Paying It Forward

This information is actually out there on the interwebs already, but it is often difficult to find if you don’t really look for it. Some of the other articles out there that talk about this are below:

https://supportforums.cisco.com/discussion/11646656/sip-trunk-lync

http://voipnorm.blogspot.com/2011/08/using-lync-with-multiple-direct-sip.html

Also note that this may not be the only reason you get a 404 Not Found with Cisco integrations, but it is often a very strong starting point that will likely get you one step closer.

15Nov/15

SIP 500 Error with Lync Server and Exchange Online UM

November 15, 2015Exchange, Lync, Office365DNS, Patching, UMTrevor Miller

Configuring Lync Server 2010/2013 for integration with Exchange Online for UM is a relatively straight forward process. You need a Lync Edge server, federation enabled, proper external DNS SRV records, the appropriate Hosted Voicemail Policy configuration applied to the on-premise Lync environment, and the Exchange Online UM dial plan configured appropriately. Despite having all that configured, I ran into an interesting issue with a Lync 2010 deployment that prevented calls from being routed to Exchange Online UM.

The Lync 2010 servers were all running the October 2012 Cumulative Update and the Lync 2010 clients were all running the latest Cumulative Update from July 2013. Every attempt to dial voicemail directly from the Lync 2010 client resulted in a failure, and an odd SIP 500 error in the tracing logs:

Peer: lyncedgepool01.corp.widgets.com:5061
Message-Type: response
Start-Line: SIP/2.0 500 The server encountered an unexpected internal error
From: “xxxx, xxxx”<sip:raj@widgets.com>;tag=ba26f8512a;epid=955dceba2a
To: <sip:raj@widgets.com;opaque=app:voicemail>;tag=2C5FF02E42C3DDCA8E3E0A5C47158DA3
CSeq: 1 INVITE
Call-ID: 100dd855adb04bfbaeafe56658380950
Via: SIP/2.0/TLS 172.20.18.68:56578;branch=z9hG4bK2415C5DE.E7A3270075FF503C;branched=TRUE;received=172.20.1.14;ms-received-ort=56578;ms-received-cid=5CF6F00
Via: SIP/2.0/TLS 172.17.98.94:54425;ms-received-port=54425;ms-received-cid=A3FC600
ms-diagnostics: 1008;reason=”Unable to resolve DNS SRV record”;domain=”widgets.com”;source=”ocsaccess.widgets.com”
Server: RTC/4.0
Content-Length: 0
ms-edge-proxy-message-trust: ms-source-type=EdgeProxyGenerated;ms-ep-fqdn=Lyncedgepool01.corp.widgets.com;ms-source-verified- user=verified

What made this odd is that this type of DNS SRV check should not be performed for outbound calls to Exchange Online, especially when this check was for the primary SIP domain configured within Lync. It was very obvious that this issue revolved around the federation logic within Lync so I went round and round on this, double-checking configuration after configuration to verify that I didn’t have something incorrect. Federation was definitely working externally and everything else checked out OK, so I had do to some extra digging.

After consulting with a colleague it turns out we ran into a similar, but not exact issue with a different client. In that issue, the resolution was to simply install the latest cumulative update, which at the time was the March 2013 CU. Following the CU update their issue was resolved, so I requested the installation of the July 2013 CU on my customer’s environment. My client installed the July 2013 CU on their Lync 2010 front-end servers and edge servers and immediately after, voicemail calls began to flow as expected and integration with Exchange Online UM was working. Success!!!

Moral of the story: patches sometimes contain undocumented fixes so don’t ever think a new patch wouldn’t potentially solve an odd issue! Stay as up to date as possible and if issues arise, it might be time to update!

10Nov/15

Logitech PTZ Pro Camera Review

November 10, 2015Interop, Lync, Office365, Skype4BDevices, VideoTrevor Miller

Video has, and continues to be, a huge push in the Unified Communications market. The difficulty in this video-centric reality is that very seldom do video solutions provide a one-size-fits all approach. Polycom, for example, has a wide array of products to meet video needs from small rooms to large rooms, but it often requires special know-how and overall higher costs for implementation. Skype has ushered in the era of cheap and easy web-cam based communications, but those cameras seldom do good for setups that are more than just a person sitting in front of a computer monitor. Intermediate room systems (such as LRS) aim to add additional collaboration features, such as touch screens, to conference room systems, but they often are still utilizing smaller web-cameras for video which limit the quality and don’t sufficiently handle medium sized rooms. The difficulty for many organizations then is how to effectively integrate video into their conference room environments without requiring the capital outlay costs and custom installation experience that is required for many solutions but still offering premium quality and doing so without breaking the bank.

What’s a PTZ Camera

There are many different versions of pan-tilt-zoom cameras out there today, many of which tie into security systems, production camera studios, video classrooms or existing video teleconference systems. All of these cameras allow the ability, through motors built into the base, to move the camera through varying degrees of position so that the focus can be moved as the subject moves across the field of vision. The biggest difficulty with many of these PTZ cameras is they are typically $1,000 or more for HD quality and they seldom easily integrate with external systems unless those systems are H.264 or H.263 IP-based video distribution systems. To compound issues, those video based distribution systems are often siloed in design and implementation, especially when concerning Skype for Business integrations, so rarely is there a good way to integrate the flexible Skype4B approach into a system which simply wasn’t designed nor built for the cloud-first, mobile-first world we now live in.

What’s Different about the Logitech PTZ Pro Camera?

Imagine you could take all the best things about a PTZ camera and put it into a device that was USB based. Now imagine that device would be compatible with any software based Unified Communications solution such as Skype for Business, Skype, WebEx, Jabber, etc. Now imagine that device is available to you for less than $1,000. That’s the power of the Logitech PTZ Pro Camera – suddenly high quality room based conferencing is available to many organizations for a reasonable cost when compared to traditional video teleconference room systems.

What’s Included?

Camera
Remote Control
10 foot USB cable with integrated power adapter connection
Power adapter
Wall mount/Table base
Quick start guide

Installation Process

The overall installation process is so deceptively simple you would think that it should be more difficult, but the truth is it actually should be this easy. In fact, the on the inside of the lid of the box, Logitech describes just how easy it is:

Step 1: plug the custom cable into the back of the PTZ camera base

Step 2: plug the power cable into the power connector on the camera cable

Step 3: plug the USB cable into an available USB port on your computer

Step 4: Fire up your Skype for Business client, set the video device for PTZ Pro Camera and GO!

Real World Use

PTZ Pro Cam – Range of Field View

C930e – Range of Field View

PTZ Pro Cam – Close Up View

C930e – Close Up View

PTZ Pro Cam – Max Zoom

What’s Hot

I’ve had many people notice a positive difference in video quality since switching over to the PTZ Pro Cam. They all claim the video is crisper, more detailed, and has better color – and that’s compared to my C930e web-cam. The Carl Zeiss optics definitely go a long way here.
Even though the camera is USB based, it still allows you flexible deployment scenarios with physical camera location within a room by utilizing Cat5 or Fiber USB extenders from Extron, Creston, or BlackBox. If you’ve got a computer stashed away, you can still use those extenders to get normal USB connectivity from wherever you decide to place the PTZ Pro Cam to the location of the PC. I consider this a huge win for do-it-yourselfers and custom installers that will integrate rooms with Skype for Business in new and unique ways.
10X optical zoom is awesome, especially for rooms that may be longer than wider.
The camera motors are mostly silent.
The overall build quality is good. You can tell the device is designed for enterprise use and is built accordingly.
The USB plug-and-play experience is top notch. It is no different than any other Skype for Business certified device – literally plug it in and go.
Logitech provides a dual-purpose table stand and wall mount, no separate mount purchase required.
The included remote is easy, simple and no frills. It just works and is easy to understand.
The price is right. $799 MSRP will please many IT budget holders and is easily justified when compared to other room setups or travel costs for employees.
The far end control app for Windows based systems allow custom control without requiring RS-232 integration.
The audio solution is up to you and can be one of many existing audio solutions from Jabra, Plantronics, ClearOne, Polycom, etc. This truly is a DIY kind of solution and is infinitely flexible because of it.

What’s Not

Autofocus is somewhat hit or miss at times. The autofocus generally does well and is fast, but occasionally in certain lighting scenarios it may go through a cycle of lose/re-gain that can be somewhat annoying.
While Logitech provides the table stand and wall mount, the metal of the mount is a bit thin and worries me that the vibrations from the camera motors or HVAC systems within walls that may cause the mount itself to vibrate thus causing the picture to vibrate. A separate wall mount from your A/V vendor will likely be a worthwhile investment.
The camera does not contain automatic voice tracking. This may be a big deal for some, as Polycom’s Eagle Eye solution provides it, but automatic tracking requires coordination and integration between the video processor and the audio processor which is why it is generally limited to single vendor solutions for audio & video. Overall I don’t see this as a big limitation, but one to note nonetheless.
The camera base build quality is fine, but it ain’t designed to take a beating. Other vendors such as Vaddio have a bit more ruggedness to their devices which allow them to take a bit more abuse, but just make sure you keep the PTZ Pro Cam out of arms reach and in a safe place and you’ll have no issues.
The camera motors are mostly silent, but then again they aren’t. I can still hear the whirring over a small fan that sits below my desk, so there is noticeable noise created.
Far end camera control is just one more app to maintain on corporate systems and likely won’t be used anyway. In my opinion it’s a “nice to have” not a “have to have”.
While camera quality is good, I’ve still seen better – albeit at much higher costs.

Final Thoughts

Overall I think Logitech has a big winner here. Super simple installation. Deployment flexibility. Good video quality. Reasonable cost. I’d be hard pressed to say that I’d prefer to pay additional money for a Vaddio solution over what Logitech provides. If you are looking for an enterprise class PTZ camera to work with your UC deployments (and especially Skype for Business deployments), the Logitech PTZ Pro Camera needs to be on your list.

09Nov/15

Disabling Edge Server File Transfer Functionality in Lync Server

November 9, 2015LyncAdministration, Edge, SecurityTrevor Miller

In the November 2011 Cumulative Update of Lync 2010, Microsoft added the ability to disable file transfers via the edge server. This feature enhancement undoubtedly was the result of customers voicing concerns about data loss protection and was a welcomed addition to the IT administrator’s arsenal. In short, the feature allowed administrators to block all file transfers that would traverse an organization’s edge server but allowed file transfers within the confines of the internal network (assuming the user was assigned a conference policy that allowed it).

Lync 2010

In Lync 2010, to block all file transfers on the edge server you would use the following command:

http://support.microsoft.com/kb/2621840

new-csfiletransferfilterconfiguration -identity edgeserver:pia-ls2013-ed01.widgets.com -blockfileextension $true -action blockall -enabled $true

Lync 2013

The same file transfer blocking capabilities exist in Lync 2013, but the commands are a bit different from what was used in Lync 2010.

In Lync 2013, to block all file transfers on the edge server you would use the following command:

http://technet.microsoft.com/en-us/library/gg425897.aspx

new-csfiletransferfilterconfiguration -identity edgeserver:pia-ls2013-ed01.widgets.com -action blockall -enabled $true

01Nov/15

Uncovering the true cost of AOL Federation for Lync

November 1, 2015Lync, Skype4BFederation, Public IM, TCOTrevor Miller

For a long time Microsoft has offered the capability of on-premises Live Communications Server, Office Communications Server and Lync Server deployments to federate with AOL for instant messaging. In order to use AOL federation there were certain basic requirements that had to be met:

You must have a functional edge server (including DNS records & necessary certificates)
You must have the appropriate Public IM Connectivity licenses for the users who will require PIC
You must request PIC provisioning through the Microsoft PIC website

Licensing

In order to use PIC federation you had to have the appropriate licenses, just as you must have the appropriate Client Access Licenses in order to use Lync, Exchange, SharePoint or any other Microsoft software. Most customers would see this cost included as part and parcel of their Enterprise Agreements with Microsoft, so while some people may dislike the cost of Enterprise Agreements, Microsoft is including a vast amount of functionality (PIC included) to you at a reasonable cost.

PIC Provisioning

Microsoft has largely handled the PIC provisioning process through their PIC website, whereby you simply request the public IM providers you want to federate with and Microsoft handles all the coordination and enablement on the back end. Having Microsoft handle the coordination was a “single throat to choke” but it often resulted in longer setup times as it could take up to 30 days for the services to be available once the request was submitted.

This is all good and well – why bring it up now?

Microsoft announced earlier this year that the PIC agreements with Yahoo! and AOL were not being renewed, meaning customers would either A) lose their ability to federate with those providers or B) have to look at third-party solutions to keep the federation alive. The EOL date for the Microsoft PIC agreements is June 30, 2014, so the clock has been ticking ever since the announcement was made. AOL recently stepped up to the plate and announced that they will begin offering direct federation for Lync customers through their PIC website, a welcome change of heart that allows customers to “keep the lights on” between their on-premises Lync deployments and the AOL cloud. Additionally, the provisioning process allowed customers to contact AOL directly and negate the need to request provisioning through Microsoft. Reading all this you would think this is a “Win-Win” situation, but as the old saying goes, the “Devil is in the details”…

The Details

I recently went through this provisioning process with a customer and discovered that the new AOL PIC federation service is not free and actually costs you additional monthly fees on top of what you’ve already paid for your Lync CALs. For this customer it would have cost them $10 per user, per month to use the AOL federation service. Add this up over a year and you could have substantial increases in your operating expenses to support AOL federation. On the plus side, however, AOL includes written SLAs as part of the service and offers customers various tiers of support for service incidents. Note: Should you decide to use AOL federation your fees could be different as AOL had different levels of fees based on the number of Lync users whom you wanted to have AOL federation enabled for.

My Thoughts

The fact that Microsoft included these costs within Enterprise Agreements was both a blessing and a curse – it meant that some customers would grumble about the costs, but it also meant that customers could simply take advantage of all the included features without a second thought. Now that Microsoft will no longer have an agreement in place after June 30, 2014, the true cost of the federation has now come to the surface and I have to admit that the AOL federation rose has some thorns. I can certainly understand that AOL has infrastructure and operational costs associated with this service so a fee is understandable, but some folks (myself included) might question if connectivity with AOL services is worth the cost they charge you, especially considering that only instant messaging is available. Skype, on the other hand, offers IM & audio federation with Lync (with video coming in the near future), so the value proposition of AOL seems to be eroding when compared to Skype.

Is AOL federation worth it? That’s up to you to decide…but suddenly Skype seems a lot more attractive to me!

26Oct/15

Adding SQL Witness Requires Rebuild of Mirror in Lync Server 2013

October 26, 2015LyncAdministration, High Availability, SQLTrevor Miller

Consider the following scenario:

Lync is installed and the initial topology is published with only mirroring configured (no witness)
At a later date (this could be weeks, days, or hours – it really does not matter the length of time) administrators decide to add a SQL witness for automatic failover of the SQL mirror
A Lync administrator adds the witness configuration into the topology and publishes the changes
Topology builder reports no errors and the Lync administrator assumes that all is properly configured for SQL automatic failover
Despite no errors encountered in topology builder, SQL Management Studio reports that NO witness configuration has been completed and attempting automatic failover results in a SQL back end outage

I have been able reproduce the scenario above in my lab, thereby removing doubt that it was an “environment issue”, and determined that the root cause is in the overall order of the SQL mirror configuration and how topology builder acts in a given configuration scenario. The bottom line (and thus, the “gotcha”) is this:

If you want to use SQL mirroring with a SQL witness, have all servers ready and initially publish the SQL mirror and SQL witness at the same time in topology builder to ensure proper activation and configuration. If you add a SQL witness after SQL mirroring was initially published, topology builder publishes the new SQL server configuration into the topology but it does not actually make any SQL-related changes on the SQL mirror nodes or SQL witness server. In the latter scenario, the witness can eventually be configured and used, but only by removing the mirroring functionality and then publishing it all again. Note: You could also have a DBA fix these issues as well through T-SQL commands, but for Lync administrators who choose to resolve it themselves these steps are your only recourse.

If you initially only configure and install SQL mirroring and then attempt to go back and add the mirroring witness, you’ll discover that nothing gets changed on the SQL principal or SQL mirror database properties after you publish your topology changes:

Notice how the Witness field is empty and that the Operating mode is set to High safety without automatic failover. Examining the Lync Topology, however, indicates that the witness is configured and active:

After a few rounds of testing I determined the following steps were required to actually get the SQL witness configured within SQL in the scenario where the mirror was installed first and a witness is added later:

Open Lync topology builder and disable the SQL witness configuration from pool properties and publish the topology changes.

Note: Do not disable the SQL mirror configuration. Only remove the SQL witness configuration.

Run the uninstall-csmirrordatabase cmdlet from the Lync management shell to remove mirroring configuration for all published databases.

Uninstall-csmirrordatabase –databasetype User –sqlserverfqdn pia-sql-be01.widgets.com –dropexistingdatabasesonmirror –verbose
Uninstall-csmirrordatabase –databasetype Application –sqlserverfqdn pia-sql-be01.widgets.com –dropexistingdatabasesonmirror –verbose
Uninstall-csmirrordatabase –databasetype Centralmgmt –sqlserverfqdn pia-sql-be01.widgets.com –dropexistingdatabasesonmirror –verbose
Uninstall-csmirrordatabase –databasetype Archiving –sqlserverfqdn pia-sql-be01.widgets.com –dropexistingdatabasesonmirror –verbose
Uninstall-csmirrordatabase –databasetype Monitoring –sqlserverfqdn pia-sql-be01.widgets.com –dropexistingdatabasesonmirror –verbose

The SQL mirror node databases should now be in an offline state. Manually remove the databases on the SQL mirror node through Management Studio. Right-click each offline database and select Delete.

Note: The database will have a small green arrow to indicate it is offline, so make sure you only remove the correct databases based on your topology.

The SQL mirroring endpoint information does not get removed automatically as part of the uninstall-csmirrordatabase cmdlet. Manually remove the endpoint information on the principal and mirror through Management Studio. Navigate to Server Objects>Endpoints>Database Mirroring. Right-click the mirroring_endpoint object and select Delete.

In Lync topology builder re-enable the mirror and witness settings on the pool properties and publish the changes.

Following the topology publish and subsequent database installation on the SQL mirror you should now notice that the witness information has been configured on the principal & mirror servers and that the Operating mode reflects the witness configuration.

Additionally, if you query the SQL witness you will see that it is successfully configured with endpoint configuration for monitoring the mirror:

Again, the bottom line here is that it is easiest to make sure the SQL witness is activated by enabling SQL mirroring and the SQL witness at the same time when you initially publish your Lync topology. If that is not an option, you will have to follow the steps above to essentially break and remove the SQL mirror, and then add the mirror AND witness configuration back in. For anyone out there that has attempted to add a SQL witness at a later date, do not assume that your witness configuration succeeded – you should absolutely verify this configuration in your environment and take the steps above if your environment did not actually get configured for SQL witness functionality.

22Oct/15

Changing SQL Mirror Endpoint Owner Breaks Mirroring in Lync Server 2013

October 22, 2015Lync, Skype4BAdministration, SQLTrevor Miller

When you install Lync 2013 and configure the back end for mirroring, a lot of things happen “under the hood” to configure the mirror (and witness, if applicable). Microsoft really has done a fantastic job of hiding the overall complexity of SQL mirroring through Lync topology builder, but suffice it so say, there are many SQL pieces that get configured and more than a few moving parts. Following the completion of the Lync back end install many companies choose to go back and perform some cleanup duties on the SQL servers. Depending on the company and the DBA, there could be lots of customization, or as I’ve encountered with most deployments, the DBAs usually just want to change the owner of the databases and mirroring endpoint that got created. The assumption by many (myself included) was that this would be a completely seamless and a benign change, but I learned during a recent deployment that it can result in SQL issues that 1) cause the mirror to stop functioning and 2) put both SQL servers in a state where databases could not be accessed on either server. Initially I thought this was something specific to the customer’s SQL configuration, but I was able to confirm this exact behavior in my lab so I knew that the issue could potentially be more widespread than I first thought. It took more than a few hours to figure out the root cause, but thankfully the fix is quick and easy. Read on for the exciting conclusion!

Note: While I expect this issue to be a “your-mileage-may-vary” type of issue, it did give me pause that I could re-create this in my lab, which is very simple and has no complexity. Some may experience this issue and others may not. If you do, the info below should save you some heartache.

Understanding SQL Mirroring

For those less familiar with the inner workings of SQL mirroring, Microsoft has some great information on TechNet that will help you better understand what we’re about to discuss. I highly suggest you read the TechNet documentation first before continuing on.

SQL mirror functionality relies on mirroring endpoints, configured within each SQL instance, to allow inter-server communication. There are two easy ways to view the mirror endpoints:

Option 1 – In SQL Management Studio, go to Server Objects>Endpoints>Database Mirroring

Option 2 – In SQL Management Studio, run this query:

select * from sys.database_mirroring_endpoints

In order to use the mirroring endpoint, logins must exist on the SQL server instance for the account that the SQL services are running under on the server. Just as a domain user account must be added to the logins to connect to a SQL server, so too must the account(s) that the SQL server instance are running under. The principal server will have logins for the mirror and witness, the mirror server will have logins for the principal and witness, and the witness server will have logins for the principal and mirror. In my lab the SQL instances are using the NETWORK SERVICE account on the server itself, so the domain computer accounts are logins within each SQL instance. If your environment uses dedicated domain user accounts to run SQL, then you would see the domain user accounts as logins within each SQL instance. For example, my principal server logins are shown below:

Additionally, the logins must be granted permissions to connect to the mirroring endpoint. Just as a domain user account must be granted access to a database to successfully connect, so too must accounts be given explicit access to consume the database mirroring endpoint. The permission required is a GRANT CONNECT permission, and this permission will match up with the logins on each server. The principal server will have GRANT CONNECT for the mirror and witness, the mirror server will have GRANT CONNECT for the principal and witness, and the witness server will have GRANT CONNECT for the principal and mirror. Remember that the permissions will always be granted to the account that SQL is running under. For example, I can use the following T-SQL query to examine the current permissions on my principal SQL server:

SELECT EPS.name, SPS.STATE,
CONVERT(nvarchar(38),
SUSER_NAME(SPS.grantor_principal_id))AS [GRANTED BY],
SPS.TYPE AS PERMISSION,
CONVERT(nvarchar(46),SUSER_NAME(SPS.grantee_principal_id))AS [GRANTED TO]
FROM sys.server_permissions SPS , sys.endpoints EPS
WHERE SPS.major_id = EPS.endpoint_id
ORDER BY Permission,[GRANTED BY], [GRANTED TO]

Lastly, the mirroring endpoint will have an owner who was the person that initially ran Lync topology builder and installed the databases. The owner will be that person’s domain user account and can be determined by the following T-SQL query:

SELECT [PrincipalName] = sp.name, [PrincipalId] = sp.principal_id, me.*
FROM sys.database_mirroring_endpoints me with(nolock)
inner join sys.server_principals sp with(nolock)
on me.principal_id = sp.principal_id

The Issue

As stated at the beginning of the post, my customer went through and changed the owner of the mirroring endpoint to SA as a part of post-installation clean-up. The process to change the owner is a very simple T-SQL command that must be run on each SQL server:

ALTER AUTHORIZATION ON ENDPOINT::mirroring_endpoint TO sa

After running that command, we verified that ownership had changed using the following T-SQL query:

SELECT [PrincipalName] = sp.name, [PrincipalId] = sp.principal_id, me.*
FROM sys.database_mirroring_endpoints me with(nolock)
inner join sys.server_principals sp with(nolock)
on me.principal_id = sp.principal_id

Initially we thought all was well, but when we began testing failover things broke down. We tested a hard failover by simply shutting down the SQL services on the principal and as expected the failover completed successfully. After turning on the SQL services on the principal following the failover, we ended up in a state where mirroring became broken and the Lync front end services couldn’t connect to the back end databases on either server. Looking at SQL Management Studio you could see that both servers thought they were the principal and mirroring was disconnected. We also tried restarting SQL services on the mirror and witness, but nothing helped. Nothing was working and it seemed that the servers, witness included, simply couldn’t talk to one another. When replicating these changes in my lab, I saw this (which was the same experience at my customer):

We were all pretty baffled by this (including the customer’s DBA) and after multiple attempts in my lab, I was able to track down what had occurred. For a reason unbeknownst to me, the ALTER AUTHORIZATION command removes the GRANT CONNECT permissions on the mirroring endpoint. Checking the permissions in my lab, I saw this:

SELECT EPS.name, SPS.STATE,
CONVERT(nvarchar(38),
SUSER_NAME(SPS.grantor_principal_id))AS [GRANTED BY],
SPS.TYPE AS PERMISSION,
CONVERT(nvarchar(46),SUSER_NAME(SPS.grantee_principal_id))AS [GRANTED TO]
FROM sys.server_permissions SPS , sys.endpoints EPS
WHERE SPS.major_id = EPS.endpoint_id
ORDER BY Permission,[GRANTED BY], [GRANTED TO]

Notice how the computer accounts have been removed. This little nugget turned out to be the proverbial “needle in a haystack”, as it allowed me to determine that the servers couldn’t connect because the permissions had been removed.

The Fix

After all the fuss, the fix was to simply add the GRANT CONNECT permissions back so that the servers could talk to one another again. Remember that the permissions will always be granted to account that SQL is running under. For example, in my lab I used the following T-SQL commands to restore the permissions:

Principal

GRANT CONNECT ON ENDPOINT::mirroring_endpoint TO [WIDGETS\PIA-SQL-BE02$]
GRANT CONNECT ON ENDPOINT::mirroring_endpoint TO [WIDGETS\PIA-SQL-BE03$]

Mirror

GRANT CONNECT ON ENDPOINT::mirroring_endpoint TO [WIDGETS\PIA-SQL-BE01$]
GRANT CONNECT ON ENDPOINT::mirroring_endpoint TO [WIDGETS\PIA-SQL-BE03$]

Witness

GRANT CONNECT ON ENDPOINT::mirroring_endpoint TO [WIDGETS\PIA-SQL-BE01$]
GRANT CONNECT ON ENDPOINT::mirroring_endpoint TO [WIDGETS\PIA-SQL-BE02$]

Note: If your environment uses domain user accounts for SQL services, make sure to apply the permission to the user accounts and not the computer accounts like I did above.

Once the permissions were in place and verified, mirroring automatically resumed and SQL Management Studio looked much happier:

At my customer we made the same changes (except we applied the permissions to the domain user accounts that SQL was running under) and ended up with the same result as my lab: SQL mirroring was again working. Additionally, the Lync services on the front ends could once again connect to the back end databases. All was once again well in my Lync world.

Conclusion

This is another example of hidden SQL “gotchas” that could cause major heartaches for Lync 2013 deployments using SQL mirroring functionality. It seems that SQL caches the original configuration (which is why mirroring initially worked after we changed it), but once SQL services are restarted it will use the new configuration (which is why it was broken afterwards). If you have SQL mirroring deployed I strongly urge you to double-check permissions if you change the owner of the DB endpoint and thoroughly verify failover following that change. If you don’t, you run the risk of an outage which is exactly what mirroring is supposed to avoid.

Lastly, if anyone out there, DBAs included, can explain why the ALTER AUTHORIZATION command would remove the permissions….I’m all mirrors….I mean ears.