How to Fix a “CertEnroll” Certificate Private Key Error in OCS 2007 R2 and SharePoint

March 24, 2010 by Chris Williams · Leave a Comment
Filed under: OCS 2007, OCS 2007 R2, Reference 

Microsoft Office Communication Server 2007 R2 servers require certificates for several authentications between servers. SharePoint servers do too. These certificates require keys in order to maintain their security.

Today's post will give a solution for a cert error that pops up occasionally: When OCS or SharePoint fails to detect your cert's private key.

  1. When setting up a new certificate for Office Communication Server 2007 R2 or SharePoint, you may receive this error.

    “CertEnroll::CX5090Enrollment::p_Install Response: ANS1 bad tag value met. 0×8009310b (ASN: 267)”

    OCS CertEnroll Certificate Request Error Message
    This error is most likely caused by a missing Private Key on the certificate you're attempting to install.
  2. In order to resolve this issue, access the MMC for certificates.

    MMC for Certificates OCS
    Select the certificate you are trying to install and view its Properties. Select the Details view and copy the Serial Number you find there to Notepad.

    Certificate Properties, Serial Number
  3. Once you have it in Notepad, remove the blanks between the numbers. So it looks like this:

    OCS Certificate Private Key in Notepad

  4. Once that's done, run the command certutil to append this now-corrected Private Key to the certificate. Enter the command as follows:

    certutil –repairstore my (insert serial Number)

    An example screenshot is below.

    Command Prompt Repair OCS Certificate with Private Key
  5. Once this repair finishes running, the certificate will be ready to use in your installation.

Save this private key, along with its relevant cert name, in another location. Just in case OCS or SharePoint prompts you for it again.

Did you encounter this error? Under what circumstances? Tell us in the comments if this solution worked for you.

Protecting External Connectivity In OCS 2007

Last Friday on TechRepublic’’s “10 Things” Blog, Brien Posey wrote about 10 Common Network Security Design Flaws.

The second flaw is “opening more firewall ports than necessary.” And what does he use as an example? OCS 2007 R2.

It’’s a very good example. As he states, OCS requires several ports opened in order to provide external connections to other networks. Without proper protection, this can be a risk to you. Ports left open (and not monitored) are little signposts saying, “Enter Here!”

(This is only if you want to use external connections. If you”re only interested in OCS for internal IM and Presence, you won”t need to open those ports.)

Brien puts forth Microsoft ForeFront as a good solution to the problem. ForeFront’’s Threat Management Gateway is a reverse proxy - intended to filter requests for access into & out of your network. At the risk of sounding too provider-loyal, it IS a natural fit. (If you use a hosted OCS provider, chances are ForeFront is in place.)

In order to protect External Connectivity completely though, you”ll have to use the OCS Edge Server. There’’s an advantage to this: depending on how you want to communicate with others, you can enable only what you need. Each of the External Connectivity services requires a service enabled on the OCS Edge Server. According to Microsoft’’s TechNet, these are:

  • Access Edge service — Lets outside users communicate with your OCS using SIP.
  • Web Conferencing Edge service — Lets outside users participate in your conferences.
  • A/V Edge service — Lets you share audio and video with external users.

The TechNet page also gives links on how to administer these services:
Microsoft TechNet — Managing External Connectivity for Your Organization with Edge Servers

I blogged about this because it’’s important to remember. Brien’’s #1 network security flaw was the “set it and forget it” mentality. Doing that with OCS can leave a lot of exploitable holes in your network. All of them preventable if you remember to protect External Connectivity.

Technorati Tags: