Hyperion / EPM -- Enabling TLS 1.2 and LDAPS in Hyperion/EPM 11.1.2.4

Recently enabled TLS 1.2 ( HTTPS ) and LDAPS for Microsoft Active Directory (MSAD) connections in a mission critical Hyperion environment. Actually, the work was quite challenging, so I decided that it's worth sharing with you.

In this project we needed to enable TLS 1.2 on Hyperion Web application side and turn off all the SSL/TLS versions except TLS 1.2 for security reasons. In addition to that, we needed to enable LDAPS for Hyperion-MSAD connections..

Let's start with the TLS/HTTPS side of the work;

We knew that Hyperion 11.1.2.4 doesn't support TLS 1.2. So the web server that comes built-in with the Hyperion was unable to speak TLS 1.2..

That's why, we needed to add/install a seperate Web Server in front of the Hyperion's web server, which was OHS 11.1.1.7.

We also needed to configure this new Web Server to play the middle man role between the Hyperion clients and Hyperion Web applications (Hyperion's Web Server actually).

This new webserver  should have been an OHS 11.1.1.9.
Note that, OHS 11.1.1.9 supports TLS 1.2...

After this configuration, the flow would be Clients -> https 11.1.1.9 -> https 11.1.1.7..

We already enabled TLS/SSL in the Hyperion. That is, we enabled the TLS/SSL in current OHS ,but it wasn't able to speak TLS 1.2.

References for enabling TLS/SSL in EPM ( SSL terminated at OHS):

Steps to Setting Up SSL Offloading with OHS Webserver From EPM 11.1.2.x (Doc ID 1530169.1)
http://learninghyperion.blogspot.com/2015/04/secure-epm-environment-ssl-terminated.html

We ensured that, currrent OHS couldn't communicate with TLS 1.2, and we tested that by disabling all the other protocols but TLS 1.2 in our browser.

We tried to reach the application login, but couldn't do that.

As mentioned, we added a new OHS 11.1.1.9 into the picture and let it do the TLS 1.2 work.. That new OHS was also assuming the duty of being a bridge between clients and old OHS 11.1.1.7.
A reverse proxy...

The final picture became as follows;

Actually we have applied the workaround documented in MOS note : Does EPM to Support TLS 1.2 Communication via OHS? (Doc ID 2179810.1), and it worked perfectly well.

The operations to implement this workaround were as follows;

1) Download OHS 11.1.1.9 via Patch 20995453.

2) Install OHS by running the OHS 11.1.1.9 installer.

Choose Install and Configure
Enter a new Oracle Middleware Home location (DO NOT use existing Oracle Middleware home, do not overwrite anyting!)
Choose "Oracle HTTP Server" only to configure only the OHS and OPMN.
Be careful in "Configure Ports" page, as All OHS and OPMN ports have to be unique.

3) After the installation, first check that you can access the new OHS using HTTP.

4) Then enable SSL in this new OHS by making changes to the httpd.conf and ssl.conf files.

See Configuring Oracle HTTP Server to Use SSL in Fusion Middleware 11g (11.1.1.X) Document 1226933.1. Note if the EPM OHS is already SSL enabled, you should be able to copy the changes made to the EPM OHS to support SSL, to the 11.1.1.9 version of OHS. You can also use the same certificate (wallet) since the new OHS is on same machine.

5) Configure the new OHS 11.1.1.9 Web Server as a reverse proxy to OHS 11.1.1.7 by editing the OHS 11.1.1.9 ssl.conf.. Just add the ProxyPass and ProxyReversePass as shown below..

Note that, in the following example, we suppose the OHS 11.1.1.7 is already ssl-enabled.. If not, you can just make the necessary modifications accordingly.

<IfModule mod_proxy.c>
SSLProxyEngine On
# Normal reverse-proxy requirements
ProxyPass / https://<ohs_11.1.1.7_host.domain.com>:<ohs_11.1.1.7_ssl_port>/
ProxyPassReverse / https://<ohs_11.1.1.7_host.domain.com>:<ohs_11.1.1.7_ssl_port>/
ProxyPreserveHost On
ProxyRequests off
# SSL specific reverse-proxy requirements
SSLProxyProtocol ALL -TLSv1.1 -TLSv1.2
SSLProxyCipherSuite HIGH:MEDIUM:!LOW:!NULL:!aNULL:!eNULL:+SHA1:+MD5:+HIGH:+MEDIUM
SSLProxyWallet </oracle/wallet_location>
</IfModule>

6) Start/Restart OHS 11.1.1.9 and try to access the EPM login page through OHS 11.1.1.9 using https.

Example url: https://NEWOHS:443/workspace/index.jsp.

7) Disable all the protocols except TLS 1.2 and try reaching the EPM login page again to ensure that you have SSL TLS 1.2 for the communication between the user’s browser and your OHS 11.1.1.9.

Actually, it could be tested by disabling all the protocols but TLS 1.2 in our browsers, as mentioned earlier. But this time we needed to disable all the protocol (except TLS 1.2) in OHS level, so that no one could reach the EPM using lower SSL/TLS versions..

In order to disable all the protocols but TLS 1.2 in OHS, we just edited the ssl.conf of our new OHS server and modified the related line like the following and restarted our OHS;

SSLProtocol -ALL +TLSv1.2

(this instruction basically says -> accept TLSv1.2 and deny all other protocols in SSL communication)

So far so good.. 

Lets take a look at the LDAPS side of the work;

You can think LDAPS,as the SSL/TLS-based version of LDAP..

We use ldap for Active Directory authentications mostly in Oracle environments.

General speaking, we gather the credential information from the user, or from user's Operating system and use them with ldapbind and if we see a successful return from our bind operation, we let the user go into our application.. 

So we authenticate the user from a centralized LDAP directory and this LDAP directory is mostly Microsoft Active Directory (MSAD).

Most of the time, we have OAM (Oracle Access Manager) and OID (Oracle Internet Directory) between the Active Directory and our application, but this is another story and it is not in the scope of this blog post..

Anyways, we were already authenticating Hyperion users from the MSAD.. 

However; our connection between Hyperion and MSAD was based on LDAP, rather than LDAPS and that was a security vulnerability.

So we must have established secure connection communication between our Hyperion EPM servers and the user directory (MSAD LDAP).

We already knew that, whenever Hyperion tries to communicate with SSL enabled MSAD/LDAP, it uses the trust certificate from its Java keystore (cacerts).

So, in order to enable LDAPS, we imported all the MSAD SSL certificates.. (by following How to Import SSL Certificates for Hyperion EPM to Use a SSL Connection to Corporate Directory (Doc ID 1599610.1 ))

We imported the root certificates of the MSAD both into the cacerts keystore file residing in Jrockit and the cacerts keystore file residing in Jdk directories..

We used commands like the following for doing this job ->

keytool -import -alias ldapca -keystore C:\bea\jrockit_160_37\jre\lib\security\cacerts -trustcacerts -file C:\ldapca.cer

After these types of moves, we restarted all the EPM services and used EPM Shared Services to configure MSAD configuration.. We configured it to use LDAPS on port 636(default LDAPs port of MSAD) but it failed !!

Basically, when we tried to configure MSAD Connection Information we got the error as follows;

"EPMCSS-05139: Failed to retrieve base DNs. Error communicating to server. Simple Bind Failed. Invalid host, port value."

At this moment, we thought that, maybe this was because the MSAD that we were trying to talk to, was configured with TLS 1.2 only.

Actually, MSAD admin said that his MSAD configuration allows TLS 1.0 and 1.1 as well, but we still suspected that kind of a configuration problem and decided to upgrade our JDK and Jrockit in the Hyperion side.

We knew that TLS 1.2 was only supported with JDK 8 version. On the other hand, our JDK and JRockit versions were 1.6, and JDK 8 upgrade was not supported with the Hyperion 11.1.2.4..

With this in mind, we made a research and reviewed the knowledge base and bug records..

The following MOS document seemed promising..

However; it was recommending a JDK upgrade..

Essbase: Unable to connect to SSL Enabled Windows 2016 MSAD External Directory from Essbase Server. ERROR: JAVAX.NAMING.COMMUNICATIONEXCEPTION: SIMPLE BIND FAILED: LDAPSERVER.COM:636 [ROOT EXCEPTION IS JAVA.NET.SOCKETEXCEPTION: CONNECTION RESET ( Doc ID 2482392.1 )

Yes.. The above document was suggesting JDK upgrade->

To Upgrade JDK 1.6 to JDK 1.6 Update 181 or higher, apply the steps outlined in Doc ID 2390603.1

To Upgrade JDK 1.6 to JDK 1.7, apply the steps outlined in Doc ID 2351499.1

So we decided to give it a try.

We had our up-to-date backups and the operation was easy..

We choosed to upgrade our JDK to JDK 1.6 Update 181, and this operation was just about downloading the new version, unzipping it and switching the names of the currently installed JDK directories with the newly downloaded ones..

We upgraded JDK to 1.6.0.181 and Jrockit to 28.3.17 by following Doc ID 2390603.1 and ID 2482392, but we were still getting the same error!!

At this point, as for throubleshooting the error we did the following;

• Worked with the LDAP admin to made sure that LDAP host is using SSLv3 or TLS1.0 and not use SSLv2Hello.
• Made sure there is no firewall issue.. LDAPS port (636) was open in the firewall between the EPM server and LDAP server.
• Made sure that the UserBaseDN, principal, credentials were configured properly. Also made sure that the server hosting LDAP was reachable and that the port (636) was open both directions.
• Installed an external LDAP browser (http://www.ldapbrowser.com/download.htm on the server where Shared Services was installed and tested the connection to MSAD. (note that external LDAP browser connected to MSAD successfully..)
• Tried re-importing CA certificates to the keystores used by EPM's Weblogic.

We couldn't get any solution from the things I just listed above, but during these throubleshooting work; we discovered something which was quite interesting.

Actually , the hostname that we were using for connecting the MSAD wasn't belong to MSAD itself.. So there was something between EPM and MSAD and that thing was a Load Balancer!

Customer was using a Load Balancer in front of its MSAD! Yes... But not only that... Customer was using that Load Balancer for offloading the SSL work between the  MSAD and the rest of the network.

This made all clear.

We needed to import Load Balancer's root certificate in order to speak LDAPs with the MSAD.

Actually we needed both Load Balancer's root certificate and MSAD 's root certificate.. This was because in case of a failure in Load Balancer we still need to reach the MSAD and be able to speak TLS with it.

As for the solution, we gathered the root certificate of the Load Balancer and imported it to our cacerts files, restarted EPM services and it was done! LDAPs was enabled! :) interesting right ? :)

Once again, we realized how important the communication between IT employees is!

So if you are working on a new environment, always ask and try to get a detailed infrastructure architecture which shows the connections between your server and other servers in the network...

If you can't get that info from the customer, then you try to find it in your own way .. 

Almost forgot ! :) One more thing for the TLS 1.2 enablement! 

We made the firewall admin to close all the ports between the clients and the OHS 11.1.1.7... We don't want to have a naughty client to reach the application using TLS 1.0 or SSLv3 by bypassing our new OHS 11.1.1.9  :)