Sametime 11 with Proxy Server -- installation notes
by Richard Pajerski
Posted on Thursday March 19, 2020 at 04:39PM in Technology
HCL Sametime 11 has been out for a few months now and brought important technical changes that, when used in conjunction with the Sametime 11 Proxy Server, make it a more compelling offering than previous versions released by IBM.
The most fundamental change is the streamlined installation that removes the DB2 and Websphere dependencies needed for the proxy server. Those components have been replaced by MongoDB and a Tomcat-based proxy server respectively, both of which are indeed simpler to install and configure. It's been rumored that an .nsf storage option will be offered in the next release and that should further smooth out the installation process.
That being said, getting everything up and running is more difficult than it should be. One notable problem is the documentation. The language is at times too informal (even ambiguous), the formatting could use some tidying up (unclear headings/inconsistent fonts for samples, etc.) and a URL for details on setting up SSL/TLS sends you to the wrong version (10) -- apparently, no version 11-specific documentation is available. More importantly, if this is a fresh Sametime installation (which is the only supported option), what's *left out* of the documentation might lead to broken communication between the proxy and the Sametime server.
After successfully completing the Windows installation, I attempted to log in to the proxy from a browser and was greeted with "Sametime is temporarily unavailable":
The problem wasn't immediately obvious because in the proxy's logs I found entries like "Sametime Proxy server is successfully connected to the Sametime community, ...". But digging further into the logs, I found "generateTokenFailed reason: 80000000". So network communication was there but
SSO wasn't working.
PRO TIP: Get more verbose logging on the proxy by uncommenting these two lines in Tomcat's logging.properties file (sametimeproxy\conf directory):
com.ibm.level=FINE
com.ibm.handlers = 2localhost.org.apache.juli.AsyncFileHandler
During the Sametime server installation, the installer creates a Web/SSO document in the Domino Directory called "LtpaToken", sets the Session authentication field to "Multiple Servers (SSO)" (in the Domino server document) and correctly references the Web/SSO document. However, on this fresh installation, the DNS Domain name field of the Web/SSO document was blank:
That's going to leave you with "HTTP Server: Error loading Web SSO Cookie Name Configuration 'LtpaToken' for Web Site ..." on the Domino server and will prevent Sametime from properly creating an SSO token to send to the proxy server.
Adding the DNS domain name (in my case, .testlab.com) to the Web/SSO document should fix the login problem for most installations. But in my Windows installation, there was a further complication that kept producing "Sametime is temporarily unavailable". It turns out the fully-qualified hostname for the Sametime server was not being passed to the proxy. From the proxy's logs:
serverFQDN: S1
cluster: CN=S1/O=TestLab
serverURL: 192.168.0.102
The fix for this was to enter the fully-qualified domain name for the Sametime server in the Net Address field of the Domino Server document (Ports > Notes Network Ports tab). After that, the login worked and the proxy reported:
serverFQDN: s1.testlab.com
cluster: CN=S1/O=TestLab
serverURL: 192.168.0.102
Conclusion
So what's *left out* of the documentation is any reference to the Web/SSO and LtpaToken configuration on the Sametime server. Whether or not the blank DNS Domain name field is an "out-of-the-box" configuration error, some mention of the Web/SSO details back on the Sametime server would be a helpful addition to the documentation.
How are your Sametime 11 installations coming along?
Thanks for the hints. While inststalling on a Windows box I ran into an issue regarding xACL (Extended Access) to names.nsf.
The error logs pointed me to an 10 year old entry from someone on the internet who had the same problem. Basically you would need to disable xACL, then install Sametime and re-enable after the installation completes. I will have to verify this today.
Hi Uwe --
Thanks for your comments and I appreciate the tip on the Extended ACL. Were you able to verify that?
We're fortunate to have those old blogs out there especially considering that the old Notes/Domino forums have been offline for months.
Best,
Richard
Posted by Richard Pajerski on April 05, 2020 at 03:21 PM EDT #
Thanks for the information.
Still not getting it working, the instructions on HCL need expanding and tidying.
On 3rd build of a sametime server, and still getting the same "Sametime is temporarily unavailable" in a browser.
Posted by Nigel on August 17, 2020 at 10:35 AM EDT #
Hi Nigel -- What are you seeing in the Tomcat logs?
Posted by Richard Pajerski on August 17, 2020 at 02:13 PM EDT #
Hi Richard,
Thanks for replying. I am new to Sametime but not Notes, so all this is a bit confusing.
The Tomcat logs are showing token errors.
17-Aug-2020 15:32:06.294 WARNING [https-jsse-nio-8443-exec-7] com.lotus.sametime.core.comparch.STCompPart.sendEvent Unable to send event, ID = 0x4 - MessageDispatcher is null.
17-Aug-2020 15:32:16.760 INFO [https-jsse-nio-8443-exec-3] com.ibm.collaboration.realtime.stproxy.services.community.CommunityService.callLoginService loginMethod: byToken, SID: 08bb600c-54f4-44ca-a1ca-84742dd36edd
17-Aug-2020 15:32:16.763 INFO [pool-1-thread-7] com.ibm.rtc.stproxy.services.ExecutorServiceManager.lambda$registerExecutorService$0 ThreadPool[pool-1-thread-7] registered for service[Status-Thread-Pool]
17-Aug-2020 15:32:16.769 WARNING [https-jsse-nio-8443-exec-3] com.ibm.collaboration.realtime.stproxy.services.community.CommunityService.loginByToken CLFRX0035E: Token was not found for login by token by user null, SID: 08bb600c-54f4-44ca-a1ca-84742dd36edd
17-Aug-2020 15:32:16.770 INFO [https-jsse-nio-8443-exec-3] com.ibm.collaboration.realtime.stproxy.services.conference.ConferenceManager.removeConference conferenceList size: 0
17-Aug-2020 15:32:16.770 WARNING [https-jsse-nio-8443-exec-3] com.ibm.collaboration.realtime.stproxy.services.community.CommunityService.abortLogin CommunityService.abortLogin()
17-Aug-2020 15:32:16.771 WARNING [https-jsse-nio-8443-exec-3] com.ibm.collaboration.realtime.stproxy.services.community.CommunityService.doService SID:[null] IllegalArgumentException[NO_TOKENS]
Any pointers would be most appreciated, as you stated you start looking and find stuff for version 8.5, 9, 10 and 11 and all gets a bit muddled as to what to do, in what order.
Thanks
Nigel
Posted by Nigel on August 19, 2020 at 03:26 AM EDT #
Hi Nigel, thanks for the update. It appears your SSO token is either not being created by Domino or not being sent to Tomcat.
The next thing I'd look at is your Web SSO Configuration document in the Web\Web Configurations view. Verify that the 'DNS Domain' field has your domain name in it with a period prepended to it. For example, the value in my setup is ".testlab.com" (without the double quotes).
Also, if you haven't already done so, this would be a good time to get more verbose logging going in Tomcat:
Tomcat's logging.properties file (sametimeproxy\conf directory):
com.ibm.level=FINE
com.ibm.handlers = 2localhost.org.apache.juli.AsyncFileHandler
...then restart Tomcat.
Posted by Richard Pajerski on August 19, 2020 at 11:17 AM EDT #
Hi Richard,
It seems that I have a very similar issue here. Even though I have completed successfully all steps for community+proxy+mongodb for level FP2, proxy is not working and displays the message "Sametime temporarily unavailable". I have checked your points and they seem to be ok. I have tried everything from documentation and blog posts, but it seems I've stuck there...
My setup is utilizing LDAP directory type, proxy 80/443 ports, user photo support. The ST logs display temporarily login/logout for the users, and from what I've seen, cookies and LTPA token is working. No web client or mobile app is able to login, however, all ST clients (embedded + Standalone) work fine. I have enabled proxy logging options as instructed.
Tomcat (catalina) log extract display the following after server startup:
25-Aug-2020 22:46:04.720 WARNING [ST async execution thread1] com.lotus.sametime.core.util.AsyncExecutionQueue.run Exception on AsyncExecutionQueue:
java.lang.UnsatisfiedLinkError: sturlcon10 (Not found in java.library.path)
at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1471)
at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1423)
at java.lang.System.loadLibrary(System.java:561)
at com.lotus.sametime.core.util.connection.WnetURLConnection.connect(WnetURLConnection.java:196)
at com.lotus.sametime.userinfo.UserInfoServletConnector.getContent(UserInfoServletConnector.java:254)
at com.lotus.sametime.userinfo.UserInfoServletConnector.getContentThroughRProxy(UserInfoServletConnector.java:231)
at com.lotus.sametime.userinfo.UserInfoServletConnector.getUserInfoDocument(UserInfoServletConnector.java:138)
at com.lotus.sametime.userinfo.UserInfoQuery.doSynchronousHttpQuery(UserInfoQuery.java:265)
at com.lotus.sametime.userinfo.UserInfoQuery.access$200(UserInfoQuery.java:46)
at com.lotus.sametime.userinfo.UserInfoQuery$1.run(UserInfoQuery.java:201)
at com.lotus.sametime.core.util.AsyncExecutionQueue.executeAll(AsyncExecutionQueue.java:128)
at com.lotus.sametime.core.util.AsyncExecutionQueue.run(AsyncExecutionQueue.java:164)
at java.lang.Thread.run(Thread.java:821)
25-Aug-2020 22:46:04.720 WARNING [ST async execution thread1] com.lotus.sametime.throwable.ThrowableReporter.caughtThrowable Notifying listeners about caught throwable: java.lang.UnsatisfiedLinkError
Any Ideas?
Regards
Posted by Zeppos Galanos on August 25, 2020 at 04:59 PM EDT #
Hi Zeppos --
Thanks for posting. Part of your post was truncated but I found these errors in your logs:
25-Aug-2020 23:10:37.366 WARNING [https-openssl-nio-443-exec-4] com.ibm.collaboration.realtime.stproxy.services.community.CommunityService.doService SID:[2e7ec173-8d42-49d3-a4e7-554a80948d64] IllegalArgumentException[JWT String argument cannot be null or empty.]
25-Aug-2020 23:13:20.284 WARNING [https-openssl-nio-443-exec-1] com.ibm.collaboration.realtime.stproxy.services.community.CommunityService.loginByToken CLFRX0035E: Token was not found for login by token by user null, SID: f2009ffd-22cb-4b5d-8f97-29462167a375
These are among the same errors that Nigel was seeing and I think they indicate Domino has a problem creating/sending the SSO token. Two things I would look at next:
1) What value do you have in the 'DNS Domain:' field of your Web/SSO document?
2) In your Tomcat logs, what value do you see for "serverFQDN:"? It should display the same fully-qualified domain name that's in the browser URL (for example, chat.mydomain.com).
Posted by Richard Pajerski on August 25, 2020 at 10:35 PM EDT #
Hi Richard,
Thanks for all the updates and help it is most appreciated.
I wish there was a one-click installer for all of this and not so convoluted and ambiguous, I am sure it could mostly be encapsulated in an NSF to do most of the installation, post sametime server install if HCL put their minds to it.
I think I am now getting confused by the whole thing, especially the domain names to use.
I now have changed so many things so may times, I can’t remember where it was when I first started and might start again from fresh install of windows again.
My set-up for example:
Our windows domain is internal.co.uk
The primary domino server is pds.internal.co.uk
its FQDN is pds.external-domain.co.uk
The sametime server is sametime.internal.co.uk
its FQDN is sametime.externaldomain.com (yes different from the main server, and whats in the browser)
The web sso is set up on the sametime server, is this is correct, and not needed on the main domino server.
So what should I be putting in the DNS domain name in here, to be honest I have tried all of the things we use.
I have turned on fine logging, and will collect some logs to send.
I have done 3 complete rebuilds of the server and installation all as instructed by HCL in the last month, things should not be this difficult to install and get working. Same as Zeppos the client and embedded worked immediately after install, just the web/phone app never got working, which is what the users want.
Posted by Nigel on September 03, 2020 at 08:30 AM EDT #
Hi Nigel --
Thanks for posting. I understand your frustration and agree that HCL needs to make the experience much simpler! By the way, this blog software did save your full responses -- but by default, it assumes anything over 1,000 characters is SPAM, giving you the message you saw.
Do you have both Domino and Sametime installed on the same server? If so, can you assign 'sametime.internal.co.uk' to the same IP address? Ideally, 'sametime.internal.co.uk' should be the URL that users enter in their browser/phone and also be the value listed in the Net Address field of your Domino Server document.
Posted by Richard Pajerski on September 03, 2020 at 11:11 AM EDT #
I wanted set it up on the same machine but I set up the sametime server as a new server as it requires server 2016/19 and the main domino server is still on server 2012.
The users will not be able to enter the AD domain address as this not available outside the organisational network, where as I can add sub-domains to our public domain names to point to the sametime server like chat.externaldomain.com.
I think it doesn't help when setting up mongo, sametime and proxy etc. using the out of date instructions from HCL, I was unsure when entering the address/name it meant AD name or FQDN, local, loopback or public IP.
It would probably all work if our internal AD Domain had the same name as our web domain name.
So apart from doing all the things in HCL instructions and any additional settings like you have stated, does anything need to be done, so a fresh windows server install, update, join domain, install and set-up Domino server, MongoDB, sametime, proxy using all the proper steps.
Are there any changes to the other non-sametime primary domino server needed at all. I was reading about Directory Assistance and other things and replicating things from one to the other, end up going down rabbit holes reading for hours, on things about 8.5, 9 and 10 versions etc.
Sorry for bombarding you, but you seem to have a good knowledge of this.
Posted by Nigel on September 04, 2020 at 08:01 AM EDT #
Hi Nigel --
No problem, I don't mind the questions!
If you only need the capacity of one server, then you should install Sametime 11 on the same server as Domino 11. You can get Sametime working on multiple servers (with Domino server clustering) but that is a more complicated setup and should only be considered if you need to support thousands of concurrent connections.
For a single server setup, no, you don't need any additional configuration for Directory Assistance, etc. Also, since the architecture changed with Sametime 11 and WebSphere server is no longer needed, the documentation for releases 8.x through 10 is (generally) no longer relevant for Sametime 11 installations.
Although the Sametime 11 server isn't officially supported under Windows 2012 it may work fine there (test first, but I've managed to get it working successfully even on a Windows 8 Pro client as a proof of concept in a non-production environment).
Posted by Richard Pajerski on September 08, 2020 at 12:15 PM EDT #
Hi Richard,
Thanks for this, I will give it a go I think as only have less than 100 users, so not really a lot of users.
Like I said only setup a new server to get running on server 2019. I can give it a go and then roll out if it works.
Will it cause problems if verse and http services are running on the server as well.
Will update with my results when I get a chance to do it.
Thanks
Posted by Nigel on September 09, 2020 at 03:33 AM EDT #
Hi Nigel --
I've not tried to run Verse on the same machine as Sametime, but in theory it should work as long as you use different ports. I'd suggest running Verse on default port 443 and Sametime on 8443. So your web and mobile users will access chat at sametime.internal.co.uk:8443.
Looking forward to hearing how this goes.
Posted by Richard Pajerski on September 09, 2020 at 09:42 AM EDT #
Nigel -- If you're using an Internet Site document, one other thing you might try is adding the following two lines to your server's sametime.ini in the [AuthToken] section:
ST_ORG=<Your Domino Organization's abbreviated name>
ST_TOKEN_TYPE=LtpaToken
Extending my example above, my sametime.ini has:
[AuthToken]
ST_ORG=TestLab
ST_TOKEN_TYPE=LtpaToken
Posted by Richard Pajerski on September 22, 2020 at 05:34 PM EDT #
Hi Richard
Thanks for this, I have been off work for a couple of weeks so will try this and let you know the results.
I installed sametme on the primary domino server and still have the same problem I was having on the separate sametime server, so must be related.
Thanks
Posted by Nigel on October 06, 2020 at 03:18 AM EDT #
