Overview
To resolve issues with News Streams or Activity Streams not showing in Hosted/On-Premise instances, a common first step is to rebuild the EAE stream. When restarting the web nodes after an EAE stream rebuild, a node may enter a deadlock (blocked state) and not finish starting up.
Solution
This happens if the EAE service sends a message too quickly to a web node.
<supportagent>
This is a known issue that was resolved in 9.0.7.1.
You can use the Jive Snaps tool to check for blocked threads, by taking a thread dump.
</supportagent>
Resolution steps:
-
Stop all web nodes & EAE service.
/etc/init.d/jive stop eae -
Start the EAE service.
/etc/init.d/jive start eae - Wait for EAE to finish starting up. You should see the below message in eae-service.log file (/usr/local/jive/var/logs/eae-service.log):
com.jivesoftware.eae.stream.lucene.LuceneFilteredStreamService - Stream service started -
Ensure the following are fully up and running: Cache, Search, Doc Converter
- Check for this message in cache-service.out:
INFO Startup completed in X ms - Check for this message in docconverter.out:
INFO: Server startup in [X] milliseconds - Check for this message in search/stdout.log:
jive_cli started process (5053)
- Check for this message in cache-service.out:
-
Start the web nodes one-by-one. Wait for one to complete startup, before starting the next. Look for below message in sbs.out file of the nodes:
Server startup in [X] milliseconds - If you hit a deadlock, restart the affected node without restarting the EAE server.
<supportagent>
It is possible to encounter the long times to start the web app nodes waiting for query results, and it would be "an indication of database issues, that need to be investigated with the help of a DBA.
</supportagent>
You can upgrade to the latest version of Jive to avoid the deadlock issue in future.
Testing
All the web nodes and EAE service should complete starting up. News and Activity Streams should be visible and should update for newly added content.
Priyanka Bhotika
Comments