I am using Pega 8.3 and I have decommissioned a JVM as a Stream node and made it a WebUser Node. When the now WebUser node starts, it appears it is still trying to start the stream service given the following error:
api.server.StreamServerService) ERROR - Unable to start stream REST endpoint
org.jboss.netty.channel.ChannelException: Failed to bind to: /x.x.x.x:7003.
If I start the new stream JVM first and then the old stream/now WebUser, both JVMs start fine with the exception of the error above. If start the old stream first and the new stream second, the new stream will not start because the port 7003 is in use by the old stream node even though it's a WebUser node. Both JVMs reside on the same server.
Thanks for the response Kevin. Working with you last week, we truncated those 3 tables once we made the new node designation in one of our environments. That environment is experiencing the issue was well where the WebNode tries to start the stream service. The odd thing is, if the WebNode boots up first, it uses the port 7003, but if you look in the temp directory, it isn't running any kafka services. I've made the node designation switch in 2 other environments as well and they all experience the same thing. I just tried truncated the tables in another environment one of the 3 environments, just to make sure and the issue still persists.
Thanks for the response. I added the DSS setting and it only moved the problem to a different port:
No services running on the host:
1. lsof -i:9889 No return (meaning the port is open)
2. started WebUser Node
3. lsof -i:9889 Returned the PID of the WebUser Node
4. Started Stream Node While starting i received the following error: ERROR - Cannot start service [StreamServer.Default]. Will retry in 180 seconds. Remaining attempts: 2
org.jboss.netty.channel.ChannelException: Failed to bind to: /xx.xx.xx.xx:9889
So with the DSS in place, it just moved the problem from port 7003 to port 9889. Again, these JVMs are on the same host.