Close popover
Donn Rochette (DonnR527)
Air Force AFRL

Air Force AFRL
DonnR527 Member since 2018 2 posts
Air Force AFRL
Posted: March 4, 2019
Last activity: March 8, 2019

Zookeeper sockets in CLOSE_WAIT state

We have encountered a situation several times where we are getting fatal errors due to too many open files. It turns out that we had 811,992 open sockets, 807,057 in CLOSE_WAIT state, 2,315 in a connected or listening state. The limit for open descriptors for the Java process is 4096. The 807k sockets in CLOSE_WAIT state are connecting to the Zookeeper socket on the same host;
"TCP host101:eforward->host101:39565 (CLOSE_WAIT)".
The eforward port, 2181, is the default Zookeeper port, the port 39565, is of course, ephemeral.

I saw reference to a post describing a similar issue with too many sockets in CLOSE_WAIT state in a SOAP scenario. The suggested action that case was to change timeouts for Axis2.

Is this a known issue? If so, is there a setting for Kafka or ZK that we need to consider?

System Administration
Moderation Team has archived post