Question

4
Replies
61
Views
Close popover
Rahul Roshan (Roshan)
Tata Consultancy Services
Rahul Roshan
Tata Consultancy Services
GB
Roshan Member since 2017 2 posts
Tata Consultancy Services
Posted: January 18, 2021
Last activity: January 19, 2021

Kubernetes Inter POD Communication for Pega search

Pega PODS are unable to communicate with each other for Pega default Elastic Search.

Background:

We are trying to build Pega  8.5.1, in Client Cloud Azure with Docker Image and Kuberenets as container orchestrator. Pega is installed and first master node is brought up with node types as - BackgroundProcessing,Search,Stream,Batch. Search Index built properly and is in available staste in Search landing page.

However, when next node as WebUser node type is brought up Search index status is Unavailable in Landing page and Rule,Data search fails with error as-"Unable to reach master node" .

Observation:

Each POD has a Private IP and through Kubernetes Services has a Public/Cluster IP. In pr_sys_statusnodes table pyclusteraddress is occupied by PublicIP:5701 whereas pyindexaddress is occupied with privateIP:9300 for each node respectively. 

Hazlecast is working properly after setting identification/cluster/public/address= Cluster IP of Master node in prconfig.xml and both node visible in Admin Studio-System & Nodes.

However when WebUser node is brought up/ restarted it throws below error in logs and search doesn't work.

2021-01-14 12:27:18,282 [oundProcessControl-6] [  STANDARD] [                    ] [                    ] (BackgroundProcessEventListener) INFO    - Starting service [Aggregation.Default] 2021-01-14 12:27:18,282 [oundProcessControl-6] [  STANDARD] [                    ] [                    ] (BackgroundProcessEventListener) INFO    - Service [Aggregation.Default] started 2021-01-14 12:27:25,067 [il INITIALIZE_SEARCH] [  STANDARD] [                    ] [                    ] (SearchManager.ESSearchProvider) INFO    - Successfully started ElastcSearch node 2021-01-14 12:27:55,140 [il INITIALIZE_SEARCH] [  STANDARD] [                    ] [                    ] (SearchManager.ESSearchProvider) WARN    - Failed to connect to the master node. Please contact your system administrator. Reason: null 2021-01-14 12:27:55,140 [il INITIALIZE_SEARCH] [  STANDARD] [                    ] [                    ] (Manager.PegaSearchProviderImpl) INFO    - FTS node initialization: updating node index directory to '' on host 'pegaforegroundnew-5d67dc8575-mbksg' - make sure that different nodes don't share it! 2021-01-14 12:28:25,357 [il INITIALIZE_SEARCH] [  STANDARD] [                    ] [                    ] (Manager.PegaSearchProviderImpl) ERROR   - Cannot enable full-text search for node PEGAFOREGROUNDNEW-5D67DC8575-MBKSG. Possible cause: None of the search index host nodes are reachable. 2021-01-14 12:28:55,400 [il INITIALIZE_SEARCH] [  STANDARD] [                    ] [                    ] (artup.SearchInitializationTask) ERROR   - Finished configuration of node for full-text search, with errors.

 

How to make PODS communicate and reach master node for Pega Search in Kubernetes?

Any help is highly appreciated.

***Edited by Moderator: Pooja Gadige to change content format from Discussion to Question, add product details tag***
Pega Cloud Pega Platform 8.5.1 Cloud Services Client-managed Cloud Financial Services Lead System Architect