Posted: 21 Apr 2016 12:31 EDT Last activity: 5 May 2016 12:57 EDT
How Pega CHAT can connect to MongoDB replica set?
We have created MongoDB replica set where 2 Mongo DB instances are running in different VM one as Primary and one as secondary. In the Pega CHAT Config.json file we only have option to mention the connection details for Primary MongoDB. Please let us know how Pega CHAT can connect to the replica set to support the automatic failover meaning if Primary goes down then Pega CHAT will automatically get connected with secondary and work as it is.
I know, but in the config.json file we are binding chat server with a specific mongo db instance by specifying the hostname and port. If the host goes down the pega chat is stopped working even if we have a secondary mongodb instance running one different host/same host but diffrent port.
Pega CHAT app has only 1 adres (url) which should always point to the primary node of replica set
On all replica set nodes there are scripts who check which one is the primary and in case it changes (some other node will be elected to be primary) the url will get updated
From what i understand about replica sets in mongodb the must be at least 3 nodes to have HA because there must be at least 2 to elect a new primary
So if you have only 2 nodes then if the primary will go down your secondary won't get elected to primary automatically , you will need to force it manually
If you will put secondary nodes in the app config you will still need to do some checks to see which one of them has been elected to be primary
I think it will be easier to have only 1 url in the config and just make sure it points to primary this will give us more flexibility in changing replica set configuration (ex. adding new replica set nodes)
Any approach we take there will always be some disruption in service because it takes replica set some time (up to 20 s) to elect a new primary
Thanks for your reply. You have mentioned that if the primary gets changed then the URL will get updated. >> 1. Is this the URL mentioned in config.json file? If so, then how will this be updated automatically? If updated somehow it will require a server restart and which will have its own downtime?
However the be the configuration is (2/3 repl set) we are concerned about the config.json file. If the node mentioned in that file goes down then how automatically other node will work without any disruption in the service? Please give us an example of the configuration you have done in EU. Thanks.
To clarify how it works We have a dns record ex. primary.mongodb.example.com this should always resolve to the ip of mongodb replica set primary node
On all replica set nodes there is a custom script that runs every minute that connects to mongodb and checks which node is the primary
If the primary node changes than this scripts connects to the dns server and updates the ip to which this primary.mongodb.example.com points to
So in jcon.config you only put this url primary.mongodb.example.com and the infrastructure makes sure that when you resolve this url you will get the current primary node ip
"If the node mentioned in that file goes down then how automatically other node will work without any disruption in the service" there will be a short distruption of service consisting of 2 things
1. Time that is needed for mongodb to elect a new node (it can take from couple of seconds to 20s)
2. Time the script takes to find out that a new primary node is elected and to update the dns (script execution time is under 2s but it's run every minute so if the primary node changes ex. 10:10:30 the script will pick it up at 10:11:00 ) worst case scenario there will be a 90s disruption in mongodb availability
"If updated somehow it will require a server restart and which will have its own downtime"
I can either create a script that automatically restarts CHAT service if i see errors (ex. mongodb connection failure ) in the logs or the app itself if it fails to connect to mongodb can automatically retry each time resolving the mongodb url from config.json
The custom script you have mentioned in your reply (custom script that runs every minute that connects to mongodb) >> is it written by you or available as a part of mongodb setup or is it a part of Pega CHAT Software?
"To achieve a fault tolerant infrastructure with no single point of failure, it is recommended that you install the Pega Chat service as a multi server setup with each Node.js based chat server configured to interact with a MongoDB replica set setup and a Redis master slave setup"
And this configuration does not work out-of-the-box.
Could you please help us here and provide us the custom mongoDB sample scripts which we can work on top of and reuse if possible or provide some other solution to make this configuration work. Many thanks!