We found the root cause of the problem. The problem is related with the roaming profiles being used on Citrix Env. The field support team disabled it and we observed that everything gets back to normal.
Actually we don't want this issue even with roaming profile on but at least we figured out the root cause and solved by turning roaming profile off.
Turning roaming profile off brings some waiting time (around additional 1min) each time the Citrix terminal restarts but this is not happening very frequently.