MongoDB replica set in kubernetes was partially broken.
For some reason MongoDB replicaset on dev env was broken:
Mongo created 2 separate replica sets - (procedure-mongo-0, procedure-mongo-1) , (procedure-mongo-2).
Fixed this issue by deleting procedure-mongo-2 pod + persistent volume.
Most likely issue was caused by init sequence of mongo pods when pod-2 was not able to contact pod-0,pod-1. We need to ensure that Monitoring system for Dev/Stage/Prod envs will show MongoDB statuses and ensure that replicas are healthy