Aha! from O'Rielly's Building Microservices

 

Many times in books there are moments where the ideas open my eye for their novelty and I say Aha!. Over the time however they are lost and forgotten. Therefore, I am sharing them here so they can be revisted.

1. Context : choosing of which type of communication, here the author is discussing Event Driven Style of communication, here he is discussing the cavets. 
1.  "We were using a transcational queue, as a worker died , its lock on the request would time-out and the message will. be put back in the queue, to be picked up my another worker to pick up and die" . This is called as Catastropic Failover.   This was solved by putting a maximum retry for the job on the queue.  And diverting these messages to a "Message Hospital" / dead letter queue where these messages could be observed. 
   
   
   For event driven communication, we also need a good system of monitoring, strong correlated Ids that trace the request across the boundries. 
   
   
   Solution to this problem is typlical with any communication type of = Timeouts.  Pattern is Timeout for communcation is a good fail safe mechanism.