In the outgoing queues we had unacknowledged messages, sometimes on server 1 sometimes on server 2.
The two servers were cloned … and they had the same QMID
Source : from John Breakwell:
When you send a message, the sender’s Queue Manager ID (QMID) is sent with it. The destination builds a cache table mapping QMID to IP address of sender. This is to speed up sending acknowledgements back.
If two sending machines have the same QMID then the destination machine will have only one cache entry for both of them so acknowledgements with either all be sent to sender1 or all be sent to sender2. Any acknowledgements sent to the wrong machine will be discarded.
More info: MSMQ prefers to be unique