The root of the problem is that invalidation evicts remote nodes which may mean that remote nodes are removed completely, losing all historic versioning data
1) Svr 1 generates state for node (data version 1)
2) Svr 2 generates newer state for same node (data version 2)
3) Svr 2 commits (data version 2), invalidates state on cache 1
4) Svr 1 commits, and this is allowed to proceed since it seems cache 1 created this data new, and it does not exist in cache 1.
5) Svr 1 will attempt to invalidate this remotely, but this will fail - but it will still be applied locally.
Solution: Invalidation not to remove nodes, but evict nodes as though they had children (remove data maps and set dataInitialised to false), and (if optimistic) attach the version of the invalidation to the node.
Eviction to have, in addition to maxTTL, a minTTL as well so that nodes spend at least a certain amount of time in the cache even if scheduled for eviction since even if we have the above eviction may come along and remove the node altogether.
MinTTL to be configured to be greater than the max length of transactions.