Apply future deadlocks if quorum cannot be reached
I believe this is a duplicate of #145 which was closed due to no activity but appears to be unresolved.
I have calls to Apply which sometimes occur at the same time as nodes in the cluster being stopped, e.g. in unit/integration tests. In this situation, waiting on the result from the future returned by Apply (deferError.Error()) will block forever, causing the test to deadlock. This is because deferError is waiting on a response from the errCh which is never going to come.
Interestingly, a change was added a while back to add a ShutdownCh to deferError to allow futures to react to shutdowns. However, this appears to only be used to address a deadlock that occurred when snapshots are taken.
It seems to me like deferError ought to have a timeout on it, either taking its own timeout on calls to Error() or by respecting the timeout passed to Apply(). Given that the future returned by Apply involves RPCs and consensus, it seems like this should have a timeout to prevent potential deadlock situations.
Source: hashicorp/raft