Nondeterministic tests are a common problem in test setups, especially with integration tests. For example, imagine you are doing an integration test of two components that exchange messages where receiving a message in one of the systems changes its internal state which you want to verify. However, the message exchange can be delayed due to external factors (network latency, load etc.). When running tests (for example using the excellent Cucumber framework), testing steps are usually run sequentially. It may happen that the tested system is lagging behind the testing steps, failing your test in some (but not all) cases. You need to make your tests more tolerant for these situations. Often, that means retrying for a couple of times.
How would you implement this pattern in Java? Well, here’s how I did it. I am using a timed runner object that tries an action until it succeeds before it times out. You can define what should happen in case the action times out, by overwriting the abstract method.
/**
* Use this class to run actions with a timeout.
* Actions have to be wrapped within
* a yield method of the Yieldable
* interface.
*/
public abstract class TimedRunner {
private int timoutLength;
private int elapsedTime;
private int checkRate = 250;
public TimedRunner(int length) {
timoutLength = length;
elapsedTime = 0;
}
public TimedRunner() {
timoutLength = 10000;
elapsedTime = 0;
}
public synchronized void reset() {
elapsedTime = 0;
}
public void run(Yieldable runMe) {
for (; ;) {
try {
Thread.sleep(checkRate);
}
catch (InterruptedException ioe) {
continue;
}
boolean success = runMe.yield();
elapsedTime += checkRate;
if (success) {
break;
}
if (elapsedTime > timoutLength) {
timeout();
break;
}
}
}
// just overwrite this method to perform the timeout action
public abstract void timeout();
}Now, in other languages, like Ruby or Python you could just call a method and pass it a block of instructions. As that’s not possible in Java, you need to wrap the actions you want to pass within a method belonging to an object you can pass along. That object will be an instance implementing the Yieldable interface:
/**
* Implement this interface when you want
* to use TimedRunner. TimedRunner
* expects a Yieldable object, class its yield
* method and calls it until
* it succeeds or a timeout is reached.
*/
public interface Yieldable {
public boolean yield();
}The yield() method needs to return a status of success or failure so that the TimedRunner knows when to stop calling it. You could extend/modify this pattern by changing the signature of yield() to match your specific needs.
What should be done in case of a timeout? The most common thing to do would be to fail the test (assuming usage of JUnit 4):
import junit.framework.Assert;
public class FailRunner extends TimedRunner {
@Override
public void timeout() {
Assert.fail("Timed out without success.");
}
public FailRunner(int timeout) {
super(timeout);
}
}In your test code, you would put the nondeterministic code within a yield method and pass it to a TimedRunner instance:
// ... within your test code...
Yieldable runMe = new Yieldable(){
public boolean yield() {
// do something here and return true
// if it succeeded or false otherwise
}
};
new FailRunner(30000).run(runMe);
// ...Feel free to leave feedback in the comments. I would be interested to learn how you handle testing issues such as this one.