DUnit tests are junit JUnit tests that extend CacheTestCase or DistributedTestCase and run code in more than one VM. Because these tests tend to involve multiple members in a distributed system, these tests are more prone to potential race conditions. Here are some tips for tracking down failures and fixing these tests.

...

Running a single test

You can use the standard gradle Gradle properties to run a single test. Or, better yet, run the test in your IDE to help you debug it.

...

To save time, the JVMs are reused between test cases. This can cause issues if previous tests leave the JVM in a bad state. You can see the list of previously executed tests as the first step in the stdout of a test, if you are looking at a failure from jenkinsJenkins:

Previously run tests: [PartitionedRegionHAFailureAndRecoveryDUnitTest, NetSearchMessagingDUnitTest, EvictionObjectSizerDUnitTest, PartitionedRegionBucketCreationDistributionDUnitTest, InterruptsConserveSocketsFalseDUnitTest, PartitionedRegionRedundancyZoneDUnitTest, DeltaFaultInDUnitTest, PutAllGlobalDUnitTest, DeltaPropagationStatsDUnitTest, P2PDeltaPropagationDUnitTest, PartitionedRegionPRIDDUnitTest, TransactionsWithDeltaDUnitTest, Bug33726DUnitTest, PartitionedRegionTestUtilsDUnitTest]

...

To run the same tests in order, you can add a suite to your junit JUnit test

public static Test suite() {
    Class[] classes = new Class[] {PartitionedRegionBucketCreationDistributionDUnitTest.class, InterruptsConserveSocketsFalseDUnitTest.class, PartitionedRegionRedundancyZoneDUnitTest.class, PartitionedRegionTestUtilsDUnitTest.class};
    return new TestSuite(classes);
  }

...

Turn on debug level in dunit tests

There 're are many ways to run a test case with a specific debug level, ; the easiest way is to temporarily add the following code into to the test program temporarily.

  @Override
  public Properties getDistributedSystemProperties() {
    Properties props = new Properties();
    props.setProperty("log-level", "debug");
    return props;
  }

...

Turn on trace level using log4j2.xml

There're Product code contains some trace code in product code like similar to the following: 

      if (logger.isTraceEnabled(LogMarker.TOMBSTONE)) {
        logger.trace(LogMarker.TOMBSTONE, "Destroyed entries sweeper starting with default sleep interval={}", this.expiryTime);
      }

They This will not be displayed in a debug level log. To specify a customized log4j2.xml, there 're are also several ways, but the easiest one is to copy your log4j2.xml into open/gemfire-core/src/main/resources/log4j2.xml. Here is an example of log4j2.xml which turned that turns on LogMarker.TOMBSTONE, LogMarker.DISTRIBUTION, LogMarker.DISK_STORE_MONITOR:

...

The default heap size for dunit a DUnit test is 768m. We can modify open/build.gradle to be use a bigger larger heap size in case the dunit DUnit test run runs out of memory. 

        maxHeapSize '768m'
        jvmArgs = ['-XX:+HeapDumpOnOutOfMemoryError', '-ea']
        systemProperties = [
          'gemfire.DEFAULT_MAX_OPLOG_SIZE' : '10',
          'gemfire.disallowMcastDefaults'  : 'true',
          'jline.terminal'                 : 'jline.UnsupportedTerminal',
        ]

...

We've introduced com.jayway.awaitility.Awaitility.await in geodeGeode. There's a better way to replace our old WaitCriterion. await() can also be used in product code, while WaitCriterion is only in DistributedTestCase. 

...