You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

 

DUnit tests are junit tests that extend CacheTestCase or DistributedTestCase and run code in more than one VM. Because these tests tend involve multiple members in a distributed system, these tests are more prone to potential race conditions. Here are some tips for tracking down failures and fixing these tests.

 

Running a single test

You can use the standard gradle properties to run a single test. Or, better yet, run the test in your IDE to help you debug it.

 

./gradlew gemfire-core:distributedTest -DdistributedTest.single=PartitionedRegionTestUtilsDUnitTest

 

Running a test multiple times

One way to run a test a lot of times is to add a new method to your case test that runs a problematic method many times. Here's an example

 

  public void testLoop() throws Exception {
    for(int i=0; i < 200; i++) {
      testGetBucketOwners();
      tearDown();
      setUp();
    }
  }

 

Running a list of tests

 

To save time, the JVMs are reused between test cases. This can cause issues if previous tests leave the JVM in a bad state. You can see the list of previously executed tests as the first step in the stdout of a test, if you are looking at a failure from jenkins:

 

Previously run tests: [PartitionedRegionHAFailureAndRecoveryDUnitTest, NetSearchMessagingDUnitTest, EvictionObjectSizerDUnitTest, PartitionedRegionBucketCreationDistributionDUnitTest, InterruptsConserveSocketsFalseDUnitTest, PartitionedRegionRedundancyZoneDUnitTest, DeltaFaultInDUnitTest, PutAllGlobalDUnitTest, DeltaPropagationStatsDUnitTest, P2PDeltaPropagationDUnitTest, PartitionedRegionPRIDDUnitTest, TransactionsWithDeltaDUnitTest, Bug33726DUnitTest, PartitionedRegionTestUtilsDUnitTest]

 

To run the same tests in order, you can add a suite to your junit test

 

public static Test suite() {
    Class[] classes = new Class[] {PartitionedRegionBucketCreationDistributionDUnitTest.class, InterruptsConserveSocketsFalseDUnitTest.class, PartitionedRegionRedundancyZoneDUnitTest.class, PartitionedRegionTestUtilsDUnitTest.class};
    return new TestSuite(classes);
  }

 

Fixing suspect strings

Distributed tests check the log output from each test for "suspect strings" which include any warnings, errors, or exceptions. If your test is supposed to log one of these strings, you can add an expected exception to the beginning of your test method, or even in your setUp method if all test cases are expected to log this message. There is no need to remove the expected exception, it is automatically removed in tearDown().

 

  public void testClientPutWithInterrupt() throws Throwable {
    addExpectedException("InterruptedException");
   //...
  }

 

Turn on debug level in dunit tests

There're many ways to run a test case with debug level, the easiest way is to add following code into the test program temporarily.

 

  @Override
  public Properties getDistributedSystemProperties() {
    Properties props = new Properties();
    props.setProperty("log-level", "debug");
    return props;
  }

 

Turn on trace level using log4j2.xml

There're some trace code in product code like following: 

 

      if (logger.isTraceEnabled(LogMarker.TOMBSTONE)) {
        logger.trace(LogMarker.TOMBSTONE, "Destroyed entries sweeper starting with default sleep interval={}", this.expiryTime);
      }

They will not be displayed in debug level log. To specify a customized log4j2.xml, there're also several ways, but the easiest one is to copy your log4j2.xml into open/gemfire-core/src/main/resources/log4j2.xml. Here is an example of log4j2.xml which turned on LogMarker.TOMBSTONE, LogMarker.DISTRIBUTION, LogMarker.DISK_STORE_MONITOR:

 

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="ERROR" shutdownHook="disable" packages="com.gemstone.gemfire.internal.logging.log4j">
  <Properties>
    <Property name="gemfire-pattern">[%level{lowerCase=true} %date{yyyy/MM/dd HH:mm:ss.SSS z} &lt;%thread&gt; tid=%tid] %message%n%throwable%n</Property>
  </Properties>
  <filters>
    <MarkerFilter marker="DISTRIBUTION" onMatch="ACCEPT" onMismatch="NEUTRAL"/>
    <MarkerFilter marker="DISK_STORE_MONITOR" onMatch="ACCEPT" onMismatch="NEUTRAL"/>
    <MarkerFilter marker="TOMBSTONE" onMatch="ACCEPT" onMismatch="NEUTRAL"/>
    <!--MarkerFilter marker="PERSIST_RECOVERY" onMatch="ACCEPT" onMismatch="NEUTRAL"/-->
    <!--MarkerFilter marker="PERSIST_WRITES" onMatch="ACCEPT" onMismatch="NEUTRAL"/-->
    <!-- have to explicitly DENY all the markers' log, then explicitly add back one by one -->
    <MarkerFilter marker="GEMFIRE_MARKER" onMatch="DENY" onMismatch="NEUTRAL"/>
  </filters>
  <Appenders>
    <Console name="STDOUT" target="SYSTEM_OUT">
      <PatternLayout pattern="${gemfire-pattern}"/>
    </Console>
    <Console name="STDERR" target="SYSTEM_ERR">
      <PatternLayout pattern="${gemfire-pattern}"/>
    </Console>
    <!--RollingFile name="RollingFile" fileName="${filename}"-->
    <File name="Log" fileName="system.log" bufferedIO="true">
      <PatternLayout pattern="${gemfire-pattern}"/>
    </File>
  </Appenders>
  <Loggers>
    <Logger name="com.gemstone" level="INFO" additivity="false">
      <AppenderRef ref="Log"/>
    </Logger>
    <Logger name="com.gemstone.gemfire.distributed.internal" level="TRACE" additivity="false">
      <AppenderRef ref="STDOUT" level="DEBUG"/>
    </Logger>
    <Logger name="com.gemstone.gemfire.cache.client.internal" level="TRACE" additivity="false">
      <AppenderRef ref="STDOUT" level="DEBUG"/>
    </Logger>
    <Logger name="com.gemstone.gemfire.internal.cache" level="TRACE" additivity="false">
      <AppenderRef ref="STDOUT" level="TRACE"/>
    </Logger>
    <!--Logger name="com.gemstone.gemfire.distributed.internal.DistributionManager" level="TRACE" additivity="false">
      <AppenderRef ref="Log" level="DEBUG"/>
    </Logger-->
    <!--Logger name="com.gemstone.gemfire.distributed.internal.DistributionMessage" level="TRACE" additivity="false">
      <AppenderRef ref="Log" level="DEBUG"/>
    </Logger-->
    <Root level="ERROR">
      <!--AppenderRef ref="STDOUT"/-->
    </Root>
  </Loggers>
</Configuration>
  • No labels