If you have a requirements like  map key1 and key2 to a value, this article is for you.

For example, In IoTDB, you want to 

(1) get some info of a given file name and a given device name.

(2) get some info of a given start time and a end time.


This is a simple test for comparing different data structure: Map<Pair<Key1, Key2>, Value>, Map<Key1, Map<Key2, Value>>, and MultiKeyMap<Key1, Key2, Value>.


import java.util.HashMap;
import java.util.Map;
import java.util.Random;
import java.util.stream.IntStream;
import org.apache.commons.collections4.map.MultiKeyMap;
import org.apache.iotdb.tsfile.utils.Pair;
import org.junit.Test;

public class MapPerformanceTest {
  int dataSize = 4000000;
  int key1Size = 1000;
  int key2Size = 300000;

  @Test
  public void test1() {
    Map<Pair<Long, Long>, Long> data = new HashMap<>();
    Random random = new Random();
    long start, write, read;
    start = System.currentTimeMillis();
    IntStream.range(1, dataSize).forEach(i -> data.put(new Pair<>(random.nextLong()%key1Size, random.nextLong()%key2Size), random.nextLong()));
    write = System.currentTimeMillis() - start;
    start = System.currentTimeMillis();
    IntStream.range(1, dataSize).forEach(i -> data.get(new Pair<>(random.nextLong()%key1Size, random.nextLong()%key2Size)));
    read = System.currentTimeMillis() - start;
    System.out.println("Map<Pair<Long, Long>, Long>:\t" + write +"\t" + read);
  }
  @Test
  public void test2() {
    MultiKeyMap<Long, Long> data = new MultiKeyMap<>();
    Random random = new Random();
    long start, write, read;
    start = System.currentTimeMillis();
    IntStream.range(1, dataSize).forEach(i -> data.put(random.nextLong()%key1Size, random.nextLong()%key2Size, random.nextLong()));
    write = System.currentTimeMillis() - start;
    start = System.currentTimeMillis();
    IntStream.range(1, dataSize).forEach(i -> data.get(random.nextLong()%key1Size, random.nextLong()%key2Size));
    read = System.currentTimeMillis() - start;
    System.out.println("MultiKeyMap<Long, Long>:\t" + write +"\t" + read);
  }
  @Test
  public void test3() {
    Map<Long, Map<Long, Long>> data = new HashMap<>();
    Random random = new Random();
    long start, write, read;
    start = System.currentTimeMillis();
    IntStream.range(1, dataSize).forEach(i ->
      data.computeIfAbsent(random.nextLong()%key1Size, k -> new HashMap<>()).put(random.nextLong()%key2Size, random.nextLong())
    );
    write = System.currentTimeMillis() - start;
    start = System.currentTimeMillis();
    IntStream.range(1, dataSize).forEach(i -> data.get(random.nextLong()%key1Size).get(random.nextLong()%key2Size));
    read = System.currentTimeMillis() - start;
    System.out.println("Map<Long, Map<Long, Long>>:\t" + write +"\t" + read );
  }

}


How to use:


For example, you want to check the performance under a workload like: I will use the map to cache the metadata of 10000 tsfiles and in each file, there is about 3 million time series. Totally, I use the cache to cache 1 million metadata.

Then, set dataSize = 1000000, set key1Size=10000, and set key2Size = 3000000. 







  • No labels