Page History

...

This document covers item 4 - submission of tests

Consolidated Test Data Overview

Elasticsearch has been chosen as the storage repository for the combined data, due to the fact that it is designed to handle text documents and it's general popularity which means a large number of people will be able to work with it.

Elasticsearch stores data as JSON documents, the data ingestion is therefore designed to be natively in JSON to avoid the need for additional processing/complexity - ie no need for logstash.

Along with the individual test results, various pieces of metadata are required to form a full picture of the test results.

These are collected in the following files from a Marvin test run:

• test_data.py -> test_data.py_contents
• marvin_cfg file -> env_data
• nose test output (in JSON) -> results
• additional test run metadata -> test_metadata

JSON document structure

{
  "testrun": {
    "testrun_uuid": "$TESTRUN_UUID",
    "results": [{
      "classname": "tests.smoke.test_global_settings:TestUpdateConfigWithScope",
      "type": "success",
      "name": "test_UpdateConfigParamWithScope",
      "time": 0.18374300003051758
      }],
    "test_data.py_contents": "%23+Licensed+to+the+Apache+Software+Foundation+%28ASF%29+under+on......",
    "test_metadata": {
      "base_cloudstack_version": "4.10",
      "pr_id": "null",
      "rc_num": "null",
      "marvin_tags": "advanced",
      "marvin_hypervisor": "kvm",
      "hypervisor_version": "CentOS 6",
      "hypervisor_minor_version": "",
      "mgmt_server_os": "CentOS 7",
      "mgmt_server_os_ver": "7.3",
      "testrun_date": ""
      },
    "env_data": ""

  }
}

Elasticsearch configuration

PUT /test_results

{
  "mappings": {
    "testruns": {
      "properties": {
        "testrun_uuid": {
          "type": "text"
        },
        "results": {
          "type": "nested"
        },
        "env_data": {
          "type": "nested"
        },
        "test_metadata": {
          "type": "nested"
        }
      }

    }
  }
}

PUT test_results/_settings
{
  "index.mapper.dynamic":true 
}