You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 24 Next »

Motivation

For now, Ignite has not build-in profiling tool for user's operations and internal processes. Such a tool will be able to collect performance statistics and create a human-readable report. It will help to analyze workload and to tune configuration and applications.

Example of similar tools in other products: AWR [1] [2] [3] (Oracle) ; pgbadger [4], pgmetrics [5], powa [6] (PostgresSQL).

Description

We should provide a way to execute cluster profiling. Consider the following scenario:

  • Enable profiling mode.
  • Executes some arbitrary workload.
  • Collects profiling info.
  • Run the tool that will create the Report contains statistics of workload.

Performance report

The performance report will be in a human-readable format (html page) and should contain:

  • Ignite and plugins versions, topology changes, profiling start/end time
  • Queries (SQL, scan, ..) timings, resources:
    • Queries that took up the most time
    • Slowest queries
    • Most frequent queries
    • Failing queries
    • Queries count by type
    • Queries that took up the most CPU/IO/Disk
  • User tasks timings, resources
    • Jobs of slowest tasks
  • Caches and cache operations statistics:
    • Get/Put/Remove
    • Transactions
    • Invoke
    • Lock
    • create/destroy caches
  • Workload by nodes
    • CPU/IO/Disk resources
  • Checkpoints statistics
  • WAL statistics
  • PME statistics

Additional investigation required to gather following statistics:

  • Query parse time
  • Lock waiting time
  • User time
  • Messages process timings

Proposed Changes

The Ignite will log some additional internal statistics using separate log category of IgniteLogger. Later it will be some binary format.

The new ignite-profiling module will be introduced. It will contain:

  • The tool to build the report: report.sh(bat)

Public API changes

The new module will be created: ignite-profiling.

Risks and Assumptions

Enabled profiling mode will cause performance degradation.

Discussion Links

// Links to discussions on the devlist, if applicable.

Report example


Reference Links

  1. https://docs.oracle.com/cd/E11882_01/server.112/e41573/autostat.htm#PFGRF94176
  2. http://www.dba-oracle.com/t_sample_awr_report.htm
  3. http://expertoracle.com/2018/02/06/performance-tuning-basics-15-awr-report-analysis/
  4. https://github.com/darold/pgbadger
  5. https://pgmetrics.io/docs/index.html#example
  6. https://powa.readthedocs.io/en/latest/

Tickets

Unable to render Jira issues macro, execution error.



  • No labels

Apache Ignite