THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
HDFS per-user Metrics aren't emitted by default. Kindly exercise caution before enabling them and make sure to refer to the details of client and service port numbers.
To be able to use the HDFS - Users dashboard in your Grafana instance as well as to view metrics for HDFS per user, you will need to add these custom properties to your configuration.
Step-by-step guide
In Ambari, HDFS > Configs > Advanced > Custom hdfs-site, Add the following properties.
| Key | Value |
|---|---|
| dfs.namenode.servicerpc-address | namenodehost:8021 |
| ipc.8020.callqueue.impl | org.apache.hadoop.ipc.FairCallQueue |
| ipc.8020.backoff.enable | true |
| ipc.8020.scheduler.impl | org.apache.hadoop.ipc.DecayRpcScheduler |
| ipc.8020.scheduler.priority.levels | 3 |
| ipc.8020.decay-scheduler.backoff.responsetime.enable | true |
| ipc.8020.decay-scheduler.backoff.responsetime.thresholds | 10,20,30 |
Things to consider:
- client port : 8020 (if different, replace it with appropriate port in all keys)
- service port: 8021 (if different, replace it with appropriate port in first value)
- namenodehost: needs to be a FQDN.
Once these properties are added, it should look like this.
Restart Metrics Collector & you should see the metrics being emitted. You should now also be able to use the HDFS - Users Dashboard in Grafana.
Related articles