THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Disable IPv6, leaving it enabled may force service to bind to IPv6 addresses only and thus resulting in inability to connect to it (source link):
Code Block sysctl -w net.ipv6.conf.all.disable_ipv6=1sysctl1 sysctl -w net.ipv6.conf.default.disable_ipv6=1echo1 echo -e "\n# Disable IPv6\nnet.ipv6.conf.all.disable_ipv6 = 1\nnet.ipv6.conf.default.disable_ipv6 = 1" >> /etc/sysctl.conf
...
| Code Block |
|---|
yum install epel-release -yyumy yum update -yyumy yum install git wget curl rpm scp tar unzip wget createrepo reposync yum-utils ntp python-pip -y |
...
| Code Block |
|---|
pip install --upgrade pippippip pip install --upgrade setuptools |
...
| Code Block |
|---|
echo 'export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s_/jre/bin/java__")' > /etc/profile.d/java_18.shchmodsh chmod +x /etc/profile.d/java_18.sh |
...
| Code Block |
|---|
wget http://apache.volia.net/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gztargz tar -zxf apache-maven-3.3.9-bin.tar.gz; mv apache-maven-3.3.9 /opt/export PATH=/opt/apache-maven-3.3.9/bin:$PATHecho$PATH echo 'export PATH=/opt/apache-maven-3.3.9/bin:$PATH' > /etc/profile.d/maven.shchmodsh chmod +x /etc/profile.d/maven.sh |
...
| Code Block |
|---|
yum install docker-io -yservicey service docker start |
Build Metron code:
...
If you install Metron on single node (not multi-node as advised) you need to modify ElasticSearch config templates to use only configuration specified below. Config templates are located in incubator-metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/ELASTICSEARCH/2.3.3/package/templates/
| Code Block |
|---|
cluster.name: metronnetworkmetron network.host: ["_eth0:ipv4_","_local:ipv4_"] discovery.zen.ping.unicast.hosts: [ <single_node_hostname> ] path.data: /opt/lmm/es_dataindexdata index.number_of_replicas: 0 |
Fix Kibana install file (it shoild should be fixed in METRON-641:
| Code Block |
|---|
sed -i 's@{}/kibana@{0}/kibana@g' incubator-metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/KIBANA/4.5.1/package/scripts/kibana_master.py |
Build Metron with HDP 2.5 profile:
| Code Block |
|---|
cd incubator-metronmvnmetron mvn clean install -DskipTests -PHDP-2.5.0.0cd0 cd metron-deployment/packaging/docker/rpm-dockermvndocker mvn clean install -DskipTests -PHDP-2.5.0.0 |
On all nodes create localrepo directory and copy RPMs from Ambari node there:
| Code Block |
|---|
mkdir /localrepocplocalrepo cp -rp /root/incubator-metron/metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/* /localrepo/ |
...
| Code Block |
|---|
scp /localrepo/* <replace_with_node_ip>:/localrepo/ |
On If passwordless ssh has not yet been set up within the cluster, then in main node generate key and add it to trusted:
| Code Block |
|---|
cat /dev/zero | ssh-keygen -q -N "" 2>/dev/nullsshnull ssh-copy-id -i ~/.ssh/id_rsa.pub <replace_with_master_node_ip>chmodip> chmod 700 ~/.ssh |
Add this key to all the slave nodes:
...
On all nodes disable security:
| Code Block |
|---|
ulimit -n 10000ulimit10000 ulimit -u 257597chkconfig257597 chkconfig ntpd onserviceon service ntpd startchkconfigstart chkconfig iptables off /etc/init.d/iptables stopsetenforcestop setenforce 0 |
Make sure each node can resolve every other node's hostname or add hostname of each node to /etc/hosts on every node. For example add following lines in /etc/hosts of each node:
...
Where 10.10.10.1, 10.10.10.2 and 10.10.10.3 are the IPs of your nodes and node1, node2 and node3 are hostnames.
On main node download and setup Ambari repoand setup Ambari repo (you may replace the "2.4.1.0" with a newer Ambari version number):
| Code Block |
|---|
wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo |
...
| Code Block |
|---|
yum install ambari-server -yambariy ambari-server setup -s |
Add Metron service to Ambari by running mpack command (make sure to specify correct path to mpack in --mpack=):
...
| Code Block |
|---|
ambari-server start |
And go to Ambari UI, Access the Ambari UI by going to the following URL in a web browser (use admin / admin as user / pass):
| Code Block |
|---|
http://<replace_with_master_node_ip>:8080/#/installer/step0 |
...
| Code Block |
|---|
cd /usr/lib/python2.6/site-packages/resource_monitoring/
python psutil/build.py |
And re-try.
...
| Code Block |
|---|
yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y |
echo -e "elasticsearch - memlock unlimited\nstorm - nproc 257597" >> /etc/security/limits.conf