Versions Compared

Old Version 4

changes.mady.by.user Lewis McGibbney

Saved on

compared with

New Version 5

changes.mady.by.user Lewis McGibbney

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Configuring and Deploying Hadoop Services

...

Hadoop was configured and deployed in pseudo-distributed mode

hdfs-site.xml Configuration

Code Block
<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

yarn-site.xml Configuration



Configuring and Deploying Tez

...

Running the Injector job on Tez

Image Added

Run #YARN Engine

# of URLs

Elapsed Time
1MapReduce11523
00:00:34
2MapReduce11523
00:00:32
3MapReduce11523
00:00:34
4Tez11523
00:00:42
5Tez11523
00:00:13
6Tez11523
00:00:14
7MapReduce1576346900:03:21
8MapReduce1576346900:03:13
9MapReduce1576346900:02:38
10MapReduce1576346900:02:37
11MapReduce1576346900:02:48
12Tez1576346900:02:14
11
13Tez1576346900:02:10
12
14Tez1576346900:02:13

From the above Tez clearly appears to offer significant runtime improvements over MapReduce. This is very promising however much more experimentation is required.

...