You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Hadoop uses some dependences that do not work natively with Apple Silicon (M1) machines. If you would like to develop and build Hadoop on your macOS with M1 chip / ARM64 CPU, follow steps in this page.

protobuf-maven-plugin

You may get following error:

[ERROR] Failed to execute goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.1:compile-custom (default) on project hadoop-yarn-csi: Missing:
[ERROR] ----------
[ERROR] 1) io.grpc:protoc-gen-grpc-java:exe:linux-aarch_64:1.15.1

This is because the grpc package is not releasing binaries for ARM64. One way of solving this is to specify the properties os.detected.classifier to be osx-x86_64 instead of osx-aarch_64 For example, you can put following code in your ~/.m2/settings.xml file:

<settings>
    <activeProfiles>
        <activeProfile>
            apple-silicon
        </activeProfile>
    </activeProfiles>
    <profiles>
        <profile>
            <id>apple-silicon</id>
            <properties>
                <os.detected.classifier>osx-x86_64</os.detected.classifier>
            </properties>
        </profile>
    </profiles>
</settings>

frontend-maven-plugin

This plugin is used to install node.js and yarn locally. YARN-6278 added `yarn-ui` profile to pom.xml leveraging this plugin to automatically install node.js and yarn locally under target/webapp directory. Latest versions of this plugin supports Apple M1 chip but Hadoop is still using an older version that does not. We can use the x64 node.js binaries as a workaround. 

  • No labels