淘先锋技术网

首页 1 2 3 4 5 6 7

Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild

解决方案:

  • 1, 在命令行执行,复制信息。冒号用逗号代替
hadoop classpath
  • 2 , 编辑yarn-site.xml,添加信息:
    </configuration>
    <property>
    	<name>namemapreduce.reduce.env/name</name>										
    	<value>valueHADOOP_MAPRED_HOME=/opt/module/hadoop-		3.1.3/value</value>
    </property>
    
      <property>
        <name>yarn.application.classpath</name>
     
        <value>复制的Hadoop classpath信息,如
        		/opt/hadoop-3.1.0/etc/hadoop,
                /opt/hadoop-3.1.0/share/hadoop/common/*,
                /opt/hadoop-3.1.0/share/hadoop/common/lib/*,
                /opt/hadoop-3.1.0/share/hadoop/hdfs/*,
                /opt/hadoop-3.1.0/share/hadoop/hdfs/lib/*,
                /opt/hadoop-3.1.0/share/hadoop/mapreduce/*,
                /opt/hadoop-3.1.0/share/hadoop/mapreduce/lib/*,
                /opt/hadoop-3.1.0/share/hadoop/yarn/*,
                /opt/hadoop-3.1.0/share/hadoop/yarn/lib/*
         </value>
      </property>
    </configuration>
    
    设置完毕后用xsync命令分发到其他服务器上并使用start-all.sh命令重启Hadoop集群,
    成功运行。

ERROR:Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

解决方案:

  • 1,编辑mapred-site.xml,添加以下信息
    <property>
      <name>yarn.app.mapreduce.am.env</name>
      <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
    </property>
    <property>
      <name>mapreduce.map.env</name>
      <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
    </property>
    <property>
      <name>mapreduce.reduce.env</name>
      <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
    </property>
    
  • 2, 设置完毕后用xsync命令分发到其他服务器上并使用start-all.sh命令重启Hadoop集群,
    成功运行。

其他

如果仍旧报错,请参考可成功运行的配置:
mapred-site.xml配置如下

<configuration>

<!-- 指定 MapReduce 程序运行在 Yarn 上 -->
<property>
	<name>mapreduce.framework.name</name>
	<value>yarn</value>
</property>

<property>
	<name>namemapreduce.reduce.env/name</name>
	<value>valueHADOOP_MAPRED_HOME=/opt/module/hadoop-3.1.3/value</value>

</property>
<property>
 	 <name>yarn.app.mapreduce.am.env</name>
  	<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
  	<name>mapreduce.map.env</name>
  	<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
  	<name>mapreduce.reduce.env</name>
  	<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>

</configuration>

yarn-site.xml 配置如下

<configuration>


<!-- 指定 MR 走 shuffle -->
<property>
	<name>yarn.nodemanager.aux-services</name>
	<value>mapreduce_shuffle</value>
</property>
<!-- 指定 ResourceManager 的地址-->
<property>
	<name>yarn.resourcemanager.hostname</name>
	<value>hadoop103</value>
</property>
<!-- 环境变量的继承 -->
<property>
	<name>yarn.nodemanager.env-whitelist</name>
	<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CO
NF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAP
RED_HOME</value>
</property>
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>


<property>
        <name>yarn.application.classpath</name>
        <value>
        /opt/module/hadoop-3.1.3/etc/hadoop,/opt/module/hadoop-3.1.3/share/hadoop/common/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/common/*,/opt/module/hadoop-3.1.3/share/hadoop/hdfs,/opt/module/hadoop-3.1.3/share/hadoop/hdfs/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/hdfs/*,/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/*,/opt/module/hadoop-3.1.3/share/hadoop/yarn,/opt/module/hadoop-3.1.3/share/hadoop/yarn/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/yarn/*
   </value>
    </property>
</configuration>

运行成功截图:在这里插入图片描述
在这里插入图片描述