淘先锋技术网

首页 1 2 3 4 5 6 7

emr的做法?

1 数据root密码修改后emr配置修改:

2/ 添加s3访问

S3:hadoop-aws-2.8.5.jarjar -tfv hadoop-aws-2.8.5.jar |grep S3FileSystem/usr/lib/hadoop-current/share/hadoop/tools/lib/usr/lib/hadoop-current/share/hadoop/common/lib放到hadoop和hive的classpath下?

su - hadoopecho $HADOOP_CLASSPAT

<property><name>fs.s3a.access.key</name><value>access.key</value></property><property><name>fs.s3a.secret.key</name><value>secret.key</value></property><property><name>fs.s3a.impl</name><value>org.apache.hadoop.fs.s3a.S3AFileSystem</value></property>#hadoop使用S3sudo cp /usr/lib/hadoop-current/share/hadoop/tools/lib/*aws* /usr/lib/hadoop-current/share/hadoop/common/lib/sudo cp /usr/lib/hadoop-current/share/hadoop/tools/lib/*jackson* /usr/lib/hadoop-current/share/hadoop/common/lib/sudo cp /usr/lib/hadoop-current/share/hadoop/tools/lib/joda-time-2.9.4.jar /usr/lib/hadoop-current/share/hadoop/common/lib/