挂载HDFS(HDFS NFS Gateway)

0. 安装依赖包

yum install -y showmount

1. 修改配置文件

修改core-site.xml

<property>
    <name>hadoop.proxyuser.root.groups</name>
    <value>*</value>
    <description>
         The 'nfsserver' user is allowed to proxy all members of the 'users-group1' and
         'users-group2' groups. Note that in most cases you will need to include the
         group "root" because the user "root" (which usually belonges to "root" group) will
         generally be the user that initially executes the mount on the NFS client system.
         Set this to '*' to allow nfsserver user to proxy any group.
    </description>
    </property>
    <property>
    <name>hadoop.proxyuser.root.hosts</name>
    <value>*</value>
    <description>
         This is the host where the nfs gateway is running. Set this to '*' to allow
         requests from any hosts to be proxied.
    </description>
    </property>

修改hdfs-site.xml

<property>
        <name>nfs.dump.dir</name>
        <value>/tmp/.hdfs-nfs</value>
    </property>
    <property>
        <name>nfs.rtmax</name>
        <value>1048576</value>
        <description>This is the maximum size in bytes of a READ request supported by the NFS gateway. If you change this, make sure you also update the nfs mount's rsize(add rsize= # of bytes to the mount directive).</description>
    </property>
    <property>
        <name>nfs.wtmax</name>
        <value>65536</value>
        <description>This is the maximum size in bytes of a WRITE request supported by the NFS gateway. If you change this, make sure you also update the nfs mount's wsize(add wsize= # of bytes to the mount directive).</description>
    </property>
    <property>
        <name>nfs.exports.allowed.hosts</name>
        <value>* rw</value>
        <description>允许所有主机对文件有rw权限</description>
    </property>

2. 重启hadoop集群服务

sbin/stop-all.sh
sbin/start-all.sh
#上述方法提示废弃,可使用
sbin/stop-dfs.sh
sbin/stop-yarn.sh
sbin/start-dfs.sh
sbin/start-yarn.sh

3. 启动hadoop nfs

systemctl stop rpcbind
systemctl stop nfs
#这里需要重启主机,否则portmap会启动失败,原因是111端口还是在被使用
sbin/hadoop-daemon.sh --script bin/hdfs start portmap
sbin/hadoop-daemon.sh --script bin/hdfs start nfs3

4. 挂载

mount -t nfs -o vers=3,proto=tcp,nolock,noacl,sync ceph-1:/  /mnt/hdfs

注意:

  1. slave节点不需要按照上面修改配置文件,只需要按照指定的yum包,然后直接挂载即可

References

  1. http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
【社区内容提示】社区部分内容疑似由AI辅助生成,浏览时请结合常识与多方信息审慎甄别。
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

相关阅读更多精彩内容

友情链接更多精彩内容