minio文件服务器搭建（分布式）

搭建

分布式Minio

启动一个分布式Minio实例，你只需要把硬盘位置做为参数传给minio server命令即可，然后，你需要在所有其它节点运行同样的命令。

注意：

分布式Minio里所有的节点需要有同样的access秘钥和secret秘钥，这样这些节点才能建立联接。为了实现这个，你需要在执行minio server命令之前，先将access秘钥和secret秘钥export成环境变量。
分布式Minio使用的磁盘里必须是干净的，里面没有数据。
分布式Minio里的节点时间差不能超过3秒，你可以使用NTP 来保证时间一致。
例子1：
四节点
cd /usr/minio
mkdir data1
mkdir data2
mkdir data3
mkdir data4
vim run.sh
chmod u+x *.sh

export MINIO_ACCESS_KEY="minio"
export MINIO_SECRET_KEY="minio123"
for i in {01..04}; do
    nohup ./minio server --address ":90${i}" http://127.0.0.1:9001/usr/minio/data1 http://127.0.0.1:9002/usr/minio/data2  http://127.0.0.1:9003/usr/minio/data3 http://127.0.0.1:9004/usr/minio/data4 > "/usr/minio/90${i}.log"& 2>&1
done

export MINIO_ACCESS_KEY="minio"
export MINIO_SECRET_KEY="minio123"
nohup ./minio server --address ":9001" http://127.0.0.1:9001/usr/minio/data1 http://127.0.0.1:9002/usr/minio/data2  http://127.0.0.1:9003/usr/minio/data3 http://127.0.0.1:9004/usr/minio/data4 > "/usr/minio/9001.log"& 2>&1
nohup ./minio server --address ":9002" http://127.0.0.1:9001/usr/minio/data1 http://127.0.0.1:9002/usr/minio/data2  http://127.0.0.1:9003/usr/minio/data3 http://127.0.0.1:9004/usr/minio/data4 > "/usr/minio/9002.log"& 2>&1
nohup ./minio server --address ":9003" http://127.0.0.1:9001/usr/minio/data1 http://127.0.0.1:9002/usr/minio/data2  http://127.0.0.1:9003/usr/minio/data3 http://127.0.0.1:9004/usr/minio/data4 > "/usr/minio/9003.log"& 2>&1
nohup ./minio server --address ":9004" http://127.0.0.1:9001/usr/minio/data1 http://127.0.0.1:9002/usr/minio/data2  http://127.0.0.1:9003/usr/minio/data3 http://127.0.0.1:9004/usr/minio/data4 > "/usr/minio/9004.log"& 2>&1

例子2：
双节点、双挂载

#!/bin/bash
export MINIO_ACCESS_KEY="minio"
export MINIO_SECRET_KEY="minio123"
cd /usr/minio/
./minio server --address :9001 http://127.0.0.1:9001/usr/minio/data1 http://127.0.0.1:9001/usr/minio/data2  http://127.0.0.1:9002/usr/minio/data3 http://127.0.0.1:9002/usr/minio/data4 > /usr/minio/minio1.log 2>&1&
./minio server --address :9002 http://127.0.0.1:9001/usr/minio/data1 http://127.0.0.1:9001/usr/minio/data2  http://127.0.0.1:9002/usr/minio/data3 http://127.0.0.1:9002/usr/minio/data4 > /usr/minio/minio2.log 2>&1&

笔记：

集群服务意外停掉后，重启文件自动会同步
双挂载磁盘，每个磁盘都有备份
高可用，单机 Minio 服务存在单点故障，相反，如果是一个 N 节点的分布式 Minio ,只要有 N/2 节点在线，你的数据就是安全的。不过你需要至少有 N/2+1 个节点来创建新的对象。
https://blog.csdn.net/erdongxt/article/details/102997081

参考

可扩容的方式，使用MinIO纠删码，如果想要扩容，必须使用这种方式

需要将新的MinIO节点架设好，假设原来是4台机器，再加4台机器进入节点即可完成扩容。

首次部署

export MINIO_ACCESS_KEY="minio"
export MINIO_SECRET_KEY="minio123"
#minio server http://host{1...32}/export{1...32}
./minio server   http://192.168.1.{69...70}:9000/usr/minio/data{1...4}

扩容方式，在原命令最后一行末尾添加扩容的节点，重启集群

export MINIO_ACCESS_KEY="minio"
export MINIO_SECRET_KEY="minio123"
#minio server http://host{1...32}/export{1...32} http://host{33...64}/export{1...32}
./minio server   http://192.168.1.{69...70}:9000/usr/minio/data{1...4} http://192.168.1.{71...72}:9000/usr/minio/data{1...4}

挂载在根目录下会报错如下：
Disk /usr/minio/data1 is a root disk. Please ensure the disk is mounted properly, refusing to use root disk.

双机Minio服务（生产）

#!/bin/bash
export MINIO_ACCESS_KEY="minio"
export MINIO_SECRET_KEY="handhand123"
cd /usr/local/minio/
./minio server --address :80 http://10.128.241.7:80/usr/local/minio/data1 http://10.128.241.7:80/usr/local/minio/data2 http://10.128.241.18:80/usr/local/minio/data1 http://10.128.241.18:80/usr/local/minio/data2 > /usr/local/minio/minio.log 2>&1&

nginx代理：

 server {
        listen       9000;
        server_name  localhost;
        #charset koi8-r;
        #access_log  logs/host.access.log  main;
client_max_body_size 100M;
    location  / {
        proxy_pass http://updbackend;
        proxy_set_header   Host             $http_host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-Proto  $scheme;
        proxy_set_header   X-Forwarded-Host  $http_host;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        client_max_body_size 100m;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

docker minio分布式

参考：https://docs.min.io/cn/deploy-minio-on-docker-compose.html
1
2

运行下面的命令

#拉镜像
docker-compose pull
#启动
docker-compose up
docker-compose -f docker-compose.yaml up -d

docker-compose.yaml
配置：

version: '3.7'

# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
  minio1:
    image: minio/minio:RELEASE.2020-12-26T01-35-54Z
    volumes:
      - data1-1:/data1
      - data1-2:/data2
    expose:
      - "9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio2:
    image: minio/minio:RELEASE.2020-12-26T01-35-54Z
    volumes:
      - data2-1:/data1
      - data2-2:/data2
    expose:
      - "9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio3:
    image: minio/minio:RELEASE.2020-12-26T01-35-54Z
    volumes:
      - data3-1:/data1
      - data3-2:/data2
    expose:
      - "9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio4:
    image: minio/minio:RELEASE.2020-12-26T01-35-54Z
    volumes:
      - data4-1:/data1
      - data4-2:/data2
    expose:
      - "9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  nginx:
    image: nginx:1.19.2-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
  data1-1:
  data1-2:
  data2-1:
  data2-2:
  data3-1:
  data3-2:
  data4-1:
  data4-2: