K8s实现nginx、tomcat动静分离

K8s环境部署nginx、tomcat动静分离，部署elk+filebeat日志分析系统。通过nginx代理连接内外网。通过添加节点标签将nginx及tomcat相关pod定向调度至node1节点，elk相关pod定向调度至node2节点，filebeat的kind类型设置为DaemonSet。

节点：

master：172.28.9.90

Node1：172.28.9.92

Node2：172.28.9.91

Nginx负载：172.28.9.93

Nginx、Tomcat网页文件目录：


Node节点标签：


镜像下载：

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.11.1

docker pull docker.elastic.co/logstash/logstash:7.11.1

docker pull docker.elastic.co/kibana/kibana:7.11.1

docker pull docker.elastic.co/beats/filebeat:7.11.1

docker pull docker.io/nginx:latest

docker pull docker.io/tomcat:latest

配置文件：

nginx.yaml、tomcat.yaml、filebeat.yaml、logstash.yaml、elasticsearch.yaml、kibana.yaml

service及pod运行状态：



Nginx及tomcat网页：



Elk网页及nginx日志：


Yaml文件内容：

nginx.yaml

---

apiVersion: v1

kind: PersistentVolume

metadata:

  name: nginx-log-pv

  labels:

    name: nginx-log-pv

spec:

  capacity:

    storage: 5Mi

  accessModes:

  - ReadWriteOnce

  persistentVolumeReclaimPolicy: Recycle

  hostPath:

    path: /var/log/nginx/

---

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

  name: nginx-log-pvc

spec:

  accessModes:

  - ReadWriteOnce

  resources:

    requests:

      storage: 5Mi

  selector:

    matchLabels:

      name: nginx-log-pv

---

apiVersion: v1

kind: ConfigMap

metadata:

  name: cm-nginx-config

data:

  nginx.conf: |

    user  nginx;

    worker_processes  1;

    error_log  /var/log/nginx/error.log warn;

    pid        /var/run/nginx.pid;

    events {

        worker_connections  1024;

    }

    http {

        include       /etc/nginx/mime.types;

        default_type  application/octet-stream;

        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '

                          '$status $body_bytes_sent "$http_referer" '

                          '"$http_user_agent" "$http_x_forwarded_for"';

        access_log  /var/log/nginx/access.log  main;

        sendfile        on;

        keepalive_timeout  65;

        upstream tomcat_server1 {

           server tomcat-sts-0.tomcatserver.default:8080;

           server tomcat-sts-1.tomcatserver.default:8080;

        }

        upstream tomcat_server2 {

           server tomcat-sts-0.tomcatserver.default:8081;

           server tomcat-sts-1.tomcatserver.default:8081;

        }

          server {

            listen       80;

            server_name  test1;

            location / {

                root   /usr/share/nginx/html1;

                index  test1.html;


            }

            location /docs {

                alias /usr/share/nginx/docs1/;

                index test1-docs.html;

            }

            location ~ .*.jsp$ {

                proxy_pass http://tomcat_server1;

                proxy_set_header Host $host:$server_port;

                proxy_set_header REMOTE-HOST $remote_addr;

                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

                proxy_set_header X-Real-IP $remote_addr;

            }

          }

          server {

            listen 81;

            server_name test2;

            location / {

              root   /usr/share/nginx/html2;

              index  test2.html;

            }

            location /docs {

              alias   /usr/share/nginx/docs2/;

              index   test2-docs.html;

            }

            location ~ .*.jsp$ {

               proxy_pass http://tomcat_server2;

                proxy_set_header Host $host:$server_port;

                proxy_set_header REMOTE-HOST $remote_addr;

                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

                proxy_set_header X-Real-IP $remote_addr;

            }

          }

    }

---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: nginx-deployment

spec:

  selector:

    matchLabels:

      app: nginx

  replicas: 2

  template:

    metadata:

      labels:

        app: nginx

    spec:

      containers:

      - name: nginx

        image: docker.io/nginx:latest

        imagePullPolicy: Never

        command: [ "nginx" ]

        args:

        - "-g daemon off;"

        - -c

        - /etc/nginx/nginx.conf

        ports:

        - containerPort: 80

          name: test1

        - containerPort: 81

          name: test2

        volumeMounts:

        - name: nginx-config

          mountPath: /etc/nginx/nginx.conf

          subPath: nginx.conf

        - name: nginx-html-docs

          mountPath: /usr/share/nginx/

        - name: nginx-log-pvc

          mountPath: /var/log/nginx/

      nodeSelector:

        type: web

      volumes:

      - name: nginx-config

        configMap:

          name: cm-nginx-config

      - name: nginx-log-pvc

        persistentVolumeClaim:

          claimName: nginx-log-pvc

      - name: nginx-html-docs

        hostPath:

          path: /usr/share/nginx/

          type: Directory

---

apiVersion: v1

kind: Service

metadata:

  name: nginx

spec:

  type: NodePort

  ports:

  - name: nginx1

    port: 80

    nodePort: 30001

  - name: nginx2

    port: 81

    nodePort: 30002

  selector:

    app: nginx



tomcat.yaml

---

apiVersion: v1

kind: PersistentVolume

metadata:

  name: tomcat-log-pv

  labels:

    name: tomcat-log-pv

spec:

  capacity:

    storage: 5Mi

  accessModes:

  - ReadWriteOnce

  persistentVolumeReclaimPolicy: Recycle

  hostPath:

    path: /var/log/tomcat/

---

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

  name: tomcat-log-pvc

spec:

  accessModes:

  - ReadWriteOnce

  resources:

    requests:

      storage: 5Mi

  selector:

    matchLabels:

      name: tomcat-log-pv

---

apiVersion: v1

kind: ConfigMap

metadata:

  name: cm-tomcat-server-config

data:

  server.xml: |

    <?xml version='1.0' encoding='utf-8'?>

    <Server port="8005" shutdown="SHUTDOWN">

      <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />

      <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />

      <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />

      <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

      <GlobalNamingResources>

        <Resource name="UserDatabase" auth="Container"

                  type="org.apache.catalina.UserDatabase"

                  description="User database that can be updated and saved"

                  factory="org.apache.catalina.users.MemoryUserDatabaseFactory"

                  pathname="conf/tomcat-users.xml" />

      </GlobalNamingResources>



      <Service name="test1">

        <Connector port="8080" protocol="HTTP/1.1"

                   connectionTimeout="20000"

                   redirectPort="8443" />

        <Engine name="test1" defaultHost="test1">

          <Realm className="org.apache.catalina.realm.LockOutRealm">

            <Realm className="org.apache.catalina.realm.UserDatabaseRealm"

                   resourceName="UserDatabase"/>

          </Realm>


          <Host name="test1"  appBase="/usr/share/tomcat/test1"

                unpackWARs="true" autoDeploy="true">

            <Valve className="org.apache.catalina.valves.AccessLogValve" directory="/var/log/tomcat/"

                   prefix="test1_access_log" suffix=".txt"

                   pattern="%{X-Forwarded-For}i %h %l %u %t %r %s %b %{Referer}i %{User-Agent}i" />


          </Host>

        </Engine>

      </Service>

      <Service name="test2">

        <Connector port="8081" protocol="HTTP/1.1"

                   connectionTimeout="20000"

                   redirectPort="8443" />

        <Engine name="test2" defaultHost="test2">

          <Realm className="org.apache.catalina.realm.LockOutRealm">

            <Realm className="org.apache.catalina.realm.UserDatabaseRealm"

                   resourceName="UserDatabase"/>

          </Realm>


          <Host name="test2"  appBase="/usr/share/tomcat/test2"

                unpackWARs="true" autoDeploy="true">

            <Valve className="org.apache.catalina.valves.AccessLogValve" directory="/var/log/tomcat/"

                   prefix="test2_access_log" suffix=".txt"

                   pattern="%{X-Forwarded-For}i %h %l %u %t %r %s %b %{Referer}i %{User-Agent}i" />


          </Host>

        </Engine>

      </Service>

    </Server>


---

apiVersion: apps/v1

kind: StatefulSet

metadata:

  name: tomcat-sts

spec:

  selector:

    matchLabels:

      app: tomcat

  replicas: 2

  serviceName: tomcatserver

  template:

    metadata:

      labels:

        app: tomcat

    spec:

      containers:

      - name: tomcat

        image: docker.io/tomcat:latest

        imagePullPolicy: Never

        command: [ "/bin/sh", "-c" ]

        args:

        - "bin/catalina.sh run;"

        ports:

        - containerPort: 8080

          name: test1

        - containerPort: 8081

          name: test2

        volumeMounts:

        - name: tomcat-server-config

          mountPath: /usr/local/tomcat/conf/server.xml

          subPath: server.xml

        - name: tomcat-webapp

          mountPath: /usr/share/tomcat/

        - name: tomcat-log-pvc

          mountPath: /var/log/tomcat/

      nodeSelector:

        type: web

      volumes:

      - name: tomcat-server-config

        configMap:

          name: cm-tomcat-server-config

      - name: tomcat-log-pvc

        persistentVolumeClaim:

          claimName: tomcat-log-pvc

      - name: tomcat-webapp

        hostPath:

          path: /usr/share/tomcat/

          type: Directory

---

apiVersion: v1

kind: Service

metadata:

  name: tomcat

  labels:

    app: tomcat

spec:

  clusterIP: None

  ports:

  - port: 8080

    name: test1

  - port: 8081

    name: test2

  selector:

    app: tomcat



filebeat.yaml

---

apiVersion: v1

kind: ConfigMap

metadata:

  name: cm-filebeat-config

data:

  filebeat.yml: |

    filebeat.inputs:

    - type: log

      enabled: true

      paths:

      - /var/log/nginx/*.log

    filebeat.config.modules:

      path: ${path.config}/modules.d/*.yml

      reload.enabled: false

    output.logstash:

      hosts: [ "logstash:5044" ]

---

apiVersion: apps/v1

kind: DaemonSet

metadata:

  name: filebeat

spec:

  selector:

    matchLabels:

      app: filebeat

  template:

    metadata:

      labels:

        app: filebeat

    spec:

      containers:

      - name: filebeat

        image: docker.elastic.co/beats/filebeat:7.11.1

        imagePullPolicy: Never

        command: [ "./filebeat" ]

        args:

        - -e

        - -c

        - filebeat.yml

        volumeMounts:

        - name: filebeat-config

          mountPath: /usr/share/filebeat/filebeat.yml

          subPath: filebeat.yml

        - name: nginx-log-pvc

          mountPath: /var/log/nginx/

      volumes:

      - name: filebeat-config

        configMap:

          name: cm-filebeat-config

      - name: nginx-log-pvc

        persistentVolumeClaim:

          claimName: nginx-log-pvc



logstash.yaml

---

apiVersion: v1

kind: ConfigMap

metadata:

  name: cm-logstash-yml-config

data:

  logstash.yml: |

    http.host: "0.0.0.0"

    xpack.monitoring.elasticsearch.hosts: ["http://elasticsearch:9200"]


---

apiVersion: v1

kind: ConfigMap

metadata:

  name: cm-logstash-config

data:

  logstash.conf: |

    input {

      beats {

        port => 5044

      }

    }


    output {

      elasticsearch {

        hosts => "elasticsearch:9200"

        index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"

      }

    }


---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: logstash

spec:

  replicas: 1

  selector:

    matchLabels:

      app: logstash

  template:

    metadata:

      labels:

        app: logstash

    spec:

      containers:

      - image: docker.elastic.co/logstash/logstash:7.11.1

        name: logstash

        imagePullPolicy: Never

        command: [ "bin/logstash" ]

        args:

        - -f

        - config/logstash.conf

        ports:

        - name: log-es

          containerPort: 5044

        volumeMounts:

        - name: logstash-yml-config

          mountPath: /usr/share/logstash/config/logstash.yml

          subPath: logstash.yml

        - name: logstash-config

          mountPath: /usr/share/logstash/config/logstash.conf

          subPath: logstash.conf

      nodeSelector:

        type: elk

      volumes:

      - name: logstash-yml-config

        configMap:

          name: cm-logstash-yml-config

      - name: logstash-config

        configMap:

          name: cm-logstash-config

---

apiVersion: v1

kind: Service

metadata:

  name: logstash

spec:

  type: ClusterIP

  ports:

  - name: log-es

    port: 5044

  selector:

    app: logstash



elasticsearch.yaml

---

apiVersion: v1

kind: ConfigMap

metadata:

  name: cm-elasticsearch-config

data:

  elasticsearch.yml: |

    network.host: 0.0.0.0

    http.port: 9200

    cluster.initial_master_nodes: '${POD_IP}'



---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: elasticsearch

spec:

  replicas: 1

  selector:

    matchLabels:

      app: elasticsearch

  template:

    metadata:

      labels:

        app: elasticsearch

    spec:

      containers:

      - image: docker.elastic.co/elasticsearch/elasticsearch:7.11.1

        name: elasticsearch

        imagePullPolicy: Never

        env:

        - name: POD_IP

          valueFrom:

            fieldRef:

              fieldPath: status.podIP

        ports:

        - name: es-kibana

          containerPort: 9200

        volumeMounts:

        - name: elasticsearch-config

          mountPath: /usr/share/elasticsearch/config/elasticsearch.yml

          subPath: elasticsearch.yml

      nodeSelector:

        type: elk

      volumes:

      - name: elasticsearch-config

        configMap:

          name: cm-elasticsearch-config

---

apiVersion: v1

kind: Service

metadata:

  name: elasticsearch

spec:

  type: ClusterIP

  ports:

  - name: log

    port: 9200

  selector:

    app: elasticsearch



kibana.yaml

---

apiVersion: v1

kind: ConfigMap

metadata:

  name: cm-kibana-config

data:

  kibana.yml: |

    server.port: 5601

    server.host: "0.0.0.0"

    elasticsearch.hosts: ["http://elasticsearch:9200"]

---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: kibana

spec:

  replicas: 1

  selector:

    matchLabels:

      app: kibana

  template:

    metadata:

      labels:

        app: kibana

    spec:

      containers:

      - image: docker.elastic.co/kibana/kibana:7.11.1

        name: kibana

        imagePullPolicy: Never

        command: [ "bin/kibana" ]

        args:

        - '--allow-root'

        ports:

        - name: web

          containerPort: 5601

        volumeMounts:

        - name: kibana-config

          mountPath: /usr/share/kibana/config/kibana.yml

          subPath: kibana.yml

      nodeSelector:

        type: elk

      volumes:

      - name: kibana-config

        configMap:

          name: cm-kibana-config

---

apiVersion: v1

kind: Service

metadata:

  name: kibana

spec:

  type: NodePort

  ports:

  - name: web

    port: 5601

    nodePort: 30003

  selector:

    app: kibana

最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 204,793评论 6赞 478
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 87,567评论 2赞 381
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 151,342评论 0赞 338
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 54,825评论 1赞 277
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 63,814评论 5赞 368
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 48,680评论 1赞 281
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 38,033评论 3赞 399
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 36,687评论 0赞 258
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 42,175评论 1赞 300
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,668评论 2赞 321
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,775评论 1赞 332
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 33,419评论 4赞 321
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 39,020评论 3赞 307
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,978评论 0赞 19
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,206评论 1赞 260
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 45,092评论 2赞 351
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 42,510评论 2赞 343

推荐阅读更多精彩内容