在基础网络架构方案上，k8s是怎么构建网络的呢，主要是service和ingress两个抽象，本文先探讨service。

service实例

首先创建一个后端service程序，就是一个简单的go语言的http服务器，DockerFile创建本地镜像，Dockerfile如下

FROM golang:latest
WORKDIR /home/goapp
ADD ./dockertest .
ADD start.sh .
RUN go build .
EXPOSE 8080
ENTRYPOINT ["./start.sh"]

mkdir /tmp
touch /tmp/healthy
./server

Dockerfile启动点是start.sh，主要是要执行多条语句，其中后面两句就是创建一个/tmp/healthy文件，这个文件是为健康探活使用的，k8s 探针会定期检查是否有这个文件，以确定Pod是否存活。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-server
spec:
  selector:
    matchLabels:
      app: my-server
  replicas: 1
  template:
    metadata:
      labels:
        app: my-server
    spec:
      containers:
      - name: my-server
        image: server
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: "0.5"
            memory: 200Mi
          limits:
            cpu: "0.5"
            memory: 200Mi
        livenessProbe:
           exec:
             command:
             - cat
             - /tmp/healthy
           initialDelaySeconds: 5
           periodSeconds: 30
      terminationGracePeriodSeconds: 60

这个文件用于创建后端服务Pod，其中有几个注意点：

• imagePullPolicy: IfNotPresent，因为想使用本地编译好的镜像，所以设置这个参数为IfNotPresent，意思是如果存在本地镜像就是用本地镜像，否则就从远程下载
• resources的request和limits设置程一样的，这样Pod的Qos就为Guaranteed
• 设置了livenessProbe，用于Pod的监控检查，检查方式为每30秒检查pod中是否有/tmp/healthy目录

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  type: NodePort
  selector:
    app: my-server
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
      nodePort: 30000

这个是个service定义，这个service代理的后端就是上面的Pod，可以通过命令

kubectl describe service

Name:              kubernetes
Namespace:         default
Labels:            component=apiserver
                   provider=kubernetes
Annotations:       <none>
Selector:          <none>
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.96.0.1
IPs:               10.96.0.1
Port:              https  443/TCP
TargetPort:        8443/TCP
Endpoints:         192.168.49.2:8443
Session Affinity:  None
Events:            <none>


Name:                     my-service
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 app=my-server
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.106.27.125
IPs:                      10.106.27.125
Port:                     <unset>  80/TCP
TargetPort:               8080/TCP
NodePort:                 <unset>  30000/TCP
Endpoints:                172.17.0.4:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

可以看到EndPoints，这个EndPoint就是后端Pod，service通过轮询的方式来发送命令到EndPoints。

踩过的小坑

通过minikube使用本地镜像，一开始使用docker编译了镜像，但是镜像怎么都使用不了本地镜像，后来找到一篇文章，才搞明白，k8s使用的docker环境和本机的docker环境并非同一个，需要通过如下命令进行切换，然后在k8s的docker环境下编译镜像，这样才能让k8s看到本地镜像

sudo minikube start --driver=docker
eval $(minikube -p minikube docker-env)

service原理

service的访问如下图

service

• cluster ip，这种类型就是一个集群ip，供k8s集群内服务访问

  
  
  cluster ip

• NodePort，这种类型在cluster ip的情况下，在服务所在的主机上开启了一个端口，通过这个端口可以从外部直接访问service，然后service再访问到主机

  
  
  NodePort
  
  

  但是这样一来，这个类型每多一个service，就要在相应的Node上开一个端口，资源消耗很大

• LoadBalance

  
  
  LoadBlance
  
  

  在NodePort的基础上增加了一个负载均衡层，这个负载均衡层由各个云厂家提供。

• ExternalName
  再外部访问的基础上，再封装一次，提供域名访问。

可以看到，除了Cluster ip仅供内部服务访问，其它三种情况都可以提供外部访问。

代码实现上的思考

service不同于其它Department等对象，这个对象并不是从Pod继承

type Service struct {
    metav1.TypeMeta `json:",inline"`
    // Standard object's metadata.
    // More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
    // +optional
    metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`

    // Spec defines the behavior of a service.
    // https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
    // +optional
    Spec ServiceSpec `json:"spec,omitempty" protobuf:"bytes,2,opt,name=spec"`

    // Most recently observed status of the service.
    // Populated by the system.
    // Read-only.
    // More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
    // +optional
    Status ServiceStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}

// ServiceSpec describes the attributes that a user creates on a service.
type ServiceSpec struct {
    // The list of ports that are exposed by this service.
    // More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
    // +patchMergeKey=port
    // +patchStrategy=merge
    // +listType=map
    // +listMapKey=port
    // +listMapKey=protocol
    Ports []ServicePort `json:"ports,omitempty" patchStrategy:"merge" patchMergeKey:"port" protobuf:"bytes,1,rep,name=ports"`

    // Route service traffic to pods with label keys and values matching this
    // selector. If empty or not present, the service is assumed to have an
    // external process managing its endpoints, which Kubernetes will not
    // modify. Only applies to types ClusterIP, NodePort, and LoadBalancer.
    // Ignored if type is ExternalName.
    // More info: https://kubernetes.io/docs/concepts/services-networking/service/
    // +optional
    // +mapType=atomic
    Selector map[string]string `json:"selector,omitempty" protobuf:"bytes,2,rep,name=selector"`

    // clusterIP is the IP address of the service and is usually assigned
    // randomly. If an address is specified manually, is in-range (as per
    // system configuration), and is not in use, it will be allocated to the
    // service; otherwise creation of the service will fail. This field may not
    // be changed through updates unless the type field is also being changed
    // to ExternalName (which requires this field to be blank) or the type
    // field is being changed from ExternalName (in which case this field may
    // optionally be specified, as describe above).  Valid values are "None",
    // empty string (""), or a valid IP address. Setting this to "None" makes a
    // "headless service" (no virtual IP), which is useful when direct endpoint
    // connections are preferred and proxying is not required.  Only applies to
    // types ClusterIP, NodePort, and LoadBalancer. If this field is specified
    // when creating a Service of type ExternalName, creation will fail. This
    // field will be wiped when updating a Service to type ExternalName.
    // More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
    // +optional
    ClusterIP string `json:"clusterIP,omitempty" protobuf:"bytes,3,opt,name=clusterIP"`

    // ClusterIPs is a list of IP addresses assigned to this service, and are
    // usually assigned randomly.  If an address is specified manually, is
    // in-range (as per system configuration), and is not in use, it will be
    // allocated to the service; otherwise creation of the service will fail.
    // This field may not be changed through updates unless the type field is
    // also being changed to ExternalName (which requires this field to be
    // empty) or the type field is being changed from ExternalName (in which
    // case this field may optionally be specified, as describe above).  Valid
    // values are "None", empty string (""), or a valid IP address.  Setting
    // this to "None" makes a "headless service" (no virtual IP), which is
    // useful when direct endpoint connections are preferred and proxying is
    // not required.  Only applies to types ClusterIP, NodePort, and
    // LoadBalancer. If this field is specified when creating a Service of type
    // ExternalName, creation will fail. This field will be wiped when updating
    // a Service to type ExternalName.  If this field is not specified, it will
    // be initialized from the clusterIP field.  If this field is specified,
    // clients must ensure that clusterIPs[0] and clusterIP have the same
    // value.
    //
    // Unless the "IPv6DualStack" feature gate is enabled, this field is
    // limited to one value, which must be the same as the clusterIP field.  If
    // the feature gate is enabled, this field may hold a maximum of two
    // entries (dual-stack IPs, in either order).  These IPs must correspond to
    // the values of the ipFamilies field. Both clusterIPs and ipFamilies are
    // governed by the ipFamilyPolicy field.
    // More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
    // +listType=atomic
    // +optional
    ClusterIPs []string `json:"clusterIPs,omitempty" protobuf:"bytes,18,opt,name=clusterIPs"`

    // type determines how the Service is exposed. Defaults to ClusterIP. Valid
    // options are ExternalName, ClusterIP, NodePort, and LoadBalancer.
    // "ClusterIP" allocates a cluster-internal IP address for load-balancing
    // to endpoints. Endpoints are determined by the selector or if that is not
    // specified, by manual construction of an Endpoints object or
    // EndpointSlice objects. If clusterIP is "None", no virtual IP is
    // allocated and the endpoints are published as a set of endpoints rather
    // than a virtual IP.
    // "NodePort" builds on ClusterIP and allocates a port on every node which
    // routes to the same endpoints as the clusterIP.
    // "LoadBalancer" builds on NodePort and creates an external load-balancer
    // (if supported in the current cloud) which routes to the same endpoints
    // as the clusterIP.
    // "ExternalName" aliases this service to the specified externalName.
    // Several other fields do not apply to ExternalName services.
    // More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
    // +optional
    Type ServiceType `json:"type,omitempty" protobuf:"bytes,4,opt,name=type,casttype=ServiceType"`

    // externalIPs is a list of IP addresses for which nodes in the cluster
    // will also accept traffic for this service.  These IPs are not managed by
    // Kubernetes.  The user is responsible for ensuring that traffic arrives
    // at a node with this IP.  A common example is external load-balancers
    // that are not part of the Kubernetes system.
    // +optional
    ExternalIPs []string `json:"externalIPs,omitempty" protobuf:"bytes,5,rep,name=externalIPs"`

    // Supports "ClientIP" and "None". Used to maintain session affinity.
    // Enable client IP based session affinity.
    // Must be ClientIP or None.
    // Defaults to None.
    // More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
    // +optional
    SessionAffinity ServiceAffinity `json:"sessionAffinity,omitempty" protobuf:"bytes,7,opt,name=sessionAffinity,casttype=ServiceAffinity"`

    // Only applies to Service Type: LoadBalancer
    // LoadBalancer will get created with the IP specified in this field.
    // This feature depends on whether the underlying cloud-provider supports specifying
    // the loadBalancerIP when a load balancer is created.
    // This field will be ignored if the cloud-provider does not support the feature.
    // +optional
    LoadBalancerIP string `json:"loadBalancerIP,omitempty" protobuf:"bytes,8,opt,name=loadBalancerIP"`

    // If specified and supported by the platform, this will restrict traffic through the cloud-provider
    // load-balancer will be restricted to the specified client IPs. This field will be ignored if the
    // cloud-provider does not support the feature."
    // More info: https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/
    // +optional
    LoadBalancerSourceRanges []string `json:"loadBalancerSourceRanges,omitempty" protobuf:"bytes,9,opt,name=loadBalancerSourceRanges"`

    // externalName is the external reference that discovery mechanisms will
    // return as an alias for this service (e.g. a DNS CNAME record). No
    // proxying will be involved.  Must be a lowercase RFC-1123 hostname
    // (https://tools.ietf.org/html/rfc1123) and requires `type` to be "ExternalName".
    // +optional
    ExternalName string `json:"externalName,omitempty" protobuf:"bytes,10,opt,name=externalName"`

    // externalTrafficPolicy denotes if this Service desires to route external
    // traffic to node-local or cluster-wide endpoints. "Local" preserves the
    // client source IP and avoids a second hop for LoadBalancer and Nodeport
    // type services, but risks potentially imbalanced traffic spreading.
    // "Cluster" obscures the client source IP and may cause a second hop to
    // another node, but should have good overall load-spreading.
    // +optional
    ExternalTrafficPolicy ServiceExternalTrafficPolicyType `json:"externalTrafficPolicy,omitempty" protobuf:"bytes,11,opt,name=externalTrafficPolicy"`

    // healthCheckNodePort specifies the healthcheck nodePort for the service.
    // This only applies when type is set to LoadBalancer and
    // externalTrafficPolicy is set to Local. If a value is specified, is
    // in-range, and is not in use, it will be used.  If not specified, a value
    // will be automatically allocated.  External systems (e.g. load-balancers)
    // can use this port to determine if a given node holds endpoints for this
    // service or not.  If this field is specified when creating a Service
    // which does not need it, creation will fail. This field will be wiped
    // when updating a Service to no longer need it (e.g. changing type).
    // +optional
    HealthCheckNodePort int32 `json:"healthCheckNodePort,omitempty" protobuf:"bytes,12,opt,name=healthCheckNodePort"`

    // publishNotReadyAddresses indicates that any agent which deals with endpoints for this
    // Service should disregard any indications of ready/not-ready.
    // The primary use case for setting this field is for a StatefulSet's Headless Service to
    // propagate SRV DNS records for its Pods for the purpose of peer discovery.
    // The Kubernetes controllers that generate Endpoints and EndpointSlice resources for
    // Services interpret this to mean that all endpoints are considered "ready" even if the
    // Pods themselves are not. Agents which consume only Kubernetes generated endpoints
    // through the Endpoints or EndpointSlice resources can safely assume this behavior.
    // +optional
    PublishNotReadyAddresses bool `json:"publishNotReadyAddresses,omitempty" protobuf:"varint,13,opt,name=publishNotReadyAddresses"`

    // sessionAffinityConfig contains the configurations of session affinity.
    // +optional
    SessionAffinityConfig *SessionAffinityConfig `json:"sessionAffinityConfig,omitempty" protobuf:"bytes,14,opt,name=sessionAffinityConfig"`

    // TopologyKeys is tombstoned to show why 16 is reserved protobuf tag.
    //TopologyKeys []string `json:"topologyKeys,omitempty" protobuf:"bytes,16,opt,name=topologyKeys"`

    // IPFamily is tombstoned to show why 15 is a reserved protobuf tag.
    // IPFamily *IPFamily `json:"ipFamily,omitempty" protobuf:"bytes,15,opt,name=ipFamily,Configcasttype=IPFamily"`

    // IPFamilies is a list of IP families (e.g. IPv4, IPv6) assigned to this
    // service, and is gated by the "IPv6DualStack" feature gate.  This field
    // is usually assigned automatically based on cluster configuration and the
    // ipFamilyPolicy field. If this field is specified manually, the requested
    // family is available in the cluster, and ipFamilyPolicy allows it, it
    // will be used; otherwise creation of the service will fail.  This field
    // is conditionally mutable: it allows for adding or removing a secondary
    // IP family, but it does not allow changing the primary IP family of the
    // Service.  Valid values are "IPv4" and "IPv6".  This field only applies
    // to Services of types ClusterIP, NodePort, and LoadBalancer, and does
    // apply to "headless" services.  This field will be wiped when updating a
    // Service to type ExternalName.
    //
    // This field may hold a maximum of two entries (dual-stack families, in
    // either order).  These families must correspond to the values of the
    // clusterIPs field, if specified. Both clusterIPs and ipFamilies are
    // governed by the ipFamilyPolicy field.
    // +listType=atomic
    // +optional
    IPFamilies []IPFamily `json:"ipFamilies,omitempty" protobuf:"bytes,19,opt,name=ipFamilies,casttype=IPFamily"`

    // IPFamilyPolicy represents the dual-stack-ness requested or required by
    // this Service, and is gated by the "IPv6DualStack" feature gate.  If
    // there is no value provided, then this field will be set to SingleStack.
    // Services can be "SingleStack" (a single IP family), "PreferDualStack"
    // (two IP families on dual-stack configured clusters or a single IP family
    // on single-stack clusters), or "RequireDualStack" (two IP families on
    // dual-stack configured clusters, otherwise fail). The ipFamilies and
    // clusterIPs fields depend on the value of this field.  This field will be
    // wiped when updating a service to type ExternalName.
    // +optional
    IPFamilyPolicy *IPFamilyPolicyType `json:"ipFamilyPolicy,omitempty" protobuf:"bytes,17,opt,name=ipFamilyPolicy,casttype=IPFamilyPolicyType"`

    // allocateLoadBalancerNodePorts defines if NodePorts will be automatically
    // allocated for services with type LoadBalancer.  Default is "true". It
    // may be set to "false" if the cluster load-balancer does not rely on
    // NodePorts.  If the caller requests specific NodePorts (by specifying a
    // value), those requests will be respected, regardless of this field.
    // This field may only be set for services with type LoadBalancer and will
    // be cleared if the type is changed to any other type.
    // This field is beta-level and is only honored by servers that enable the ServiceLBNodePortControl feature.
    // +featureGate=ServiceLBNodePortControl
    // +optional
    AllocateLoadBalancerNodePorts *bool `json:"allocateLoadBalancerNodePorts,omitempty" protobuf:"bytes,20,opt,name=allocateLoadBalancerNodePorts"`

    // loadBalancerClass is the class of the load balancer implementation this Service belongs to.
    // If specified, the value of this field must be a label-style identifier, with an optional prefix,
    // e.g. "internal-vip" or "example.com/internal-vip". Unprefixed names are reserved for end-users.
    // This field can only be set when the Service type is 'LoadBalancer'. If not set, the default load
    // balancer implementation is used, today this is typically done through the cloud provider integration,
    // but should apply for any default implementation. If set, it is assumed that a load balancer
    // implementation is watching for Services with a matching class. Any default load balancer
    // implementation (e.g. cloud providers) should ignore Services that set this field.
    // This field can only be set when creating or updating a Service to type 'LoadBalancer'.
    // Once set, it can not be changed. This field will be wiped when a service is updated to a non 'LoadBalancer' type.
    // +featureGate=LoadBalancerClass
    // +optional
    LoadBalancerClass *string `json:"loadBalancerClass,omitempty" protobuf:"bytes,21,opt,name=loadBalancerClass"`

    // InternalTrafficPolicy specifies if the cluster internal traffic
    // should be routed to all endpoints or node-local endpoints only.
    // "Cluster" routes internal traffic to a Service to all endpoints.
    // "Local" routes traffic to node-local endpoints only, traffic is
    // dropped if no node-local endpoints are ready.
    // The default value is "Cluster".
    // +featureGate=ServiceInternalTrafficPolicy
    // +optional
    InternalTrafficPolicy *ServiceInternalTrafficPolicyType `json:"internalTrafficPolicy,omitempty" protobuf:"bytes,22,opt,name=internalTrafficPolicy"`
}

它本身并不创建Pod对象，而是创建网络资源，存储上和其它对象一样，存储再etcd，通过controller watch变化。所以对于cluster ip对象，因为这个ip要在etcd上获取对那个的映射，因此只能供k8s集群内部访问，在集群外无法直接访问cluster ip类型的service。

而service对象背后的原理，还是k8s基础网络方案，如Flannel、Callico，这些基础镜像背后的原理，还是操作系统路由、iptable等基本原理。

小结

k8s service 是个有别于Depoloyment、Job等的对象，它不从Pod继承，只创建网络，并且提供了四种类型，其中cluster ip只提供集群内部访问，NodePort、LoadBalance、ExternalName提供集群外部访问。