kubernetes系列教程(十六)基于nginx ingress实现服务暴露-深蓝源码网


时间: 2020-09-03 00:08:26 人气: 2275 评论: 0

文章目录

写在前面

本章介绍kubernetes系列教程的ingress概念,在kubernetes中对外暴露服务的方式有两种:service(NodePort或者外部LoadBalancer)和ingress,其中service是提供四层的负载均衡,通过iptables DNAT或lvs nat模式实现后端Pod的代理请求。如需实现http,域名,URI,证书等请求方式,service是无法实现的,需要借助于ingress来来实现,本文将来介绍ingress相关的内容。

1. Ingress简介

An API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination and name-based virtual hosting.

引用官方关于ingress的介绍我们可以得知,ingress是一种通过http协议暴露kubernetes内部服务的api对象,即充当Edge Router边界路由器的角色对外基于七层的负载均衡调度机制,能够提供以下几个功能:

  • 负载均衡,将请求自动负载均衡到后端的Pod上;
  • SSL加密,客户端到Ingress Controller为https加密,到后端Pod为明文的http;
  • 基于名称的虚拟主机,提供基于域名或URI更灵活的路由方式
Ingress Controller

实现Ingress包含的组件有:

  • Ingress,客户端,负责定义ingress配置,将请求转发给Ingress Controller;
  • Ingress Controller,Ingress控制器,实现七层转发的Edge Router,通过调用k8s的api动态感知集群中Pod的变化而动态更新配置文件并重载, Controller需要部署在k8s集群中以实现和集群中的pod通信,通常以DaemonSets或Deployments的形式部署,并对外暴露80和443端口,对于DaemonSets来说,一般是以hostNetwork或者hostPort的形式暴露,Deployments则以NodePort的方式暴露,控制器的多个节点则借助外部负载均衡ExternalLB以实现统一接入;
  • Ingress配置规则,Controller控制器通过service服务发现机制动态实现后端Pod路由转发规则的实现;
  • Service,kuberntes中四层的负载均衡调度机制,Ingress借助service的服务发现机制实现集群中Pod资源的动态感知;
  • Pod,后端实际负责响应请求容器,由控制器如Deployment创建,通过标签Labels和service关联,服务发现。

简而言之,ingress控制器借助service的服务发现机制实现配置的动态更新以实现Pod的负载均衡机制实现,由于涉及到Ingress Controller的动态更新,目前社区Ingress Controller大体包含两种类型的控制器:

  • 传统的七层负载均衡如Nginx,HAproxy,开发了适应微服务应用的插件,具有成熟,高性能等优点;
  • 新型微服务负载均衡如Traefik,Envoy,Istio,专门适用于微服务+容器化应用场景,具有动态更新特点;

类型

常见类型

优点

缺点

传统负载均衡

nginx,haproxy

成熟,稳定,高性能

动态更新需reload配置文件

微服务负载均衡

Traefik,Envoy,Istio

天生为微服务而生,动态更新

性能还有待提升

2. Nginx Ingress

2.1 Nginx ingress介绍

By default, pods of Kubernetes services are not accessible from the external network, but only by other pods within the Kubernetes cluster. Kubernetes has a built‑in configuration for HTTP load balancing, called Ingress, that defines rules for external connectivity to Kubernetes services. Users who need to provide external access to their Kubernetes services create an Ingress resource that defines rules, including the URI path, backing service name, and other information. The Ingress controller can then automatically program a frontend load balancer to enable Ingress configuration. The NGINX Ingress Controller for Kubernetes is what enables Kubernetes to configure NGINX and NGINX Plus for load balancing Kubernetes services.

Nginx ingress

Nginx Ingress Controller是实现ingress的具体实现,包含有两个版本:Ngnix OSS和Nginx Plus版,后者是商业化增强版,支持更多的功能,详情参考官方文档介绍https://www.nginx.com/products/nginx/kubernetes-ingress-controller#compare-versions

Nginx ingress版本features对比

2.2 Nginx ingress安装

首先需要安装Nginx Ingress Controller控制器,控制器安装方式包含两种:DaemonSets和Deployments。

  • DaemonSets通过hostPort的方式暴露80和443端口,可通过Node的调度由专门的节点实现部署
  • Deployments则通过NodePort的方式实现控制器端口的暴露,借助外部负载均衡实现高可用负载均衡

除此之外,还需要部署Namespace,ServiceAccount,RBAC,Secrets,Custom Resource Definitions等资源,如下开始部署。

2.2.1 基础依赖环境准备

1、github中下载源码包,安装部署文件在kubernetes-ingress/deployments/目录下

[root@node-1 ~]# git clone https://github.com/nginxinc/kubernetes-ingress.git
[root@node-1 ~]# tree kubernetes-ingress/deployments/
kubernetes-ingress/deployments/
├── common
│   ├── custom-resource-definitions.yaml  自定义资源
│   ├── default-server-secret.yaml        Secrets
│   ├── nginx-config.yaml
│   └── ns-and-sa.yaml                    Namspace+ServiceAccount
├── daemon-set
│   ├── nginx-ingress.yaml                DaemonSets控制器
│   └── nginx-plus-ingress.yaml
├── deployment
│   ├── nginx-ingress.yaml                Deployments控制器
│   └── nginx-plus-ingress.yaml
├── helm-chart                            Helm安装包
│   ├── chart-icon.png
│   ├── Chart.yaml
│   ├── README.md
│   ├── templates
│   │   ├── controller-configmap.yaml
│   │   ├── controller-custom-resources.yaml
│   │   ├── controller-daemonset.yaml
│   │   ├── controller-deployment.yaml
│   │   ├── controller-leader-election-configmap.yaml
│   │   ├── controller-secret.yaml
│   │   ├── controller-serviceaccount.yaml
│   │   ├── controller-service.yaml
│   │   ├── controller-wildcard-secret.yaml
│   │   ├── _helpers.tpl
│   │   ├── NOTES.txt
│   │   └── rbac.yaml
│   ├── values-icp.yaml
│   ├── values-plus.yaml
│   └── values.yaml
├── rbac                                RBAC认证授权
│   └── rbac.yaml
├── README.md
└── service                            Service定义
    ├── loadbalancer-aws-elb.yaml
    ├── loadbalancer.yaml              DaemonSets暴露服务方式
    └── nodeport.yaml                  Deployments暴露服务方式

2、创建Namespace和ServiceAccount, kubectl apply -f common/default-server-secret.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: nginx-ingress 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress 
  namespace: nginx-ingress

3、创建Secrets自签名证书,kubectl apply -f common/default-server-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: default-server-secret
  namespace: nginx-ingress
type: Opaque
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN2akNDQWFZQ0NRREFPRjl0THNhWFhEQU5CZ2txaGtpRzl3MEJBUXNGQURBaE1SOHdIUVlEVlFRRERCWk8KUjBsT1dFbHVaM0psYzNORGIyNTBjbTlzYkdWeU1CNFhEVEU0TURreE1qRTRNRE16TlZvWERUSXpNRGt4TVRFNApNRE16TlZvd0lURWZNQjBHQTFVRUF3d1dUa2RKVGxoSmJtZHlaWE56UTI5dWRISnZiR3hsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUwvN2hIUEtFWGRMdjNyaUM3QlBrMTNpWkt5eTlyQ08KR2xZUXYyK2EzUDF0azIrS3YwVGF5aGRCbDRrcnNUcTZzZm8vWUk1Y2Vhbkw4WGM3U1pyQkVRYm9EN2REbWs1Qgo4eDZLS2xHWU5IWlg0Rm5UZ0VPaStlM2ptTFFxRlBSY1kzVnNPazFFeUZBL0JnWlJVbkNHZUtGeERSN0tQdGhyCmtqSXVuektURXUyaDU4Tlp0S21ScUJHdDEwcTNRYzhZT3ExM2FnbmovUWRjc0ZYYTJnMjB1K1lYZDdoZ3krZksKWk4vVUkxQUQ0YzZyM1lma1ZWUmVHd1lxQVp1WXN2V0RKbW1GNWRwdEMzN011cDBPRUxVTExSakZJOTZXNXIwSAo1TmdPc25NWFJNV1hYVlpiNWRxT3R0SmRtS3FhZ25TZ1JQQVpQN2MwQjFQU2FqYzZjNGZRVXpNQ0F3RUFBVEFOCkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWpLb2tRdGRPcEsrTzhibWVPc3lySmdJSXJycVFVY2ZOUitjb0hZVUoKdGhrYnhITFMzR3VBTWI5dm15VExPY2xxeC9aYzJPblEwMEJCLzlTb0swcitFZ1U2UlVrRWtWcitTTFA3NTdUWgozZWI4dmdPdEduMS9ienM3bzNBaS9kclkrcUI5Q2k1S3lPc3FHTG1US2xFaUtOYkcyR1ZyTWxjS0ZYQU80YTY3Cklnc1hzYktNbTQwV1U3cG9mcGltU1ZmaXFSdkV5YmN3N0NYODF6cFErUyt1eHRYK2VBZ3V0NHh3VlI5d2IyVXYKelhuZk9HbWhWNThDd1dIQnNKa0kxNXhaa2VUWXdSN0diaEFMSkZUUkk3dkhvQXprTWIzbjAxQjQyWjNrN3RXNQpJUDFmTlpIOFUvOWxiUHNoT21FRFZkdjF5ZytVRVJxbStGSis2R0oxeFJGcGZnPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdi91RWM4b1JkMHUvZXVJTHNFK1RYZUprckxMMnNJNGFWaEMvYjVyYy9XMlRiNHEvClJOcktGMEdYaVN1eE9ycXgrajlnamx4NXFjdnhkenRKbXNFUkJ1Z1B0ME9hVGtIekhvb3FVWmcwZGxmZ1dkT0EKUTZMNTdlT1l0Q29VOUZ4amRXdzZUVVRJVUQ4R0JsRlNjSVo0b1hFTkhzbysyR3VTTWk2Zk1wTVM3YUhudzFtMApxWkdvRWEzWFNyZEJ6eGc2clhkcUNlUDlCMXl3VmRyYURiUzc1aGQzdUdETDU4cGszOVFqVUFQaHpxdmRoK1JWClZGNGJCaW9CbTVpeTlZTW1hWVhsMm0wTGZzeTZuUTRRdFFzdEdNVWozcGJtdlFmazJBNnljeGRFeFpkZFZsdmwKMm82MjBsMllxcHFDZEtCRThCay90elFIVTlKcU56cHpoOUJUTXdJREFRQUJBb0lCQVFDZklHbXowOHhRVmorNwpLZnZJUXQwQ0YzR2MxNld6eDhVNml4MHg4Mm15d1kxUUNlL3BzWE9LZlRxT1h1SENyUlp5TnUvZ2IvUUQ4bUFOCmxOMjRZTWl0TWRJODg5TEZoTkp3QU5OODJDeTczckM5bzVvUDlkazAvYzRIbjAzSkVYNzZ5QjgzQm9rR1FvYksKMjhMNk0rdHUzUmFqNjd6Vmc2d2szaEhrU0pXSzBwV1YrSjdrUkRWYmhDYUZhNk5nMUZNRWxhTlozVDhhUUtyQgpDUDNDeEFTdjYxWTk5TEI4KzNXWVFIK3NYaTVGM01pYVNBZ1BkQUk3WEh1dXFET1lvMU5PL0JoSGt1aVg2QnRtCnorNTZud2pZMy8yUytSRmNBc3JMTnIwMDJZZi9oY0IraVlDNzVWYmcydVd6WTY3TWdOTGQ5VW9RU3BDRkYrVm4KM0cyUnhybnhBb0dCQU40U3M0ZVlPU2huMVpQQjdhTUZsY0k2RHR2S2ErTGZTTXFyY2pOZjJlSEpZNnhubmxKdgpGenpGL2RiVWVTbWxSekR0WkdlcXZXaHFISy9iTjIyeWJhOU1WMDlRQ0JFTk5jNmtWajJTVHpUWkJVbEx4QzYrCk93Z0wyZHhKendWelU0VC84ajdHalRUN05BZVpFS2FvRHFyRG5BYWkyaW5oZU1JVWZHRXFGKzJyQW9HQkFOMVAKK0tZL0lsS3RWRzRKSklQNzBjUis3RmpyeXJpY05iWCtQVzUvOXFHaWxnY2grZ3l4b25BWlBpd2NpeDN3QVpGdwpaZC96ZFB2aTBkWEppc1BSZjRMazg5b2pCUmpiRmRmc2l5UmJYbyt3TFU4NUhRU2NGMnN5aUFPaTVBRHdVU0FkCm45YWFweUNweEFkREtERHdObit3ZFhtaTZ0OHRpSFRkK3RoVDhkaVpBb0dCQUt6Wis1bG9OOTBtYlF4VVh5YUwKMjFSUm9tMGJjcndsTmVCaWNFSmlzaEhYa2xpSVVxZ3hSZklNM2hhUVRUcklKZENFaHFsV01aV0xPb2I2NTNyZgo3aFlMSXM1ZUtka3o0aFRVdnpldm9TMHVXcm9CV2xOVHlGanIrSWhKZnZUc0hpOGdsU3FkbXgySkJhZUFVWUNXCndNdlQ4NmNLclNyNkQrZG8wS05FZzFsL0FvR0FlMkFVdHVFbFNqLzBmRzgrV3hHc1RFV1JqclRNUzRSUjhRWXQKeXdjdFA4aDZxTGxKUTRCWGxQU05rMXZLTmtOUkxIb2pZT2pCQTViYjhibXNVU1BlV09NNENoaFJ4QnlHbmR2eAphYkJDRkFwY0IvbEg4d1R0alVZYlN5T294ZGt5OEp0ek90ajJhS0FiZHd6NlArWDZDODhjZmxYVFo5MWpYL3RMCjF3TmRKS2tDZ1lCbyt0UzB5TzJ2SWFmK2UwSkN5TGhzVDQ5cTN3Zis2QWVqWGx2WDJ1VnRYejN5QTZnbXo5aCsKcDNlK2JMRUxwb3B0WFhNdUFRR0xhUkcrYlNNcjR5dERYbE5ZSndUeThXczNKY3dlSTdqZVp2b0ZpbmNvVlVIMwphdmxoTUVCRGYxSjltSDB5cDBwWUNaS2ROdHNvZEZtQktzVEtQMjJhTmtsVVhCS3gyZzR6cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=

4、创建ConfigMap自定义配置文件,kubectl apply -f common/nginx-config.yaml

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:

5、为虚拟云主机和虚拟云主机路由定义自定义资源,支持自定义虚拟主机和虚拟路由,kubectl apply -f common/custom-resource-definitions.yaml

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: virtualservers.k8s.nginx.org
spec:
  group: k8s.nginx.org
  versions:
  - name: v1
    served: true
    storage: true
  scope: Namespaced
  names:
    plural: virtualservers
    singular: virtualserver
    kind: VirtualServer
    shortNames:
    - vs
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: virtualserverroutes.k8s.nginx.org
spec:
  group: k8s.nginx.org
  versions:
  - name: v1
    served: true
    storage: true
  scope: Namespaced
  names:
    plural: virtualserverroutes
    singular: virtualserverroute
    kind: VirtualServerRoute
    shortNames:
    - vsr

6、配置RBAC认证授权,实现ingress控制器访问集群中的其他资源,kubectl apply -f rbac/rbac.yaml

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-ingress
rules:
- apiGroups:
  - ""
  resources:
  - services
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get
  - list
  - watch
  - update
  - create
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs:
  - list
  - watch
  - get
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - k8s.nginx.org
  resources:
  - virtualservers
  - virtualserverroutes
  verbs:
  - list
  - watch
  - get
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-ingress
subjects:
- kind: ServiceAccount
  name: nginx-ingress
  namespace: nginx-ingress
roleRef:
  kind: ClusterRole
  name: nginx-ingress
  apiGroup: rbac.authorization.k8s.io

2.2.2 部署Ingress控制器

1、 部署控制器,控制器可以DaemonSets和Deployment的形式部署,如下是DaemonSets的配置文件

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
     #annotations:
       #prometheus.io/scrape: "true"
       #prometheus.io/port: "9113"
    spec:
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:edge
        imagePullPolicy: Always
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
          hostPort: 80            #通过hostPort的方式暴露端口
        - name: https
          containerPort: 443
          hostPort: 443
       #- name: prometheus
         #containerPort: 9113
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 101 #nginx
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
         #- 
						 技术沙龙 教程文章 热点综合					

评论