ole的知识小站

nginx安装

—

由

wangjm
于

nginx安装

graph LR;
 client([客户端])-. Ingress 所管理的<br>负载均衡器 .->ingress[Ingress];
 ingress-->|路由规则|service[服务];
 subgraph cluster
 ingress;
 service-->pod1[Pod];
 service-->pod2[Pod];
 end
 classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000;
 classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff;
 classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5;
 class ingress,service,pod1,pod2 k8s;
 class client plain;
 class cluster cluster;

实际这里的nginx分为两种

  • 用作ingres的 nginx ingress。 是作为k8s组件对外提供服务的。
  • k8s当前命名空间中的nginx pod与service，
    • 挂载了前端文件的pv （或pvc），用于提供静态页面访问。
    • 作为二级路由，提供更细粒度的路由控制等。

prerequisite

本文涉及的文件中，私有的nginx镜像，调整为了公版 的nginx:latest镜像

本文涉及的文件中，原来的tst都调整为了jtest。

本文涉及的文件中，原来的woyunsoft.com都调整为ole12138.cn

本文涉及的文件中，原来的wotongsoft.com都调整为ole12138.com

aoyunsoft.com相关的都先注释掉了，proxy_pass中出现的域名一定要存在才行。

jumpserver相关的配置也注掉了。

jtest作为当前操作的默认命名空间

kubectl create ns jtest
kubectl config set-context --current --namespace jtest

新建retain类型的storage class

前面建的默认nfs-storage这个默认存储类型是Delete类型的，当没有pod挂载对应的pvc时，之前的pvc会被删除。

现在需要一个Retain类型的storage class。

# 以原有的存储类型作为模板，
kubectl get sc nfs-storage -o yaml > nfs-storage-retain.yaml

#修改一下
vim nfs-storage-ratain.yaml

#看下修改后的内容
cat nfs-storage-retain.yaml 

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage-retain
parameters:
  archiveOnDelete: "true"
provisioner: fuseim.pri/ifs
reclaimPolicy: Retain
volumeBindingMode: Immediate

#创建sc
kubectl apply -f ./nfs-storage-ratain.yaml

nginx pod和service配置

pvc-h5-nginx.yaml

这里我注掉了指定的存储类，适用默认存储类（我这边默认是nfs存储）。

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
   name: h5-nginx-pvc
spec:
   accessModes:
      - ReadWriteMany
   storageClassName: "nfs-storage-retain"
   resources:
      requests:
         storage: 5Gi

应用

kubectl apply -f ./pvc-h5-nginx.yaml

web-cm

需要先建下相应的configmap

server.conf

# H5项目 ========== start
server {
    listen       10002;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/insurance/non_car/v3";
    }
}

server {
    listen       10003;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/template/v3";
    }
}

server {
    listen       10004;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/insurance/car/v3";
    }
}

server {
    listen       10005;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/union";
    }
}

server {
    listen       10006;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/sport/v3";
    }
}

server {
    listen       10007;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/customer";
    }
}

# vue测试项目
server {
    listen       10009;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/vue_demo";
    }
}

# H5悦安康
server {
    listen       10010;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/health";
    }
}

# H5项目 ========== end

# PC子目录
server {
    listen       12001;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/sys";
    }
}

server {
    listen       12002;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/shop";
    }
}

server {
    listen       12003;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/policy_manage";
    }
}
server {
    listen       12004;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/bi";
    }
}

server {
    listen       12005;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/customer";
    }
}

server {
    listen       12006;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/pc_car";
    }
}

server {
    listen       12007;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/settlement";
   }
}

## PC IOT管理
server {
    listen       12008;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/iot";
   }
}

## PC 数据大屏
server {
    listen       12009;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/data_screen";
   }
}

#PC SCRM系统
server {
    listen       13000;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/biz";
   }
}

mobile.conf

server{
    listen       80;
    server_name  mobilejtest.ole12138.cn;

     location ~ \.txt$ {
        add_header 'Access-Control-Allow-Origin' '*';
        root "/etc/nginx/conf.d";
    }

    location / {
        ssi on;
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        add_header 'Access-Control-Allow-Origin' '*';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        index index.html;
        try_files $uri $uri/ /;
        root "/usr/local/web/test/h5-components";
    }
    location ~ /components/*.*\.(js|css)$ {
        expires 10d;
    }
    location /wkbins {
        proxy_pass http://test.m.wkbins.com/;
    }
    location /resources{
        proxy_pass http://test.m.wkbins.com/resources/;
    }
    location /baiduaip {
        proxy_pass https://aip.baidubce.com/;
    }
 
    location ~ /\.txt$ {
        ssi on;
        add_header 'Access-Control-Allow-Origin' '*';
        index index.html;
        root "/usr/local/web/test/h5-components";
    }
    #new
    location ^~/non/car/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10002/;
    }
    location ^~/plate/ {
        add_header 'Access-Control-Allow-Origin' '*';
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10003/;
    }
    location ^~/insurance/car/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10004/;
    }
    location ^~/union/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10005/;
    }

    location ^~/sport/v3/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10006/;
    }

# vue测试用例
    location ^~/vue/demo/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10009/;
    }

# 悦安康
    location ^~/health/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10010/;
    }



# v4
location ^~/non/car/v4/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10012/;
    }
    location ^~/plate/v4/ {
        add_header 'Access-Control-Allow-Origin' '*';
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10013/;
    }
    location ^~/insurance/car/v4/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10014/;
    }
    location ^~/sport/v4/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10008/;
    }

}

scrm.conf

#PC SCRM系统
server{
    listen       80;
    server_name  cloudjtest.ole12138.cn;
    location ^~/biz/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:13000/;
    }
}

hmp.conf

#PC 沃享健康后台
server{
    listen       80;
    server_name  hmpjtest.ole12138.com;

    location ^~/sys/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For 
        $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12001/;
    }

    location ^~/{
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For 
        $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12008/;
    }

location ^~/data/screen/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For 
        $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12009/;
    }
}

bmp_imjtest.conf

server{
    listen       80;
    server_name  bmp.imjtest.ole12138.cn;
    location / {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12005/;
    }
}

mobile_imjtest.conf

server{
    listen        80;
    server_name   mobile.imjtest.ole12138.cn;
    location / {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10007/;
    }
}

server_v4.conf

# h5 v4
server {
    listen       10008;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/sport/v4";
    }
}
server {
    listen       10012;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/insurance/non_car/v4";
    }
}
server {
    listen       10013;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/template/v4";
    }
}
server {
    listen       10014;
    server_name  localhost;
    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/h5-components/insurance/car/v4";
    }
}

pay.conf

server{
    listen       80;
    server_name  payjtest.ole12138.cn;
    location / {
        ssi on;
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        add_header 'Access-Control-Allow-Origin' '*';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        index index.html;
        try_files $uri $uri/ /;
        root "/usr/local/web/test/h5-components";
    }
    location ~ /components/*.*\.(js|css)$ {
        expires 10d;
    }
    location /wkbins {
        proxy_pass http://test.m.wkbins.com/;
    }
    location /resources{
        proxy_pass http://test.m.wkbins.com/resources/;
    }
    location /baiduaip {
        proxy_pass https://aip.baidubce.com/;
    }
 
    location ~ /\.txt$ {
        ssi on;
        add_header 'Access-Control-Allow-Origin' '*';
        index index.html;
        root "/usr/local/web/test/h5-components";
    }
    #new
    location ^~/non/car/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10002/;
    }
    location ^~/plate/ {
        add_header 'Access-Control-Allow-Origin' '*';
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10003/;
    }
    location ^~/insurance/car/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10004/;
    }
    location ^~/union/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10005/;
    }

    location ^~/sport/v3/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10006/;
    }

# h5 v4
location ^~/non/car/v4/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10012/;
    }
    location ^~/plate/v4/ {
        add_header 'Access-Control-Allow-Origin' '*';
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10013/;
    }
    location ^~/insurance/car/v4/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10014/;
    }
    location ^~/sport/v4/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10008/;
    }
}

pc.conf

server{
    listen       80;
    server_name  bmpjtest.ole12138.cn portaljtest.ole12138.cn playjtest.ole12138.cn;
    location / {
        ssi on;
        index index.html;
        root "/usr/local/web/test/pc-components/web-pc-main";
    }

    location /components {
        ssi on;
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
                add_header Cache-Control public,max-age=604800;
        }
        index index.html;
        root "/usr/local/web/test/pc-components";
    }

    #location /h5 {
    #    proxy_pass http://mobilejtest.aoyunsoft.com/;
    #}
    location /baiduaip {
        proxy_pass https://aip.baidubce.com/;
    }
    location /image {
        proxy_pass https://pri-biz.oss-cn-shenzhen.aliyuncs.com/;
    }


    #new
    location ^~/sys/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12001/;
    }


    location ^~/shop/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12002/;
    }
    location ^~/policy/manage/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12003/;
    }
    location ^~/bi/{
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12004/;
    }
    
    location  ^~/pc/car/insurance/ {
          proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12006/;
    }
location ^~/settlement/ {
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:12007/;
    }
}

website.conf

# 沃通官网
server{
    listen       80;
    server_name  wwwjtest.ole12138.com;

    location / {
        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
            add_header Cache-Control public,max-age=604800;
        }
        ssi on;
        index index.html;
        try_files $uri $uri/ /index.html;
        root "/usr/local/web/test/pc-components/wotongsoft";
    }
}

从这些文件创建cm

kubectl create configmap web-cm --from-file bmp_imjtest.conf --from-file hmp.conf --from-file mobile.conf --from-file mobile_imjtest.conf --from-file pay.conf --from-file pc.conf --from-file scrm.conf --from-file server.conf --from-file server_v4.conf --from-file website.conf

svc-h5-nginx.yaml

创建nginx的pod和svc

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: h5-nginx
  namespace: jtest
  labels:
    app: h5-nginx
  annotations:
    reloader.stakater.com/auto: "true"
spec:
  serviceName: h5-nginx
  selector:
    matchLabels:
      app: h5-nginx
  replicas: 1
  template:
   metadata:
     labels:
       app: h5-nginx
   spec:
     containers:
       - name: nginx
         image: nginx:latest
         env:
           - name: TZ
             value: Asia/Shanghai
         ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
        ##configMap 挂载nginx配置文件
         volumeMounts:
           - mountPath: /etc/nginx/conf.d/
             name: h5-nginx-vm
             readOnly: true
           - mountPath: /usr/local/web
             name: h5-nginx-data
     volumes:
       ## 从外部挂载configMap映射nginx配置文件
       - name: h5-nginx-vm
         configMap:
           name: web-cm
       ## 从外部挂载nginx映射的NAS 磁盘文件
       - name: h5-nginx-data
         persistentVolumeClaim:
           claimName: h5-nginx-pvc
---
kind: Service
apiVersion: v1
metadata:
  namespace: jtest
  name: h5-nginx
spec:
  selector:
    app: h5-nginx
  clusterIP: None
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
      name: h5-nginx-srv-port

应用

kubectl apply -f ./svc-h5-nginx.yaml

backstage-cm

创建需要用到的configmap

api.conf

server {
    charset utf-8;
    #access_log  logs/host.access.log  main;

    listen   80;
    server_name apijtest.ole12138.cn;
    location / {
        proxy_pass   http://openapi-gateway:8067;
        proxy_set_header   Host    $host;
        proxy_set_header   X-Real-IP   $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location ^~/openapi/third-interface/upgrade/warning{
        default_type application/json;
        return 200 '{"state":"1","transId":"CEFB24F1-A3D0-40D9-BF84-9BF94F456B13","transTime":"2020-06-24 09:22:16.634","responseTime":"2020-06-24 09:22:16","transFor":"http://izwz9gwebxoid0ppb92uohz:5014/upgrade/warning","userName":"auto","message":{"code":"SYS1-0001","detail":"","info":"操作成功"},"data":{"status":"0","message":"升级完成"}}';
    }
}

renew.conf

server{
    listen       80;
    server_name   test.renew.ole12138.cn;

    location / {
        proxy_pass  http://openapi-gateway:8067/openapi/policy/renew/auth/;
    }

    location  /mission/auth{
        proxy_pass  http://openapi-gateway:8067/openapi/policy/mission/auth;
    }
}

jumpserver.conf

server{
    listen       80;
    server_name  jump.ole12138.cn;
    client_max_body_size 100m;  # 录像及文件上传大小限制
    location / {
        proxy_pass http://jumpserver:80;
        proxy_buffering off;
        proxy_http_version 1.1;
        proxy_request_buffering off;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $http_connection;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        access_log off;
    }
}

oauth2.conf

server{
    listen       80;
    server_name  oauth2jtest.ole12138.cn;
    location ~ \.txt$ {
        add_header 'Access-Control-Allow-Origin' '*';
        root "/etc/nginx/conf.d";
    }

    location / { 
        proxy_pass   http://openapi-gateway:8067/openapi/pay/open/getOpenId;
    }
    location /share {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/open/getOpenId/share;
    }
    location /proxy{
        proxy_pass http://openapi-gateway:8067/openapi/pay/open/getOpenId/proxy;
    }
    location /unify{
        proxy_pass http://openapi-gateway:8067/openapi/pay/open/getOpenId/unify;
    }
    location /mini{
        proxy_pass http://openapi-gateway:8067/openapi/pay/open/getOpenId/mini;
    }
    location /wxresolve{
        proxy_pass http://openapi-gateway:8067/openapi/pay/m/wechatChannelPartner/getOpenId/share;
    }
    location /thirdUserIdResolve{
        proxy_pass http://openapi-gateway:8067/openapi/pay/m/thirdChannel/menuUrl/resolve/userId;
    }
    location /thirdMobileResolve{
        proxy_pass http://openapi-gateway:8067/openapi/pay/m/thirdChannel/menuUrl/resolve/mobile;
    }
    location /agent/share {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/open/getOpenId/agent/share;
    }
    location /agent/share/oauth2 {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/open/getOpenId/agent/share/oauth2;
    }

    location /wxoauth2 {
        proxy_pass  http://openapi-gateway:8067/openapi/pay/open/getOpenId/wld;
    }

    location /wld/oauth2{
        proxy_pass  http://openapi-gateway:8067/openapi/pay/open/getOpenId/wld/oauth2;
    }

    location /alioauth2{
        proxy_pass http://openapi-gateway:8067/openapi/pay/open/getOpenId/ali/authCallBack;
    }

    location /authorize2{
        proxy_pass http://openapi-gateway:8067/openapi/pay/open/getOpenId/ali/authorize2;
    }

    location /shopping/share {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/open/getOpenId/shopping/share;
    }
    location /shopping/share/oauth2 {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/open/getOpenId/shopping/share/oauth2;
    }

    ##支付回调   
    location /unifyPayBackResolve {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/payCallBack/unifyPayBackResolve;
    }
    ##待支付回调
    location /unifyWaitPayResolve {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/payCallBack/unifyWaitPayResolve;
    }
    ##统一的微信跳转
    location /unifyRedirect/ {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/payCallBack/unifyRedirect/oauth2;
    }
    ##待支付订单分享
    location /nonCarWaitPay/share {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/payCallBack/non/waitPay/share;
    }
    location /carWaitPay/share {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/payCallBack/car/waitPay/share;
    }
    location /wld/extension {
        proxy_pass   http://openapi-gateway:8067/openapi/wld-v2/url/extension/auth;
    }

    location /iot/resolve {
        proxy_pass   http://openapi-gateway:8067/openapi/wld-v2/m/user/thirdPageConfig/resolver;
    }

    location /iot/share{
        proxy_pass   http://openapi-gateway:8067/openapi/wld-v2/m/user/thirdPageConfig/share;
    }

    location /channelOpenId {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/m/channelPartner/getOpenId/redirect;
    } 
    location /resolveOpenId {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/m/wechatChannelPartner/getOpenId/redirect;
    } 
    location /wxshare {
        proxy_pass   http://openapi-gateway:8067/openapi/pay/m/channelPartner/getOpenId/share;
    }
    location /member/auth{
        proxy_pass   http://openapi-gateway:8067/openapi/wld-v2/weChatAccount/auth;
    }
    location /dev/member/auth{
        proxy_pass http://10.16.11.39:15008/weChatAccount/auth;
    }

    location /commonOpenId {
        proxy_pass    http://openapi-gateway:8067/openapi/pay/officialAccountsMessage/getOpenId/redirect;
    }
    location /wxPush/message{
        proxy_pass  http://openapi-gateway:8067/openapi/pay/wechatMessage/messageLink;
    }
    location /car/waitPay {
        proxy_pass    http://openapi-gateway:8067/openapi/pay/payCallBack/car/WaitPay;
    }
    location /car/waitPay/shareV2 {
        proxy_pass    http://openapi-gateway:8067/openapi/pay/payCallBack/car/waitPay/shareV2;
    }

    location /activityCallBack{
        proxy_pass   http://openapi-gateway:8067/openapi/member/order/payShortLink;
    }

    ##iot平台微信公众号登录授权短链
    location /iotBiz/member/auth{
        proxy_pass  http://wld-service-zuul:8077/openapi/iot-biz/weChatAccount/auth;
    }

    ##iot平台微信公众号登录授权回调短链
        location /iotBiz/member/auth/callback{
    proxy_pass  http://wld-service-zuul:8077/openapi/iot-biz/weChatAccount/auth/callback;
    }

    ##续保推送亚美
    location /agent/renew/policyRenew{
        proxy_pass  http://openapi-gateway:8067/openapi/pay/renew/policyRenew;
    }

    ##团队邀请
    location /invited/team{
        proxy_pass  http://openapi-gateway:8067/openapi/channel/agentTeamRelation/invitedLink;
    }

    location /thirdPlatform{
        proxy_pass    http://openapi-gateway:8067/openapi/pay/thirdPlatform;
    }

    location /wxMenu{
        proxy_pass    http://openapi-gateway:8067/openapi/wld-v2/wx/application/menu;
    }

    location /wxMenu/analyse {
        proxy_pass   http://openapi-gateway:8067/openapi/wld-v2/m/user/thirdPageConfig/analyse;
    }

    location /eventMessage {
        proxy_pass   http://openapi-gateway:8067/openapi/wld-v2/eventRule/eventMessage;
    }
}

wt.conf

server{
    listen       80;
    server_name  jtest.ole12138.com;

    location  /{
        proxy_pass  http://wld-service-zuul:8077;
        proxy_set_header   Host    $host;
        proxy_set_header   X-Real-IP   $remote_addr;
        proxy_set_header   X-Forwarded-For 
        $proxy_add_x_forwarded_for;
    }
}

push.conf

server{
    listen       80;
    server_name  test.push.ole12138.cn;

    location  /auth {
        proxy_pass  http://openapi-gateway:8067/openapi/policy/mission/auth;
    }
}

从以上创建configmap

kubectl create configmap backstage-cm --from-file api.conf --from-file jumpserver.conf --from-file oauth2.conf --from-file push.conf --from-file renew.conf --from-file wt.conf

svc-oauth-nginx.yaml

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: oauth-nginx
  namespace: jtest
  labels:
    app: oauth-nginx
  annotations:
    reloader.stakater.com/auto: "true"
spec:
  serviceName: oauth-nginx
  selector:
    matchLabels:
      app: oauth-nginx
  replicas: 1
  template:
   metadata:
     labels:
       app: oauth-nginx
   spec:
     containers:
       - name: nginx
         image: nginx:latest
#         lifecycle:
#           postStart:
#             exec:
#               command:
#                 - /bin/sh
#                 - '-c'
#                 - >-
#                   rm -rf /etc/nginx/nginx.conf
         env:
           - name: TZ
             value: Asia/Shanghai
         ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
         ##configMap 挂载nginx配置文件
         volumeMounts:
           - mountPath: /etc/nginx/conf.d/
             name: h5-nginx-vm
             #readOnly: true
           - mountPath: /usr/local/web
             name: h5-nginx-data
     volumes:
       ## 从外部挂载configMap映射nginx配置文件
       - name: h5-nginx-vm
         configMap:
           name: backstage-cm
       ## 从外部挂载nginx映射的NAS 磁盘文件
       - name: h5-nginx-data
         persistentVolumeClaim:
           claimName: h5-nginx-pvc
---
kind: Service
apiVersion: v1
metadata:
  namespace: jtest
  name: oauth-nginx
spec:
  selector:
    app: oauth-nginx
  clusterIP: None
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
      name: oauth-nginx-srv-port

这里暂先忽略reloader.stakater.com/auto: "true"相关的配置.

应用

kubectl apply -f ./svc-oauth-nginx.yaml

wotong-backstage-cm

api.conf

server {
    charset utf-8;
    #access_log  logs/host.access.log  main;

    listen   80;
    server_name jtest.ole12138.com;


    location ^~/openapi/third-interface/upgrade/warning{
      default_type application/json;
      return 200 '{"state":"1","transId":"CEFB24F1-A3D0-40D9-BF84-9BF94F456B13","transTime":"2020-06-24 09:22:16.634","responseTime":"2020-06-24 09:22:16","transFor":"http://izwz9gwebxoid0ppb92uohz:5014/upgrade/warning","userName":"auto","message":{"code":"SYS1-0001","detail":"","info":"操作成功"},"data":{"status":"0","message":"升级完成"}}';
    }
}

oauth2.conf

server{
    listen       80;
    server_name  oauth2jtest.ole12138.com;
    location ~ \.txt$ {
        add_header 'Access-Control-Allow-Origin' '*';
        root "/etc/nginx/conf.d";
    }
    ##iot平台微信公众号登录授权回调短链
    location /iotBiz/member/auth/callback{
          proxy_pass  http://wld-service-zuul:8077/openapi/iot-biz/weChatAccount/auth/callback;
     }

#     location / {
#        if ($request_uri ~* "\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$"){
#            add_header Cache-Control public,max-age=604800;
#        }
#        ssi on;
#        index index.html;
#        try_files $uri $uri/ /index.html;
#        root "/usr/local/";
#    }

    location /wcThirdPlatform{
        proxy_pass http://openapi-gateway:8067/openapi/pay/wechat/thirdPlatform;
    }

    location /mini/register{
        proxy_pass http://openapi-gateway:8067/openapi/wld-v2/miniProgram/fastRegisterAuth;
    }

}

根据以上配置文件，创建configmap

kubectl create configmap wotong-backstage-cm --from-file api.conf  --from-file oauth2.conf

svc-wotong-nginx.yaml

目前看来，这个服务只做了一些二次路由，而没有静态数据

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: wotong-nginx
  namespace: jtest
  labels:
    app: wotong-nginx
  annotations:
    reloader.stakater.com/auto: "true"
spec:
  serviceName: wotong-nginx
  selector:
    matchLabels:
      app: wotong-nginx
  replicas: 1
  template:
   metadata:
     labels:
       app: wotong-nginx
   spec:
     containers:
       - name: nginx
         image: nginx:latest
#         lifecycle:
#           postStart:
#             exec:
#               command:
#                 - /bin/sh
#                 - '-c'
#                 - >-
#                   rm -rf /etc/nginx/nginx.conf
         env:
           - name: TZ
             value: Asia/Shanghai
         ports:
            - name: http
              containerPort: 80
#              hostPort: 80
            - name: https
              containerPort: 443
#              hostPort: 443
        ##configMap 挂载nginx配置文件
         volumeMounts:
           - mountPath: /etc/nginx/conf.d/
             name: wotong-nginx-vm
#             readOnly: true
#           - mountPath: /usr/local/web
#             name: wotong-nginx-data
     volumes:
       - name: wotong-nginx-vm
        ## 从外部挂载configMap映射nginx配置文件
         configMap:
           name: wotong-backstage-cm
#           items:
#             - key: oauth2.conf
#               path: oauth2.conf
       ## 从外部挂载nginx映射的NAS 磁盘文件
#       - name: wotong-nginx-data
#         persistentVolumeClaim:
#           claimName: wotong-nginx-nas-csi-pvc
---
kind: Service
apiVersion: v1
metadata:
  namespace: jtest
  name: wotong-nginx
spec:
  selector:
    app: wotong-nginx
  clusterIP: None
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
      name: wotong-nginx-srv-port

应用

kubectl apply -f ./svc-wotong-nginx.yaml

依赖的相关服务

oauth-nginx的statefulset可能会启动失败。

[root@jingmin-kube-archlinux backstage-cm]# kubectl describe pod/oauth-nginx-0 
...
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  18s                default-scheduler  Successfully assigned jtest/oauth-nginx-0 to jingmin-kube-archlinux
  ...
  Normal   Pulled     12s                kubelet            Successfully pulled image "nginx:latest" in 2.233s (2.233s including waiting)
  Warning  BackOff    10s (x2 over 11s)  kubelet            Back-off restarting failed container nginx in pod oauth-nginx-0_jtest(84e0a0a4-2c1a-4379-858e-438c0327048a)
  

[root@jingmin-kube-archlinux backstage-cm]# kubectl logs pod/oauth-nginx-0 
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf is not a file or does not exist
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/09/12 15:44:45 [emerg] 1#1: host not found in upstream "openapi-gateway" in /etc/nginx/conf.d/api.conf:8
nginx: [emerg] host not found in upstream "openapi-gateway" in /etc/nginx/conf.d/api.conf:8

这是是由于nginx配置文件中，服务地址（或者说，k8s中的私有域名）openapi-gateway解析不到。

类似的，还有jumpserver,wld-service-zuul的服务地址解析不到，都会导致nginx的pod启动失败。

jumpserver是堡垒机服务，可以先不考虑。在前面的配置文件中可以先注释掉。

而openapi-gateway和wld-service-zuul对应的服务，一定要先启动才行。

nginx ingress配置

创建ingress

导出并调整一下原本的tst命名空间下的ingress

$env:KUBECONFIG="C:\Users\wangjm\.kube\config_yak"

kubectl config set-context --current --namespace tst

kubectl get ingress -o yaml > ingress-tst.yaml

vim ingress-tst.yaml

mv ingress-tst.yaml ingress-jtest.yaml

以download下来的ingress作为模板

调整namespace

删除status相关的内容

调整apiVersion（1.19有变动，将apiVersion: extensions/v1beta1改为apiVersion: networking.k8s.io/v1）

删除creationTimestamp

删除resourceVersion

删除generation

删除selfLink

删除uid

woyunsoft.com都调整为ole12138.cn （根据自己的域名调整）

wotongsoft.com都调整为ole12138.com （根据自己的域名调整）

tst都替换为jest

暂时先注释掉elasticsearch相关的内容

暂时先注释掉kafka相关的内容

暂时先注释掉jira相关的内容

创建对应的ingress

kubectl apply -f ./ingress-jtest.yaml

会发现创建失败。提示serviceName和servicePort不合法。同时pathType也不再有默认值。

pathType` no longer has a default value in v1; "Exact", "Prefix", or "ImplementationSpecific" must be specified

unknown field "servicePort" in io.k8s.api.networking.v1.IngressBackend

unknown field "serviceName" in io.k8s.api.networking.v1.IngressBackend

这是由于nginx ingress 语法在1.19之后有变动。

参考：https://stackoverflow.com/questions/64125048/get-error-unknown-field-servicename-in-io-k8s-api-networking-v1-ingressbacken

需要手动编辑ingress的配置。

添加pathType参数配置。

修改serviceName和servicePort相关的配置。

这是修改后的ingress-jtest.yaml

apiVersion: v1
items:
#- apiVersion: networking.k8s.io/v1
#  kind: Ingress
#  metadata:
#    annotations:
#      nginx.ingress.kubernetes.io/service-weight: ""
#    name: elasticsearch
#    namespace: jtest
#  spec:
#    rules:
#    - host: elasticsearch.c253e0c129d8f453a82dfb1ae4ba19613.cn-shenzhen.alicontainer.com
#      http:
#        paths:
#        - backend:
#            service:
#              name: elasticsearch-es-http
#              port:
#                number: 9200
#          path: /
#          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      nginx.ingress.kubernetes.io/service-weight: 'h5-nginx: 100, oauth-nginx: 100,
        h5-nginx: 100, h5-nginx: 100, h5-nginx: 100, h5-nginx: 100, h5-nginx: 100,
        h5-nginx: 100, oauth-nginx: 100, h5-nginx: 100, h5-nginx: 100, h5-nginx: 100'
    name: h5-nginx
    namespace: jtest
  spec:
    rules:
    - host: mobilejtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: oauth2jtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: portaljtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: bmpjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: payjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: bmp.imjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: mobile.imjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: playjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: test.renew.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: cloudjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: hmpjtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: wwwjtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      nginx.ingress.kubernetes.io/service-weight: ""
    name: im-zuul
    namespace: jtest
  spec:
    rules:
    - host: api.imjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: server-im-zuul
              port:
                number: 8068
          path: /
          pathType: Prefix
#- apiVersion: networking.k8s.io/v1
#  kind: Ingress
#  metadata:
#    annotations:
#      nginx.ingress.kubernetes.io/service-weight: ""
#    name: jira
#    namespace: jtest
#  spec:
#    rules:
#    - host: jira.ole12138.cn
#      http:
#        paths:
#        - backend:
#            service:
#              name: jira
#              port:
#                number: 8080
#          path: /
#          pathType: Prefix
#    tls:
#    - hosts:
#      - jira.ole12138.cn
#      secretName: jira-secret0
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      nginx.ingress.kubernetes.io/service-weight: ""
    name: job-nginx
    namespace: jtest
  spec:
    rules:
    - host: jobjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: service-job-admin
              port:
                number: 5019
          path: /
          pathType: Prefix
#- apiVersion: networking.k8s.io/v1
#  kind: Ingress
#  metadata:
#    annotations:
#      nginx.ingress.kubernetes.io/service-weight: ""
#    name: kafka-eagle
#    namespace: jtest
#  spec:
#    rules:
#    - host: kafka-eagle.c253e0c129d8f453a82dfb1ae4ba19613.cn-shenzhen.alicontainer.com
#      http:
#        paths:
#        - backend:
#            service:
#              name: kafka-eagle
#              port:
#                number: 8048
#          path: /
#          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      nginx.ingress.kubernetes.io/service-weight: ""
    name: openapi-h5-gatway
    namespace: jtest
  spec:
    rules:
    - host: m.apijtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: openapi-h5-gateway
              port:
                number: 8057
          path: /
          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      nginx.ingress.kubernetes.io/proxy-body-size: 100m
      nginx.ingress.kubernetes.io/service-weight: ""
      nginx.org/client-max-body-size: 100m
    name: openapi-ingress
    namespace: jtest
  spec:
    rules:
    - host: apijtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: openapi-gateway
              port:
                number: 8067
          path: /delete
          pathType: Prefix
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: openapi.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: openapi-gateway
              port:
                number: 8067
          path: /
          pathType: Prefix
    - host: test.push.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    tls:
    - hosts:
      - apijtest.ole12138.cn
      secretName: openapi-tls
    - hosts:
      - openapi.ole12138.cn
      secretName: openapi-tls
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      nginx.ingress.kubernetes.io/service-weight: ""
    name: wotong-nginx
    namespace: jtest
  spec:
    rules:
    - host: oauth2jtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: wotong-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      nginx.ingress.kubernetes.io/service-weight: ""
    name: wotong-jtest
    namespace: jtest
  spec:
    rules:
    - host: jtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
        - backend:
            service:
              name: wotong-nginx
              port:
                number: 80
          path: /openapi/third-interface/upgrade/warning
          pathType: Prefix
    - host: apijtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: wld-service-zuul
              port:
                number: 8077
          path: /
          pathType: Prefix
    tls:
    - hosts:
      - apijtest.ole12138.com
      secretName: apijtest-wotong
kind: List
#metadata:
#  resourceVersion: ""

重新创建对应的ingress

kubectl apply -f ./ingress-jtest.yaml

创建ingress成功

使用cert-manager自动配置https

之前章节配好了cert-manager，在当前命名空间下还是建一下staging和production环境的issuer (由Let’s Encrypt提供服务)

修改其中的邮箱部分，用于创建账号，以及将来有证书将要过期相关的内容会发到对应的邮箱

[root@jingmin-kube-archlinux issuer]# vim staging-issuer.yaml 
[root@jingmin-kube-archlinux issuer]# cat staging-issuer.yaml 
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    # The ACME server URL
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: 784319947@qq.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-staging
    # Enable the HTTP-01 challenge provider
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx

部署staging-issuer

kubectl create -f ./staging-issuer.yaml

类似的方式，创建production-issuer

wget https://raw.githubusercontent.com/cert-manager/website/master/content/docs/tutorials/acme/example/production-issuer.yaml

同样，修改其中的邮箱为自己的邮箱

[root@jingmin-kube-archlinux issuer]# vim production-issuer.yaml 
[root@jingmin-kube-archlinux issuer]# cat production-issuer.yaml 
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: 784319947@qq.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx

部署到当前命名空间中

kubectl create -f ./production-issuer.yaml

这两个issuer都通过http01的方式向Let’s Encrypt 发出challenge.

kubectl describe issuer

可以看到description中都有一条Message: The ACME account was registered with the ACME server

向ingress中，

添加cert-manager的issuer注解cert-manager.io/issuer: letsencrypt-staging，

类似这样

- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-staging

添加tls的hosts和secretsName部分

（如果是不需要tls加密的http服务，可以略过）

（secretsName名称随便起，cert-manager会自动生成. 但是不能重名）

（上面有重名的openapi-tls需要改名为openapi-tls1和openapi-tls2）

vim ingress-jtest.yaml 
kubectl apply -f ./ingress-jtest.yaml

可以看下，会自动生成对应的secret

# get secret | grep Opaque |grep -v letsencrypt

apijtest-wotong-9chkl   Opaque                           1      21m
openapi-tls1-q4lqw      Opaque                           1      92s
openapi-tls2-w9brf      Opaque                           1      92s

然后需要到域名服务商那里，配置相应子域名。

在浏览器中，使用https访问ingress地址，比如https://openapi.ole12138.cn/, 会有提示警告，看下证书，以及颁发者（虽然是提示无效，但不是k8s提供默认的fake证书，而是Let’s Encrypt提供的staging证书）。

现在再修改一下ingress中annotations中的issuer，切换为production环境的issuer。注意其中一行： cert-manager.io/issuer: letsencrypt-prod

vim ingress-jtest.yaml
:%s/letsencrypt-staging/letsencrypt-prod/g

重新应用新的ingress配置

kubectl apply -f ./ingress-jtest.yaml

稍等一分钟，再次在浏览器中，以https方式，访问nacos的ingress地址https://openapi.ole12138.cn/. 正常的话，可以直接访通，没有任何警告。 看下地址栏前面的锁头标志，点看看下证书，确认是Let’s Encrypt颁发的。

最后提供一下最终的ingress-jtest.yaml配置

apiVersion: v1
items:
#- apiVersion: networking.k8s.io/v1
#  kind: Ingress
#  metadata:
#    annotations:
#      cert-manager.io/issuer: letsencrypt-prod
#      nginx.ingress.kubernetes.io/service-weight: ""
#    name: elasticsearch
#    namespace: jtest
#  spec:
#    rules:
#    - host: elasticsearch.c253e0c129d8f453a82dfb1ae4ba19613.cn-shenzhen.alicontainer.com
#      http:
#        paths:
#        - backend:
#            service:
#              name: elasticsearch-es-http
#              port:
#                number: 9200
#          path: /
#          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/service-weight: 'h5-nginx: 100, oauth-nginx: 100,
        h5-nginx: 100, h5-nginx: 100, h5-nginx: 100, h5-nginx: 100, h5-nginx: 100,
        h5-nginx: 100, oauth-nginx: 100, h5-nginx: 100, h5-nginx: 100, h5-nginx: 100'
    name: h5-nginx
    namespace: jtest
  spec:
    rules:
    - host: mobilejtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: oauth2jtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: portaljtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: bmpjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: payjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: bmp.imjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: mobile.imjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: playjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: test.renew.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: cloudjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: hmpjtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: wwwjtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: h5-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/service-weight: ""
    name: im-zuul
    namespace: jtest
  spec:
    rules:
    - host: api.imjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: server-im-zuul
              port:
                number: 8068
          path: /
          pathType: Prefix
#- apiVersion: networking.k8s.io/v1
#  kind: Ingress
#  metadata:
#    annotations:
#      cert-manager.io/issuer: letsencrypt-prod
#      nginx.ingress.kubernetes.io/service-weight: ""
#    name: jira
#    namespace: jtest
#  spec:
#    rules:
#    - host: jira.ole12138.cn
#      http:
#        paths:
#        - backend:
#            service:
#              name: jira
#              port:
#                number: 8080
#          path: /
#          pathType: Prefix
#    tls:
#    - hosts:
#      - jira.ole12138.cn
#      secretName: jira-secret0
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/service-weight: ""
    name: job-nginx
    namespace: jtest
  spec:
    rules:
    - host: jobjtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: service-job-admin
              port:
                number: 5019
          path: /
          pathType: Prefix
#- apiVersion: networking.k8s.io/v1
#  kind: Ingress
#  metadata:
#    annotations:
#      cert-manager.io/issuer: letsencrypt-prod
#      nginx.ingress.kubernetes.io/service-weight: ""
#    name: kafka-eagle
#    namespace: jtest
#  spec:
#    rules:
#    - host: kafka-eagle.c253e0c129d8f453a82dfb1ae4ba19613.cn-shenzhen.alicontainer.com
#      http:
#        paths:
#        - backend:
#            service:
#              name: kafka-eagle
#              port:
#                number: 8048
#          path: /
#          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/service-weight: ""
    name: openapi-h5-gatway
    namespace: jtest
  spec:
    rules:
    - host: m.apijtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: openapi-h5-gateway
              port:
                number: 8057
          path: /
          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/proxy-body-size: 100m
      nginx.ingress.kubernetes.io/service-weight: ""
      nginx.org/client-max-body-size: 100m
    name: openapi-ingress
    namespace: jtest
  spec:
    rules:
    - host: apijtest.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: openapi-gateway
              port:
                number: 8067
          path: /delete
          pathType: Prefix
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    - host: openapi.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: openapi-gateway
              port:
                number: 8067
          path: /
          pathType: Prefix
    - host: test.push.ole12138.cn
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
    tls:
    - hosts:
      - apijtest.ole12138.cn
      secretName: openapi-tls1
    - hosts:
      - openapi.ole12138.cn
      secretName: openapi-tls2
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/service-weight: ""
    name: wotong-nginx
    namespace: jtest
  spec:
    rules:
    - host: oauth2jtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: wotong-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      cert-manager.io/issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/service-weight: ""
    name: wotong-jtest
    namespace: jtest
  spec:
    rules:
    - host: jtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: oauth-nginx
              port:
                number: 80
          path: /
          pathType: Prefix
        - backend:
            service:
              name: wotong-nginx
              port:
                number: 80
          path: /openapi/third-interface/upgrade/warning
          pathType: Prefix
    - host: apijtest.ole12138.com
      http:
        paths:
        - backend:
            service:
              name: wld-service-zuul
              port:
                number: 8077
          path: /
          pathType: Prefix
    tls:
    - hosts:
      - apijtest.ole12138.com
      secretName: apijtest-wotong
kind: List
#metadata:
#  resourceVersion: ""

评论

发表回复

您的电子邮箱地址不会被公开。 必填项已用 * 标注