Contents
redis安装
参考： https://redis.com/redis-enterprise-software/redis-enterprise-on-kubernetes/
参考： https://docs.redis.com/latest/kubernetes/
如上两个官网的参考，看了下是redis-enterprise版本的，不是oss版本的。不适用。
背景
redis高可用（主从、哨兵、集群）
参考： https://zhuanlan.zhihu.com/p/177000194
redis的高可用主要有三种：主从、哨兵、集群。
简单主从模式，一主多从。master节点故障时，需要手动切换主节点。
哨兵模式，是建立在一主多从基础上，会监测节点故障，主节点故障时会选举从节点作为新的主节点。
集群模式，是redis3之后引入的。使用hash槽，分布式存储数据。而且可以多主多从。各节点都可以接收流量，并转发到合适的节点。
安装yaml选型
一些Sentinal模式的文章
参考： https://www.airplane.dev/blog/deploy-redis-cluster-on-kubernetes
参考： https://sarweshsuman-1.medium.com/deploying-redis-ha-cluster-in-kubernetes-437162337625
一些Cluster模式的文章，看起来是手动建的。没去试，感觉维护是个问题。
参考： https://cloud.tencent.com/developer/article/1796311
参考： https://juejin.cn/post/7042173689948798989
参考： https://segmentfault.com/a/1190000039196137
参考： https://www.cnblogs.com/Sunzz/p/15237497.html
helm官网查询redis，以star数排序
https://artifacthub.io/packages/search?ts_query_web=redis&sort=stars&page=1注意到其中两个chart
https://artifacthub.io/packages/helm/bitnami/redis
https://artifacthub.io/packages/helm/bitnami/redis-cluster点开有这两个chart的说明和区别，一个是基于sentinal的，一个是基于cluster的：Choose between Redis® Helm Chart and Redis® Cluster Helm Chart 在 Redis® Helm Chart 和 Redis® Cluster Helm Chart 之间进行选择
You can choose any of the two Redis® Helm charts for deploying a Redis® cluster. While Redis® Helm Chart will deploy a master-slave cluster using Redis® Sentinel, the Redis® Cluster Helm Chart will deploy a Redis® Cluster with sharding. The main features of each chart are the following: 您可以选择两个 Redis® Helm 图表中的任意一个来部署 Redis® 集群。 Redis® Helm Chart 将使用 Redis® Sentinel 部署主从集群，而 Redis® Cluster Helm Chart 将部署带分片的 Redis® 集群。每个图表的主要特点如下：
| Redis® Redis® | Redis® Cluster Redis®集群 |
|---|---|
| Supports multiple databases 支持多种数据库 | Supports only one database. Better if you have a big dataset 仅支持一个数据库。如果你有一个大数据集就更好了 |
| Single write point (single master) 单写入点（单主控） | Multiple write points (multiple masters) 多个写入点（多个主控） |
 |
 |
Looking to use Redisreg; Cluster in production? Try VMware Application Catalog, the enterprise edition of Bitnami Application Catalog.
对应的github仓库地址是这两个
https://github.com/bitnami/charts/tree/main/bitnami/redis
https://github.com/bitnami/charts/tree/main/bitnami/redis-cluster最终选择使用helm安装redis和redis-cluster，都试一下。
helm安装redis
Redis® Helm Chart will deploy a master-replica cluster, with the option of enabling using Redis® Sentinel. Redis® Helm Chart 将部署主副本集群，并可选择启用使用 Redis® Sentinel。
参考： https://artifacthub.io/packages/helm/bitnami/redis#configuration-and-installation-details
参考： https://artifacthub.io/packages/helm/bitnami/redis#cluster-topologies
对于Redis® Helm Chart 这个chart，有好几种使用方式。
-
主从模式（默认）。主读写，从读。分两个service，一个读写service，一个读service。
-
standalone模式，只有一个实例。只有一个service，读写，6379端口。
-
启用sentinal的主从模式。Redis® service: Exposes port 6379 for Redis® read-only operations and port 26379 for accessing Redis® Sentinel.
-
多主模式。实验性的，暂不考虑。
测试部署主从模式redis
先单独建个命名空间，设为当前默认命名空间，试下吧
kubectl create ns redis
kubectl config set-context --current --namespace redis使用oci兼容的helm仓库镜像，
参考： https://helm.sh/docs/topics/registries/
参考： https://blog.bitnami.com/2023/01/bitnami-helm-charts-available-as-oci.html
参考： https://github.com/bitnami/charts/tree/main/bitnami/redis
然后下载下redis的helm配置，看下
helm pull oci://registry-1.docker.io/bitnamicharts/redis --untar
cd redis
复制默认配置作为模板，复制，编辑新文件，保留需要调整的内容
cp ./values.yaml ./my-override-values.yaml
vim ./my-override-values.yaml 如下是我需要覆盖调整的配置：
[root@jingmin-kube-archlinux redis]# cat ./my-override-values.yaml
global:
storageClass: "nfs-storage"
redis:
password: "Redis12345"
## @param architecture Redis® architecture. Allowed values: `standalone` or `replication`
##
architecture: replication
## @param commonConfiguration [string] Common configuration to be added into the ConfigMap
## ref: https://redis.io/topics/config
##
commonConfiguration: |-
# Enable AOF https://redis.io/topics/persistence#append-only-file
appendonly yes
# Disable RDB persistence, AOF persistence already enabled.
save ""
## @param existingConfigmap The name of an existing ConfigMap with your custom configuration for Redis® nodes
##
existingConfigmap: ""
## @section Redis® master configuration parameters
##
master:
## @param master.configuration Configuration for Redis® master nodes
## ref: https://redis.io/topics/config
##
configuration: ""
## @param master.disableCommands Array with Redis® commands to disable on master nodes
## Commands will be completely disabled by renaming each to an empty string.
## ref: https://redis.io/topics/security#disabling-of-specific-commands
##
disableCommands:
- FLUSHDB
- FLUSHALL
## Redis® master service parameters
##
service:
## @param master.service.type Redis® master service type
##
type: ClusterIP
## @param master.service.ports.redis Redis® master service port
##
ports:
redis: 6379
## @param master.service.nodePorts.redis Node port for Redis® master
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
## NOTE: choose port between <30000-32767>
##
nodePorts:
redis: ""
## @section Redis® replicas configuration parameters
##
replica:
## @param replica.replicaCount Number of Redis® replicas to deploy
##
replicaCount: 3
## @param replica.configuration Configuration for Redis® replicas nodes
## ref: https://redis.io/topics/config
##
configuration: ""
## @param replica.disableCommands Array with Redis® commands to disable on replicas nodes
## Commands will be completely disabled by renaming each to an empty string.
## ref: https://redis.io/topics/security#disabling-of-specific-commands
##
disableCommands:
- FLUSHDB
- FLUSHALL
## Redis® replicas service parameters
##
service:
## @param replica.service.type Redis® replicas service type
##
type: ClusterIP
## @param replica.service.ports.redis Redis® replicas service port
##
ports:
redis: 6379
## @param replica.service.nodePorts.redis Node port for Redis® replicas
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
## NOTE: choose port between <30000-32767>
##
nodePorts:
redis: ""
## Autoscaling configuration
##
autoscaling:
## @param replica.autoscaling.enabled Enable replica autoscaling settings
##
enabled: false
## ServiceAccount configuration
##
## @section Redis® Sentinel configuration parameters
##
sentinel:
## @param sentinel.enabled Use Redis® Sentinel on Redis® pods.
## IMPORTANT: this will disable the master and replicas services and
## create a single Redis® service exposing both the Redis and Sentinel ports
##
enabled: false
这里主要改了下全局的redis密码和使用的存储类storageclass。
没有启用sentinal（默认），这里列出只是为了更明确些。
使用此配置，覆盖部分默认配置，创建redis
[root@jingmin-kube-archlinux redis]# cd ..
[root@jingmin-kube-archlinux k8s]# helm install redis -f ./redis/my-override-values.yaml ./redis/
NAME: redis
LAST DEPLOYED: Fri Sep 1 16:25:24 2023
NAMESPACE: redis
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 18.0.1
APP VERSION: 7.2.0
** Please be patient while the chart is being deployed **
Redis® can be accessed on the following DNS names from within your cluster:
redis-master.redis.svc.cluster.local for read/write operations (port 6379)
redis-replicas.redis.svc.cluster.local for read-only operations (port 6379)
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace redis redis -o jsonpath="{.data.redis-password}" | base64 -d)
To connect to your Redis® server:
1. Run a Redis® pod that you can use as a client:
kubectl run --namespace redis redis-client --restart='Never' --env REDIS_PASSWORD=$REDIS_PASSWORD --image docker.io/bitnami/redis:7.2.0-debian-11-r0 --command -- sleep infinity
Use the following command to attach to the pod:
kubectl exec --tty -i redis-client \
--namespace redis -- bash
2. Connect using the Redis® CLI:
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-master
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-replicas
To connect to your database from outside the cluster execute the following commands:
kubectl port-forward --namespace redis svc/redis-master 6379:6379 &
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379上面有一些提示，告诉我们如何在集群内和集群外连接到redis。
先看下是否部署正常。
[root@jingmin-kube-archlinux k8s]# kubectl get all,cm,secrets,cr
NAME READY STATUS RESTARTS AGE
pod/redis-master-0 1/1 Running 0 5m22s
pod/redis-replicas-0 1/1 Running 0 5m22s
pod/redis-replicas-1 1/1 Running 0 4m35s
pod/redis-replicas-2 1/1 Running 0 4m3s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis-headless ClusterIP None <none> 6379/TCP 5m22s
service/redis-master ClusterIP 172.31.12.114 <none> 6379/TCP 5m22s
service/redis-replicas ClusterIP 172.31.10.65 <none> 6379/TCP 5m22s
NAME READY AGE
statefulset.apps/redis-master 1/1 5m22s
statefulset.apps/redis-replicas 3/3 5m22s
NAME DATA AGE
configmap/kube-root-ca.crt 1 2d7h
configmap/redis-configuration 3 5m22s
configmap/redis-health 6 5m22s
configmap/redis-scripts 2 5m22s
NAME TYPE DATA AGE
secret/redis Opaque 1 5m22s
secret/sh.helm.release.v1.redis.v1 helm.sh/release.v1 1 5m22s可以看到pod都起来了，服务也正常，statefulset状态显示也正常。
然后可以按照前面的部署完成后，helm给出的提示。来试下连接redis。
[root@jingmin-kube-archlinux k8s]# export REDIS_PASSWORD=$(kubectl get secret --namespace redis redis -o jsonpath="{.data.redis-password}" | base64 -d)
[root@jingmin-kube-archlinux k8s]# echo $REDIS_PASSWORD
Redis12345可以看到密码，就是之前设置的密码。
在集群内创建个客户端pod，连上去看看
[root@jingmin-kube-archlinux k8s]# kubectl run --namespace redis redis-client --restart='Never' --env REDIS_PASSWORD=$REDIS_PASSWORD --image docker.io/bitnami/redis:7.2.0-debian-11-r0 --command -- sleep infinity
pod/redis-client created
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 6s
redis-master-0 1/1 Running 0 11m
redis-replicas-0 1/1 Running 0 11m
redis-replicas-1 1/1 Running 0 10m
redis-replicas-2 1/1 Running 0 10m
[root@jingmin-kube-archlinux k8s]# kubectl exec --tty -i redis-client --namespace redis -- bash
I have no name!@redis-client:/$ pwd
/
I have no name!@redis-client:/$ redis-cli -h redis-master
redis-master:6379> auth Redis12345
OK
redis-master:6379> keys *
(empty array)
redis-master:6379>
连到了master上，可以看到数据库是空的。
另起一个终端，也连上去，这次是连 从库replicas（按服务地址连接，连接会负载均衡到多个pod中）。
[root@jingmin-kube-archlinux k8s]# kubectl exec --tty -i redis-client --namespace redis -- bash
I have no name!@redis-client:/$ redis-cli -h redis-replicas
redis-replicas:6379> keys *
(error) NOAUTH Authentication required.
redis-replicas:6379> auth Redis12345
OK
redis-replicas:6379> keys *
(empty array)
redis-replicas:6379> set test_key1 test_value
(error) READONLY You can't write against a read only replica.
redis-replicas:6379> keys *
(empty array)
可以看到，从库也是空的。而且拒绝了写入操作。
切换连接到master的终端，试着写库
redis-master:6379> set test_key2 test_value2
OK
redis-master:6379> keys *
1) "test_key2"
redis-master:6379> get test_key2
"test_value2"
确认master可以读写
再切换回从库（另一个窗口终端）
redis-replicas:6379> keys *
1) "test_key2"
redis-replicas:6379> get test_key2
"test_value2"
确认获取到了主库写过的值。
再切回连接master的终端， 退出
redis-master:6379> exit
I have no name!@redis-client:/$ exit
exit
看下存储和存储请求
[root@jingmin-kube-archlinux k8s]# kubectl get pvc,pv
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/redis-data-redis-master-0 Bound pvc-17fb8a38-93d4-48d3-9814-c6e7a0978429 8Gi RWO nfs-storage 26m
persistentvolumeclaim/redis-data-redis-replicas-0 Bound pvc-a2b76be2-e866-4053-a4ba-67e5ee6ae69d 8Gi RWO nfs-storage 26m
persistentvolumeclaim/redis-data-redis-replicas-1 Bound pvc-b66cce97-05ec-4eac-b056-cde17370aa22 8Gi RWO nfs-storage 25m
persistentvolumeclaim/redis-data-redis-replicas-2 Bound pvc-0831b907-2cc3-4c9d-9a0e-54759e5e063a 8Gi RWO nfs-storage 24m
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-0831b907-2cc3-4c9d-9a0e-54759e5e063a 8Gi RWO Delete Bound redis/redis-data-redis-replicas-2 nfs-storage 24m
persistentvolume/pvc-17fb8a38-93d4-48d3-9814-c6e7a0978429 8Gi RWO Delete Bound redis/redis-data-redis-master-0 nfs-storage 26m
persistentvolume/pvc-a2b76be2-e866-4053-a4ba-67e5ee6ae69d 8Gi RWO Delete Bound redis/redis-data-redis-replicas-0 nfs-storage 26m
persistentvolume/pvc-b66cce97-05ec-4eac-b056-cde17370aa22 8Gi RWO Delete Bound redis/redis-data-redis-replicas-1 nfs-storage 25m
pv是cluster全局资源，这里只选择性展示了一部分。
试着删下master pod
[root@jingmin-kube-archlinux k8s]# kubectl delete pods redis-master-0
pod "redis-master-0" deleted
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 17m
redis-master-0 0/1 Running 0 6s
redis-replicas-0 1/1 Running 0 28m
redis-replicas-1 1/1 Running 0 28m
redis-replicas-2 1/1 Running 0 27m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 17m
redis-master-0 0/1 Running 0 13s
redis-replicas-0 1/1 Running 0 29m
redis-replicas-1 1/1 Running 0 28m
redis-replicas-2 1/1 Running 0 27m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 17m
redis-master-0 0/1 Running 0 20s
redis-replicas-0 1/1 Running 0 29m
redis-replicas-1 1/1 Running 0 28m
redis-replicas-2 1/1 Running 0 27m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 17m
redis-master-0 1/1 Running 0 22s
redis-replicas-0 0/1 Running 0 29m
redis-replicas-1 1/1 Running 0 28m
redis-replicas-2 1/1 Running 0 27m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 18m
redis-master-0 1/1 Running 0 31s
redis-replicas-0 0/1 Running 1 (10s ago) 29m
redis-replicas-1 0/1 Running 1 (8s ago) 28m
redis-replicas-2 0/1 Running 1 (6s ago) 28m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 18m
redis-master-0 1/1 Running 0 42s
redis-replicas-0 0/1 Running 1 (21s ago) 29m
redis-replicas-1 0/1 Running 1 (19s ago) 28m
redis-replicas-2 0/1 Running 1 (17s ago) 28m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 18m
redis-master-0 1/1 Running 0 44s
redis-replicas-0 0/1 Running 1 (23s ago) 29m
redis-replicas-1 0/1 Running 1 (21s ago) 28m
redis-replicas-2 0/1 Running 1 (19s ago) 28m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 18m
redis-master-0 1/1 Running 0 45s
redis-replicas-0 0/1 Running 1 (24s ago) 29m
redis-replicas-1 0/1 Running 1 (22s ago) 28m
redis-replicas-2 0/1 Running 1 (20s ago) 28m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 18m
redis-master-0 1/1 Running 0 47s
redis-replicas-0 1/1 Running 1 (26s ago) 29m
redis-replicas-1 0/1 Running 1 (24s ago) 28m
redis-replicas-2 0/1 Running 1 (22s ago) 28m
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 18m
redis-master-0 1/1 Running 0 49s
redis-replicas-0 1/1 Running 1 (28s ago) 29m
redis-replicas-1 1/1 Running 1 (26s ago) 28m
redis-replicas-2 1/1 Running 1 (24s ago) 28m
多次查询pods信息，可以看到，删除master之后，造成master和replicas实例的依次重启。
切换到另一个窗口（连接从库的那个窗口）
redis-replicas:6379> get test_key2
Error: Broken pipe
not connected>
redis-replicas:6379>
redis-replicas:6379> get test_key2
(error) NOAUTH Authentication required.
redis-replicas:6379> auth Redis12345
OK
redis-replicas:6379> get test_key2
"test_value2"
之前的连接断开了，redis-cli会自动重连。但需要重新验证。之前存储的内容也还在。
切换到上一个窗口，还是连接master库
[root@jingmin-kube-archlinux k8s]# kubectl exec --tty -i redis-client --namespace redis -- bash
I have no name!@redis-client:/$ redis-cli -h redis-master
redis-master:6379> auth Redis12345
OK
redis-master:6379> keys *
1) "test_key2"
redis-master:6379> get test_key2
"test_value2"
可以看到还是可以看到，删除pod之前的内容。看起来默认配置，redis使用了存储来保存数据的。
综上，集群内使用主从redis，是没有问题的。
在本地连接上k8s
PS C:\users\wangjm\.kube> kubectl port-forward --namespace redis svc/redis-master 6379:6379在本地使用redis的GUI客户端（第三方客户端），连接127.0.0.1:6379. 可以看到k8s中的数据。
综上，集群外连接主从redis ，也是没问题的。（仅调试使用）
参考： https://artifacthub.io/packages/helm/bitnami/redis#external-dns
但是如果生产环境要在集群外使用redis， 需要进一步的配置。暂未涉及，略。
测试部署启用sentinel的主从模式
与上一节类似。
（后来发现有错别字，实践过程中，我在很多地方把sentinel误写成了sentinal，连部署内容也是。）
先单独建个命名空间，设为当前默认命名空间，试下吧
kubectl create ns redis-sentinal
kubectl config set-context --current --namespace redis-sentinal使用oci兼容的helm仓库镜像，
参考： https://helm.sh/docs/topics/registries/
参考： https://blog.bitnami.com/2023/01/bitnami-helm-charts-available-as-oci.html
参考： https://github.com/bitnami/charts/tree/main/bitnami/redis
下载下redis的helm配置，看下（此步骤略过）
helm pull oci://registry-1.docker.io/bitnamicharts/redis --untar
cd redis
因为使用的是同一个原始chart，这里选择复制
cp -rdp ./redis ./redis_sentinal
cd redis_sentinal/
调整一下需要覆盖的配置(修改./my-override-values.yaml，仅保留需要覆盖的配置)
vimdiff ./my-override-values.yaml ./values.yaml如下是我需要覆盖调整的配置：
[root@jingmin-kube-archlinux redis_sentinal]# vimdiff ./my-override-values.yaml ./values.yaml
2 files to edit
[root@jingmin-kube-archlinux redis_sentinal]# vim my-override-values.yaml
[root@jingmin-kube-archlinux redis_sentinal]# cat my-override-values.yaml
global:
storageClass: "nfs-storage"
redis:
password: "Redis12345"
## @param architecture Redis® architecture. Allowed values: `standalone` or `replication`
##
architecture: replication
## @param commonConfiguration [string] Common configuration to be added into the ConfigMap
## ref: https://redis.io/topics/config
##
commonConfiguration: |-
# Enable AOF https://redis.io/topics/persistence#append-only-file
appendonly yes
# Disable RDB persistence, AOF persistence already enabled.
save ""
## @param existingConfigmap The name of an existing ConfigMap with your custom configuration for Redis® nodes
##
existingConfigmap: ""
## @section Redis® master configuration parameters
##
master:
## @param master.configuration Configuration for Redis® master nodes
## ref: https://redis.io/topics/config
##
configuration: ""
## @param master.disableCommands Array with Redis® commands to disable on master nodes
## Commands will be completely disabled by renaming each to an empty string.
## ref: https://redis.io/topics/security#disabling-of-specific-commands
##
disableCommands:
- FLUSHDB
- FLUSHALL
## Redis® master service parameters
##
service:
## @param master.service.type Redis® master service type
##
type: ClusterIP
## @param master.service.ports.redis Redis® master service port
##
ports:
redis: 6379
## @param master.service.nodePorts.redis Node port for Redis® master
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
## NOTE: choose port between <30000-32767>
##
nodePorts:
redis: ""
## @section Redis® replicas configuration parameters
##
replica:
## @param replica.replicaCount Number of Redis® replicas to deploy
##
replicaCount: 3
## @param replica.configuration Configuration for Redis® replicas nodes
## ref: https://redis.io/topics/config
##
configuration: ""
## @param replica.disableCommands Array with Redis® commands to disable on replicas nodes
## Commands will be completely disabled by renaming each to an empty string.
## ref: https://redis.io/topics/security#disabling-of-specific-commands
##
disableCommands:
- FLUSHDB
- FLUSHALL
## Redis® replicas service parameters
##
service:
## @param replica.service.type Redis® replicas service type
##
type: ClusterIP
## @param replica.service.ports.redis Redis® replicas service port
##
ports:
redis: 6379
## @param replica.service.nodePorts.redis Node port for Redis® replicas
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
## NOTE: choose port between <30000-32767>
##
nodePorts:
redis: ""
## Autoscaling configuration
##
autoscaling:
## @param replica.autoscaling.enabled Enable replica autoscaling settings
##
enabled: false
## ServiceAccount configuration
##
## @section Redis® Sentinel configuration parameters
##
sentinel:
## @param sentinel.enabled Use Redis® Sentinel on Redis® pods.
## IMPORTANT: this will disable the master and replicas services and
## create a single Redis® service exposing both the Redis and Sentinel ports
##
enabled: true
persistence:
## @param sentinel.persistence.enabled Enable persistence on Redis® sentinel nodes using Persistent Volume Claims (Experimental)
##
enabled: false
相比前面简单主从模式的配置，这里只是启用了sentinal。
sentinal.persistence.enabled: false 没有调整，还是使用默认值。只是看起来比较奇怪，单独列出来看下。
部署安装带sentinal的redis
[root@jingmin-kube-archlinux redis_sentinal]# cd ..
[root@jingmin-kube-archlinux k8s]# helm install redis-sentinal -f ./redis_sentinal/my-override-values.yaml ./redis_sentinal/
NAME: redis-sentinal
LAST DEPLOYED: Fri Sep 1 18:10:29 2023
NAMESPACE: redis-sentinal
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 18.0.1
APP VERSION: 7.2.0
** Please be patient while the chart is being deployed **
Redis® can be accessed via port 6379 on the following DNS name from within your cluster:
redis-sentinal.redis-sentinal.svc.cluster.local for read only operations
For read/write operations, first access the Redis® Sentinel cluster, which is available in port 26379 using the same domain name above.
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace redis-sentinal redis-sentinal -o jsonpath="{.data.redis-password}" | base64 -d)
To connect to your Redis® server:
1. Run a Redis® pod that you can use as a client:
kubectl run --namespace redis-sentinal redis-client --restart='Never' --env REDIS_PASSWORD=$REDIS_PASSWORD --image docker.io/bitnami/redis:7.2.0-debian-11-r0 --command -- sleep infinity
Use the following command to attach to the pod:
kubectl exec --tty -i redis-client \
--namespace redis-sentinal -- bash
2. Connect using the Redis® CLI:
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinal -p 6379 # Read only operations
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinal -p 26379 # Sentinel access
To connect to your database from outside the cluster execute the following commands:
kubectl port-forward --namespace redis-sentinal svc/redis-sentinal 6379:6379 &
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379也是有一些集群内外连接redis的方法提示。
看下有没有部署成功
[root@jingmin-kube-archlinux k8s]# kubectl get all,cm,secrets,pvc
NAME READY STATUS RESTARTS AGE
pod/redis-sentinal-node-0 2/2 Running 0 3m18s
pod/redis-sentinal-node-1 2/2 Running 0 2m35s
pod/redis-sentinal-node-2 2/2 Running 0 118s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis-sentinal ClusterIP 172.31.10.171 <none> 6379/TCP,26379/TCP 3m18s
service/redis-sentinal-headless ClusterIP None <none> 6379/TCP,26379/TCP 3m18s
NAME READY AGE
statefulset.apps/redis-sentinal-node 3/3 3m18s
NAME DATA AGE
configmap/kube-root-ca.crt 1 19m
configmap/redis-sentinal-configuration 4 3m18s
configmap/redis-sentinal-health 8 3m18s
configmap/redis-sentinal-scripts 4 3m18s
NAME TYPE DATA AGE
secret/redis-sentinal Opaque 1 3m18s
secret/sh.helm.release.v1.redis-sentinal.v1 helm.sh/release.v1 1 3m18s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/redis-data-redis-sentinal-node-0 Bound pvc-ad4791fb-952a-46b6-8989-a3805b05236b 8Gi RWO nfs-storage 3m18s
persistentvolumeclaim/redis-data-redis-sentinal-node-1 Bound pvc-b45dc0d0-5262-4ede-afa0-e3984c6f28ff 8Gi RWO nfs-storage 2m35s
persistentvolumeclaim/redis-data-redis-sentinal-node-2 Bound pvc-51c0d7b2-72f7-4eb3-854d-fdce46b3156a 8Gi RWO nfs-storage 118s
看到这里只有一套statefulset，pod没有分主从。
按照部署完成时的提示，试下集群内连接redis.
先看下redis密码
[root@jingmin-kube-archlinux k8s]# export REDIS_PASSWORD=$(kubectl get secret --namespace redis-sentinal redis-sentinal -o jsonpath="{.data.redis-password}" | base64 -d)
[root@jingmin-kube-archlinux k8s]# echo $REDIS_PASSWORD
Redis12345
与我们前面调整的配置文件中的密码一致。
先在集群内创建个用于连接的客户端pod
[root@jingmin-kube-archlinux k8s]# kubectl run --namespace redis-sentinal redis-client --restart='Never' --env REDIS_PASSWORD=$REDIS_PASSWORD --image docker.io/bitnami/redis:7.2.0-debian-11-r0 --command -- sleep infinity
pod/redis-client created
[root@jingmin-kube-archlinux k8s]# kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-client 1/1 Running 0 9s
redis-sentinal-node-0 2/2 Running 0 10m
redis-sentinal-node-1 2/2 Running 0 9m48s
redis-sentinal-node-2 2/2 Running 0 9m11s
连接到6379端口
[root@jingmin-kube-archlinux k8s]# kubectl exec --tty -i redis-client --namespace redis-sentinal -- bash
I have no name!@redis-client:/$ REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinal -p 6379
redis-sentinal:6379> keys *
(empty array)
redis-sentinal:6379> set test_key3 test_value3
(error) READONLY You can't write against a read only replica.
exit
exit仅支持读操作。
连接到26379端口
[root@jingmin-kube-archlinux ~]# kubectl exec --tty -i redis-client --namespace redis-sentinal -- bash
I have no name!@redis-client:/$ REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinal -p 26379
redis-sentinal:26379> info
...
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_tilt_since_seconds:-1
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379,slaves=2,sentinels=3
redis-sentinal:26379> sentinel masters
1) 1) "name"
2) "mymaster"
3) "ip"
4) "redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local"
5) "port"
6) "6379"
...
redis-sentinal:26379> sentinel master mymaster
1) "name"
2) "mymaster"
3) "ip"
4) "redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local"
5) "port"
6) "6379"
...
redis-sentinal:26379> sentinel slaves mymaster
1) 1) "name"
2) "redis-sentinal-node-1.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379"
3) "ip"
4) "redis-sentinal-node-1.redis-sentinal-headless.redis-sentinal.svc.cluster.local"
5) "port"
6) "6379"
...
2) 1) "name"
2) "redis-sentinal-node-2.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379"
3) "ip"
4) "redis-sentinal-node-2.redis-sentinal-headless.redis-sentinal.svc.cluster.local"
5) "port"
6) "6379"
...
exit
1、redis基本命令 1）获取sentinel的状态 (1）info 查看sentinel的状态 (2）sentinel masters 获取sentinel中监控的所有master的节点 (3）sentinel master
获取master-name节点redis的状态信息 (4）sentinel slaves 获取master-name节点下所有的slaves的状态信息。 (5) SENTINEL get-master-addr-by-name 通过sentinel中的节点名获取其ip地址 参考： https://www.modb.pro/db/622365
从上面sentinel查询到master节点和slave节点的信息。
试着连下master节点
I have no name!@redis-client:/$ REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local -p 6379
redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> keys *
(empty array)
redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> set test_key4 test_value4
OK
redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> keys *
1) "test_key4"
redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> get test_key4
"test_value4"
redis-sentinal-node-0.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> exit连接成功，读写成功。
试着连接其中一个slave节点
I have no name!@redis-client:/$ REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinal-node-1.redis-sentinal-headless.redis-sentinal.svc.cluster.local -p 6379
redis-sentinal-node-1.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> keys *
1) "test_key4"
redis-sentinal-node-1.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> get test_key4
"test_value4"
redis-sentinal-node-1.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> set test_key5 test_value5
(error) READONLY You can't write against a read only replica.
redis-sentinal-node-1.redis-sentinal-headless.redis-sentinal.svc.cluster.local:6379> exit
exit连接成功，读成功，写失败。
综上，集群内使用sentinel-redis集群，是没有问题的。
再试下集群外连接(本地电脑上，做好k8s连接配置)
kubectl port-forward --namespace redis-sentinal svc/redis-sentinal 6379:6379试了下，在集群外是可以读写的。
？？？集群内 此服务端口6379只允许读，集群外port-forward后反而能读写？？？
默认做了网络策略还是什么?
测试部署redis-cluster
与上面两个部署基于主从模式及sentinel不同，这里使用redis cluster模式（当然使用的chart也不同）.
参考： https://artifacthub.io/packages/helm/bitnami/redis-cluster#configuration-and-installation-details
参考： https://artifacthub.io/packages/helm/bitnami/redis-cluster#cluster-topology
Cluster topology 集群拓扑
To successfully set the cluster up, it will need to have at least 3 master nodes. The total number of nodes is calculated like-
nodes = numOfMasterNodes + numOfMasterNodes * replicas. Hence, the defaultscluster.nodes = 6andcluster.replicas = 1means, 3 master and 3 replica nodes will be deployed by the chart. 要成功设置集群，至少需要 3 个主节点。节点总数的计算方式如下：nodes = numOfMasterNodes + numOfMasterNodes * replicas。因此，默认的cluster.nodes = 6和cluster.replicas = 1意味着图表将部署 3 个主节点和 3 个副本节点。By default the Redis® Cluster is not accessible from outside the Kubernetes cluster, to access the Redis® Cluster from outside you have to set
cluster.externalAccess.enabled=trueat deployment time. It will create in the first installation only 6 LoadBalancer services, one for each Redis® node, once you have the external IPs of each service you will need to perform an upgrade passing those IPs to thecluster.externalAccess.service.loadbalancerIParray. 默认情况下，无法从 Kubernetes 集群外部访问 Redis® 集群，要从外部访问 Redis® 集群，您必须在部署时设置cluster.externalAccess.enabled=true。它将在第一次安装中仅创建 6 个 LoadBalancer 服务，每个 Redis® 节点一个，一旦您拥有每个服务的外部 IP，您将需要执行升级，将这些 IP 传递到cluster.externalAccess.service.loadbalancerIP阵列。The replicas will be read-only replicas of the masters. By default only one service is exposed (when not using the external access mode). You will connect your client to the exposed service, regardless you need to read or write. 副本将是主服务器的只读副本。默认情况下只暴露一项服务（不使用外部访问模式时）。无论您需要读取还是写入，您都将把您的客户端连接到公开的服务。 When a write operation arrives to a replica it will redirect the client to the proper master node. For example, using
redis-cliyou will need to provide the-cflag forredis-clito follow the redirection automatically. 当写操作到达副本时，它将把客户端重定向到正确的主节点。例如，使用redis-cli您需要为redis-cli提供-c标志才能自动遵循重定向。Using the external access mode, you can connect to any of the pods and the slaves will redirect the client in the same way as explained before, but the all the IPs will be public. 使用外部访问模式，您可以连接到任何 Pod，并且从属设备将以与之前解释的方式相同的方式重定向客户端，但所有 IP 都将是公共的。
In case the master crashes, one of his slaves will be promoted to master. The slots stored by the crashed master will be unavailable until the slave finish the promotion. If a master and all his slaves crash, the cluster will be down until one of them is up again. 万一主人崩溃了，他的一个奴隶就会被提升为主人。崩溃的主站存储的槽位将不可用，直到从站完成升级为止。如果一个主节点和他的所有从节点都崩溃了，集群就会宕机，直到其中一个节点重新启动。 To avoid downtime, it is possible to configure the number of Redis® nodes with
cluster.nodesand the number of replicas that will be assigned to each master withcluster.replicas. For example: 为了避免停机，可以使用cluster.nodes配置 Redis® 节点的数量，并使用cluster.replicas配置分配给每个主服务器的副本数量。例如：
cluster.nodes=9( 3 master plus 2 replicas for each master)cluster.nodes=9（3 个主控加上每个主控 2 个副本）cluster.replicas=2Providing the values above, the cluster will have 3 masters and, each master, will have 2 replicas. 提供上述值，集群将有 3 个主节点，每个主节点将有 2 个副本。
看后面的升级说明，发现redis-cluster的缩扩容，还是需要很多手工操作，和其他人手动创建集群也差不多了。
但是初次部署启动倒是挺方便的。
还是先单独建个命名空间，设为当前默认命名空间，试下吧
kubectl create ns redis-cluster
kubectl config set-context --current --namespace redis-cluster先下载下redis-cluster的helm配置，看下
helm pull oci://registry-1.docker.io/bitnamicharts/redis-cluster --untar
cd redis-cluster
复制默认配置作为模板，复制，编辑新文件，保留需要调整的内容
cp ./values.yaml ./my-override-values.yaml
vim ./my-override-values.yaml 如下是我需要覆盖调整的配置：
[root@jingmin-kube-archlinux redis-cluster]# cat my-override-values.yaml
global:
storageClass: "nfs-storage"
redis:
password: "Redis12345"
## Redis® Service properties for standalone mode.
##
service:
## @param service.ports.redis Kubernetes Redis service port
##
ports:
redis: 6379
## Node ports to expose
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
## @param service.nodePorts.redis Node port for Redis
##
nodePorts:
redis: ""
## @param service.type Service type for default redis service
## Setting this to LoadBalancer may require corresponding service annotations for loadbalancer creation to succeed.
## Currently supported types are ClusterIP (default) and LoadBalancer
##
type: ClusterIP
## Enable persistence using Persistent Volume Claims
## ref: https://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistence:
## @param persistence.enabled Enable persistence on Redis®
## If enabled, nodes are using Persistent Volume Claims
## If disabled, an emptyDir volume is used. This is not recommended.
## ref: https://github.com/bitnami/charts/tree/main/bitnami/redis-cluster#persistence
##
enabled: true
## @param persistence.path Path to mount the volume at, to use other images Redis® images.
##
path: /bitnami/redis/data
## @param persistence.subPath The subdirectory of the volume to mount to, useful in dev environments and one PV for multiple services
##
subPath: ""
## @section Redis® statefulset parameters
##
redis:
## @param redis.containerPorts.redis Redis® port
## @param redis.containerPorts.bus The busPort should be obtained adding 10000 to the redisPort. By default: 10000 + 6379 = 16379
##
containerPorts:
redis: 6379
bus: 16379
## @section Cluster management parameters
##
## Redis® Cluster settings
##
cluster:
## @param cluster.init Enable the initialization of the Redis® Cluster
##
init: true
## Number of Redis® nodes to be deployed
##
## Note:
## This is total number of nodes including the replicas. Meaning there will be 3 master and 3 replica
## nodes (as replica count is set to 1 by default, there will be 1 replica per master node).
## Hence, nodes = numberOfMasterNodes + numberOfMasterNodes * replicas
##
## @param cluster.nodes The number of master nodes should always be >= 3, otherwise cluster creation will fail
##
nodes: 6
## @param cluster.replicas Number of replicas for every master in the cluster
## Parameter to be passed as --cluster-replicas to the redis-cli --cluster create
## 1 means that we want a replica for every master created
##
replicas: 1
## Configuration to access the Redis® Cluster from outside the Kubernetes cluster
##
externalAccess:
## @param cluster.externalAccess.enabled Enable access to the Redis
##
enabled: false
## This section allows to update the Redis® cluster nodes.
##
update:
## @param cluster.update.addNodes Boolean to specify if you want to add nodes after the upgrade
## Setting this to true a hook will add nodes to the Redis® cluster after the upgrade. currentNumberOfNodes and currentNumberOfReplicas is required
##
addNodes: false
## @param cluster.update.currentNumberOfNodes Number of currently deployed Redis® nodes
##
currentNumberOfNodes: 6
## @param cluster.update.currentNumberOfReplicas Number of currently deployed Redis® replicas
##
currentNumberOfReplicas: 1
## @param cluster.update.newExternalIPs External IPs obtained from the services for the new nodes to add to the cluster
##
newExternalIPs: []
其实这里只调整了全局使用的storageClass和密码。
其他内容，遵从默认。这里列出，只是感觉比较重要。
使用默认的配置，6个nodes, 而replicas为1（replicas为1表示每个master节点有1个replicas，换算一下即3主3从）。
安装部署
[root@jingmin-kube-archlinux redis-cluster]# cd ..
[root@jingmin-kube-archlinux k8s]# helm install redis-cluster -f ./redis-cluster/my-override-values.yaml ./redis-cluster/
NAME: redis-cluster
LAST DEPLOYED: Fri Sep 1 23:49:47 2023
NAMESPACE: redis-cluster
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis-cluster
CHART VERSION: 9.0.1
APP VERSION: 7.2.0** Please be patient while the chart is being deployed **
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace "redis-cluster" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)
You have deployed a Redis® Cluster accessible only from within you Kubernetes Cluster.INFO: The Job to create the cluster will be created.To connect to your Redis® cluster:
1. Run a Redis® pod that you can use as a client:
kubectl run --namespace redis-cluster redis-cluster-client --rm --tty -i --restart='Never' \
--env REDIS_PASSWORD=$REDIS_PASSWORD \
--image docker.io/bitnami/redis-cluster:7.2.0-debian-11-r0 -- bash
2. Connect using the Redis® CLI:
redis-cli -c -h redis-cluster -a $REDIS_PASSWORD
上面提示，实际的创建过程可能会比较慢。
然后给了一些集群内连接redis-cluster的小提示。
看下集群是否起来了,主要是看下pods的状态是不是都是running,是不是都已经ready了。
[root@jingmin-kube-archlinux k8s]# kubectl get all,pvc,cm,secrets
NAME READY STATUS RESTARTS AGE
pod/redis-cluster-0 1/1 Running 0 2m51s
pod/redis-cluster-1 1/1 Running 1 (89s ago) 2m51s
pod/redis-cluster-2 1/1 Running 0 2m51s
pod/redis-cluster-3 1/1 Running 1 (89s ago) 2m51s
pod/redis-cluster-4 1/1 Running 1 (84s ago) 2m51s
pod/redis-cluster-5 1/1 Running 0 2m51s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis-cluster ClusterIP 172.31.7.126 <none> 6379/TCP 2m51s
service/redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 2m51s
NAME READY AGE
statefulset.apps/redis-cluster 6/6 2m51s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/redis-data-redis-cluster-0 Bound pvc-595b70d5-a3b8-4fec-84a1-b00cb27a33a0 8Gi RWO nfs-storage 2m51s
persistentvolumeclaim/redis-data-redis-cluster-1 Bound pvc-f3202c6a-a513-438b-b6a7-ae98264efc46 8Gi RWO nfs-storage 2m51s
persistentvolumeclaim/redis-data-redis-cluster-2 Bound pvc-ed81dd2e-5ac3-4b33-8114-d516263472b1 8Gi RWO nfs-storage 2m51s
persistentvolumeclaim/redis-data-redis-cluster-3 Bound pvc-e6319f74-55dd-4fa2-b608-12e5842078da 8Gi RWO nfs-storage 2m51s
persistentvolumeclaim/redis-data-redis-cluster-4 Bound pvc-8796b2c0-6d7d-4bc7-84c3-291a43cfc3c1 8Gi RWO nfs-storage 2m51s
persistentvolumeclaim/redis-data-redis-cluster-5 Bound pvc-3b59bfbe-d4bf-45f5-9f23-2c6bc3533388 8Gi RWO nfs-storage 2m51s
NAME DATA AGE
configmap/kube-root-ca.crt 1 3m37s
configmap/redis-cluster-default 1 2m51s
configmap/redis-cluster-scripts 2 2m51s
NAME TYPE DATA AGE
secret/redis-cluster Opaque 1 2m51s
secret/sh.helm.release.v1.redis-cluster.v1 helm.sh/release.v1 1 2m51s
可以看到，pod都已经起来了， statefulset只有一个，pvc也都bound好了，其他资源看起来也都正常。
看下密码
[root@jingmin-kube-archlinux k8s]# export REDIS_PASSWORD=$(kubectl get secret --namespace "redis-cluster" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)
[root@jingmin-kube-archlinux k8s]# echo $REDIS_PASSWORD
Redis12345
密码确实是前面设置的密码。
试着在集群内连接下redis-cluster
先在集群里创建个客户端pod,用于连接redis-cluster
[root@jingmin-kube-archlinux k8s]# kubectl run --namespace redis-cluster redis-cluster-client --rm --tty -i --restart='Never' \
--env REDIS_PASSWORD=$REDIS_PASSWORD \
--image docker.io/bitnami/redis-cluster:7.2.0-debian-11-r0 -- bash
分配了tty，并进入bash shell. 直接就是pod里的shell界面。
在这个shell里连接下集群中的redis-cluster, 注意这里是集群模式，要加参数-c
redis-cli -c -h redis-cluster -a $REDIS_PASSWORD做一些读写测试
I have no name!@redis-cluster-client:/$ redis-cli -c -h redis-cluster -a $REDIS_PASSWORD
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
redis-cluster:6379> keys *
(empty array)
redis-cluster:6379> set test_key10 test_value10
-> Redirected to slot [9166] located at 172.30.0.75:6379
OK
172.30.0.75:6379> get test_key10
"test_value10"
172.30.0.75:6379> set test_key11 test_value11
-> Redirected to slot [13295] located at 172.30.1.170:6379
OK
172.30.1.170:6379> keys *
1) "test_key11"
172.30.1.170:6379> keys \*
(empty array)
172.30.1.170:6379> exit
I have no name!@redis-cluster-client:/$ redis-cli -c -h redis-cluster -a $REDIS_PASSWORD --cluster call 172.30.0.75:6379 keys \*
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Calling keys *
172.30.0.75:6379: test_key10
172.30.1.172:6379:
172.30.0.77:6379:
172.30.1.171:6379: test_key10
172.30.0.76:6379: test_key11
172.30.1.170:6379: test_key11redis集群模式，好像是使用分片存储（哈希槽）。
一个redis-cluster有固定的slot个数，分配给不同的master, 允许读写。而slave则复制master的slot中的内容，仅读和转发写，并用作master异常时的故障转移。
客户端可以连接到任意的master或slave节点。在对key操作时，每个key都计算下hash,对应到相应的slot. 如果是在本节点上的槽，而且是读操作，直接读即可。非本节点的slot,做转发。 如果本节点是slave, 但是是写操作，转发给对应的master.
这里还注意到，集群模式，keys 命令只能看到本节点上的数据。如果要看所有节点的数据， 需要连接时加--cluster call <ip>:<port> keys \*
综上，在k8s集群内访问redis-cluster，没有问题。
试下外部访问（仅调试， 使用端口转发）
kubectl port-forward svc/redis-cluster 6379:6379新打开个窗口，连接下本地的6379端口
[wangjm@jingmin-kube-archlinux ~]$ redis-cli -h 127.0.0.1
127.0.0.1:6379> keys *
(error) NOAUTH Authentication required.
127.0.0.1:6379> auth Redis12345
OK
127.0.0.1:6379> keys *
1) "test_key10"
127.0.0.1:6379> exit
[wangjm@jingmin-kube-archlinux ~]$ redis-cli -h 127.0.0.1 -a Redis12345 -c --cluster call 127.0.0.1:6379 keys \*
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Calling keys *
127.0.0.1:6379: test_key10
172.30.0.75:6379: test_key10
172.30.0.77:6379:
172.30.1.170:6379: test_key11
172.30.1.172:6379:
172.30.0.76:6379: test_key11综上，在k8s集群外访问redis-cluster，没有问题。
发表回复