ole的知识小站

rabbitmq安装

—

由

wangjm
于

RabbitMQ安装

参考： https://www.rabbitmq.com/documentation.html

参考： https://www.rabbitmq.com/distributed.html

参考： https://www.rabbitmq.com/clustering.html

参考： https://www.rabbitmq.com/ha.html

参考： https://www.rabbitmq.com/quorum-queues.html

参考： https://www.rabbitmq.com/kubernetes/operator/operator-overview.html

介绍

很多中文资料，尤其是生产级环境下的说明。主要说明了下

  • rabbitmq的基本结构和概念（producer，broker，consumer， exchange，queue，binding， routingKey），
  • 基本集群（队列消息非高可用），
  • 以及镜像队列集群（队列消息也高可用）。

实际上，根据官网文档。3.7、3.8之后的rabbitmq，

  • 引入了Streams（消息类似于日志，持久化存盘，允许消息重放）、
  • 引入了Quorum Queue（仲裁队列，基于raft多数共识协议，优化主从队列复制，队列故障转移不会导致消息重发，从节点增量同步，强调安全，持久队列）
  • classic queue在不强调安全的情况下继续可用。而 classic queue mirroring （镜像队列，或者说经典队列的副本）变为deprecated，4.0+之后将会删除。

选型

rabbitmq在k8s中的使用，可以考虑以下几个：

  • artifacthub.io找star最多的helm包，不出意外是bitnami的rabbitmq的chart，https://artifacthub.io/packages/helm/bitnami/rabbitmq
  • rabbitmq官方提到的operator，https://www.rabbitmq.com/kubernetes/operator/install-operator.html
  • bitnami的rabbitmq-cluster-operator, 看起来是对官网 operator 的打包, https://artifacthub.io/packages/helm/bitnami/rabbitmq-cluster-operator
  • 网上其他人做的yaml

先按官网的operator 方式试试,有时间再试试bitnami的rabbitmq和rabbitmq operator

部署安装

官网rabbitmq-cluster-operator安装

参考: https://www.rabbitmq.com/kubernetes/operator/operator-overview.html

参考: https://www.rabbitmq.com/kubernetes/operator/install-operator.html

参考: https://www.rabbitmq.com/kubernetes/operator/quickstart-operator.html

参考： https://www.rabbitmq.com/kubernetes/operator/using-operator.html

官网提供了两个Operator， 这里暂时只试用了下 RabbitMQ Cluster Kubernetes Operator

参考： https://www.rabbitmq.com/kubernetes/operator/operator-overview.html

The RabbitMQ team develop and maintain two kubernetes operators: the RabbitMQ Cluster Kubernetes Operator and the RabbitMQ Messaging Topology Operator. RabbitMQ 团队开发和维护两个 kubernetes Operator：RabbitMQ Cluster Kubernetes Operator 和 RabbitMQ Messaging Topology Operator。

  • The RabbitMQ Cluster Kubernetes Operator automates provisioning, management, and operations of RabbitMQ clusters running on Kubernetes. RabbitMQ Cluster Kubernetes Operator 可自动配置、管理和操作在 Kubernetes 上运行的 RabbitMQ 集群。
  • The RabbitMQ Messaging Topology Operator manages RabbitMQ messaging topologies within a RabbitMQ cluster deployed via the RabbitMQ Cluster Kubernetes Operator. RabbitMQ Messaging Topology Operator 管理通过 RabbitMQ Cluster Kubernetes Operator 部署的 RabbitMQ 集群内的 RabbitMQ 消息拓扑。

Kubernetes Operators are software extensions to Kubernetes that provide custom resources for management of applications, services and their components. Kubernetes Operator 是 Kubernetes 的软件扩展，提供用于管理应用程序、服务及其组件的自定义资源。

切回default命名空间操作先

kubectl config set-context --current --namespace default

下载并执行crd以及cluster级rbac

mkdir rabbitmq-official
cd rabbitmq-official/
wget https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
kubectl apply -f ./cluster-operator.yml

然后会发现多了个命名空间

[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get ns
NAME                 STATUS   AGE
...
rabbitmq-system      Active   84s
...

然后是这样的

[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get all,cm,secrets,crd,cr -n rabbitmq-system 
NAME                                            READY   STATUS    RESTARTS   AGE
pod/rabbitmq-cluster-operator-d74f89f55-p7648   1/1     Running   0          2m48s

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/rabbitmq-cluster-operator   1/1     1            1           2m48s

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/rabbitmq-cluster-operator-d74f89f55   1         1         1       2m48s

NAME                         DATA   AGE
configmap/kube-root-ca.crt   1      2m48s

NAME                                                                                CREATED AT
...
customresourcedefinition.apiextensions.k8s.io/rabbitmqclusters.rabbitmq.com         2023-09-10T14:53:53Z

可以安装下kubectl-rabbitmq插件

参考： https://www.rabbitmq.com/kubernetes/operator/kubectl-plugin.html

kubectl krew install rabbitmq
kubectl krew install tail
kubectl rabbitmq help

然后可以试下部署hello-world例子

参考: https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/hello-world

[root@jingmin-kube-archlinux rabbitmq-official]# kubectl create ns rabbitmq-official
namespace/rabbitmq-official created
[root@jingmin-kube-archlinux rabbitmq-official]# kubectl config set-context --current --namespace rabbitmq-official
Context "kubernetes-admin@kubernetes" modified.
[root@jingmin-kube-archlinux rabbitmq-official]# vim hello-world.yaml
[root@jingmin-kube-archlinux rabbitmq-official]# cat hello-world.yaml 
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
    name: hello-world
[root@jingmin-kube-archlinux rabbitmq-official]# kubectl apply -f ./hello-world.yaml 
rabbitmqcluster.rabbitmq.com/hello-world created

简单看下都有哪些东西

[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get all
NAME                       READY   STATUS    RESTARTS   AGE
pod/hello-world-server-0   1/1     Running   0          4m14s

NAME                        TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                        AGE
service/hello-world         ClusterIP   172.31.6.38   <none>        15672/TCP,15692/TCP,5672/TCP   4m15s
service/hello-world-nodes   ClusterIP   None          <none>        4369/TCP,25672/TCP             4m15s

NAME                                  READY   AGE
statefulset.apps/hello-world-server   1/1     4m15s

NAME                                       ALLREPLICASREADY   RECONCILESUCCESS   AGE
rabbitmqcluster.rabbitmq.com/hello-world   True               True               4m15s

看下具体自定义资源配置

[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get rabbitmqclusters.rabbitmq.com hello-world -o yaml
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"rabbitmq.com/v1beta1","kind":"RabbitmqCluster","metadata":{"annotations":{},"name":"hello-world","namespace":"rabbitmq-official"}}
  creationTimestamp: "2023-09-10T15:09:32Z"
  finalizers:
  - deletion.finalizers.rabbitmqclusters.rabbitmq.com
  generation: 3
  name: hello-world
  namespace: rabbitmq-official
  resourceVersion: "3906034"
  uid: 303e2a09-85ba-4333-b8e1-d08538192541
spec:
  delayStartSeconds: 30
  image: rabbitmq:3.12.2-management
  override: {}
  persistence:
    storage: 10Gi
  rabbitmq: {}
  replicas: 1
  resources:
    limits:
      cpu: "2"
      memory: 2Gi
    requests:
      cpu: "1"
      memory: 2Gi
  secretBackend:
    externalSecret: {}
  service:
    type: ClusterIP
  terminationGracePeriodSeconds: 604800
  tls: {}
status:
  binding:
    name: hello-world-default-user
  conditions:
  - lastTransitionTime: "2023-09-10T15:10:25Z"
    reason: AllPodsAreReady
    status: "True"
    type: AllReplicasReady
  - lastTransitionTime: "2023-09-10T15:10:25Z"
    reason: AtLeastOneEndpointAvailable
    status: "True"
    type: ClusterAvailable
  - lastTransitionTime: "2023-09-10T15:09:33Z"
    reason: NoWarnings
    status: "True"
    type: NoWarnings
  - lastTransitionTime: "2023-09-10T15:10:32Z"
    message: Finish reconciling
    reason: Success
    status: "True"
    type: ReconcileSuccess
  defaultUser:
    secretReference:
      keys:
        password: password
        username: username
      name: hello-world-default-user
      namespace: rabbitmq-official
    serviceReference:
      name: hello-world
      namespace: rabbitmq-official
  observedGeneration: 3

试着调整下replicas. (>=3, 避开2)

[root@jingmin-kube-archlinux rabbitmq-official]# kubectl edit rabbitmqclusters.rabbitmq.com hello-world 
rabbitmqcluster.rabbitmq.com/hello-world edited
[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get rabbitmqclusters.rabbitmq.com hello-world 
NAME          ALLREPLICASREADY   RECONCILESUCCESS   AGE
hello-world   False              True               9m9s
[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get rabbitmqclusters.rabbitmq.com hello-world 
NAME          ALLREPLICASREADY   RECONCILESUCCESS   AGE
hello-world   True               True               9m48s

NAME                                       ALLREPLICASREADY   RECONCILESUCCESS   AGE
rabbitmqcluster.rabbitmq.com/hello-world   False              True               12m
[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get all
NAME                       READY   STATUS    RESTARTS   AGE
pod/hello-world-server-0   1/1     Running   0          60s
pod/hello-world-server-1   1/1     Running   0          2m19s
pod/hello-world-server-2   1/1     Running   0          3m39s

NAME                        TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                        AGE
service/hello-world         ClusterIP   172.31.6.38   <none>        15672/TCP,15692/TCP,5672/TCP   12m
service/hello-world-nodes   ClusterIP   None          <none>        4369/TCP,25672/TCP             12m

NAME                                  READY   AGE
statefulset.apps/hello-world-server   3/3     12m

NAME                                       ALLREPLICASREADY   RECONCILESUCCESS   AGE
rabbitmqcluster.rabbitmq.com/hello-world   True               True               12m


[root@jingmin-kube-archlinux rabbitmq-official]# kubectl get rabbitmqclusters.rabbitmq.com hello-world -o yaml
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"rabbitmq.com/v1beta1","kind":"RabbitmqCluster","metadata":{"annotations":{},"name":"hello-world","namespace":"rabbitmq-official"}}
  creationTimestamp: "2023-09-10T15:09:32Z"
  finalizers:
  - deletion.finalizers.rabbitmqclusters.rabbitmq.com
  generation: 4
  name: hello-world
  namespace: rabbitmq-official
  resourceVersion: "3908131"
  uid: 303e2a09-85ba-4333-b8e1-d08538192541
spec:
  delayStartSeconds: 30
  image: rabbitmq:3.12.2-management
  override: {}
  persistence:
    storage: 10Gi
  rabbitmq: {}
  replicas: 3
  resources:
    limits:
      cpu: "2"
      memory: 2Gi
    requests:
      cpu: "1"
      memory: 2Gi
  secretBackend:
    externalSecret: {}
  service:
    type: ClusterIP
  terminationGracePeriodSeconds: 604800
  tls: {}
status:
  binding:
    name: hello-world-default-user
  conditions:
  - lastTransitionTime: "2023-09-10T15:21:57Z"
    reason: AllPodsAreReady
    status: "True"
    type: AllReplicasReady
  - lastTransitionTime: "2023-09-10T15:10:25Z"
    reason: AtLeastOneEndpointAvailable
    status: "True"
    type: ClusterAvailable
  - lastTransitionTime: "2023-09-10T15:09:33Z"
    reason: NoWarnings
    status: "True"
    type: NoWarnings
  - lastTransitionTime: "2023-09-10T15:10:32Z"
    message: Finish reconciling
    reason: Success
    status: "True"
    type: ReconcileSuccess
  defaultUser:
    secretReference:
      keys:
        password: password
        username: username
      name: hello-world-default-user
      namespace: rabbitmq-official
    serviceReference:
      name: hello-world
      namespace: rabbitmq-official
  observedGeneration: 4

查看日志

参考: https://www.rabbitmq.com/kubernetes/operator/quickstart-operator.html

kubectl logs hello-world-server-0
kubectl rabbitmq tail hello-world

获取默认帐号密码,并作测试

username="$(kubectl get secret hello-world-default-user -o jsonpath='{.data.username}' | base64 --decode)"
password="$(kubectl get secret hello-world-default-user -o jsonpath='{.data.password}' | base64 --decode)"
service="$(kubectl get service hello-world -o jsonpath='{.spec.clusterIP}')"
kubectl run perf-test --image=pivotalrabbitmq/perf-test -- --uri amqp://$username:$password@$service

比如, 我这边获取的初始帐号和密码是如下内容

[root@jingmin-kube-archlinux rabbitmq-official]# username="$(kubectl get secret hello-world-default-user -o jsonpath='{.data.username}' | base64 --decode)"
[root@jingmin-kube-archlinux rabbitmq-official]# echo $username
default_user_ydsSKDyVKTaeebbtx8R
[root@jingmin-kube-archlinux rabbitmq-official]# password="$(kubectl get secret hello-world-default-user -o jsonpath='{.data.password}' | base64 --decode)"
[root@jingmin-kube-archlinux rabbitmq-official]# echo $password
5MpT73c3OtAuU8H_tzMhkaC6DIGhOtXU

也可以使用rabbitmq 插件的方式获取账号密码

[root@jingmin-kube-archlinux ~]# kubectl rabbitmq secrets hello-world
username: default_user_ydsSKDyVKTaeebbtx8R
password: 5MpT73c3OtAuU8H_tzMhkaC6DIGhOtXU

本地做下端口转发(仅debug或临时用下)

kubectl port-forward services/hello-world 8081:15672

或者也可以使用插件打开管理ui的端口转发

kubectl rabbitmq manage hello-world

然后浏览器访问127.0.0.1:8081端口,使用上面的初始帐号密码登录.

可以正常登入.集群正常. 可以进一步配vhost,user,exchange,queue,policy等(略).

如何使用rabbitmq-cluster-operator，官方有具体的文档。 参考： https://www.rabbitmq.com/kubernetes/operator/using-operator.html

官方提供了一些例子,用以参考,方便进一步使用operator

参考: https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/

bitnami版本rabbit安装

暂略（暂未试用）

参考： https://artifacthub.io/packages/helm/bitnami/rabbitmq

参考： https://docs.bitnami.com/kubernetes/infrastructure/rabbitmq/

bitnami版本rabbitmq-cluster-operator安装

暂略（暂未试用）

参考： https://artifacthub.io/packages/helm/bitnami/rabbitmq-cluster-operator

参考： https://docs.bitnami.com/kubernetes/infrastructure/rabbitmq/

参考： https://www.rabbitmq.com/kubernetes/operator/install-operator.html#helm-chart

感觉应该是官网Operator的包装。

RabbitMQ 和 RabbitMQ Operator Helm 图表之间的差异

转载来源： https://docs.bitnami.com/kubernetes/infrastructure/rabbitmq/get-started/compare-solutions/

The RabbitMQ chart deploys a single RabbitMQ installation using a Kubernetes StatefulSet object (together with Services, PVCs, ConfigMaps, etc.). The figure below shows the deployed objects in the cluster after chart installation: RabbitMQ 图表使用 Kubernetes StatefulSet 对象（以及服务、PVC、ConfigMap 等）部署单个 RabbitMQ 安装。下图展示了chart安装后集群中部署的对象：

                    +--------------+             +-----+
                    |              |             |     |
 Service            |   RabbitMQ   +<------------+ PVC |
<-------------------+              |             |     |
                    |  StatefulSet |             +-----+
                    |              |
                    +-----------+--+
                                ^                +------------+
                                |                |            |
                                +----------------+ Configmaps |
                                                 | Secrets    |
                                                 +------------+

Its lifecycle is managed using Helm and, at the RabbitMQ container level, the following operations are automated: persistence management, configuration based on environment variables and plugin initialization. 它的生命周期是使用 Helm 进行管理的，并且在 RabbitMQ 容器级别，以下操作是自动化的：持久化管理、基于环境变量的配置和插件初始化。 The StatefulSet does not require any ServiceAccounts with special RBAC privileges, so this solution fits better in more restricted Kubernetes installations. StatefulSet 不需要任何具有特殊 RBAC 权限的 ServiceAccount，因此该解决方案更适合更受限制的 Kubernetes 安装。

The RabbitMQ Operator chart deploys a RabbitMQ Operator installation using a Kubernetes Deployment. The figure below shows the RabbitMQ operator deployment after chart installation: RabbitMQ Operator 图表使用 Kubernetes 部署来部署 RabbitMQ Operator 安装。下图显示了图表安装后的RabbitMQ算子部署：

+--------------------+
|                    |      +---------------+
|  RabbitMQ Operator |      |               |
|                    |      |     RBAC      |
|     Deployment     |      | Privileges    |
+-------+------------+      +-------+-------+
        ^                           |
        |   +-----------------+     |
        +---+ Service Account +<----+
            +-----------------+

The Operator extends the Kubernetes API with the RabbitmqCluster object. The user will then be able to deploy these objects and the previously deployed Operator will take care of deploying all the required StatefulSets, ConfigMaps and Services for running a RabbitMQ instance. Operator 使用 RabbitmqCluster 对象扩展 Kubernetes API。然后，用户将能够部署这些对象，之前部署的 Operator 将负责部署运行 RabbitMQ 实例所需的所有 StatefulSet、ConfigMap 和服务。 Its lifecycle is managed using kubectl on the RabbitmqCluster objects. The following figure shows the deployed objects after deploying a RabbitmqCluster object using kubectl: 它的生命周期是使用 RabbitmqCluster 对象上的 kubectl 进行管理的。下图显示了使用 kubectl 部署 RabbitmqCluster 对象后部署的对象：

  +--------------------+
  |                    |      +---------------+
  |  RabbitMQ Operator |      |               |
  |                    |      |     RBAC      |
  |     Deployment     |      | Privileges    |
  +-------+------------+      +-------+-------+
    |     ^                           |
    |     |   +-----------------+     |
    |     +---+ Service Account +<----+
    |         +-----------------+
    |
    |
    |
    |
    |    |-----------------------------------------------------------------------|
    |    |                                                                       |
    |    |                        +--------------+             +-----+           |
    |    |                        |              |             |     |           |
    |---->     Service            |   RabbitMQ   +<------------+ PVC |           |
         |    <-------------------+              |             |     |           |
         |                        |  StatefulSet |             +-----+           |
         |                        |              |                               |
         |                        +-----------+--+                               |
         |                                    ^                +------------+    |
         |                                    |                |            |    |
         |                                    +----------------+ Configmaps |    |
         |                                                     | Secrets    |    |
         |                                                     +------------+    |
         |                                                                       |
         |                                                                       |
         |-----------------------------------------------------------------------|

This RabbitMQ Operator chart allows users to easily deploy multiple RabbitMQ instances compared to the RabbitMQ chart. 与 RabbitMQ 图表相比，此 RabbitMQ Operator 图表允许用户轻松部署多个 RabbitMQ 实例。

NOTE: As the Operator automatically deploys RabbitMQ installations, the RabbitMQ Operator pods will require a ServiceAccount with privileges to create and destroy multiple Kubernetes objects. This may be problematic for Kubernetes clusters with strict role-based access policies. 注意：由于 Operator 自动部署 RabbitMQ 安装，因此 RabbitMQ Operator Pod 将需要具有创建和销毁多个 Kubernetes 对象权限的 ServiceAccount。对于具有严格的基于角色的访问策略的 Kubernetes 集群来说，这可能会出现问题。

评论

发表回复

您的电子邮箱地址不会被公开。 必填项已用 * 标注