Major Version Upgrade (Blue-Green Deployment)

RabbitMQ 3.12.x cannot be directly upgraded to 4.2.x through an in-place upgrade. You need to create a new 4.2.x instance and migrate data from the old instance. This document provides two migration approaches based on whether downtime is acceptable.

Choose Your Approach

Option 1: Drain & Migrate — If you can tolerate a brief service interruption, consume all messages first, then switch to the new instance. This is the simplest approach.
Option 2: Blue-Green Upgrade — If you cannot tolerate downtime, use the Federation plugin to achieve a seamless migration with zero message loss.

Option 1: Drain & Migrate (Acceptable Downtime)

This approach involves a brief service interruption. If your business can tolerate downtime, this is the most straightforward approach:

Drain Messages: Stop all producers and wait for consumers to process all remaining messages. Ensure all queues are empty.
Export Definitions: Export the definitions (exchanges, queues, bindings, users, vhosts) from the old 3.12.x instance.
Create New Instance: Deploy a new RabbitMQ 4.2.x instance.
Import Definitions: Import the definitions into the new 4.2.x instance.
Switch Traffic: Update your application configurations to connect to the new instance and restart them.
Decommission: Once everything is confirmed working, delete the old 3.12.x instance.

Option 2: Blue-Green Upgrade (Zero Downtime)

This approach uses the RabbitMQ Federation plugin to seamlessly transfer messages from the old instance (blue) to the new instance (green), without stopping producers or consumers. For more details, refer to the official Blue-Green Deployment guide.

How It Works

Deploy a new 4.2.x instance (green) alongside the existing 3.12.x instance (blue).
Import definitions from blue to green (exchanges, queues, bindings, users, etc.).
Configure queue federation on the green cluster with an upstream pointing to blue.
Migrate consumers to green — federation automatically pulls messages from blue to green.
Drain remaining messages from blue.
Migrate producers to green.
Decommission the blue cluster.

Federation Principle

Federated queues work on the principle that consumers connected to the green cluster will receive messages published to the blue cluster, as long as there are no local consumers on blue for those queues. Local consumers always take precedence.

Prerequisites

The old RabbitMQ 3.12.x instance (blue) is healthy and in Active state.
The rabbitmq_federation plugin is available in both instances.

Procedure

CLI

Step 1: Create a new 4.2.x instance (green cluster)

cat << 'EOF' | kubectl -n <namespace> create -f -
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: <green-instance-name>
spec:
  replicas: 3
  version: "4.2"
  resources:
    limits:
      cpu: 1
      memory: 2Gi
    requests:
      cpu: 1
      memory: 2Gi
  persistence:
    storage: 1Gi
    storageClassName: <storage-class>
  service:
    type: ClusterIP
  rabbitmq:
    additionalPlugins:
      - rabbitmq_federation
      - rabbitmq_federation_management
    additionalConfig: |-
      channel_max=1000
      default_vhost=/
      handshake_timeout=10000
      heartbeat=60
      max_message_size=134217728
      queue_index_embed_msgs_below=4096
      raft.wal_max_size_bytes=64000000
      vm_memory_calculation_strategy=allocated
      vm_memory_high_watermark.relative=0.4
EOF

Wait for the green instance to become Active:

kubectl -n <namespace> get rabbitmqclusters <green-instance-name> -w

Step 2: Export definitions from blue and import into green

# Export definitions from blue
kubectl exec -n <namespace> <blue-pod-0> -- \
  rabbitmqctl export_definitions /tmp/definitions.json

# Copy to local
kubectl cp <namespace>/<blue-pod-0>:/tmp/definitions.json ./definitions.json

# Copy to green pod
kubectl cp ./definitions.json <namespace>/<green-pod-0>:/tmp/definitions.json

# Import definitions into green
kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl import_definitions /tmp/definitions.json

Verify definitions are imported correctly:

# Check exchanges, queues, and bindings on green
kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_exchanges name type
kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_queues name
kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_bindings

Step 3: Configure federation upstream on green cluster

Get the blue cluster's internal service address first:

kubectl -n <namespace> get svc -l app.kubernetes.io/name=<blue-instance-name>

Then get the blue cluster's credentials:

# Get the default user secret name
kubectl -n <namespace> get rabbitmqclusters <blue-instance-name> \
  -o jsonpath='{.status.defaultUser.secretReference.name}'

# Retrieve username and password
BLUE_USER=$(kubectl -n <namespace> get secret <blue-secret-name> \
  -o jsonpath='{.data.username}' | base64 -d)
BLUE_PASS=$(kubectl -n <namespace> get secret <blue-secret-name> \
  -o jsonpath='{.data.password}' | base64 -d)

echo "Blue cluster credentials: $BLUE_USER / $BLUE_PASS"

URL Encoding

If your username or password contains special characters (like @, :, /), ensure they are URL-encoded in the connection URI below.

Configure the upstream on the green cluster pointing to blue:

kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl set_parameter federation-upstream blue \
  '{"uri":"amqp://<blue-user>:<blue-password>@<blue-service-name>.<namespace>.svc.cluster.local"}'

Set a federation policy to match all queues:

kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl set_policy --apply-to queues blue-federation ".*" \
  '{"federation-upstream":"blue"}'

Policy Priority Note

In RabbitMQ, if multiple policies match a queue, only the one with the highest priority is applied. If your imported definitions already contain policies covering your queues, the blanket blue-federation policy above might not take effect (or it might overwrite existing rules if given higher priority).

In such cases, you should manually update the existing policies on the green cluster to include the "federation-upstream": "blue" key alongside their current definitions.

Verify federation status:

kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl eval \
  'rabbit_federation_status:status().'

Step 4: Migrate consumers to the green cluster

Update your consumer application configurations to point to the green cluster's service address. At this point:

Producers still publish to the blue cluster.
Consumers read from the green cluster.
Federation automatically pulls messages from blue to green.

You can get the green cluster's service address using the following command:

# Get the green cluster's service address
kubectl -n <namespace> get svc -l app.kubernetes.io/name=<green-instance-name>

Verify messages are being fed through:

# Check that consumers on green are receiving messages
kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl list_queues name messages consumers

# Check federation link status
kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl eval 'rabbit_federation_status:status().'

Important

Before migrating consumers off the blue cluster, make sure the federation links on green are in running state. If a link is not running, messages will not be transferred.

Step 5: Drain messages from the blue cluster

Once all consumers have moved to green, federation will drain remaining messages from blue. Monitor progress:

# Watch queue message counts on blue decrease to 0
kubectl exec -n <namespace> <blue-pod-0> -- \
  rabbitmqctl list_queues name messages

For large backlogs, you can optionally use the Shovel plugin to accelerate draining:

# Enable shovel on green if not already enabled
kubectl exec -n <namespace> <green-pod-0> -- rabbitmq-plugins enable rabbitmq_shovel

# Create a shovel for a specific queue with large backlog
kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl set_parameter shovel shovel-blue-queue1 \
  '{"src-protocol":"amqp091","src-uri":"amqp://<blue-user>:<blue-password>@<blue-service-name>.<namespace>.svc.cluster.local","src-queue":"<queue-name>","dest-protocol":"amqp091","dest-uri":"amqp://","dest-queue":"<queue-name>"}'

Note

Using shovel concurrently with federation will transfer messages in parallel, which may result in messages arriving out of order. If message ordering is critical, rely on federation alone.

Step 6: Migrate producers to the green cluster

Once the queues on blue are nearly empty:

Stop all producers.
Wait for the remaining messages to be fully drained via federation.

Verify all queues on blue are empty:

kubectl exec -n <namespace> <blue-pod-0> -- \
  rabbitmqctl list_queues name messages

Update producer configurations to point to the green cluster and restart them.

Step 7: Clean up and decommission the blue cluster

Remove the federation policy and upstream configuration from green:

kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl clear_policy blue-federation

kubectl exec -n <namespace> <green-pod-0> -- \
  rabbitmqctl clear_parameter federation-upstream blue

Delete the blue cluster:

kubectl -n <namespace> delete rabbitmqcluster <blue-instance-name>

Verification Checklist

Use the following checklist to verify each stage of the migration. Each step includes a CLI command that can be executed directly.

AI Testing

The commands below are designed to be executable step-by-step. Replace all placeholders (e.g. <namespace>, <blue-pod-0>) with actual values before running.

#	Check Item	Command	Expected Result
1	Green cluster is Active	`kubectl -n <namespace> get rabbitmqclusters <green-instance-name>`	`PHASE: Active`
2	Definitions imported	`kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_queues name`	Same queues as blue
3	Federation plugin enabled	`kubectl exec -n <namespace> <green-pod-0> -- rabbitmq-plugins list -e`	`rabbitmq_federation` listed
4	Federation upstream set	`kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_parameters`	`federation-upstream` → `blue` exists
5	Federation links running	`kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl eval 'rabbit_federation_status:status().'`	Status is `running`
6	Consumers on green receiving	`kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_queues name messages consumers`	`consumers > 0`
7	Blue queues draining	`kubectl exec -n <namespace> <blue-pod-0> -- rabbitmqctl list_queues name messages`	Message count decreasing
8	Blue queues empty	Same as above	All queues show `0`
9	Producers on green	`kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_queues name messages`	Messages increasing
10	Federation cleaned up	`kubectl exec -n <namespace> <green-pod-0> -- rabbitmqctl list_parameters`	No federation parameters

Rollback

If issues are encountered during the migration:

Before consumer migration: Simply delete the green cluster. No impact on existing services.
After consumer migration, before producer migration: Switch consumers back to the blue cluster. Delete the green cluster.
After full migration: If the green cluster has issues, create a new 3.12.x instance, import definitions, and switch back. Note that messages produced to green after the switch will need to be drained back using shovel or federation in reverse.

#Major Version Upgrade (Blue-Green Deployment)

#TOC

#Option 1: Drain & Migrate (Acceptable Downtime)

#Option 2: Blue-Green Upgrade (Zero Downtime)

#How It Works

#Prerequisites

#Procedure

#Verification Checklist

#Rollback