Applicable to:
- SolusVM 2
Symptoms
- Solus agent restart on SolusVM 2 Compute Resource makes RabbitMQ to push old already executed tasks. The following message can be found in /var/log/solus/agent.log:
CONFIG_TEXT: INFO[2021-09-30 02:54:40]/usr/src/agent/pkg/queue/queue.go:253 agent/pkg/queue.(*queue).Consume() Consumer "cr-4-worker-vm-backup-restore-0" registered! processing messages...
- Connectivity issue from Compute Resource to RabbitMQ service on SolusVM 2 Management node. The following error can be found in /var/log/solus/agent.log on Compute Resource:
CONFIG_TEXT: /usr/src/agent/pkg/queue/queue.go:96 agent/pkg/queue.(*queue).bindQueue() Queue "" declaration failed: Exception (504) Reason: "channel/connection is not open"
Cause
SolusVM 2 bug with ID #SIO-3671 which was fixed in 1.1.20949
Resolution
Update SolusVM 2 to the latest version:
Workaround:
Note: There should not be any running tasks in SolusVM 2
- Connect to Management node via SSH.
- Backup SolusVM 2 Management node
-
Enter rabbitmq container:
# docker exec -ti $(docker ps --format '{{ .ID }}' -f name=solus_rabbitmq) sh
-
Fetch the list of all queues:
# rabbitmqctl list_queues
-
Remove the queues pushed to Compute Resource with the command:
# rabbitmqctl delete_queue QUEUE_NAME
Replace QUEUE_NAME with the corresponding queue name.
-
Exit rabbitmq container:
# exit
-
Restart rabbitmq with the commands:
# docker service scale solus_rabbitmq=0
# docker service scale solus_rabbitmq=1 - Connect to Compute Resource via SSH
-
Restart Solus agent with the command:
# systemctl restart solus-agent
Comments
0 comments
Please sign in to leave a comment.