Operations grimoire/RabbitMQ
From Nasqueron Agora
RabbitMQ is deployed through Docker.
Clusters
| Cluster name | Management interface | Description |
|---|---|---|
| white-rabbit | https://white-rabbit.nasqueron.org | Main cluster, used for CI and CD purpose |
Procedures
- To redeploy containers, use
salt-call --local state.apply roles/paas-docker/containers/rabbitmq - To enable a plugin, edit this Dockerfile and rebuild image
- To enable a new port for a new protocol or metrics, edit relevant file under rOPS: pillar/paas/
- To bind queue or check what happens, use web interface
Services using RabbitMQ
- white-rabbit, dev vhost
- Notifications center: intelligent bus to get notifications from GitHub, Phabricator, Jenkins, DockerHub, normalize them and publish them to RabbitMQ
- Wearg: IRC bot, read wearg-notifications queue to get notifications to publish them on Libera Chat channels
Troubleshoot
Metrics
If the ready messages is high, it means a queue is not read. As of August 2024, the only permanent queue is the one for notifications used by Wearg. If Wearg is offline or disconnected from RabbitMQ, it's expected to have those messages in the queue.
Metrics doesn't work anymore
Base image disable metrics. They renamed once the configuration file to do so.
If metrics disappears, check in the container the /etc/rabbitmq/conf.d/ directory and update edit this Dockerfile accordingly.
As of July 2024, Prometheus plugin is also needed to allow data scraping.
