This document describes the steps to stop and start the database and RabbitMQ in a HA environment.
For more details about restarting a Galera cluster, see https://galeracluster.com/library/training/tutorials/restarting-cluster.html
Check Galera cluster leader
To stop or start the Galera cluster you should first identify the leader node.
On the datanodes, check the state of Galera:
[root@datanodes2 /]# cd /var/lib/mysql/ [root@datanodes2 mysql]# cat grastate.dat # GALERA saved state version: 2.1 uuid: c0e2c5f4-e045-11e9-988e-d6d16254ad99 seqno: -1 safe_to_bootstrap: 0
If there is a node with safe_to_bootstrap=1, you should start the cluster on this node
If no node is marked as safe_to_bootstrap, force it (using the virtual IP).
Stop Galera cluster
To stop the Galera cluster do these steps.
Find the leader node as described above
Starting with the nodes that are not the leader. Stop the database on all nodes, .
[root@datanodes2 /]# systemctl stop mariadb [root@datanodes1 /]# systemctl stop mariadb [root@datanodes0 /]# systemctl stop mariadb
Start Galera cluster
To start the Galera cluster, log in to the leader node.
Start the cluster with this command
[root@datanodes0 /]# galera_new_cluster
On the other nodes, just start MariaDB or MySQL service
[root@datanodes1 /]# systemctl start mariadb [root@datanodes2 /]# systemctl start mariadb
You can use these commands to check the cluster status:
[root@datanodes1 mysql]# maxadmin show servers [root@datanodes1 mysql]# maxadmin list servers
Check and rejoin RabbitMQ cluster
On each datanode server do these commands.
rabbitmqctl stop_app rabbitmqctl join_cluster rabbit@datanodes1 rabbitmqctl join_cluster rabbit@datanodes2 rabbitmqctl start_app rabbitmqctl cluster_status
Start services and servers
After you restart the Galera, MySQL and RabbitMQ on the datanode servers, do these steps
On the services datanodes, manually start the Redis service
Start the monitoring cluster (remember to wait 5 minutes for Cassandra and 1 minute for KairosDB)
Start other Abiquo servers (or restart tomcat services)