Upgrade Abiquo HA platform to v5.2.x

Before you shut down the platform servers you should check that no operations are in progress on the platform.

1. Check that the Abiquo RabbitMQ queues are empty on the Abiquo Monolithic Server, Abiquo Server or Datanode server

   The number of messages in all queues must be 0.

   # rabbitmqctl list_queues messages name

   # rabbitmqctl list_queues messages name
   Listing queues ...
   0	abiquo.am.notifications
   0	abiquo.bpm.notifications
   0	abiquo.datacenter.requests.ADatacenter.bpm
   0	abiquo.datacenter.requests.ADatacenter.virtualfactory
   0	abiquo.ha.tasks
   0	abiquo.nodecollector.notifications
   0	abiquo.pcrsync.messages
   0	abiquo.pcrsync.parking-expect-no-consumers
   0	abiquo.scheduler.fast.requests
   0	abiquo.scheduler.requests
   0	abiquo.scheduler.slow.requests
   0	abiquo.tracer.traces
   0	abiquo.virtualfactory.notifications
   0	abiquo.virtualmachines.definitionsyncs
   0	abiquo.vsm.eventsynk
   ...done.

2. On the V2V Server, check for any active conversions by checking for the V2V or Mechadora processes

   $ ps aux | grep v2v
   $ ps aux | grep mechadora

When user VM operations are blocked and all of the above checks show that no tasks are running, it is safe to halt the platform.

To perform a basic backup of the platform, run the following backups:

# On DB server, back up the Abiquo MySQL DBs
mysqldump --routines --triggers kinton            > kinton.sql-$(date +%Y%m%d_%H%M%S)
mysqldump --routines --triggers kinton_accounting > kinton_accounting.sql-$(date +%Y%m%d_%H%M%S)

# On the Monitoring server, back up Watchtower DB
mysqldump --routines --triggers watchtower            > watchtower.sql-$(date +%Y%m%d_%H%M%S)

# On the Abiquo API Server or UI server, back up the UI
tar cvfz /var/www/html/ui.tgz-$(date +%Y%m%d_%H%M%S) /var/www/html/ui

# If you are using SAML authentication with multiple IdPs, on ALL Abiquo API Servers, back up this file
cp 
/opt/abiquo/tomcat/webapps/api/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml /opt/abiquo/tomcat/webapps/api/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml-$(date
 +%Y%m%d_%H%M%S)

# On ALL servers, back up the /opt/abiquo folder
tar cvfz /opt/abiquo.tgz-$(date +%Y%m%d_%H%M%S) /opt/abiquo

# On Abiquo Monolithic or API Server, Remote Services, or Datanode, back up Redis
redis-cli save
cp -a /var/lib/redis /var/lib/redis-$(date +%Y%m%d_%H%M%S)

This section describes how to stop platform services on all servers.

To stop platform services:

1. Stop the API on the API server or Monolithic server

   systemctl stop abiquo-tomcat

2. Stop the UI on the API server or Monolithic server or dedicated UI server

   systemctl stop httpd

3. Stop Remote services server

   systemctl stop abiquo-tomcat

4. Stop the database on the Monolithic server or Database server or Datanode server.

   systemctl stop mariadb

   For a datanode configuration, you will also need to stop the Galera cluster. For more details, see Stop and start HA configuration

5. Stop RabbitMQ (on the monolithic server or API Server or Datanode)

   systemctl stop rabbitmq-server

6. V2V server -
   You do not need to stop anything because the BPM remote service is run on-demand only

7. Stop monitoring server

   systemctl stop abiquo-delorean
   systemctl stop abiquo-emmett
   systemctl stop kairosdb
   systemctl stop cassandra

8. On the monitoring server, check if Cassandra is really dead

   ps auwx | grep cassandra

   Get the process number for Cassandra (the first number in the output of the previous command), and kill it. In this example, Cassandra is process 12345.

   kill -9 12345

All processes on platform servers should now be halted.

To start the Abiquo platform servers and services, do these steps:

1. On the Abiquo server, restart the HTTP daemon to refresh the user interface files, and bring up the Tomcat server.

   service httpd start
   service abiquo-tomcat start

2. On the Remote services server, start the Tomcat server

   service abiquo-tomcat start

3. On the Monitoring server do these steps.

   1. Edit the file /opt/kairosdb/conf/kairosdb.properties to update the name of the following variable and to remove the port from it:

      1. Replace the line kairosdb.datastore.cassandra.host_list=192.168.888.999:9160.

      2. With this line kairosdb.datastore.cassandra.cql_host_list=192.168.888.999. Please note the new cql_ prefix for host_list.

   2. Edit the file /etc/cassandra/conf/cassandra.yml OR /etc/cassandra/default.conf/cassandra.yaml (whichever exists) to remove a variable:

      1. Remove the line starting with kairosdb.datastore.cassandra.datapoint_ttl and save and close the file.

   3. Edit the file /opt/kairosdb/conf/kairosdb.properties to add a new variable (ref: Internal JIRA SUP-333):

      1. Add kairosdb.datastore.cassandra.datapoint_ttl = 15768000

4. On the Monitoring server, start the Cassandra service

   sudo service cassandra start

5. WAIT about 5 minutes until the service is up and running

6. Start the KairosDB service

   sudo service kairosdb start

7. Start the other services in this order

   sudo service abiquo-emmett start
   sudo service abiquo-delorean start

8. On the V2V server: restart the Tomcat server:

   service abiquo-tomcat restart

9. In Abiquo, re-enable the physical machines!

10. Clear your browser cache to prevent glitches in the UI