Standard Abiquo minor or patch version update

This section describes how to upgrade from Abiquo 5.4.0 or above to an Abiquo 5.4.x version. Please contact Abiquo Support for further information.

 

Abiquo YUM repositories are no longer open, please contact Abiquo Support to obtain your credentials

This upgrade process starts from Abiquo 5.4.0 or above. To upgrade from earlier versions, please see Upgrade Abiquo from 5.2.x to 5.4.x

  • The upgrade will overwrite lang_en_US.json. If you have customized labels or translations, back them up before the upgrade and restore them afterwards accordingly.
  • The upgrade will overwrite the SAML login configuration for multiple IdPs. If you use SAML with multiple IdPs, back up the configuration files before the upgrade and add the multiple IdPs afterwards accordingly. See Restore SAML security beans after upgrade


To prepare your upgrade document, you can use these upgrade docs per installation profile:


1. Prevent cloud users from performing VM operations

  1. In the UI in the Infrastructure view, select each physical machine and click Disable
  2. Using the API, set the state of each physical machine to DISABLED




2. Check for operations in progress on the platform

Before you shut down the platform servers you should check that no operations are in progress.

Before you shut down the platform servers you should check that no operations are in progress on the platform.

  1. Check that the Abiquo RabbitMQ queues are empty on the Abiquo Monolithic Server, Abiquo Server or Datanode server

    The number of messages in all queues must be 0.

    # rabbitmqctl list_queues messages name
     Click here to show/hide more details
    # rabbitmqctl list_queues messages name
    Listing queues ...
    0	abiquo.am.notifications
    0	abiquo.bpm.notifications
    0	abiquo.datacenter.requests.ADatacenter.bpm
    0	abiquo.datacenter.requests.ADatacenter.virtualfactory
    0	abiquo.ha.tasks
    0	abiquo.nodecollector.notifications
    0	abiquo.pcrsync.messages
    0	abiquo.pcrsync.parking-expect-no-consumers
    0	abiquo.scheduler.fast.requests
    0	abiquo.scheduler.requests
    0	abiquo.scheduler.slow.requests
    0	abiquo.tracer.traces
    0	abiquo.virtualfactory.notifications
    0	abiquo.virtualmachines.definitionsyncs
    0	abiquo.vsm.eventsynk
    ...done.
  2. On the V2V Server, check for any active conversions by checking for the V2V or Mechadora processes

    $ ps aux | grep v2v
    $ ps aux | grep mechadora

When users' VM operations are blocked and all of the above checks show that no tasks are running, it is safe to halt the platform.




3. Back up the main platform elements

To perform a basic backup of the platform, run the following backups:

Before you begin, stop platform services, and check you have enough space on your destination systems.


# On DB server, back up the Abiquo MySQL DBs
mysqldump --routines --triggers kinton            > kinton.sql-$(date +%Y%m%d_%H%M%S)
mysqldump --routines --triggers kinton_accounting > kinton_accounting.sql-$(date +%Y%m%d_%H%M%S)

# On the Abiquo API Server or UI server, back up the UI
tar cvfz /var/www/html/ui.tgz-$(date +%Y%m%d_%H%M%S) /var/www/html/ui

# If you are using SAML authentication with multiple IdPs, on ALL Abiquo API Servers, back up this file
cp 
/opt/abiquo/tomcat/webapps/api/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml /opt/abiquo/tomcat/webapps/api/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml-$(date
 +%Y%m%d_%H%M%S)

# On ALL servers, back up the /opt/abiquo folder
tar cvfz /opt/abiquo.tgz-$(date +%Y%m%d_%H%M%S) /opt/abiquo

# On Abiquo Monolithic or API Server, Remote Services, or Datanode, back up Redis
redis-cli save
cp -a /var/lib/redis /var/lib/redis-$(date +%Y%m%d_%H%M%S)





4. Stop platform services for minor upgrade

This section describes how to stop platform services on all servers.

if there are operations in progress, DO NOT STOP the platform services because this can cause serious issues with your platform.
See Check for operations in progress on the platform and wait for all operations to complete

To stop platform services:

  1. Stop the API on the API Server or Monolithic Server

    systemctl stop abiquo-tomcat
  2. Stop the UI on the API Server or Monolithic Server or dedicated UI Server

    systemctl stop httpd
  3. Stop Remote Services server

    systemctl stop abiquo-tomcat
  4. V2V Server
    You do not need to stop anything because the BPM remote service is run on-demand only

  5. Stop Monitoring server

    systemctl stop abiquo-delorean
    systemctl stop abiquo-emmett
    systemctl stop kairosdb
    systemctl stop cassandra
  6. On the Monitoring server, check if Cassandra is really dead

    ps auwx | grep cassandra

    Get the process number for Cassandra (the first number in the output of the previous command), and kill it. In this example, Cassandra is process 12345.

    kill -9 12345

All processes on platform servers should now be halted.




5. Optionally make snapshots and backups of all platform servers

Although you are upgrading to one of the minor versions, we recommend that you make a snapshot and/or a backup of your platform servers.



6. Prepare yum repositories on all servers

  1. Check that you have the repository URL and credentials

  2. To upgrade to a version with patch number that is not zero, for example, version 6.0.x, enable both repositories:

    yum-config-manager --enable abiquo-base
    yum-config-manager --enable abiquo-updates
  3. Optionally add your username and password to the Abiquo repos

    yum-config-manager --save --setopt=abiquo-*.username=MYUSERNAME
    yum-config-manager --save --setopt=abiquo-*.password=MYPASSWORD

    Don't forget to use a backslash to escape any shell special characters. For more details, see https://www.oreilly.com/library/view/learning-the-bash/1565923472/ch01s09.html

  4. Clean yum and make cache

    yum clean all && yum makecache
  5. If you did not make snapshots of all servers already, then you could do this now.



7. Upgrade packages on ALL servers

These instructions are for monolithic, distributed, and HA environments.

  1. Update Abiquo packages:

    yum clean all && yum makecache && yum update 'abiquo-*'
  2. The Abiquo services must run as the tomcat user (not root), so set the required permissions and enforce the use of the package configuration files.
    If you have a MONOLITHIC SERVER do all of the following steps on the Monolithic server

    1. On Abiquo Server and Remote Services

      chown -R tomcat /opt/abiquo
    2. On the Remote Services with Appliance Manager and the V2V Server (i.e. servers that mount the NFS repository)

      chown -R tomcat /opt/vm_repository
      chmod -R a+r /opt/vm_repository
    3. On the V2V server, check the /etc/sysconfig/abiquo/ec2-api-tools file exists.

8. Update the Abiquo databases

  1. Check that your hostname is in your DNS or in your /etc/hosts file

  2. Upgrade the Abiquo API databases

    abiquo-db [-h DB hostname] [-P DB port] [-u user] [-p password] update


    If the liquibase update fails with a message similar to the following: 

    Liquibase update Failed: Validation Failed:
    1 change sets check sum
    src/X.X.X-XX/kinton-X.X.X.xml::ABICLOUDPREMIUM-XXXX-XxxxxxxxXXxXxxxxxxXxxxxx::XXXXXXXXX is now: 7:ee2fa6e058ec76c7abf801567898917d
    For more information, use the --logLevel flag

    Do the following steps

    1. Clear the database checksums

      abiquo-db clearCheckSums
    2. Retry the above abiquo-db update command.
  3. To upgrade the Abiquo Watchtower database, on the monitoring appliance, run the command below:

    watchtower-db [-h DB hostname] [-P DB port] [-u user] [-p password] update

9. Upgrade additional elements

Start with your original version and perform the steps to the final version.


By default, you will find the upgrade scripts (e.g. for Redis definitions) on the Abiquo Server under the folder: /usr/share/doc/abiquo-redis/redis/

For each upgrade step, run the Redis database script to remove old VSM definitions, which can be found in the 4.2.3 subfolder

On Remote Services server:

# cd /usr/share/doc/abiquo-redis/redis/
# bash ./4.2.3/00-old-vsm-definitions.sh


Upgrade steps for Abiquo 5.4.1

  1. Standard Redis database upgrade step


    Run Redis database script
    (tick) These scripts can always be found on the Abiquo Server.
    On each Remote Services server (including Monolithic sever) or Datanode server run the Redis scripts

    # cd /usr/share/doc/abiquo-redis/redis/
    # bash ./4.2.3/00-old-vsm-definitions.sh


Upgrade steps for Abiquo 5.4.2

  1. Standard Redis database upgrade step


Additional services to be upgraded

If you are using an Abiquo service customization that is not distributed with OS pacakges (you will already know that, like the e2c service), then please make sure to update it now before starting the services again.


10. Configure Abiquo after the upgrade

  1. Before you start the Abiquo tomcat server, add Abiquo Configuration Properties to the abiquo.properties file.
    By default the abiquo.properties file is found in the /opt/abiquo/config/ folder.
    See Changes to Abiquo Configuration Properties

  2. Configure the user interface. The default UI location is /var/www/html/ui.
    Optional: Add custom labels and translations in the lang_xx_XX_custom.json files in the lang folder
    Add custom configuration to client-config-custom.json. See Configure Abiquo UI
    If your API is not in the same domain as the UI, set the API endpoint pointing to your Abiquo API server:

    {
        "config.endpoint": "http://myAPIserver/api"
    }
  3. SAML: If you are using SAML with multiple IDPs, restore the configuration as described at Restore SAML security beans after upgrade.

11. Start Abiquo server and services

To start the Abiquo platform servers and services, do these steps:

  • On the Abiquo Server, restart the HTTP daemon to refresh the user interface files, and bring up the Tomcat server.

    service httpd restart
    service abiquo-tomcat start
  • On the Remote Services server, start the Tomcat server

    service abiquo-tomcat start
  • On the Monitoring Server: start the Cassandra service

    sudo service cassandra start
  • WAIT about 5 minutes until the service is up and running
  • Start the KairosDB service

    sudo service kairosdb start
  • Start the other services in this order

    sudo service abiquo-emmett start
    sudo service abiquo-delorean start

Copyright © 2006-2022, Abiquo Holdings SL. All rights reserved