Versions Compared

Old Version 40

changes.mady.by.user MS

Saved on

compared with

New Version Current

changes.mady.by.user MS

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Upgrade monolithic server

Upgrade Abiquo Monolithic to v5.2.x

Include Page
isMissingRequiredParameterstrue

Upgrade monitoring server

Upgrade monitoring server to v5.2

...

Upgrade steps from 4.7.x to 5.0.x

Start with your original version and perform the steps to the final version.

...

titleClick for Upgrade steps from 4.7.x to 5.0.x

Upgrade steps from 4.7.x to 5.0.x

...

Upgrade steps from 5.0.x to 5.1.x

Start with your original version and perform the steps to the final version.

...

titleClick for Upgrade steps from 5.0.x to 5.1.x for Veeam

Upgrade steps from 5.0.x to 5.1.x for Veeam

...

titleClick for Upgrade steps from 5.0.x to 5.1.2 or 5.2.x for VM names

Upgrade steps from 5.0.x to 5.1.2 or 5.2.x for VM names

...

Upgrade steps from 5.1.x to 5.2.x

Start with your original version and perform the steps to the final version.

When upgrading from 5.1.0 or 5.1.1 to 5.1.2 or above, follow the steps from the Upgrade steps from 5.0.x to 5.1.x block.

...

titleClick for Upgrade steps from 5.1.x to 5.2.x for Azure

Upgrade steps from 5.1.x to 5.2.x for Azure

...

titleClick for Upgrade steps from 5.1.x to 5.2.x for Tomcat (step 3 is mandatory)

Upgrade steps from 5.1.x to 5.2.x for Tomcat with TLS

...

titleClick for Upgrade steps to remove old VSM definitions for Redis

Upgrade steps to remove old VSM definitions for Redis

...

Upgrade steps from 5.1.x to 5.2.x for billing dashboards

...

Upgrade steps for 5.2 versions

These steps apply to upgrades starting from version 5.2.0 and above.

...

titleClick for Upgrade steps for 5.2 versions

Upgrade steps for 5.2 versions

Upgrade steps for 5.2 versions

...

Configure Abiquo after the upgrade

...

Before you start the Abiquo tomcat server, add Abiquo configuration properties to the abiquo.properties file.
By default the abiquo.properties file is found in the /opt/abiquo/config/ folder.
See Changes to Abiquo configuration properties

Configure the user interface. The default UI location is /var/www/html/ui.
Optional: Add custom labels and translations in the lang_xx_XX_custom.json files in the lang folder
Add custom configuration to client-config-custom.json. See Configure Abiquo UI
If your API is not in the same domain as the UI, set the API endpoint pointing to your Abiquo API server:

Code Block
{
    "config.endpoint": "http://myAPIserver/api"
}

...

Reporting changes: To upgrade the Abiquo Reports database for the upgrade to Abiquo 4.7.x+, contact Abiquo Support for the file and procedure.

Start Abiquo servers and services

Start Abiquo servers and services

...

These steps are for a Monolithic Abiquo Server, with API and Remote Services on a single server.

Abiquo will upgrade to new versions of RabbitMQ and MySQL that require a manual upgrade path of dependencies.

Note

These steps are for a server that contains RabbitMQ AND MySQL

  1. Remove the Erlang and Galera packages.

    Code Block
    yum remove erlang* galera
yum clean all && yum makecache

  2. Install new RabbitMQ and MariaDB server

    Code Block
    yum install rabbitmq-server MariaDB-server

    Before you confirm, check that the packages will be installed from the abiquo-base repository

    1. rabbitmq-server: 3.8.2.1

    2. MariaDB-server: 10.4.10.1

  3. Upgrade Abiquo to 5.2.x

    Code Block
    yum update

    Before you confirm, check the following packages will be installed from the abiquo-base repository:

    1. jdk: 11.0.6u10

    2. redis: 5.0.7.1

  4. After the update, check versions

    Check that the symbolic link to the latest version of Java points to Java 11 on Monolithic Server or Datanode

    Code Block
    # cd /usr/java
# ll
latest -> /usr/java/jdk-11.0.6+10

    Check that the correct Tomcat version will be used on the Monolithic Server

    Code Block
    # cd /opt/abiquo/tomcat/bin
# sh version.sh
Using CATALINA_BASE:   /opt/abiquo/tomcat
Using CATALINA_HOME:   /opt/abiquo/tomcat
Using CATALINA_TMPDIR: /opt/abiquo/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /opt/abiquo/tomcat/bin/bootstrap.jar:/opt/abiquo/tomcat/bin/tomcat-juli.jar
NOTE: Picked up JDK_JAVA_OPTIONS:  --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
Server version: Apache Tomcat/9.0.33
Server built:   Mar 11 2020 09:31:38 UTC
Server number:  9.0.33.0
OS Name:        Linux
OS Version:     3.10.0-1160.15.2.el7.x86_64
Architecture:   amd64
JVM Version:    11.0.6+10
JVM Vendor:     AdoptOpenJDK

    On the Monolithic Server in the JDK folder, check certificate migration for AM download. The API Server certificate should be listed for Java 11 with its FQDN. You may need to enter the storepass option to supply the password and you can use the alias option to search for the hostname or FQDN of your server

    Code Block
    # cd /usr/java/jdk-11.0.6+10/lib/security
# keytool -list -keystore cacerts -alias (hostname or fqdn)

    For example

    Code Block
    sudo keytool -list -keystore cacerts -alias abiquo.example.com

  5. Enable and start the new services

    Code Block
    systemctl enable mariadb
systemctl enable rabbitmq-server
systemctl start mariadb
systemctl start rabbitmq-server

  6. Upgrade MySQL (you might need to specifiy the user and password in the command line):

    Code Block
    mysql_upgrade

    The mysql_upgrade step may detect errors that trigger messages such as the following:

    Code Block
    kinton_accounting.ACCOUNT_PERIOD_USAGE_VW Error : Column count of mysql.proc is wrong. Expected 21, found 20. Created with MariaDB 100129, now running 100410. Please use mysql_upgrade to fix this error

    These messages are expected and this same mysql_upgrade process will automatically fix these errors in its next stages.

    Check that the mysql_upgrade process completes correctly.

  7. Check that your hostname is in your DNS or in your /etc/hosts file

  8. Upgrade the Abiquo API databases

    Code Block
    abiquo-db [-h DB hostname] [-P DB port] [-u user] [-p password] update

  9. If the liquibase update fails with a message similar to the following: 

    Code Block
    Liquibase update Failed: Validation Failed:
1 change sets check sum
src/X.X.X-XX/kinton-X.X.X.xml::ABICLOUDPREMIUM-XXXX-XxxxxxxxXXxXxxxxxxXxxxxx::XXXXXXXXX is now: 7:ee2fa6e058ec76c7abf801567898917d
For more information, use the --logLevel flag

    Do the following steps

    1. Clear the database checksums

      Code Block
      abiquo-db clearCheckSums

    2. Retry the above abiquo-db update command.

  10. On the monolithic server, change file owners to tomcat user

    Code Block
    chown -R tomcat /opt/abiquo

  11. On the monolithic server running Remote Services that mount the NFS repository (AM, V2V), change file owners to tomcat user

    Code Block
    chown -R tomcat /opt/vm_repository
chmod -R a+r /opt/vm_repository


...


Upgrade monitoring server

This step is for Watchtower monitoring servers in all installations.

  1. To continue using Java 8 (for Cassandra), remove the JDK through the package manager. If you use yum, it will be delete all the dependencies and you will have to reinstall them

    Code Block
    rpm -e --nodeps jdk

  2. Check the jdk to install is version 8

    Code Block
    rpm -qa | grep jdk

  3. Check that Java is correctly installed:

    1. If you execute this command and get that error, please execute the command from the next bullet:

      Code Block
      $ java
-bash: java: command not found

    2. Fix it by executing this command, then selecting the option "java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/bin/java)":

      Code Block
      $ sudo alternatives --config java

There are 3 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
   1           /usr/java/jdk1.8.0_144/bin/java
 + 2           /usr/java/jdk-11.0.6+10/bin/java
*  3           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/bin/java)

Enter to keep the current selection[+], or type selection number: 3

  4. As for the Abiquo DB steps, remove Galera, which will also remove MariaDB

    Code Block
    yum remove galera

  5. Clean yum and make cache

    Code Block
    yum clean all && yum makecache

  6. Install MariaDB

    Code Block
    yum install MariaDB-server

  7. Update Abiquo

    Code Block
    yum update

  8. Check that the Java package is now "javajdk" to use Java 1.8.0u144 with Cassandra:

    Code Block
    # rpm -qa 'javajdk*'
javajdk-1.8.0u144-6691.el7.x86_64

  9. If you have a HA datanode that runs the watchtower database, skip directly to Step 10 to update watchtower database

  10. Enable and start the services that were reinstalled

    Code Block
    systemctl enable mariadb
systemctl start mariadb

  11. Upgrade MySQL

    Code Block
    mysql_upgrade

    The mysql_upgrade step may detect errors that trigger messages such as the following message (note: this example is from the Abiquo database server).

    Code Block
    kinton_accounting.ACCOUNT_PERIOD_USAGE_VW Error : Column count of mysql.proc is wrong. Expected 21, found 20. Created with MariaDB 100129, now running 100410. Please use mysql_upgrade to fix this error

    These messages are expected and this same mysql_upgrade process will automatically fix these errors in its next stages.

    Check that the mysql_upgrade process completes correctly.

  12. Update the Abiquo Watchtower database

    Code Block
    watchtower-db [-h DB hostname] [-P DB port] [-u user] [-p password] update

...

Upgrade steps from 4.7.x to 5.0.x

Start with your original version and perform the steps to the final version.

Expand
titleClick for Upgrade steps from 4.7.x to 5.0.x

Run Redis database script

On each Remote Services server:

Code Block
# cd /usr/share/doc/abiquo-redis/redis/
# bash ./4.2.3/00-old-vsm-definitions.sh

Post-upgrade steps:

  • Change the tutorials' softlinks in the frontend servers. If there is a softlink /var/www/html/ui/config/tutorials pointing to /opt/abiquo/config/tutorials, change it to point to /opt/abiquo/tutorials instead.

  • If your Apache redirect addresses end in trailing slash characters ("/"), remove the trailing slashes



...


Upgrade steps from 5.0.x to 5.1.x

Start with your original version and perform the steps to the final version.

Expand
titleClick for Upgrade steps from 5.0.x to 5.1.x for Veeam

Run script to update job names on Veeam server

When upgrading from 5.0.x or 5.1.x to 5.1.2 or above, if you are using Veeam, run the following application that will update the job names on the Veeam server to allow duplicate VM names.

The instructions for running the tool are in the following file.

Code Block
/usr/share/doc/abiquo-model/scripts/veeam-job-modifier/README.md

To run the tool, do these steps:

  1. Edit the veeam.properties file and set the MySQL jdbc properties. Replace the examples with the values for the Abiquo database

    Code Block
    abiquo.database.jdbcurl = jdbc:mysql://abiquoserver.example.com:3306/kinton
abiquo.database.user = mysqlUser
abiquo.database.password = mysqlPassword

  2. Obtain the Veeam version and credentials to pass as arguments to the script

    • veeam95u4 Run for usage with Veeam server 9.5u4 or veeam10 Run for usage with Veeam server 10

    • -ip IP address of Veeam server

    • -u, --user veeamServerUser

    • -p, --password veeamServerPassword

  3. The other argument defines a dry-run mode to test the script

    • -no-dry, --no-dry-run If true, update Veeam job names. If false, only show what will the names will look like after the change

  4. Run the tool in dry-run mode (with the argument "--no-dry-run false" and check the Veeam jobs that it will update.

    Code Block
    java -jar /usr/share/doc/abiquo-model/scripts/veeam-job-modifier/veeam-job-modifier-jar-with-dependencies.jar veeam10 -ip \ http://veeamserver.example.com:9399/api -u veeamServerUser -p veeamServerPassword -no-dry false

  5. Run the tool in production mode to update the Veeam server

    Code Block
    java -jar /usr/share/doc/abiquo-model/scripts/veeam-job-modifier/veeam-job-modifier-jar-with-dependencies.jar veeam10 -ip \ http://veeamserver.example.com:9399/api -u veeamServerUser -p veeamServerPassword -no-dry true

  6. After you run the tool, edit the veeam.properties file, and remove the temporary properties with the database credentials.


Expand
titleClick for Upgrade steps from 5.0.x to 5.1.2 or 5.2.x for VM names

Run mandatory database scripts on Redis database and Watchtower SQL database

When upgrading from any version lower than 5.1.2, do the following steps once (e.g. during upgrade to 5.1.2 OR upgrade to 5.2.x)

On the Abiquo Server, run the following tool that will generate two scripts to run on the Redis database and Watchtower SQL database.

These scripts change the use of the Abiquo VM Identifier (name attribute) as the UID to the ID of the VM as the new UID.

The instructions for running the tool are in the following file:

Code Block
/usr/share/doc/abiquo-model/scripts/wt-db-redis-db-migration-tool/README.md

To run the tool and scripts, do these steps:

  1. On the Abiquo Server, run the tool to generate the scripts, supplying the database credentials

    Code Block
    $ java -jar /usr/share/doc/abiquo-model/scripts/wt-db-redis-db-migration-tool/wtrtool.jar \
 --kinton "jdbc:mysql://localhost:3306/kinton?autoReconnect=true&user=mysqlUser&password=mysqlPassword" \
 --output .

    You should find the following scripts in the output directory (by default the current directory)

    • redisdelta, which is Bash script

    • watchtowerdelta.sql

  2. For each Remote Services server, copy the redisdelta Bash script to the server.

  3. On each Remote Services server, run the Bash script. Note that this script requires Bash version 4, which is the default version on Abiquo servers

    Code Block
    bash ./redisdelta

  4. Copy the SQL script to the Watchtower Monitoring Server.

  5. Run the SQL script

    Code Block
    mysql watchtower < ./watchtowerdelta.sql

Run Redis database script

On each Remote Services server:

Code Block
# cd /usr/share/doc/abiquo-redis/redis/
# bash ./4.2.3/00-old-vsm-definitions.sh


...


Upgrade steps from 5.1.x to 5.2.x

Start with your original version and perform the steps to the final version.

When upgrading from 5.1.0 or 5.1.1 to 5.1.2 or above, follow the steps from the Upgrade steps from 5.0.x to 5.1.x block.


Expand
titleClick for Upgrade steps from 5.1.x to 5.2.x for Azure

Run Redis Script to update Azure VSM subscriptions

If you have VMs deployed in Azure, on the Remote Services server, run this script that will use the Redis address and port configured in abiquo.properties. The script will add the VM's Resource Group to the VSM Redis subscription of the VM. We assume that VMs are in the same Resource Group as their virtual datacenter. This is certainly true if the VMs were deployed from Abiquo. In any case, if you previously onboarded a VM in a different RG, the virtual machine definition synchronization process will update it. This script gets the resource group from the VDC provider ID with the format resourceGroup/vdcproviderId, e.g. abiquo-westeurope/abqvnet-vdcname.

  1. First run the tool in dry-run mode and check the VMs to update

    Code Block
    java -jar /usr/share/doc/abiquo-model/scripts/azure-resourcegroup-migration-tool/vm-resourcegroup.jar

  2. Run the tool in production mode to update the subscriptions

    Code Block
    java -jar /usr/share/doc/abiquo-model/scripts/azure-resourcegroup-migration-tool/vm-resourcegroup.jar -no-dry="true"



Expand
titleClick for Upgrade steps from 5.1.x to 5.2.x for Tomcat (step 3 is mandatory)

Configure HTTPS TLS between Tomcat servers

If you are using HTTPS TLS between the Tomcat servers, do these steps. For details of this configuration, see Configure Abiquo Tomcat with HTTPS for Remote RS.

  1. On each Tomcat server (API and RS), restore the backup of the server.xml files from before the upgrade

  2. Edit the server.xml files and add the new secretRequired="false" option to the HTTPS connector

    Code Block
    <Service name="Catalina">

        <Connector
           protocol="org.apache.coyote.http11.Http11NioProtocol"
           port="8009" maxThreads="200"
           scheme="https" secure="true" SSLEnabled="true"
           keystoreFile="/opt/abiquo/tomcat/conf/remoters.jks" keystorePass="changeit" keyAlias="myremoters.bcn.abiquo.com"
           clientAuth="false" secretRequired="false"
           sslProtocol="TLS"/>

  3. If you have a separate API and UI server, on the API server, edit the server.xml file and allow the AJP connector to listen everywhere, not just on localhost, by setting address="0.0.0.0". Also add secretRequired="false".

    Code Block
    <Connector port="8010" protocol="AJP/1.3"
               enableLookups="false"
               tomcatAuthentication="false"
               connectionTimeout="20000"
               secretRequired="false"
               address="0.0.0.0"
               />


Expand
titleClick for Upgrade steps to remove old VSM definitions for Redis

Run Redis database script

(tick) These scripts can always be found on the Abiquo Server.

On each Remote Services server (including Monolithic sever) or Datanode server run the Redis scripts.

Code Block
# cd /usr/share/doc/abiquo-redis/redis/
# bash ./4.2.3/00-old-vsm-definitions.sh


Expand
titleClick for upgrade steps from 5.1.x to 5.2.x for billing dashboards

Remove deprecated billing dashboard scripts

In Abiquo 5.2, the billing dashboard feature has been integrated into the Abiquo platform.
The upgrade process will migrate most of the enterprise properties for billing scripts, which will make the old billing dashboard scripts invalid.

If you were using the billing dashboard in a previous version, do these steps.

  1. To prevent errors, before the upgrade, you should have disabled the old billing dashboard scripts by removing the old cron jobs. If not, disable them now. see Upgrade billing dashboards

  2. In Abiquo, upgrade enterprise properties to ensure valid billing data is displayed. See Upgrade billing dashboards

  3. After you complete the upgrade, remove scripts and data. See /wiki/spaces/doc/pages/311371999





Upgrade steps for 5.2 versions

These steps apply to upgrades starting from version 5.2.0 and above.

Expand
titleClick for Upgrade steps for 5.2 versions

Abiquo 5.2.0 or later to latest 5.2.x version

  • Run Redis database script
    (tick) These scripts can always be found on the Abiquo Server.
    On each Remote Services server (including Monolithic sever) or Datanode server run the Redis scripts

    Code Block
    # cd /usr/share/doc/abiquo-redis/redis/
# bash ./4.2.3/00-old-vsm-definitions.sh


...


Configure Abiquo after the upgrade

  1. Before you start the Abiquo tomcat server, add Abiquo configuration properties to the abiquo.properties file.
    By default the abiquo.properties file is found in the /opt/abiquo/config/ folder.
    See Changes to Abiquo configuration properties

  2. Configure the user interface. The default UI location is /var/www/html/ui.
    Optional: Add custom labels and translations in the lang_xx_XX_custom.json files in the lang folder
    Add custom configuration to client-config-custom.json. See Configure Abiquo UI
    If your API is not in the same domain as the UI, set the API endpoint pointing to your Abiquo API server:

    Code Block
    {
    "config.endpoint": "http://myAPIserver/api"
}


  3. Reporting changes: To upgrade the Abiquo Reports database for the upgrade to Abiquo 4.7.x+, contact Abiquo Support for the file and procedure.


...


Start Abiquo servers and services

To start the Abiquo platform servers and services, do these steps:

  1. On the Abiquo server, restart the HTTP daemon to refresh the user interface files, and bring up the Tomcat server.

    Code Block
    service httpd start
service abiquo-tomcat start

  2. On the Remote services server, start the Tomcat server

    Code Block
    service abiquo-tomcat start

  3. On the Monitoring server do these steps.

    1. Edit the file /opt/kairosdb/conf/kairosdb.properties to update the name of the following variable and to remove the port from it:

      1. Replace the line kairosdb.datastore.cassandra.host_list=192.168.888.999:9160.

      2. With this line kairosdb.datastore.cassandra.cql_host_list=192.168.888.999. Please note the new cql_ prefix for host_list.

    2. Edit the file /etc/cassandra/conf/cassandra.yml OR /etc/cassandra/default.conf/cassandra.yaml (whichever exists) to remove a variable:

      1. Remove the line starting with kairosdb.datastore.cassandra.datapoint_ttl and save and close the file.

    3. Edit the file /opt/kairosdb/conf/kairosdb.properties to add a new variable (ref: Internal JIRA SUP-333):

      1. Add kairosdb.datastore.cassandra.datapoint_ttl = 15768000

  4. On the Monitoring server, start the Cassandra service

    Code Block
    sudo service cassandra start

  5. WAIT about 5 minutes until the service is up and running

  6. Start the KairosDB service

    Code Block
    sudo service kairosdb start

  7. Start the other services in this order

    Code Block
    sudo service abiquo-emmett start
sudo service abiquo-delorean start

  8. On the V2V server: restart the Tomcat server:

    Code Block
    service abiquo-tomcat restart

  9. In Abiquo, re-enable the physical machines! (smile)

  10. Clear your browser cache to prevent glitches in the UI