...
For your convenience, we have also prepared the following concise upgrade document that groups the commands by servers.
However, we encourage you to use the current Upgrade Abiquo document as your main reference:
Please contact Abiquo Support for further information.
...
From version | To version | Upgrade type | Notes |
---|---|---|---|
6.1.x | 6.2.x | StandardMajor | Follow the process in this document (or at Upgrade Abiquo 6.1.x to 6.2.x distributed by servers
|
...
Before you begin, contact Abiquo Platform and Customer Success department to obtain your credentials because for Abiquo YUM repositories require a login.
Warning |
---|
Major upgrade The upgrade to Abiquo 6.2.0 is a major upgrade with upgrades of third-party software packages that are critical platform dependencies You must make snapshots of ALL servers before you start the upgrade |
The upgrade will overwrite
lang_en_US_labels.json
. If you have Back up your customized labels or translations , back them up before the upgrade, and restore them afterwards.
Note |
---|
SAML upgrade
|
...
3. Prevent cloud users from performing VM operations
In the UI in the Infrastructure view, select each physical machine and click Disable
Using the API, set the state of each physical machine to
DISABLED
...
Code Block | ||
---|---|---|
| ||
# On DB server, back up the Abiquo MySQLAPI DBsServer mysqldumpor --routines --triggers kinton > kinton.sqlUI server, back up the UI tar cvfz /var/www/html/ui.tgz-$(date +%Y%m%d_%H%M%S) mysqldump --routines --triggers kinton_accounting > kinton_accounting.sql-$(date +%Y%m%d_%H%M%S) # On the Monitoring server, back up Watchtower DB mysqldump --routines --triggers watchtower > watchtower.sql/var/www/html/ui # Copy the api.xml file to back up the database connection details cp /opt/abiquo/tomcat/conf/Catalina/localhost/api.xml /opt/abiquo/api.xml-$(date +%Y%m%d_%H%M%S) # On theALL Abiquoservers, APIback Serverup or UI server, back up the UI the /opt/abiquo folder tar cvfz /var/www/html/uiopt/abiquo.tgz-$(date +%Y%m%d_%H%M%S) /var/www/html/uiopt/abiquo # IfOn youAbiquo areMonolithic usingor SAMLAPI authenticationServer, withRemote multiple IdPsServices, on ALL Abiquo API Serversor Datanode, back up thisRedis fileredis-cli save cp -a /optvar/abiquolib/tomcatredis /webappsvar/apilib/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml /opt/abiquo/tomcat/webapps/api/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml-$(date redis-$(date +%Y%m%d_%H%M%S) # On DB server, back up the Abiquo MySQL DBs mysqldump --routines --triggers kinton > kinton.sql-$(date +%Y%m%d_%H%M%S) mysqldump # Copy the api.xml file to back up the database connection details cp /opt/abiquo/tomcat/conf/Catalina/localhost/api.xml /opt/abiquo/api.xml--routines --triggers kinton_accounting > kinton_accounting.sql-$(date +%Y%m%d_%H%M%S) # On ALLthe Monitoring serversserver, back up the /opt/abiquo folder tar cvfz /opt/abiquo.tgzWatchtower DB mysqldump --routines --triggers watchtower > watchtower.sql-$(date +%Y%m%d_%H%M%S) /opt/abiquo # On Abiquo Monolithic or API Server, Remote Services, or Datanode # If you are using SAML authentication with multiple IdPs, on ALL Abiquo API Servers, back up Redisthis redis-clifile save cp -a /var/lib/redis /var/lib/redis-$(date +%Y%m%d_%H%M%S/opt/abiquo/tomcat/webapps/api/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml /opt/abiquo/tomcat/webapps/api/WEB-INF/classes/springresources/security/saml/security-saml-generated-beans.xml-$(date +%Y%m%d_%H%M%S) |
...
7. Make snapshots and backups of all platform servers
This is a major upgrade, so we recommend that you make a snapshot and/or a full backup of your platform servers.
...
8.
...
Check that you have the repository URL and credentials
On all Abiquo Servers, such as Abiquo API, Remote Services, Watchtower monitoring.
Install release package
To upgrade to a version with a patch number of zero, for example, version 6.2.0
...
Remove the previous version Abiquo release packages.
Code Block |
---|
yum remove 'abiquo-release-*' |
...
Find the abiquo-release-ee
package for your version in the list of available versions
Install the release package.
For Abiquo 6.2.0, the command would be similar to the following:
Code Block |
---|
yum localinstall http://usr:passwd@mirror.abiquo.com/el7/6.2/os/x86_64/abiquo-release-ee-6.2.0-20240205_155724.el7.noarch.rpm |
For Abiquo 6.2.1, the release package would be in the format:
Code Block |
---|
http://usr:passwd@mirror.abiquo.com/el7/6.2/updates/x86_64/abiquo-release-ee-6.2.1-BBBBBBBBBBBB.el7.noarch.rpm |
...
For major versions, disable the updates repo. See https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/sec-managing_yum_repositories
Remove Docker and unmount shared folders
On your RS server(s) uninstall Docker
Code Block rpm -e --nodeps docker rpm -e --nodeps docker-client rpm -e --nodeps docker-common
If you have multiple instances of the NARS Remote Service in your datacenter, check on your RS servers for an NFS shared folder for the NARS results (with the Abiquo configuration property
abiquo.nars.async.fileresults.path
):Code Block language bash $ mount | grep -i "nfs\|results" | grep -v sunrpc 10.60.37.87:/abq-pro-nfs on /opt/abiquo/results-nars type nfs4 (rw,nosuid,nodev,noexec,relatime,sync,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.60.37.77,local_lock=none,addr=10.60.37.87,user)
Unmount the shared folder. You will remount it after the upgrade
Code Block language bash sudo umount /opt/abiquo/results-nars
...
9. Prepare yum repositories for 6.2 or 6.2.x
...
Check that you have the repository URL and credentials
...
On all Abiquo Servers, such as Abiquo API, Remote Services, Watchtower monitoring.
Install release package
To upgrade to a version with a patch number that is not of zero, for example, version 6.2.x, enable both repositories:.0
Remove the previous version Abiquo release packages.
Code Block yum
Optionally add your username and password to the Abiquo repos
Code Block yum-config-manager --save --setopt=abiquo-*.username=MYUSERNAME yum-config-manager --save --setopt=abiquo-*.password=MYPASSWORD
Don't forget to use a backslash to escape any shell special characters. For more details, see https://www.oreilly.com/library/view/learning-the-bash/1565923472/ch01s09.html
Clean yum and make cache
Code Block yum clean all && yum makecache
If you did not make snapshots of all servers already, then you could do this now.
9. Upgrade packages on ALL servers
These instructions are for monolithic, distributed, and HA environments.
Check in your RS server(s) if there is an NFS shared folder for the NARS results (related to the Abiquo configuration property
abiquo.nars.async.fileresults.path
):Code Block language bash $ mount | grep -i "nfs\|results" | grep -v sunrpc 10.60.37.87:/abq-pro-nfs on /opt/abiquo/results-nars type nfs4 (rw,nosuid,nodev,noexec,relatime,sync,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.60.37.77,local_lock=none,addr=10.60.37.87,user)
Unmount the previous shared folder if you have it mounted:
Code Block language bash sudo umount /opt/abiquo/results-nars
Also on your RS server(s) uninstall Docker
Code Block yum remove docker
Update all packages:
yumCode Block remove 'abiquo-release-*'
Find the
abiquo-release-ee
package for your version in the list of available versionsInstall the release package.
For Abiquo 6.2.0, the command would be similar to the following:Code Block yum localinstall http://usr:passwd@mirror.abiquo.com/el7/6.2/os/x86_64/abiquo-release-ee-6.2.0-AAAAAAAA_BBBBBBB.el7.noarch.rpm
For Abiquo 6.2.1, the release package would be in the format:
Code Block http://usr:passwd@mirror.abiquo.com/el7/6.2/updates/x86_64/abiquo-release-ee-6.2.1-BBBBBBBBBBBB.el7.noarch.rpm
For major versions, disable the updates repo.
Code Block yum-config-manager --disable abiquo-updates
Prepare yum repositories for 6.2.x (minor version)
Check that you have the repository URL and credentials
To upgrade to the latest version with a patch number that is not zero, for example, version 6.2.x, enable both repositories:
Code Block yum-config-manager --enable abiquo-base yum-config-manager --enable abiquo-updates
Optionally, add your username and password to the Abiquo repos
Code Block yum-config-manager --save --setopt=abiquo-*.username=MYUSERNAME yum-config-manager --save --setopt=abiquo-*.password=MYPASSWORD
Don't forget to use a backslash to escape any shell special characters. For more details, see https://www.oreilly.com/library/view/learning-the-bash/1565923472/ch01s09.html
Clean yum and make cache
Code Block yum clean all && yum makecache && yum update
The MariaDB update requires a manual install. But first download and run the gpmigrator tool
Mount the shared folder from the second step again.
Code Block language bash sudo mount -a
The Abiquo services must run as the
tomcat
user (notroot
), so set the required permissions and enforce the use of the package configuration files.
If you have a MONOLITHIC SERVER do all of the following steps on the Monolithic serverOn Abiquo server and remote services
Code Block chown -R tomcat /opt/abiquo
On the remote services with appliance manager and the V2V server (i.e. servers that mount the NFS repository)If you did not make snapshots of all servers already, then you could do this now.
...
10. Upgrade packages on ALL servers
These instructions are for monolithic, distributed, and HA environments.
Update all packages.
Code Block yum clean all && yum makecache && yum update
The MariaDB update requires a manual install. But first download and run the
gpmigrator
toolIf you earlier unmounted shared folders from your Remote Services server, mount this folder again (and any other folders!)
Code Block language bash sudo mount -a
The Abiquo services must run as the
tomcat
user (notroot
), so set the required permissions and enforce the use of the package configuration files.
If you have a MONOLITHIC SERVER do all of the following steps on the Monolithic serverOn Abiquo server and remote services
Code Block chown -R tomcat /opt/vm_repository chmod -R abiquo
On the remote services with appliance manager and the V2V server (i.e. servers that mount the NFS repository)
Code Block chown -R tomcat /opt/vm_repository chmod -R a+r /opt/vm_repository
On the V2V server, check the
/etc/sysconfig/abiquo/ec2-api-tools
file exists.
The file must contain the following configuration. If the file does not exist, create it and add this configuration.For CentOS 7
Code Block EC2_HOME=/opt/aws
...
...
11. Run the guest password migrator
Download and install the
guestpasswordmigrator
packageRun the tool to migrate the guest passwords of the current VMs togpmigrator
packageCode Block yum localinstall http://user:passwd@mirror.abiquo.com/el7/6.2/os/x86_64/abiquo-gpmigrator-6.2.0-20240205_155724.el7.noarch.rpm
Run the tool to migrate the guest passwords of the current VMs to a new column that will be added in 6.2.0.
The tool requires access to the
kinton
database and to the API RedisUsage:
Code Block --jdbc-url VAL : Replace VAL with Kinton JDBC URL including user and password --output-file VAL : Replace VAL with the output file (by default: 'guestpasswords.sql') --redis-host VAL : Redis host to connect --redis-port N : Redis port to connect (by default: 6379)
For example:
Code Block java -jar gpmigrator.jar --jdbc-url="jdbc:mariadb://localhost:3306/kinton?user=root&password=root" --redis-host=localhost
Save the
guestpasswords.sql
file and run it (or whatever you called it) to run after you update the Abiquo database withabiquo-db
.
...
12. Upgrade MariaDB for Abiquo 6.2.0
On all database servers, including the Watchtower server, upgrade MariaDB
Code Block rpm --nodeps -ev MariaDB-server-10.4.10-1.el7.centos.x86_64 yum install MariaDB-server service mariadb start # OR for datanode: galera_new_cluster mysql_upgrade
...
13. Update the Abiquo databases
Check that your hostname is in your DNS or in your /etc/hosts file
Upgrade the Abiquo API databases
Code Block abiquo-db [-h DB hostname] [-P DB port] [-u user] [-p password] update
If the Liquibase database update fails with a message similar to the following:
Do the following stepsCode Block Liquibase update Failed: Validation Failed: 1 change sets check sum src/X.X.X-XX/kinton-X.X.X.xml::ABICLOUDPREMIUM-XXXX-XxxxxxxxXXxXxxxxxxXxxxxx::XXXXXXXXX is now: 7:ee2fa6e058ec76c7abf801567898917d For more information, use the --logLevel flag
Clear the
databasechecksums and run the update again
Code Block abiquo-db clearCheckSums
Retry the above
abiquo-db update
command. In MariaDB, for Abiquo 6.2.For Abiquo 6.2.0, run the
gpmigrator
SQL file output from the guest password migrator script (with the appropriate user and credentials)Code Block mysql kinton < guestpasswords.sql
To upgrade the Abiquo Watchtower database, on for the monitoring appliance, run the command below:
Code Block watchtower-db [-h DB hostname] [-P DB port] [-u user] [-p password] update
abiquo-db [-h DB hostname] [-P DB port] [-u user] [-p password] update |
...
...
14. Upgrade additional elements
By default, you will find the upgrade scripts (e.g. for Redis definitions) on the Abiquo Server under the folder: /usr/share/doc/abiquo-redis/redis/
For each upgrade step, run the Redis database script to remove old VSM definitions, which can be found in the 4.2.3 subfolder
On remote services server:
Code Block |
---|
# cd /usr/share/doc/abiquo-redis/redis/
# bash ./4.2.3/00-old-vsm-definitions.sh |
For Abiquo 6.2.0+, update the database connector configurations
...
On the Abiquo API Server, edit /opt/abiquo/tomcat/conf/Catalina/localhost/api.xml
...
If your database is on the Abiquo Server or a single server
For the database connector, remove
jdbc:mysql://dbhost
and replace withjdbc:mariadb:${DBHOST}
If you are using a separate database cluster:
...
For Abiquo 6.2.0+, update the database connector configurations
On the Abiquo API Server, edit
/opt/abiquo/tomcat/conf/Catalina/localhost/api.xml
The latest version of the file should contain new options as shown below.
Update the file and/or variables marked with${}
for your environment.Code Block <Context> <Resource name="jdbc/abiquoDB" auth="Container" type="javax.sql.DataSource" factory="com.zaxxer.hikari.HikariJNDIFactory" dataSourceClassName="org.mariadb.jdbc.MariaDbDataSource" connectionTimeout="120000" maximumPoolSize="100" username="${server.database.username}" password="${server.database.password}" dataSource.url="jdbc:mariadb://${server.database.host}:${server.database.port}/kinton" /> </Context>
If you have a DB cluster, change the
datasource.url
to point to your Galera cluster.
This will not affect the multi-master configuration of the clusterCode Block jdbc:mariadb:loadbalance://address=(host=${DBMAIN})(port=3306)(type=master), address=(host=${DBR1})(port=3306)(type=slave), address=(host=${DBR2})(port=3306)(type=slave)/kinton
On the Watchtower server, edit the
emmett.conf
anddelorean.conf
files at:Code Block /etc/abiquo/watchtower/delorean.conf /etc/abiquo/watchtower/emmett.conf
From the connection details, remove
jdbc:mysql:
and replace withjdbc:mariadb:
Add your Galera configuration if your monitoring DB is in a cluster
...
15. Configure Abiquo after the upgrade
Before you start the Abiquo tomcat server, add Abiquo configuration properties to the
abiquo.properties
file.
By default theabiquo.properties
file is found in the/opt/abiquo/config/
folder.
See Changes to Abiquo configuration propertiesConfigure the user interface. The default UI location is
/var/www/html/ui
.
Optional: Add custom labels and translations in thelang_xx_XX_custom.json
files in the lang folder
Add custom configuration toclient-config-custom.json
. See Configure Abiquo UI and Changes to UI configuration
If your API is not in the same domain as the UI, set the API endpoint pointing to your Abiquo API server:Code Block { "config.endpoint": "http://myAPIserver/api" }
SAML: If you are using SAML with multiple IDPs, restore the configuration as described at Restore SAML security beans after upgrade.
Abiquo 6.2.0 updates Java and other packages, so restore your certificates. To do this, you can follow the steps at Update certificates in your Abiquo environment.
...
...
16. Start Abiquo server and services
To start the Abiquo platform servers and services, do these steps:
On Abiquo server or datanode, restart RabbitMQ
Code Block systemctl start rabbitmq-server
On the Abiquo server, restart the HTTP daemon to refresh the user interface files, and bring up the Tomcat server.
Code Block service httpd start service abiquo-tomcat start
On the Remote services server, start the Tomcat server
Code Block service abiquo-tomcat start
On the Monitoring server: start the Cassandra service
Code Block sudo service cassandra start
WAIT about 5 minutes until the service is up and running
Start the KairosDB service
Code Block sudo service kairosdb start
Start the other services in this order
Code Block sudo service abiquo-emmett start sudo service abiquo-delorean start
On the V2V server: restart the Tomcat server:
Code Block service abiquo-tomcat restart
...