...
The Abiquo API can be balanced amongst several servers to spread load and increase service capacity. As you know, Abiquo provides an HTML5 client application that runs on your browser that interacts with the API to operate the cloud.
So, as As the UI is a regular HTML app, there may be several options to balance both the UI and the API. A couple of examples are given below
Info | ||
---|---|---|
| ||
This document covers a setup sample for an Abiquo UI and Abiquo API balanced cluster.
|
API plus UI cluster
In this setup, we are spreading the load amongst a cluster of nodes running both API and UI. Please note that any API clustered setup REQUIRES using zookeeper to work.Zookeeper.
For the sake of simplicity, we are using a two node setup, but this is easily scalable to several nodes and using Apache web server as a load balancer.
CHANGE THIS
Assume the following IP addressing for our cluster nodes.
...
Start by installing backend severs using the Server profile of the Abiquo ISO installer. This profile will install API and UI webapps, as well as the zookeeper Zookeeper daemon needed for API to work in a clustered configuration.
...
Once you have both servers with the Server profile installed, keep in mind that you need to keep the abiquo.properties file synchronized on every node in the API cluster.
...
First, for the API to function as a clustered API, you need to setup zookeeper. The zookeeper package is installed using the "Server" profile of the Abiquo installer ISO. If it is not installed, you can do so typingZookeeper. You can run Zookeeper on the LoadBalancer node or on a separate box, as preferred.
Zookeeper requires JVM to run, ensure that JVM (Oracle or OpenJDK) is installed.
Code Block |
---|
# java -v |
The zookeeper package is installed using the "Server" profile of the Abiquo installer ISO. But if you need to install it manually, use:
Code Block |
---|
# yum -y install zookeeper |
At Then add the command line. Then, you need to add the following line following line in the [server] section of the abiquo.properties file on all Abiquo API server nodes:
Code Block |
---|
abiquo.api.zk.serverConnection = <zookeeper_ip>:2181 |
Replace <zookeeper_ip> to match your environment. Also , add or modify the abiquo.server.api.location property to match the load balancer URL for the API service (shown below):
Code Block |
---|
abiquo.server.api.location = https://abiquo.example.com/api |
You will need to edit file Edit the files /opt/abiquo/tomcat/conf/Catalina/localhost/api.xml and /opt/abiquo/tomcat/conf/Catalina/localhost/m.xml and set the values for the MySQL kinton database :
...
connection. Change the url, username and password attributes to match your environment.
Code Block |
---|
<Resource name="jdbc/abiquoDB" auth="Container" type="javax.sql.DataSource" initialSize="10" suspectTimeout="60" timeBetweenEvictionRunsMillis="30000" minEvictableIdleTimeMillis="60000" maxActive="100" minIdle="10" maxIdle="50" maxWait="10000" removeAbandoned="true" removeAbandonedTimeout="60" logAbandoned="true" username="rootmysql_user" password="user_password" driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhostmysql.example.com:3306/kinton?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8"/> |
The same configuration on the client-premium MySQL jdbc connection should be done in /opt/abiquo/tomcat/conf/Catalina/localhost/client-premium.xml file.
You need to change the url, username and password attributes to match your environment.
UI configuration
In a clustered UI setup, you will need to configure the LB as API endpoint for each UI node. Edit /var/www/html/ui/config/client-config.json file and set same config.endpoint property for each node. For example:
...
UI configuration
In a clustered UI setup, you will need to configure the LB as the API endpoint for each UI node. Edit /var/www/html/ui/config/client-config.json file and set the same config.endpoint property for each UI node. For example:
Code Block |
---|
"config.endpoint": "https://abiquo.example.com/api", |
*** In this setup, maybe it would be nice that by default each UI users the API hosted in the same node and in case of API failure, balance petitions to other nodes. This should be configurable at LB level I guess. ***
Tomcat configuration
Make sure the jvmRoute parameter is different in each host as this will be used by Apache tomcat to route requests to each host.
To do that, edit the conf/server.xml file inside the tomcat directory and edit the following values:
Code Block |
---|
<Engine name="Catalina" defaultHost="localhost" jvmRoute="node1"> |
Restart tomcat after the change to apply new configuration.
Apache configuration
First, you need to make sure that Apache's required modules are loaded. Be sure to include the following lines in your Apache config file:
Code Block |
---|
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer |
LoadBalancer Apache node configuration
To ensure that Apache's required modules are loaded, add the following lines to your Apache config file:
Code Block |
---|
LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_balancer_module modules/mod_proxy_balancer.so LoadModule proxy_ftp_module modules/mod_proxy_ftp.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_connect_module modules/mod_proxy_balancerconnect.so LoadModule proxy_ftpajp_module modules/mod_proxy_ftp.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_connect_module modules/mod_proxy_connect.so LoadModule proxy_ajp_module modules/mod_proxy_ajp.so |
...
ajp.so |
On the LoadBalancer node running Apache, create a new config file in /etc/httpd/conf.d/api-balancer.conf with the following contents:
Code Block |
---|
<VirtualHost *:443> ServerName abiquo.example.com SSLEngine on SSLProtocol all -SSLv2 SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW SSLCertificateFile /etc/pki/tls/certs/abiquo.example.com.crt SSLCertificateKeyFile /etc/pki/tls/private/abiquo.example.com.key # Enable the balancer manager console in the server root <Location /> SetHandler balancer-manager </Location> # Configure the API AJP cluster nodes <Proxy balancer://api-cluster> BalancerMember ajp://10192.60168.132.39100:8010 route=node1 ping=1 BalancerMember ajp://10192.60168.132.39101:8010 route=node2 ping=1 </Proxy> # Configure the UI HTTP cluster nodes <Proxy balancer://ui-cluster> BalancerMember http://10192.60168.132.39100 route=node1 ping=1 BalancerMember http://10192.60168.132.25101 route=node2 ping=1 </Proxy> # Configure the modules we want to load balance ProxyPass /api/ balancer://api-cluster/api/ ProxyPass /ui/ balancer://ui-cluster/ui/ stickysession=JSESSIONID|jsessionid </VirtualHost> |
Restart or reload your Apache server to apply the new configuration.
Tip |
---|
Note that this config enables Apache's balancer-manager to get information on the balancer cluster and perform basic operations on it. If you want to disable it, just comment or remove the <Location /> mark. |
Now, accessing your balancer IP at http://10.60.13.10/client-premium or /api, will balance requests between the two backend nodes.
Tip | ||
---|---|---|
Check the API balancing with the following curl command:
Make sure the href links returned by the curl API call point to the correct location. Otherwise, check your configuration again. |
API cluster and client-premium cluster
In this setup, there are two separate clusters. One is for balancing API requests, and the other one is to balance client-premium traffic.
Assume the following IP addressing for our cluster nodes.
Node | IP addressing |
---|---|
API LB | 10.60.13.10 |
API node1 | 192.168.2.100 |
API node2 | 192.168.2.101 |
Client LB | 10.60.13.50 |
Client node1 | 192.168.2.200 |
Client node2 | 192.168.2.201 |
Installation
Start by installing backend severs using the Server profile of the Abiquo ISO installer. This profile will install API and client-premium webapps, as well as the zookeeper daemon needed for API to work in a clustered configuration.
Tip |
---|
Follow instructions in Distributed Install of Abiquo Server v2.6 or Distributed Install of Abiquo Server v2.4 to get the Server profile installed. |
Once you have all servers with Server profile installed, follow steps below:
In client-premium nodes, delete de API webapp from tomcat's webapp folder, and API config file:
Code Block # rm -rf /opt/abiquo/tomcat/webapps/api # rm /opt/abiquo/tomcat/conf/Catalina/localhost/api.xml
In the API nodes, delete the client-premium webapp and client-premium's config file
Code Block # rm -rf /opt/abiquo/tomcat/webapps/client-premium # rm /opt/abiquo/tomcat/conf/Catalina/localhost/client-premium.xml
Keep in mind that you need to keep abiquo.properties file in sync in every node of the cluster.
API Cluster
You should follow the procedure explained in API plus client-premium cluster with a couple of modifications:
- Remove the client-premium webapp on each node running API.
- Use the following Apache load balancer configuration file:
Code Block |
---|
# Enable the balancer manager console in the server root
<Location />
SetHandler balancer-manager
</Location>
# Configure the cluster nodes
<Proxy balancer://cluster>
BalancerMember ajp://192.168.2.100:8010 route=node1 ping=1
BalancerMember ajp://192.168.2.101:8010 route=node2 ping=1
</Proxy>
# Configure the modules we want to load balance
ProxyPass /api balancer://cluster/api |
It is enough for this balancer to just proxy requests to /api context. Remember to set the required properties in Abiquo tomcat servers.
Client cluster
The client-preimium cluster configuration is quite similar to the API cluster. For convenience, client-premium nodes should share Abiquo properties file with nodes running API and you need to remove every webapp other than client-premium one. Also, delete the file /opt/abiquo/tomcat/conf/Catalina/localhost/api.xml so Tomcat does not complain about the missing API webapp.
Now, in the Apache balancer for client-premium instances, use the following configuration file:
Code Block |
---|
# Enable the balancer manager console in the server root
<Location />
SetHandler balancer-manager
</Location>
# Configure the cluster nodes
<Proxy balancer://cluster>
BalancerMember ajp://192.168.2.200:8010 route=node1 ping=1
BalancerMember ajp://192.168.2.201:8010 route=node2 ping=1
</Proxy>
# Configure the modules we want to load balance
ProxyPass /client-premium/ balancer://cluster/client-premium/ stickysession=JSESSIONID|jsessionid |
Now you have a balanced client-premium environment that connects to a balanced API cluster.
Adding SSL
If you want to use SSL connections to the Abiquo GUI, follow the steps in Apache Frontend with the following modifications:
- Be sure to set a common name for your certificate. This needs to match ServerName parameter in Apache's virtual host definition and abiquo.server.api.location property in Abiquo configuration file.
- Use the following Apache virtual host configuration, replacing ServerName value with your certificate's CN:
Code Block |
---|
<VirtualHost *:443>
ServerName apibalancer
SSLEngine on
SSLProtocol all -SSLv2
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
SSLCertificateFile /etc/pki/tls/certs/ca.crt
SSLCertificateKeyFile /etc/pki/tls/private/ca.key
# Enable the balancer manager console in the server root
<Location />
SetHandler balancer-manager
</Location>
# Configure the cluster nodes (secondary disabled by default)
<Proxy balancer://cluster>
BalancerMember ajp://192.168.2.216:8009 route=node1 ping=1
BalancerMember ajp://192.168.2.217:8009 route=node2 ping=1
</Proxy>
# Configure the modules we want to load balance
ProxyPass /api balancer://cluster/api
ProxyPass /client-premium/ balancer://cluster/client-premium/ stickysession=JSESSIONID|jsessionid
</VirtualHost>
|
- Set abiquo.server.api.location property in Abiquo configuration file to point to the "ServerName" host name and using HTTPS protocol:
Code Block |
---|
abiquo.server.api.location = https://apibalancer/api |
- You need to import Apache server's certificate and CA into Java truststore in order for the client-premium webapp to be able to connect to API. This should be done in every node running the client-premium webapp:
Code Block |
---|
# echo -n | openssl s_client -connect <ServerName>:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/temp.cert
# /usr/java/default/bin/keytool -import -file /tmp/temp.cert -keystore /usr/java/jdk1.6.0_37/jre/lib/security/cacerts
# /usr/java/default/bin/keytool -import -trustcacerts -file /path/to/ca/ca.pem -alias CA_ALIAS -keystore /usr/java/jdk1.6.0_37/jre/lib/security/cacerts |
Tip |
---|
Note that default password for a JVM truststore is changeit. Also note you may need to adjust paths for both keytool command and cacerts truststore depending on your java version. |
Edit /opt/abiquo/tomcat/webapps/client-premium/config/client-config.xml.jsp and change USE_SECURE_CHANNEL_LOGIN value to 1:
Code Block |
---|
...
<name>USE_SECURE_CHANNEL_LOGIN</name> <value>1</value>
... |
...
Info | ||
---|---|---|
| ||
In the Apache configuration sample shown above, the LoadBalancer node also provides the SSL layer and hence, the Proxy balancing rule should be done without SSL against the UI nodes. |
Abiquo API Tomcat configuration
Ensure the jvmRoute parameter is different on each host because this parameter will be used by Apache tomcat to route requests to each host.
To do this, edit the /opt/abiquo/tomcat/conf/server.xml file in the tomcat directory and modify the following values:
Code Block |
---|
<Engine name="Catalina" defaultHost="localhost" jvmRoute="node1"> |
You need to import the LoadBalancer's Apache SSL certificate and CA into the Java truststore to enable the API to complete SSL connections to API endpoints. If you are using a self-signed certificate for testing purposes, importing the SSL certificate will suffice. This should be done on every node running the API webapp:
Code Block |
---|
# echo -n | openssl s_client -connect abiquo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > abiquo.example.com.cert
# /usr/java/default/bin/keytool -import -file abiquo.example.com.cert -keystore /path/to/cacerts
# /usr/java/default/bin/keytool -import -trustcacerts -file /path/to/ca/ca.pem -alias CA_ALIAS -keystore /path/to/cacerts |
Tip | ||
---|---|---|
| ||
|
Restart tomcat after the change to apply the new configuration.
Now, access your balancer at https://abiquo.example.com/ui/ or https://abiquo.example.com/api/ . Requests between the two backend nodes will be adequately balanced.
Tip | ||
---|---|---|
Check the API balancing with the following curl command:
Make sure the href links returned by the API call point to the correct location. Otherwise, check your configuration again. |
Other setups
The sample configuration described in this document covers the setup of a balanced Abiquo API and Abiquo UI cluster but you can change node distribution and LoadBalancer configuration to provide different setups depending on your requirements. For example, you may want to provide separate nodes for Abiquo UI and Abiquo API or maybe you may want just to cluster Abiquo UI and provide a single and shared Abiquo API node, etc.
Also, this sample uses Apache to provide the Load Balancing functionality but this can be achieved with other HA / Load Balancing solutions such as HAProxy or other vendor hardware / software tools.