Deploy distributed scalable monitoring server

Install monitoring appliance

The Monitoring appliance is optional and it contains the Abiquo Watchtower components for VM monitoring and metrics, along with its required services and databases. For a production environment, you can run the monitoring database on the datanode services server. In a distributed scalable environment, you will create a monitoring cluster with one node.

Installation requirements

2 CPUs
8 GB RAM
64 GB HD

Installation parameters

Parameter	Notes
Abiquo Server IP	This must be the same as the Abiquo Server appliance management IP address
Abiquo RabbitMQ server IP	Abiquo datanode appliance IP, or comma separated list of IPs for a cluster

Firewall requirements

From	To	Notes
Remote Services	36638/tcp	Watchtower
Datanode services	36638/tcp	Watchtower
Watchtower	7000/tcp	Emmett (One of the Abiquo monitoring services)
KairosDB	9042/tcp	Cassandra
KairosDB	9160/tcp	Cassandra
Watchtower	8080/tcp	Kairos DB
Watchtower	3306/tcp	Database

Post install steps

Do these steps on the monitoring node.

Log in to the monitoring server
Edit the Cassandra configuration file at:
```
/etc/cassandra/conf/cassandra.yaml
```
1. Check the seeds value, which should be the IP of the monitoring VM (or the master VM in the cluster)
```
seeds: x.x.x.x
```
Edit the KairosDB properties file at:
```
/opt/kairosdb/conf/kairosdb.properties
```
1. Remove the this property
```
kairosdb.datastore.cassandra.host_list
```
2. Add this property and set the value as the IP address of the monitoring server, without the port. Note the cql_ in the name. For example.
```
kairosdb.datastore.cassandra.cql_host_list=10.60.10.102
```
Edit the watchtower configuration files at:
```
/etc/abiquo/watchtower/delorean.conf 
/etc/abiquo/watchtower/emmett.conf
```
1. Set the URL of the mysql server connection (as on the API server) with the IP address of the watchtower server
2. Set the database credentials as for the Abiquo server

Edit the watchtower properties files at:

/etc/abiquo/watchtower/delorean.properties
/etc/abiquo/watchtower/emmett.properties

Add the port of the RabbitMQ connection to the datanode services server address
```
abiquo.rabbitmq.addresses = 10.60.10.102:5672
```

Check the status of the monitoring services
```
systemctl status abiquo-delorean.service abiquo-emmett.service kairosdb.service cassandra.service
```
1. If they are running, stop monitoring services
```
systemctl stop abiquo-delorean.service abiquo-emmett.service kairosdb.service cassandra.service
```
2. Check if Cassandra is really dead
```
ps auwx | grep cassandra
```
  Get the process number for Cassandra (the first number in the output of the previous command), and kill it. In this example, Cassandra is process 12345.
```
kill -9 12345
```
3. Stop the firewalld (and allow connections as required)
```
systemctl stop firewalld.service
```
Start cassandra and wait 5 minutes before continuing
Start kairosdb and wait 1 minute
Start abiquo-emmett and abiquo-delorean (the order is not important here)
Start firewalld

The monitoring cluster of one server should now be fully functioning.

Enable and validate monitoring

To enable monitoring:

On the Abiquo Server and the Abiquo Remote Services servers (but not the V2V server), edit the abiquo.properties file and check or set the following properties:
1. abiquo.monitoring.enabled = yes
2. abiquo.watchtower.host = {$MONITORING_IP_ADDRESS}
3. abiquo.watchtower.port = 36638
Restart your Abiquo Tomcat services.

To validate monitoring:

This step assumes you have followed the Abiquo quick tutorial to deploy a VM in Abiquo
Log in to the Abiquo UI and create a new VM and edit it
Go to Monitoring and select the Fetch metrics checkbox and the metrics to display
Save the VM and deploy it
Wait for Abiquo to obtain metrics for the VM (around 5 minutes)
On the VM icon, click the graph metrics button to display your VM metrics