Introduction to Monitoring and Metrics

The Abiquo VM monitoring and metrics feature enables you to obtain a rapid and convenient overview of VM performance through the Abiquo cloud console. Abiquo supports monitoring and metrics of VMs on AWS, Azure, ESXi with vCenter, KVM, and XenServer, as well as Docker.

This feature supports the following metrics:

Built-in metrics from each hypervisor and public cloud plugin; the monitor manager (VSM remote service) fetches these metrics periodically
Custom metrics with two options:
- user configures and populates custom metrics using the Abiquo API
- user configures custom metrics in collectd and the Abiquo collectd plugin populates them

The following diagram shows a basic monitoring and metrics configuration.

In addition to metrics, the Monitoring Server also manages:

Metric alarms that detect when metrics reach threshold levels
Alerts that activate when a set of alarms are triggered. Alerts can notify users or trigger actions on VMs, such as autoscaling

Install Monitoring

In a test environment, install the Monitoring OVA that is a CentOS server with the monitoring server components (Watchtower server, KairosDB, and Cassandra). For full instructions on how to install it, see OVA deployment. In a production environment, a monitoring server would typically have 4 CPUs and 16 GB RAM.

For a larger production environment, you can install separate servers:

Watchtower Server
KairosDB
Cassandra, with one of these configurations in order of scale:
1. On Kairos DB server
2. Separate server
3. Cluster
Cassandra servers require JRE 7 and ideally disk sizes are 1 TB per node of spinning disks or 3.5 TB per node of SSD.

Post-install configuration

After you complete the install, perform the following configuration:

In cassandra.yaml, set the following properties:
1. batch_size_warn_threshold_in_kb: 50
2. batch_size_fail_threshold_in_kb: 70
Set KairosDB properties (with default location /opt/kairosdb/conf/kairosdb.properties) as required for your environment and noting the following recommendations:
- Monitoring data retention time: KairosDB manages the data in Cassandra and you can set the the "time to live" for datapoints with the kairosdb.datastore.cassandra.datapoint_ttl property as described here: https://github.com/kairosdb/kairosdb/blob/v0.9.4/src/main/resources/kairosdb.properties#L91-L95
  Cassandra will automatically delete a datapoint after its time to live has elapsed. By default, the property is not set and Cassandra will keep datapoints forever. By default the datapoints are stored in the /var/lib/cassandra/data/kairosdb folder
  - Note that when the platform deletes a metric, it will delete the datapoints in KairosDB
- The number of datapoints to send to Cassandra in large environments. Before you begin, set the Cassandra batch_size properties above. For best performance, set:
  - kairosdb.queue_processor.batch_size=10000. See https://github.com/super-collider/docker-openjdk8-kairosdb/blob/master/kairosdb.properties#L193-L204
In cassandra.yaml, configure seeds with the KairsoDB IP (even if it's on the same server) and set Broadcast address as described in the Cassandra documentation
Firewall:
1. on the Watchtower server, enable the Abiquo Server and ALL the Remote Services servers (except separate V2V services) to access TCP 36638
2. on the Abiquo Server, allow Watchtower to access RabbitMQ
3. in multi-server environments:
  1. on the KairosDB server, allow Abiquo Watchtower to access TCP 8080
  2. on the Cassandra servers, allow KairosDB to access Cassandra on the configured ports
Synchronize server clocks: you MUST configure and synchronize NTP across all Abiquo servers and all hypervisors before you enable metrics. Alerts will not be sent if the time is not synchronized on all servers, including the monitoring and metrics server and the hypervisors (ESXi and vCenter servers)

On remote services (Monolithic and Remote Services, but not V2V services), edit the abiquo.properties file to add the following properties.

# Enable/disable monitoring and alarms
abiquo.monitoring.enabled = true
 
# Address where watchtower is located
abiquo.watchtower.host=IP_OF_YOUR_WATCHTOWER_INSTALLATION

# Port where watchtower is listening
abiquo.watchtower.port=36638

Remember to restart your Abiquo Tomcat service.

Optional configuration for KVM hypervisor
Click here to expand...
Configure KVM properties
1. In the AIM properties file in the [stats] section to manage statistics for metrics and monitoring:
  1. collectFreqSeconds = 60
  2. refreshFreqSeconds = 30
  3. database = /var/lib/abiquo-aim.db
  See the AIM agent for further details.
2. To support metric dimensions in KVM through AIM, there is a related abiquo.properties entry called abiquo.vsm.measures.pusher.frequency. See Abiquo Configuration Properties#vsm
Within Abiquo, to configure fetch and display of metrics by default, go to Configuration → General and set the "Enable virtual machine monitoring by default" property. The new value will apply to all VMs created after you change the value.
Within Abiquo, assign users privileges to view and manage metrics, alarms, and alerts as required

Monitoring architecture

Abiquo Watchtower uses two types of storage: MySQL (the watchtower schema on the Abiquo MySQL database server) and KairosDB with Cassandra. For messaging, it uses the Abiquo RabbitMQ instances used by the Abiquo API server with queues for Evaluation and Notification.

Abiquo Watchtower has two main modules, called "abiquo-emmett" and "abiquo-delorean". These services receive metric data, and evaluate and notify alarms and alerts, as shown in the following diagram.

How to use VM monitoring, metrics, alarms and alerts

Note that the monitoring configuration files are stored under the /etc/abiquo/watchtower path.

The Abiquo UI displays the VM monitoring information, and enables you to manage alarms and alerts. See the following pages:

The monitoring information is also available through the API, and for information about alarms and alerts, see the following pages:

The functionality to push custom metrics is not available through the UI. See the API documentation pages for each data entity that supports custom metrics. For example, for a VM:

Similar methods are available for other resources.

Use collectd to gather metrics

Abiquo supports collectd for custom metrics collection on Linux VMs. The Abiquo-collectd-plugin automatically pushes the values gathered by collectd to the Abiquo API. There is a specific Abiquo API endpoint that the plugin uses to perform the push. As with other custom metrics, the Abiquo API then pushes the collectd metrics to the Abiquo Monitoring Server. You can use this method for hypervisors and providers where Abiquo does not support built-in metrics.

Install and configure collectd

To install and configure collectd

In Abiquo, the tenant administrator or user configures the collectd application (authentication and permissions) using OAuth. The user must have the “Allow user to push own metrics” privilege and assign this privilege to the plugin.
Users install collectd on their VMs, and then install the Abiquo collectd plugin (from source or using the Abiquo Chef cookbook).
The user configures their metrics in collectd.
The Abiquo collectd plugin automatically pushes all collectd metrics to the Abiquo API in PUTVAL JSON format.

Suggested Installation and Configuration Using Chef

Abiquo recommends the following use case:

The enterprise administrator, who manages the Chef server for the tenant, creates the monitoring application with only the "Allow user to push own metrics" privilege.
The enterprise administrator creates a role on the Chef server with the following:
1. The monitoring recipes to install the Abiquo collectd plugin
2. The OAuth tokens for the application configured
The users do not need to configure anything. They can just select the the appropriate role (from the default Chef view with no need to check individual recipes) and they are ready to go (no need to configure attributes in JSON).

This use case is recommended because the OAuth application belongs to the tenant administrator. This means that the configuration is already prepared for users by the enterprise administrator and it is also protected from users when they load their recipes, etc.

Links

Collectd website: http://collectd.org/
Abiquo-collectd plugin: https://github.com/abiquo/collectd-abiquo
Install Abiquo-collectd plugin with Abiquo Chef cookbook: https://github.com/abiquo/collectd-abiquo-cookbook
Abiquo API VM resource: VirtualMachinesResource

Browser not supported