| Table of Contents |
|---|
Introduction to Abiquo
Abiquo is a cloud computing platform that enables you to manage public clouds and private clouds. You can use your own physical infrastructure to provide infrastructure as a service. It is a hypervisor agnostic model (no vendor lock-in) and it provides a unified interface for managing:
VM templates
Compute
Network
Storage
Events
Abiquo datacenters
In private cloud, a datacenter consists of:
A set of hypervisors
Storage servers (for optional managed storage)
Network File System (NFS)
DHCP
Abiquo remote services
...
Abiquo is designed to manage multiple datacenters and public cloud providers.
...
Abiquo manages multiple hypervisors and conversions between hypervisors.
Managed by the vendor-provided API:
VMware
Hyper-V
Managed by the Abiquo AIM agent using libvirt
KVM
And Abiquo also works with public cloud providers.
...
The Abiquo platform uses Remotes Services to manage the hypervisors. The Abiquo remote services found in each datacenter are:
Appliance manager
Business process manager
Discovery manager
Virtualization manager
Virtual system monitor
Remote access manager
Services manager
Appliance manager
Provides the ability to upload and download VM templates into repositories available to the datacenter. Administrators may use the AM API to manage templates.
...
Provides conversion of VM templates between disk formats supported by hypervisors and export to public cloud providers, thus eliminating vendor lock-in.
Conversion requests and responses are submitted via message queues
Requests are processed one at a time, because working at NFS level implies being careful of I/O conflicts
Uses RabbitMQ as a message broker
Discovery manager
Provides hypervisor discovery of the following:
Hypervisor type
Deployed VMs
Physical machine capabilities and resources
Uses the API provided by the hypervisor vendor.
...
The Abiquo Server manages an arbitrary number of datacenters by accessing their remote services. Communications between modules and datacenters are performed via RabbitMQ. MySQL stores the system configuration and state. The platform also offers enterprise functionality: pricing, scheduling, security. The server exposes its data and operations via a REST API.
Architecture diagram
The Abiquo architecture components fit together as shown in the following diagram.
...
Technology stack
The Abiquo technology stack includes the following projects.
Apache projects
Thrift
Commons
Others
MySQL (MariaDB)
Redis
RabbitMQ
Akka
Libvirt
Jersey
Virtual machine operations
...
This document assumes a good working knowledge of the Abiquo platform.
Advantages
Scalability
asynchronous VM operations
independent
concurrent
Traceability
simple, standardized VM definition for all hypervisors
error messages and logs
monitoring console
Performance
non-blocking operations
Stability
improved reconfigure
improved rollback
standardized and unified hypervisor communication
automated testing
Scalability and stability
...
Virtual machine and hypervisor actions
Hypervisor Connection
login
logout
Virtual Machine
configure
reconfigure
unconfigure
snapshot
get state
power on
power off
stop
resume
Virtual machine states diagram
See VM and VApp states.The VMs can transition between states as shown in the following diagram.
...
See Virtual appliance states and Virtual machine states table
Deployment scenario
To deploy a virtual appliance containing N virtual machines, we operate directly on each VM. We perform
N virtual machine configurations
N power-on requests
We wish to run these N configurations in parallel (as resources permit) and request a power-on of each VM after configuration. These operations are concurrent and independent.
...
The virtual factory coordinates these tasks.
...
Each request to the virtual factory is a complete entity and includes the hypervisor connection to avoid reconnects.
A request contains:
tasks and jobs
virtual machine definitions
hypervisor connection
...
The Server/API and virtual factory communicate using RabbitMQ which uses AMQP.
...
Each datacenter has its own IN and OUT queue. The queue name is configured with the Abiquo configuration property abiquo.datacenter.id for the Remote Services (V2V/BPM and virtual factory). This property identifies the AMQP datacenter queues. It is needed for building routing keys and queue names, so that they are unique for each datacenter. Do not change this property! The value of this property is generated at installation, but you can recover the property through the API with Get Datacenter.
...
The components of the virtual factory and their basic functions are as follows.
Queue manager: handles incoming queue messages
Task orchestrator: decides when to send a job
Virtualization worker: executes the VM operation
Many jobs can run concurrently with multiple instances of the virtualization worker, but the number of jobs in a specific installation is dependent on both the available resources and the configuration.
The number of concurrent sessions is limited by resources because you cannot have more open sessions than the number of cores in your hypervisor host. It can also be controlled at the platform level, and for hypervisors/providers and for VDCs using the Abiquo Configuration Properties. For example, for the platform, the abiquo.virtualfactory.openSession property controls the default number of simultaneous operations on a single hypervisor or provider, and by default it is set to 2. To configure the number of concurrent connections, see Control the number of concurrent operations.
...
Traceability
...
The diagram below shows the interaction between the elements of the virtual factory in dealing with requests and responses.
Queue Manager: handles incoming queue messages
Task Orchestrator: decides when to send a job
Virtualization Worker: executes the VM operation
...
Queue manager
Receives task message from the API or Server using the RabbitMQ connection.
checks the message and sends the task to the task orchestrator
Receives task response (from task orchestrator) then acknowledges the AMQP message
the message is deleted from the queue
Task orchestrator
Receives task messages
saves the current task message
saves a map of task IDs to jobs IDs
if it's a sequential task, send the first job to a VW
it it's a concurrent task, send all the jobs to many VWs
if the job being sent is a task, send to itself
Receives job results
notify end of job to the outgoing AMQP queue
check if the task is completed, then check if it should notify itself or the Queue Manager
if not completed and a sequential task, send the next job
Receives task results
only intermediate task results, because task is also a job.
perform the same logic as a Job Result
Virtualization worker
Receives job messages
create a new instance of Hypervisor Connection and perform the VM operation
send back a Job Result when it's done