Abiquo accounting architecture

Introduction to accounting architecture

The Abiquo accounting architecture consists of accounting tables, triggers for data capture, and procedures.

This detailed diagram shows the basic architecture of the Accounting system.

Triggers and stored procedures

The Abiquo kinton schema contains triggers for accounting integration. The triggers run stored procedures in the kinton_accounting schema to automatically capture data in if/then/when scenarios.

update_virtualmachine_update_stats AFTER UPDATE ON virtualmachine

create_rasd_management_update_stats AFTER INSERT ON rasd_management

delete_rasd_management_update_stats AFTER DELETE ON rasd_management

update_ip_pool_management_update_stats AFTER UPDATE ON ip_pool_management

update_rasd_management_update_stats AFTER UPDATE ON rasd_management

create_vlan_network_update_stats AFTER INSERT ON vlan_network

delete_vlan_network_update_stats AFTER DELETE ON vlan_network

rack_update AFTER UPDATE ON rack

physicalmachine_create AFTER INSERT ON physicalmachine

physicalmachine_delete AFTER DELETE ON physicalmachine

physicalmachine_update AFTER UPDATE ON physicalmachine

virtualimage_create AFTER INSERT ON virtualimage

virtualimage_delete AFTER DELETE ON virtualimage

virtualimage_update AFTER UPDATE ON virtualimage

virtualimageconversion_update AFTER UPDATE ON virtualimage_conversion

Stored procedures for event accounting

The triggers collect the information necessary and call stored procedures responsible for recording accounting events. The stored procedures are:

AccountingVMRegisterEvents
AccountingStorageRegisterEvents
AccountingIPsRegisterEvents
AccountingVLANRegisterEvents

Accounting tables

Accounting triggers are always active. The resultant data is stored in event-based database tables that store resource data, and the time resource usage begins and ends.

These tables all follow a similar design pattern and are prefixed accounting_event_"resource", where "resource" can be substituted depending on the accounting resource type. For example, the data table on virtual machines (CPU, memory, storage) shown below.

Table accounting_event_vm

Table accounting_event_pm

This table stores details of reserved physical servers - one row for each reserved server.

 

Field Name	Field Type	Description
idPMAccountingEvent	BIGINT	Unique column identifier for the accounting event
idPhysicalMachine	INT	ID of the physical machine, as defined in the kinton.physicalmachine table. NOTE: Use the kinton_accounting ABQ_PM_ID_TO_NAME(id) function to obtain the machine name associated with the ID
idEnterprise	INT	ID of the ENTERPRISE, as defined in the kinton.enterprise table. NOTE: Use the kinton_accounting ABQ_ENT_ID_TO_NAME(id) function to obtain the machine name associated with the ID
cpu	INT	The number of physical CPUs in the machine
ram	INT	The physical machine’s RAM (in MB)
startTime	TIMESTAMP	The time at which the machine reservation started
stopTime	TIMESTAMP	The time at which the reservation finished.   This value is NULL if the reservation is still active
idDataCenter	INT	ID of the physical datacenter, as defined in the kinton.datacenter table. NOTE: Use the kinton_accounting ABQ_DC_ID_TO_NAME(id) function to obtain the datacenter name associated with the ID
consolidated	TINYINT	Indicates whether this row has been consoliated with other rows.   Currently this value is always 0
version_c	INT	Software controlled data version

 

Table accounting_event_repository

This table contains details of the repository usage. There is one row for each VM Template and Instance stored in the repository. Persistent VM templates are not included in this table, because they are stored as external volumes and charged separately.

 

Field Name	Field Type	Description
idRepoAccountingEvent	BIGINT	Unique column identifier for the accounting event
idImage	INT	ID of the repository image, as defined in the kinton.virtualimage table. NOTE: Use the kinton_accounting ABQ_IMG_ID_TO_NAME(id) function to obtain the image name associated with the ID
idEnterprise	INT	ID of the ENTERPRISE, as defined in the kinton.enterprise table. NOTE: Use the kinton_accounting ABQ_ENT_ID_TO_NAME(id) function to obtain the machine name associated with the ID
idImageTypeName	INT	The ID of the image type name in the generic accounting name table, use the kinton_accounting ABQ_OBJECT_ID_TO_NAME (id)  function to retrieve the actual type name.   Names are one of: INSTANCE, TEMPLATE, INSTANCE-CONVERSION, TEMPLATE-CONVERSION
idImageFormatName	INT	The ID of the image format name in the generic accounting name table, use the kinton_accounting ABQ_OBJECT_ID_TO_NAME (id)  function to retrieve the actual format name.   Names are: VMDK_FLAT, RAW, etc. see the wiki API reference for further details
imageSize	BIGINT	Physical Size of the repository image, in bytes
idRepository	INT	The ID of the image repository as defined in the kinton.repository table
idRepositoryName	INT	The ID of the repository name in the generic accounting name table, use the kinton_accounting ABQ_OBJECT_ID_TO_NAME (id)  function to retrieve the actual repository name. THIS FIELD WAS DEPRECATED IN ABIQUO 3.4
startTime	TIMESTAMP	The time at which the image was created in the repository
stopTime	TIMESTAMP	The time at which the image was deleted from the repository.  This value is NULL if the image still exists.
idDataCenter	INT	ID of the physical datacenter, as defined in the kinton.datacenter table. NOTE: Use the kinton_accounting ABQ_DC_ID_TO_NAME(id) function to obtain the datacenter name associated with the ID.
consolidated	TINYINT	Indicates whether this row has been consoliated with other rows.   Currently this value is always 0.
version_c	INT	Software controlled data version.

List of accounting event tables

The accounting event tables include the following.

• backup
• ds_storage
• firewall
• ips
• loadbalancer
• pm
• protect
• repository
• storage
• vlan
• vm
• vm_off
• vm_on

accounting_event_detail table

The accounting_event_detail table contains the information needed for accounting and billing functions. This table should not be accessed directly - if you need to access non-aggregated data then please use the kinton_accounting schema ACCOUNT_PERIOD_USAGE_VW (or the HOURLY_USAGE_MAX_VW or HOURLY_USAGE_MAX_2_VW) views to access the content of this table.

Warning: Running unoptimized or long SQL queries against this table can have a significant impact on the overall performance of the transactional database, and might affect the proper functioning of AbiquoServer. Therefore Abiquo recommends the use of database replication to replicate this table to another database server which has been prepared to process OLAP type queries.

The following rows are returned from ACCOUNT_PERIOD_USAGE_VW view:

Every hour, the status of each resource implemented in the system is recorded, which can generate a large number of rows. Abiquo recommends that the table is periodically purged of resource information which is no longer required.

Resource IDs

The resources in the accounting_event_detail table are represented with a numerical value, as defined in the table below in the idAccountingResourceType column.

Resource owners

Abiquo records resource ownership at the following levels:

• virtual machine
• virtual datacenter 
• enterprise

Description of resource owners

1. Virtual Machine

• Cores
• RAM
• Local storage

• Cost codes (if the VM template has a cost code)

  Grouped virtual machine components and virtual machines are accounted per group, not per individual virtual machine.

2. Virtual Appliance

• Groups of virtual machines
  
  • Cores
  • RAM
  • Local storage

  • Cost codes (if the VM template has a cost code)

    Grouped virtual appliances are accounted per group, not per individual virtual appliance.

3. Virtual Datacenter

• Groups of virtual machines
• Groups of virtual appliances
  
  • Cores
  • RAM
  • Local storage
  • VLANS
  • Public IPs

  • External storage

    Virtual Datacenter accounting is the total of the resources reserved and/or consumed by the Virtual Machines, Virtual Appliances, and users of a Virtual Datacenter.

4. Enterprise
An Enterprise has no managed resources that do not belong to a Virtual Datacenter, so there is no accounting per Enterprise. However, the Enterprise associated with each resource is recorded for aggregating resources at Enterprise level, and this information can be retrieved through the database views.

Legacy accounting views

For accounting and billing, Abiquo has previously implemented database views (in both the kinton and kinton_accounting schemas) to help you access the resource usage information correctly. However, the use of these views is no longer recommended for scalability and performance reasons. Instead, use the equivalent queries described above.

ACCOUNT_PERIOD_USAGE_VW - Returns the resource use for the accounting period with the maximum values. If the same resource has been used more than once in the same time zone, it takes the greater value. Not grouped by virtual machine and virtual appliance. The columns returned ar the same as the HOURLY_USAGE_MAX_2_VW but the name has been changed to reflect the change to the accounting period.

Legacy maximum usage views

• HOURLY_USAGE_MAX_VW - Returns the resource use for hours with their maximum values. If the same resource has been used more than once in the same time zone, it takes the greater value. Not grouped by virtual machine and virtual appliance.
• HOURLY_USAGE_MAX_2_VW

Legacy sum views

• HOURLY_USAGE_SUM_VW - Returns the resource use for hours with their values added together. The data is aggregated to virtual datacenter level, but can be further aggregated to enterprise level if required.
• DAILY_USAGE_SUM_VW - Returns the resource use for days with their values added together. The data is aggregated to virtual datacenter level, but can be further aggregated to enterprise level if required.
• MONTHLY_USAGE_SUM_VW - Returns the resource use for months with their values added together. The data is aggregated to virtual datacenter level, but can be further aggregated to enterprise level if required.

Configuration view

• ACCOUNTING_CONFIG_VW  - Returns the current accounting configuration as a single row