Introduction

The Abiquo accounting service is a usage metering system. It was originally designed to enable Abiquo customers to charge their users on a pay per use basis for private cloud resources. The Abiquo accounting service records how much of each resource a tenant uses in a virtual datacenter or datacenter. For example, if a user deploys 5 VMs for a total of 25 hours each, then by default the system will record 125 VM hours during the accounting period. The resources to account are fully configurable and you can also create your own custom resources and queries for managed services. You can use the Abiquo billing integration to create a file for your billing system.

This is a basic technical overview of the Abiquo accounting system:

1. Abiquo records when users reserve or use resources in the kinton_accounting DB.
2. The administrator configures which resources are accounted and the accounting periods used for it.
3. Some accounting procedures are run periodically to consolidate/update resource usage and purge old records.

Accounting information backup

The accounting master tables are in kinton_accounting DB. Include these tables in your backup procedures:

• accounting_event_ips
• accounting_event_pm
• accounting_event_repository
• accounting_event_storage
• accounting_event_vlan
• accounting_event_vm
• accounting_event_ds_storage
• accounting_event_vm_on
• accounting_event_vm_off
• accounting_event_backup
• accounting_event_firewall
• accounting_event_loadbalancer

Accounting parameters

To retrieve the current accounting parameters, run the query below in kinton_accounting DB:

SELECT * FROM accounting_parameters;

+---------+-------------------------------------+------------+
| idParam | paramName                           | paramValue |
+---------+-------------------------------------+------------+
|       1 | AccountingEnabled                   | 1          |
|       2 | DeleteRegEventsUseSPParam           | 0          |
|       3 | DeleteRegEventsDeleteHours          | 26280      |
|       4 | MaximumPeriodsToFirstInit           | 1          |
|       5 | MaximumPeriodsToProcess             | 24         |
|       6 | AccountPeriodSize                   | HOUR       |
|       7 | AccountPeriodGranularity            | HOUR       |
|       8 | LoggingLevel                        | INFO       |
|       9 | VirtualMachine-vcpu                 | 1          |
|      10 | VirtualMachine-vram                 | 1          |
|      11 | VirtualMachine-vhd                  | 1          |
|      12 | VirtualMachine-vtierhd              | 1          |
|      13 | VirtualMachine-backup               | 1          |
|      14 | ExternalStorage                     | 1          |
|      15 | IPAddress                           | 1          |
|      16 | VLAN                                | 1          |
|      17 | VirtualMachine-hypervisorType       | 1          |
|      18 | Firewall                            | 1          |
|      19 | LoadBalancer                        | 1          |
|      20 | VirtualMachine-vcpuon               | 1          |
|      21 | VirtualMachine-vcpuoff              | 1          |
|      22 | VirtualMachine-vramon               | 1          |
|      23 | VirtualMachine-vramoff              | 1          |
|      24 | VirtualMachine-hp                   | 1          |
|      25 | VirtualMachine-hpon                 | 1          |
|      26 | VirtualMachine-hpoff                | 1          |
|      27 | VirtualMachine-haHosted             | 0          |
|      28 | ReservedPhysicalMachine-cpu         | 0          |
|      29 | ReservedPhysicalMachine-ram         | 0          |
|      30 | RepositoryStorage                   | 0          |
|      31 | VirtualMachine-antiAffinity         | 0          |
|      32 | Consolidation-time-sensitivity-secs | 30         |
|    1419 | draas_protection                    | 1          |
+---------+-------------------------------------+------------+

To check the value of an accounting parameter, use the query below:

SELECT * FROM accounting_parameters WHERE paramName="PARAM_NAME";

SELECT * FROM accounting_parameters WHERE paramName="VirtualMachine-hypervisorType";
+---------+-------------------------------+------------+
| idParam | paramName                     | paramValue |
+---------+-------------------------------+------------+
|      17 | VirtualMachine-hypervisorType | 0          |
+---------+-------------------------------+------------+
1 row in set (0.00 sec)

Description of resources

Abiquo stores this data about the use of these resources in the Accounting events tables. Then Administrators can configure which resources are accounted in the Accounting event detail table.

Key

• - Resource accounted

•  - Deprecated
• Blue shading - disabled by default
• Pink shading - custom resource

Enable or disable accounting of resources

Abiquo always tracks all resources in the accounting master tables, but Administrators can configure which resources will be consolidated for billing and reporting in the accounting_event_detail table.

To select which resources Abiquo will record in this table, enable/disable the corresponding accounting parameters by setting the corresponding accounting parameter value.

UPDATE accounting_parameters SET paramValue=1 WHERE paramName="VirtualMachine-antiAffinity";

Accounting time parameters

In the accounting_event_detail table, Abiquo stores resource usage information by account period size and account period granularity. Always set these parameters before allowing users to work with your system.

Account period size parameter

For each accounting period, the update procedure checks the Accounting event tables for resources that have been reserved and/or used in the last period by granularity and records the usage data in the accounting_event_detail table.

The minimum period of consumption of a resource is always one accounting period. If a resource is active for an interval of less than one accounting period, the system automatically rounds consumption up to one accounting period. But please note that the value stored will depend on the granularity.

Resources may be turned on and off during an accounting period. Abiquo stores all these events in the Accounting master tables. However, accounting_event_detail only contains the maximum value for an accounting period.

You can set the accounting period size parameter in the accounting_parameters table.

UPDATE accounting_parameters SET paramValue="WEEK" WHERE paramName="AccountPeriodSize";

Accounting period granularity

For each accounting period, the Update procedure checks the master tables for resources reserved and/or consumed in the last period by granularity  and records the usage data in the Accounting event detail table.

The value stored for each accounting period can be divided into smaller time units using the granularity. However, the granularity does not affect the minimum usage.

The minimum period of usage is always determined by the accounting period:

• If accounting period = DAY and granularity = HOUR, then for a single CPU VM, Abiquo would record 24 resource units for a day record. So this could represent a charge for 24 hours of vCPU usage.
• If accounting period = DAY and granularity = DAY, then Abiquo will record 1 resource unit. So this could represent a charge for 1 day of vCPU usage.

You can set the granularity parameter in the accounting_parameters table.

The following table shows compatibles periods and granularities. Abiquo uses calendar months, so week granularity is not compatible with month.

UPDATE accounting_parameters SET paramValue="DAY" WHERE paramName="AccountPeriodGranularity";

Change the accounting period parameters in an existing system

To change the accounting period, follow these steps: 

1. Backup the accounting_event_detail table.
2. Delete the existing content of the accounting_event_detail table.
3. Change the accounting parameters to the desired values. Set MaximumPeriodsToFirstInit to ensure old data is accounted correctly.
4. Monitor the content of accounting_event_detail to ensure the generated data meets your needs. If not, then repeat from step 2.

Accounting periodic procedures

The accounting system runs the accounting procedures below periodically to:

• UpdateAccounting processes the data in the accounting master tables to consolidate the resource usage in the accounting_event_detail table.
  
• DeleteOldRegisteredEvents deletes data in the accounting master tables older than the DeleteRegEventsDeleteHours accounting parameter (26280 hours or 3 years by default)

These procedures are configured in the /etc/cron.d/abiquo-accounting cron job. Note that DeleteOldRegisteredEvents does not use its legacy parameter to set the maximum record age by default.

To view the cron job configuration for the accounting procedures, run the command:

cat /etc/cron.d/abiquo-accounting  

# Runs every hour
0 * * * * root mysql -uroot -hlocalhost -e "CALL kinton.UpdateAccounting();"
# Runs every Sunday at 12:00 and deletes records older than a week.
0 12 * * 0 root mysql -uroot -hlocalhost -e "CALL kinton.DeleteOldRegisteredEvents(168);"

Set the frequency of procedure run to delete old records

To change the maximum record age of the accounting master table records, set the DeleteRegEventsDeleteHours parameter in accounting_parameters table as desired.

It is also possible to use the stored procedure parameter as maximum record age by enabling the DeleteRegEventsUseSPParam parameter in accounting parameters, but it is not recommended to use this method.

UPDATE accounting_parameters SET paramValue="17520" WHERE paramName="DeleteRegEventsDeleteHours";

Retrieve Accounting data

Use the following queries as templates for creating queries to view aggregate resource usage as required. These queries should retrieve information from the replicated accounting database.

Previously the accounting system used views, which have been maintained for legacy applications. See Abiquo accounting architecture.

SELECT
  v.startTime AS startTime,
  v.endTime AS endTime,
  v.idAccountingResourceType AS idAccountingResourceType,
  v.resourceType AS resourceType,
  SUM(v.resourceUnits) AS resourceUnits,
  v.idEnterprise AS idEnterprise,
  v.idVirtualDataCenter AS idVirtualDataCenter,
  v.enterpriseName AS enterpriseName,
  v.virtualDataCenter AS virtualDataCenter
FROM
  kinton_accounting.HOURLY_USAGE_MAX_VW v
WHERE
  # APPLY CUSTOM QUERY FILTERS HERE, E.G. BY DATE RANGE AS BELOW
  v.startTime >= '2012-04-16' AND v.endTime < '2012-04-17'
GROUP BY
  v.startTime,
  v.idAccountingResourceType,
  v.idEnterprise, v.idVirtualDataCenter;

SELECT
  CAST(a.startTime AS DATE) AS startTime,
  CAST(a.startTime AS DATE) AS endTime,
  a.idAccountingResourceType AS idAccountingResourceType,
  a.resourceType AS resourceType,
  SUM(a.resourceUnits) AS resourceUnits,
  a.idEnterprise AS idEnterprise,
  a.idVirtualDataCenter AS idVirtualDataCenter,
  a.enterpriseName AS enterpriseName,
  a.virtualDataCenter AS virtualDataCenter
FROM
  kinton_accounting.HOURLY_USAGE_MAX_VW a
WHERE
  # APPLY CUSTOM QUERY FILTERS HERE, E.G. BY DATE RANGE AS BELOW
  a.startTime >= '2012-04-16' AND a.endTime < '2012-04-17'
GROUP BY
  CAST(a.startTime AS DATE),
  a.idAccountingResourceType,
  a.idEnterprise, a.idVirtualDataCenter;

SELECT
  CAST((a.startTime - INTERVAL (DAYOFMONTH(a.startTime) - 1) day) AS DATE) AS startTime,
  LAST_DAY(a.startTime) AS endTime,
  a.idAccountingResourceType AS idAccountingResourceType,
  a.resourceType AS resourceType,
  SUM(a.resourceUnits) AS resourceUnits,
  a.idEnterprise AS idEnterprise,
  a.idVirtualDataCenter AS idVirtualDataCenter,
  a.enterpriseName AS enterpriseName,
  a.virtualDataCenter AS virtualDataCenter
FROM
  kinton_accounting.HOURLY_USAGE_MAX_VW a
WHERE
  # APPLY CUSTOM QUERY FILTERS HERE, E.G. BY DATE RANGE AS BELOW
  a.startTime >= '2012-04-01' AND a.endTime < '2012-05-01'
GROUP BY
  CAST((a.startTime - INTERVAL (dayofmonth(a.startTime) - 1) day) AS DATE),
  a.idAccountingResourceType,
  a.idEnterprise, a.idVirtualDataCenter;

Accounting consistency checks

You can use the Abiquo accounting consistency check to monitor your accounting system.

See Abiquo accounting consistency check