Custom accounting resource definitions

1 Overview
2 The CUSTOM_GENERATE_ACCOUNTING_DATA_V1 stored procedure
3 Identify custom resource types for Abiquo
4 Calculating the correct number of resources for the account period granularity
5 Abiquo upgrades and stored procedure handling
6 An example custom accounting metric

Abiquo offers standard resource metrics and you can also define your own as described in this document. You can define custom metrics using Abiquo data or from another source.

Use case to account for network bandwidth usage

If you have OpenFlow or Netflow network data for your platform, you can import consolidated network data into a table or schema for the Abiquo database. Then create a custom resource definition for it. Abiquo will read the data and create custom network usage records in the accounting_event_detail table. You can then use this data to bill your customers for their network bandwidth usage.

Overview

The UpdateAccounting stored procedure (located in the ‘kinton’ schema) runs.
The UpdateAccounting procedure processes one account period at a time.
It runs the ‘GenerateAccountingPeriodDataExt’ to process the period:
- It creates temporary tables with a subset of data from the accounting_event_... tables for the time period
- It creates a temporary table to hold unconsolidated accounting_event_detail data, which is populated with data from the temporary accounting_event_... tables
- It consolidates the unconsolidated temporary data into the main accounting_event_detail table by calling the ‘FLUSH_TMP_ACCOUNTING_EVENT_DETAIL’ stored procedure.
  - For each duplicate resource usage it accounts for the MAXIMUM resource identified. For example, if the number of CPUs is increased from 4 to 6 CPUs on a VM during the account period, then two rows will exist in the accounting_event_vm table for that VM. The consolidation process identifies the maximum CPU used 6, and accounts for that.

When the above ‘core’ resources have been processed for a given account period the platform process the custom resources. The accounting process looks for the custom stored procedure and runs it with details of the account period.

The CUSTOM_GENERATE_ACCOUNTING_DATA_V1 stored procedure

The CUSTOM_GENERATE_ACCOUNTING_DATA_V1 stored procedure is the entry point for generating custom accounting data.
The procedure should be defined as follows:

CREATE PROCEDURE kinton_accounting.CUSTOM_GENERATE_ACCOUNTING_DATA_V1
(IN period_start TIMESTAMP,
 IN period_end TIMESTAMP, 
 IN period_hours INTEGER,
 IN granularity_hours INTEGER)
    NOT DETERMINISTIC
    SQL SECURITY DEFINER
BEGIN
	# Write any custom metrics to the temporary table
	# ...
	# Then Flush them to the permanent table:
	CALL FLUSH_TMP_ACCOUNTING_EVENT_DETAIL();
END

To define a custom stored procedure, start with the template version of the CUSTOM_GENERATE_ACCOUNTING_DATA_TEMPLATE_V1 defined within the kinton_accounting schema.

When you run the custom stored procedure, it has access to the temporary accounting_event_... tables, and also to the (empty) tmp_accounting_event_detail table which holds unconsolidated temporary data. The custom stored procedure may access any Abiquo database data (because it is running in the context of the Abiquo system) and use it populate the unconsolidated temporary table. Depending on availability and configuration, the custom stored procedure can access non-Abiquo DBMS data too. Finally, as the last thing it does, the custom procedure should call the accounting procedure used to flush the data from the temporary table into the main accounting table, which will automatically consolidate data in the process.

Identify custom resource types for Abiquo

Abiquo needs to uniquely identify custom accounting resources by their resource name and resource ID. And it will need to avoid clashes with any further extensions to the core Abiquo accounting resources. Abiquo stores resource names in the accounting_event_detail table (and its temporary equivalent table) using a name ID rather than the resource name itself.

Give each custom resource a unique name
Set a unique custom resource IDs as a numeric value in the range of -1 to -127
Use the 'ABQ_OBJECT_NAME_TO_ID' function to add the reource name to the name table
If any of the core accounting resources are disabled, add the names of VMs, VApps, VDCs, and Enterprises to the appropriate name tables

Calculating the correct number of resources for the account period granularity

The accounting process now facilitates flexible accounting periods and granularities of resource usage. Clearly, with custom accounting metrics, resource usage can be accounted in an appropriate mechanism for that resource.
Since the accounting granularity is never smaller than one hour, internally, Abiquo represents the granularity in hours, and this can provide an easy way to calculate a count of resource usage for a resource, assuming that the resource has been in use for the entire accounting period. For example, to count the number of CPUs used by a given VM over the account period, you can perform the following simple calculation:

Number_of_VM_CPUs *
(Account_Period_Hours / Account_Period_Granularity_Hours)

Abiquo upgrades and stored procedure handling

The Abiquo product will never overwrite or update any existing copy of a custom stored procedure during an upgrade. However, in the case that the existing procedure is known to be no longer compatible with the newer version of Abiquo, then the expected version of the stored procedure will be incremented and the old procedure will no longer be called - the old procedure will need to be adjusted to the required format of the new stored procedure before it will be invoked by the accounting procedure.

An example custom accounting metric

This section defines a simple customer accounting stored procedure, which should demonstrate how to use existing Abiquo DBMS data to generate additional accounting usage metrics. The example below shows how the data in the acccounting_event_vm table can be used to generate an accounting metric to count which indicates the number of VMs hosted in an Abiquo HA enabled environment (which may require a premium charge). The example procedure is given below:

DELIMITER |
CREATE PROCEDURE kinton_accounting.CUSTOM_GENERATE_ACCOUNTING_DATA_V1
(IN period_start TIMESTAMP, 
 IN period_end TIMESTAMP, 
 IN period_hours INTEGER,
 IN granularity_hours INTEGER)
    NOT DETERMINISTIC
    SQL SECURITY DEFINER
BEGIN
  INSERT INTO kinton_accounting.tmp_accounting_event_detail(
    startTime,
    endTime, 
    idAccountingResourceType,
    resourceType,
    resourceUnits,
    resourceName, 
    idEnterprise, 
    idVirtualDataCenter, 
    idVirtualApp, 
    idVirtualMachine, 
    costCode,
    idStorageTier)
    SELECT DISTINCT
        period_start,
        period_end,
        -1,
        ABQ_OBJECT_NAME_TO_ID('VirtualMachine-haHosted'),
        1 * (period_hours/granularity_hours),
        ABQ_OBJECT_NAME_TO_ID(ABQ_VM_ID_TO_NAME(acc_vm.idVM)),
        acc_vm.idEnterprise, # Assumes already in name table from 'core' processing
        acc_vm.idVirtualDataCenter, # Assumes already in name table from 'core' processing
        acc_vm.idVirtualApp, # Assumes already in name table from 'core' processing
        acc_vm.idVM, # Assumes already in name table from 'core' processing
        NULL,
        NULL
    FROM accounting_event_vm acc_vm
    WHERE
        # Is currently active, and was active prior to the end of the account period
        ((acc_vm.startTime <= period_end AND 
         (acc_vm.stopTime IS NULL OR acc_vm.stopTime >= period_end)) OR
        # OR stopped running during the account period
        ((acc_vm.stopTime >= period_start) AND (acc_vm.stopTime < period_end))) AND
        acc_vm.haEnabled=1;


    # Finally, consolidate and write the entries from the TMP table into the 
    # accounting_event_detail table...
    CALL FLUSH_TMP_ACCOUNTING_EVENT_DETAIL();
END|
DELIMITER ;