In addition to the standard resource metrics offered by Abiquo, you can define your own custom accounting resource metrics. These metrics can be defined from existing data in the Abiquo database or from another source. For example, if you have access of OpenFlow or NetFlow network data for your Abiquo environment, you can import consolidated network data into a table/schema that is accessible to the Abiquo database. Then you can create a custom resource definition so that Abiquo will reference the network data and create network usage records in the Abiquo accounting_event_detail table. This will extend the current accounting system to account for network bandwidth usage.
Overview
- The UpdateAccounting stored procedure (located in the ‘kinton’ schema) is invoked.
- The UpdateAccounting procedure loops around, processing one account period at a time.
- When processing an account period, the ‘GenerateAccountingPeriodDataExt’ stored procedure is invoked to process the period:
- A number of temporary tables are created which contain a subset of data from the accounting_event_... tables, effectively including only rows relevant to the time period being generated.
- A temporary table is created to hold unconsolidated accounting_event_detail data, which is populated with data from the temporary accounting_event_... tables
- The unconsolidated temporary data is then consolidated into the main accounting_event_detail table by calling the ‘FLUSH_TMP_ACCOUNTING_EVENT_DETAIL’ stored procedure. The consolidation process works by identifying any duplicate resource definitions for the accounting period, and for each duplicate it accounts for the MAXIMUM resource identified. For example, if the number of CPUs is increased from 4 to 6 CPUs on a VM during the account period, then two rows will exist in the accounting_event_vm table for that VM. The consolidation process identifies the maximum CPU used 6, and accounts for that.
Any custom resources defined are processed when the above ‘core’ resources have been processed for a given account period. At that point, the accounting process looks for a well-known custom stored procedure, which if found, is invoked with details of the account period being processed.
The CUSTOM_GENERATE_ACCOUNTING_DATA_V1 Stored Procedure
The CUSTOM_GENERATE_ACCOUNTING_DATA_V1 stored procedure is the entry point for
generating custom accounting data. The procedure should be defined as follows:
CREATE PROCEDURE kinton_accounting.CUSTOM_GENERATE_ACCOUNTING_DATA_V1
(IN period_start TIMESTAMP,
IN period_end TIMESTAMP,
IN period_hours INTEGER,
IN granularity_hours INTEGER)
NOT DETERMINISTIC
SQL SECURITY DEFINER
BEGIN
# Write any custom metrics to the temporary table
# ...
# Then Flush them to the permanent table:
CALL FLUSH_TMP_ACCOUNTING_EVENT_DETAIL();
END
Note that there is a template version of the CUSTOM_GENERATE_ACCOUNTING_DATA_TEMPLATE_V1 procedure defined within the kinton_accounting schema, which can be used as the basis of any custom procedure.
When invoked, the custom stored procedure has access to the temporary accounting_event_... tables, and also to the (empty) tmp_accounting_event_detail table which holds unconsolidated temporary data. The custom stored procedure may access any Abiquo database data that it likes (as it is running in the context of the Abiquo system) and use it populate the unconsolidated temporary table. Subject to availability and configuration, it can also be used to access non-Abiquo DBMS data too. Finally, as the last thing it does, the custom procedure should call the accounting procedure used to flush the data from the temporary table into the main accounting table, which will automatically consolidate data in the process.
Custom Accounting resource types and resource ID generation
Any custom accounting resources must be uniquely identifiable by their resource name and resource ID. Therefore, in order to identify custom usages and to avoid clashes with any further extensions to the core Abiquo accounting resources, custom resource IDs MUST be allocated an ID in the numeric range -1 through -127.
Additionally, note that resource name is stored in the accounting_event_detail table (and its temporary equivalent table) using a name ID rather than the resource name itself. Therefore, you should ensure that the resource name is correctly added to the name table using the ‘ABQ_OBJECT_NAME_TO_ID’ function (please see the example below to see how the function can be used).
Finally, you should ensure that the names of any referenced VMs, VApps, VDCs, and Enterprises exist in the accounting name tables - this should not be a problem if all of the ‘core’ accounting resources are active, since they will populate the name tables for you - but if they are disable then be sure to add the names to the appropriate name tables.
Calculating the correct number of resource for the account period granularity
The accounting process now facilitates flexible accounting periods and granularities of resource usage. Clearly, with custom accounting metrics, resource usage can be accounted in an appropriate mechanism for that resource.
Since the accounting granularity is never smaller than one hour, internally, Abiquo represents the granularity in hours, and this can provide an easy way to calculate a count of resource usage for a resource, assuming that the resource has been in use for the entire accounting period. For example, to count the number of CPUs used by a given VM over the account period, you can perform the following simple calculation:
Number_of_VM_CPUs * (Account_Period_Hours / Account_Period_Granularity_Hours)
Abiquo upgrades and stored procedure handling
The Abiquo product will never overwrite or update any existing copy of a custom stored procedure during an upgrade. However, in the case that the existing procedure is known to be no longer compatible with the newer version of Abiquo, then the expected version of the stored procedure will be incremented and the old procedure will no longer be called - the old procedure will need to be adjusted to the required format of the new stored procedure before it will be invoked by the accounting procedure.
An example Custom Accounting metric
This section defines a simple customer accounting stored procedure, which should demonstrate how to use existing Abiquo DBMS data to generate additional accounting usage metrics. The example below shows how the data in the acccounting_event_vm table can be used to generate an accounting metric to count which indicates the number of VMs hosted in an Abiquo HA enabled environment (which may require a premium charge). The example procedure is given below:
DELIMITER |
CREATE PROCEDURE kinton_accounting.CUSTOM_GENERATE_ACCOUNTING_DATA_V1
(IN period_start TIMESTAMP,
IN period_end TIMESTAMP,
IN period_hours INTEGER,
IN granularity_hours INTEGER)
NOT DETERMINISTIC
SQL SECURITY DEFINER
BEGIN
INSERT INTO kinton_accounting.tmp_accounting_event_detail(
startTime,
endTime,
idAccountingResourceType,
resourceType,
resourceUnits,
resourceName,
idEnterprise,
idVirtualDataCenter,
idVirtualApp,
idVirtualMachine,
costCode,
idStorageTier)
SELECT DISTINCT
period_start,
period_end,
-1,
ABQ_OBJECT_NAME_TO_ID('VirtualMachine-haHosted'),
1 * (period_hours/granularity_hours),
ABQ_OBJECT_NAME_TO_ID(ABQ_VM_ID_TO_NAME(acc_vm.idVM)),
acc_vm.idEnterprise, # Assumes already in name table from 'core' processing
acc_vm.idVirtualDataCenter, # Assumes already in name table from 'core' processing
acc_vm.idVirtualApp, # Assumes already in name table from 'core' processing
acc_vm.idVM, # Assumes already in name table from 'core' processing
NULL,
NULL
FROM accounting_event_vm acc_vm
WHERE
# Is currently active, and was active prior to the end of the account period
((acc_vm.startTime <= period_end AND
(acc_vm.stopTime IS NULL OR acc_vm.stopTime >= period_end)) OR
# OR stopped running during the account period
((acc_vm.stopTime >= period_start) AND (acc_vm.stopTime < period_end))) AND
acc_vm.haEnabled=1;
# Finally, consolidate and write the entries from the TMP table into the
# accounting_event_detail table...
CALL FLUSH_TMP_ACCOUNTING_EVENT_DETAIL();
END|
DELIMITER ;