Page Comparison

...

Public cloud billing check tool

Tool that This tool checks public cloud billing and customer to Abiquo enterprise mapping.

Change history

Build the tool

You can run this tool on the Abiquo Server. The project is built using Maven. You can build it with the following command:

mvn clean install

This will output a .jar script as follows.

target/public-cloud-billing-check-tool.jar

Configuration

To use this tool in AWS, update your billing-api-user for Abiquo to add the following AWS IAM policy:

"Effect": 
	"Allow", 
	"Action": [ "ce:*", ], 
"Resource": "*"

For both providers, add your connection details to the YAML configuration file

1. Go to /opt/abiquo-public-cloud-billing-check-tool/config/

2. Edit the config.yml file and complete the values for your system.

   Code Block
   language none
   Abiquo_database: host: localhost user: password: port: 3306 database: kinton socket: /var/lib/mysql/mysql.sock abiquo: api_url: 'http://localhost/api' version: '6.0' user: #Basic Auth credentials password: #OAuth credentials access-token: access-token-secret: consumer-key: consumer-secret:

General usage

The tool has two main commands to run on Azure or AWS.

...

...

Install the tool

You can run this tool on the Abiquo Server.

In Abiquo >= 6.0.1, you can install the package abiquo-billing-public-cloud-check-tool with yum.

The default installation directory is /opt/abiquo-public-cloud-billing-check-tool.

...

Amazon usage

When running the tool on Amazon, you can use the -h option to print the parameters, as shown here

And the tool file name is public-cloud-billing-check-tool.jar

Configuration

To use this tool in AWS, update your billing-api-user for Abiquo to add access to AWS Cost Explorer.

For example, for full access you can use the following AWS IAM policy:

"Effect": 
	"Allow", 
	"Action": [ "ce:*", ], 
"Resource": "*"

For both providers, add your connection details to the YAML configuration file

1. Go to /opt/abiquo-public-cloud-billing-check-tool/config/

2. Edit the config.yml file and complete the values for your system.

   Code Block
   language none
   Abiquo_database: host: localhost user: password: port: 3306 database: kinton socket: /var/lib/mysql/mysql.sock abiquo: api_url: 'http://localhost/api' version: '6.0' user: #Basic Auth credentials password: #OAuth credentials access-token: access-token-secret: consumer-key: consumer-secret:

General usage

The tool has two main commands to run on Azure or AWS.

java -jar public-cloud-billing-check-tool.jar aws
-hhelp Usage: <main class> aws [-abhvVhV] [-spp] [-cfp=<configYAMLPath>] [-d=<resellerDiscount>] [-m=<margin>] [-mt=<month>] -r=<rootEnterpriseId> [-y=<year>]
Check for AWS organization billing costs and account mappings.

-a, --accounts             Run the check for accounts inside an organization mapped to abiquo
-b, --billing              Run check for billing information in Abiquo for the last consolidated billing period
-cfp, --configPath=<configYAMLPath>
              COMMAND] 
Check public cloud billing information. 
-h, --help      Show this help message and exit.
-V, --version   Print version information and exit.
Commands: 
azure  Check for Azure CSP customer costs and subscription mappings.
aws    Check for AWS organization billing costs and account mappings.

Amazon usage

When running the tool on Amazon, you can use the -h option to print the parameters, as shown here.

java -jar public-cloud-billing-check-tool.jar aws -h
Usage: <main class> aws [-abhvV] [-spp] [-cfp=<configYAMLPath>] [-d=<resellerDiscount>] [-m=<margin>] [-mt=<month>] -r=<rootEnterpriseId> [-y=<year>]
Check for AWS organization billing costs and account mappings.

-a, --accounts         Specify a different path forRun the configcheck file.for Defaultaccounts pathinside isan theorganization samemapped whereto .jar is located
-dabiquo
-b, --discount=<resellerDiscount>billing              Run check for billing information in Abiquo    Definefor the discountlast necessaryconsolidated to subtract to total costbilling period
-hcfp, --helpconfigPath=<configYAMLPath>
                Show this help message and exit. -m, --margin=<margin>    Specify a Desireddifferent marginpath tofor comparethe costconfig fromfile. AWSDefault andpath Abiquo database. If difference is bigger than margin, it will be consideredis the same where .jar is located
-d, --discount=<resellerDiscount>
                           Define the discount asnecessary anto error.subtract Defaultto istotal 0cost
-mth, --month=<month>help       Month of the billing period to check. Default is previous month
-rShow this help message and exit.
-m, --rootEnterpriseIdmargin=<rootEnterpriseId><margin>      Desired margin to compare cost from AWS and Abiquo database. If difference is bigger than margin, it will be considered
Enterprise id of the reseller or key node from which obtain the pricing credentials -spp, --sppDiscount        If true, include solution    provideras programan discountserror. inDefault theis price0
-vmt, --verbosemonth=<month>       Month of the billing period to check. DisplayDefault moreis detailed information about the process executionprevious month
-Vr, --versionrootEnterpriseId=<rootEnterpriseId>
              Print version information and exit. -y, --year=<year>       Enterprise Yearid of the reseller billingor periodkey tonode check.from Defaultwhich isobtain the yearpricing ofcredentials
the previous month

Only root enterprise option is required. An example of execution using all parameters will be: java -jar public-cloud-billing-check-tool.jar aws -a -b -r=1

With the "verbose" option, the tool will display a detailed list of all accounts mapped to Abiquo and the respective enterprises that contain public cloud credentials of each account.

With the "accounts" option, the tool will check for accounts that are:

1. Used in more than one enterprise in Abiquo

2. Mapped to the reseller enterprise hierarchy but not present in the reseller organization

3. Present in the reseller organization but not present mapped to Abiquo

If "verbose" is also selected, it will also print all the accounts of the organization.

With the "billing" option, the the tool will check the difference between AWS cost and Abiquo costs for each mapped account. If "verbose" is activated, the script will print the difference for all enterprises, otherwise it will only print entries with cost difference between AWS and Abiquo.

Azure usage

You can use this tool if you have a set of Azure Plans subscriptions mapped to enterprises. The tool has two main functions:

• Check that all subscriptions from a CSP Azure Plan are mapped correctly to Abiquo enterprises

• For Azure billing, check the costs of the last month at customer level, not subscription level.
  

To run the tool for Azure with the help option to discover the parameters, run the following command:

java -jar public-cloud-billing-check-tool.jar azure -h

Here is an example of the help parameters output. The "-r" parameter is mandatory.

-azp (--azureplan)      : Run the check for Azure Plan subscriptions in Abiquo-spp, --sppDiscount        If true, include solution provider program discounts in the price
-v, --verbose              Display more detailed information about the process execution
-V, --version              Print version information and exit.
-y, --year=<year>          Year of the billing period to check. Default is the year of the previous month

The -r root enterprise option is required and the other parameters are optional.

An example of execution using all parameters is: java -jar public-cloud-billing-check-tool.jar aws -a -b -r=1.

With the verbose option, the tool will display a detailed list of all accounts mapped to Abiquo and the respective enterprises that contain public cloud credentials of each account.

With the accounts option, the tool will check for accounts that are:

1. Used in more than one enterprise in Abiquo

2. Mapped to the reseller enterprise hierarchy but not present in the reseller organization

3. Present in the reseller organization but not present mapped to Abiquo

If verbose is also selected, it will also print all the accounts of the organization.

With the billing option, the the tool will check the difference between AWS cost and Abiquo costs for each mapped account. If verbose is activated, the script will print the difference for all enterprises, otherwise it will only print entries with a cost difference between AWS and Abiquo.

...

Azure usage

You can use this tool if you have a set of Azure Plans subscriptions mapped to enterprises. The tool has two main functions:

• Check that all subscriptions from a CSP Azure Plan are mapped correctly to Abiquo enterprises

• For Azure billing, check the costs of the last month at customer level, not subscription level.
  

To run the tool for Azure with the help option to discover the parameters, run the following command:

java -jar public-cloud-billing-check-tool.jar azure -h

Here is an example of the help parameters output. The -r parameter is mandatory.

java -jar public-cloud-billing-check-tool.jar azure -h
Usage: <main class> azure [-bhvV] [-azp] [-c=<currencyCode>] [-cfp=<configYAMLPath>] [-d=<resellerDiscount>] [-epp=<excludedProductIdsPath>]
                          [-m=<margin>] [-mt=<month>] -r=<resellerId> [-y=<year>] [-ep=<excludedPublishers>[,<excludedPublishers>...]]...
Check for Azure CSP customer costs and subscription mappings.
      -azp, --azureplan      Run the check for Azure Plan subscriptions in Abiquo with the CSP data
  -b, --billing              Run check for billing information in Abiquo for the last consolidated billing period
  -c, --currencyCode=<currencyCode>
                             Currency code used to retrieve billing line items, Default code is USD
  -cfp, --configPath=<configYAMLPath>
with the CSP data  -b (--billing)          : Run check for billing information in Abiquo for the    Specify a different path for the config file. Default path is the same where .jar is located
  -d, --discount=<resellerDiscount>
   last consolidated billing period  -cfp (--configPath) VAL : Specify a different path for the config file.          Define the discount necessary to substract to total cost
  -ep, --excludedPublishers=<excludedPublishers>[,<excludedPublishers>...]
     The default path is the same as where the .jar is located  -d (--discount)         : DefineSpecify theAzure discountpublishers to subtractexclude fromtheir totalproducts
cost  -epepp, --excludedPublishersexcludedProductIdsPath=<excludedPublishers>[,<excludedPublishers>...<excludedProductIdsPath>]
 -s (--seed)             : Path to seed file used to decrypt access and key tokens   Specify a path to                      from database
 -r (--reseller)a CSV file of Azure product IDs to exclude
  -h, --help         : Enterprise ID of the reseller from which obtainShow thethis help message and exit.
  -m, --margin=<margin>      Desired margin to compare cost from CSP and Abiquo database. If difference is  pricing credentials
 -m (--margin) N   bigger than margin, it will be considered
     : Desired margin to compare cost from CSP and Abiquo                 as an error. Default is 0
  -mt, --month=<month>  database. If the difference is biggerMonth thanof margin,the itbilling period to check. Default is previous month
    -r, --reseller=<resellerId>
               will be considered an error. Default is 0  -oe (--onlyErrors)    Enterprise id :of Configurethe scriptreseller tofrom onlywhich promptobtain errorsthe orpricing warningscredentials
  -v, --verbose              Display more detailed information about the process execution
  in logs-V, --version  -v (--verbose)          : Print more detailedversion information aboutand theexit.
process  -y, --year=<year>          Year of the billing period to check. Default is the year of the previous  executionmonth

Here is an example of running the tool on Azure using parameters:

java -jar public-cloud-billing-check-tool.jar azure -azp -b -cfp "/path/to/config.yml" -s "/path/to/seed/.store"azp -b -m 2 -v -r 10

And another example:

java -jar /opt/abiquo-public-cloud-billing-check-tool/public-cloud-billing-check-tool.jar azure --azureplan --billing --configPath /home/ansible/config.yml --margin 0 --verbose --discount '0.15' --reseller 3

Exclude publishers option

...

/abiquo-public-cloud-billing-check-tool/public-cloud-billing-check-tool.jar azure --azureplan --billing --configPath /home/ansible/config.yml --margin 0 --verbose --discount '0.15' --reseller 3

Exclude publishers option

To exclude products from specific publishers from Azure billing, you can set and Abiquo property: abiquo.azure.billing.parser.lineitem.publisher.ignore.
This property contains a string of publishers to exclude from billing line items in CSV format, such as Microsoft Office. 
When you are using this property, for the public-cloud-billing-check-tool for Azure, you can use the --excludedPublishers option.

Exclude product IDs option

To exclude specific products by ID from Azure billing, you can set a new system property "abiquo.azure.billing.parser.lineitem.publisher.ignore".
This property contains a string of publishers to exclude from billing line items in CSV format. For example "Microsoft Office". 
When .billing.parser.lineitem.productid.ignore.path system property to define a fileystem path to a CSV file containing all the product IDs to ignore.
Effectively, the CSV file will be a list of product IDs to ignore, one per line. When you are using this property, for the public-cloud-billing-check-tool for Azure, you can use the " --epp or --excludedPublishers" excludedProductIdsPath option.

Azure example output

In verbose mode the tool will print a detailed list of all the CSP customers, and the respective enterprises that contain public cloud credentials of the same customer.

...

• A positive number means there are more entitlements in the CSP Azure Plan than there are in Abiquo

• A negative number means there are more subscriptions in Abiquo than in the Microsoft Azure portal. This could be because credentials are not set correctly, or because a subscription was deleted/canceledcancelled/deactivated in Azure but the subscription still exists in Abiquo.

...

2021-04-21 10:43:58,678 WARN  com.abiquo.azureplans.Main - Customer "X" with identifier XXXX-XXXX-XXXX-XXXX has a summary cost of N USD, and in Abiquo database total for all the enterprises of same customer is N-1 USD.
2021-04-21 10:43:58,678 WARN  com.abiquo.azureplans.Main - There's a difference of 1 for customer "X" with identifier XXXX-XXXX-XXXX-XXXX
2021-04-21 10:43:58,678 WARN  com.abiquo.azureplans.Main - Detailed cost for every enterprise belonging to customer "X" with identifier XXXX-XXXX-XXXX-XXXX
2021-04-21 10:43:58,679 WARN  com.abiquo.azureplans.Main - 	-> Cost for enterprise "X Ent 1" with id 45 is X
2021-04-21 10:43:58,679 WARN  com.abiquo.azureplans.Main - 	-> Cost for enterprise "X Ent 2" with id 56 is X

Azure example errors

The tool tries to log all error messages without stopping. Most of those errors would be errors returned from the Azure CSP API.

...