Versions Compared

Old Version 11

changes.mady.by.user Lou M

Saved on

compared with

New Version 12

changes.mady.by.user MS

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Public cloud billing check tool

...

You can run this tool on the Abiquo Server. The project is built using Maven. You can build it with the following command:

Code Block
languagebashnone
mvn clean install

This will output a .jar script as follows.

Code Block
languagenone
target/public-cloud-billing-check-tool.jar

...

To use this tool in AWS, update your billing-api-user for Abiquo to add the following AWS IAM policy:

Code Block
languagenone
"Effect": 
	"Allow", 
	"Action": [ "ce:*", ], 
"Resource": "*"

...

  1. Go to /opt/abiquo-public-cloud-billing-check-tool/config/ 

  2. Edit the config.yml file and complete the values for your system.

    Code Block
    languagenone
    Abiquo_database:
  host: localhost
 
user:
 
password:
 
port: 3306
  database: kinton
 
socket: /var/lib/mysql/mysql.sock
abiquo:
  api_url: 'http://localhost/api'
  version: '6.0'
  user:
 
#Basic Auth credentials
 
password:

 #OAuth credentials
  access-token:

 access-token-secret:
  consumer-key:
 
consumer-secret:

General usage

The tool has two main commands to run on Azure or AWS.

Code Block
languagenone
java -jar public-cloud-billing-check-tool.jar 
-help Usage: <main class> [-hV] [COMMAND] 
Check public cloud billing information. 
	-h, --help      Show this help message and exit.

	-V, --version   Print version information and exit.

Commands: 
	azure  Check for Azure CSP customer costs and subscription mappings.
aws   	aws Check for AWS organization billing costs and account mappings.

...

When running the tool on Amazon, you can use the -h option to print the parameters, as shown here.

Code Block
languagenone
java -jar public-cloud-billing-check-tool.jar aws -h

Usage: <main class> aws [-abhvV] [-spp] [-cfp=<configYAMLPath>] [-d=<resellerDiscount>] [-m=<margin>] [-mt=<month>] -r=<resellerId><rootEnterpriseId> [-s=<seed>] [-y=<year>]
Check for AWS organization billing costs and account mappings.

  -a, --accounts             Run the check for accounts inside an organization mapped to abiquo
  -b, --billing              Run check for billing information in Abiquo for the last consolidated billing period
      -cfp, --configPath=<configYAMLPath>
  
                          Specify a different path for the config file. Default path is the same where .jar is located
  -d, --discount=<resellerDiscount>

                            Define the discount necessary to subtract to total cost
 
-h, --help                 Show this help message and exit.
 
-m, --margin=<margin>      Desired margin to compare cost from AWS and Abiquo database. If difference is bigger than margin, it will be considered
   
                           as an error. Default is 0
      -mt, --month=<month>   Month of   Month of the billing period to check. Default is lastprevious month
 
-r, --resellerrootEnterpriseId=<resellerId>
<rootEnterpriseId>
                            Enterprise id of the reseller or key node from which to obtain the pricing credentials
  -sspp, --seed=<seed>sppDiscount        If true, Pathinclude tosolution seedprovider fileprogram useddiscounts toin decryptthe accessprice
and key tokens from database
  -v, --verbose   -v, --verbose              Display more detailed information about the process execution

 -V, --version              Print version information and exit.
  -y, --year=<year>          Year of the billing period to check. Default is lastthe year of the previous month

Only the reseller parameter root enterprise option is required. An example of execution with using all parameters would will be:

...

java

...

-jar

...

public-cloud-billing-check-tool.jar

...

aws

...

-a

...

-b

...

-r=1

With the "verbose" option, the tool will display a detailed list of all accounts mapped to Abiquo and the respective enterprises that contain public cloud credentials of each account.

...

  1. Used in more than one enterprise in Abiquo

  2. Mapped to the reseller enterprise hierarchy but not present in the reseller organization

  3. Present in the reseller organization but not present mapped to Abiquo If "verbose" is also selected, it will also print all the accounts of the organization.

With the "billing" option, the the tool will check the difference between AWS cost and Abiquo costs for each mapped account. If "verbose" is activated, the script will print the difference for all enterprises, otherwise it will only print entries with cost difference between AWS and Abiquo.

...

  • Check that all subscriptions from a CSP Azure Plan are mapped correctly to Abiquo enterprises

  • For Azure billing, check the costs of the last month at customer level, not subscription level.

To run the tool for Azure with the help option to discover the parameters, run the following command:

Code Block
languagenone
java -jar public-cloud-billing-check-tool.jar azure -h

Here is an example of the help parameters output. The "-r" parameter is mandatory.

Code Block
-azp (--azureplan)      : Run the check for Azure Plan subscriptions in Abiquo
Code Block
languagenone
Usage: <main class> azure [-bhvV] [-azp] [-c=<currencyCode>] [-cfp=<configYAMLPath>] [-d=<resellerDiscount>] [-m=<margin>] [-mt=<month>]
                   with the CSP data  -br=<resellerId> (--billing)          : Run check for billing information in Abiquo for the
     [-y=<year>] [-ep=<excludedPublishers>[,<excludedPublishers>...]]...
Check for Azure CSP customer costs and subscription mappings.
    -azp, --azureplan      Run the check for Azure Plan subscriptions in Abiquo with lastthe consolidated billing period
CSP data
-cfpb, (--configPath)billing  VAL : Specify a different path for the config file.   Run check for billing information in Abiquo for the last consolidated billing period
-c, --currencyCode=<currencyCode>
          The default path is the same as where the .jar is located  -d (--discount)   Currency code used to retrieve billing :line Defineitems, theDefault discountcode tois subtractUSD
from total cost  -epcfp, --excludedPublishers=<excludedPublishers>[,<excludedPublishers>...]
 -s (--seed)configPath=<configYAMLPath>
                    : Path to seed file used to decryptSpecify accessa anddifferent keypath tokensfor the config file. Default path is the same where .jar is located
-d, --discount=<resellerDiscount>
             from database  -r (--reseller)         : EnterpriseDefine IDthe ofdiscount thenecessary resellerto fromsubstract whichto obtaintotal thecost
 -ep, --excludedPublishers=<excludedPublishers>[,<excludedPublishers>...]
 
-h, --help                 Show this help message pricingand credentialsexit.
 -m, (--margin) N    =<margin>     : Desired margin to compare cost from CSP and Abiquo database. If difference is bigger than margin, it will be considered
                database. If the difference is bigger than margin, it             as an error. Default is 0
    -mt, --month=<month>   Month of the billing period to check. Default is previous month
-r, --reseller=<resellerId>
       will be considered an error. Default is 0  -oe (--onlyErrors)      : Configure script to onlyEnterprise promptid errorsof orthe warningsreseller from which obtain the pricing credentials
-v, --verbose              Display more detailed information about the inprocess logs
execution
-vV, (--verbose)version          : Print more detailed informationPrint aboutversion theinformation processand exit.
-y, --year=<year>          Year of the billing period to check. Default is the year of the previous  execution

Here is an example of running the tool on Azure using parameters:

Code Block
java -jar month

Only reseller option is required. An example of execution using all parameters will be: java -jar public-cloud-billing-check-tool.jar

...

azure

...

-azp

...

-b -r=1

When verbose is activated, script will display a detailed list of all the CSP customers, and the respective enterprises which contains public cloud credentials of the same customer

And another example:

Code Block
languagenone
java -jar /opt/abiquo-public-cloud-billing-check-tool/public-cloud-billing-check-tool.jar azure --azureplan --billing --configPath /home/ansible/config.yml --margin 0 --verbose --discount '0.15' --reseller 3

...

To exclude products from specific publishers from Azure billing, you can set a new system property "abiquo.azure.billing.parser.lineitem.publisher.ignore".
This property contains a string of publishers to exclude from billing line items in CSV format. For example "Microsoft Office". 
When you are using this property, for the public-cloud-billing-check-tool for Azure, you can use the "--excludedPublishers" option.

Azure example output

In verbose mode the tool will print a detailed list of all the CSP customers, and the respective enterprises that contain public cloud credentials of the same customer.

Code Block
languagenone
2021-04-13 12:03:45,267 [main] INFO  com.abiquo.azureplans.Main - Customer "Customer Name" with identifier XXXXX-XXXXXX-XXXXXX-XXXXX
2021-04-13 12:03:45,267 [main] INFO  com.abiquo.azureplans.Main - 	- Enterprise "Abiquo Enterprise Name" with id 241
2021-04-13 12:03:45,267 [main] INFO  com.abiquo.azureplans.Main - 	- Enterprise "Abiquo Other Enterprise Name " with id 244

For the Azure plans check, in verbose mode, the tool will then create output as shown below. Without verbose mode, only warnings will be shown.

Code Block
languagenone
2021-04-13 12:05:37,588 [main] INFO  com.abiquo.azureplans.Main - Customer on CSP "Customer X" with identifier XXXX-XXXX-XXXX-XXXX has all the subscriptions of the azure plan Azure plan with identifier XXXX-XXXX-XXXX-XXXX mapped to Abiquo enterprises
2021-04-13 12:05:44,989 [main] WARN  com.abiquo.azureplans.Main - There's 1 subscriptions missing in Abiquo from the Azure Plan of customer "Customer missing subscriptions" with identifier XXXX-XXXX-XXXX-XXXX. Present subscriptions in Abiquo are: 
-> "Subscription 1" with identifier XXXX-XXXX-XXXX-XXXX mapped to enterprise "Enterprise Abiquo 1" with id 502
-> "Subscription 2" with identifier XXXX-XXXX-XXXX-XXXX mapped to enterprise "Abiquo Enterprise 2" with id 508

...

You can specify a margin for the log levels of differences between the provider and Abiquo. Differences that are less than or equal to the margin will be INFO level, and differences larger than the margin will be WARN level.

Code Block
languagenone
2021-04-21 10:43:58,678 WARN  com.abiquo.azureplans.Main - Customer "X" with identifier XXXX-XXXX-XXXX-XXXX has a summary cost of N USD, and in Abiquo database total for all the enterprises of same customer is N-1 USD.
2021-04-21 10:43:58,678 WARN  com.abiquo.azureplans.Main - There's a difference of 1 for customer "X" with identifier XXXX-XXXX-XXXX-XXXX
2021-04-21 10:43:58,678 WARN  com.abiquo.azureplans.Main - Detailed cost for every enterprise belonging to customer "X" with identifier XXXX-XXXX-XXXX-XXXX
2021-04-21 10:43:58,679 WARN  com.abiquo.azureplans.Main - 	-> Cost for enterprise "X Ent 1" with id 45 is X
2021-04-21 10:43:58,679 WARN  com.abiquo.azureplans.Main - 	-> Cost for enterprise "X Ent 2" with id 56 is X

Azure example errors

The tool tries to log all error messages without stopping. Most of those errors would be errors returned from the Azure CSP API.

A customer in Abiquo was not found in the Azure CSP. You should delete these credentials from Abiquo.

Code Block
languagenone
2021-04-21 10:39:50,956 ERROR com.abiquo.azureplans.Main - Customer with identifier XXXX-XXXX-XXXX-XXXX could not be found in Azure CSP. This could be due that the customer was deleted

A recently created customer won't have any costs for the last month, so the CSP API returns an error. The tool controls this condition, and logs an error.

Code Block
languagebody
2021-04-21 10:43:51,220 ERROR com.abiquo.azureplans.Main - Customer "X" with identifier XXXX-XXXX-XXXX-XXXX does not have any summary cost. This could be due to the creation date of the customer. Execution will continue