Add public cloud

A public cloud region represents a region of a cloud provider in Abiquo, to enable users to have compute access: to use virtual resources and deploy VMs in the cloud, and/or billing access: to display cloud billing data on the dashboard.

Before you begin:

• For vCloud Director, obtain credentials for the Administrator or Organization. For more details see VMware vCloud Director integration

To create a public cloud region:

1. Go to Infrastructure → Public

2. Click the + add button at the bottom of the Public cloud regions list 

3. On the Create public cloud region dialog, enter the base Name and select the Provider.

4. Select the Regions

   • The platform will create a region with Name, and for the next regions it will add a suffix of _1, _2, and so on, and the region’s location.  

   • If the platform cannot create a region, it will go on to the next region on the list.

     

5. Click Next

6. For the Virtualization manager, in the remote services IP address field, enter the domain name of the Remote Services server.

   1. You can share the remote services with multiple public cloud regions or private cloud.

      1. You can select the Remote Services server from the pulldown, and click Duplicate IP address

   2. We recommend that you Check the remote services before you save your regions

      

7. Click Save

   The platform will create your public cloud region.

To allow users to deploy in the public cloud region, edit your enterprises to:

1. Edit Datacenters and add the new region to the Allowed datacenters list

2. Add Credentials for the cloud provider API. See Obtain public cloud credentials

   1. You will require one separate account for each enterprise using a public cloud region, which means one account or subscription per enterprise

   2. For certain regions, such as those in China, you will require separate credentials, which you can enter separately after you select the appropriate provider, such as Amazon CHINA

   3. For vCloud Director, you will require Organization and Administrator credentials 

To allow administrators to manage the public cloud region, edit your user scopes and add the new region

To display Azure billing data for resellers and their customers:

1. Abiquo can obtain Azure billing data with the credentials you enter when you configure Abiquo to:

   1. Create a reseller with CSP pricing credentials; OR

   2. Create an enterprise with compute or billing only credentials, with a Contributor role.
      See Onboard an Azure CSP or AWS organization account.

2. Edit the CSP reseller enterprise and set the following enterprise properties:

   1. azurecompute-arm_discount: decimal value (with dot separator), used to calculate the user invoice for billing dashboard. CSP APIs return prices with a discount applied, which we remove to display for customers. We use the formula: total = TotalFromCSP/(1 -  azurecompute-arm_discount). This is a required property

   2. azurecompute-arm_currency_code: The default is USD. Three character currency code of the bill. Warning - this is different from currency_code used for conversion factor. This is a required property

   3. billing.azure.country_code: Two digit ISO code representing the country where you purchased the subscription. The default of US is set in abiquo.properties on Remote Services as abiquo.billing.azure.country_code.

3. Optionally, configure price factors for markups. See Manage price factors

4. Edit reseller customer enterprises and add this enterprise property:

   1. azurecompute-arm_discount: Decimal value (with dot separator), used to calculate the user invoice for billing dashboard.

5. If the customer enterprise has Azure plans, they may have a keynode enterprise and several standard enterprises below it to use the plans credentials. Edit the standard enterprises and add this property.

   1. azurecompute-arm_only_bill_subscription: If this property is true, for an Azure plan, only bill for usage - do not include the customer's products and services in the billing data for this enterprise.

Abiquo properties for Azure billing data display

On the Abiquo Server, you can configure the following properties for Azure billing.

• abiquo.enterprise.property.billing.monthoffset: By default, the platform will retrieve billing data for the last two months. To change this set the following property to the number of months to retrieve.
  Default value: 2

• abiquo.azure.billing.parser.lineitem.publisher.ignore: By default, Abiquo will bill SaaS resources, such as a Twilio subscription, to the enterprise that the subscription is assigned to. This corresponds to the billing scope ID of a billing line item. To exclude products from specific publishers from Azure billing, add the following property, and enter a string of publishers to exclude from billing line items in CSV format. With the public-cloud-billing-check-tool for Azure, use the --excludedPublishers option. Default value: Microsoft Office

On the Remote Services, you can configure the following properties for Azure billing.

• abiquo.billing.azure.country_code: For Azure price factors. Two digit ISO code representing the country where you will obtain the product list. Default value: US

Obtain credentials for an AWS standard or reseller account

This section describes how to create a new user and obtain credentials for an AWS standard or reseller account. For reseller customers, see the section below.

• An AWS IAM user for Abiquo:

  • Requires programmatic access

  • Does not require a password for login or MFA.

To obtain AWS access, you will need to assign a group and/or access policy to your user, as described in the sections below. If you already have a group and/or access policy, you can use them again.

AWS credentials are an Access Key ID and Secret Access Key that allow access the AWS APIs.

Before you begin:

1. Check the access policy examples in the sections below with your Security Administrator

To create a user in a new group with a new access policy, do these steps. 

1. Open the AWS console

2. On the top left menu, go to Services →  IAM

3. On the left sidebar, go to Users

4. Click Add user

   1. Enter the User name

   2. Select Programmatic access

   3. Click Next: Permissions

5. Under Add user to group, select a group or click Create group.
   To create a group do these steps:

   1. Enter the Group name

   2. Select existing policies, or click Create policy

   3. In the Create policy section, go to JSON, and enter your access policy, as provided by your Security Administrator. See below for example policies

   4. Click Review policy

   5. Enter the Name and Description, check the resource access, and click Create policy

6. Go back to the previous browser tab with the Create group dialog open

   1. Click Refresh

   2. If you can't see the policy, search and filter by Name to display it, and select the policy

   3. Click Create group

7. On the Add user to group page, select the group

8. Click Next: Tags and enter the Key and Value of tags as required

9. Click Next: Review

10. Click Create user

11. To obtain user credentials, do these steps.

    1. Copy the Access key ID

    2. Click Show and copy the Secret access key. 

    3. To download a credentials file, click Download .csv.
       By default the file will be saved as credentials.csv in your Downloads folder. 

    4. Close the Credentials window

Manage credentials

For an existing user, to manage access keys.

1. From IAM → Users, click on the user name

2. Go to Security Credentials

3. Create, deactivate, and delete access keys as required

Credentials file format

This is an example of the format of the credentials.csv file when you open it in a text editor:

User Name,Access Key Id,Secret Access Key
"MJSB",AKIAJHWYJYNWV2RAAAAA,YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

1. The Access Key ID is the string of characters before the comma, which is AKIAJHWYJYNWV2RAAAAA in the above example.

2. The Secret Access Key for the Access Key ID is the string of characters after the comma, which is YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY in the above example.

AWS Policy examples

These policies are examples only. Please check your policies with your system administrator.

Example compute policy

This is an example of a policy that will let you work with the compute features in AWS.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "ec2:*",
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "elasticloadbalancing:*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "cloudwatch:*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:DeleteServerCertificate",
                "iam:UploadServerCertificate",
                "iam:ListServerCertificates",
                "iam:GetServerCertificate"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "autoscaling:*",
            "Resource": "*"
        }
    ]
}

Standard account pricing and billing policy

The following policy lets the user manage pricing and billing for a standard account that is not a partner organization.

The “Sids” for specific actions are as follows.

• Pricing requires: Pricing

• To run the billing check tool requires: BillingCheckTool

• To run the billing process requires: Billing

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "Pricing",
			"Effect": "Allow",
			"Action": [
				"pricing:*"
			],
			"Resource": [
				"*"
			]
		},
		{
			"Sid": "BillingCheckTool",
			"Effect": "Allow",
			"Action": [
				"ce:GetCostAndUsage"
				"organizations:ListAccounts",
				"organizations:ListRoots"
			],
			"Resource": [
				"*"
			]
		},
		{
			"Sid": "Billing",
			"Effect": "Allow",
			"Action": [
				"s3:GetObject",
				"s3:ListBucket"
			],
			"Resource": [
				"arn:aws:s3:::billingprogramatic",
				"arn:aws:s3:::billingprogramatic/*"
			]
		}
	]
}

Reseller pricing and billing policy

The following policy lets the user manage pricing and billing for an AWS partner organization.

The “Sids” for specific actions are as follows.

• Pricing requires: Pricing

• To add billing only credentials requires: BillingOnlyCredentials

• To run the billing check tool requires: BillingCheckTool

• To run the billing process requires: Billing

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "Pricing",
			"Effect": "Allow",
			"Action": [
				"pricing:*"
			],
			"Resource": [
				"*"
			]
		},
		{
			"Sid": "BillingOnlyCredentials",
			"Effect": "Allow",
			"Action": [
				"organizations:DescribeAccount"
			],
			"Resource": [
				"*"
			]
		},
		{
			"Sid": "BillingCheckTool",
			"Effect": "Allow",
			"Action": [
				"ce:GetCostAndUsage",
				"organizations:ListAccounts",
				"organizations:ListRoots"
			],
			"Resource": [
				"*"
			]
		},
		{
			"Sid": "Billing",
			"Effect": "Allow",
			"Action": [
				"s3:GetObject",
				"s3:ListBucket"
			],
			"Resource": [
				"arn:aws:s3:::billingprogramatic",
				"arn:aws:s3:::billingprogramatic/*"
			]
		}
	]
}

Policy to create an account using Abiquo

To use the Create account feature in Abiquo, add this statement to your AWS account policy.

		{
			"Sid": "CreateAccount",
			"Effect": "Allow",
			"Action": [
				"organizations:CreateAccount",
				"organizations:DescribeCreateAccountStatus",
				"sts:AssumeRole"
			],
			"Resource": []
		}

Policy to create an account in an organizational unit using Abiquo

To use the Create account feature and select or create an organizational unit in which to create the account, add this statement to your AWS account policy, to replace the basic Create account policy.

		{
			"Sid": "CreateAccount",
			"Effect": "Allow",
			"Action": [
				"organizations:CreateAccount",
				"organizations:DescribeCreateAccountStatus",
				"organizations:CreateOrganizationalUnit",
				"organizations:ListOrganizationalUnitsForParent",
				"organizations:ListRoots",
				"organizations:MoveAccount",
				"sts:AssumeRole"
			],
			"Resource": "*"
		}

Obtain billing-only credentials for customers of AWS resellers

For the customer of a reseller account, the AWS credentials are the customer Account ID.

1. In AWS, go to the customer account, NOT the main AWS partner account

2. Obtain the customer’s Account ID

Add this customer credential to a customer enterprise on the Public tab and select the Billing only checkbox

Changes to AWS billing

The following changes apply to AWS billing:

• In Abiquo 6.1.0+, replace the price_factor enterprise property with the Abiquo price factors for Amazon. See Manage price factors

• In Abiquo 6.1.2+, you can also add a managed costs using a price factor. See Manage price factors

Configure AWS to supply billing data

To configure AWS to supply billing data for standard or reseller accounts, do the following steps.

1. Create an S3 bucket, for example, costandusagebillingreport

   1. Within the bucket, create a folder where AWS will store your reports. Give it the name of your report, for example costandusagebillingreport

   2. Note the billing bucket name, for example, costandusagebillingreport

2. Create a new user, such as programmaticbilling to create the reports

   1. Assign the AmazonS3ReadOnlyAccess policy

   2. Activate the IAM user's access to billing information. See https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/control-access-billing.html

3. Configure Cost & Usage Reports

   1. Go to Cost & Usage Reports

   2. Enter the Report  name, and note the value to enter in Abiquo later, then click Next

   3. Click Configure and select the S3 bucket. Click Next, then select I have confirmed that this policy is correct.

   4. By default, Amazon will put the reports in a folder with the name format /report-name/date-range/. Note this as the value for the amazon_bucket_prefix in Abiquo. Click Next

   5. Review your configuration and check that the following parameters are set: 

      1. bucket name

      2. path (folder/subfolder)

      3. time detail: Hourly

      4. GZ or ZIP format

4. The AWS account with credentials to use in Abiquo should be an account with pricing and billing permissions.

   1. For partner accounts, see https://abiquo.atlassian.net/wiki/spaces/doc/pages/311370749/Obtain+AWS+credentials#Reseller-pricing-and-billing-credentials

   2. For standard accounts, see https://abiquo.atlassian.net/wiki/spaces/doc/pages/311370749/Obtain+AWS+credentials#Standard-account-pricing-and-billing-policy

Configure dashboard display in Abiquo

To configure billing dashboards in Abiquo, do these steps in Abiquo.

1. Edit the enterprise and create the following enterprise properties:

   1. amazon_bucket: bucket_name

   2. amazon_bucket_region: code for the AWS region of the bucket, such as us-east-1

   3. amazon_report_name: amazon_bucket/amazon_bucket_prefix/amazon_report_name/file.csv

   4. amazon_bucket_prefix: amazon_bucket/ amazon_bucket_prefix /amazon_report_name/file.csv

   5. amazon_billing_compress_format: ZIP or GZ

   6. amazon_mpa: set to dedicated or no to use blended costs; if not present or shared, use unblended costs

Additional configuration for resellers

This section describes additional configuration for reseller accounts.

1. Create an additional enterprise property for resellers:

   1. amazon_discount: as agreed with Amazon, usually with a value such as 0.03, 0.02

2. In Pricing view, create price factors as required for the reseller, the customers, and for a percentage of managed costs. See Manage price factors

3. For your customer enterprises, add credentials for compute and/or billing

4. Assign the privilege to View bills to user roles

5. Optionally, configure Abiquo properties

   1. By default, the platform will retrieve billing data for the last two months. To change this value, on the Abiquo Server, set the following property

      abiquo.enterprise.property.billing.monthoffset=2

   2. By default, the platform will process all items in the CSV file, so the partner discount (SppDiscount) value can be visible on customer dashboards. To discard items from the CSV file, on the Remote Services server, set the following property with a list of codes of any items to discard.

      abiquo.ec2billing.parser.lineItemType.ignore=SppDiscount

The platform can import hardware profile prices from public cloud for use in pricing.

• The prices are for Linux instances with no pre-installed software

• For AWS, the only currency available is US dollars.
  

Before you begin:

1. Check that there is a public cloud region for the provider. See Create a public cloud region.

2. Check that the tenant has a pricing model assigned. See Create a new pricing model

   1. For Amazon regions, use US Dollars as the currency

   2. If you are onboarding price lists for a customer of the CSP and/or reseller, go to Pricing → Price factors and create price factors to add markups or discounts to the cloud provider prices.
      See Manage price factors

3. Obtain credentials to retrieve pricing information from the provider.

   1. For AWS pricing, an IAMS user must have the AWSPriceListServiceFullAccess permission. You can add the permission to the regular public cloud user account or create a separate user. See Obtain AWS credentials

   2. For Azure, see Obtain Azure ARM credentials.

4. Check you have the pricing credentials in the right format. See Public cloud pricing credentials table below

To retrieve the hardware profile prices:

1. Add pricing credentials for the tenant 

   1. From the username menu go to Edit credentials or go to Users → edit enterprise

   2. Go to Credentials → Public (for compute or combined credentials in Amazon) or
      Credentials → Pricing (for Azure or separate Amazon credentials)

   3. Enter the credentials

2. If the tenant is a CSP and/or reseller, go to Edit enterprise → Properties and enter the following properties with appropriate values:

   azurecompute-arm_discount=0.2
   amazon_discount=0

   Note that CSP accounts return the prices with the discount factor applied, so the platform will not apply it again.
   To configure a custom suffix of the discount properties, set abiquo.enterprise.property.discount.suffix in abiquo.properties. See Abiquo Configuration Properties#enterprise.

3. When you save the tenant, if the pricing credentials are present, the platform will retrieve the prices. 

To display and edit the prices of public cloud hardware profiles:

1. Go to Pricing → edit pricing model.

2. Go to Resource Prices → select the public cloud region

3. For each hardware profile, enter a New price as required

4. Click Save

The platform will update the hardware profile prices from the public cloud provider every 24 hours.

• To set a custom interval, set abiquo.pricing.import.check.delayInHrs in abiquo.properties. See Abiquo configuration properties#pricing

• To prevent the platform from updating the prices from the public cloud provider, remove the pricing credentials

Public cloud pricing credentials table