Upload and replace template disks via API

1. Introduction

This document describes how to upload and replace existing template disks using the Appliance manager API.

Use case

Use this process to replace a template disk directly in the repository file system and ensure that Abiquo has the correct template details for the new disk.

This request requires direct access to the Appliance Manager API
Make a backup of your template folder before you begin

If you are replacing a disk that was created from an instance of a captured VM, add an Abiquo OVF file to the original VM template folder before you upload your new disk. You may be able to copy a file from another template and modify it for your template.

The Appliance Manager API does not validate the request, so be sure to provide the correct path to your disk file or you could overwrite some other part of your file system!!

Here is an example use case

Upload or download a template
Use it to create a VM
Update the template. Abiquo supports changes to:
1. file content
2. size
3. format
4. capacity
5. controller
Update the disk using the Appliance manager API.
When the user next deploys the VM, the platform will copy the updated disk

Basic steps

Get the template details from the Abiquo API
Create the replacement data object
Replace the file using the Appliance manager API

2. Log in to the API and obtain a token

If you wish to use token authentication, which is more secure and required for 2FA, do these steps.

To obtain an authorization token, send a GET request to the API login resource.

Use basic authentication and get the token from the X-Abiquo-Token header.

curl --verbose -X GET "https://abiquo.example.com:443/api/login" -u adminuser:password -k

If you are using 2FA, you can get the 2FA verification code in the usual way (by email or from Google Authenticator).

Log in again and send the 2FA verification code.

curl --verbose -X GET "https://abiquo.example.com:443/api/login" -H "X-Abiquo-OTP: your2FAcode"

Now you can get the token from the X-Abiquo-Token header of the response.

In your API requests, use the token in a header with the following format. In this example we have shortened the token. In the workflow below, we represent the token with {api_token}.

-H 'Authorization: Token a2e19816735381c1d074441cbd002aa01f9ed7dc4cd9f4e0683c3c88...'

For more details, see Authentication.

The API token may expire during the process, and you may need to obtain a new token and use it in your requests

You can also get the token from the UI! Go to the browser Developer console on the Network tab and in the request responses look for the X-Abiquo-Token header.

3. Get the template details

Get the enterprise, for example, by using the has parameter to filter by a text string in the enterprise names from the list enterprises request.

Replace the base_api_url with the URL of your Abiquo API server, for example, https://abiquo.example.com/api, and replace {api_token} with the token from the X-Abiquo-Token response header.

Here we use JQ to filter the API response to only display the enterprise names and IDs.

curl -X GET '{base_api_url}/admin/enterprises?has=Abiquo' \
     -H 'Accept: application/vnd.abiquo.enterprises+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.collection[]| {name: .name, id: .id}'

Reference: https://wiki.abiquo.com/api/latest/EnterprisesResource.html#list-enterprises

If you are not already working with the selected enterprise, switch enterprise in the UI (by clicking the Switch enterprise button) or by switching enterprises via the API.

For more details see How to switch enterprises via API

Get the enterprise and filter the response to only supply the link to its datacenter repositories.

curl -s -X GET '{base_api_url}/admin/enterprises/1' \
     -H 'Accept: application/vnd.abiquo.enterprise+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.links[] | select(.rel=="datacenterrepositories")'

Get the link to the enterprise's datacenter repository for the appropriate datacenter. The link to the datacenter repositories can be found in the Enterprise entity.

{
  "title": "repositories",
  "rel": "datacenterrepositories",
  "type": "application/vnd.abiquo.datacenterrepositories+json",
  "href": "{base_api_url}/admin/enterprises/1/datacenterrepositories"
}

From this link you can get the datacenter repositories.

curl -s -X GET '{base_api_url}/admin/enterprises/1/datacenterrepositories' \
     -H 'Accept: application/vnd.abiquo.datacenterrepositories+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.collection[].links[]| select(.rel=="datacenter", .rel=="edit")'

The JQ filter above will print the datacenter and edit links for each of the datacenter repositories.

Reference: https://wiki.abiquo.com/api/latest/DatacenterRepositoriesResource.html#list-datacenter-repositories

Select the appropriate repo and then use the edit link to get the datacenter repository and select the link to the virtual machine templates.

curl -s -X GET '{base_api_url}/admin/enterprises/1/datacenterrepositories/1' \
     -H 'Accept: application/vnd.abiquo.datacenterrepository+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.links[]| select(.rel=="virtualmachinetemplates")'

In our datacenter, the JQ filter returned this link to the virtual machine templates

 {
  "title": "virtual machine templates",
  "rel": "virtualmachinetemplates",
  "type": "application/vnd.abiquo.virtualmachinetemplates+json",
  "href": "{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates"
}

Get the templates, and find the appropriate template. You can use the has parameter to search for the template by name.

curl -s -X GET '{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates?has=yVM2_disk_replace' \
     -H 'Accept: application/vnd.abiquo.virtualmachinetemplates+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.collection[].links[] | select (.rel=="edit")'

The above JQ filter will list the edit links of all the templates that the request returns. From our example.

{
  "title": "yVM2_disk_replace",
  "rel": "edit",
  "type": "application/vnd.abiquo.virtualmachinetemplate+json",
  "href": "{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72"
}

Reference: https://wiki.abiquo.com/api/latest/VirtualMachineTemplatesResource.html#list-virtual-machine-templates-in-a-datacenter-repository

Get the template that you wish to use. The template will contain a link to the template’s disks.

curl -s -X GET '{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72' \
     -H 'Accept: application/vnd.abiquo.virtualmachinetemplate+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.links[] | select (.rel=="disks")'

From the JQ filter in the above example, we got the following disk link.

{
  "title": "disks",
  "rel": "disks",
  "type": "application/vnd.abiquo.disks+json",
  "href": "{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks"
}

Get the template disks, and find the disk you wish to replace.

curl -s -X GET '{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks' \
     -H 'Accept: application/vnd.abiquo.disks+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.collection[] | (.links[] | select (.rel=="edit")), {sequence: .sequence,  path: .path}'

For the repository, you will need the diskUrl and currentPath of this disk. In the Abiquo API, these are the href URL from the disk edit link and the path of the disk.

The above query and JQ filter returned the following.

{
  "title": "disk",
  "rel": "edit",
  "type": "application/vnd.abiquo.disk+json",
  "href": "{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks/74"
}
{
  "sequence": 0,
  "path": "1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/4ba3b6e1-6f06-47d9-8703-9e9eae1c2a9b-snapshot-yVM2b8c91aa4-14b1-44a6-b27b-aa12c7ca3433-flat.vmdk"
}

You will need the href and the path for the replacement object!

4. Create the replacement object

From the above response, we need

the href URL from the disk in the edit link
the path of the disk

From the example above, for the first disk with a sequence number of 0, we have the following.

"href": "{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks/74"
"path": "1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/4ba3b6e1-6f06-47d9-8703-9e9eae1c2a9b-snapshot-yVM2b8c91aa4-14b1-44a6-b27b-aa12c7ca3433-flat.vmdk"

diskReplace.json

From the above response, you will need to set the following

diskUrl = the href URL from the disk edit link
currentPath = path

To get the VM template URL, just remove disks/XX from the end of the diskUrl!

Here is an example object.

{ 
    "virtualMachineTemplateUrl":"{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72",
    "diskUrl":"{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks/74",
    "currentPath":"1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/4ba3b6e1-6f06-47d9-8703-9e9eae1c2a9b-snapshot-yVM2b8c91aa4-14b1-44a6-b27b-aa12c7ca3433-flat.vmdk",
	"diskController": "lsilogic",
	"diskControllerType": "SCSI",
    "diskFileFormat":"VMDK_STREAM_OPTIMIZED",
	"diskFilePath": "",
	"label": "Hard disk 1",
    "requiredHDInMB":120,
    "sequence":0
}

For the requiredHDInMB you must enter the correct capacity or deployed size of the disk. The capacity value is stored in the disk file itself.

See Determine the size of a VM disk file

Abiquo does not validate the value that you enter and if you enter an incorrect value, the platform will try to deploy the disk with its true capacity. The deploy may fail and the platform will not be able to properly check that the disk will fit on the hypervisor datastore.

Important note: You cannot resize (expand) a boot disk before you deploy a VM. After you expand any disk, remember to update the configuration of the disk in the VM operating system.

The default value for the diskControllerType is IDE and the default value for the diskController is NULL.

If your template uses a different controller type and value, such as SCSI and lsilogic, you must add these values in the diskReplace.json entity, as shown above. If you do not add them, the platform will overwrite your template with its default values.

You should note that the AM API does not support all of the values that are supported by the Abiquo API.

5. Obtain the URLs to replace the file

Obtain the URLs to build the request to replace the disk.

If your file name or folder name contains spaces, do the steps below in “Steps for folder names with spaces” to obtain the folder path, file name and URL

The URL to post to contains the enterprise ID and the template folder in the following format.

/am/erepos/{enterpriseId}/templates/{templateFolder}

For template folder, from our example with the following href and path:

"href": "{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks/74"
"path": "1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/4ba3b6e1-6f06-47d9-8703-9e9eae1c2a9b-snapshot-yVM2b8c91aa4-14b1-44a6-b27b-aa12c7ca3433-flat.vmdk"

The template folder is the folder path on the NFS Repository, without the file name.

1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9

So the full URL to post to will be template path containing the prefix /am/erpos/{enterpriseId}/templates and the folder path

/am/erepos/1/templates/1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9

The template path does not contain the ID of the datacenter repository. It contains the folder name from /opt/vm_repository.
So in our example, the datacenter repository ID is 1 and the path is "1/bundle.." but they are not related

If the original disk file is not available, you can create a placeholder file in the folder using the touch command, in the format touch {currentPath}. For example:
touch 1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/4ba3b6e1-6f06-47d9-8703-9e9eae1c2a9b-snapshot-yVM2b8c91aa4-14b1-44a6-b27b-aa12c7ca3433-flat.vmdk

Steps to obtain the URL for folder names with spaces

The URL to post to contains the enterprise ID and the template folder in the following format.

/am/erepos/{enterpriseId}/templates/{templateFolder}

The template folder is the folder path on the NFS Repository, without the file name.

So for a folder name with spaces, for example, abc def.

We would have the following href and path:

"href": "{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks/74"
"path": "1/bundle/abc%20def/olddisk-flat.vmdk"

The folder path is as follows:

1/bundle/abc%20def

The format of the URL to post to must encode the special characters in the folder path again. So for our example:

/am/erepos/1/templates/1/bundle/abc%2520abc

So, in our example, the first part of the post request would look as follows:

curl -v -k -X POST "https://remoteservices.example.com:443/am/erepos/1/templates/1/bundle/abc%2520abc"

6. Replace the disk

This request is to the Appliance Manager remote service, so we need to use its URL.

In the following cURL, the base_url_NO_API is the URL of the server, without any /api path. For example, https://remoteservice.example.com.

We recommend that you use double quotation marks around all the parameters, including the file parameters for diskInfo and diskFile.

curl -v -k -X POST '{base_url_NO_API}/am/erepos/1/templates/1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9' \
  -H 'Authorization: Token {api_token}' \
  -F "diskInfo=@diskReplace.json" -F "diskFile=@newdisk.vmdk"

Replace newdisk.vmdk with the name of your disk file on the local file system.

Save the disk replacement object you created previously to a file called diskReplace.json. This is the real data object from our example.

{ 
    "virtualMachineTemplateUrl":"{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72",
    "diskUrl":"{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks/74",
    "currentPath":"1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/4ba3b6e1-6f06-47d9-8703-9e9eae1c2a9b-snapshot-yVM2b8c91aa4-14b1-44a6-b27b-aa12c7ca3433-flat.vmdk",
	"diskControllerType": "IDE",
    "diskFileFormat":"VMDK_STREAM_OPTIMIZED",
	"label": "Hard disk 1",
    "requiredHDInMB":120,
    "sequence":0
}

Example request:

Use double quotation marks around all the parameters, including the file parameters for diskInfo and diskFile, especially if the disk file name contains spaces!

curl -v -k -X POST 'https://remoteservices.example.com:443/am/erepos/1/templates/1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9' \
  -H 'Authorization: Token {api_token}' \
  -F "diskInfo=@diskReplace.json" -F "diskFile=@newdisk.vmdk"

> POST /am/erepos/1/templates/1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9 HTTP/1.1
> Host: abiquo.example.com
> User-Agent: curl/7.85.0
> Accept: */*
> Authorization: Token a2e19816735381c1d074441cbd002aa006febf2c4888bc2d79b9d2dfbcd9cbed84cbb0089197da8f0406f7a78ae2b25a64df188f10233541e5c45639b013b0c0
> Content-Length: 118752233
> Content-Type: multipart/form-data; boundary=------------------------bb8e15ec8adeba87
> Expect: 100-continue
> 

< HTTP/1.1 100 Continue
< HTTP/1.1 201 
< Date: Wed, 25 Jan 2023 14:44:47 GMT
< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
< Location: https://abiquo.example.com/am/erepos/1/templates/1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/8611a4ed-adbd-49aa-a22e-81fd4b60e833-newdisk.vmdk
< Content-Length: 0
<

Success status code: 201 Created

7. Check the disk file

You can check the disk file in Abiquo UI, which will display the path and file size.

You can also go to the datacenter repository on the NFS share (in the /opt/vm_repository folder) and check the new disk file on the file system using the details in the Location link in the above response.

[user@abiquo e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9]% ls -1
4ba3b6e1-6f06-47d9-8703-9e9eae1c2a9b-snapshot-yVM2b8c91aa4-14b1-44a6-b27b-aa12c7ca3433.ovf
7ae33505-fefc-452c-843d-7a843370f641-snapshot-yVM2b8c91aa4-14b1-44a6-b.vmdk
8611a4ed-adbd-49aa-a22e-81fd4b60e833-newdisk.vmdk
formats
[user@abiquo e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9]%

Note that if the disk file had spaces, it would display with + signs in the file name, for example:

daf2f2fb-d3d0-4263-a466-d1b27a0d6864-new+dsl+disk.vmdk

And for the example, we also retrieved the template disks and checked the details of the disk file in the API.

curl -s -X GET '{base_api_url}/admin/enterprises/1/datacenterrepositories/1/virtualmachinetemplates/72/disks' \
     -H 'Accept: application/vnd.abiquo.disks+json;version=6.1' -k \
     -H 'Authorization: Token {api_token}' \
     | jq '.collection[] | (.links[] | select (.rel=="diskfile")), {sequence: .sequence,  path: .path}'

Which for our disk, shows the newdisk.vmdk file has been loaded into Abiquo.

{
  "title": "8611a4ed-adbd-49aa-a22e-81fd4b60e833-newdisk",
  "rel": "diskfile",
  "href": "http://abiquo.example.com:8009/am/files/1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/8611a4ed-adbd-49aa-a22e-81fd4b60e833-newdisk.vmdk"
}
{
  "sequence": 0,
  "path": "1/bundle/e05785d2-b49a-4034-9bb9-3440e1693589-4ba3b6e1-6f06-47d9-8703-9e9/8611a4ed-adbd-49aa-a22e-81fd4b60e833-newdisk.vmdk"
}

The new disk is now ready to use in the VM template.