vRealize Automation Cloud – Infrastructure as Code

In the last blog post I explained Swisscom’s requirements regarding infrastructure as code, automated testing and continuous integration and deployment. This post will go in the technical implementation we are demoing at VMworld Barcelona. I will show what is already possible out of the box and how you can bridge the gaps by using the vRealize Automation Cloud APIs.

High Level Implementation Overview

 

vRealize Automation Cloud Entity Diagram

This diagram depicts the most important entities that must be created to set up a working environment. We are managing the light blue components by yaml definitions and the vRAC API calls for these entities. The entities with a green border are already available through GIT natively, the orange border shows further dependencies to external FaaS and vRO workflows.

Dark blue is used for the part that we have not automated, yet. We hope that organizations will be manageable through VMware Cloud Provider Hub for the SaaS offering, for a on-site vRA we plan extending our definition to a full VPC definition. Quite challenging will be the part of “bring your own identity” – maybe this will be done manually.

Automation makes sense if you can lower costs by scaling effects; for us as service provider these onboarding tasks on an organization level are executed frequently with standardized structures. Within a single enterprise, it might be good enough to have only project templates.

Out-of-the-box Functionality

We are very pleased that VMware vRealize Automation Suite 8.0 and vRealize Automation Cloud is already supporting GIT integration for blueprints and actions (see here for GitHub or GitLab). This is a major improvement of the developer friendliness and maintainability in complex environments. It is possible to keep blueprints and actions in different branches (i.e. per customer) and manage them in a highly automated fashion solely through GIT. Synchronizing repositories or versioning assets can be done by API as well.

vRealize Orchestrator 8.0 comes with a first version of GIT integration as well.

Implemented Functionality

In this blog post, I just document the needed API calls. I will come up with the GIT repository link later.

Most of the calls are still undocumented and provided as is. I use Chrome Developer Tools to inspect the network traffic while I simply do all the needed steps manually through the UI. As I work with Java, I use jsonschema2pojo to create the JSON domain classes. Take care to model the Pageable base class properly ( @see org.springframework.data.domain.Page<T> ), it will decrease the number of classes significantly. And yes, it would be nice to have the data structures defined in a open API definition (swagger file) enabling generation of all the needed classes.

Please use the following host for the SaaS installation:

{api} = https://api.mgmt.cloud.vmware.com

Create Project

This function is defined by a json file ( example1, example2, yaml ).

POST {api}/project-service/api/projects/
{
  "name" : "INT",
  "description" : "INT",
  "administrators" : [ ],
  "members" : [ ],
  "operationTimeout" : 0,
  "sharedResources" : false
}

PATCH https://api.mgmt.cloud.vmware.com/project-service/api/projects/{projectId}/principals

GET {api}/provisioning/resources/placement-zones?expand

 

Collect all cloud zones and prepare project config.

PATCH {api}/provisioning/mgmt/project-config

 

Kubernetes

GET {api}/cmx/api/resources/k8s-zones

 

Add Project ID to collection

PUT {api}/cmx/api/resources/k8s-zones/{zoneId}

 

Lookup Integrations

GET {api}/provisioning/uerp/provisioning/mgmt/endpoints?expand&external&$filter=((customProperties.__facade_endpoint_type%20ne%20%27vmc%27)%20and%20(customProperties.isExternal%20eq%20%27true%27))&$orderby=name%20asc&$top=20&$skip=0

 

Create new project integration

POST {api}/content/api/sources
{
   "name" : "Danieles Github",
   "type" : "com.github.saas",
   "projectIds" : [ "{projectId}" ],
   "config" : {
      "path" : "blueprints",
      "branch" : "vmworld2019",
      "repository" : "bluebossa63/scscopcas",
      "contentType" : "BLUEPRINT",
      "projectName" : "INT",
      "integrationId" : "126192f657b3d87558e235740ac2e"
   },
   "syncEnabled" : true
}

POST {api}/content/api/sources
{
   "name" : "Danieles Github",
   "type" : "com.github.saas",
   "projectIds" : [ "{projectId}" ],
   "config" : {
      "path" : "actions",
      "branch" : "vmworld2019",
      "repository" : "bluebossa63/scscopcas",
      "contentType" : "ABX_SCRIPTS",
      "projectName" : "INT",
      "integrationId" : "126192f657b3d87558e235740ac2e"
   },
   "syncEnabled" : true
}

 

POLLING until enough blueprints (GIT synch may take a while)

GET {api}/blueprint/api/blueprints?expand=true&projects={projectId}&size=50&page=0

END POLLING

 

FOR EACH blueprint and version:

GET {api}/blueprint/api/blueprints/{blueprintId}/versions/?expand=true&size=10&orderBy=version+DESC&page=0
GET {api}/blueprint/api/blueprints/{blueprintId}/versions/1.0
POST {api}/blueprint/api/blueprints/{blueprintId}/versions/1.0/actions/release
{
   "id" : "1.0",
   "selfLink" : "/blueprint/api/blueprints/{blueprintId}/versions/1.0",
   …
}

 

Create new Service Broker Content Source

POST {api}/catalog/api/admin/sources
{
   "name" : "INT",
   "description" : "INT blueprints",
   "typeId" : "com.vmw.blueprint",
   "config" : {
      "sourceProjectId" : "{projectId}"
   },
   "itemsImported" : 0,
   "itemsFound" : 0,
   "lastImportErrors" : [ ],
   "global" : true
}

 

Create Service Broker Entitlement

POST {api}/catalog/api/admin/entitlements
{
   "id" : "",
   "projectId" : "{projectId}",
   "definition" : {
      "type" : "CatalogSourceIdentifier",
      "id" : "{contentSolurceId}",
      "name" : "",
      "description" : "",
      "sourceType" : "",
      "numItems" : 0
   }
}

 

POLLING for Service Broker catalog items

GET {api}/catalog/api/items/?size=100&page=0&projectIds={projectId}

END POLLING

 

Update catalog items with Icon (check uploading icons with Chrome, I just reuse already uploaded icons)

PATCH https://api.mgmt.cloud.vmware.com/catalog/api/admin/items/{itemId}

 

Find blueprint to attach custom form:

GET {api}/catalog/api/admin/items/?search=esc-event-subscriber&size=20&sort=name%2Casc&page=0

GET {api}/catalog/api/admin/items/{contentItemId}

POST {api}/form-service/api/forms/designer/request?sourceId={contentItemId}&formType=requestForm&sourceType=com.vmw.blueprint
{
   Content of property "schema"
}

POST {api}/form-service/api/forms

See the sample form I used in a previous post on custom forms.

 

Create Subscriptions

Get Actions

GET {api}/abx/api/resources/actions?size=250&page=0

 

Get Topics

GET {api}/event-broker/api/topics?size=250&page=0

 

Create Subscription per Action and Topic

POST {api}/event-broker/api/subscriptions
{
   "id" : "sub_1572260902920",
   "type" : "RUNNABLE",
   "eventTopicId" : "compute.provision.pre",
   "name" : "compute.provision.pre_FAB",
   "subscriberId" : "temporary-value",
   "blocking" : false,
   "broadcast" : false,
   "description" : "Compute provision",
   "criteria" : "",
   "constraints" : {
      "projectId" : [ "{projectId}" ]
   },
   "timeout" : 0,
   "priority" : 10,
   "runnableType" : "extensibility.abx",
   "runnableId" : "8a76897e6c9affcc016ca211dbde0810",
   "messageTTL" : 0
}

Delete Project

Get Service Broker Project Sources and delete each item

GET {api}/content/api/sources?expand=true&size=50&projectIds={projectId}&page=1

DELETE {api}/content/api/sources/{sourceId}

 

Get Cloud Assembly blueprints for project and delete each item

GET {api}/blueprint/api/blueprints?expand=true&projects={projectId}&size=50&page=0

DELETE {api}/blueprint/api/blueprints/{blueprintId}

 

Get project subscriptions and delete each item

GET {api}/event-broker/api/subscriptions?expand=true&projects={projectId}&size=50&page=0

DELETE {api}/event-broker/api/subscriptions/{subscriptionId}

 

Get Project Actions and delete each item

GET {api}/abx/api/resources/actions?expand=true&projects={projectId}&size=50&page=0
DELETE {api}/abx/api/resources/actions/{actionId}?projectId={projectId}

 

Remove all zones and users from project (get project definition and simply clear the collections)

PATCH {api}/provisioning/mgmt/project-config

 

Remove project from kubernetes zones

GET {api}/cmx/api/resources/k8s-zones

Remove projectId from collection

PUT {api}/cmx/api/resources/k8s-zones/{zoneId}

 

Delete Entitlement

GET {api}/catalog/api/admin/entitlements?projectId={projectId}
 

DELETE {api}/catalog/api/admin/entitlements/{entitlementId}

 

Delete Content Sources

GET {api}/catalog/api/admin/sources?size=100&page=0
 

DELETE {api}/catalog/api/admin/sources/{contentSolurceId}

 

Delete Project

DELETE {api}/project-service/api/projects/{projectId}

 

VMworld Barcelona

If you’re attending don’t forget to register for the Breakout Session HBO3559BE. We will demo this functionality live on stage. And stay tuned: I will explain in detail how we have configured Code Stream for our evaluation test.