Thursday, March 24, 2011

Composting vApp(s)

I was so tempted to write code here but I should point the readers to the community (link available in the resource section) where complete example is available. Never the less let me add the pointers here.

When we compose a vApp, we are creating a new vApp which can contain its child VM(s) from various sources. These sources can be VM from an existing template or a VM from existing vApp.  Simply put we are picking up assortment of VM that we like and making a copy of them to be part of the new vApp.
Let’s see step by step, how to go about it.
1.    First and foremost we need to create composition parameters by using ComposeVAppParams object.  We need to set basic information such as Name, Boolean indicating if we want to deploy it and so forth. We also need to set up the instantiation details as we did while instantiating template to vApp.  That basically involves setting up the vApp Network configuration.
2.    Composition parameters contain a list of ‘Source Items’. These source items are nothing but references to VM that need to be added to the composed vApp.

List<SourcedCompositionItemParamType> items =  composeVAppParamsType.getSourcedItem();
ReferenceType vappTemplateVMRef = new ReferenceType();
vappTemplateVMRef.setHref(vmHref);
vappTemplateVMRef.setName(“ChildVm1”);
vappTemplateItem.setSource(vappTemplateVMRef);
   
// Now add this VM reference to the Composition parameters’ item list
items.add(vappTemplateItem);
// More items (VM) can be added to this ‘items’ list

3.    Optionally set up networking for the all child VM(s) now.

One can chose to use the same VM and add it multiple times such that the composed vApp has  more than one such child VM.  It is necessary to use different name for the Reference used in the step 2 above.  We are mainly interested in the HREF that we set on the reference object.


The actual step of composing the vApp using SDK looks as simple as
Vapp vapp = vdc.composeVapp(createComposeParams(vappTemplateRef, vdc));

The actual work is creating the composing details
When recomposing the vApp if you are adding new VM(s) then the steps are exactly same as above where in actual work is creating RecomposeVAppParamsType. You need to add new SourcedComposition Item. If VM needs to be deleted from existing vApp then you need to populate the ‘Delete Item’ list.

// create the recompose vapp params type.
RecomposeVAppParamsType recomposeVAppParamsType = new RecomposeVAppParamsType();
recomposeVAppParamsType.setName("RecomposedVapp");
       
 // adding the vm item.
 List<SourcedCompositionItemParamType> newItems = recomposeVAppParamsType.getSourcedItem();
  newItems.add(vmItem);

Thursday, January 6, 2011

How to create a new vApp

There are many ways one can create a vApp in vCloud, and they vary depending on where one starts. Here are some of the options. As promised we will use the SDK to illustrate the example(s).

  • Create one form a template (POST .../api/v1.0/vdc/<vdc-id>/action/instantiateVAppTemplate)
  • Start from OVF package.
  • Assorted vApp:
    • Compose new vApp from templates and/or existing vApp
      (POST .../api/v1.0/vdc/<vdc-id>/action/composeVAp)
    • Recompose vApp to add/delete VM in it.
      POST .../api/v1.0/vApp/<vapp-id>/action/recomposeVApp (This is not really creating a new vApp so we will visit it next time).
  • Clone from existing vApp POST .../api/v1.0/vdc/<vdc-id>/action/cloneVApp
  • Importing from vSphere 
Let’s visit the above one by one:
1.    Start from OVF Package:
This is the most ground up operation one can think of for creating a vApp. You are basically loading the vApp from the OVF as a vApp Template in a catalog and then its matter of details for instantiating vApp from template as in #2. So let’s look at the first thing first: uploading OVF as vApp Template. The OVF file essentially describes how the vApp is configured using various ‘Sections’ and bundles with the  disk files of the virtual machine. Even before we upload this vApp description we need to create a ‘vApp Template’ object in the vCloud. The SDK provides a method on Vdc object ‘createVappTemplate’.

UploadVAppTemplateParamsType vappTemplParams = new
UploadVAppTemplateParamsType();

vappTemplParams.setDescription("New Template Description");
vappTemplParams.setName("NewVAppTemplate");

VappTemplate vappTemplate = vdc.createVappTemplate(vappTemplParams);

It goes without saying that this template needs to be later added to a Catalog. Once the object is created we upload the OVF descriptor using method on newly created vApp template ‘uploadOVFFile’.

File ovfFile = new File(ovfFileLocation);

FileInputStream ovfFileInputStream = new FileInputStream(ovfFile);

vappTemplate.uploadOVFFile(ovfFileInputStream, ovfFile.length());

 As a result of resolving the OVF descriptor, the Template object knows how many virtual machines are part of the vApp and how many disk every machine has. We are yet to upload the disk files so as a pre-requisite for that it creates a place holder link for those disk files and adds details to the vApp Template object. Make sure that you refresh the vApp Template object on the client side so as to get this newly available configuration data. Now you can upload the disk files to the cloud and take a coffee break.

// wait to make sure OVF descriptor is uploaded.
// Refresh the vApp Template object.
File vmdkFile = new File(vmdkFileLocation);
FileInputStream vmdkFileInputStream = new FileInputStream(vmdkFile);

vappTemplate.uploadFile(vmdkfileName, vmdkFileInputStream, vmdkFile.length());

 Once done go on to the next step of creating a vApp from template as follows. All this available in the sample provided as part of the vCloud SDK.

Wednesday, November 24, 2010

vCloud SDK

VMware released SDK for vCloud API for Java, PHP and C# to bring the API in developer's own world. The success of API relies on not just how technically superior it is but how easy it is to adapt in real life in developer's choice of language. The SDK is an attempt to exactly do the same.
We will use Java SDK in most of our examples primarily because it was first one to get released but more importantly its widely used in the ecosystem.
As we saw that one of the important tenet of API is the data model. The users expect the solid object model that is in line with their use cases and their choice of language and the Java SDK exactly brings it to them. The vCloud API defines this model using XML schemas which are commonly available as part of the specification. Java SDK brings these REST resources to developers as Java object using JAXB. This completely hides the clients from the XML that defines the request/response payload for the vCloud API. Simply put NO XML processing code in clients.
It also handles the HTTP communication so that the client is dealing with only application level Java objects. Developer's life is further eased by providing utility or wrapper classes that hides some of the complexities and help avoid boiler plate code.

API Commandments
•    Less is more.
•    vCloud API Resource model types should be accessible to the clients as is; do not hide them
•    Use composition to handle the use cases and not inheritance. 
•    Hide the URL semantics
•    Hide HTTP communication details
•    Hide object marshalling/un-marshalling to and from the XML needed for the REST request-response
•    Expose the Object Oriented nature of REST resources
•    Associate the resources with their operations

Here are some of the good starting points for SDK and vCloud API. Do not forget to check the samples bundled with the SDK.

http://communities.vmware.com/community/developer/forums/vcloudapi
http://communities.vmware.com/community/developer/forums/vcloudsdkjava
http://communities.vmware.com/community/developer/forums/vcloudsdk-net

Very helpful presentations  @  VMworld.
http://www.vmworld.com/docs/DOC-5153
http://www.vmworld.com/docs/DOC-5405

We will visit the vCloud API again, but now using the SDK.  I will refrain from repeating the information that’s already available in the SDK samples.:-)

Thursday, October 28, 2010

HATEOS and resource navigation in vCloud API

HATEOS stands for Hypermedia as the Engine of Application State. This is a principle or constraint used by the designers when defining resources in RESTful way. Bottom line of this principle is that client needs to know the a fixed or top level URL and be able to access all the necessary resources. Which means every response or resource description should further indicate how to access either related resource or perform certain actions on the resource. Since resources are identified by URL and actions are also represented in the form of URL as we saw in earlier post, the resource description or the HTTP responses should include links or URLs for the navigation purpose.

This is the third level of maturity as described by Richardson. Lets see how vCloud API adheres to this principle.
vCloud API defines a top level URL http://my.vcloud.com/api/versions This URL provides information about all the versions supported by the vCloud Server. For every version supported there is log in URL associated with every version number (if there are more than one :-), which is obviously not the case for the version 1.0)
This is how it looks:
GET http://my.vcloud.com/api/versions
<SupportedVersions xmlns="http://www.vmware.com/vcloud/versions" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/versions http://my.vcloud.com/api/versions/schema/versions.xsd">
    <VersionInfo>
        <Version>1.0</Version>
        <LoginUrl>http://my.vcloud.com/api/v1.0/login</LoginUrl>
        <MediaTypeMapping> ... </MediaTypeMapping>
        ...
    </VersionInfo>
    ....
</SupportedVersions>

Now user needs to fetch the log in URL and perform the log in operation. Which in turn returns a list of 'Reference' to all the Organizations available to this user to access.

<OrgList  ...>
    <Org type="application/vnd.vmware.vcloud.org+xml" name="my-org2" href="https://my.vcloud.com/api/v1.0/org/537058365"/>
    <Org type="application/vnd.vmware.vcloud.org+xml" name="my-org1" href="https://my.vcloud.com/api/v1.0/org/1041520205"/>
</OrgList>
You must have realized by now that the above are refrences/links to the Organization resources. They are not the actual resources. SO we get the URL for the Organization using the 'href' attribute and perfrom the GET operation on it. Notice in following response that the resource provides link to itself as well.

e.g. GET https://my.vcloud.com/api/v1.0/org/537058365
This is of the form https://<server>/api/v1.0/org/{id}. One of the other tenet of HATEOS implemented by vCLoud API is that the URL are opaque. Which means client is discouraged to cache these URL and rely instead on the 'href' and links.

Now lets look into the Organization resource in response to above request
<Org xmlns="http://www.vmware.com/vcloud/v1" name="my-org2" type="application/vnd.vmware.vcloud.org+xml" href="https://my.vcloud.com/api/v1.0/org/537058365" ... >
    <Link rel="down" type="application/vnd.vmware.vcloud.vdc+xml" name="org2_vdc1" href="https://my.vcloud.com/api/v1.0/vdc/1700638305"/>
    <Link rel="down" type="application/vnd.vmware.vcloud.tasksList+xml" href="https://my.vcloud.com/api/v1.0/tasksList/537058365"/>
    <Description>my-org2</Description>
    <FullName>my-org2</FullName>
</Org>

The difference in this response to note is the 'Link' element. This element provides further navigation paths. Specifically how to access the TaskList resource and the vDC resource contained in this Organization. (refer to the UML diagram in my earlier post for detailed relationsships between resources.) Another point to note is the 'rel' attribute. In addition to just giving the navigation paths it also provides the relationsship between this resource and the linked resource using the 'rel' attribute. for example the 'down' indicates containment such that this resource contains the linked resource.

The Link with 'rel' attribute also indicates what actions can be perfromed on this resource. Following Link from the 'Vdc' indicates that we can upload a vApp Template using the URL associated with it.

<Link rel="add" type="application/vnd.vmware.vcloud.uploadVAppTemplateParams+xml" href="https://my.vcloud.com/api/v1.0/vdc/1700638305/action/uploadVAppTemplate"/>
And when it comes to resources like vApp we indicate the available actions as following. This vApp gives out the link to power it off and also indicates what type of relationship (actionin this case) the Link has with the resource.
<VApp ... >
    <Link rel="power:powerOff" href="https://my.vcloud.com/api/v1.0/vApp/vapp-1299560622/power/action/powerOff"/>
    ...
</VApp>

The vCloud API provides only links that are relevant to the curent state of the resource. For example the above link in vApp will only be available if the vApp is in poer on state. If the vApp is powered off then there will be corresponding link to power on operation.

so in conclusion we just saw how the vCloud API adheres to the third level of REST maturity but while doing so we also saw how we can navigate the vCloud resources. Let me promise you one thing, this blog post would be much smaller if I avoid using those XML examples, and you know what it will make your code also very small, readable and maintainable if some one else takes care of those XML and HTTP verbs such as GET/POST. We will see how the SDK released along with the vCloud API can make our life easy in subsequent posts. 

Wednesday, October 27, 2010

RESTful vCloud API


Many people use REST in many different ways depending on their requirements and more importantly their understanding. One of the favorite topics of blogs is comparison of REST style with SOA and when it comes to OO API and Service oriented API. I will stay clear of opinions but map the vCloud API to the REST pattern. This will help understand why the vCloud API looks the way it is.

Before we get into the details we need to understand the design model and its maturity levels. This job is done by the stalwarts and we will stand on their shoulders to get the better view of vCloud API. Check out these links for more details on the REST Maturity model by Martin Fowler and Leonard Richardson. ( Richardson Maturity Model

Many people call their API; ‘REST API’ when in practice the end points are actually defined as service endpoints rather than resources. They use the XML (or JSON) payload and define the end points very similar to SOAP or XML-RPC style interfaces (e.g. CreateImage, DeregisterImage) . Main point to note these are verbs and not nouns. Those who have done OO programming and defining the objects know that when they read the problem story they underline the nouns to define objects and verbs to define the methods.
When we get into more matured REST API we focus on the nouns to define the resources that are implemented by the server and then expose them to client such that the client now makes requests to the resources rather than using the singular service.
In very simple terms it is almost like calling procedure (services/actions) against operating on objects/resources.

 The vCloud API takes the REST approach by defining resources such as Organization, vDC, vApp, vAppTemplate and so forth.  Client uses HTTP verbs GET, POST, DELETE to inquire the state of the resource (Get the details of the resource), update/create resource and delete the resource. The actual resources and their relationship using the UML diagram.

There is small caveat where the pure REST approach falls short and vCloud API addresses it by introducing ‘action’ verb.  These actions not necessarily create a new resource as one would expect from the generally accepted semantics of HTTP POST operation. As mentioned above one may chose to create URL such as POST …/api/v1.0/poweronvapp which more or less looks like service/operation but vCloud API takes REST like approach where it uses the resource where HTTP verb can be directly used and use the ‘action’ paradigm in conjunction with it. Following examples gives this example.

GET .../api/v1.0/org/<org-id> get organization for given id.

POST .../api/v1.0/catalog/<catalog-id>/catalogItems Create a Catalog Item in given Catalog.

The actions are performed using the POST as follows:  

POST .../api/v1.0/vdc/<vdc-id>/action/instantiateVAppTemplate  : Create/Instantiate new  vApp from given Template (this is sent as part of the request payload, Instantiation Parameters). The action is performed on the vDC which will house the vApp for execution.

POST .../api/v1.0/vApp/<vapp/vm-id>/power/action/powerOn Power on vApp.

The vCloud API handles the long running operations by responding to such actions with the Task resource that represents the tasks in progress. The client can then treat this Task resource like any other resource to enquire the status by doing GET operations.  This resource further provides means (links) to cancel the task (action on the Task resource) and hence does not have any direct POST operations.  Typically the Task is maintained even if it is completed (with success/failure) or canceled and hence not deleted by the user directly.

Hope it gives the general idea of the RESTfullness of the vCloud API. We will visit MOST important feature of the API, which make it Level 3 mature as it uses the HATEOAS (‘Hypertext as the Engine of Application State’) when we discuss how to navigate the resources.

Friday, October 22, 2010

Introduction to vCloud API

As many of you, who are keeping track of cloud space and in particular Vmware offering, are aware that Vmware released vCloud API. Vmware vCloud Director implements the version 1.0 of the vCloud API. Since the release Vmware has also released SDK in Java, C# and PHP. Since I am involved with the API and SDK for some time let’s take our first steps for embracing the vCloud API.

Instead of writing the programming guide which available here along with few others documents
vCloud API is REST based API. The three main tenets of any API are the Object Mode, Interfaces and the Protocol.  As you know the API being REST based presents the Object Model as the REST resources which carry small set of fixed methods. The REST lends its personality of ‘Many nouns and few verbs’ to this API as well.  In this posts let me just introduce you the Object or resource model in using UML.

Basically the vCloud API has 3 parts: User API, Admin API and Extension API
User API is focused on the self provisioning for the end user and exposes the pure virtual resources such as vApp, vApp Templates, Catalogs, Organization and so forth. Although it allows access to objects such Organization, vDC it does not allow any administrative functions on these objects to create or delete them. This is left to the Admin API which is geared more towards the System or Organization administrators. Finally the Extension API is based on the extensibility framework of the vCloud API that allows access to different system mainly for the purpose of the administration.
 

vCloud Director 1.0 provides Admin API extensions that provides information about the underlying vSphere platform entities(Not all of them but the significant ones of the interest of the cloud administrator) .


Following UML diagram shows the resources participating in User API. The resources marked in colored boxes cannot be modified using the User API.



The following UML diagram shows the resources participating in the Admin API. Note that a user logging in with Administrative privileges always gets enhanced view of the resource such as Organization, vDC etc. The actual type they get back is AdminOrg, AdminVdc and so forth.

















The Extension API uses the extensibility framework to extend the vCloud API capability to let access the objects from underlying vShpere infrastructure. This is implemented in vCloud Director Product. Any vendor implementing the vCloud API on platform other than vSphere can chose to expose the data model of the underlying platform or integrated systems using these Admin API. We will focus on the extension API available in vCloud Director and the following UML shows how it crosses the boundaries to relate to the vShpere objects.


In subsequent blogs we will address the other two tenets of the API namely interfaces and the protocol.

Friday, October 8, 2010

First step

Although I have spent last couple of years working on vCloud, its still a first step for me in the blogosphere on this subject. So thought I will set some guidelines and objectives so we know what to expect here.
This blog is for all those who are set out on cloud journey and those who are packing their bags. We will start this journey together with first step here.
Since my focus is on Cloud API and cloud solutions expect some discussion on these topics to start with. But we are not restricted to only these topics. Neither we are restricted to only VMware vCloud.
so stay tuned ...
If you missed the VMworld 2010 in San Francisco, you have one more chance in Copenhagen, next week.