You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Bug Reference

coming soon...

Branch

coming soon...

Introduction

Purpose

This document lays out the integration of the OpenStack Swift Object Store as an object store service made available through the CloudStack UI.  This integration allows CloudStack users to authenticate against Swift using their CloudStack credentials as well as exposing a UI in CloudStack for users to manage their the documents in Swift.  

A key enabling component for this integration is a Swift Auth middleware that I wrote called 'mauth'.  This middleware supports CloudStack by default, but it is also extensible and open source.  

How 'mauth' works:

To get thing started I will show a curl example using mauth to authenticate a CloudStack user.
In this example, http://127.0.0.1:8080/v1.0 is pointing at the Swift cluster.
The $username and $apikey reflect the CloudStack user and his api key respectively. 

    Request for authentication
    curl -v -H "X-Auth-User: $username" -H "X-Auth-Key: $apikey" http://127.0.0.1:8080/v1.0
    returns: $auth_token and $swift_storage_url

    Request container list
    curl -v -X GET -H "X-Auth-Token: $auth_token" $swift_storage_url

In this example, the user requests authentication and they are given back a token and their storage url for where their objects are stored.  They can then use that token and url for subsequent requests.

How the integration works:

A UI for Object Store is added to CloudStack in the left hand services menu.  When a user clicks on the Object Store menu option, they are authenticated with the Swift backend (assuming they have api keys generated, if they don't, it will notify them that they need to generate an api key to use the service).  Once authenticated, the user's containers and objects are listed in the main area using the treeView mechanism.  Selecting a container or object gives details about it as well as shows actions that can be taken on it.

A technical overview of the integration:

There are a few challenges with this integration.  I will give a basic overview of the components in play and touch on some of the reasons for the decisions, but more detail will be added later in the document.

Because the object storage data is often large and very network intensive, we do not want the data to actually flow through the CloudStack box as it would introduce a huge bottleneck.  Because of this, there is no Java component of this implementation on CloudStack.  Instead, the traffic is routed to Swift or CloudStack based on the url structure by a load balancer which sits in front of the two services.  The technology used for load balancing (and ssl termination) is not important, but to illustrate the concept, I used HAproxy and Pound for these tasks.

 

 

References

  • mauth - An extensible middleware for OpenStack Swift which enables Swift to authenticate CloudStack users.  (Developed by Will Stevens @ CloudOps)

Document History

Glossary

Feature Specifications

  • 'Object Store' added to the main services menu on the left
  • User can List all containers and objects in their Swift account
  • User can Add/Delete Swift Containers
    • Supports cascade delete, so it will first delete all the objects in the container
  • User can Add/Delete Swift Objects
  • User can Add/Delete Folders in containers to group objects (unique to this implementation)
    • Supports cascade delete of folders, so it will first delete all the objects in the folder
  • User can create containers as Public or Private
    • Public - Each object in the container can be accessed without needing authentication
    • Private - Each object in the container requires the user to authenticate to access it
  • User can modify the Public/Private setting for existing containers
  • A public URL is shown in the details for each object in a public container


  • When a user logs in, a change had to be made to add the 'api key' to the CS user object in the browser
  • The load balancer exposes the URL that is used by both CloudStack and Swift calls
  • All Swift calls are done using AJAX and the load balancer handles routing the calls to Swift
  • Swift uses the HTTP PUT verb and browsers do not support the PUT verb, so the client sends the requests as POST and the LB translates to PUT
    • This means that the current implementation does not support any of the Swift API calls that require POST
    • More details on this can be found in the HAproxy config attached at the end of this document
  • This functionality requires the 'mauth' middleware to be installed on Swift
  • A global variable needs to be added to CS to enable/disable the Object Store functionality

  • put a summary or a brief description of the feature in question 
  • list what is deliberately not supported or what the feature will not offer - to clear any prospective ambiguities
  • list all open items or unresolved issues the developer is unable to decide about without further discussion
  • quality risks (test guidelines)
    • functional
    • non functional: performance, scalability, stability, overload scenarios, etc
    • corner cases and boundary conditions
    • negative usage scenarios
  • specify supportability characteristics:
    • what new logging (or at least the important one) is introduced
    • how to debug and troubleshoot
    • what are the audit events 
    • list JMX interfaces
    • graceful failure and recovery scenarios
    • possible fallback or work around route if feature does not work as expected, if those workarounds do exist ofcourse.
    • if feature depends other run-time environment related requirements, provide sanity check list for support people to run
  • explain configuration characteristics:
    • configuration parameters or files introduced/changed
    • branding parameters or files introduced/changed
    • highlight parameters for performance tweaking
    • highlight how installation/upgrade scenarios change
  • deployment requirements (fresh install vs. upgrade) if any
  • system requirements: memory, CPU, desk space, etc
  • interoperability and compatibility requirements:
    • OS
    • xenserver, hypervisors
    • storage, networks, other
  • list localization and internationalization specifications 
  • explain the impact and possible upgrade/migration solution introduced by the feature 
  • explain performance & scalability implications when feature is used from small scale to large scale
  • explain security specifications
    • list your evaluation of possible security attacks against the feature and the answers in your design* *
  • explain marketing specifications
  • explain levels or types of users communities of this feature (e.g. admin, user, etc)

Use cases

put the relevant use case/stories to explain how the feature is going to be used/work

Architecture and Design description

  • discussion of alternatives amongst design ideas, their resources/time tradeoffs and limitations. Explain why a certain design idea is chosen over others
  • highlight architectural patterns being used (queues, async/sync, state machines, etc)
  • talk about main algorithms used
  • explain what components are being changed and what the dependent components are
  • regarding database: talk about tables being added/modified
  • performance implications: what are the improvements or risks introduced to capacity, response time, resources usage and other relevant KPIs
  • preferably show class diagrams, sequence diagrams and state diagrams
  • if possible, publish signatures of all methods classes and interfaces implement, and the explain the object information of different classes

Web Services APIs

list changes to existing web services APIs and new APIs introduced with signatures and throughout documentation

UI flow

  • either demonstrate it visually here or link to relevant mockups

IP Clearance

  • what dependencies will you be adding to the project?
  • are you expecting to include any code developed outside the Apache CloudStack project?

Appendix

Appendix A:

Appendix B:

  • No labels