Bug Reference

Branch

master, 4.2.0

Introduction

Trusted compute pools with Intel Trusted Execution Technology enable isolation and tamper detection in boot process and complement run time protections. Meanwhile, hardware-based trust provides verification useful in compliance and trust status, which security and policy applications use to control workload.

Using Intel's TXT technology, a number of painpoints of a secure computing environments can be addressed. For example, Isolation is a key concern in a shared infrastructure where a lack of traditional guarantees of physical separation are lacking and multiple workloads may interfere with each other. Enforcement i.e. controls needed to enforce protection of Infrastructure can prevent pre-runtime environments are target of new attacks and low-level attacks are hard to detect and can be difficult to recover from. Encryption is another problem that can get worse in a cloud where data protection can be harder due to lack of boundaries and multi-tenancy.

Source: Intel TXT Overview

Purpose

This document describes the specifications and design of the feature.

References

http://www.intel.com/content/www/us/en/architecture-and-technology/trusted-execution-technology/malware-reduction-general-technology.html

Document History

Requirement:

• CloudStack will work with an attestation server to secure the deployed Hosts - the attestation server has the capability to compare launch values against "known good"
• When setting up a cloudstack environment, automatically understand which hosts are "trustworthy" and present it to the admin.
• Administrators are able to create a service offering that will allow users to select if they need the VMs to be deployed on trusted hosts.
• Ensure that instances requested in such a manner are always placed on trusted hosts. Instances that do not require a trusted host will not be blocked from getting deployed on a trusted host.
• Whenever a trusted host or the attestation server itself is rebooted, verify the trustworthiness.
• Migration of VM from a trusted to untrusted host should be allowed but it should raise an alert.

Non requirements

• The attestation server will not be managed by cloudstack. It'll have to be setup and configured and then registered with cloudstack for checking the trust attributes of a host.
• For checking the trust assertions of a host, a trust agent should be running on the host. Attestation server checks with the agent the trust relationship of the host. The trust agent should already have been configured.

Glossary

Feature Specification

Uses Cases

• A user wants to deploy an instance on a platform/OS/VMM for which trust verification has been done.
  • Administrator creates a service offering which enables deployment of an instance on only trusted user.
  • The service offering is made available to the user.
  • The user chooses this service offering while deploying an instance. Cloudstack picks up only trusted host in the deployment plan for deploying the instance.
• A user wants to move his instance to a host for which trust verification has been done.
  • The instance is shutdown/stopped by issuing a stopVirtualMachine call.
  • The service offering for the instance is updated. The new service offering allows deployment of an instance on only trusted host.
  • Instance start request is placed. Cloudstack only picks up trusted hosts to deploy it.

Architecture and Design description

Dependency on the attestation server client library
A java client library is available for easy integration with the attestation server. Cloudstack will be using it to register with the attestation server and to check for the trust relationship of a host. The library and this feature will be made available under non-oss.

Feature will be contained in a plugin

• The feature will be implemented and contained in a new plugin.
• It will provide the api for registering the attestation server with cloudstack.
• It will implement a listener for getting host connect notifications. It'll register with the agent manager to get the processConnect callbacks when agent connects to a host. On getting the callback it'll check with the attestation server (if configured) whether the given host is trusted or not. The callback gets called in the following scenarios
  • When a new host is added to the infrastructure.
  • When management server comes up and connects to the host.
  • When the host reboots/boots up, cloudstack will issue a processDisconnect callback when it looses connection and a processConnect callback on being able to connect again.

Registering an attestation server with cloudstack

• Only one attestation server can be registered with a cloudstack management server.
• The attestation service can be enabled or disabled through a global configuration parameter 'enable.attestation.service' (Boolean: true/false). It'll be disabled by default.
• A root administrator can register the details of an attestation server by making a registerAttestationServer api call. This is an async call. Cloudstack management server will open a connection to the attestation server and it'll use the KeystoreUtil.createUserInDirectory client library api call to register/create a user. On successful registration the attestation server details will be persisted in the db.
• The above request for a new user needs to be approved by an attestation server administrator. This is a manual process and will be included in the documentation.
• If an attestation server is already registered with the management server, any subsequent requests to register another attestation server will override the older registration.

Checking the trust relationship of an host

• Whenever management server connects to a host, the processConnect callback routine gets triggered for the plugin.
• It verifies if attestation check is enabled in the global config and if an attestation server has been registered.
• It opens a connection to the attestation server with the credentials registered with cloudstack.
• It then checks the assertion attributes for the host; i.e if the host is trusted or not. For that it makes an api.getSamlForHost(<HostIp>) call.
  • If a host assertion not available exception is thrown, it means the whitelist configuration for the host hasn't been done and it hasn't been registered with the attestation server.
  • To do the whitelist configuration and registration of host a configureWhiteList followed by registerHost api calls are made to the attestation server. Open Issue 1.
  • These calls will be needed only when management server connects to a host for the first time.
• The assertion attributes are checked to make sure the host is trusted. Accordingly the host_details table is updated with a new name value pair 'trusted':true/false. Open Issue 2.

Deploying an instance on a trusted host
Open Issue 3.

Migration and HA

• When hosts are listed for migration of an instance, if an instance is using an offering which requires trusted host, the host allocators will be updated to not to include hosts that are not trusted (information stored in the host_details table in the db). These hosts will be marked as 'Unsuitable' for migration (as seen in UI).
• Migration of instances using offering requiring trusted hosts, to untrusted hosts will be allowed. However, an alert will be raised to bring this to administrators attention.
• If an instance using a offering requiring trusted hosts goes down and HA is triggered for it, management server will try to bring it up on host that is trusted. If no trusted hosts are available HA for the instance will fail.

Database modifications

A new table will be created in the db to hold the attestation server details. Open Issue 4.

Web Services APIs

1. registerAttestationServer : A new api to register an attestation server with cloudstack. It will take the details of the attestation server as a parameter and check if a connection can be established to it.

   Parameters	Type	Required/Optional	Comments
   url	String	Required	Url of the attestation server
   username	String	Required	Username with which cloudstack should register and connect with the attestation server
   password	String	Required	Password with which cloudstack should register and connect with the attestation server

Test Guidelines

<TBD>

Hypervisor support

The functionality will be made available for VmWare, KVM and XenServer.

Supportability characteristics

Logging

All successful operations are logged to INFO, all exceptions/failures to ERROR, and all synchronization checks to DEBUG.

Events/Alerts

If an instance created from a service offering requires a trusted host and it is placed on an untrusted or unattested host, an alert will be raised to bring it to administrators attention.

UI Flow

<TBD>

Open Issues

1. The documentation available suggests that the configureWhiteList and registerHost api calls can be used for registering a server for attestation. These apis haven't been tried as part of poc. Need to confirm this. Should we keep the white list configuration and registration of a host out of the scope? Additionally, should management server check with the attestation server if a host is trusted or not and on getting a host not registered exception, register it? Or instead, keep the information on whether a host is registered or not in the db.
2. Proposal is to fill the host_details table with an additional attribute for an host on checking if it is trusted or not. Should this information be stored elsewhere, probably in a separate table?
3. For Anti-affinity and Dedicated resources features changes are being made to address how an instance is deployed. It'll also affect this feature. For the current feature changes are needed in the host allocators to not to pick up an untrusted host if the service offering requires trusted host. How should a service offering be updated to include the information that a trusted host is required? Should it be done using name/value pairs, as has been suggested for Dedicated resource feature. However, name/value pairs will be the property of a planner (I could be wrong); and here we need to pass this information to the allocators.
4. Should the attestation server details be kept in a new table or can they be kept in the host table itself.