EC2 Server Configuration¶
The OpenNebula EC2 Query is a web service that enables you to launch and manage virtual machines in your OpenNebula installation through the Amazon EC2 Query Interface. In this way, you can use any EC2 Query tool or utility to access your Private Cloud. The EC2 Query web service is implemented upon the OpenNebula Cloud API (OCA) layer that exposes the full capabilities of an OpenNebula private cloud; and Sinatra, a widely used light web framework.
The current implementation includes the basic routines to use a Cloud, namely: image upload and registration, and the VM run, describe and terminate operations. The following sections explain you how to install and configure the EC2 Query web service on top of a running OpenNebula cloud.
The OpenNebula EC2 Query service provides an Amazon EC2 Query API-compatible interface to your cloud, that can be used alongside the native OpenNebula CLI or OpenNebula Sunstone. The OpenNebula distribution includes the tools needed to use the EC2 Query service.
Requirements & Installation¶
You must have an OpenNebula site properly configured and running. Be sure to check the OpenNebula Installation and Configuration Guides to set up your private cloud first. This guide also assumes that you are familiar with the configuration and use of OpenNebula.
The OpenNebula EC2 Query service was installed during the OpenNebula installation, and also the dependencies of this service, as explained in the installation guide
The service is configured through the
/etc/one/econe.conf file, where you can set up the basic operational parameters for the EC2 Query web service. The available options are:
tmpdir: Directory to store temp files when uploading images
onedXML-RPC service, default
host: Host where the
econeserver will run
port: Port which the
econeserver will use
ssl_server: URL for the EC2 service endpoint, when configured through a proxy
debug_level: Log debug level,
auth: Authentication driver for incoming requests
core_auth: Authentication driver to communicate with OpenNebula core. Check this guide for more information about the core_auth system.
File based templates
use_file_templates: Use former file-based templates for instance types instead of OpenNebula templates
instance_types: DEPRECATED. The VM types for your cloud
describe_with_terminated_instances: Include terminated instances in the
describe_instancesXML. When this parameter is enabled, all the VMs in the DONE state will be retrieved in each
describe_instancesaction and then filtered. This can cause performance issues when the pool of VMs in the DONE state is huge.
terminated_instances_expiration_time: Terminated VMs will be included in the list until the termination date + terminated_instances_expiration_time is reached.
datastore_id: Datastore in which the Images uploaded through EC2 will be allocated, by default 1.
cluster_id: Cluster associated with the EC2 resources. By default no Cluster is defined.
elasticips_vnet_id: Virtual Network containing the elastic IPs to be used with EC2. If not defined, the Elastic IP functionality is disabled.
associate_script: Script to associate a public IP with a private IP. Arguments: elastic_ip private_ip vnet_template (where the template is base64-encoded).
disassociate_script: Script to disassociate a public IP. Arguments: elastic_ip private_ip vnet_template (where the template is base64-encoded).
ebs_fstype: FSTYPE that will be used when creating new volumes (DATABLOCKs)
:host must be a FQDN; do not use IPs here.
The cloud users have to be created in OpenNebula by
oneadmin using the
oneuser utility. Once a user is registered in the system, using the same procedure as to create private cloud users, they can start using the system.
The users will authenticate using the Amazon EC2 procedure with
AWSAccessKeyId (their OpenNebula user name) and
AWSSecretAccessKey (their OpenNebula hashed password).
The cloud administrator can limit the interfaces that these users can use to interact with OpenNebula by setting the driver
public for them. Using that driver, cloud users will not be able to interact with OpenNebula through Sunstone, CLI or XML-RPC.
$ oneuser chauth cloud_user public
Defining VM Types¶
You can define as many Virtual Machine types as you want. Just:
- Create a new OpenNebula template for the new type and make it available for the users group. You can use restricted attributes and set permissions like any other OpenNebula resource. You must include the EC2_INSTANCE_TYPE parameter inside the template definition, otherwise the template will not be available to be used as an instance type in EC2.
# This is the content of the /tmp/m1.small file NAME = "m1.small" EC2_INSTANCE_TYPE = "m1.small" CPU = 1 MEMORY = 1700 ...
onetemplate create /tmp/m1.small onetemplate chgrp m1.small users onetemplate chmod m1.small 640
The template must include all the required information to instantiate a new virtual machine, such as network configuration, capacity, placement requirements, etc. This information will be used as a base template and will be merged with the information provided by the user.
The user will select an instance type, along with the AMI id, keypair and user data, when creating a new instance. Therefore, the template should not include the OS, since it will be specified by the user with the selected AMI.
The templates are processed by the EC2 server to include specific data for the instance.
Starting the Cloud Service¶
To start the EC2 Query service, just issue the following command:
You can find the econe server log file in
To stop the EC2 Query service:
In order to benefit from the Keypair functionality, the images that will be used by the econe users must be prepared to read the EC2_PUBLIC_KEY and EC2_USER_DATA from the CONTEXT disk. This can be achieved easily with the new contextualization packages, generating a new custom contextualization package like this one:
#!/bin/bash echo "$EC2_PUBLIC_KEY" > /root/.ssh/authorized_keys
Enabling Elastic IP Functionality¶
An Elastic IP address is associated with the user, not a particular instance, and users control that address until they choose to release it. This way, users can remap their public IP addresses to any of their instances.
In order to enable this functionality you have to follow the following steps in order to create a VNET containing the elastic IPs:
- Create a new Virtual Network as oneadmin, containing the public IPs that will be controlled by the EC2 users. Each IP must be placed in its own AR:
NAME = "ElasticIPs" PHYDEV = "eth0" VLAN = "YES" VLAN_ID = 50 BRIDGE = "brhm" AR = [IP=10.0.0.1, TYPE=IP4, SIZE=1] AR = [IP=10.0.0.2, TYPE=IP4, SIZE=1] AR = [IP=10.0.0.3, TYPE=IP4, SIZE=1] AR = [IP=10.0.0.4, TYPE=IP4, SIZE=1] # Custom Attributes to be used in Context GATEWAY = 18.104.22.168
onevnet create /tmp/fixed.vnet ID: 8
This VNET will be managed by the oneadmin user; therefore
USE permission for the EC2 users is not required.
- Update the
econe.conffile with the VNET ID:
- Provide associate and disassociate scripts
The interaction with the infrastructure has been abstracted. Therefore two scripts have to be provided by the cloud administrator in order to interact with each specific network configuration. These two scripts enable us to adapt this feature to different configurations and data centers.
These scripts are language agnostic and their path has to be specified in the econe configuration file:
:associate_script: /usr/bin/associate_ip.sh :disassociate_script: /usr/bin/disassociate_ip.sh
The associate script will receive three arguments: elastic_ip to be associated; private_ip of the instance; Virtual Network template, base64-encoded.
The disassociate script will receive similar arguments: elastic_ip to be disassociated, private_ip, and Virtual Network template.
Scripts to interact with OpenFlow can be found in an ecosystem project
Using a Specific Group for EC2¶
It is recommended to create a new group to handle the EC2 cloud users:
onegroup create ec2 ID: 100
Create and add the users to the EC2 group (ID:100):
oneuser create clouduser my_password ID: 12 oneuser chgrp 12 100
Also, you will have to create ACL rules so that the cloud users are able to deploy their VMs in the allowed hosts.
onehost list ID NAME CLUSTER RVM ALLOCATED_CPU ALLOCATED_MEM STAT 1 kvm1 - 2 110 / 200 (55%) 640M / 3.6G (17%) on 1 kvm2 - 2 110 / 200 (55%) 640M / 3.6G (17%) on 1 kvm3 - 2 110 / 200 (55%) 640M / 3.6G (17%) on
These rules will allow users inside the EC2 group (ID:100) to deploy VMs in the hosts kvm01 (ID:0) and kvm03 (ID:3)
oneacl create "@100 HOST/#1 MANAGE" oneacl create "@100 HOST/#3 MANAGE"
You have to create a VNet network using the
onevnet utility with the IPs you want to lease to the VMs created with the EC2 Query service.
onevnet create /tmp/templates/vnet ID: 12
Remember that you will have to add this VNet (ID:12) to the users group (ID:100) and give USE (640) permissions to the group in order to get leases from it.
onevnet chgrp 12 100 onevnet chmod 12 640
You will have to update the NIC template, inside the
/etc/one/ec2query_templates directory, in order to use this VNet ID.
Configuring an SSL Proxy¶
The OpenNebula EC2 Query Service runs natively just on normal HTTP connections. If the extra security provided by SSL is needed, a proxy can be set up to handle the SSL connection that forwards the petition to the EC2 Query Service and returns the answer to the client.
This set up needs:
- A server certificate for the SSL connections
- An HTTP proxy that understands SSL
- EC2Query Service configuration to accept petitions from the proxy
You can find instructions on configuring lighttpd as a proxy for the example
cloudserver.org in the Sunstone Setup guide. Follow those, but use
9869, as the
econe.conf needs to define the following:
# Host and port where econe server will run :host: localhost :port: 4567 #SSL proxy URL that serves the API (set if is being used) :ssl_server: https://cloudserver.org:8443/
Once the lighttpd server is started, EC2Query petitions using HTTPS URIs can be directed to
https://cloudserver.org:8443, that will then be unencrypted, passed to localhost, port 4567, satisfied (hopefully), encrypted again and then passed back to the client.
:ssl_server must be a URL that may contain a custom path.