How To: Build your own Cloud

July 2nd, 2013 by

Building CloudsBuild your own cloud: this has been on my mind for awhile.  With “cloud computing” becoming a buzzword the last couple years, I figured it was about time I put one together.  In the past, I have put together cloud-like solutions and arguably clouds, by some people’s standards.  The clouds have been traditional VMware with vCloud Director, or leveraged OnApp. I have also played with Amazon Web Services, Azure, Google App Engine, etc. in the public space. I really wanted to test and see how simple and quick I could get some of the open source cloud offerings up and running.  The 3 I looked at were CloudStack, OpenStack, and Eucalyptus.  The main reason for these 3 was the buzz and market traction I have seen with them.  What I wanted to see was how long and the pain factor for getting them up and running, as well as how quickly after the install I could start deploying compute instances and storage.  Lets start the comparison:

* Note: Much of the content and steps below are pulled directly from the different cloud providers’ guides with some of my own commentary laced in.

CloudStack:

Cloudstack: Prerequisites: For step by step instructions see the guide.

Management Server, Database, and Storage System Requirements

The machines that will run the Management Server and MySQL database must meet the following requirements. The same machines can also be used to provide primary and secondary storage, such as via localdisk or NFS. The Management Server may be placed on a virtual machine.
  • Operating system:
    • Preferred: CentOS/RHEL 6.3+ or Ubuntu 12.04(.1)
  • 64-bit x86 CPU (more cores results in better performance)
  • 4 GB of memory
  • 250 GB of local disk (more results in better capability; 500 GB recommended)
  • At least 1 NIC
  • Statically allocated IP address
  • Fully qualified domain name as returned by the hostname command

Host/Hypervisor System Requirements

This host is where the cloud services run in the form of guest virtual machines. Each host is one machine that meets the following requirements:
  1. Must support HVM (Intel-VT or AMD-V enabled).
  2. 64-bit x86 CPU (more cores results in better performance)
  3. Hardware virtualization support required
  4. 4 GB of memory
  5. 36 GB of local disk
  6. At least 1 NIC

Prerequisites for building Apache CloudStack
There are a number of prerequisites needed to build CloudStack. This walk-through assumes compilation on a Linux system that uses RPMs or DEBs for package management.
You will need, at a minimum, the following to compile CloudStack:

  1. Maven (version 3)
  2. Java (OpenJDK 1.6 or Java 7/OpenJDK 1.7)
  3. Apache Web Services Common Utilities (ws-commons-util)
  4. MySQL
  5. MySQLdb (provides Python database API)
  6. Tomcat 6 (not 6.0.35)
  7. genisoimage
  8. rpmbuild or dpkg-dev

After you get all this work done, you need to extract the source. Extracting the CloudStack release is relatively simple.

$ tar -jxvf apache-cloudstack-4.1.0.src.tar.bz2

You can now move into the directory:

$ cd ./apache-cloudstack-4.1.0-src

Then you need to Build the DEB Packages.  Next you need to set up an APT repository and configure your machines to use the APT repository.   Once you have the machines using the APT repository you need to build RPMs as mentioned in the prerequisite. Once the RPMs are built you need generate the RPMs.

Generating RPMs is done using the package.sh script:

$./package.sh
That will run for a bit and then place the finished packages in dist/rpmbuild/RPMS/x86_64/.
Note: If you need support for the VMware, NetApp, F5, NetScaler, SRX, or any other non-Open Source Software (nonoss) plugins, you’ll need to download a few components on your own and follow a slightly different procedure to build from source.
From this point you only have 10 more steps to go.
  1. Make sure you have the required hardware ready. See Section 4.3, “Minimum System Requirements”
  2. Install the Management Server (choose single-node or multi-node). See Section 4.5, “Management Server Installation”
  3. Log in to the UI. See Chapter 5, User Interface
  4. Add a zone. Includes the first pod, cluster, and host. See Section 6.3, “Adding a Zone”
  5. Add more pods (optional). See Section 6.4, “Adding a Pod”
  6. Add more clusters (optional). See Section 6.5, “Adding a Cluster”
  7. Add more hosts (optional). See Section 6.6, “Adding a Host”
  8. Add more primary storage (optional). See Section 6.7, “Add Primary Storage”
  9. Add more secondary storage (optional). See Section 6.8, “Add Secondary Storage”

Simple, fast, and quick, right?   Not so much.  I have read and seen some more prepacked solutions that help speed this process up, but the process still isn’t a quick install, and you’re not up and running very fast.  I will give the CloudStack team huge credit; the guides are very clean and straightforward.  You just need to have a little patience and know more than the average user to get things going.

 

OpenStack:

For starters I find it pretty amusing that you are presented with 2 paths:

OpenStack Start

I will have to be honest, looking at the 2 options I was afraid of option 2 since it appears, at face value, to be a lot of work just by the content above.  However, OpenStack’s documentation is very clear and tries to make it much simpler.  With OpenStack, you have a handful of different install and build choices.  I started going down a couple different build paths, and came to the realization that it was more effort and time than I wanted to invest.   I was grateful to discover a much more hidden approach to build an All-in-One OpenStack that you can deploy on VMware Workstation/ESXi, or pretty much any other hypervisor deployment.  So I cheated a little by using the RackSpace alamo.iso.  So much easier than building it myself. So props to RackSpace for having that as an option.  I know there are other solutions similar to Rackspace iso, but it was easy to find and the documentation on step-by-step building made it the path of least resistance.

Suggested system requirements:

To install Private Cloud Software, you will need compute resources for the following 2 key roles within the infrastructure.

Chef server (optional – Opscode Hosted Chef may be used)

  1. 16GB RAM
  2. 60GB disk space
  3. 4 CPU cores

OpenStack controller node

  1. 16GB RAM
  2. 144GB disk space (more if Glance resides on the same server)
  3. 4 CPU cores

The process using the alamo.iso was pretty painless: follow the prompts, know your network settings, and away you go.

OpenStack Install

The nice part with this install is there was a lot of thought into all the screens to help make sure you know what is going on as the install builds and you configure all the pieces.
OpenStack install 2

I was pleasantly surprised, for as complex and how many moving parts OpenStack has, Rackspace’s ISO makes it very simple to get you up and running.  Really it’s as simple as following some user/password prompts, some network questions, and then you’re done.  Once the wizard is completed, you point the browser and cross your fingers your network settings are right, and away you go.

openstack login

Once logged in I went straight to seeing how robust the dashboard is and the speed in which I could deploy an instance.  It wasn’t bad at all. Fairly straight forward, and things provisioned without too much pain. I’m not going to lie- one of the first images I created and deployed on OpenStack was a ScienceLogic software AIO. Pretty sweet.  I see an OpenStack ScienceLogic software collector in the future.

Openstack Em7

 

Eucalyptus:

Eucalyptus FastStart Minimum Requirements:

  1. At least one machine with 100GB disk space
  2. A range of IP addresses to assign to Eucalyptus instances
  3. Burning an ISO file to a DVD
  4. Configuring a range of unused IP addresses on your network
  5. Provisioning a private subnet

FastStart is intended for smaller deployments. For larger deployments, requiring separation of components, download the standard open source software.

Euco Fast

One thing I need to give credit to the Eucalyptus team for is around clear and painless access to getting the pieces I needed to get the cloud going.  No forms to fill out, no crazy amount of documentation to read, or long install manual.

Eucalyptus consists of the following components:
  1. Cloud Controller (CLC): this component provides EC2 functionality
  2. Walrus: this component provides S3 functionality
  3. Cluster Controller (CC): this component provides management service for a cluster in your cloud
  4. Storage Controller (SC): this component provides EBS functionality
  5. Node Controller (NC): this component controls virtual machine instances
In the Frontend+NC configuration, the CLC, Walrus, CC, and SC are installed on one machine, called the Frontend.
The NC is installed on another machine, called the Node. In this configuration you can have one Frontend and one or more Nodes.
In the Cloud-in-a-box configuration, all components are installed on one machine. The simplest way to install Eucalyptus is to install Cloud-in-a-Box. It is also the least like an actual cloud, but it’s a great way to learn the basics about how Eucalyptus works. All components are installed in a single system, and most of the configuration is handled automatically.
One more observation; Eucalyptus fast start of cloud-in-a-box solution is seriously only 8 steps. One of which is downloading the iso. 8 wizard prompts, ips, passwords and users entry later.

Euco Login

It was really that fast and simple.  It was, however, a little eerie how much Eucalyptus looks like a fake AWS.  The layout and menu structure is remarkably close to AWS.  I would have thought, even though it is based on the same set of code and cloud design, apis, etc , you would have at least made the UI more your own.

You tell me?  Although I think the wording and look are too close for my taste, I completely understand the argument that it was on purpose, so if you use one, using the other is no problem.  This also allows you to have quicker ramp for a hybrid cloud model between the two.
Euco Dashboard
aws dashboard

Conclusion:

I know all clouds aren’t created equal and they all have their pros and cons, but from being able to download a package and getting it up and running with instances on it, in the shortest and least painful time there are some differences.  By far CloudStack and OpenStack give you more freedom on your kernel and packages of choice, which is very important to people.  However, for just plug and play speed and ease of use,  I would have to give the win to Eucalyptus. Which of the cloud frameworks win out or if they all co-exist will be interesting to see.  Either way I enjoyed the pros and cons of all 3 and can completely understand the passion and polar opinions on which one is really a better platform.

Tagged with: , , , , , , , , , ,

3 comments

3 Comments Add your own

  • 1. Dawn Bost, MD; Board Certified Internal Medicine Specialist, Graduate Student in Health Informatics  |  July 2nd, 2013 at 9:42 am

    Very informative post Jeremy. Of course, my interest in Cloud storage is because it is an appealing option for storage and exchange of health information. I understand that security and ptivacy continues to be a problem. What are your ideas on this issue?

    Reply

  • 2. Jeremy Sherwood, Cloud Strategist  |  July 2nd, 2013 at 9:45 am

    Dawn-I would be more than happy to setup a call to go over the security side of the cloud. You are absolutely correct, there are safe and unsafe clouds to place and operate data in. I look forward to our discussion. – Direct Email Sent

    Reply

  • 3. mary  |  July 15th, 2013 at 7:02 am

    I am not in the cloud yet as there are some reasons for it. I am still considering the technology itself and in fact thinking hard about it. The post is great as you describe the basic systems which is rather helpful, at least for me.

    Reply

Leave a Comment

Required

Required, hidden

 

Some HTML allowed:
<a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Trackback this post  |  Subscribe to the comments via RSS Feed


Our blog’s authors aren’t just experts in their field, they’re also key contributors to our world-class monitoring platform. If you’d like to see how these topics play out in a real-world setting, please register for a free, no pressure demo:

Request a demo

Search


type keywords | hit enter

Navigation

Recent Posts

Categories

Archives

Recent Comments

Subscribe