Author Archive

Bootstrap your Agile Java Startup Infrastructure in < 30 minutes

March 25, 2010

So, you have a great idea for the next web-based killer application. You have assembled a small but geographically dispersed team with the requisite amount of Agile Java development-fu to get the job done. What you need now is some hosted infrastructure so you can all get to work effectively.

In less than 30 minutes.

Step 1 – Get some IDE

Get Eclipse Galileo (JEE version). Hopefully your pipe is fat enough (~190 MB download). Fire it up and use update sites to obtain m2eclipse and subclipse (we’re assuming SVN for a repository, but you could use Git). Install Tomcat and configure as a server.

Step 2 – Get some Scaffolding

Within Eclipse go to File>New>Other, to open the Select a wizard dialog. From the scrollbox, select Maven Project and then click Next. Move through the wizard to select an archetype of an appropriate type (e.g. Appfuse). Click Finish. Validate you can build and deploy your app.

Step 3 – Get some Version Control

While you are waiting for your build to complete, pick a hosted version control provider. There are a number who provide low-cost or even free hosted Subversion for private projects, typically with a trial period. Here is a list. Once signed-up with a clean repository, use subclipse to share the scaffolding app created in (2).

Step 4 – Get some Project Tracking and Wiki

The majority of providers listed in (3) also offer something in the area of hosted task/issue tracking apps that often have sufficient wiki capabilities. Alternatively you can try those who specialise
in this area such as FogBugz or Agilo. We use Agilo on Mike. And you might want to get Basecamp. We use that too.

Step 5 – Get some CI

You could roll your own on Amazon EC2, but that isn’t happening in 30 minutes or 30 hours, probably! Hmmm…Oh I almost forgot – you could use our excellent hosted service ;).

Step 6 – Get some BRM

Whats a BRM? Its Binary Repository Manager. If you’re using Maven, I’d recommend you use one. The only hosted one I’m aware of is provided by Artifactory Online.

Right. We are good to go. What are you waiting for?

Cloud Considerations: 4 tips for getting started on Amazon EC2

December 8, 2009

You need an environment and have decided that Amazon is the way to go. There are some things that should be considered before building your OS on, or uploading to the cloud.

1. Getting started

There are a couple of ways to get started with Amazon Cloud Hosting. You can build your own AMI ( Amazon Machine Image ) or you can modify an existing public AMI.
I am not going to give a tutorial on how to go about either of these options, that is well documented by Amazon.

Firstly you need to select your operating system, Amazon has a decent selection of supported OSs, you have the main Linux flavours that you’d want to see:

  • Red Hat
  • Fedora
  • Ubuntu
  • Debian
  • Gentoo
  • openSUSE

along with:

  • OpenSolaris

and one token Windows version:

  • Windows Server 2003

After selecting your OS see if there is a public image available of the version you would like to help you make the decision of whether to build your own.

If you want to go with a public AMI you need to decide if you can trust the provider, you can boot up an instance and check out the machine, see what is installed, ensure there is no proprietary software or at least you know the software that is installed. This should be thought about as well as the considerations to be applied to building your own AMI.

The easiest way to boot your first AMI is through the AWS Management Console. It is a fairly intuitive front end to the commands you can run when you install the EC2 tools, once you learn your way around the console, it will  give you a better insight into the tasks you need to get used to when using the command line tools.

If you want to build your own AMI then you will be in control of exactly what is installed, but you should consider the next point.

2. Is it Repeatable?

Now you have decided how to get started, are you able to rebuild your environment to the same specifications in a short period of time? If the worst happens and for whatever reason your server gets shutdown, lost, deleted, can you re-provision it?

Whether you are building your own or modifying an existing AMI there are ways to make the process repeatable. The simplest way is to keep a record of the process, a list of what is installed, the location of the binaries, where they are installed and any configuration steps.

If you have read the Amazon docs, and I suggest you do, then it should be fairly clear that all of the processes involved in either building your own or starting with an existing AMI are scriptable ( at least on the Linux side of things ). Using Amazon’s EC2 tools you can control AMI instances through scripts quite nicely, boot instances, attach EBS volumes, upload files and execute commands, everything necessary for automated environment creation. This in theory could be the beginning of your own Amazon DSL, but that is another blog post…

When creating your AMI through a loopback file, once again you have almost infinite control which should make the scripting easier. Once you have finished with your masterpiece you need to be sure to bundle it and upload it to Amazon.

3. Persistence

Now you know how you want to create your AMI and make sure the process is repeatable you need to think about the persistence of your data. By default none of your information will be saved when your AMI instance is shut down, unless you re-bundle that instance to a new bucket or with a new manifest name everything you did since you booted will be lost.

Obviously this isn’t workable, Amazon have a solution, and they call it Elastic Block Storage (EBS). Again there are Amazon docs for how to create and use EBS volumes. EBS volumes are attached to your instances as devices which you can mount as you would any other device, they are easy to use and it is simple to take backups by using the snapshot feature.

The real consideration is how to structure the installation of your software so you get the benefits of data persistence. This is obviously going to vary massively on the type of system you are running and the software you are installing, but components like web applications and databases and such should be configured to use EBS for information storage. If you plan carefully you can have EBS volumes with discrete services installed that can be cloned and attached to new instances at will, allowing for a truly scaleable architecture.

4. Versioning

How can you keep a track of differences in your environments from one upload to the next? There are a couple of really manageable ways, if you are scripting your setup then you can use version control and a release or tagging process of the script to keep a track of environmental changes.

If you are only at the stage of keeping a track of changes through component lists then it is possible to bundle your AMIs to a bucket appended with a version, or indeed specify a manifest file versioned in the same way, meaning you will keep your historic changes as an entire environment. If you are version controlling your scripts I would suggest uploading to unique bucket names appended with versions to begin with anyway, it just means you can immediately revert to the old environment if anything goes wrong.

After using Amazon EC2 for a while it will become clear how environments can be used as just another architectural component, they can be provisioned on demand, have services attached and started and within minutes you have a custom “box” available for use in the cloud, which can be torn down and brought back to its initial state very quickly.

One more thing, whatever you do, don’t block your SSH port 🙂