Posts Tagged ‘version control’

Kiln Repository Support

May 25, 2010

Today we are pleased to announce support for projects hosted within Kiln On-Demand repositories.

Kiln Logo

For those of you who are unfamiliar with Kiln, it is based upon the popular open source Distributed Version Control System (DVCS) Mercurial.

We have created a short screen cast that demonstrates this new feature using the Java Petstore (built using Maven) as an example project.

You can also find additional information about our support for Kiln in our new, comprehensive, on-line user guide

Bootstrap your Agile Java Startup Infrastructure in < 30 minutes

March 25, 2010

So, you have a great idea for the next web-based killer application. You have assembled a small but geographically dispersed team with the requisite amount of Agile Java development-fu to get the job done. What you need now is some hosted infrastructure so you can all get to work effectively.

In less than 30 minutes.

Step 1 – Get some IDE

Get Eclipse Galileo (JEE version). Hopefully your pipe is fat enough (~190 MB download). Fire it up and use update sites to obtain m2eclipse and subclipse (we’re assuming SVN for a repository, but you could use Git). Install Tomcat and configure as a server.

Step 2 – Get some Scaffolding

Within Eclipse go to File>New>Other, to open the Select a wizard dialog. From the scrollbox, select Maven Project and then click Next. Move through the wizard to select an archetype of an appropriate type (e.g. Appfuse). Click Finish. Validate you can build and deploy your app.

Step 3 – Get some Version Control

While you are waiting for your build to complete, pick a hosted version control provider. There are a number who provide low-cost or even free hosted Subversion for private projects, typically with a trial period. Here is a list. Once signed-up with a clean repository, use subclipse to share the scaffolding app created in (2).

Step 4 – Get some Project Tracking and Wiki

The majority of providers listed in (3) also offer something in the area of hosted task/issue tracking apps that often have sufficient wiki capabilities. Alternatively you can try those who specialise
in this area such as FogBugz or Agilo. We use Agilo on Mike. And you might want to get Basecamp. We use that too.

Step 5 – Get some CI

You could roll your own on Amazon EC2, but that isn’t happening in 30 minutes or 30 hours, probably! Hmmm…Oh I almost forgot – you could use our excellent hosted service ;).

Step 6 – Get some BRM

Whats a BRM? Its Binary Repository Manager. If you’re using Maven, I’d recommend you use one. The only hosted one I’m aware of is provided by Artifactory Online.

Right. We are good to go. What are you waiting for?

New Mike Release with GitHub integration and Functional Testing support

March 18, 2010

We are very pleased to be able announce a new release of our hosted CI platform. The release includes some major enhancements to allow our users to do much more with their builds while at the same time ensuring that they happen within a highly secure, controlled environment.

Here is summary of the new features:

CI support for projects hosted within public or private GitHub repositories that use Ant, Maven or Eclipse.

Each build is now executed in an entirely separated environment, which allows our users to:

  • Start, stop and deploy to embedded containers as part of their build process (e.g. Jetty or Glassfish).
  • Start, stop and deploy to provided (by Mike) instances of Apache Tomcat (v5 and 6) as part of their build process.
  • Run integration and functional tests, using frameworks such as HtmlUnit or JWebUnit against applications deployed to said containers or instances.
  • Fork additional processes as part of a build, such as bash scripts and other non-Java based tooling.
  • Use the full range of available Ant tasks and Maven plug-ins as part of a build.

Security hardening:

  • All network communication within Mike is encrypted using either key or token-based authentication.
  • All outbound network access from a build is transparently proxied and validated.
  • IP protection: all builds run in a clean, repeatable sand-boxed environment and all source code processed by a build is accessed using token-based authentication over encrypted channels. See the security section of our website for further details

Also:

  • Support for Maven Ant Tasks (v2.1.0) – Ant-based projects can now use Maven dependency management for their binary artefacts.
  • Gravatar integration for developer email addresses.

Be sure to check out our no-obligation, 14-day trial. You simply need to provide us with your company/project name and an email address. You can also use our sample applications to rapidly get started and experience our platform features. Our prices start from only $10 per month.

Cloud Considerations: 4 tips for getting started on Amazon EC2

December 8, 2009

You need an environment and have decided that Amazon is the way to go. There are some things that should be considered before building your OS on, or uploading to the cloud.

1. Getting started

There are a couple of ways to get started with Amazon Cloud Hosting. You can build your own AMI ( Amazon Machine Image ) or you can modify an existing public AMI.
I am not going to give a tutorial on how to go about either of these options, that is well documented by Amazon.

Firstly you need to select your operating system, Amazon has a decent selection of supported OSs, you have the main Linux flavours that you’d want to see:

  • Red Hat
  • Fedora
  • Ubuntu
  • Debian
  • Gentoo
  • openSUSE

along with:

  • OpenSolaris

and one token Windows version:

  • Windows Server 2003

After selecting your OS see if there is a public image available of the version you would like to help you make the decision of whether to build your own.

If you want to go with a public AMI you need to decide if you can trust the provider, you can boot up an instance and check out the machine, see what is installed, ensure there is no proprietary software or at least you know the software that is installed. This should be thought about as well as the considerations to be applied to building your own AMI.

The easiest way to boot your first AMI is through the AWS Management Console. It is a fairly intuitive front end to the commands you can run when you install the EC2 tools, once you learn your way around the console, it will  give you a better insight into the tasks you need to get used to when using the command line tools.

If you want to build your own AMI then you will be in control of exactly what is installed, but you should consider the next point.

2. Is it Repeatable?

Now you have decided how to get started, are you able to rebuild your environment to the same specifications in a short period of time? If the worst happens and for whatever reason your server gets shutdown, lost, deleted, can you re-provision it?

Whether you are building your own or modifying an existing AMI there are ways to make the process repeatable. The simplest way is to keep a record of the process, a list of what is installed, the location of the binaries, where they are installed and any configuration steps.

If you have read the Amazon docs, and I suggest you do, then it should be fairly clear that all of the processes involved in either building your own or starting with an existing AMI are scriptable ( at least on the Linux side of things ). Using Amazon’s EC2 tools you can control AMI instances through scripts quite nicely, boot instances, attach EBS volumes, upload files and execute commands, everything necessary for automated environment creation. This in theory could be the beginning of your own Amazon DSL, but that is another blog post…

When creating your AMI through a loopback file, once again you have almost infinite control which should make the scripting easier. Once you have finished with your masterpiece you need to be sure to bundle it and upload it to Amazon.

3. Persistence

Now you know how you want to create your AMI and make sure the process is repeatable you need to think about the persistence of your data. By default none of your information will be saved when your AMI instance is shut down, unless you re-bundle that instance to a new bucket or with a new manifest name everything you did since you booted will be lost.

Obviously this isn’t workable, Amazon have a solution, and they call it Elastic Block Storage (EBS). Again there are Amazon docs for how to create and use EBS volumes. EBS volumes are attached to your instances as devices which you can mount as you would any other device, they are easy to use and it is simple to take backups by using the snapshot feature.

The real consideration is how to structure the installation of your software so you get the benefits of data persistence. This is obviously going to vary massively on the type of system you are running and the software you are installing, but components like web applications and databases and such should be configured to use EBS for information storage. If you plan carefully you can have EBS volumes with discrete services installed that can be cloned and attached to new instances at will, allowing for a truly scaleable architecture.

4. Versioning

How can you keep a track of differences in your environments from one upload to the next? There are a couple of really manageable ways, if you are scripting your setup then you can use version control and a release or tagging process of the script to keep a track of environmental changes.

If you are only at the stage of keeping a track of changes through component lists then it is possible to bundle your AMIs to a bucket appended with a version, or indeed specify a manifest file versioned in the same way, meaning you will keep your historic changes as an entire environment. If you are version controlling your scripts I would suggest uploading to unique bucket names appended with versions to begin with anyway, it just means you can immediately revert to the old environment if anything goes wrong.

After using Amazon EC2 for a while it will become clear how environments can be used as just another architectural component, they can be provisioned on demand, have services attached and started and within minutes you have a custom “box” available for use in the cloud, which can be torn down and brought back to its initial state very quickly.

One more thing, whatever you do, don’t block your SSH port 🙂

Could your next development environment be in the cloud?

November 17, 2009

Your new project has been given the green light. You need to get your team up and running quickly.  Could a cloud based development environment be the answer? This blog discusses some of the options  and issues for moving to a  hosted development environment.

Cloud City

One of the first tasks for any Agile project team is to establish a robust, reliable set of development tools and associated infrastructure. Generally on any new development endeavour the following needs to be put in place:

  • Locate or create a source code repository and import your skeleton project.
  • Locate or create a continuous integration server to build your source code, run your tests and notify the team via email of any failures.
  • Locate or create a task/user story tracking server and/or issue management tool and add your project to it.
  • Locate or create your test environments for developer smoke, integration, and functional testing.

Now you might be lucky and have much of this infrastructure in place, in which case adding some extra users, a new project and associated set of build jobs might be relatively straightforward. However, if it isn’t you will need to   allocate a decent chunk of time to setting things up yourself. This may involve procurement of hardware, installation of an appropriate OS, installation of the relevant applications, providing secure access to project team members and so on. This gets more complex if the team is distributed and the infrastructure must be accessible beyond a LAN.

Often these development tools are open source, which means that while the cost of acquisition for the software may be zero, the ongoing maintenance and support will probably require specialist knowledge.  Any time your team spends doing this is time that could be spent progressing the project.

With either option, server space and administration time is required and there are obviously costs associated with this, and the costs may be disproportionately large to small and medium-sized organisations.

In the same way that SaaS offerings for email and collaboration suites (such as Google Apps) have sought over the past few years sought to turn these services into low-cost, click-and-go commodities, there are now equivalent hosted solution options available for Agile development infrastructure.

Hosted version control solutions have been available for a while and the market has expanded rapidly over the past few years. Collabnet (the people behind Subversion) and CVSdude are probably the best known. Both companies have expanded their offerings, and not only provide version control but are bundled with or integrate to a host of other tools.  These now compete with newer entrants like Beanstalk, Assembla and Unfuddle (which also does Git hosting). GitHub itself has seen huge growth over the past year, with over 400 new users and 1,000 projects being added each day.

Coming from the other direction are suppliers of Development collaboration and productivity tools, Atlassian are probably the biggest player here and their hosted JIRA studio suite based around the very popular Jira issue tracking tool was launched in May 2008.  Fogbugz is another alternative, based around an integrated Project Management, Wiki, Issue Management and Helpdesk set of tools.

Each vendor has a different focus with JIRA studio and Collabnet’s Team-Forge perhaps the most fully featured for development teams. Both offer a very comprehensive stack and are moving towards the idea of an Application Lifecycle Management (ALM) suite. It will be interesting to see how these platforms develop, and how the traditional Enterprise tool vendors (IBM Rational, Microsoft and Borland) respond.

If  a platform sounds too restrictive and you want to “pick and mix” your own set of tools you can.  There are also some great tools which focus on a specific area – Basecamp for Project Management and Lighthouse for Issue Management are a couple of well-known examples worth looking at. Most of these tools have open APIs that enable you to integrate easily with others, so getting together an integrated  set of tooling is easily achievable.

My advice here is to be clear about what you want from these tools. Some are very feature rich and developer centric while others do a great job of providing a clean and simple process and interface. So which tools suits you will depend on your project, your team, and your organisation. What is clear though is that there is growth in this sector, increased competition and greater integration between the providers. All of this can only be good for those who are happy to outsource their development environments – increased choice and competition against a backdrop of  decreasing hosting costs.

Moving on from  managing your project to testing it, the use of externalized environments that allow teams to deploy a release of a web-based application and run functional tests against it, is trickier. Depending on the nature of the application and its associated runtime dependencies this may require the creation of a bespoke environment. However, recent developments in cloud computing should soon make this much easier. Google’s App Engine for example allows you to run Java (and Python) applications on their infrastructure. So if the AppEngine is your production target, creating a test environment that is a clone of production should be a relatively straight-forward activity. More recently in August this year, SpringSource launched their Cloud Foundry which allows you to rapidly deploy a test (and also production) environment for your Java web application with a few mouse clicks and Microsoft have also weighed in with Azure. Both Google and Microsoft are promising tighter integration with the IDE, and I’ll be watching these platforms closely.

One area which is less mature is hosted continuous integration. There are currently only a small number of pioneering providers in this space, which may surprise some, as the practice of continuous integration is at the heart of the Agile development process. The SaaS multi-tenant application model does not fit easily with the requirements for continuous and often complex software builds. It is computing resource intensive activity, especially for programming languages such as Java, and this will inevitably impact the cost of such a service to the end-user. Mike CI is one of these pioneers and there is a good analysis of the others here.

Now I’m not saying trusting your code to a 3rd party is a simple decision to make. There are often legal, security and organisational hurdles to consider. It isn’t be for everyone and for many large corporates it might be a step too far. But for many people the cost, convenience and management overhead of maintaining it all yourself does not stack up. Your team is in place to write great software. For your next project, I’d recommend that you seriously consider using these low-cost, on-demand hosted services.