Continuous deployment environment with Docker, AWS EB and Codeship

In the last two episodes we have configured the Docker environment and automated deployment with Elastic Beanstalk, but still the process requires some manual actions and I personally would like to avoid any unnecessary interference in the process. So, today I will walk you through combining all the pieces together and automating the process fully. A continuous integration system will be placed between developer’s environment and final servers. I’ll present how to achieve all of that with Codeship. What make me choose this particular mechanism? The simplicity of setting up, number of additional tools ready to use without installation and finally the fact that it isn’t time consuming.

Continuous integration (CI) is the practice that requires integrating code into a shared repository, that needs to be build automatically. In order to detect problems early, each check-in should be verified by automated tests. A good CI tool includes build, tests and deploy and that means you won’t waste your time on any of above mentioned elements.

Connecting Codeship with github repository

The fastest way to connect the github repository with Codeship is to login with github or bitbucket account at https://codeship.com/sessions/new. Codeship will ask about permissions not only for personal information, but also repositories. The second will be useful in the next part.

After successful login, the splash screen with a list of configured projects will be displayed. It’s empty for now, so there is a big green button “Let’s do it! Create my first project!”. So, let’s do it.

The project wizard shows two available and most popular services to connect with – the github and bitbucket. Even if the account is registered with github, it can be connected with bitbucket too. After choosing the provider, a list of existing projects will be displayed. Choose one and go to the next step which is configuring the testing process.

Continuous Integration process

Unfortunately, Codeship doesn’t support the Docker integration yet. Codeship is currently preparing a Docker-based CI infrastructure which will be based on docker-compose tool and additional yaml configuration. The beta version has been already released and should be finished soon. There is a possibility to register to the beta version at http://beta.codeship.com/.

In the meanwhile, the integration specification has been published. The docker-compose up command is run as default (this part is ready in the application) and the only thing necessary to configure Codeship to run tests with docker is to specify what commands should be executed in which containers. A special file, codeship-steps.yml is dedicated to do that. It should look like this:

That means Codeship will run simultaneously two containers and execute phpunit and phpspec commands.

Tests’ configuration

The current implementation of Codeship allows to run tests inside predefined server based on linux. To configure tests procedure select ‘Project Settings’ from top right corner dropdown menu and choose ‘Test Settings’.

There will appear two textarea fields purposed for setting up commands like app/console doctrine:schema:update or app/console doctrine:fixtures:load, and Configure Test Pipelines dedicated for running actual tests – in our case phpunit and phpspec.

Moreover, on top of the site, there is a selectbox with available sample configurations for plenty programming languages – Ruby, Java, node.js, PHP, Go or Dart. For each of them there are prepared a couple of distributions with selected version of primary language e.g. PHP 5.4 or 5.5.

Those configured commands will be executed each time when the code will be pushed or committed to the repository. Codeship will build the project from scratch, install all dependencies from composer automatically and run specific tests. If any test fails, an administrator or development team will be notified by email and a red flag will appear next to the build on last builds list.

There is also a way to check why tests don’t pass or why build failed. After clicking the broken record, a full stack of executed commands will be presented. The last failed command will be expanded and show error message. Next time, if the build goes green the administrator will also receive an email that the project came back on track.

Continuous Deployment process

In the last part an AWS EB application with an environment has been configured. Now, the time to fully automate the deployment process has come.To complete that task, go to the project’s settings in top right corner at Codeship and choose Deployment tab from left menu.

Top tabs are presenting different purposes of deployment which depend on the name of the branch. The options used by me most often are develop, qa, acceptance and production. A selected branch is assigned to each of them. For instance, when a pull request will merge to the develop branch then the code will be automatically deployed to the develop environment on AWS EB. You can execute similar process with qa (qa branch), acceptance (acceptance branch) and production (master branch).
There are plenty of other services to connect with listed in “Add another deployment to your pipeline” section. In my opinion, only one important hosting provider is missing on that list – Digital Ocean. But the rest of most popular services have already been integrated. Choose Amazon ElasticBeanstalk from the list and configure with Amazon’s credentials.

Amazon’s credentials

Codeship ask for some information and credentials required by Amazon API. TheApplication Name, Application Environment and Region can be found in AWS EB panel at list of available services.

S3 Bucket name can be filled by any random string. If the bucket will not be found then the AWS will create it. In this bucket all packed deploys will be stored. Moreover, the contents of each bucket can be viewed in S3 at https://console.aws.amazon.com/s3/home.

AWS Access Key ID and AWS Secret Access Key require more work to get. Login to AWS and click the account’s name in the top right corner, then select Security Credentials (http://docs.aws.amazon.com/IAM/latest/UserGuide/IAMBestPractices.html.

After closing the popup window a list of six access methods is displayed. The Access Keys (Access Key ID and Secret Access Key) will be most useful with Codeship. Expand this option and click “Create New Access Key”. The Access and Secret Keys can be downloaded as csv file or showed by clicking a little link “Show Access Key”. Copy and paste them into Codeship.

Now, the deploy configuration is ready to be completed. Click ‘Create’ button below the form. From now on, when the code will be merged into a master branch, tests will be run and if passed, the branch will be deployed automatically, by the Codeship to the assigned environment on AWS EB. Your automated deployment is now up and running – congratulations!

This series was purposed to share with you how to carry out configuring the automated deployment process from A to Z. And the goal might seem to be achieved. But there are still some improvements to introduce and of course I will try to share other tips & tricks about Docker, Amazon WebServices and Codeship. In the meanwhile, I encourage you to share your own experience with automated deployment, thoughts or questions in the comments.