Category Archives: Deployment

Create a LEMP stack in AWS with EC2 (Linux, Nginx, MySQL and PHP 7.2)

Hopefully, this quick run-through will help you get a LEMP server up and running quite quickly in AWS. If you have any questions, feel free to ask in the comments.

Important! Please, choose the required data center from the top right hand corner. Remember to do this for any configuration changes or new instances when logging into the AWS console.

Follow the navigation to start launching a new EC2 instance:
AWS > Compute > EC2 > Instances > Launch Instance

Select the Amazon Linux Candidate we want to use:
Amazon Linux 2 LTS Candidate 2 AMI (HVM), SSD Volume Type – ami-921423eb

And the following flavour with our desired specifications:
General Purpose: t2.xlarge

Click on the ‘next’ buttons in the bottom right hand corner until you get to ‘Configure Security Group’. Leave all other configuration options as default.

The SSH rule should be added already, but you will need to add the 443 and 80 port open from anywhere. Name this security group something relevant with a similar description, so it can be reused for other instances.

Finally, click ‘Review and launch’.

When prompted, create a new key pair and download this to your machine. This will be in the form of a .pem file which you should add to your .ssh directory. For now, this is the quickest way to configure a role to allow you to access the server:

Where aws.pem is the name of your key pair. The usage of CI means no one else should need access to the server, and less human intervention will prevent any configuration from changing. Cloud setups should be kept extremely automated.

If you wish for this to change, consider creating an IAM role, such as, ‘DevOps’, and create and add the necessary users, which will all have different key pairs, AWS logins and API keys. Use different user accounts per employee, to make it easy to revoke them where and when necessary.

Once your key pair is downloaded, you will be able to finally launch your instance. Head back to the ‘Instances’ page and wait for your instance to complete setup. The instance should be in a ‘Running’ state and the status check should show as ‘Initializing’.

When the status checks finalizes and shows ‘2/2 checks passed’, you will be able to use the public IVP4 address of your instance. Use this to login via SSH using the ec2-user user:

Where xxx.xxx.xxx.xxx is your public IPV4 address of the instance.

Please note, if you are unable to ping or access this IP address, your security group may be misconfigured.

Once logged into the box, run the following commands to configure:

Find the following line:

Add this line below it:

Come out of the file, and create the sites-available directory:

Now, create a new virtual host entry:

Paste in the following for a virtual host setup:

For non-dev environments, the SSL certificate will need to be properly generated via a third party (such as CloudFlare or Let’s Encrypt). This will then need to be uploaded to the server(s) and the location of the key changed in the virtual host entry. After applying any changes, Nginx will need to be restarted for these changes to take effect.

Please be sure to change the server_name directive, and set up the relevant DNS (pointing to the load balancer or Elastic IP). After that, create the virtual host symlink:

Finally, we will need to create a server key for the SSL binding. For the key generation, use an empty passphrase, and leave the challenge password blank:

And change the matching keys to the following values:

Feel free to comment out the default virtual host after you have confirmed everything is working as expected, which can be done in:

Once complete, Nginx and PHP-FPM should be running. You can install your website at /var/www/my-site to get started.

At a later stage, when the website is fully deployed, an image should be taken to allow us to recreate boxes at the click of a button, as well as being able to set up auto-scaling in the future. This will drastically reduce the time needed to re-create an instance, and could be used in an emergency situation if an instance is unresponsive.

Please note, it is not recommended to take an image when the instance is running, as Amazon is not able to guarantee the integrity of the file system on the created image. When creating images, please bear in mind the instance will be shut down. If taking images on a working production environment, the instance should be taken out of the load balancer before working on it.

If you have any problems or need to troubleshoot, check your Nginx configuration by using:

And using the Nginx error log to check for errors when accessing the web server:

Once a configuration file has changed, you will need to restart nginx with:

The perfect deployment system

If you’ve worked in an agency that have their processes sorted, you’ll have come across some sort of automated or well managed deployment system. It’s definitely a step up from manually uploading files via FTP (or even editing them directory from the server via FTP). Whilst this makes my toes curl, we all had to start somewhere.

From manually uploading files, the next step is to start working with version control systems (VCSs). SVN and Git are two very well known VCSs which are used to save all your changes to a repository. Some use this as their deployment by simply cloning the repository on the server and pulling (generally through a staging/production branch) directly via the command line. This takes away the annoyance of having to upload all your files by only deploying changed files, but it’s still not the right way to go about it.

There are third party solutions out there such as DeployHQ and DeployBot (formally known as Dploy), both of which are great tools and enable you to set up deployments via a hook on a specific branch or by manually deploying (either by adding some text in the commit or pushing the ‘deploy’ button on their website). Personally, I have used both of these systems and found DeployBot has much more to offer as a deployment service however, there’s still a few bottle necks (one of them described in this post).

All of these systems use SFTP/FTP to upload files to the server. Although there are other types of deployments you can set up yourself (such as SSH deployments) the FTP versions are the only ones that actually upload the files for you.

Something which I think these systems have missed out on is the use of a command called rsync. For more information about this command, take a look at the manual page by typing in man rsync from your command line. Directly from the manual, the part we want to take advantage of is:

The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network connection, using an efficient checksum-search algorithm described in the technical report that accompanies this package.

How great is that?

This means if we’re deploying a file we already have on the server (chances are, we do, since we’re generally updating files as well as uploading new ones), we’re only deploying the difference in the file, not the whole file itself. If we had a 90MB file and changed a few words, rsync will be able to upload a much smaller amount of bytes compared to the whole 90MB file, which FTP will not be able to do. Since FTP is pretty slow anyway (other copy commands such as SCP will be much faster than FTP), rsync seems like a great solution for deploying web applications.

Let’s take DeployBot’s setup as it currently is and develop from there to make the perfect deployment system. If you’re not familiar with DeployBot, take a look at their guide on deploying a Laravel app to Digital Ocean and read from ‘Configuring DeployBot’.

If we take the technology from rsync and the setup DeployBot already has, we’re heading in the right direction. One of the biggest downfalls I found using DeployBot is the fact it has to re-upload the entire project via FTP. On a large site, that can take quite a while. It does create revision directories and uses a symlink to change the public directory to the new revision (when it’s been deployed, and when it’s ready).

Rather than having to re-upload the entire project, why not just copy the previous revision directory (which will be quite fast on a good server – much faster than uploading everything again) and then use the rsync command to deploy the differences for that particular revision?

If we were to deploy something to our server from our local version, we would use rsync like this:

With all that in mind, if DeployBot were to copy the previous release, upload changes to that directory via the rsync command and continue with their usage of system links and containers, we’d find it to be perfect deployment system.

Quick rollback via system link in DeployBot

I’ve been using DeployBot (formally known as Dploy) to deploy complex websites which require compiling Sass, minifying and uploading assets to Amazon S3 using Gulp.js, installation of Composer dependancies and a number of other configuration and cache settings to be changed.

DeployBot does quite a good job of managing this infrastructure. Although the build tools are still in beta, you can’t fault them for their support.

As per this blog post they mention:

You could always rollback by triggering a deployment with a previous commit selected. However, when things go wrong you want to react fast. We hope this button will go a long way to help you save time and not make a mistake in a rush.

The problem is, if you have quite a large site (that requires compiling before it even starts uploading the files), rollbacks are painfully slow. There’s nothing worse than a live site in a state that the client (or even yourself) isn’t happy with. Whatever the reason, you’d want rollbacks to be instantaneous.

DeployBot don’t currently support ‘quick’ rollbacks via symlink changes alone (since these can be quite complicated). At the moment, rollbacks are done just as any other deployment – code gets compiled, uploaded to a new release, all the files are uploaded to the release and if it’s successful, the release will switch the $BASE/current symlink. The other issue is if you have code that needs to be executed before/after deployments such as database migrations. This is where things can get a bit tricky, and you may want to stick with DeployBot’s current rollback solution.

I’ll leave that to you work out for your own applications but on a basic level, I’ve come up with a quick shell script that makes this ‘quick’ rollback possible:

Be sure to change the $base variable to your correct path.

Please check my GitHub account for the latest version of this code.

This works by first checking to see if the current deployment is a rollback. If it is, we check through all the current releases that are stored on the server and comparing the .revision file with the current commit. If it matches, the symlink is changed.

I’ve put this code in a new server (selecting ‘Shell’ as the type) upon deployment. Adding this code alone will quickly check to see if the rolled back release is available, switch it and ends with an exit code (0). Your application would have reverted back instantly.

Panic over. Go make a cuppa.

Whilst you’re making your tea, since these deployments are running in parallel, the Atomic FTP server is currently deploying the rollback from scratch.

After speaking to DeployBot, one of the concerns they had with this script is you would have two deployments running in parallel, so you couldn’t be sure the shell deployment ran first. Generally speaking, your Atomic FTP deployment will take a good while to deploy, and you’d want it to carry on deploying anyway – this is more as a fallback until the actual deployment is finished.

However, if you would prefer to have your deployments running in series, simply set up a second environment. The first being your shell deployment with the quick rollback and the second being your Atomic FTP deployment. Head over to the ‘overview’ of your deployments which should show your environments. Click the settings button on your shell environment and you should be in the ‘Servers & settings’ section. Scroll down to ‘Triggers’ and you’ll see something like this:

screenshot

Simply check the ‘Deploy another environment’ box and choose your Atomic FTP environment. Be sure to name your shell environment just the environment you wish to choose. In this case, I’ve named my shell environment ‘Staging’ and my Atomic FTP environment ‘Staging (Atomic FTP)’. This is so the manual commit trigger can be picked up by adding [deploy: staging] to your commit message.

And there we have it, instant rollbacks. Neato.