AWS, Drupal and Caching: pt.1 Intro and Plan
Oct. 15, 2014So, here we are at another series of blog posts by your bearded badger storyteller. The reason this one is going to be another of those long series is just because it breaks up into separate pieces nicely, and is altogether a fairly complex topic if you haven't played with it all before.
But, first things first, why am I doing this? Well, there are many reasons, one is that I am sick of all the excuses youhear when on shared hosting such as 'adding caching will cause us performance issues'. Another, more sensible, reasonis just that whilst I know of a lot of the AWS elements, until you have really broken it apart and played, you can only know so much. So I want to see what it takes to build a server to host my site, what I can do to it, and how much of my time it will take to maintain such a behemoth.
Let's Plan
Before doing anything with AWS, you kind of need to have a feel for what you want, and understand the parts to make it up. Usually the most satisfying approach, is to create a nice diagram such as the beautiful ones you can find on the AWS documentation pages (http://media.amazonwebservices.com/architecturecenter/AWS_ac_ra_web_01.pdf). There's also lot of reading on cloud computing, and how it allows us to modularize our servers, but ultimately I wanted a single web node running Debian that I could modify as much as needed. I also wanted this to have a fixed IP and not be over expensive, yet giving me a good base to build upon (due to inexperience in devops field). Oh, of course, we want this to be based in the EU as most of my users are likely to be in the UK. Although of course, if this changes, it's nothing a good CDN cant help with. This said, a diagram wouldn't help me all that much, so sadly I decided to save time and not draw one, instead I just listed the tools I want to sit on my web node.
This final point of using a single web node came about once I realised that whilst AWS comes with some cool technology like RDS (Relational Database Service), these things aren'tnecessarily cheap. Another standard of web architecture is to seperate caching layers, and introduce a load balancer, which would also add cost, and until I can absorb this in some way, was not worth it yet. Therefore, I chose to use the Turnkey lampstack (I tried Bitnami first, but will go into that in a later chapter)as seemed a nice base (http://www.turnkeylinux.org/lampstack).
Now we have a base OS decided upon, we need to think about what is going to go on there. Turnkey gave us the base lampstack, but there are several things I am used to on a server, and so will have to make sure these are added to the server as one of my first tasks:
- bash
- Individual user account(s)
- DB user(s)for each site
- sudo
- drush
- memcache
- apc
- varnish
- slow query logs (always handy to receive)
Alongside the above tools, a backup scheduler is required. Now, there are several online systems you can pay for, or even use free trials for. But tbh, this isn't necessary at all so long as you put in a little time up-front. This is because AWS comes with a nice CLI interface, part of which allows you to trigger off backups. If only there was some system tool to run AWS CLI functions via some chronological peri... okay, yes, we plug cron in to use this. Very little code, very powerful backup system FTW.
So, is this it? Not quite, one of the requirements was to have a static IP for my server. There are many reasons for this, the short-term reason is that my DNS is hosted elsewhere, so I don't want to have to repoint it everytime AWS restarts my server (which can happen due to server issues, or Amazon updates). Other reasons include scaling up the server and needing to know which varnish servers to purge, aswell as point a load balancer at. So AWS comes with the concept of EIPs (Elastic IPs) which are free, so long as they are in use, which is exactly what I need.
The final step in most cases is cost, how much do we think this will come to. TBH that is part of why I am running this test. All the docs imply this will come to something around the £200 p/a mark, but will no more once the free trial period of AWS is over.
Oh yeah, did I mention that, if you are new to AWS, a lot of the basic parts are free for a set period of time. It's not a lot, but it meant I did not get charged whilst I was configuring the server. Bonus!
Final point which I imagine several people I know will whine about, saying things like 'why didn't you use chef/puppet you FOOL'. Now, whilst I am not a chef expert, I know the parts, and have it running on a local vagrant box nicely. In the long term, if my server needs multiple boxes due to how popular things get (heh) I will up it to use chef. The main reason I did not use chef in this though was that it hides a lot of the config and settings (by design), and as this was a learning process for me to understand servers better, it seemed counter productive to use such an automation tool.