Ansible

How to Create EC2 Duplicate Instance with Ansible

Tomasz Szandała — Wed, 28 Oct 2020 19:32:42 +0000

Many companies like mine use AWS infrastructure as a service (IaaS) heavily. Sometimes we want to perform a potentially risky operation on an EC2 instance. As long as we do not work with immutable infrastructure it is imperative to be prepared for instant revert.

One of the solutions is to use a script that will perform instance duplication, but in modern environments, where unification is an essence it would be wiser to use more common known software instead of making up a custom script.

Here comes the Ansible!

Ansible is a simple automation software. It handles configuration management, application deployment, cloud provisioning, ad-hoc task execution, network automation, and multi-node orchestration. It is marketed as a tool for making complex changes like zero-downtime rolling patching, therefore we have used it for this straightforward snapshotting task.

Requirements

For this example we will only need an Ansible, in my case it was version 2.9 - in subsequent releases there is a major change with introducing collections so let's stick with this one for simplicity.

Due to working with AWS we require a minimal set of permissions, which include permissions to create:

AWS snapshots
Register images (AMI)
Start and stop EC2

Environment preparation

Since I am forced to work on Windows I have utilized Vagrant instances. Please find below a Vagrantfile content.

We are launching a virtual machine, with Centos 7 and Ansible installed.

For security reasons Ansible, by default, has disabled reading configuration from mounted location, therefore we have to implcity indicate path /vagrant/ansible.cfg.

Listing 1. Vagrantfile for our research

Vagrant.configure("2") do |config|
  config.vm.box = "geerlingguy/centos7"
  config.vm.hostname = "awx"
  config.vm.provider "virtualbox" do |vb|
    vb.name = "AWX"
    vb.memory = "2048"
    vb.cpus = 3
  end
  config.vm.provision "shell", inline: "yum install -y git python3-pip"
  config.vm.provision "shell", inline: "pip3 install ansible==2.9.10"
  config.vm.provision "shell", inline: "echo 'export ANSIBLE_CONFIG=/vagrant/ansible.cfg' >> /home/vagrant/.bashrc"
end

First tasks

In the first lines of the Ansible we specify few meta values. Most of them, like name, hosts and tasks are mandatory. Others provide auxiliary functions.

Listing 2. duplicate_ec2.yml playbook first lines

---
- name: yolo
  hosts: localhost
  connection: local
  gather_facts: false
  become: false
  vars:
    instance_id: i-deadbeef007

tasks:

Go to Full Article

Digging Through the DevOps Arsenal: Introducing Ansible

Petros Koutoupis — Wed, 07 Aug 2019 18:15:00 +0000

by Petros Koutoupis

If you need to deploy hundreds of server or client nodes in parallel, maybe on-premises or in the cloud, and you need to configure each and every single one of them, what do you do? How do you do it? Where do you even begin? Many configuration management frameworks exist to address most, if not all, of these questions and concerns. Ansible is one such framework.

You may have heard of Ansible already, but for those who haven't or don't know what it is, Ansible is a configuration management and provisioning tool. (I'll get to exactly what that means shortly.) It's very similar to other tools, such as Puppet, Chef and Salt.

Why use Ansible? Well, because it's simple to master. I don't mean that the others are not simple, but Ansible makes it easy for individuals to pick up quickly. That's because Ansible uses YAML as its base to provision, configure and deploy. And because of this approach, tasks are executed in a specific order. During execution, if you trip over a syntax error, it will fail once you hit it, potentially making it easier to debug.

Now, what's YAML? YAML (or YAML Ain't Markup Language) is a human-readable data-serialization language mostly used to capture configuration files. You know how JSON is easier to implement and use over XML? Well, YAML takes a more simplistic approach than JSON. Here's an example of a typical YAML structure containing a list:


data:
    - var1:
        a: 1
        b: 2
    - var2:
        a: 1
        b: 2
        c: 3

Now, let's swing back to Ansible. Ansible is an open-source automation platform freely available for Linux, macOS and BSD. Again, it's very simple to set up and use, without compromising any power. Ansible is designed to aid you in configuration management, application deployment and the automation of assorted tasks. It works great in the realm of IT orchestration, in which you need to run specific tasks in sequence and create a chain of events that must happen on multiple and different servers or devices.

Here's a good example: say you have a group of web servers behind a load balancer. You need to upgrade those web servers, but you also need to ensure that all but one server remains online for the upgrade process. Ansible can handle such a complex task.

Ansible uses SSH to manage remote systems across the network, and those systems are required to have a local installation of not only SSH but also Python. That means you don't have to install and configure a client-server environment for Ansible.

Go to Full Article

Weekend Reading: Ansible

Shawn Powers — Sat, 05 Jan 2019 12:30:00 +0000

by Shawn Powers

I've written about and trained folks on various DevOps tools through the years, and although they're awesome, it's obvious that most of them are designed from the mind of a developer. There's nothing wrong with that, because approaching configuration management programmatically is the whole point. Still, it wasn't until I started playing with Ansible that I felt like it was something a sysadmin quickly would appreciate.

Part of that appreciation comes from the way Ansible communicates with its client computers—namely, via SSH. As sysadmins, you're all very familiar with connecting to computers via SSH, so right from the word "go", you have a better understanding of Ansible than the other alternatives.

With that in mind, I've written a few articles exploring how to take advantage of Ansible. It's a great system, but when I was first exposed to it, it wasn't clear how to start. It's not that the learning curve is steep. In fact, if anything, the problem was that I didn't really have that much to learn before starting to use Ansible, and that made it confusing. For example, if you don't have to install an agent program (Ansible doesn't have any software installed on the client computers), how do you start?

Ansible, Part I: the Automation Framework That Thinks Like a Sysadmin

How to get started with Ansible. Shawn tells us the reason Ansible was so difficult for him at first was because it's so flexible with how to configure the server/client relationship, he didn't know what he was supposed to do. The truth is that Ansible doesn't really care how you set up the SSH system; it will utilize whatever configuration you have. This article will get you set up.

Ansible, Part II: Making Things Happen

Finally, an automation framework that thinks like a sysadmin. Ansible, you're hired.

Ansible is supposed to make your job easier, so the first thing you need to learn is how to do familiar tasks. For most sysadmins, that means some simple command-line work. Ansible has a few quirks when it comes to command-line utilities, but it's worth learning the nuances, because it makes for a powerful system.

Ansible, Part III: Playbooks

Playbooks make Ansible even more powerful than before.

Go to Full Article

Ansible, Part IV: Putting It All Together

Shawn Powers — Fri, 02 Mar 2018 15:31:56 +0000

by Shawn Powers

Roles are the most complicated and yet simplest aspect of Ansible to learn.

I've mentioned before that Ansible's ad-hoc mode often is overlooked as just a way to learn how to use Ansible. I couldn't disagree with that mentality any more fervently than I already do. Ad-hoc mode is actually what I tend to use most often on a day-to-day basis. That said, using playbooks and roles are very powerful ways to utilize Ansible's abilities. In fact, when most people think of Ansible, they tend to think of the roles feature, because it's the way most Ansible code is shared. So first, it's important to understand the relationship between ad-hoc mode, playbooks and roles.

Ad-hoc Mode

This is a bit of a review, but it's easy to forget once you start creating playbooks. Ad-hoc mode is simply a one-liner that uses an Ansible module to accomplish a given task on a set of computers. Something like:


ansible cadlab -b -m yum -a "name=vim state=latest"

will install vim on every computer in the cadlab group. The -b signals to elevate privilege ("become" root), the -m means to use the yum module, and the -a says what actions to take. In this case, it's installing the latest version of vim.

Usually when I use ad-hoc mode to install packages, I'll follow up with something like this:


ansible cadlab -b -m service -a "name=httpd state=started
 ↪enabled=yes"

That one-liner will make sure that the httpd service is running and set to start on boot automatically (the latter is what "enabled" means). Like I said at the beginning, I most often use Ansible's ad-hoc mode on a day-to-day basis. When a new rollout or upgrade needs to happen though, that's when it makes sense to create a playbook, which is a text file that contains a bunch of Ansible commands.

Playbook Mode

I described playbooks in my last article. They are YAML- (Yet Another Markup Language) formatted text files that contain a list of things for Ansible to accomplish. For example, to install Apache on a lab full of computers, you'd create a file something like this:

Go to Full Article

Ansible, Part III: Playbooks

Shawn Powers — Mon, 19 Feb 2018 16:50:14 +0000

by Shawn Powers

Playbooks make Ansible even more powerful than before.

To be quite honest, if Ansible had nothing but its ad-hoc mode, it still would be a powerful and useful tool for automating large numbers of computers. In fact, if it weren't for a few features, I might consider sticking with ad-hoc mode and adding a bunch of those ad-hoc commands to a Bash script and be done with learning. Those few additional features, however, make the continued effort well worth it.

Tame the Beast with YAML

Ansible goes out of its way to use an easy-to-read configuration file for making "playbooks", which are files full of separate Ansible "tasks". A task is basically an ad-hoc command written out in a configuration file that makes it more organized and easy to expand. The configuration files use YAML, which stands for "Yet Another Markup Language". It's an easy-to-read markup language, but it does rely on whitespace, which isn't terribly common with most config files. A simple playbook looks something like this:


---

- hosts: webservers
  become: yes
  tasks:
    - name: this installs a package
      apt: name=apache2 update_cache=yes state=latest

    - name: this restarts the apache service
      service: name=apache2 enabled=yes state=restarted

The contents should be fairly easy to identify. It's basically two ad-hoc commands broken up into a YAML configuration file. There are a few important things to notice. First, every filename ends with .yaml, and every YAML file must begin with three hyphen characters. Also, as mentioned above, whitespace matters. Finally, when a hyphen should precede a section and when it should just be spaced appropriately often is confusing. Basically every new section needs to start with a - symbol, but it's often hard to tell what should be its own section. Nevertheless, it starts to feel natural as you create more and more playbooks.

The above playbook would be executed by typing:


ansible-playbook filename.yaml

And that is the equivalent of these two commands:


ansible webservers -b -m apt -a "name=apache2
 ↪update_cache=yes state=latest"
ansible webservers -b -m service -a "name=apache2
 ↪enabled=yes state=restarted"

Handling Your Handlers

But a bit of organization is really only the beginning of why playbooks are so powerful. First off, there's the idea of "Handlers", which are tasks that are executed only when "notified" that a task has made a change. How does that work exactly? Let's rewrite the above YAML file to make the second task a handler:

Go to Full Article

Ansible: Making Things Happen

Shawn Powers — Tue, 30 Jan 2018 16:07:28 +0000

by Shawn Powers

Finally, an automation framework that thinks like a sysadmin. Ansible, you're hired.

In my last article, I described how to configure your server and clients so you could connect to each client from the server. Ansible is a push-based automation tool, so the connection is initiated from your "server", which is usually just a workstation or a server you ssh in to from your workstation. In this article, I explain how modules work and how you can use Ansible in ad-hoc mode from the command line.

Command Module

This is the safest module to execute remote commands on the client machine. As with most Ansible modules, it requires Python to be installed on the client, but that's it. When Ansible executes commands using the Command Module, it does not process those commands through the user's shell. This means some variables like $HOME are not available. It also means stream functions (redirects, pipes) don't work. If you don't need to redirect output or to reference the user's home directory as a shell variable, the Command Module is what you want to use. To invoke the Command Module in ad-hoc mode, do something like this:


ansible host_or_groupname -m command -a "whoami"

Your output should show SUCCESS for each host referenced and then return the user name that the user used to log in. You'll notice that the user is not root, unless that's the user you used to connect to the client computer.

If you want to see the elevated user, you'll add another argument to the ansible command. You can add -b in order to "become" the elevated user (or the sudo user). So, if you were to run the same command as above with a "-b" flag:


ansible host_or_groupname -b -m command -a "whoami"

you should see a similar result, but the whoami results should say root instead of the user you used to connect. That flag is important to use, especially if you try to run remote commands that require root access!

Go to Full Article

Ansible: the Automation Framework That Thinks Like a Sysadmin

Shawn Powers — Fri, 05 Jan 2018 13:02:34 +0000

by Shawn Powers

With that in mind, I'm planning to write a few articles exploring how to take advantage of Ansible. It's a great system, but when I was first exposed to it, it wasn't clear how to start. It's not that the learning curve is steep. In fact, if anything, the problem was that I didn't really have that much to learn before starting to use Ansible, and that made it confusing. For example, if you don't have to install an agent program (Ansible doesn't have any software installed on the client computers), how do you start?

Getting to the Starting Line

The reason Ansible was so difficult for me at first is because it's so flexible with how to configure the server/client relationship, I didn't know what I was supposed to do. The truth is that Ansible doesn't really care how you set up the SSH system; it will utilize whatever configuration you have. There are just a couple things to consider:

Ansible needs to connect to the client computer via SSH.
Once connected, Ansible needs to elevate privilege so it can configure the system, install packages and so on.

Unfortunately, those two considerations really open a can of worms. Connecting to a remote computer and elevating privilege is a scary thing to allow. For some reason, it feels less vulnerable when you simply install an agent on the remote computer and let Chef or Puppet handle privilege escalation. It's not that Ansible is any less secure, but rather, it puts the security decisions in your hands.

Next I'm going to list a bunch of potential configurations, along with the pros and cons of each. This isn't an exhaustive list, but it should get you thinking along the right lines for what will be ideal in your environment. I also should note that I'm not going to mention systems like Vagrant, because although Vagrant is wonderful for building a quick infrastructure for testing and developing, it's so very different from a bunch of servers that the considerations are too dissimilar really to compare.

Some SSH Scenarios

1) SSHing into remote computer as root with password in Ansible config.

Go to Full Article