DevOps

Three Ways To Improve Your Programming Skills

Steve King — Tue, 08 Feb 2022 17:00:00 +0000

The ability to write code is a huge differentiator for every job role in an enterprise Linux environment. As an Operations and DevOps manager, I was constantly challenged to improve my team’s programming skills, and the team genuinely wanted to be more proficient.

But how?

Structured training is a standard answer: take a course! Our company, like many, invested enormously in learning resources. I’d sit with an engineer one-on-one and we’d ponder the online portal together, puzzling out the most appropriate Python learning path.

There are two issues, however. Problem one: classroom material is almost immediately forgotten, if not directly applied. Problem two: I’d lose visibility of progress for days, weeks, or even months. I’d find out too late the material was inappropriate or too advanced.

I want to share three supplemental approaches I found effective in helping engineers get comfortable with programming while becoming immediately productive. Each is suited to a different baseline skill-set.

New hires: teach the framework, first

No, or very little, programming experience? Don’t ask the engineer to write code; instead, write documentation. This sounds contradictory, I realize. But this is the key: the engineer is to treat what he or she writes exactly like code. Your team’s development practices are to be followed scrupulously, regardless of content. And if your shop isn’t already handling technical documentation like code, please start!

Really -- it's not basic language constructs and procedures that are a struggle for the inexperienced -- it’s everything else. Immediately more important than writing code are the concepts of a repository and mechanics of Git; continuous integration, continuous delivery (CI/CD) pipelines, installing and configuring an integrated development environment (IDE), etc.

In my team, I challenged very new programmers to write “runbooks” – a set of written procedures used for incident response by Level I support and sysadmins. And I expected an iterative cycle; the runbooks would be properly merged in our repository, and be continually updated and revised, reinforcing this practical learning. You “get” Git by using it over and over.

Asking new programmers to write documentation helps in these three ways:

The employee develops a hands-on understanding of version control and other tools fundamental to best practices.
The output is immediately productive, and appreciated by peers.
It’s an easy transition to the next logical step, which IS programming: replacing the manual actions with automation.

Go to Full Article

Experts Attempt to Explain DevOps--and Almost Succeed

Bryan Lunduke — Wed, 07 Aug 2019 20:00:00 +0000

by Bryan Lunduke

What is DevOps? How does it relate to other ideas and methodologies within software development? Linux Journal Deputy Editor and longtime software developer, Bryan Lunduke isn't entirely sure, so he asks some experts to help him better understand the DevOps phenomenon.

The word DevOps confuses me.

I'm not even sure confuses me quite does justice to the pain I experience—right in the center of my brain—every time the word is uttered.

It's not that I dislike DevOps; it's that I genuinely don't understand what in tarnation it actually is. Let me demonstrate. What follows is the definition of DevOps on Wikipedia as of a few moments ago:

DevOps is a set of software development practices that combine software development (Dev) and information technology operations (Ops) to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.

I'm pretty sure I got three aneurysms just by copying and pasting that sentence, and I still have no clue what DevOps really is. Perhaps I should back up and give a little context on where I'm coming from.

My professional career began in the 1990s when I got my first job as a Software Test Engineer (the people that find bugs in software, hopefully before the software ships, and tell the programmers about them). During the years that followed, my title, and responsibilities, gradually evolved as I worked my way through as many software-industry job titles as I could:

Automation Engineer: people that automate testing software.
Software Development Engineer in Test: people that make tools for the testers to use.
Software Development Engineer: aka "Coder", aka "Programmer".
Dev Lead: "Hey, you're a good programmer! You should also manage a few other programmers but still code just as much as you did before, but, don't worry, we won't give you much of a raise! It'll be great!"
Dev Manager: like a Dev Lead, with less programming, more managing.
Director of Engineering: the manager of the managers of the programmers.
Vice President of Technology/Engineering: aka "The big boss nerd man who gets to make decisions and gets in trouble first when deadlines are missed."

During my various times with fancy-pants titles, I managed teams that included:

Go to Full Article

Digging Through the DevOps Arsenal: Introducing Ansible

Petros Koutoupis — Wed, 07 Aug 2019 18:15:00 +0000

by Petros Koutoupis

If you need to deploy hundreds of server or client nodes in parallel, maybe on-premises or in the cloud, and you need to configure each and every single one of them, what do you do? How do you do it? Where do you even begin? Many configuration management frameworks exist to address most, if not all, of these questions and concerns. Ansible is one such framework.

You may have heard of Ansible already, but for those who haven't or don't know what it is, Ansible is a configuration management and provisioning tool. (I'll get to exactly what that means shortly.) It's very similar to other tools, such as Puppet, Chef and Salt.

Why use Ansible? Well, because it's simple to master. I don't mean that the others are not simple, but Ansible makes it easy for individuals to pick up quickly. That's because Ansible uses YAML as its base to provision, configure and deploy. And because of this approach, tasks are executed in a specific order. During execution, if you trip over a syntax error, it will fail once you hit it, potentially making it easier to debug.

Now, what's YAML? YAML (or YAML Ain't Markup Language) is a human-readable data-serialization language mostly used to capture configuration files. You know how JSON is easier to implement and use over XML? Well, YAML takes a more simplistic approach than JSON. Here's an example of a typical YAML structure containing a list:


data:
    - var1:
        a: 1
        b: 2
    - var2:
        a: 1
        b: 2
        c: 3

Now, let's swing back to Ansible. Ansible is an open-source automation platform freely available for Linux, macOS and BSD. Again, it's very simple to set up and use, without compromising any power. Ansible is designed to aid you in configuration management, application deployment and the automation of assorted tasks. It works great in the realm of IT orchestration, in which you need to run specific tasks in sequence and create a chain of events that must happen on multiple and different servers or devices.

Here's a good example: say you have a group of web servers behind a load balancer. You need to upgrade those web servers, but you also need to ensure that all but one server remains online for the upgrade process. Ansible can handle such a complex task.

Ansible uses SSH to manage remote systems across the network, and those systems are required to have a local installation of not only SSH but also Python. That means you don't have to install and configure a client-server environment for Ansible.

Go to Full Article

Writing GitHub Web Hooks with Bash

Andy Carlson — Wed, 07 Aug 2019 16:30:00 +0000

by Andy Carlson

Bring your GitHub repository to the next level of functionality.

For the past year since Microsoft has acquired GitHub, I've been hosting my Git repositories on a private server. Although I relished the opportunity and challenge of setting it all up, and the end product works well for my needs, doing this was not without its sacrifices. GitHub offers a clean interface for configuring many Git features that otherwise would require more time and effort than simply clicking a button. One of the features made easier to implement by GitHub that I was most fond of was web hooks. A web hook is executed when a specific event occurs within the GitHub application. Upon execution, data is sent via an HTTP POST to a specified URL.

This article walks through how to set up a custom web hook, including configuring a web server, processing the POST data from GitHub and creating a few basic web hooks using Bash.

Preparing Apache

For the purpose of this project, let's use the Apache web server to host the web hook scripts. The module that Apache uses to run server-side shell scripts is mod_cgi, which is available on major Linux distributions.

Once the module is enabled, it's time to configure the directory permissions and virtual host within Apache. Use the /opt/hooks directory to host the web hooks, and give ownership of this directory to the user that runs Apache. To determine the user running an Apache instance, run the following command (provided Apache is currently running):


ps -e -o %U%c| grep 'apache2\|httpd'

These commands will return a two-column output containing the name of the user running Apache and the name of the Apache binary (typically either httpd or apache2). Grant directory permission with the following chown command (where USER is the name of the user shown in the previous ps command):


chown -R USER /opt/hooks

Within this directory, two sub-directories will be created: html and cgi-bin. The html folder will be used as a web root for the virtual host, and cgi-bin will contain all shell scripts for the virtual host. Be aware that as new sub-directories and files are created under /opt/hooks, you may need to rerun the above chown to verify proper access to files and sub-directories.

Here's the configuration for the virtual host within Apache:

Go to Full Article

My Favorite Infrastructure

Kyle Rankin — Wed, 07 Aug 2019 15:45:00 +0000

by Kyle Rankin

Take a tour through the best infrastructure I ever built with stops in architecture, disaster recovery, configuration management, orchestration and security.

Working at a startup has many pros and cons, but one of the main benefits over a traditional established company is that a startup often gives you an opportunity to build a completely new infrastructure from the ground up. When you work on a new project at an established company, you typically have to account for legacy systems and design choices that were made for you, often before you even got to the company. But at a startup, you often are presented with a truly blank slate: no pre-existing infrastructure and no existing design choices to factor in.

Brand-new, from-scratch infrastructure is a particularly appealing prospect if you are at a systems architect level. One of the distinctions between a senior-level systems administrator and architect level is that you have been operating at a senior level long enough that you have managed a number of different high-level projects personally and have seen which approaches work and which approaches don't. When you are at this level, it's very exciting to be able to build a brand-new infrastructure from scratch according to all of the lessons you've learned from past efforts without having to support any legacy infrastructure.

During the past decade, I've worked at a few different startups where I was asked to develop new infrastructure completely from scratch but with high security, uptime and compliance requirements, so there was no pressure to cut corners for speed like you might normally face at a startup. I've not only gotten to realize the joy of designing new infrastructure, I've also been able to do it multiple times. Each time, I've been able to bring along all of the past designs that worked, leaving behind the bits that didn't, and updating all the tools to take advantage of new features. This series of infrastructure designs culminated in what I realize looking back on it is my favorite infrastructure—the gold standard on which I will judge all future attempts.

In this article, I dig into some of the details of my favorite infrastructure. I describe some of the constraints around the design and explore how each part of the infrastructure fits together, why I made the design decisions I did, and how it all worked. I'm not saying that what worked for me will work for you, but hopefully you can take some inspiration from my approach and adapt it for your needs.

Go to Full Article

The DevOps Issue

Bryan Lunduke — Thu, 01 Aug 2019 15:00:00 +0000

by Bryan Lunduke

Every few years a new term is coined within the computer industry—big data, machine learning, agile development, Internet of Things, just to name a few. You'd be forgiven for not knowing them all.

Some of these are new ideas. Some are refinements on existing ideas. Others still are simply notions we've all had for a long time, but now we have a new word to describe said notions.

Which brings me to a topic we cover in depth in this issue of Linux Journal: DevOps.

Not sure what DevOps is? Need it explained to you? It's okay, I was in the same boat. Start off by reading "Experts Attempt to Explain DevOps—and Almost Succeed" to get a high-level explanation of what this whole DevOps brouhaha is all about.

Once you've got the concept of DevOps firmly implanted in your brain, it's time to dive in and look at how specific parts of DevOps can be implemented, starting with "Continuous Integration/Continuous Development with FOSS Tools" by Quentin Hartman, Director of Infrastructure and DevOps at Finalze.

Next, turn to Linux Journal's very own Editor at Large (and senior performance software engineer at Cray), Petros Koutoupis, for a look at how to install and utilize Ansible to deploy and configure large numbers of Linux servers all at once. It's a nifty tool to have in your toolbelt, especially when looking to do things "The DevOps Way".

Okay, you've got the idea of DevOps, and you know some of the tools you can utilize with it as you build out a big, expansive online service. But what does a truly excellent system really look like? What components does it consist of? How does one go about selecting said components?

Luckily, we've got Kyle Rankin's aptly titled "My Favorite Infrastructure" to answer those questions. Linux Journal's illustrious Tech Editor (and Chief Security Officer at Purism) gives a tour of, what he considers to be, the best infrastructure he ever built. Including details on the architecture, configuration management, security and disaster recovery.

Oh, but we're not done! Ever want to build an OpenStack implementation on top of Fedora, openSUSE or Debian? John S. Tonello, the Global Technical Marketing Manager at SUSE, walks through exactly that with the help of free software tools like Kolla, Docker, qemu and pip. It's a veritable smorgasbord of Linux server-y goodness.

Go to Full Article

Continuous Integration/Continuous Development with FOSS Tools

Quentin Hartman — Wed, 31 Jul 2019 11:30:00 +0000

by Quentin Hartman

Up your DevOps game! Get the fundamentals of CI/CD with FOSS tools now!

One of the hottest topics within the DevOps space is Continuous Integration and Continuous Deployment (CI/CD). This attention has drawn lots of investment dollars, and a vast array of proprietary Software As A Service (SaaS) tools have been created in the CI/CD space, which traditionally has been dominated by free open-source software (FOSS) tools. Is FOSS still the right choice with the low cost of many of these SaaS options?

It depends. In many cases, the cost of self-hosting these FOSS tools will be greater than the cost to use a non-FOSS SaaS option. However, even in today's cloud-centric and SaaS-saturated world, you may have good reasons to self-host FOSS. Whatever those reasons may be, just don't forget that "Free" isn't free when it comes to keeping a service running reliably 24/7/365. If you're looking at FOSS as a means to save money, make sure you account for those costs.

Even with those costs accounted for, FOSS still delivers a lot of value, especially to small and medium-sized organizations that are taking their first steps into DevOps and CI/CD. Starting with a commercialized FOSS product is a great middle ground. It gives a smooth growth path into the more advanced proprietary features, allowing you to pay for those only once you need them. Often called Open Core, this approach isn't universally loved, but when applied well, it has allowed for a lot of value to be created for everyone involved.

An Embarrassment of Riches

The DevOps concept exploded in the past several years. The term quickly saturated the mainstream technology industry. With this increased mindshare comes a corresponding increase in the number of tools available to accomplish DevOps-related tasks. That's a blessing and a curse as a DevOps practitioner. Thanks to the endless buffet of options, you're sure to find something that meets your needs, but to a newcomer, the multitude of choices is overwhelming. Combine that with the vast scope of tasks that fall under the DevOps umbrella and the competing claims of "best" from all sides, and you have a recipe for paralysis. A good place for finding tools and filtering by a variety of criteria is DevOpsBookmarks.com. The content is all open source, and the maintainers are diligent about merging contributions, but it hasn't seen a lot of updates lately. Despite that, it makes a great jumping off point. If you find something noteworthy that should be included, a pull request would be appreciated!

Go to Full Article

How to Build an Enterprise Kubernetes Strategy

LJ Staff — Mon, 22 Jul 2019 04:27:42 +0000

How to Build an Enterprise Kubernetes Strategy LJ Staff Sun, 07/21/2019 - 23:27

In today’s emerging cloud-native environments, Kubernetes is everywhere.

Organizations love Kubernetes because it helps significantly increase the agility and efficiency of their software development teams, enabling them to reduce the time and perils associated with putting new software into production. Information technology operations teams love Kubernetes because it helps boost productivity, reduce costs and risks, and moves organizations closer to achieving their hybrid cloud goals.

Simply put, Kubernetes makes it easier to manage software complexity. As enterprise applications become more complex, development and operations (DevOps) teams need a tool that can orchestrate that complexity. They need a way to launch all the services dependent on these applications, making sure the applications and services are healthy and can connect to one Another.

Containers have dramatically risen in popularity because they provide a consistent way to package application components and their dependencies into a single object that can run in any environment. By packaging code and its dependencies into containers, a development team can use standardized units of code as consistent building blocks. The container will run the same way in any environment and can start and terminate quickly, allowing applications to scale to any size.

In fact, development teams are using containers to package entire applications and move them to the cloud without the need to make any code changes. Additionally, containers can make it easier to build workflows for applications that run between on-premises and cloud environments, enabling the smooth operation of almost any hybrid environment.

You may download this special Kubernetes ebook here.

Rancher Labs has written an ebook about this subject and they’re sharing it with Linux Journal readers. Topics include:

The Dangers of Too Many Good Things
Understanding Your Organization’s Current Kubernetes Adoption 4
Where Will You Be Running Kubernetes in Five Years?
Who Should Own the Kubernetes Strategy?
Centralized vs Decentralized Kubernetes Management
Containerization and Kubernetes Will Disrupt Some of Your Other Plans
Our Organization is Heavily Investing in Cloud Computing
We Are Investing in Hyper-Converged Infrastructure as Part of a Data Center Upgrade
We Are Trying to Modernize Our Existing Applications to Improve Security and Stability
We Need to Cut Our Infrastructure/Cloud Spending
Preparing Your Teams for Broader Kubernetes Adoption
Evaluating Container Management Platforms and Delivering Kubernetes- as-a-Service
Kubernetes Distribution, Cluster Provisioning and Lifecycle Management
Multi-Cluster Kubernetes Management
User Management and Delegated Administration
Policy Management
User Experience and the Entire Cloud Native Stack
Kubernetes Security and Audit
Open Source, SaaS, and Support
A Few Final Thoughts
Appendix A — Case Study: How Life Sciences Leader Illumina Implemented an
Enterprise Kubernetes Strategy
A Complex Puzzle with Many Parts
Connecting All the Pieces with Rancher and Kubernetes
How Rancher and Kubernetes Can Work for Any Organization

You may download this special Kubernetes ebook here.

Webinar: Operationalizing DevSecOps

Carlie Fairchild — Sun, 16 Jun 2019 13:42:13 +0000

by Carlie Fairchild

In this webinar, Twistlock's James Jones and Linux Journal's Katherine Druckman discuss hardening your DevOps environments and processes. Topics covered:

The keys to DevSecOps success
Tangible benefits of DevSecOps
Steps and tools involved with building, shipping, and running containers
DevSecOps creates a feedback loop
Seven steps to containers
And more

https://zoom.us/webinar/register/WN_h6Z3aGxtQzSdHIa2kFv_VA

Go to Full Article

CI/CD and the New Generation of Software Delivery: an Interview with Harness

Petros Koutoupis — Fri, 21 Dec 2018 13:15:15 +0000

by Petros Koutoupis

Continuous integration and continuous delivery (CI/CD) is all the rage in the modern world of software development. But actually what is this pipeline process? It's a method or set of principles for which development teams implement and deliver code more frequently and reliably.

Continuous integration embodies a coding philosophy and set of practices propelling teams to implement small and frequent code changes into version control repositories, while the continuous delivery picks up where the CI ends and automates the application's delivery

Many platforms, such as Jenkins and CircleCI, exist to help companies and teams streamline the development and integration of their software stacks, but not much exists in the way of easing and automating the process of delivery. And with what does exist, the solutions tend to fall short with features and functionality, or they are overly complicated to configure in the first place.

This is where Harness comes into the picture. Harness produces the industry's very first Continuous Delivery-as-a-Service platform. Using machine learning, it simplifies and automates the entire CD process. Steve Burton, VP of marketing at Harness, recently took the time to share more details with me.

Petros Koutoupis: Please introduce yourself to our readers.

Steve Burton: While officially the VP of marketing, I am a DevOps Evangelist over at Harness. What this means is that I do a little bit of everything. While most of my career has been in product management and marketing, I stepped out of the university with a bachelor's degree in computer science and an initial career in Java development (ca. 2004 at Sapient), working on large-scale enterprise J2EE implementations. Prior to Harness, I did geek stuff at AppDynamics, Moogsoft and Glassdoor. And when not knee-deep in take, I enjoy spending my time watching F1 and researching cars on the web.

PK: What is Harness?

SB: We provide Continuous Delivery as-a-Service. It's the CD bit of the CI/CD equation that helps customers automate how their software is deployed and delivered to end users in production.

We basically allow customers to move fast without breaking things, so they can increase developer velocity without the risk of downtime or failure.

PK: What problem or problems does Harness solve?

SB: Developers are under tremendous pressure to deliver applications to production, fast and with zero error. It's a constant pain, one that I personally dealt with as a former Java developer. Our founders had also seen this challenge firsthand, and that's why they started Harness.

Go to Full Article