Monitoring

Bare-Bones Monitoring with Monit and RRDtool

Andy Carlson — Thu, 21 Mar 2019 12:00:00 +0000

How to provide robust monitoring to low-end systems.

When running a critical system, it's necessary to know what resources the system is consuming, to be alerted when resource utilization reaches a specific level and to trend long-term performance. Zabbix and Nagios are two large-scale solutions that monitor, alert and trend system performance, and they each provide a rich user interface. Due to the requirements of those solutions, however, dedicated hardware/VM resources typically are required to host the monitoring solution. For smaller server implementations, options exist for providing basic monitoring, alerting and trending functionality. This article shows how to accomplish basic and custom monitoring and alerting using Monit. It also covers how to monitor long-term trending of system performance with RRDtool.

Initial Monit Configuration

On many popular Linux distros, you can install Monit from the associated software repository. Once installed, you can handle all the configuration with the monitrc configuration file. That file generally is located within the /etc directory structure, but the exact location varies based on your distribution.

The config file has two sections: Global and Services. The Global section allows for custom configuration of the Monit application. The Monit service contains a web-based front end that is fully configurable through the config file. Although the section is commented out by default, you can uncomment items selectively for granular customization. The web configuration block looks like this:


set httpd port 2812 and
    use address localhost
    allow localhost
    allow admin:monit

The first line sets the port number where you can access Monit via web browser. The second line sets the hostname (the HTTP Host header) that's used to access Monit. The third line sets the host from which the Monit application can be accessed. Note that you also can do this using a local firewall access restriction if a firewall is currently in place. The fourth line allows the configuration of a user name/password pair for use when accessing Monit. There's also a section that allows SSL options for encrypted connections to Monit. Although enabling SSL is recommended when passing authentication data, you also could reverse-proxy Monit through an existing web server, such as nginx or Apache, provided SSL is already configured on the web server. For more information on reverse-proxying Monit through Apache, see the Resources section at the end of this article.

The next items you need to enable deal with configuring email alerts. To set up the email server through which email will be relayed to the recipient, add or enable the following line:

Go to Full Article

Taking System Monitoring to the Next Level: an Interview with Scalyr CEO Steve Newman

Petros Koutoupis — Fri, 22 Feb 2019 13:00:00 +0000

by Petros Koutoupis

As computing ecosystems become more complex, monitoring and analyzing those often disconnected moving parts becomes increasingly challenging.

Today's data center has evolved from a single supplier producing and selling all-in-one offerings, such as the days when EMC, NetApp, HP or even Sun owned your data center and you chose a vendor and stuck with it. Those same vendors provided you with the required tools to monitor, analyze and troubleshoot their entire stack.

Shifting focus to the present, the landscape now appears to be quite different. Instead, you will find environments of mixed offerings provided by an assortment of vendors, both large and small. Proprietary machines work side by side with off-the-shelf commodity devices hosting software-defined software. Half of your applications may be hosted in virtual machines over a hypervisor or just spun up in a container. How does a modern data-center administrator or DevOps professional manage such an environment?

An assortment of platforms and frameworks exist that provide such capabilities, but they're not all one and the same. In some cases, those same tools will need to be coupled with others to produce something useful (for example, ELK: Elasticsearch + Logstash + Kibana). Unfortunately, this arrangement just adds to the complication and frustration when attempting to diagnose or discover problems in your computing environment.

Putting an end to this level of complexity, one company stands out among the rest: Scalyr. Scalyr develops and offers a complete suite of server monitoring, log management, visualization and analysis tools, which integrate with cloud services. I recently had the pleasure of chatting with Scalyr CEO Steve Newman.

His is not a household name, like Steve Jobs or Bill Gates, but you will be familiar with his work and contributions to cloud-enabled technologies. Although this is likely to change with Scalyr, Steve is best known for his work with Writely, a technology that later was acquired by Google and relabeled as Google Docs. In our conversation, Steve and I took the opportunity to discuss Scalyr, its solution and the problem it solves.

Steve Newman, Scalyr CEO

Petros Koutoupis: Tell me a bit about yourself. Who is Steve Newman?

Steve Newman: I am an engineer by both training and background and have spent most of my career in the startup environment. This is because I enjoy building things. I was at Google for a number of years following an acquisition, and while the experience itself was great, the startup bug in me drove me to Scalyr.

PK: So, now you founded a company called Scalyr. Please tell us, what is Scalyr?

Go to Full Article

Redefining the Landscape of System Monitoring: an Interview with Pulseway's Founder

Petros Koutoupis — Fri, 25 Jan 2019 13:15:15 +0000

by Petros Koutoupis

Pulseway provides a product of the same name that's built to enable IT personnel and give them the ability to monitor, manage and automate their systems and the tasks or applications that they host. And, the best part is that they can do all of these things anywhere and everywhere, from their pockets. In fact, I wrote about Pulseway once before, so check out that article for an introduction.

Pulseway is the Swiss Army knife of IT management, all accessible from your fingertips. You don't need to be glued to physical computer or connected to your employer's network. You are able to manage everything from either a web browser or mobile device—all you need is internet access.

I recently sat down with the founder and CEO of Pulseway, Marius Mihale, to ask him not only about exciting new things going on with the company, but also to find out where the company is heading.

Petros Koutoupis: Please, tell us a bit about yourself.

Marius Mihale: I am the Founder and CEO of Pulseway. I created both the software and solution about eight years ago. I initially designed the product with the goal of making the lives of IT administrators easier. It all started when I was attempting to shut down a server remotely but could not find a mobile application to aid me in this. This is how Pulseway was born. And while users can also access the same administration functions via a web browser and through our website, our core application is the mobile app: you can monitor Windows, Linux and Mac OS alongside various applications from your mobile device and take the necessary actions as they are needed.

PK: What has the demand been for such a solution?

MM: In 2011, we released a trial version of our product and almost immediately received a lot of wonderful feedback from the industry. It was this feedback that helped us shape the application we have today. Today, there are more than 300K registered free user accounts and more than 4,500 paid business and managed service provider (MSP) accounts worldwide—more specially, in both Europe and the United States.

PK: How has the IT management landscape evolved in the past year?

MM: Most of the actions we take and the features we implement are based on the needs of our users. We pay careful attention to our customer feedback and requests. And we implement a lot of this feedback, with simplicity in mind.

Go to Full Article

Why Your Server Monitoring (Still) Sucks

Mike Julian — Thu, 01 Nov 2018 15:25:46 +0000

by Mike Julian

Five observations about why your your server monitoring still stinks by a monitoring specialist-turned-consultant.

Early in my career, I was responsible for managing a large fleet of printers across a large campus. We're talking several hundred networked printers. It often required a 10- or 15-minute walk to get to some of those printers physically, and many were used only sporadically. I didn't always know what was happening until I arrived, so it was anyone's guess as to the problem. Simple paper jam? Driver issue? Printer currently on fire? I found out only after the long walk. Making this even more frustrating for everyone was that, thanks to the infrequent use of some of them, a printer with a problem might go unnoticed for weeks, making itself known only when someone tried to print with it.

Finally, it occurred to me: wouldn't it be nice if I knew about the problem and the cause before someone called me? I found my first monitoring tool that day, and I was absolutely hooked.

Since then, I've helped numerous people overhaul their monitoring systems. In doing so, I noticed the same challenges repeat themselves regularly. If you're responsible for managing the systems at your organization, read on; I have much advice to dispense.

So, without further ado, here are my top five reasons why your monitoring is crap and what you can do about it.

1. You're Using Antiquated Tools

By far, the most common reason for monitoring being screwed up is a reliance on antiquated tools. You know that's your issue when you spend too much time working around the warts of your monitoring tools or when you've got a bunch of custom code to get around some major missing functionality. But the bottom line is that you spend more time trying to fix the almost-working tools than just getting on with your job.

The problem with using antiquated tools and methodologies is that you're just making it harder for yourself. I suppose it's certainly possible to dig a hole with a rusty spoon, but wouldn't you prefer to use a shovel?

Great tools are invisible. They make you more effective, and the job is easier to accomplish. When you have great tools, you don't even notice them.

Maybe you don't describe your monitoring tools as "easy to use" or "invisible". The words you might opt to use would make my editor break out a red pen.

This checklist can help you determine if you're screwing yourself.

Go to Full Article

The Monitoring Issue

Bryan Lunduke — Thu, 01 Nov 2018 14:20:53 +0000

by Bryan Lunduke

In 1935, Austrian physicist, Erwin Schrödinger, still flying high after his Nobel Prize win from two years earlier, created a simple thought experiment.

It ran something like this:

If you have a file server, you cannot know if that server is up or down...until you check on it. Thus, until you use it, a file server is—in a sense—both up and down. At the same time.

This little brain teaser became known as Schrödinger's File Server, and it's regarded as the first known critical research on the intersection of Systems Administration and Quantum Superposition. (Though, why Erwin chose, specifically, to use a "file server" as an example remains a bit of a mystery—as the experiment works equally well with any type of server. It's like, we get it, Erwin. You have a nice NAS. Get over it.)

...

Okay, perhaps it didn't go exactly like that. But I'm confident it would have...you know...had good old Erwin had a nice Network Attached Storage server instead of a cat.

Regardless, the lessons from that experiment certainly hold true for servers. If you haven't checked on your server recently, how can you be truly sure it's running properly? Heck, it might not even be running at all!

Monitoring a server—to be notified when problems occur or, even better, when problems look like they are about to occur—seems, at first blush, to be a simple task. Write a script to ping a server, then email me when the ping times out. Run that script every few minutes and, shazam, we've got a server monitoring solution! Easy-peasy, time for lunch!

Whoah, there! Not so fast!

That server monitoring solution right there? It stinks. It's fragile. It gives you very little information (other than the results of a ping). Even for administering your own home server, that's barely enough information and monitoring to keep things running smoothly.

Even if you have a more robust solution in place, odds are there are significant shortcomings and problems with it. Luckily, Linux Journal has your back—this issue is chock full of advice, tips and tricks for how to keep your servers effectively monitored.

You know, so you're not just guessing of the cat is still alive in there.

Mike Julian (author of O'Reilly's Practical Monitoring) goes into detail on a bunch of the ways your monitoring solution needs serious work in his adorably titled "Why Your Server Monitoring (Still) Sucks" article.

We continue "telling it like it is" with Corey Quinn's treatise on Amazon's CloudWatch, "CloudWatch Is of the Devil, but I Must Use It". Seriously, Corey, tell us how you really feel.

Go to Full Article

CloudWatch Is of the Devil, but I Must Use It

Corey Quinn — Wed, 31 Oct 2018 11:30:00 +0000

by Corey Quinn

Let's talk about Amazon CloudWatch.

For those fortunate enough to not be stuck in the weeds of Amazon Web Services (AWS), CloudWatch is, and I quote from the official AWS description, "a monitoring and management service built for developers, system operators, site reliability engineers (SRE), and IT managers." This is all well and good, except for the part where there isn't a single named constituency who enjoys working with the product. Allow me to dispense some monitoring heresy.

Better, let me describe this in the context of the 14 Amazon Leadership Principles that reportedly guide every decision Amazon makes. When you take a hard look at CloudWatch's complete failure across all 14 Leadership Principles, you wonder how this product ever made it out the door in its current state.

"Frugality"

I'll start with billing. Normally left for the tail end of articles like this, the CloudWatch billing paradigm is so terrible, I'm leading with it instead. You get billed per metric, per month. You get billed per thousand metrics you request to view via the API. You get billed per dashboard per month. You get billed per alarm per month. You get charged for logs based upon data volume ingested, data volume stored and "vended logs" that get published natively by AWS services on behalf of the customer. And, you get billed per custom event. All of this can be summed up best as "nobody on the planet understands how your CloudWatch metrics and logs get billed", and it leads to scenarios where monitoring vendors can inadvertently cost you thousands of dollars by polling CloudWatch too frequently. When the AWS charges are larger than what you're paying your monitoring vendor, it's not a wonderful feeling.

"Invent and Simplify"

CloudWatch Logs, CloudWatch Events, Custom Metrics, Vended Logs and Custom Dashboards all mean different things internally to CloudWatch from what you'd expect, compared to metrics solutions that actually make some fathomable level of sense. There are, thus, multiple services that do very different things, all operating under the "CloudWatch" moniker. For example, it's not particularly intuitive to most people that scheduling a Lambda function to invoke once an hour requires a custom CloudWatch Event. It feels overly complicated, incredibly confusing, and very quickly, you find yourself in a situation where you're having to build complex relationships to monitor things that are themselves far simpler.

Go to Full Article

Pulseway: Systems Management at Your Fingertips

Petros Koutoupis — Mon, 30 Jul 2018 20:51:51 +0000

by Petros Koutoupis

In today's IT world, staying on top of anything and everything related to the most mission-critical applications or machines is increasingly important. With this need in mind, Pulseway provides a product of the same name built to give IT personnel the ability to monitor, manage and automate these very systems and the tasks or applications that they host. Managing an entire computing ecosystem (consisting of both physical and virtual machines) never should be too difficult a task, and Pulseway has proven that to be the case.

I recently was fortunate to have the opportunity to take this product for a spin. It's extremely simple to install and configure, and if you need help, everything is well documented in the User Manual on the company's website.

So, how does it work? First, you need to register an account on the Pulseway website. Two offerings currently are available: a limited free offering and a paid subscription offering. As you might expect, the limited free account limits the numbers of nodes you can manage, and it also restricts users from leveraging additional features and functionality, including an antivirus, backup/disaster recovery and more.

Once registered, you can sign in to the website and even download the mobile application to your phone or tablet—either Android or iOS. The last step is to download and install the monitoring agents to your mission-critical machines.

In my case, I installed the DEB file into an instance of Ubuntu 18.04 LTS. Once installed, the dæmon configuration file was modified to use my account credentials, and as soon as it started, the dashboard on both the website and on my mobile device saw the system, and it immediately began reporting CPU utilization, memory usage and so much more.

Figure 1. The Pulseway Web User Interface Dashboard

Both the web user interface and the mobile device share the same set of functions, so for the purposes of this review, I'm continuing with the mobile user interface.

Figure 2. The Pulseway Mobile User Interface Dashboard Summary of a Single System

Here's a rundown of some things you can do with Pulseway:

1) You can monitor historical CPU utilization to see how active or inactive your CPU cores are.

Figure 3. The Pulseway Mobile User Interface System CPU Graph

Go to Full Article

Tracking Down Blips

Shawn Powers — Tue, 26 Sep 2017 14:23:27 +0000

by Shawn Powers

In a previous article, I explained the process for setting up Cacti, which is a great program for graphing just about anything. One of the main things I graph is my internet usage. And, it's great information to have, until there is internet activity you can't explain. In my case, there was a "blip" every 20 minutes or so that would use about 4mbps of bandwidth (Figure 1). In the grand scheme of things, it wasn't a big deal, because my connection is 60mbps down. Still, it was driving me crazy. I don't like the idea of something on my network doing things on the internet without my knowledge. So, the hunt began.

Figure 1. That blip drove me crazy for weeks.

Most folks immediately told me to use Wireshark to analyze the data. That's good advice, but the problem makes me want a real-time monitoring system rather than a one-off packet search. Plus, even with Wireshark, you need to address the issue of capturing all the data flowing to and from the internet. Modern switching hardware purposefully directs traffic only to the ports on your switch where the traffic is intended. That means you can't just "sniff" the whole network without some effort. So regardless of how I was going to analyze the traffic, I had to be able to see the traffic. Thankfully, there are a few ways to accomplish that.

Sniffing All the Data

Network hubs were very common 20 years ago. The idea with a hub is that the network data coming in is repeated to every port on the hub, and whichever computer the packet was intended for accepts it. Every other computer just ignored the data. This worked fine when the amount of data was low and the speed of the data was slow, but as more devices were added to the network, it quickly became congested. About that time, "switching" technology entered the picture. A switch would accept data on every port, but repeat the packets only to the single port on which the intended device was listening. At first, switches were extremely expensive, so it wasn't uncommon to see a four-port rackmount switch that had hubs connected to each port. It was a way to segregate the congestion into manageable chunks. Eventually, switching technology became mainstream. Now even the $10 eight-port devices you can buy online are switches instead of hubs, and the idea of a hub is outdated.

Go to Full Article

HOSTING Monitoring Insights

James Gray — Fri, 24 Mar 2017 17:54:11 +0000

by James Gray

An important need for today's CIOs is gaining greater granular visibility into hybrid cloud and on-premises environments to maximize the business value of their IT assets. To make progress in this arena, two natural allies—managed cloud services provider HOSTING and hybrid IT monitoring specialist ScienceLogic—teamed up to develop HOSTING Monitoring Insights, an innovative hybrid cloud monitoring solution that delivers "the industry's first holistic and comprehensive view of hybrid cloud environments".

A view into today's situation for many organizations reveals the use of multiple platforms to monitor devices across disparate cloud and on-premises environments. The core customer benefit from Monitoring Insights is capacity to manage the health of critical business processes proactively across the whole environment and significantly reduce the quantity of monitoring tools required. Monitoring Insights is available both directly from HOSTING or through a HOSTING partner, and at various service levels based on customer need.

Go to Full Article