Getting Started With Monit on Red Hat

Monit Console Via Browser
 
As is common in many professional environments today, the technology infrastructure that we support and maintain at my day job includes a wide variety of platforms. Narrowing our focus to operating systems, we run not only Microsoft products, but also several Linux distributions including Red Hat Enterprise Linux, Ubuntu and openSUSE. And while we monitor the applications that ride atop these operating systems using a variety of means, we encountered an interesting new requirement this week. How do we monitor, restart and alert administrators to particular failed services on an instance of Red Hat Enterprise Linux, running only a relatively lightweight process on the server itself?

Potential Solution
A few minutes with Google led us to Monit, where their web site suggests that it comes “With all features needed for system monitoring and error recovery.” Better yet, you can get “Up and running in 15 minutes!” It’s freely available for Linux, BSD and OS X. But would the claims prove true? Let’s put it to the test.

Getting Started
Those familiar with various Linux distros are well aware that package installation differs from one version to the next. The following exercise involves installing Monit on RHEL (Red Hat Enterprise Linux) 6.5. We’ll assume that we’re logged in as root for the duration of this process.

  1. First we have to enable the EPEL (Extra Packages for Enterprise Linux) repository on our RHEL server using the following two commands. The first of these commands is longer than may be depicted in this blog post, but if you triple-click on the line of text to highlight it all, you should be able to copy the full command to your buffer and then paste it into a text editor or an SSH session.
  2. Then we install Monit and start it.
  3. To this point we’ve spent maybe 2 minutes, and technically we’ve already installed Monit. Not bad, but we can’t use it yet. Monit creates it’s own web page that runs on port 2812 by default. If you’re running a firewall, you may wish to open port 2812 using the commands below. Even so, Monit doesn’t allow access to its web page without further configuration beyond the firewall, which will discuss later.
  4. All Monit configuration is done via the file /etc/monit.conf. We’ll use vim to edit it. If you’ve installed the Desktop portion of RHEL, you could use the graphic gedit in place of vim.
  5. You’ll see that virtually the entire monit.conf file is commented out with # signs by default. We’ll have to uncomment various sections and add our own parameters to make it work. Look for the portion that looks like the following and remove the # markers at the beginning of the lines. (Navigate within vim using the keyboard arrow keys, and hit insert before attempting to add any new content.) You might set the daemon value down to 60 if you wish to check at one-minute intervals.
  6. Scrolling further, uncomment the following line and replace mail.bar.baz with the name or TCP/IP address of an internal SMTP mail server through which you want to send your e-mail alerts. We ran our alerts through our local Microsoft Exchange server.
  7. Monit has a default e-mail format that’s usable, but I found myself wanting to customize it. In the example below, everything following the set mail-format, inside { }, represents the content of the alert messages. You can insert these lines following a section of monit.conf that displays the default format.
  8. Next we have to identify who receives e-mail alerts. You’ll see the example syntax: set alert manager@foo.bar. I prefer to only be alerted on things that I should be concerned about, so I might limit my alerts to only the following types. Again, triple-click to highlight and then copy-paste the following line, to make sure that you got it all.
  9. Now we come to the parameters that control the Monit web page on port 2812 as discussed earlier. Uncomment the following code. There are a few things to note here, which we’ve already incorporated into this example. We don’t need the use address parameter if we wish to listen on all interfaces. If we wish to be able to connect from client workstations in a particular IP range, you can do an allow statement with a network number and subnet mask, as seen here with 192.168.1.0. The allow admin:monit section will require that username and password when connecting. You might want to change it from the default.
  10. Finally, we have to decide what to monitor. This is where it gets a bit more complicated, and Google is your friend. The following example can be configured to monitor the Apache web server. Unfortunately, the path to httpd.pid within monit.conf isn’t accurate for Red Hat, but it is accurate in the example below. If you’re having trouble locating the pid file for a particular service, you can always try locate *.pid.
  11. Let’s also put in an entry to alert if our root file system get’s larger than 90% full.
  12. At this point we’re done editing monit.conf for purposes of this conversation. If editing with vim, we’ll hit Esc if we’re still in insert mode, and then :wq to write our changes and quit. Let’s restart Monit and give it a try. You’ll know if you got a syntax error pretty quickly, as Monit will alert you to the error and fail to restart if one exists. (Go back and modify /etc/monit.conf to fix any syntax errors.)
  13. Assuming Monit restarts successfully, you should now be able to connect via a web browser to your host’s name or IP address, followed by :2812.
  14. If you wish to test monit, you might force down apache to see if it restarts, and whether you receive an alert message. Keep in mind that Monit only checks every 120 seconds, or whatever interval we specified.
  15. Assuming your e-mail configuration is correct, and that your SMTP server is relaying e-mail for you, you should receive one message indicating that apache is not running, and a second indicating that it is running once again.

Extending Monit Further
While Monit is free, there’s an M/Monit license that provides consolidated monitoring of multiple Monit clients, as well as a mobile monitoring site. Pricing starts at 65 Euros for five hosts.

Conclusion
Monit meets our requirement of having lightweight application monitoring, restart and alerting running directly atop specific Red Hat Enterprise Linux servers. And while it takes longer than fifteen minutes to fully understand one’s configuration options, one can indeed be “Up and running in 15 minutes” with a little practice. All in all, Monit is a pretty impressive free product.