Four Good Open Source Log Monitoring and Management Tools for Linux

When an operating system such as Linux is running, there are many events happening and processes that run in the background to enable efficient and reliable use of system resources. These events may happen in system software for example the init process or user applications such as Apache, MySQL, FTP and many more.

In order to understand the state of the system and different applications and how they are working, System Administrators have to keep reviewing logfiles on daily basis in production environments.

You can imagine having to review logfiles from several system areas and applications, that is where logging systems come in handy. They help to monitor, review, analyzer and even generate reports from different logfiles as configured by a System Administrator.

In this article, we shall look at the top four most used open source logging management systems in Linux today, the standard logging protocol in most if not all distributions today is syslog.

۱٫ Graylog 2

 

This is a fully integrated open source log management system that enables System Administrators to collect, index, and analyze both framed, systematic and disorganized data from just about any available source systems.

This logging system is highly pluggable and enables centralized log management from many systems. It is integrated with external components such as MongoDB for metadata and Elasticsearch used to keep logfiles and enable text search.

Graylog 2 has the following features:

1. Ready for enterprise level production
2. Includes a dashboard and an alerting system
3. Can work on data from any log source
4. Enables real time log processing
5. Enables parsing of unstructured data
6. Extensible and highly customizable
7. Offers an operational data hub

For more information view the Graylog 2 website.

۲٫ Logcheck

Logcheck is an open source log management system that helps System Administrators automatically identify unknown problems and security violations in logfiles. It periodically sends messages about the analysis results to a configured e-mail address.

Logcheck is designed as a cronjob on an hourly basis and on every system reboot by default. Three are different levels of logfile filtering are developed in this logging system which include:

1. Paranoid: is intended for high-security systems that are running very few services as possible.
2. Server: this is the default filtering level for logcheck and its rules are defined for many different system daemons. The rules defined under paranoid level are also included under this level.
3. Workstation: it is for sheltered systems and helps to filter most of the messages. It also includes rules defined under paranoid and server levels.

Logcheck is also capable of sorting messages to be reported into three possible layers which include, security events, system events and system attack alerts. A System Administrator can choose the level of details to which system events are reported depending on the filtering level though this does not affect security events and system attack alerts.

Read more about it at the Development team’s logcheck website

۳٫ Logwatch

Logwatch is a Linux/Unix system logfile analyzer and reporter that can be easily customized and it also allows a System Administrator to add additional plugins, create custom scripts that serve specific logging needs.

What it does is to review system logfiles for a given period to time and then generates a report based on system areas that you wish to collect information from. One feature of this logging system is that it is easy to use for new System Administrator and it also works on most Linux distributions available and many Unix systems.

Visit the project homepage of Logwatch

۴٫ Logstash

Logstash is also an open source data collection and logging system available on Linux, which capable of real-time pipelining, which was originally designed for data collection but its new versions now integrated several other capabilities such as using a wide range of input data formats, filtering and also output plugins and formats.

It can effectively unify data from various log source systems and normalize the data into targets of a System Administrators’ choice. Logstash also allows System Administrators to cleanse, compare and standardize all their logging data for distinct advanced analytics and also create visualization use cases as well.

Read more about it at Logstash website.

Summary

That is it for now and remember that these are not all the available log management systems that you can use on Linux. We shall keep reviewing and updating the list in future articles, I hope you find this article useful and you can let us know of other important logging tools or systems out there by leaving a comment.

References :‌tecmint.com