<span style="display:block;height:15px!important"></span>
<p align="center"><img src="https://git.doublebastion.com/shas-probe/raw/develop/img/System_Health_and_Security_Probe_logo_simple.png" alt="System Health and Security Probe" width="300px" height="187px"/></p>

<span style="display:block;height:20px!important"></span>

**System Health and Security Probe is a program that runs periodically to investigate general system health and security problems and to send report emails to admins.**

This program sends report emails to administrators if it detects at least one of the following problems:

* Any service running on the host server is in failed state.

* The periodic ClamAV scan has detected any viruses in the mail directories or in the Nextcloud upload
  directories or the logs show recent virus detections in the incoming emails or in the files uploaded
  to Nextcloud.

* Any new IP address has been banned during the last run interval due to repeated failed log in
  attempts against one of the applications monitored by Fail2ban.

* The free disk space on all partitions is less than a threashold established by the admin.

* The average CPU load in the last 15 minutes exceeded 100% utilization of all the CPU cores.

All the IP addresses banned by Fail2ban are stored in the database and if one IP has been
banned more than once, the 'whois' data for that IP is included in the email report. System Health 
and Security Probe searches for the 'abuse' email address in the 'whois' data and writes a draft 
email that can be sent manually or automatically to the entity that owns the offending IP. The draft 
email includes the log lines containing the failed log in attepts of the respective IP. The text of 
the draft email is included in the periodic email report sent to the admin, below the 'whois' data of 
each IP that was banned more than once. If the 'automatic\_emails\_to\_isp' parameter in the 
'shsp-config.php' file is set to 'yes', the program will automatically send the abuse report email 
to the entity that owns the offending IP, at the moment its ban number increases by 1, if it has 
been banned at least once in the past.

Before sending the email report to the admin, the program compares it with the last sent report, and if 
there is nothing important or new to report, the new email will not be sent, since it is assumed that
the admin has already been informed about the current problems in the previous email. This is to avoid
flooding admins with too many emails. For example, if at a particular moment it detects that the only
problem of the server is that there are 75 banned IPs, and at the moment of the next run it detects that
4 of those IPs have been debaned due to their bantime being over, it won't send the email report, because
the debanning of the 4 IPs is not important enough to deserve a new email. However, if it detects that
since the last run a new IP has been banned, the email report will be sent.

<span style="display:block;height:20px!important"></span>

<span style="display:block;text-align:center;">Sample email report</span>
<span style="display:block;height:10px!important"></span>
![Sample email report image](https://git.doublebastion.com/shas-probe/raw/develop/img/shas-probe-1.png)
<span style="display:block;height:40px!important"></span>
<span style="display:block;height:10px!important"></span>
![Sample email report image](https://git.doublebastion.com/shas-probe/raw/develop/img/shas-probe-2.png)
<span style="display:block;height:40px!important"></span>
<span style="display:block;height:10px!important"></span>
![Sample email report image](https://git.doublebastion.com/shas-probe/raw/develop/img/shas-probe-3.png)
<span style="display:block;height:40px!important"></span>
<span style="display:block;height:10px!important"></span>
![Sample email report image](https://git.doublebastion.com/shas-probe/raw/develop/img/shas-probe-4.png)
<span style="display:block;height:40px!important"></span>
<span style="display:block;height:10px!important"></span>
![Sample email report image](https://git.doublebastion.com/shas-probe/raw/develop/img/shas-probe-5.png)
<span style="display:block;height:40px!important"></span>

## Programming Languages
<span style="display:block;height:10px!important"></span>

System Health and Security Probe only uses PHP, SQL and HTML. This means it's efficient, light-weight and easy to maintain and debug.

<span style="display:block;height:20px!important"></span>

## Minimum Requirements
<span style="display:block;height:10px!important"></span>

This program is designed to be used as part of 'RED SCARF Suite'. It requires at least an abridged version 
of RED SCARF Suite and Postfix, installed and configured on a Debian server as described in the 
<a href="https://www.doublebastion.com/free-server/complete-guide-to-a-complete-linux-server/" rel="noreferrer noopener" target="_blank">Complete Guide to a Complete Linux Server.</a> 
At least the following jails need to be configured in '/etc/fail2ban/jail.local': sshd, postfix and postfix-sasl.

The user 'root' has to be allowed to send emails from command line, by specifying it in the
'/etc/postfix/main.cf' file, in the 'authorized\_submit\_users' parameter.

The log files written by the programs installed on the server should be rotated only when 
they reach about 2M in size (with the exception of WordPress websites, whose logs need to be rotated 
once every day because of Matomo), which can be configured in their respective logrotate files located 
in the '/etc/logrotate.d' directory. This is because 'System Health and Security Probe' needs to read 
a large number of log lines in order to find enough failed log in attempts that can be included in the 
draft abuse report emails.

Important !!! The ban time set for all the jails in '/etc/fail2ban/jail.local' (in seconds) must be 
greater than the interval between two consecutive runs of the program, that you set in the 
'shsp-config.php' file in the '$time\_in\_hours' parameter. If the run interval is set to 8 hours,
which is the default, all the jails in Fail2ban must have a ban time greater than 8 hours (which
means greater than 28800 seconds). Also, in '/etc/fail2ban/jail.local' it's necessary to specify a 
ban time (in seconds) for each and every jail, even for the jails who use the default ban time, for 
which it's not required. So, in every jail block, a 'bantime' parameter should be included.

Important !!! If the run interval is changed in the 'shsp-config.php' file (in the '$time\_in\_hours' 
parameter), the cron job should be changed accordingly. To run the script every 8 hours, at 5 minutes 
past, the cron job should be:

   5 */8 * * * php /srv/scripts/shas-probe/system-health-and-security-probe.php > /dev/null 2>&1

To read the email reports sent by System Health and Security Probe you will also need an email client
capable of displaying HTML emails: either a standalone email client like Thunderbird, or a web-based 
client, like Roundcube.

<span style="display:block;height:20px!important"></span>

## Installation
<span style="display:block;height:10px!important"></span>

<a href="https://www.doublebastion.com/install-system-health-and-security-probe/" rel="noreferrer noopener" target="_blank">This chapter</a> of our Complete Guide to a Complete Linux Server 
explains in detail how to install and use this application.

<span style="display:block;height:20px!important"></span>

## Contribute
<span style="display:block;height:10px!important"></span>

This is the official git repository of System Health and Security Probe. The <a href="https://github.com/DoubleBastionAdmin/shas-probe" rel="noreferrer noopener" target="_blank">GitHub 
System Health and Security Probe repository</a> is just a pointer to this repository. We don’t use GitHub for developing System Health and Security Probe because GitHub is owned by one 
of the companies that proved their disrespect for digital freedom over the years and because centralized services create autonomy and privacy issues, in spite of all their advantages.

If you want to contribute code to this project, please submit <a href="https://git.doublebastion.com/shas-probe/pullrequests/contrib" rel="noreferrer noopener" target="_blank">this form</a>, 
mentioning your intended changes. We'll send you the credentials needed to push code to the "contrib" branch of this repository. After we review the changes, we can include them in the 
project.

Please post any bugs that are not security related, or feature requests, on the <a href="https://git.doublebastion.com/shas-probe/issues/develop" rel="noreferrer noopener" target="_blank">
issue tracker</a>. If you notice bugs related to security, don’t post them on the issue tracker; instead, send them to manager [at] doublebastion [dot] com .

<span style="display:block;height:20px!important"></span>

## License
<span style="display:block;height:10px!important"></span>

System Health and Security Probe is licensed under the GNU General Public License Version 3 or any later version.
