Whitepaper : Alerting: Finding the Signal in the Noise

A key ingredient to managing security or performance for database administrators is having well configured alerting. It is unfortunate that any software which generates alerts can have both false positives and false negatives.

We define a false positive as a test result which is incorrect when it shows that a particular condition or attribute is present. For example, a false positive refers to alerting on an exceeded default threshold, but which the system did not account for using baselining. Refer to the section “Baseline Alerts”, below. The dangers of false positives will cause a database administrator to investigate alerts which are not actual issues leading to wasted effort.

We define a false negative as a test result which is incorrect when it shows that a particular condition or attribute is absent. For example, a false negative refers to not catching a database intrusion attempt because the number of failed logins did not exceed a certain amount over a defined period. For example, a threshold was set for ten failed logins in a five-minute window, but the attacker only tried nine times. False negatives are trickier to handle because they may mislead a database administrator into thinking they do not have a problem when they do. This could lead to unanticipated downtime, application impacts, or worse, a security breach.

False positives and false negatives can lead to a loss of confidence in the monitoring software being used. In such cases, the problem may only be the lack of a tuned alerting system.

Read this whitepaper to learn about the strategies to reduce false positives and false negatives without undue risk, how to prioritize alerts, escalation strategies, and how to tune thresholds.