SIM and SEM
Information technology security in the 1990’s was heavily focused on protecting the perimeter. Early security information and event management (SIEM) solutions developed during this time were basic and came as either a security information management (SIM) or security event management (SEM) solution. They were largely available as appliances that had to be deployed on-premises in your data center. This limited your ability to scale, because adding more capacity required buying more equipment from vendors.
Early solutions were also built on proprietary data bases that locked customers into one vendor’s technology. If you wanted to move your data to another system, the process was long and complicated. Connectors, if they were available, had to be heavily customized. Storage was more expensive, so only the most valuable data was collected.
Even if you got all the data you needed into your solution, searching and alerting was rudimentary and depended on knowledgeable security analysts to research, understand, and interpret what they found in the data.
As data became cheaper and technology more powerful, SIEMs became better able to ingest, process, and store data. SIEMs were now able to use signature-based alerts to find threats in your data. However, signature-based alerts can only find known alerts, and you needed to know the indicators of compromise (IOC) for a threat before you could find it. Threat intelligence feeds were created by various companies and organizations to share relevant IOC data.
For all the benefits, there were also serious drawbacks to this generation of SIEMs. They had a very limited capability to discover zero-day or unknown threats. For example, you could write a rule such as, ‘give an alert if a user enters 10 wrong passwords in a row.’ In theory this could be used to detect brute force password attacks. However, what if the attacker only tried 9 passwords in a row? Or what if you had a very forgetful user? Signature-based attempts to find unknown threats inevitably generated a lot of noise and not a lot of value. It doesn’t help that for many legacy SIEMs, the more rules you had running, the slower your system ran. With the rapid proliferation of threats these days, legacy SIEMs have become completely overwhelmed.