Scalability of Hash-based Pattern Matching for High-speed Network Security and Monitoring

Gradually increasing throughput of high-speed networks puts continuous
pressure on the performance of operations over a stream of network data.
Probably the most affected operation in the area of network security
and monitoring is pattern matching, which is in the core of widely
deployed intrusion detection systems (IDSes) like Snort, Suricata and
Bro. This paper therefore proposes several optimizations of a hash-based
pattern matching architecture that together allow to increase its
throughput to 100 Gbps and beyond. Proposed optimizations target an
interconnection network between parallel hash function engines and
independent memory blocks addressed by hashes computed over short
strings of input data. Specifically, the optimizations reduce resource
utilization by sharing parts of the full interconnection network among
its several outputs and lower collision rate in these shared parts by
aggregation and distributed buffering of memory access requests. The
optimized pattern matching architecture is therefore able to utilize a
higher number of parallel hash functions, each of which can use the
interconnection network to access any memory block. This allows not only
to increase the throughput of a key component within IDSes to more than
100 Gbps, but also to support a larger set of network threat patterns
and to update this set dynamically.