Enhancing Network Attack Detection with Distributed and In-Network Data Collection System

The collection of network data poses a significant challenge for machine/deep learning-driven network defense systems. This paper proposes a new paradigm, namely In-network Serverless Data Collection (ISDC), to eliminate the bottleneck between network infrastructure (where data is generated) and security application servers (where data is consumed). Considering the extremely mismatched scale between traffic volume and in-network resources, we stress the need to prioritize flows based on the application's interests, and a sublinear prediction algorithm is proposed to prioritize specific flows to optimize resource consumption effectively. Additionally, a negotiation-free task migration mechanism with task-data isolation is introduced to allocate tasks dynamically across the network to enhance resource efficiency. Furthermore, ISDC incorporates a serverless data migration and aggregation mechanism to ensure data integrity and serves as a reliable and distributed data source for network defense systems. We present two use cases to demonstrate the feasibility of ISDC, namely covert channel detection and DoS/DDoS attack detection. In both scenarios, ISDC achieves significantly higher flow coverage and feature accuracy compared to existing schemes, leading to improved attack detection accuracy. Remarkably, ISDC's data integrity addresses a model self-poisoning issue caused by duplicated and fragmented flow measurements generated during collaborative measurements.