Domain-specific compression algorithm for pcap network trace files


For network optimisation and analysis, network traces are important to capture and store. Such traces can take a lot of storage space, even if only packet headers are captured and stored. Pattern-recognising compression algorithms like lempel-ziv will perform very good on such pcap files, but it may be possible to improve on these generic algorithms by exploiting domain-specific knowledge about the target. Very different strategies may be employed if doing real-time compression vs. compression of already captured files. It may be possible to utilise domain knowledge to also achieve a decent compression algorithm for real-time capture and integrate it in the libpcap system.



Explore the creation of a domain-specific compression algorithm for pcap files. The scope should be limited to IP networks using IPv4 and IPv6. Compare the compression ratio of the developed solution with the best existing pattern-recognition compression algorithms. Optionally explore making a real-time compression algorithm that can improve storage for traces made on nodes without ample storage space.


Learning outcome:

Knowledge of network and packet capture strategies.

Knowledge about compression algorithms.

Knowledge of scientific evaluation and comparison.

Possibility of contributing to libpcap.



Programming experience.

Network experience.

Algorithms experience.



Networks and distributed systems / Simula Research Laboratory


Contact person:

Andreas Petlund