Parsing large number of small files in binary format. Currently getting OutofMemoryError looking for your contribution.
This library is distributed under the LGPL.
See: https://raw.github.com/RIPE-NCC/hadoop-pcap/master/LICENSE
<repositories>
<repository>
<id>hadoop-pcap</id>
<url>http://dl.bintray.com/hadoop-pcap/hadoop-pcap</url>
</repository>
</repositories>
We have created a screencast showing the use of the Hadoop PCAP SerDe in Hive using Amazon Elastic MapReduce.
You can find the video on YouTube: http://www.youtube.com/watch?v=FLxeQciax-Q
This project consists of two components:
Bundles the code used to read PCAPs. Can be used within MapReduce jobs to natively read PCAP files.
See: https://github.com/RIPE-NCC/hadoop-pcap/tree/master/hadoop-pcap-lib
Implements a Hive Serializer/Deserializer (SerDe) to query PCAPs using SQL like commands.
See: https://github.com/RIPE-NCC/hadoop-pcap/tree/master/hadoop-pcap-serde
=====================================================================================================================================