MemeStreams | Netflow stuff

When I was messing around with this stuff the trickiest part to get right was accounting for drift between exports from multiple hops in the path. The export times and bytes of each flow are always going to vary slightly at each hop. My experience is with V5. For TCP packets you get a hint about connection state from the ANDed flags field. UDP was more complicated because you don't get those connection state hints and have to make other assumptions like seeing the source port change on a "different" flow, but not every protocol obeys that. IKE come to mind with both source and dest port being 500. Good luck making any sense of TFTP or some of the RPC protocols with only V5 records.

Deduplication and stitching are fun engineering problems and I think Lancope gets it right for the most part.

Best aha moment for me was discovering how ICMP is reported in flows. The source port field is set to 0 and the high and low bytes of the destination port field are used to encode the type/code tuple: 256*[Type] + [Code]. Clever. A play out of the old FTP protocol book ;)