Tcp Stream Reassembly Output Normalizer V2 (Libtsron)

LibTSRONv2

https://code.google.com/p/jamaal-re-tools/source/browse/tsron/

Requirements: python2.7 | dpkt 1.8 | pypcap 1.1

Summary 

• LibTSRON builds and dumps ordered TCP data streams from Pcaps like TCPFlow and WireSharks (Follow TCP Stream).  It Also supports GRE and UDP.
• Data streams can be returned to a variable which is perfect for decoding/decrypting network data.  
• Data streams can be parsed by a user supplied header giving you complete control over each ordered packet returned in a data stream.
• Raw data from ordered TCP streams can be saved to an output directory.
• LibTsron can quickly analyze pcaps displaying the available streams in a pcap numerically.
• Specific streams can be returned or written to disk based upon user input.'
• LibTsron works!  The assumption that LibTsron "simply puts packets together that have the same (ip.src,tcp.sport,ip.dst,tcp.dport)." is quite simply incorrect.  
• LibTsron does indeed reassemble packets in order based off simple mathematics. 

Basic Usage: Using the built in command line utility to dump TCP/UDP or GRE streams to file

View TCP Streams 

$ python libtsron.py -p out_libpcap.pcap -D

Dump TCP Streams 

$ python libtsron.py -p out_libpcap.pcap -d out

Analyze the first stream using hexdump: 

$ hexdump -C TCP_173.194.63.178_80_192.168.1.30_56195 | less 

Calling LibTSRON from python: 

LibTSRON is perfect for creating decoders.  This example will show you the basics of using the class in order to returned ordered data to your own applications. Analyze the code below and read the comments.  I use an advanced example below which covers everything in a little more detail.  But for now, the basics.  

exampletsron.py

Running exampletsron.py 

Notice in the hexdump __libtsron_packet__ splits each packet.  So libtsron can rebuild tcp streams while allow the user to specify a delimiter for each packet dumped.  This is perfect for writing decoders.

Set 'connheader' to True and see the results: 

$ python exampletsron.py out_libpcap.pcap  | less

Each ordered packet now has a header that looks similar to this:

<--__=64.233.171.189_443_192.168.1.30_55038=__-->__libtsron_packet__

The connheader or "connection header" option appends Source host and port plus destination host and port to returned streams.   This might be useful if you're only interested in decoding source tcp data for example.  Parsing individual packets from streams can also be achieved using this option.  Note the "header" option does not need to be specified when using "connheader".  

Utilizing LibTSRON to Decode zlib compressed TCP Data (la Gh0st Rat) 

This example illustrates using LibTsron to find and decompress zlib compressed data.   zlib is used in a lot of different malware samples I've analyzed and written decoders for.  Ordered data decompression is required for properly recovering executables sent by malware for example.  

pcapzlib.py

Running pcapzlib.py

python pcapzlib.py gh0strat.pcap  | less

This decompressor builds on the exampletsron.py described above.  We are using the header variable to parse data packets defined on line 11 which is utilized on lines 35 and 37.  Once the data packets offsets are found between headers, we search for the zlib header defined on line 10.  Once the zlib header is found on line 42, we try and decompress the data at line 50.  Normally this data would be written to file, but for this example I return the printable data to the screen on line 52 and 53 producing the decompressed output you see above.  

Utilizing LibTSRON to Decode Poison Ivy Traffic:

Poison Ivy Rat (Pivy) uses Camellia to encrypt and decrypt network data along with NTDLL.RtlCompressBuffer and NTDLL.RtlDecompressBuffer for compression and decompression of certain data.  Poison Ivy provides a perfect example to illustrate the capabilities of LibTSRON because in order for an investigator to properly recover exfiltrated Pivy data from a pcap, each packet must be in its proper order for decompression to work.  Using this type of compression is brilliant because it makes data recovery via pcap that much more difficult.

Note: I've ported Pivys implementation of Camellia encryption/decryption functions plus Windows RtlDecompressBuffer function to OSX (.dylib) for this example.  The associated files and decoder are outside the scope of this post and will be covered later

1) Import the Tsron Class 

2) Like before we're using defining a header to use as a delimiter. 

3) Using ctypes we call the port of PIVYs encryption/decryption and key table functions similar to how you would using the .DLL in windows on lines 28 and 29.  For PIVY, the password must be less than 12 character lengths.  The password providing from the command line used as an argument in the C_SK function to generate a keytable for decryption  on lines 38 and 39.

4) "targ" defines the options used to send to Tsron.  Note the header variable we defined previously will be used again.

5) streamObj.TCP() returns ordered TCP data to variable "x".  Next we use the header again to distinguish the start of new packets.  Camellia decrypts 16 bytes at a time so our nested for loop handles that at on line 73.  On line 75 our port of  Camellia is called producing decrypted data.  The decrypted data is saved to a temp variable and written to disk on line 77

Typically because we're just dealing with decryption at this stage, I would generate a new pcap using nothing but pypcap and dpkt while calling libpivy.C_D(decryption function).  Once the PIVY data has been decrypted on line 75,  it could have been placed into a new decrypted pcap.  It would have been much easier to just call libtsrons at that point to reassemble ordered TCP data from the new decrypted pcap containing for decompression.  Doing things using the method shown above leaves the possibility to decompress decrypted data streams inline without using multiple decoders/applications.  The challenge in doing things this way is dealing with decompressed data sizes that extend multiple packets.  This issue can be tackled using the "header" delimiter along with the offset variables to concatenate multiple ordered packets based off the amount of data that is needed to be decompressed.  Enabling the connheader option when creating an instance or Tsron will allow you to keep track of TCP flow, which in return could be used to reconstruct a complete decrypted and decompressed pcap. :D   

But for now I will just call the Pivy data decompressor which in return will use the "pivydecrypted.bin" file to locate compressed data and decompress it to its proper file type based off ofs Pivys data scheme.

6) I ran the Pivy Decompresser against the decrypted data written to disk by LibTsron which has been placed in the "decryptedpivy" directory.

7) Which seeks through decrypted data looking for offsets to decompress based off of PIVYs packet data structure.  

8) After decompression we can see the true content of the data stolen using the PIVY RAT.  

EOF