Rule2Alert Project Release

This is an idea I had a while ago and the Rule2Alert project implemented it (awesome work Josh)

Step1. Take a snort rule like this:
alert tcp $HOME_NET any -> $EXTERNAL_NET 80 (msg:"Snort alert"; flow:to_server,established; content:"|56 24 5a 63|"; content:"hey"; distance:5; within:12; sid:2000000; rev:1;)

Step2. Create a pcap (packet capture file) or network traffic to trigger this rule.
(Mind you certain things like pcre are incredibly challenging to implement)

Why is this useful you say?
I can see many uses -

1) You can test your rules to make sure they work

2) You can determine if overlap exists between one or more different rule (therefore, consider disabling one, or merging the rule into one). This was a goal of one of the public community/EmergingThreats/Sourcefire project that I think has dropped off the radar since it was initially started.

3) You can also test other non-snort Intrusion Detection Engines for known signatures to determine how much overlap exists.

4) An attacker could also use this tool to do some sneaky things. I won't go into much detail here, but leave a comment if you think of anything.

More on the project from "New Project - Rule2Alert" at http://malforge.com/node/22 or get the source code from http://code.google.com/p/rule2alert/

Jsunpack-n update 0.3.1b: Functionality Updates

Today I am releasing a new version of jsunpack-n that fixes several different bugs and increases the functionality of jsunpack-n.

This release REQUIRES an up to date version of YARA (1.3 or greater), because the rules file makes use of the new rule syntax. If you experience problems compiling the YARA rules, this is likely the reason.

In this release, I added support for lastModified, which attackers have used in attacks to prevent analysis (this will only work if you use a pcap file), since it is part of the network traffic that an attacker sends from a malicious server.

Here is the full CHANGELOG below:

1) rule updates for yara 1.3 rule language
2) fixes in PDF JavaScript parsing
3) improvements to the tree structure, made appending children better
4) cmdline options for logging and temporary directories
5) additions to pre.js and post.js to handle App.eval, String.eval, and better definitions for Adobe version variables
6) handle document.write and document.writeln with multiple parameters

Using a Custom Spidermonkey Version and bug fixes

I've added instructions within "INSTALL.spidermonkey" showing how to build a js-1.8.0rc1 custom version of spidermonkey with hooked eval() statements. Upon doing so, your decodings will be more reliable and effective!

Attackers can use techniques that change the scope of variables, therefore JavaScript hooks are not sufficient to handle them. When you modify the spidermonkey engine in this manner, you do not change the scope and such cases will continue to decode successfully. I do not distribute spidermonkey with jsunpack-n so you still have some choices in this area, and therefore I did not disable the eval() hooks in pre.js, which this spidermonkey modification replaces.

I've been working much more on parsing SWF files and I have a development version of a Flash Decompiler (for ActionScript code) in the works, stay tuned for that!

From the CHANGELOG:
Updates 2009-10-08 version 0.3.1a
1) bug fixes release
1a) I now distribute an optional gzip.py file (on by default)
This file was built to fix gzip decompression errors (from python2.5), you may not want to use js this if you use python2.6
2) rule detection updates
3) updates to pre.file
4) added instructions for compiling and using custom spidermonkey version INSTALL.spidermonkey
5) (not new) you can type "make clean" to destroy all temporary and log files

Jsunpack-n update v0.3a: SWF parsing and Bug fixes release

The main new feature in this release is the "swf.py" file, as a standalone you can run it like this:

$ ./swf.py sample-swf-js.file
processing flash file [version 4] (length 115, actual length 115)type=0x9 length=3 name=SetBackgroundColor
type=0x18 length=31 name=Protect
type=0xc length=46 name=DoAction
actionCode 0x83 len(42) ActionGetURL javascript:eval(fV6("ZlY4KGZWMSwxKQ=="))
actionCode 0x0 len(0) unknownAction

tags (with counts) of length=0
End:1, ShowFrame:1
sample-swf-js.file ['javascript:eval(fV6("ZlY4KGZWMSwxKQ=="))']

$ ./swf.py sample-swf-url.file
processing flash file [version 8] (length 1125772, actual length 1125772)type=0x45 length=4 name=FileAttributes
type=0x9 length=3 name=SetBackgroundColor
type=0xc length=65 name=DoAction
actionCode 0x83 len(45) ActionGetURL http://5173vip.seawww.cn/cuteqq.htm (_blank)
actionCode 0x96 len(12) ActionPush datatype[0]=string(text)
actionCode 0x1d len(0) ActionSetVariable
...

As you can see, you can embed both URLs and javascript within Flash SWF files. jsunpack-n uses this module to follow those links and report any obtained information.

The changelog follows:

Updates 2009-09-25 version 0.3a
1) new extraction of URLs/JavaScript from Flash files (CWS/FWS) with swf.py
2) significant performance improvements in shellcode processing
3) bug fixes
3a) fixed tree structure of urls (specific to pcap processing)
when a node could detatch itself from the tree incorrectly

Jsunpack-n update v0.1f: Active Mode and Client version Enumeration

Attackers frequently try to hide their exploits using version detection. They profile the client software (browser or PDF reader), then only launch an exploit or decode the payload provided you use a vulnerable version. To counteract this, jsunpack-n now uses multiple different version strings and uses the best result.

More signatures and better PDF decoding (pdf.py) with Ascii85Decode support!

I also added active mode (-a), which fetches any [not analyzed] URLs and can be used with URL fetching (-u):

$ ./jsunpack-n.py -u "www.bbkmobile.com" -a
URL fetch www.bbkmobile.com
      (referer=www.google.com/trends/hottrends)
      saved 1647 bytes to ./files/fetch_b8df4c6607205922a41d6448de0dda45d3885951

Active Mode, fetching x new URLs
      [...cut...]

[nothing detected;children=malicious:10] (script) www.bbkmobile.com/
      [suspicious:5] (script) www.crcf.org.cn/logo.gif?b
            suspicious: DecodedIframe detected <iframe
            [nothing detected;children=malicious:10] (iframe) knownsec.7766.org/wwj2/1.htm?
[...cut...]

More info from the CHANGELOG:

Updates 2009-09-18 version 0.1f
1) active fetching of with -a, and evaluation of urls with -u, use both (-u URL and -a) for purely active analysis
2) evaluation of multiple different client version strings:
2a) version enumeration: adobe reader for pdf
2b) version enumeration: IE7, IE8, Firefox, Opera
2c) cumulative evaluation time limits per decoding, and inference of code coverage based upon evaluation time
3) added pdf decoding support for ASCII85Decode and made other improvements to pdf decoding
4) rules updates

Get it here!

Jsunpack-n Update v0.1e: Graphical output, directory output, command line options and fixes

A new version of jsunpack-n is available 0.1e! This version makes some major improvements in the rule language (using Yara) and allows you to make pretty pictures like this:



One thing I think you will like is the new output ./files/ directory and command line options! Enjoy.

Here is it running with the sample files included in the archive:

$ ./jsunpack-n.py sample-http-exploit.pcap
[malicious:10] hifgejig.cn/nuc/
[impact=5] DecodedGenericCLSID detected F0E42D60-368C-11D0-AD81-00A0C90DC8D9
[impact=10] MSOfficeSnapshotViewer detected F0E42D60-368C-11D0-AD81-00A0C90DC8D9
[impact=5] ObfuscationPattern detected location eval
[impact=10] MSIENestedSpan detected CDATA[<image SRC=http://&# DATAFORMATAS= <SPAN DATASRC= DATASRC= DATAFLD=
[info] find_urls: [javascript var] hifgejig.cn/nuc/exe.php

$ ./jsunpack-n.py sample-pdf.pcap
[suspicious:5] trughtsa.com/
[impact=5] DecodedIframe detected <iframe
[info] [iframe /] trughtsa.com/img/pfqa.php

[malicious:10] trughtsa.com/img/pfqa.php
[impact=10] PDFexploit detected util.printf Collab.getIcon getAnnots Collab.collectEmailInfo spell.customDictionaryOpen
[malicious] identified shellcode of length 1533 (./files/shellcode_9ac3a76f70caef94f2773abc1043e9511d2d0f09)
[info] XOR key [shellcode]: 33
[info] shellcode url [xor] trughtsa.com/img/uet.php
[impact=5] ObfuscationPattern detected eval String.fromCharCode

[malicious:10] trughtsa.com/img/uet.php
[incident:10] [0] requested by 192.168.203.60
[incident:10] [0] origin trughtsa.com/img/pfqa.php
[incident:10] [0] method=GET type=shellcode
[incident:10] [0] filetype MS-DOS executable PE for MS Windows (GUI) Intel 80386 32-bit

$ ./jsunpack-n.py sample-pdf.file
[malicious:10] sample-pdf.file
[impact=10] PDFexploit detected collab.getIcon
[suspicious] likely NOP sled shellcode variable of length 167
[malicious] identified shellcode of length 1526 (./files/shellcode_da344d16e814e40dec67592bdccdf3ad50e0069d)
[info] XOR key [shellcode]: 33
[info] shellcode url [xor] b35.info/w/who.exe
[suspicious] likely NOP sled shellcode variable of length 2048
[suspicious] likely NOP sled shellcode variable of length 1714
[suspicious] likely NOP sled shellcode variable of length 522574

More from the RELEASE notes:

RELEASE NOTES:
Updates 2009-09-02 version 0.1e

First and foremost, thanks to Victor! (for creating the Yara detection library)
Yara is now a required dependency and the supported format for the 'rules' file

1) improved URL tracking using 'urlattr' class and urls dictionary
1a) new command line option -g, to create a URL graph (only when pcap contains 10 or fewer URL requests)
2) bug fixes for stream reassembly and pdf parsing
2a) stream reassembly now handles all streams when processing a pcap file,
regardless of whether the nids state is in end_states
4) detection of NOP sled shellcode and performance improvements in shellcode processing
(this was one of the performance bottlenecks)
5) new output format with ./files/ directory or -d OUTDIR command line option
6) CVE references are available in the 'rules' file but are temporarily unavailable in alerts

Jsunpack-n Update v0.1d: Great New Features, Detection, and Other Improvements

The coolest feature in my opinion is demonstrated below using the sample-pdf.pcap file included in the archive:

$ ./jsunpack-n.py sample-pdf.pcap
[suspicious:2] trughtsa.com/
[impact=2] CVE-NO-MATCH (id 9) detected eval
[suspicious] [0]
[info] [iframe /] trughtsa.com/img/pfqa.php

[malicious:10] trughtsa.com/img/pfqa.php
[impact=2] CVE-NO-MATCH (id 9) detected eval
[suspicious] [0]
[suspicious] [0] decoded 25275 bytes
[impact=2] CVE-NO-MATCH (id 9) detected eval
[suspicious] [1]
[suspicious] [1] decoded 12269 bytes
[impact=10] CVE-2008-2992 (id 1) detected util.printf
[impact=10] CVE-2009-1493 (id 2) detected spell.customdictionaryopen
[impact=10] CVE-2009-1492 (id 3) detected getannots
[impact=10] CVE-2007-5659 (id 4) detected collab.collectemailinfo
[impact=10] CVE-2009-0927 (id 5) detected collab.geticon
[impact=3] CVE-NO-MATCH (id 8) detected string.fromcharcode
[impact=2] CVE-NO-MATCH (id 9) detected eval
[malicious] identified shellcode of length 1533
[info] XOR key [shellcode]: 33
[info] exploit_watch append [shellcode] trughtsa.com/img/uet.php
[malicious] [2]
[malicious] [2] decoded 4626 bytes

[malicious:10] trughtsa.com/img/uet.php
[incident] [0] Exploit successful [origin trughtsa.com/img/pfqa.php]
[incident] [0] Exploit successful [victim 192.168.203.60]
[incident] [0] Exploit successful [type MS-DOS executable PE for MS Windows (GUI) Intel 80386 32-bit]

Notice that each URL is flagged as suspicious/malicious (or [benign] in case you use -v or -V). Check out the "rules" file, which uses a Snort-like syntax to express what exactly you want to detect. For now, it is very simple and only allows pcre-style detection rules, each of which must match for it to classify the URL according to the impact level (-1 is experimental, 0 is benign, 0-5 is suspicious, 6-10 is malicious). If you create some great new rules, post them as a reply to this thread or send them to me and I'll add them to the "rules" file that I distribute with jsunpack-n.

Another cool thing about this version is that you no longer need the pynids/libnids libraries for it to work with non-PCAP files. A lot of people wanting to use jsunpack-n been having issues installing this library so that is the reason I've made it optional. Here is an example of processing a PDF file on the local system (sample-pdf.file is an example file included in the jsunpack-n package for testing that your local installation functions properly).

$ ./jsunpack-n.py -V sample-pdf.file
Processing sample-pdf.file
[malicious:10] sample-pdf.file
[info] [0] found JavaScript
[info] [0] decoded 14602 bytes
[info] [1] found JavaScript
[malicious] analysis exceeded 30 seconds (125644 bytes, incomplete)
[info] [1] decoded 125644 bytes
[impact=10] CVE-2009-0927 (id 5) detected collab.geticon
[malicious] identified shellcode of length 35223
[info] XOR key [shellcode]: 33
[info] exploit_watch append [shellcode] b35.info/w/who.exe
[malicious] [2]
[malicious] [2] no JavaScript

One thing you will notice about this case is that collab.geticon is not visible from the immediate decoding (but is visible via a print "//jsunpack.called collab.getIcon". It also demonstrates the timeout on JavaScript evaluation (the -t command line option). The new command line arguments that are available can be listed via the following:

$ ./jsunpack-n.py -h
Usage:
./jsunpack-n.py [fileName]
./jsunpack-n.py [interfaceName]
jsunpack-network version 0.1d (alpha)

Options:
-h, --help show this help message and exit
-t TIMEOUT, --timeout=TIMEOUT
limit on number of seconds to evaluate JavaScript
(default 30 seconds)
-v, --verbose verbose mode displays status for all files, even if
they are benign
-V, --very-verbose shows all decoding errors (noisy)
-D, --debug (experimental) debugging option to profile memory
usage

Additionally, you can inspect the CHANGELOG file for all new features:
Updates 2009-08-01 version 0.1d
1) determination of whether the code is malicious or benign (see detection.py)
2) better tracking with exploit_watch and ability incident alert for infected IP address
3) pynids 'import nids' library is now optional due to user feedback
4) additional command line arguments -h (help), -t (timeout), -v (verbose), and -V (very verbose)
5) bug fixes and performance improvements
6) added debug option -D, which profiles memory usage (get Heapy from http://guppy-pe.sourceforge.net/#Heapy)