Jsunpack Website Database Optimizations

I just published some new optimizations for the jsunpack.jeek.org database. This should dramatically improve the performance of the website.

Contact me if you need anything related to this update. I removed the search functionality (and associated RSS feeds) because that was one of the most performance intensive features on the database. While this type of function is still possible, I think I'll either have to limit terms that may be searched for or build a separate index structure so that it is better optimized.

[edit] In case you are curious, the jsunpack database has 186,459 submissions and 686,232 evalated scripts and URLs since 2010-01-29 14:17:36. This year was pretty active for web exploits. The majority of the submissions to jsunpack were publicly released in one of the RSS feeds with 167,356 submissions over this year. The opposite of that were the entries where users wished those submissions to remain private totaling 19,107 submissions.

Importing jsunpackn.py as a python library

In case you were wondering, it *IS* possible to import jsunpack-n from your own python programs. The benefits include:
1) greater control of the options (that you'd normally specify on the command line or in the configuration file)
2) control of the output in their native data structures (no more $ ./jsunpackn.py file|grep -E "suspicious|malicious" >> readme.log)

Check out the new exampleImport.py script, now available in the project's google code svn site.

If you review the main code for the exampleImport.py script, you will see that it defines a function that allows you to directly pass JavaScript... for example:
main('eval("var a=123;");')

Then you can see these loops in the main function:

        for type,hash,data in js.rooturl[url].files:
            print 'file              type=%s, hash=%s, data=%d bytes' % (type,hash,len(data))
        for printable,impact,msg in js.rooturl[url].msg:
            print 'output message    printable=%d, impact=%d, msg=%s' % (printable,impact,msg)

The point of this post is to show you that you can directly access these files and perform decoding using jsunpackn.py as a python library. If you need examples for how to operate on more than just JavaScript as the exampleImport.py shows, then try looking at the main function of jsunpackn.py, and you'll see how I process the various different types of input.

Cheers,