View RSS Feed

Godwin Austin

Collective Intelligence Framework – An awesome and pretty useful project

Rating: 3 votes, 2.33 average.
Hello Hackers!

How are you doing?
I am here today to shed some light on a nice and open source project called Collective Intelligence Framework (CIF).

About 70 % of internet traffic is …. Wait for it …. SPAM! If you don’t believe me, install a service honeypot, give it about 10 minutes and then see the magic. Or get your machine direct public interface and start TCPDump.

The internet if full of crap / awesome stuff (in the eye of the beholder) like exploit kits, botnet C&Cs, phishing frameworks etc. If one checks out the behavioral patterns of a specific IP address over time, one can surely identify traits of phases in malware propagation campaigns. This project simply keeps taking intelligence data from various sources and stores it into database. These sources could be public or private. With default installation there are about 15 to 17 public data sources. You can surly add your own by writing up a simple plugin.

So in my current CIF instance, what sources do I have? Here is a list. All these are open source publically available sources. Planning to install a bunch of honeypots to create private feeds.


  • Spyeye Tracker (abuse.ch)
  • Zeus Tracker (abuse.ch)
  • Feodo Tracker (abuse.ch)
  • Spamhaus
  • Shadowserver
  • Phishtank
  • OpenBL
  • BruteForceBlocker (danger.rulez.sk)
  • Threat Expert
  • Malware Blacklist
  • Malware Domains
  • Malware Domain List
  • Malc0de
  • Dragon Research Group
  • Clean MX


My instance runs downloads from some of these sources every hour and every day for rest of them. One can query the CIF database and get results. As CIF is holding historical data too, one can go back in history as back as the age of the CIF instance.

So what things can we query to the CIF database? Here is a list.


  • IP address
  • Domain Name
  • Malware Hash (MD5 / SHA1 / UUID)
  • CIDR Notation
  • URI


Alright, so in what formats do we get the output? Again, here is the list.


  • Bind Zone
  • Bro
  • CSV
  • HTML
  • JSON
  • PcapFilter
  • Snort
  • IPTables
  • Table (Default)


Well this has been about what one as a user can inquire about. But there is more to it. One can also make the system to shell out much updated lists. For your amusement, again…. Here is a list.

One can get lists of IP addresses / URLs / Domains / Email Addresses belonging to


  • Botnets
  • Scanners
  • Phishing pages
  • Spammers
  • Spamvertisers
  • Fastflux Botnets
  • Waraz
  • Showing suspicious behavior
  • Showing malware presence


If you are interested to know more about the project, you can visit the Google Code project link here
I thank Wes Young and team for developing this awesome project and moreover making it open source.

Ro(Ha)ck on !

--

Godwin Austin

Comments

  1. 41.w4r10r's Avatar
    Nice Blog... can you add few images showing interface...

Trackbacks

Total Trackbacks 0
Trackback URL: