Integrating VirusTotal within ELK

VirusTotal Scan[This blogpost has also been published as a guest diary on isc.sans.org]

Visualisation is a key when you need to keep control of what’s happening on networks which carry daily tons of malicious files. virustotal.com is a key player in fighting malwares on a daily basis. Not only, you can submit and search for samples on their website but they also provide an API to integrate virustotal.com in your software or scripts. A few days ago, Didiers Stevens posted some SANS ISC diaries about the Integration of VirusTotal into Microsoft sysinternal tools (here, here and here). The most common API call is to query the database for a hash. If the file was already submitted by someone else and successfilly scanned, you’ll get back interesting results, the most known being the file score in the form “x/y”. The goal of my setup is to integrate virustotal.com within my ELK setup. To feed virustotal, hashes of interesting files must be computed. I’m getting interesting hashes via my Suricata IDS which inspect all the Internet traffic passing through my network.

The first step is to configure the MD5 hashes support in Suricata. The steps are described here. Suricata logs are processed by a Logstash forwarder and MD5 hashes are stored and indexed via the field ‘fileinfo.md5‘:

MD5 Hash

(Click to enlarge)

Note: It is mandatory to configure Suricata properly to extract files from network flows. Otherwise, the MD5 hashes won’t be correct. It’s like using a snaplen of ‘0’ with tcpdump. In Suricata, have a look at the inspected response body size for HTTP requests and the stream reassembly depth. This could also have an impact on performances, fine tune them to match your network behavior.

To integrate VirusTotal within ELK, a Logstash filter already exists, developed by Jason Kendall. The code is available on github.com. To install it, follow this procedure:

# cd /data/src
# git clone https://github.com/coolacid/logstash-filter-virustotal.git
# cd logstash-filter-virustotal
# gem2.0 build logstash-filter-awesome.gemspec
# cd /opt/logstash
# bin/plugin install /data/src/logstash-filter-virustotal/logstash-filter-virustotal-0.1.1.gem

Now, create a new filter which will call the plugin and restart Logstash.

filter {
    if ( [event_type] == "fileinfo" and
         [fileinfo][filename] =~ /(?i)\.(doc|pdf|zip|exe|dll|ps1|xls|ppt)/ ) {
        virustotal {
            apikey => '<put_your_vt_api_key_here>'
            field => '[fileinfo][md5]'
            lookup_type => 'hash'
            target => 'virustotal'
        }
    }
}

The filter above will query for the MD5 hash stored in ‘fileinfo.md5‘ to virustotal;com if the event contains file information generated by Suricata and if the filename contains an interesting extension. Of course, you can adapt the filter to your own environment and match only specific file format using ‘fileinfo.magic‘ or a minimum file size using ‘fileinfo.size‘. If conditions match a file, a query will be performed using the virustotal.com API and results stored into a new ‘virustotal‘ field:

VirusTotal Results

(Click to enlarge)

Now, it’s up to you to build your ElasticSearch queries and dashboard to detect suspicious activities in your network. During the implementation, I detected that too many requests sent in parallel to virustotal.com might freeze my Logstash (mine is 1.5.1). Also, keep an eye on your API key consumption to not break your request rate or daily/monthly quota.

The Rough Life of Defenders VS. Attackers

Scale of JusticeYesterday, It was the first time that I heard the expression “Social Engineering” in Belgian public media! If this topic came in the news, you can imagine that something weird (or juicy from a journalist perspective) happened. The Flemish administration had the good idea to test the resistance of their 15K officials against a phishing attack. As people remain the weakest link, it sounds a good initiative right? But if it was disclosed in the news, you can imagine that it was in fact … a flop! (The article is available here in French)

Read More →

Don’t (always) blame the user!

BlamingOften, as security professionals, we tend to blame our users. Not all people are security aware and take the right decision when facing a potential security issue. Yes, we know: they click, they open, they answer questions, they trust, …

Read More →

$HOME Sweet $HOME

Nest-ThreatYesterday, I talked at RMLL (“Rencontres Mondiales du Logiciel Libre“) or LSM in English (“Libre Sofware Meeting“) held in Beauvais, France. The presentation title was “$HOME Sweet $HOME” and covered the security of our home networks regarding the invasion of connected gadgets also known as the Internet of Things. I gave some tips & tricks to improve your security when you connect such devices on your network.

Read More →

BSidesLisbon 2015 Wrap-Up

BSidesLisbonHere is a quick wrap-up about the just-ended BSidesLisbon event. This is the second edition of this BSides event organized in Portugal. The philosophy of those events is well known: organized by and for the community, free, open and creating a lot of opportunities to meet peers. A classic but effective organization: talks, lightning talks, a CTF but two tracks in parallel. Here is a review of the ones I attended.

Read More →

Attackers Make Mistakes But SysAdmins Too!

Keep Calm & Avoid ErrorsA few weeks ago I blogged about “The Art of Logging” and explained why it is important to log efficiently to increase changes to catch malicious activities. They are other ways to catch bad guys, especially when they make errors, after all they are humans too! But it goes the other way around too with system administrators. Last week, a customer asked me to investigate a suspicious alert reported by an IDS. It looked like an restricted web server (read: which was not supposed to be publicly available!) was hit by an attack coming from the wild Internet.

Read More →

BSidesLondon 2015 Wrap-Up

BSidesLondon-logoHere is a quick wrap-up of the just finished BSidesLondon. It was already the 5th edition (and my 5th participation!). This year, they moved to a new location close to Earls Court where is organized InfoSec Europe at the same time, good idea for those who want to attend both worlds: Hackers wearing t-shirts VS. Vendors wearing ties! This year, it was just a one-day journey for me, I’m writing this blog post while waiting for my train back to Belgium.

Read More →

Playing with IP Reputation with Dshield & OSSEC

Reputation[This blogpost has also been published as a guest diary on isc.sans.org]

When investigating incidents or searching for malicious activity in your logs, IP reputation is a nice way to increase the reliability of generated alerts. It can help to prioritize incidents. Let’s take an example with a WordPress blog. It will, sooner or later, be targeted by a brute-force attack on the default /wp-admin page. In this case, IP reputation can be helpful: An attack performed from an IP address reported as actively scanning the Internet will not (or less) attract my attention. On the contrary, if the same kind of attack is coming from an unkown IP address, this could be more suspicious…

Read More →

HITB Amsterdam Wrap-Up Day #2

HITB GoodiesI left Amsterdam after the closing keynote and I just arrived at home. This is my quick wrap-up for the second day of Hack in the Box!

The second keynote was presented by John Matherly: “The return of the Dragons”. John is the guy behind Shodan, the popular devices search engine. Shodan started because Nmap was not designed to scan the whole Internet. With Shodan, Stateless scanning is used. Today, technologies allow to scan one specific port on the Internet in less time than making a coffee. Shodan allows security researchers to request some specific scan on their projects. If the search engine is of course used by attackers to perform reconnaissance phases, it can also be used for many other purposes like:

  • You can track if a US company is selling devices to prohibited areas
  • You can track the impact of a patch which screws up a vendor infrastructure
  • Same for malware analysis (bots, C&C)
  • Security rating of a company that you want to buy? Can be used to create “security ratings”.
  • Market share
  • Do we need to push sales in a specific geographic area?
  • IoT (machine 2 machine communications), printers, wind turbines, …

John on stage

John insisted in the IoT hot topic with many examples. Such devices tend to be connected on mobile networks. It makes them very difficult to contact the owner to say “Fix your shit”. Many IoT devices also completely ignore our privacy and can expose sensitive data about our life, health. If you’re interested in IoT security, have a look at the iotdb on github.com, another project maintained by Shodan (Internet of Things nmap fingerprints). For sure, IoT are a pain and they will enforce their position of major threat in the near future.

After the first coffee break, I attended a talk by Filippo Valsorda and George Tankersley about Tor: “Non-hidden hidden services considered harmful: Attacks and detection”. Everybody knows Tor but what about the underlying protocol? Hidden services is a Tor feature which provides bidirectional anonymity and support generic services. They are identified with the .onion TLD. Well known example are drug markets ;-).

Filippo & George on stage

Services are published in a database that is a DHT (Distributed Hash Table) and the question is: How do we choose where to publish this database? The speakers explained the process of HSDir selection. Why is it critical? Hidden services users are facing a greater risks of de-anonymisation that regular Tor users. See the published slides for the details. The problem is low-latency implies correlation attacks (worried about entry and exit nodes). You can of course run your own HSDir and it will receive a subset of hidden service look-ups from users (after a few days). This will help in mapping relative popular hidden services. They explained how useful information can be collected by running your own HSDir (you can trust it if you run it yourself). Finally, as a conclusion, they presented the tool that developed to analyse Tor HSDirs (available here). Nice presentation but, if you are interesting in this topic, I recommend you to read the slides which contains a lot of details.

Then, back in track one, Lucas de Fulgentis presented “The Windows Phone Freakshow”. His research was based on the following process: installation of apps, extraction of binaries, decompilation and review. Guess what? He discovered lot of bad stuff. Windows phone apps can be bases on two technologies: SilverLight or Windows Runtime. The result of his research is some kind of catalog of bad API usages that he called “freaks”. A freak is just an example of vulnerable code.

Lucas on stage

The first freak presented was: self-defending apps. A lack of binary protection results in an app that can be analysed, reverse-engineered and modified. This is possible due to a lack of anti-debugging, obfuscation and resources integrity (+/-95% of tested apps lack of proper binary protection). There are tools like dofFuscator that helps to protest binaries. Keep in mind that binary protections simply mitigate but do not solve binary attacks. The next freak: data transport security. It concerns the transmission of sensitive data over unencrypted channels. Common issues are: No HTTPS and vulnerable to Mitm. As example, Lucas demonstrated how to abuse a vulnerable CNN application by changing or adding a breaking news. More applications are vulnerable: LinkedIn, eBay, e-banking applications. A good tip is to use SSL everywhere.  Windows Phone 8 automatically discard invalid certificates. But if the CA is compromized and the attacker generates malicious cerfificates, the problem remains! To protect against this, implement certificate pinning. Next freak; data storage security. WP8 supports BitLocker (AES128) but… it is disabled by default (I t can be enable via an ActiveSync policy). The device physical memory attacks allow access to some content. Do we really need sensitive data on our mobile device? Data can be stored in 3 locations: File system, external SD card or in the cloud. A common issue with credentials? There are stored in plain text! Use the DataProtection API. There is also a PasswordVault class available. The other freaks reviewed by Lucas: data leakage (involuntary data disclosure due to a side effect, authentication and authorisation and, last but not least, client side injection. For this point, a key point is to never trust data from the wild. “All input is evil” as said Lucas. Using an undocumented IPC call (Shell_PostMessageToast), Lucas demonstrated how to bypass the 4-digits PIN code protection in the DropBox application. Being a contributor to the OWASP Mobile Security Project, Lucas demonstrated the vulnerabilities and also the countermeasures that are available for the developers. Very nice talk!

After a good lunch and interesting discussions with peers, the last half-day started with Julien Vehent from Mozzila who presented “Mozilla Investigator: Distributed and real-time digital forensics at the speed of the cloud”. MIG = “Mozilla InvestiGator”. Mozilla has approx 15000 servers.

Mozilla InvestiGator

It’s a challenge to monitor this and perform security operations. Julien explained what IOC’s are (“Indicator of Compromise”) like IP addresses, filenames, hashes, URLs, domains, … ). The good point, they can be shared! A security incident can be caused by internal mistakes. Did you know the unfortunate commit?

$ git commit -a . &&& git push github master

Password or credentials can be leaked in this case. How to know the impact on the completed infrastructure based on thousands of server? Mozilla’s goal is to build a safer Internet with a strong startup/incubator mindeset. As said Julien, they experiment and fail fast. The current infrastructure is based on 400+ active websites, a dozen of offices, remote workers, 2 data centers, tons of AWS accounts. How to do incident response in such environemnet (it’s like looking for a needle). For Julioen, the  OpSec problems are:

  • Visibility
  • Reachability
  • Heterogeneity

The faster we can investigate, more we will find but Mozilla followed this principle: “If you can’t find the right tool, write your own”. Julien demonstrate some usage of MIG.

Example 1: Locating a cron job that contains a password:

$ mig file -e 30s -path /etc/cron.d -content “badpassword”

Example 2: Is that bonet IP connected anywhere?

$ mig netstat -e 20s -ci 78.204.52.20

Each server is running an agent connected to a RabbitMQ relay. MIG is light, written in Go, one static bin, config is pushed by tools like Puppet. Julien reviewed all the securty controls in place to let the agent work safely:  Intensive use of PGP, no raw data returned to the server, ACLs. It looks to be a very powerful tool! MIG is available here.

The next talk is the one that everybody expects at security conference: It focus on infrastructures that everybody used in his/her day life: Parking systems. Jose Antion Guasch started this research a few years ago during his spare time and fully compromised the parking system of a well know vendor (not disclosed). His talk title was “Yes Parking: Remotely Owning “Secure” Parking Systems”’.

Jose on stage

This time, no SCADA devices, no complex technical skills but classic vulnerabilities like we can find on plenty of vulnerable web servers. But focus on public parkings? They have complex infrastructures and there is money involved! (a lot of!). For Jose, a kid’s dream was to play with the parking gates just by clicking on a button and he succeeded. Every started with a simple login page. The rest of the research was based on the same structure as a classic pentest: information gathering, enumeration, analysts, exploitation, etc. A parking system is based on main components:

  • The entrance (gate)
  • A management / maintenance / payement system
  • An exit (another gate)

Behind those components, we find dump personal computers connected to a network. Jose analysed different versions of the application (which changed after the original company was taken over by another one. For the old version, it was piece of cake. To resume: Indexable directory on the management server, a directory /backup with database dumps, restore the last one, get credentials and use them. Game over! For the other versions, it was a little bit more complex but the systems got also pwned. There, Jose found credit card numbers and was able to play with the gates, record new notification messages. But a real attacker could be much more evil by selling back monthly tickets, CC, monitoring people, controlling license plates.

Old version: Directory index -> /backup directory -> database dump -> restore the last one -> Get credentials -> game over

The newest version was more difficult to pwn but it remained vulnerable… Bad passwords (again), credit card numbers in clear text, hability to record new voice messages, play with gates, … Of course, a real attacker will play more evil and resell monthly tickets, lists of CC, monitor people, control license plates, …  Jose’s advice to the application owners: Don’t put anything online, harden, don’t use default config and restrict accesses!

The last talk was presented by Marina Krotofil and Jason Larson: “What you always wanted  and now can: Hacking chemical processes”. They talked about Industrial Control Systems. They are “IT” systems implemented in “physical” systems. Components convert analog to digital signals (ex: temperature, pressure). In traditional security, TLS is the xavier of all things. But different in industrial processes, the process can’t stop due to an IT issue. Can you imagine: crypto key invalid => stop the process? Industrial systems can be controlled without modifiing the content of the message!

Marina & Jason on stage

After a SCACA 101 5 mins sessions (always useful), they came back to process manipulation. In some process, the freshness of data is critical. They reviewed different types of attacks and their impact on a plant.

And for the closing keynote, the word was given to Runa A. Sandvick. She talked about “Bringing security and privacy to where the wild thing are”. Runa’s talk was addressing the securiy communitty but without technical content, only nice facts and ideas. She worked on projects like Tor and the Freedom of the Press association.

Runa on stage

What’s the link with IoT and mobile phones? Back in 2010, we discovered that apps are leaking data to external services (ex: the Pandora music app). Another example, a riffle running Linux and connecting to apps… Drones with a mobile phone and Wireless toys, etc. Keep in mind: “Just because you can does not mean you should”. So what do we do it? Exciting? Innovation? Managing expectation. Nobody takes care of updates on mobile phones. What if the OS reaches its end of life? No more updates? From a baseline and building on it. Baseline changes in time… Today the new baseline has many more options (example was given with cars). Why don’t we have the same reflex with electronic devices? “NSA-proof” is a term created by journalists. What does it mean really? We, as community, can help journalists to help to better understand. But hackers also do mistakes with logos, brands, … Think about the latest one, VENOW… We can do better! Security vs privacy. New roles are coming which replace the workd security by privacy: privacy analysts, privacy engineers, … We need more transparency but often, there is a lack of context. Example: the screen with app permissions on Android. Facebook needs to access the microphone? But in which context?  Where do we go from here? How to educate people? Be responsible (story of Chris Roberts who hacked a plane). Two interesting initiatives mentioned by Runa: I’m The Cavalry and BuiltItSecure.ly. I really liked her keynote which contained interesting ideas to keep in mind and to share!

All the conferences material is already online, I suggest you to download the slides and read them, great content! Next step for me, BSidesLondon next Wednesday!

HITB Amsterdam Wrap-Up Day #1

HITB Track 1The HITB crew is back in the beautiful city of Amsterdam for a new edition of their security conference. Here is my wrap-up for the first day!

Read More →