6 Post Tagged as data

What's in a Web Crawl?

Since releasing our host attribute dataset (pairs, components, trackers), we’ve gotten a lot of great feedback from our community. Users are reporting faster investigation times, more substantial connections and new research leads they wouldn’t have found otherwise. While these datasets are great, they are only a fraction of the data RiskIQ stores on a daily basis. What makes RiskIQ’s web crawling technology powerful is that it’s not just a simulation, it’

read more

Hashes or it Didn't Happen

If you’ve been in the trenches of security research, you may be familiar with the phrase, “hashes or it didn’t happen”. It’s a testament to the importance of having malware when conducting an investigation and it’s something PassiveTotal has historically lacked inside the platform. Our focus has always been to provide the most comprehensive infrastructure solution while working with companies dedicated to the processing malware to fill our gaps. Starting today,

read more

Derived Host Pairs from Web Crawling

Did you realize that in loading this blog post, your web browser made over 50 network requests for resources in order to construct it? The modern web is a complex graph of dependent requests made up of images, code libraries, page content and other references. Every day, RiskIQ’s crawling technology makes nearly 2 billion HTTP requests online and saves the contents of the session inside of a database. Using years of this data, engineers

read more

Surfacing Infrastructure with Trackers

Imagine this -- you’re looking to harvest a bunch of cloud-storage user credentials through a phishing campaign. Your email is perfect, domains registered and now all you have left to do is make a copy of your target’s web page, modify the POST script and hit send. Within minutes, you are collected credentials and your campaign is a success.

This process may sound complicated, but it’s not. Attackers can easily copy your

read more

New API, New Data

Building a scalable API means having a deep understanding of your users' usage patterns. Over the past few months, we have made significant changes to our application architecture to meet our communities needs and as a final piece to the puzzle, we are releasing a new foundation for all our future APIs. We recognize updating code to use a new API is never fun, so we tried to make it easier by building an extensive

read more