I develop scalable machine learning techniques that help us understand and predict emergent global processes,
such as disease epidemics
and crowd package delivery.
The goal is to model human behavior from our day-to-day interactions recorded online, rather than relying on surveys.
For example, we have shown that from this real-time data, we can calculate the likelihood that you will catch a cold tomorrow on the basis of who you met and what they recently tweeted about.
Working with Henry Kautz,
we found that organic sensor networks—made up of people running around with their smartphones and occasionally posting to online social media—
are an important complement to the explicit social networks we create manually. Location plays a key role here.
Applying a combination of machine learning and human computation, we have leveraged organic sensor networks to predict the
spread of flu between specific individuals,
and to identify restaurants with a high risk of food poisoning (project nEmesis).
Given that three of your friends have flu-like symptoms, and that you have recently met eight people, possibly strangers, who complained about having runny noses and headaches, what is the probability that you will soon become ill as well? Our models enable you to see the spread of infectious diseases, such as flu, throughout a real-life population observed through online social media.
We apply machine learning and natural language understanding techniques to determine the health state of Twitter users at any given time. Since a large fraction of tweets is geo-tagged, we can plot them on a map, and observe how sick and healthy people interact. Our model then predicts if and when an individual will fall ill with high accuracy, thereby improving our understanding of the emergence of global epidemics from people's day-to-day interactions.
This video interview provides a quick overview of our work in the health space:
The following video shows a heatmap visualization of the prevalence of flu in New York City, as observed through public Twitter data. The more red an area is, the more people are afflicted by flu at that location. We show emergent aggregate patterns in real-time, with second-by-second resolution. By contrast, previous state-of-the-art methods (including Google Flu Trends and government data) entail time lags from days to weeks.
The fine-grained epidemiological models we show here are just one instance of the general class of problems that our system solves. Other domains include understanding public sentiment, the diffusion of information throughout a population, and predicting customer behavior.
By augmenting existing datasets with real-time insights and cues from social media, we are able to connect the dots, visualize patterns, and refine models based on user feedback.
Learning about Human Behavior from Social Media
Predicting your friends from where you go and what you say
Visualization of the movement and interactions of sampled cliques of friends who
geo-tag their posts on Twitter in New York City:
Contact network of a sample of Twitter users in New York City. Each node is a unique user and an edge denotes at least one physical meeting (based on users' GPS). The width of an edge is proportional to the number of encounters the two individuals had. Bold links show users who not only met, but are also friends (mutual followers). Individuals who at some point in time indicated increased level of sickness we labeled red.
Heatmap of Twitter activity in New York City
Location-Based Modeling of Complex Multi-Agent Activities