Detecting consumer decisions within messy data

Millions of people each month report positive and negative health care feedback across the Web. Some jump into forums to complain about ineffective prescriptions or to discuss which drugs are best to treat illnesses. Others take to blogs to describe symptoms and how to get relief.

MIT spinout dMetrics believes this online chatter is an information treasure-trove for the health care industry. “In health care, there’s this gigantic world of unstructured data that needs to be translated into useable information,” says Paul Nemirovsky PhD ’06, who co-founded dMetrics with Ariadna Quattoni PhD ’09.

The startup has developed a platform called DecisionEngine that uses machine learning and natural language processing — which helps computers better understand human speech — to mine billions of conversations about drugs, medical devices, and other health care products. These discussions are happening on blogs, Facebook, Twitter, forums, and even in comments accompanying news articles and videos.

From those vast stores of messy data, the software reveals insights into consumer decisions, Nemirovsky says: “What people do, don’t do, consider doing, may do, did in the past, as well as what needs, fears, and hopes they have.”

Today, Nemirovsky explains, dMetrics has a database that includes every public comment about patient-reported illnesses, solutions, and outcomes, pulled from more than 1 million online sources. This includes information on more than 14,000 health care products.

Clients, including Fortune 500 companies and nonprofit organizations, can use dMetrics software to answer specific questions, such as how many patients used a specific medication for a particular reason in certain time frame, or which customers are considering switching from their drug to a competitor’s drug.

Although focusing on the health care industry, dMetrics, headquartered in Brooklyn, New York, is also trialing its platform with consumer finance and political organizations. Credit card companies, for instance, can analyze why consumers favor specific credit cards over others. Political scientists could use the software to determine which issues people care about and how strongly they stand behind their opinions.

“For all these types of questions, you have to understand not only the words people use but the concepts behind the words,” Nemirovsky says.

Decoding language and expression

Other software generally relies on ontologies — formal naming and definitions — to sense overall sentiment and popularity of brands, Nemirovsky says. The software may count, for example, the number of mentions of a word (such as the name of a specific drug) to determine if it’s important, or it may detect “positive” or “negative” words.

“Language and expression doesn’t work like that,” Nemirovsky says. “We’re a bit more complex as humans.”

DecisionEngine, Nemirovsky says, better derives meaning from text because the software — which now consists of around 2 million lines of code — is consistently trained to recognize various words and synonyms, and to interpret syntax and semantics. “Online text is incredibly tough to analyze properly,” he says. “There’s slang, misspellings, run-on sentences, and crazy punctuation. Discussion is messy.”

Visualize the software as a three-tiered funnel, Nemirovsky suggests, with more refined analysis happening as the funnel gets narrower. At the top of the funnel, the software mines all mentions of a particular word or phrase associated with a certain health care product, while filtering out “noise” such as fake websites and users, or spam. The next level down involves separating out commenters’ personal experiences over, say, marketing materials and news. The bottom level determines people’s decisions and responses, such as starting to use a product — or even considering doing so, experiencing fear or confusion, or switching to a different medication.

To explain, Nemirovsky provides an example comment that could appear in an online forum: “I'm now on Drug A and took 10 mgs of Drug B, and it seemed to sync well. I'm seeing my doc tomorrow to ask about adding Drug C to my current meds. For me personally Drug A is a very tricky drug, only helpful if I'm getting good sleep, eat and exercise well and limit the use to couple times a week.”

Other software, he says, may only detect positive and negative words (such as “well” and “good” versus “tricky” and “limit”). DecisionEngine, on the other hand, would identify many more pieces of information, including the use and effectiveness of Drugs A and B combined; the dosage of Drug B; consideration for adopting Drug C; potential dissatisfaction with Drug A, depending on lifestyle choices such as “getting good sleep”; the commenter’s use of three concurrent medications; and plans of visiting a health care professional.

These insights allow clients to take action, Nemirovsky says. If consumers are planning to switch drugs, for instance, a pharmaceutical firm may want to ensure that the consumers are using their products properly, and to find a means to address any issues.

Recently, Nemirovsky says, a pharmaceutical firm used DecisionEngine to determine if an allergy medication had improved the quality of life for a subgroup of patients. Analyzing specific issues associated with the subgroup, the firm discovered that the drug had an outsized positive impact, more so than several competing brands. The firm used the results in a regulatory submission — a critical stage in bringing any health care product to market. “It’s rare for the regulatory authorities to consider online patient reports as part of the regulatory approval process,” Nemirovsky says.

Everyone’s an expert

In the late 2000s at MIT, Nemirovsky, who was an MIT Media Lab graduate student, and Quattoni, who was studying at the Computer Science and Artificial Intelligence Laboratory (CSAIL), came together with a lofty goal: Use big data to make everyone experts.

The plan was to combine machine learning with natural language processing to decode mountains of unstructured data and provide pertinent information, about anything, to anyone who wanted. “If you give people the right information, at the right time, anyone can be an expert,” Nemirovsky says.

In building the software, they discovered that an important topic for most people on a daily basis is health care. “Patients go to the doctor with complex conditions, and sometimes they leave with less certainty they had before,” Nemirovsky says. “Then they go online and say, ‘What on Earth is going on? What do I do?’”

Focusing on the health care industry, they turned to MIT’s Venture Mentoring Service, which helped them navigate various startup issues: fundraising, operations, marketing, legal issues, and other things. “Things that sound obvious now, were not obvious to us at all,” Nemirovsky says. “We were helped a lot by the VMS, especially as first-time entrepreneurs.”

Soon after Nemirovsky graduated, he and Quattoni launched dMetrics in Boston, before relocating to Brooklyn. Over the years, the startup expanded from two to 16 employees — whose machine learning and natural language processing research has been cited in more than 4,500 academic journals total — and earned four National Science Foundation grants to develop its technology.

Moving forward, dMetrics aims to bring its software to more sectors than health care, politics, and consumer finance, with aims of empowering everyone with data. In that way, Nemirovsky says, the dMetrics mission hasn’t changed much from its early MIT days: “It’s our vision that we need to open means of expertise to everyone.”

Nanodevices at one-hundredth the cost

Microelectromechanical systems — or MEMS — were a $12 billion business in 2014. But that market is dominated by just a handful of devices, such as the accelerometers that reorient the screens of most smartphones.

That’s because manufacturing MEMS has traditionally required sophisticated semiconductor fabrication facilities, which cost tens of millions of dollars to build. Potentially useful MEMS have languished in development because they don’t have markets large enough to justify the initial capital investment in production.

Two recent papers from researchers at MIT’s Microsystems Technologies Laboratories offer hope that that might change. In one, the researchers show that a MEMS-based gas sensor manufactured with a desktop device performs at least as well as commercial sensors built at conventional production facilities.

In the other paper, they show that the central component of the desktop fabrication device can itself be built with a 3-D printer. Together, the papers suggest that a widely used type of MEMS gas sensor could be produced at one-hundredth the cost with no loss of quality.

The researchers’ fabrication device sidesteps many of the requirements that make conventional MEMS manufacture expensive. “The additive manufacturing we’re doing is based on low temperature and no vacuum,” says Luis Fernando Velásquez-García, a principal research scientist in MIT’s Microsystems Technology Laboratories and senior author on both papers. “The highest temperature we’ve used is probably 60 degrees Celsius. In a chip, you probably need to grow oxide, which grows at around 1,000 degrees Celsius. And in many cases the reactors require these high vacuums to prevent contamination. We also make the devices very quickly. The devices we reported are made in a matter of hours from beginning to end.”

Welcome resistance

For years, Velásquez-García has been researching manufacturing techniques that involve dense arrays of emitters that eject microscopic streams of fluid when subjected to strong electric fields. For the gas sensors, Velásquez-García and Anthony Taylor, a visiting researcher from the British company Edwards Vacuum, use so-called “internally fed emitters.” These are emitters with cylindrical bores that allow fluid to pass through them.

In this case, the fluid contained tiny flakes of graphene oxide. Discovered in 2004, graphene is an atom-thick form of carbon with remarkable electrical properties. Velásquez-García and Taylor used their emitters to spray the fluid in a prescribed pattern on a silicon substrate. The fluid quickly evaporated, leaving a coating of graphene oxide flakes only a few tens of nanometers thick.

The flakes are so thin that interaction with gas molecules changes their resistance in a measurable way, making them useful for sensing. “We ran the gas sensors head to head with a commercial product that cost hundreds of dollars,” Velásquez-García says. “What we showed is that they are as precise, and they are faster. We make at a very low cost — probably cents — something that works as well as or better than the commercial counterparts.”

To produce those sensors, Velásquez-García and Taylor used electrospray emitters that had been built using conventional processes. However, in the December issue of the Journal of Microelectromechanical Systems, Velásquez-García reports using an affordable, high-quality 3-D printer to produce plastic electrospray emitters whose size and performance match those of the emitters that yielded the gas sensors.

Made to order

In addition to making electrospray devices more cost-effective, Velásquez-García says, 3-D printing also makes it easier to customize them for particular applications. “When we started designing them, we didn’t know anything,” Velásquez-García says. “But at the end of the week, we had maybe 15 generations of devices, where each design worked better than the previous versions.”

Indeed, Velásquez-García says, the advantages of electrospray are not so much in enabling existing MEMS devices to be made more cheaply as in enabling wholly new devices. Besides making small-market MEMS products cost-effective, electrospray could enable products incompatible with existing manufacturing techniques.

“In some cases, MEMS manufacturers have to compromise between what they intended to make, based on the models, and what you can make based on the microfabrication techniques,” Velásquez-García says. “Only a few devices that fit into the description of having large markets and not having subpar performance are the ones that have made it.”

Electrospray could also lead to novel biological sensors, Velásquez-García says. “It allows us to deposit materials that would not be compatible with high-temperature semiconductor manufacturing, like biological molecules,” he says.

“For sure, the paper opens new technical paths to making gas microsensors,” says Jan Dziuban, head of the Division of Microengineering at Wroclaw University of Technology in Poland. “From a technical point of view, the process may be easily adapted to mass fabrication.”

“But promising results must be proved statistically,” he cautions. “Personal experience tells me that plenty of very promising materials for new sensors, utilizing nanostructured materials, which have been published in high-level scientific papers, haven't resulted in reliable products.”

Deep-learning algorithm predicts photos’ memorability at “near-human” levels

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have created an algorithm that can predict how memorable or forgettable an image is almost as accurately as humans — and they plan to turn it into an app that subtly tweaks photos to make them more memorable.

For each photo, the “MemNet” algorithm — which you can try out online by uploading your own photos — also creates a heat map that identifies exactly which parts of the image are most memorable.

“Understanding memorability can help us make systems to capture the most important information, or, conversely, to store information that humans will most likely forget,” says CSAIL graduate student Aditya Khosla, who was lead author on a related paper. “It’s like having an instant focus group that tells you how likely it is that someone will remember a visual message.”

Team members picture a variety of potential applications, from improving the content of ads and social media posts, to developing more effective teaching resources, to creating your own personal “health-assistant” device to help you remember things.

Part of the project the team has also published the world’s largest image-memorability dataset, LaMem. With 60,000 images, each annotated with detailed metadata about qualities such as popularity and emotional impact, LaMem is the team’s effort to spur further research on what they say has often been an under-studied topic in computer vision.

The paper was co-written by CSAIL graduate student Akhil Raju, Professor Antonio Torralba, and principal research scientist Aude Oliva, who serves as senior investigator of the work. Khosla will present the paper in Chile this week at the International Conference on Computer Vision.

How it works

The team previously developed a similar algorithm for facial memorability. What’s notable about the new one, besides the fact that it can now perform at near-human levels, is that it uses techniques from “deep-learning,” a field of artificial intelligence that use systems called “neural networks” to teach computers to sift through massive amounts of data to find patterns all on their own.

Such techniques are what drive Apple’s Siri, Google’s auto-complete, and Facebook’s photo-tagging, and what have spurred these tech giants to spend hundreds of millions of dollars on deep-learning startups.

“While deep-learning has propelled much progress in object recognition and scene understanding, predicting human memory has often been viewed as a higher-level cognitive process that computer scientists will never be able to tackle,” Oliva says. “Well, we can, and we did!”

Neural networks work to correlate data without any human guidance on what the underlying causes or correlations might be. They are organized in layers of processing units that each perform random computations on the data in succession. As the network receives more data, it readjusts to produce more accurate predictions.

The team fed its algorithm tens of thousands of images from several different datasets, including LaMem and the scene-oriented SUN and Places (all of which were developed at CSAIL). The images had each received a “memorability score” based on the ability of human subjects to remember them in online experiments.

The team then pitted its algorithm against human subjects by having the model predicting how memorable a group of people would find a new never-before-seen image. It performed 30 percent better than existing algorithms and was within a few percentage points of the average human performance.

For each image, the algorithm produces a heat map showing which parts of the image are most memorable. By emphasizing different regions, they can potentially increase the image’s memorability.

“CSAIL researchers have done such manipulations with faces, but I’m impressed that they have been able to extend it to generic images,” says Alexei Efros, an associate professor of computer science at the University of California at Berkeley. “While you can somewhat easily change the appearance of a face by, say, making it more ‘smiley,’ it is significantly harder to generalize about all image types.”

Looking ahead

The research also unexpectedly shed light on the nature of human memory. Khosla says he had wondered whether human subjects would remember everything if they were shown only the most memorable images.

“You might expect that people will acclimate and forget as many things as they did before, but our research suggests otherwise,” he says. “This means that we could potentially improve people’s memory if we present them with memorable images.”

The team next plans to try to update the system to be able to predict the memory of a specific person, as well as to better tailor it for individual “expert industries” such as retail clothing and logo design.

“This sort of research gives us a better understanding of the visual information that people pay attention to,” Efros says. “For marketers, movie-makers and other content creators, being able to model your mental state as you look at something is an exciting new direction to explore.”

The work is supported by grants from the National Science Foundation, as well as the McGovern Institute Neurotechnology Program, the MIT Big Data Initiative at CSAIL, research awards from Google and Xerox, and a hardware donation from Nvidia.

There’s an app for that: an easy, fast and reliable way to record causes of death

Researchers have developed a revolutionary new app to capture accurate global cause of death data on tablets and mobile phones.

Worldwide, two in three deaths – 35 million each year – are unregistered. Around 180 countries that are home to 80 per cent of the world’s population do not collect reliable cause of death statistics.

The app is the result of a decade-long global collaboration, led by the University of Melbourne and researchers at the Institute for Health Metrics and Evaluation (IHME) at the University of Washington.

A new paper, published today in BMC Medicine, explains the process behind the app. The research team redesigned a short ‘verbal autopsy’ questionnaire and tested it in India, the Philippines, Mexico, and Tanzania. The app was then field-tested in China, Sri Lanka and Papua New Guinea.

Family members of the deceased were given surveys in hand-held devices. A computer then analysed the data to make a diagnosis, bypassing the need to rely on doctors to do this work.

University of Melbourne Laureate Professor Alan Lopez led the study. 

Prof Lopez said in the age of big data, we still know next to nothing about what kills people in poor countries.

“Without accurate cause of death information, we can’t monitor disease and injury trends, we can’t keep track of emerging health problems and we don’t have any markers to show us whether programs and policies are actually working.

“So if you live in a country where no-one is dying from malaria, then why are you pouring money into malaria-prevention programs? And conversely, if people are dying from lung cancer, why aren’t you investing in tobacco control?

“Up-to-date, reliable information on what people are dying from and at what age, is really important for policies to prevent premature death. Our app provides a way to do this, quickly, simply, cheaply and effectively, in real time, with the power of technology.” 

IHME Director Dr Christopher Murray added: “Verbal autopsy research has shown that computer models are just as accurate as physicians in making diagnoses based on verbal autopsy data, at a fraction of the cost.

“In countries with scarce data on causes of death, policymakers need this information to better understand local disease burden and effectively allocate resources.”

The problem in many regions around the world is that only registered doctors are qualified to determine a cause of death, but the process is expensive, time-consuming and can be unreliable.

Computers can reliably provide a diagnosis by linking symptoms with a specific cause of death in real-time. The instant provision of information overcomes what can be a 10-year lag between the death and the doctor’s report. 

“Relying on doctors to collect information about causes of death in rural populations is not helpful,” Prof Lopez said.

“Our method involves data collection by health workers, registrars and village officials, who use the app to administer the surveys.

“The data is fed into a computer, which makes a diagnosis. It requires very minimal training. This way doctors are free to do what they do best, which is providing essential medical care to their communities. 

“Governments now have a way to gather data to inform their health policies, that costs nothing and can be provided in real time. Even if you’re sitting out in the remote bush in Africa and you can do this. Anywhere you’ve got power, it’s possible.”

Study finds altered brain chemistry in people with autism

Neuroscientists link autism to reduced activity of key neurotransmitter in human brain. 

MIT and Harvard University neuroscientists have found a link between a behavioral symptom of autism and reduced activity of a neurotransmitter whose job is to dampen neuron excitation. The findings suggest that drugs that boost the action of this neurotransmitter, known as GABA, may improve some of the symptoms of autism, the researchers say.

Brain activity is controlled by a constant interplay of inhibition and excitation, which is mediated by different neurotransmitters. GABA is one of the most important inhibitory neurotransmitters, and studies of animals with autism-like symptoms have found reduced GABA activity in the brain. However, until now, there has been no direct evidence for such a link in humans.

“This is the first connection in humans between a neurotransmitter in the brain and an autistic behavioral symptom,” says Caroline Robertson, a postdoc at MIT’s McGovern Institute for Brain Research and a junior fellow of the Harvard Society of Fellows. “It’s possible that increasing GABA would help to ameliorate some of the symptoms of autism, but more work needs to be done.”

Robertson is the lead author of the study, which appears in the Dec. 17 online edition ofCurrent Biology. The paper’s senior author is Nancy Kanwisher, the Walter A. Rosenblith Professor of Brain and Cognitive Sciences and a member of the McGovern Institute. Eva-Maria Ratai, an assistant professor of radiology at Massachusetts General Hospital, also contributed to the research.

Too little inhibition

Many symptoms of autism arise from hypersensitivity to sensory input. For example, children with autism are often very sensitive to things that wouldn’t bother other children as much, such as someone talking elsewhere in the room, or a scratchy sweater. Scientists have speculated that reduced brain inhibition might underlie this hypersensitivity by making it harder to tune out distracting sensations.

In this study, the researchers explored a visual task known as binocular rivalry, which requires brain inhibition and has been shown to be more difficult for people with autism. During the task, researchers show each participant two different images, one to each eye. To see the images, the brain must switch back and forth between input from the right and left eyes.

For the participant, it looks as though the two images are fading in and out, as input from each eye takes its turn inhibiting the input coming in from the other eye.

“Everybody has a different rate at which the brain naturally oscillates between these two images, and that rate is thought to map onto the strength of the inhibitory circuitry between these two populations of cells,” Robertson says.

She found that nonautistic adults switched back and forth between the images nine times per minute, on average, and one of the images fully suppressed the other about 70 percent of the time. However, autistic adults switched back and forth only half as often as nonautistic subjects, and one of the images fully suppressed the other only about 50 percent of the time.

Performance on this task was also linked to patients’ scores on a clinical evaluation of communication and social interaction used to diagnose autism: Worse symptoms correlated with weaker inhibition during the visual task.

The researchers then measured GABA activity using a technique known as magnetic resonance spectroscopy, as autistic and typical subjects performed the binocular rivalry task. In nonautistic participants, higher levels of GABA correlated with a better ability to suppress the nondominant image. But in autistic subjects, there was no relationship between performance and GABA levels. This suggests that GABA is present in the brain but is not performing its usual function in autistic individuals, Robertson says.

“GABA is not reduced in the autistic brain, but the action of this inhibitory pathway is reduced,” she says. “The next step is figuring out which part of the pathway is disrupted.”

“This is a really great piece of work,” says Richard Edden, an associate professor of radiology at the Johns Hopkins University School of Medicine. “The role of inhibitory dysfunction in autism is strongly debated, with different camps arguing for elevated and reduced inhibition. This kind of study, which seeks to relate measures of inhibition directly to quantitative measures of function, is what we really to need to tease things out.”

Early diagnosis

In addition to offering a possible new drug target, the new finding may also help researchers develop better diagnostic tools for autism, which is now diagnosed by evaluating children’s social interactions. To that end, Robertson is investigating the possibility of using EEG scans to measure brain responses during the binocular rivalry task.

“If autism does trace back on some level to circuitry differences that affect the visual cortex, you can measure those things in a kid who’s even nonverbal, as long as he can see,” she says. “We’d like it to move toward being useful for early diagnostic screenings.”