Debate on Human Longevity’s WGS Trait Prediction Publication Rages On

September 13, 2017
Debate on Human Longevity’s WGS Trait Prediction Publication Rages On
Source: the-lightwriter/Getty Images

Christina Bennett

A controversial study published last week in the Proceedings of the National Academy of Sciences (PNAS) by researchers from Human Longevity Inc. (HLI), the San Diego company cofounded by J. Criag Venter, which said they could scour an individual’s genome sequence and predict a handful of physical traits—including facial structure—has continued to spark widespread debate.

The major headline was that HLI researchers claimed they can identify a person from their genome sequence—boldly raising contentious issues around genomic privacy and forensic DNA profiling.

No sooner had the embargo lifted on Venter’s article than the backlash started, with several geneticists—including a former HLI staffer and coauthor, and a reviewer of an earlier version of the manuscript—disputing various claims.

Clouded Context

The paper was eventually published on September 5, 2017 in PNAS, but this was not Venter’s first choice. Science—where Venter published the original Celera human genome report in February 2001—was, but the publication rejected it following criticism from reviewers including Yaniv Erlich, Ph.D., an expert on genome deidentification at Columbia University. Dr. Erlich is also chief scientific officer at MyHeritage.

Dr. Venter turned to PNAS: As a member of the National Academy of Sciences (NAS), Venter had access to a privileged publishing track. PNAS has two publishing mechanisms: contributed submissions and direct submissions. Direct submissions, which make up the bulk of published papers, are the standard route for submitting to PNAS. Competition is naturally fierce—the PNAS office receives around 50 papers a day submitted via this track, ultimately accepting between 16% to 19% of them.

Contributed submissions, on the other hand, are open only to NAS members. Moreover, contributors can handpick their reviewers (unlike most journals, which use a blinded peer-review approach to review submissions).

PNAS Editor-in-Chief Inder M. Verma has acknowledged that the contributed track is not without controversy, writing in 2015 that the track is a “source of debate with NAS members and nonmembers alike.” To dampen some of that controversy, PNAS has enacted stricter editorial policies over the years, increasing transparency and accountability. As of 1996, contributors are capped at four submissions per year, and the current PNAS editorial policy states, “Members must select reviewers who have not collaborated with the authors in the past 48 months or be from the same institution.”

Starting in October 2015, reviewer names were published alongside contributed articles. Alongside Venter’s article are the names of the three reviewers: Jean-Pierre Hubaux, Bradley Malin and Effy Vayena. Hubaux’s expertise is data and genomic privacy. Malin tells GEN his expertise is in data privacy and data sciences, more generally. Vayena’s expertise is bioethics and health policy. None of the listed reviewers appear to be experts in genome analysis.

As HLI notes to GEN, “The reviewers were chosen by input from several of the authors. They were known to those authors and chosen with their expertise and backgrounds in mind.”

Forensic Applications

GEN discussed the Venter/HLI study with coauthor M. Cyrus Maher, Ph.D., who left HLI before publication and is now head of data science at Cytovale. Dr. Maher was formerly technical lead of the project and is listed among the primary authors of the article.

“I'm proud of the study. I thought it was good science. It was great working with the whole team,” Dr. Maher said. “My hat goes off to Franz Och, who was the head of the project.” Och, Ph.D., the former head of Google Translate, joined HLI in 2014 but left HLI last year to take a new job with Grail, Illumina’s liquid biopsy spin-off company.

According to Dr. Maher, the central idea behind the study was to match a spot of blood with a driver’s license photo. In other words, the goal was to physically identify a person from his or her DNA. The ability to do this has major forensic applications. For example, DNA recovered from a crime scene could be used to reveal physical traits about a suspect, and potentially facilitate his/her identification.

The traits of most interest to HLI researchers were 3D facial structure, voice, age, sex, height, weight, eye color, skin color, and body mass index—in general, the type of information that can be found on a driver’s license.

As stated in the PNAS article, distinct models for predicting skin color, eye color, and facial structure have already been reported. What HLI researchers did was first create machine-learning models for several traits and then integrate them into one souped-up model.

The HLI researchers took the genome sequences of 1,061 participants from an ethnically diverse population. About half of the population was African American and a quarter European. The rest were Latino, East Asian, South Asian, or labeled as Other. Then they applied their proprietary model to identify people based on their DNA. The outcome? They accurately picked a person out of a 10-person mixed-ethnicity lineup 74% of the time. However, when ethnicity was consistent within a group, the accuracy fell. When the 10-person group was all African American or all European, the researchers accurately identified the person only about half of the time.

Though this method may be unique, the ability to deduce a person’s physical traits from their genome—called DNA phenotyping—is not. DNA phenotyping is already used by law enforcement agencies in certain cases to reveal a suspect’s skin color, eye color, and basic facial structure. For instance, earlier this year the Idaho Falls Police Department used DNA phenotyping to try to identify the murderer in a 20-year-old cold case. According to a local news article, two pictures of the suspect’s face were rendered from DNA collected at the crime scene: one reflecting how the suspect might have looked at the time of the murder, and the other with a rendering of the subject aged to the present day. The pictures could resemble thousands of people, but the intent is to exclude suspects and narrow down the suspect list.

Digital Whiplash

Within hours of Venter’s study being posted online, Dr. Erlich, who had reviewed an earlier version of the paper for Science, unleashed a pointed critique on bioRxiv, a platform for (non-peer-reviewed) preprints in biology. Preprints—that is, papers published before peer review—are relatively new in biology and gaining in popularity.

Dr. Erlich heavily critiqued the method used in the paper. “Much simpler techniques could achieve a similar success rate,” he said. This was demonstrated by creating a simple procedure that used only age, sex, and self-reported ethnicity to reidentify a person. Erlich achieved a similar success rate, accurately identifying a person in a 10-person lineup 75% of the time.

“The take-home message should be that identifying someone in a group of ten people requires very little effort. Anyone with access to even low-dimensional data, such as basic demographic [information], can do that,” Erlich wrote.

But Venter’s report has received qualified support from other leading geneticists. “I think the findings in the [HLI] paper are technically correct, and I would believe them,” Atul Butte, M.D., Ph.D., tells GEN.

Dr. Butte is director of the Institute for Computational Health Sciences at the University of California, San Francisco

“However, I do also agree with what Dr. Erlich is pointing out: you can get pretty far in figuring out an individual through predicting demographic features, like race and sex,” Dr. Butte continues. “Predicting characteristics like age and fine-tuning the details are going to be much harder to get right.”

“I really don’t see Dr. Erlich saying there was anything wrong or inaccurate in the manuscript,” he adds.  “I think he is questioning the novelty and significance of the work.”

Dr. Erlich also contested the study’s face prediction model, writing that the face prediction works poorly, relying heavily on sex and genomic ancestry. “Clearly, the Venter algorithm predicts a ‘genetic white male face’ rather than the individual level face of the person,” he wrote.

Facial structure is obviously a complex trait and difficult to glean from DNA data. At this time, scientists can at best make crude (though still helpful) portraits. However, interestingly enough, scientists can do the reverse. They can use facial recognition software to scan the face and detect certain genetic diseases. For example, earlier this year, researchers at the National Human Genome Research Institute, led by Maximillian Muenke, M.D., reported on the use of facial analysis technology to diagnose genetic disorders such as DiGeorge syndrome.

The HLI report also took heat from another coauthor and former HLI employee, Jason Piper, Ph.D., who is currently employed by Apple. Dr. Piper also lashed out against the claim that faces can be predicted from genomes. Coauthor and current HLI employee Alena Harley, Ph.D., vouched for the study, however, posting her thoughts to Medium on September 9, 2017.

“Our amazing team has conducted exciting work—let’s cut through the noise and discuss it on the basis of scientific merit!” she wrote.

Genomic Privacy

One of the major ramifications of the HLI report concerns genomic privacy. In a press release, Dr. Venter said, “The public and the research community at large are not adequately focused on the need for better safeguards and policies for individual privacy in the genomics era.”

“A core belief from the HLI researchers is that there is now no such thing as true deidentification and full privacy in publicly accessible databases,” HLI disclosed in a statement. “Put simply, if you have a genome from the public domain, researchers can sketch a picture of that individual, thus identifying that person,” HLI continued.

But to Erlich, who published a major review on the subject in 2014, the HLI paper does not single-handedly make the threat of genomic data breaches more ominous. “Genetic privacy is an area of active research,” Dr. Erlich wrote. “While it is important to identify new risks, it is equally important to supply policy makers with accurate information based on scientific evidence.”

Dr. Erlich also detailed how an adversary using the HLI method to reidentify a person from his or her genome is not feasible.

Dr. Butte agrees. He tells GEN: “A potential adversary using this methodology would need a large database of comparative data on many individuals already to learn from and would have to figure out how to reidentify across measurement platforms (not everyone can sequence at the same level or quality as Human Longevity). These are both hard to accomplish today.”

“I don’t really think [the HLI article] changes anything for us in academia,” Dr. Butte continues. “We are already extremely sensitive about sharing genome sequences. For instance, when required for scientific publications, many researchers commonly share genetics data through dbGaP [database of Genotypes and Phenotype], a secure monitored site that includes governance and tracking at many levels.”

GEN also sought comment from personal genomics firm 23andMe. In a statement from Kate Black, privacy officer and corporate counsel, the company added: “There's no question that emerging technologies, including in the field of genetics, present new and unique privacy and ethical questions that often outpace applicable legal and regulatory frameworks… We look forward to following the progress of the work described, participating in public discourse regarding new ethical questions in genetic data use, and maintaining the privacy of our customers.”

Meanwhile, HLI prepared a rebuttal to Erlich’s bioRxiv critique, which was also published on bioRxiv on Monday, September 11, 2017. In it, the HLI group wrote, “In the current public debate of our work, it is useful to separate the scientific results and the importance/implications of these results. For the former, the question is whether we did correctly what we set out to do. Hopefully we did. For the latter, the question is whether the ideas, methods or results turn out to be important. This discussion showed that it is a sensitive topic triggering a sometimes emotional debate. We definitely welcome a constructive debate on the topic of genomic privacy, a topic that is of relevance for policy and for genomic sciences."

A Slice of Redemption?

Amidst the public criticism, one aspect of the study has drawn praise: the ability to predict a person’s age from his or her genome. HLI researchers created a model that estimated telomere length (the telomere tips at the end of chromosomes shorten with age) and loss of the X chromosome (for women) or Y (for men) from cells in the body. (Loss of sex chromosomes occurs with age.) Using this model, the researchers could predict a person’s age.

“Seems quite accurate, in their experiment,” Dr. Butte said, referring to the finding. “But how accurate this is on lower-quality sequencing will be an issue, just like the rest of the findings.”

Dr. Erlich complimented the authors for this finding, but remained cautious about the study’s immediate applications when he spoke to GEN. According to Erlich, HLI's method “requires a lot of work and [a] carefully calibrated pipeline,” which he concludes is “not feasible for regular forensic work.”

×

Oops! Please type your email in the following format: [email protected]

You’re all set!
Thank you for subscribing to
Clinical OMICs Weekly
.