Galileo once observed that nature is written in math. Biology might be written in words. Natural language processing (NLP) algorithms are now able to generate protein sequences and predict the mutations of viruses, including key ones used by the coronavirus to evade the immune system.
The key insight that makes this possible is that many properties of biological systems can be interpreted in terms of words and sentences. “We’re learning the language of evolution,” says Bonnie Berger, a computational biologist at the Massachusetts Institute of Technology.
In the last few years, a handful of researchers—including teams from geneticist George Church’s lab and Salesforce—have shown that protein sequences and genetic codes, which can both be represented as strings of letters, can be modelled using NLP techniques.
In a study published in Science today, Berger and her colleagues pull several of these strands together and use NLP to predict mutations in viruses that allow them to avoid being detected by antibodies in the human immune system, a process known as viral immune escape. The basic idea is that the interpretation of a virus by an immune system is analogous to the interpretation of a sentence by a human.
“It’s a neat paper, building off the momentum of previous work,” says Ali Madani, a scientist at Salesforce, who is using NLP to predict protein sequences.
Berger’s team uses two different linguistic concepts: grammar and semantics (or meaning). The genetic or evolutionary fitness of a virus—such as how good it is at infecting a host—can be interpreted in terms of grammatical correctness. A successful, infectious virus is grammatically correct; an unsuccessful one is not.
Similarly, mutations of a virus can be interpreted in terms of semantics. A virus that mutates in a way that changes how things in its environment see it—such as mutations in its surface proteins that make it invisible to certain antibodies—has changed its meaning. Viruses with different mutations can have different meanings, and a virus with a different meaning may need different antibodies to read it.
To model these properties, the researchers used an LTSM, a type of neural network that predates the transformer-based ones used by large language models like GPT-3. These older networks can be trained on far less data than transformers and still perform well for many applications.
Instead of millions of sentences, they trained the NLP model on thousands of genetic sequences taken from three different viruses: 45,000 unique sequences for a strain of influenza, 60,000 for a strain of HIV and between 3000 and 4000 for a strain of Sars-Cov-2, the virus that causes covid-19. “There’s less data for the coronavirus because there’s been less surveillance,” says Brian Hie at MIT, who built the models.
NLP models work by encoding words in a mathematical space such that words with similar meanings are closer together in the model than words with different meanings; this is known as an embedding. For viruses, the embedding of the genetic sequences grouped viruses according to how similar their mutations were. This makes it easy to predict which mutations are more likely for a particular strain than others.
The overall aim of the approach is to identify mutations that might let a virus escape an immune system without making it less infectious—that is, mutations that change a virus’s meaning without making it grammatically incorrect. To test the tool, the team used a common metric for assessing predictions made by machine-learning models that scores accuracy on a scale between 0.5 (no better than chance) and 1 (perfect). In this case, they took the top mutations identified by the tool and checked how many of them were actual escape mutations, using real viruses in a lab. Their results ranged from 0.69 for HIV and 0.85 for one coronavirus strain. This is better than other state of the art models, they say.
Advance warning
Knowing what mutations might be coming could make it easier for hospitals and public health authorities to plan ahead. For example, asking the model to tell you how much a flu strain has changed its meaning since last year would give you a sense of how well the antibodies that people have already developed are going to work this year.
The team says it is now running models on new variants of the coronavirus, including the so-called UK mutation, the mink mutation from Denmark, and variants taken from South Africa, Singapore and Malaysia. They have found a high potential for immune escape in nearly all of them—although this hasn’t yet been tested in the wild. One exception is the so-called South Africa variant, which has raised fears that it may be able to escape vaccines but was not flagged by the tool. They are trying to understand why that is.
Using NLP accelerates a slow process. Previously the genome of the virus taken from a covid-19 patient in hospital could be sequenced and its mutations recreated and studied in a lab. But that can take weeks, says Bryan Bryson, a biologist at MIT, who also works on the project. The NLP model predicts potential mutations straight away, which focuses the lab work and speeds it up.
“It’s a mind blowing time to be working on this,” says Bryson. New virus sequences are coming out each week. “It’s wild to be simultaneously updating your model and then running to the lab to test it in experiments. This is the very best of computational biology.”
But it’s also just the beginning. Treating genetic mutations as changes in meaning could be applied in different ways across biology. “A good analogy can go a long way,” says Bryson.
For example, Hie thinks that their approach can be applied to drug resistance. “Think about a cancer protein that acquires resistance to chemotherapy or a bacterial protein that acquires resistance to an antibiotic,” he says. These mutations can again be thought of as changes in meaning. “There’s a lot of creative ways we can start interpreting language models.
“I think synthetic biology is on the cusp of a revolution,” says Madani. “We are now moving from simply gathering loads of data to learning how to deeply understand it.”
Researchers are watching advances in NLP and thinking up new analogies between language and biology to take advantage of them. But Bryson, Berger and Hie believe that this crossover could go both ways, with new NLP algorithms inspired by concepts in biology. “Biology has its own language,” says Berger.