Researchers took to Twitter this week to discuss a possible methodological issue in gene expression studies — specifically, those that use an enrichment analysis to test whether genes with certain functions are overly switched on or off in a sample.
The issue arises when researchers use the entire genome to determine whether their sample overrepresents particular gene functions, a mistake that “can lead to 56% of enrichment results being false,” tweeted Mark Ziemann, senior lecturer of biotechnology and bioinformatics at Deakin University in Geelong, Australia.
So how bad are the consequences of this mistake?
In a recent paper we showed that this mistake can lead to 56% of enrichment results being false!
See Fig 4Dhttps://t.co/ZkLiODwnm0— Mark Ziemann???????? (@mdziemann) February 17, 2023
Instead, researchers should always create a custom list of background genes that were detected in the experiment and then base their estimates of overrepresentation solely on those genes, Ziemann explained. “Any enrichment tool that doesn’t accept a background list is invalid and should never be presented as evidence of anything.”
Any enrichment tool that doesn’t accept a background list is invalid and should never be presented as evidence of anything. Developers of these tools need to add a background as a MANDATORY part of the analysis, and force the end-users to think about why it might impact results.
— Mark Ziemann???????? (@mdziemann) February 17, 2023
Ziemann shared links to two tools that take background lists: The Database for Annotation, Visualization and Integrated Discovery and ShinyGO 0.77. Another tool, g:Profiler, has the “advantage of keeping access to previous versions … for reproducibility,” tweeted Ozan Ozisik, a postdoctoral researcher at Marseille Medical Genetics in France.
g:Profiler is also a good example. It also has the great advantage of keeping access to previous versions (application+data) for reproducibility https://t.co/LgRsGGEDwx.
— Ozan Ozisik (@OzisikOzan) February 18, 2023
“If you’re doing pathway analysis and aren’t using a custom background … then all you’re doing is finding out what tissue you sequenced,” tweeted Scott Tyler, a postdoctoral fellow in Supinda Bunyavanich’s lab at the Icahn School of Medicine at Mount Sinai in New York City.
What he said ????????. If you’re doing pathway analysis and aren’t using a custom background, (i.e. relative to what could have been a DEG), then all you’re doing is finding out what tissue you sequenced. Custom backgrounds need to be mandatory! https://t.co/0vFCctOPnW
— Scott Tyler (@[email protected]) (@ScienceScottT) February 17, 2023
“I flag this every time I review a paper with enrichment,” tweeted James Whiting, a postdoctoral associate at the University of Calgary in Canada.
Glad to see this being discussed, I flag this every time I review a paper with enrichment. Always consider what your background or ‘universe’ gene set is! https://t.co/IFcsECtyCb
— Jim Whiting (@jimwhiting_sci) February 18, 2023
A different tool, called Neuro-stack — “a wearable platform that records human single-neuron activity during walking” — made its debut in a paper in Nature Neuroscience this week, tweeted Nanthia Suthana, associate professor of psychiatry, neurosurgery, bioengineering and psychology at the University of California, Los Angeles.
???? Our latest paper is out today in Nature Neuro ????
Introducing the Neuro-stack, a wearable platform that records human single-neuron activity during walking????????♀️https://t.co/rT7cVlwScp pic.twitter.com/gkvX96ngyF
— Nanthia Suthana (@SuthanaLab) February 20, 2023
Neuro-stack enables researchers to take recordings of neurons without bulky cart-mounted equipment, has a wireless controller and can be customized to deliver different types of stimulation, Suthana tweeted.
The Neuro-stack is not only wearable, but customizable too.
It can deliver programmable stimulation (frequency, amplitude, timing, & shape are all configurable) & be used to predict memory performance based on decoded neural activity.
— Nanthia Suthana (@SuthanaLab) February 20, 2023
“Gosh. The future is here,” tweeted Alexander Shaw, lecturer of psychology at the University of Exeter in England.
Gosh. The future is here ????
— Alexander ???????? @[email protected] (@DrAlexanderShaw) February 20, 2023
“A big step towards moving neuroscience out of the lab and into the real world!” tweeted Matthias Nau, guest researcher in psychology at the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig, Germany.
Amazing setup @SuthanaLab! A big step towards moving neuroscience out of the lab and into the real world! https://t.co/rZv9PJ82Zp
— Matthias Nau (@NauMatt) February 20, 2023
Moustafa Algamal, a research fellow in neurology at Massachusetts General Hospital in Boston, tweeted, “I can’t imagine how neuroscience will advance if, in the future, we are able to measure single-neuron activity non-invasively in healthy subjects.”
This work is a positive step towards improving our understanding of the human single-neuron activity. I can’t imagine how neuroscience will advance if, in the future, we are able to measure single-neuron activity non-invasively in healthy subjects. https://t.co/dmOipKiGtV
— Moustafa Algamal (@moustaam) February 20, 2023
Elsewhere on Twitter, researchers discussed the growing difficulty of getting manuscripts peer reviewed. One contributing factor is that in some countries, Ph.D. students are required to publish multiple papers to graduate, so “work that could be one bigger paper is split to many small ones,” which causes a backlog in the peer-review process, tweeted Tuuli Lappalainen, senior associate member at the New York Genome Center in New York City.
It’s increasingly difficult to get manuscripts peer-reviewed. One factor inflating the number of papers: Some countries (????????????????, others?) requiring a PhD student to publish multiple (~4) papers to graduate. Thus, work that could be one bigger paper is split to many small ones.
— Tuuli Lappalainen (@tuuliel) February 16, 2023
Lappalainen suggests that it would be better to let students graduate when they have done enough work, even if that means they have only published one paper. If delays in the publication process continue to worsen, “we will start to see more and more delayed graduations when papers are stuck in journal limbo for years,” she tweeted.
Finally, as the delays in the publication process get longer and longer due to the struggle to find reviewers, and some universities require a fixed number of accepted papers, we will start to see more and more delayed graduations when papers are stuck in journal limbo for years.
— Tuuli Lappalainen (@tuuliel) February 16, 2023
A publication requirement “disincentivizes participation in large (and inevitably slower) consortiums,” tweeted Brooke Wolford, a postdoctoral fellow in the K.G. Jebsen Center for Genetic Epidemiology at the Norwegian University of Science and Technology in Trondheim.
Yes, agreed. ???????? requires 3 articles and I also think it does a disservice to trainees forcing them to invest in potentially smaller and quicker low impact papers. Disincentivizes participation in large (and inevitably slower) consortiums too which was a great part of my training. https://t.co/Omg43eyMAn
— Brooke Wolford, PhD (@bnwolford) February 16, 2023
“Putting an arbitrary number on something that will significantly impact a person’s career forces them to game the system … It’s just bad for everyone,” tweeted Noel Carter, associate professor of molecular biology at the University of Sunderland in England.
Also putting an arbitrary number on something that will significantly impact a person’s career forces them to game the system and possibly start to make stuff up / p-hack etc. It’s just bad for everyone.
— DrNoelCarter (@DrNoelCarter) February 16, 2023
Offering a different opinion, Daniel Fischer, senior scientist at the Natural Resources Institute Finland in Helsinki, tweeted that “learning to write papers is a skill a PhD should learn and that requires practice,” so requirements to publish multiple papers are important.
I believe learning to write papers is a skill a PhD should learn and that requires practice. Whereas in the first paper the supervisor can help a lot, the support in following papers should gradually go down.
Imho, multiple PhD papers are important.
— Daniel Fischer @[email protected] (@Fischuu) February 17, 2023
That’s it for this week’s Community Newsletter! If you have any suggestions for interesting social posts you saw in the autism research sphere, feel free to send an email to [email protected].
Follow us on Facebook, Twitter (@Spectrum), Instagram and LinkedIn.
Cite this article: https://doi.org/10.53053/WRMB3060
By joining the discussion, you agree to our privacy policy.