The Effortless Literature Review
Literature review does not need to be time consuming or confusing. Effortlessly find literature, use AI to screen for relevance and "3-step-filtering" to find the top 10% papers in your collection.
My Blog has moved
If you want to receive new content, tips, and updates, check out my website.
Getting into a new field, starting a project or PhD can be challenging. The problem: You don’t know what you don’t know (aka unknown unknowns).
Most people therefore start “hoarding” publications – everything seems relevant. You quickly collect a 100 papers but do you have the 2 months it takes to study them? Probably not.
There is a better way and I call it: Effortless Literature Review. After testing dozens of tools I polished a workflow that leverages AI, modern discovery tools and creates an automated discovery & priotization engine.
Let’s look at the whole process and then go into details:
This seems like many steps - but you probably know some of them already. The effortlessness comes from integrating them with one another and leveraging the synergy between them.
Let’s look at the steps:
1. Finding “Seeds”
The old ways of finding papers is to “google” them, e.g. on Google Scholar or Arxiv.
If you search for “yeast” you may find papers on:
Brewing science (yeast as a fermentation agent)
Molecular biology (yeast as a model organism)
Medical science (yeast as infection)
Evolution (yeast by itself)
Google would usually rank higher cited papers at the top. Molecular biology and medicine are huge domains. So effectively you will find much less on evolution.
Problem: Finding all relevant literature in such cross-domain fields becomes challenging. Separating important from irrelevant is even harder.
Scite.ai (20$/mo) can help here by hinting at the importance of individual papers. It can tell you how often a paper has been supported or contrasted.
Papers with supporting claims are not “just mentioned” but actually validated. So the connection in our publication network is stronger and thus indicated relevance.
Luckily you can use both - as scite integrates into google scholar.
Scite.ai will also allow us to find where and how exactly a paper is mentioned (i.e. let us see the mentioning snippet). It is an amazing tool that integrates well with the rest of the academic tool ecosystem.
This combination of Google Scholar and scite.ai is a great way to find the first few papers. Their citations and references will give us access to all the other relevant papers (Step 3).
I am making a workshop on this workflow, if you are inclined to learn more and would like a demo and to ask questions. Saturday, March 18th on Zoom.
2. Analyze abstracts with AI
A while ago I tweeted about how to use ChatGPT to read abstracts using a little-known plugin called ArxivGPT.
With a little bit of creativity you can use ArxivGPT not to summarize abstracts but to ask it to RATE them.
People are weary of using ChatGPT in science, because it creates fake citations, doesn’t know math, and is over confident.
But ChatGPT is good at one thing: Comparing text.
If you design your prompts to describe what you are looking for and then add the abstract of a paper the results will surprise you.
In this example, I used a simple prompt and asked it to rate for relevance. And indeed the results are spot-on!
This allows me to screen papers for relevance. You can ask ANY question that could be answered from the abstract and without domain knowledge.
There are so many ways this workflow can be used:
Filter method papers, from theory papers
Identify papers working with a specific model organism, algorithm, or methodology
Identify what the authors themselves see as potential limitations (discussion)
Summarize in different formats (keywords, bullets, sentences)
In the upcoming workshop on Literatur Review, you will learn even more ways to use this, to integrate and automate it. I would love to see you there.
3. Growing your collection
Once you have acquired a few seed papers, it is time to start exploring the citation network. Go through the list of references at the end of your seed paper and you will find all the papers in the past.
The problem here is that you can only go back in time - not forward.
References are easy to find, but they only look into the past. We need citations to find out how the field has developed since.
In the past, you would look into the journal of the publication and read a more recent issue. But citations can be scattered across many journals.
Today the journal method is outdated. There are tools to find everything much more easily and visually.
My all-time favourite tool for the job is Litmaps. In Gourmet Literature Review I described how we can use it, so I won’t go into too much detail here. But to give you an overview, this is how it looks:
Seed maps in Litmaps show us all publications related (references and citations) to a single publication. They are an effortless way of discovery where we don’t have to drill through reference lists.
Discovery in Litmaps shows us similar papers to a collection of papers we already have.
So each time we look at either of these we can add new papers and grow our collection.
Litmaps will be at the very core of the Effortless Literature Review Workshop (March 18th or as a download after).
Bonus: I invited the CEO of Litmaps and you can ask him anything you want about the tool or even make suggestions!
4. Importance Filtering
After you have grown your collection in the previous step you will probably end up with 100s of papers. Impossible to read.
My method is to filter in 3 steps or cuts. Identify the:
10% most recent
10% most cited
10% reviews
Archive the rest of the papers for later (or never).
All of this is easy to do with Litmaps and Zotero. Luckily those two work together through an exchange format called BibTeX. I can import and export papers from one to another.
This workflow is also partially described in the literature review article. Since writing the article I added scite.ai to also look and sort for controversial papers.
In the picture above you see that the Mori 2018 paper did receive the least citations of the 5. But it has more debate going on as there are many supporting claims as well as someone contrasting it.
This paper is therefore debated. (And indeed it is, this paper is about the link of biodiversity to ecological functions or ecosystem services. First of all, we don’t know for sure where biodiversity comes from and second, we only start to understand what services ecosystems provide (i.e. air/water purification, climate regulation, flood protection) etc. Of course, it is a subject of scrutiny! )
This integration allows us to identify ongoing debates in the field. And this is where “science” happens. You can quickly learn arguments around your field with precisely these papers.
I will show you many other examples in the webinar - join us, it is just 15$ as last time or a recording if you can’t attend.
5. Reading and taking notes
Note-taking is actually the more important skill when doing research. In my last article, I wrote about boosting creativity and generating ideas on auto-pilot by using the right tools and methods.
Problem: We summarize what we read but we focus less on connecting them.
That is because our brain is a connecting system. So even by summarizing, we get new ideas through the subconscious workings of the mind.
But our mind has limits. That means the number of ideas generally won’t scale indefinitely with the amount of information we consume. At some point, you will start forgetting and this replaces older knowledge.
The right note-taking system can help here. We can let our tools generate some of the connections we miss with our minds. Plus everything is stored forever.
This graph visualizes the idea.
In gray are notes that summarize what I know (here: on proteins A, B, C, and D…). While some of them are connected because I thought of it, others are connected in two steps by the green and purple nodes. These are tags and maps of content respectively.
By tagging two notes with the same label I connect them, but without explicitly thinking of it.
This is the “auto-pilot” aspect of this method. We only think of tagging N notes. The number of possible connections between them is N * (N - 1). For 100 notes this is 9900 connections - impossible to think of manually.
In the upcoming workshop, we will talk about this as well as the idea of visual reference management (VRM).
If you have used Zotero or another reference manager you have encountered these endless lists of papers with the only way to navigate is the search bar.
But you can only search for what you can recall i.e. title, author, or tag of a paper you saved.
VRM allows you to lay out the papers visually on a canvas and connect them in groups or individually.
The result is that you can now use your spatial memory to remember your publications. Humans have evolved from animals that had to roam vast landscapes in search of food, naturally, our spatial memory is quite well-developed.
Concepts on the other hand are abstract and only a very recent evolutionary development. That’s why you can remember the neighbourhood of your childhood even decades later.
We will develop a visual reference manager in Obsidian in the Effortless Literature Review workshop. Join if you like the idea.
Summary
Literature Review is central to any researcher and also the most time-consuming part of science.
Tools and polished workflows can cut the time you spend on it. They can help you create an overview, identify debates and “hot topics”.
Just 5 years ago most of the tools I am using here did not exist. If you have been in research that long, chances are you might have missed out on some of them.
This is why I created the Effortless Literature Review workshop. It will get you up to date and hopefully boost your career in the long run.
Those tools are not complicated. They are increasingly more customer-focused and user experience-based. That is why I am very confident that you can learn most of it in just about 2 hours.
I hope to see you in this workshop (March 18th, ~noon in the US, ~evening in the EU). I priced it at 15$ dollars to keep it accessible to everyone. Reach out to me with questions you have.