Adding AI to your Academic Workflow

ChatGPT has taken the world by storm, but it has issues. Let's learn how it works to avoid them and integrate it with our notes for the AI ready academic workflow.

---

My Blog has moved

If you want to receive updates, new content, and tips, check out my website

---

ChatGPT is powerful enough to conjure fears, hopes and promises beyond our imagination. Italy recently banned ChatGPT as a rash reaction to privacy concerns, a group called “Future Life Institute” (including leading AI researchers, Elon Musk and one of Apple’s founders Steve Wozniak) signed a petition to halt development for at least 6 months.

Twitter is abuzz with “prompt engineering Gurus” foreshadowing how it will transform everything you know and are used to and companies pay >300.000$ for “AI whisperers”.

I want to take a slightly different approach and explain a bit about how it works. As academics we are interested in the details, so we don’t fall into the hype loop.

The first part of this article is about what ChatGPT is, feel free to skip it if you are familiar with it.

The second part is how to add it to Obsidian and create powerful templates for prompts specifically for academics.

Obsidian is a note-taking app, the best for academics in my opinion. It integrates with Zotero, embeds PDFs, allows you to search globally in your vault and if you set it up well will help find novel ideas and connections in your notes.

The Effortless Academics Manual is an 8-hour course that I recorded with other experts like Dr. Mushtaq Bilal and Prof Lennart Nacke that first teaches you the basics of Obsidian and then goes into detail on how to take notes well.

What exactly is ChatGPT?

ChatGPT’s capabilities are very impressive but they are also limited.

This limitation is not apparent if you don’t look into the details of how it works - a big commitment. Stephen Wolfram known for the the AI tool Wolfram Alpha that answers queries factually and correctly, wrote a very long and eye-opening essay on how ChatGPT works.

His conclusion is that “language is at a fundamental level somehow simpler than it seems” which made it possible for a model like ChatGPT to learn it. He points out that it is far from being an “Artificial General Intelligence” and that ChatGPT does not yet have reasoning capabilities that go beyond what is available through language manipulation.

To describe it simply: ChatGPT learned to predict the next sentence, given a set of previous sentences (or context).

Its training implies solving billions of “fill in the blanks” questions. Correct answers “teach” the model to fill in the same words in similar contexts. Wrong answers teach it to avoid the results.

During this training fact and fiction are often close by and hard to differentiate. So its biggest problem is to be FACTUAL.

You can imagine that if you fill in the blank in “An eagle is a ___” (bird), you essentially learn that the words eagle and bird often go together, which allows to “speak about birds and eagles”.

Imagine now going from filling in words, to filling in sentences and whole paragraphs. This is roughly what ChatGPT learned to do. And here we can have a lot of intelligent features emerge from JUST dealing with language, but NOT with logic.

Earlier versions of ChatGPT therefore had immense troubles with the simplest mathematical questions or even such simple tasks as balancing the parenthesis in a piece of text. Logically trivial problems, that are yet hard to solve from understanding language alone.

Think of an example of adding numbers on paper. I could teach you that every time you see 5 and 2 appearing in a column you answer 7. Or if you see 4 and 1 you answer 5. (…and so on for the remaining 48 combinations)

Just learning this you could arrive at a system adding numbers together.

Now I ask you to subtract numbers. If you learned by “filling in the blanks”, I need to retrain you to say 3 when you see 5 and 2 in a column or 3 when you see 4 and 1. With enough examples this can be learned by just looking at the text/numbers.

So teaching you these combinations will make it seem like you learned to add arbitrary numbers and learned ADDITION. But in reality addition of big numbers is just a much simpler operation that is broken down into simpler steps (adding one digit numbers).

ChatGPT does this but for WORDS instead of numbers, and it turns out language is simpler than we thought. It requires a system much less powerful than the human brain to create it.

A general intelligence on the other hand understands the concept of NUMBERS (1 then 2 then 3 …) and the concept of OPERATORS subtraction (numbers get smaller) or addition (numbers get bigger). And can conclude how to subtract, multiply, divide etc from just understanding the concept of NUMBERs.

ChatGPT can’t do this (at the moment). It needs to explicitly train on examples.

It feels like ChatGPT understands you, but it rather predicts what you would expect to hear.

More important still is the fact that language is ambiguous. There are limitless ways how this paragraph could be phrased without changing its meaning.

That means: that there are just less errors you can make in language once you have learned its overall structure and grammar.

It becomes apparent when you ask ChatGPT for academic references. It creates you “probable titles” (based on millions of titles it encountered and trained on in the past).

“On the breeding and mating behaviour of extinct New Zealand birds by James Moa et al” SOUNDS entirely reasonable and probable as a title for a scientific publication, but it is not TRUE. If language was math you could instantly see an error like (2 + 2 = 5). But language is ambiguous and errors not easily noticed.

Remember: ChatGPT generates probable sounding text based on its training (billions of english words from articles, books websites etc) and the context (i.e. prompt) you give it. Nothing more and nothing less.

When can we use ChatGPT then?

Generating probable sounding text is by no means an easy feature. Children spend a decade in school learning to do just this by writing essays.

So it is invaluable for our daily work, as we deal with “text acrobatics” all the time when writing, communicating etc.

But as an academic I have to ask myself “Do I trust ChatGPT”? As we have to be logically sound at all costs.

Here are some guidelines to reflect on when using ChatGPT to remain truthful:

Is all information included in the prompt?

If yes, it has a good chance of doing a good job.

✅ Paraphrasing, summarizing, expanding bullet points into text (or vice versa), removing redundancies.

In this example I join my short factual note collection that has citations into a readable text that maintains the citations. Very useful for first draft academic writing.

Do I require logical reasoning?

❌ Don’t use ChatGPT if you need to make logical or mathematical conclusions.

Here is a very simple question (asked on Apr 7th 2023). The result is reminiscent of someone who tries to give a correct sounding answer (i.e. very capable of language) without fully understanding the math (i.e. not very capable of math).

The answer is of course 8$ and not 10$.

Here you see the point of how a generative language model like ChatGPT is brilliant at making a good well explained text, but fails to answer the question.

Remember: It generates statistically plausible sounding LANGUAGE, not meaning.

Do I require something entirely novel?

❌ ChatGPT was trained on existing data and extracted the most probable configurations from it. The opposite of that is ART. Especially surrealist or dadaist art, that is completely non-sensical random, crazy and most of all NOVEL.

ChatGPT will fail here. The same goes for true humour or novel scientific insight.

Here is a comparison of a dadaist poem that ChatGPT created, that is very square, predictable and somewhat cringy vs a true dadaist poem, that makes literally no sense (no it is not german).

The plum cake is an homage to Elsa von Freytag-Loringhoven who famously wore a plum cake hat, fur coat and nothing else to an immigration appointment in the united states consulate about 100 years ago.

Do I require factual statements?

⚠️ ChatGPT is extensively trained on data from wikipedia, books and the web; it will know some facts. For example it knows the dates of the french revolution.

The three scholars it cites DO exist. But “The French Revolution: A very Short Introduction” was not written by François Furet for example, but by William Doyle.

It is very hard to spot, since the answer is “close enough” to sound plausible and even somewhat accurate.

Since the book exists and the scholar exists we can let it slide as an “innocent mistake”. In other words if I were to ask YOU this question and you would have mixed up the authors, I wouldn’t be too upset. (The difference is probably that you would let me know that you are not 100% sure about it.)

However we tend to treat machines and AI as less fallible and more trustworthy than humans.

Remember the movie “Ex Machina” where the robot had to learn to become human and to do so, it learned to lie.

ChatGPT is a different type of machine, that feels human precisely because it is fallible and somewhat inaccurate.

How to integrate ChatGPT into the academic workflow?

I hope the examples above make you understand a bit more the innards of ChatGPT as a generative model. Armed with these we can think of perfect use cases where it can save us hours of time, without falling into the problems it has.

Here is an overview of how we are going to make our notes “AI-ready” and use ChatGPT on them:

1. Install TextGenerator Plugin for Obsidian

I use Obsidian for all my notes, so this workflow will assume you use it as well. My course “The Effortless Academic’s Manual” will teach you everything about using Obsidian for academic note taking.

A wonderful plugin for Obsidian I found is called TextGenerator. It allows you to connect to ChatGPT from within your notes AND to create so called “Prompt Templates”.

Click on Settings > Community Plugins > Browse and type in “Text Generator” into the box on the top left:

2. Install "Prompt Templates” for Text Generator

Once you have installed the Text Generator plugin. Click Ctrl/Cmd + P to activate the Command Palette and type in “Text Generator: Template Packages Manager”

This will open a dialogue to install prompt templates. Click on “Default Prompts” and install it.

In your file explorer on the left you should have a set of new notes inside a folder called “textgenerator”:

In this example I opened a template called “checkgrammar”, which as the name says will help me correct grammar/spelling in my writing (a task ChatGPT is exceptionally good at).

What is a Text Generator Prompt Template?

The templates are notes in your vault. You can use placeholders that will be replaced with contents of your note (in 99% of cases you will use the placeholder {{context}}). See the screenshot above, for my check grammar template.

The placeholder {{context}} gets replaced by either:

• The entire note if no text is selected.

• Selected text, if you select some.

• The line of text if you cursor is in the middle of a line.

You can find all the details here (which is the source for the image below):

3. Connect to ChatGPT

To connect TextGenerator to ChatGPT you will need an API key. Think of it as a unique password that will identify your requests to ChatGPT and link them to your account.

Navigate to your OpenAI account. Login with your OpenAI credentials (or get a new account).

Click on "Create API key", copy the text.

In Obsidian open the Text Generator settings and paste the key.

You might wonder why you need this. ChatGPT is not free. While you can use it in the browser for free, these integrations cost something. You have a free quota, check how much you used up here.

ChatGPT runs billions of calculations to generate a single word of output. This costs electricity and requires hardware, which is what you pay for. More complex models like GPT-4 have even more parameters and run even more calculations, that is why they are much more expensive. GPT-4 is available for 20$/month.

4. Use a Text Generator Prompt Template

To use a template press Ctrl/Cmd + P to to open the command palette and search for “Text Generator: Templates: Generate & Insert”. It is advisable to add a hotkey to this command as well, if you use it frequently.

In the following popup you can select a prompt template. This is just a file in the textgenerator folder. You can add your own here by just adding new templates in the same format.

5. See some examples

In the above example we had a template to check the grammar. This is how it looks, when I use it:

The metadata at the top of the prompt template is used just for describing the template. Make sure however that whatever is typed after “promptId” is different for every prompt template you create.

Here are a few more prompts you can use:

5 more prompt template examples for academics

This tweet and that tweet contain more explanation on the examples above.

This and many more templates will soon be available as part of the “The Effortless Academic’s Manual” (free update, if you have purchased the course previously).

Use Caution with ChatGPT

Notice that in all examples we use ChatGPT exclusively to work with text, and we provide all information as part of the prompt.

A hack I found very useful is to provide my academic sources as links in the text. This way I can ask it to maintain these and end up with academic writing, containing real (i.e. my own) citations!

But always remain sceptical when using ChatGPT. Remember it paraphrases your text in a smart but not always correct manner. There is nothing more embarrassing than relying on truthfulness of answers only to find out that it is pure fiction. Always provide the full context to your ChatGPT request.

Summary

ChatGPT is a generative language model. It means it is an expert at generating sentences that most likely follow the prompt (other sentences) you provide. What is likely and what not is determined by the text it was trained on.

If I ask you to complete “An eagle is a bird of prey.”, most people will likely answer “It feeds on other animals” but not “It prefers green vegetables for its diet”. ChatGPT was trained on billions of such words and sentences found just like this article online or in books. So it “knows” a few things, but from repetition not from fact or logic.

When using ChatGPT use caution and ensure that:

• All information needed to answer the request is included in the prompt.

• No logical reasoning is required (especially avoid math).

• No factual knowledge is required

• Don’t expect true random novelty outside of what has been done similarly.

If you do this you can use the tool to its full potential, without running into problems. To remain truthful use short notes, always add citations and be realistic about your expectations.

Integrate ChatGPT with your notes using the “TextGenerator” plugin. Download the prompt templates and start creating your own by duplicating them. Good use cases are:

• Checking grammar in your writing.

• Paraphrasing and moving from text to summaries or vice versa.

• Extracting questions of interest from a text.

• Coming up with tags for your text.

Support this newsletter

I hope this newsletter helps you to learn tools and best practices for academics. If you want to support me, please check any of my paid courses here: