Many of us have added ‘developing skills to work with ChatGPT’ to our learning plans for 2023. I called this out in a recent post on my own target skills for the year.

Why is this important?

ChatGPT from Open.ai is a tipping point in how people perceive AIs and how we interact with them. It is an intuitive conversational interface to one of the best of the Large Language Models (LLMs) that have been under development over the past decade (the key architectural advance came from Google in 2017, see Attention is All You Need). You will probably be using a conversational interface to ask questions of and generate text with a LLM in 2023 and you will be unable to avoid it by 2024. Learning how to do this effectively will become a standard skill that all of us are expected to have. ChatGPT style AIs will show up in Microsoft, Adobe, Google, Salesforce and other standard tools of the trade at some point in 2023. They will even show up in Ibbaka Valio (our pricing and customer value management platform) and Ibbaka Talio (our skill management platform).

So what skills do we need to work with ChatGPT? Not engineering or natural language processing (NLP). Not probability, statistics and Bayesian networks. Not deep learning and neural nets. All of those are important skills, and fun to learn, but the skills needed to work with conversational interfaces to large languages models are more foundational.

1. How to ask questions (design prompts)

2. How to explore (iterate)

3. How to evaluate responses

4. How to detect bias

5. How to generate new ideas (generative thinking)

How to Ask Questions (Designing Prompts for ChatGPT)

The place to start is with designing prompts. The prompt is the text that you input into ChatGPT (or other content generation AIs including image and music generation engines) that generates a response. The response is probabilistic, so repeating the same prompt will give you different results (one reason that iterating is so important).

There is already a large literature on this, and places where you can go to see prompts and get ideas. There are even emerging marketplaces for prompts!

• Prompt Libraries on Github

• Prompt Marketplace (for image generation and not (yet) for ChatGPT)

There are two basic styles to designing prompts and you need to be fluid in both styles: prompt sequencing and large prompts.

Prompt sequencing

ChatGPT is responsive to a sequence of prompts. There are two approaches here: General to Specific and Specific to General.

Start with a rather general prompt and then focus in by asking about terms or ideas that show up in the response. Create, cultivate and prune branches of prompts. An example (with ChatGPT responses deleted, try this sequence yourself).

1. How can I learn better?
   

2. What is a mental map?

3. Can I have more than one mental map of the same domain?

4. How can I combine or connect two mental maps?
   

5. What are learning styles?

6. How strong is the evidence for different learning styles?

7. Can I have more than one learning style?
   

8. What is social learning?

9. Can I really learn with other people?

10. What happens when two people with different learning styles try to learn together?
    (This was a way to connect or blend the two branches - I wanted to ask about concept blending, but I needed to get on with writing this post)

Sometimes this feels like following links on the Internet, but it is important to be more goal driven and to consciously shape your prompts and the connections between them.

Trained librarians are very good at this (ask ChatGPT ‘How do research librarians research a topic?’).

Direction and discipline are good, but it can get you on too narrow a track, so sometimes throw ChatGPT a curveball. Take a phrase from a response and try to interpret it differently and set ChatGPT off in a different direction.

An alternative to the general to specific style is to begin with something very narrow and specific and then try to open the ‘conversation’ up. Here is another possible sequence (I know, a bit geeky, but I am coaching children on math these days).

1. How do I learn how to prove propositions about triangles?
   

2. Can I apply the same approach to squares?

3. What about circles?

4. What kind of things can I claim about triangles?

5. What can I say about geometric figures in general?
   

6. What is a proposition about a mathematical figure?

7. What other types of proposition are there?

8. How do propositions generate new propositions?

And so on.

Try going back and forth between the general-to-specific and specific-to-general styes. They reinforce each other. Most of us have a preferred approach, and will find it stimulating to try the other style from time to time.

Large prompts

Part of the power of ChatGPT comes from its ability to accept very large prompts of thousands of words. You can post in a whole article and ask ChatGPT to summarize for you. Try dropping in a piece of code and asking ChatGPT to explain what is does, if there are bugs, how it could be optimized. There are even people experimenting with gene sequences in ChatGPT to see if it can say anything useful about such sequences. There is now a whole sub literature on ChatGPT (and other LMMs) and bioinformatics, see for example Can ChatGPT solve bioinformatic problems with Python?

There is no need to stop with just one article. Try putting in two articles and asking ChatGPT to find the common themes and then to identify contradictions. Or one can take a series of articles and ask it to combine them into one article.

An especially fun thing to do is to post in the start of a story and ask ChatGPT ‘what will happen next?’ I have not tried it yet, but when I have an open morning I want to see if I can write a renga with ChatGPT (I am reading Karthika Nair and Marilyn Hacker’s wonderful riff on renga A Different Distance, one of the best books I have read on the Covid Pandemic).

If your style is to give ChatGPT a series of short prompts it is fun to switch up and try some large (100s of words) or very large (thousands of words) prompts from time to time. Cut and paste is your friend here!

How to Explore (Iterating with ChatGPT)

ChatGPT is probabilistic. It is built on a Bayesian Network, graph (in the mathematical sense) where the links between nodes are probabilistic (they are not always applied). This means that ChatGPT will generally not give the same response to the same question. That is a very good thing. Most knowledge is not absolute and can be expressed in different ways.
A best practice is to make a habit of using the ‘ask again’ function several times and then comparing responses. You can even use the trick of generating several responses to the same prompt and then putting them all into one large prompt and asking ChatGPT to summarize and then to find contradictions.

Learning requires repetition, and by asking the same question multiple times, and then asking it in slightly different ways, one gets a much deeper understanding of how ChatGPT works and how ChatGPT can work with you.

• Ask the same question different ways

• Get multiple responses to the same prompt and then compare the results

• Repeat over time

Evaluating Results given by ChatGPT

ChatGPT has condensed and connected knowledge from a large volume of text from the Internet. It is a summary of what we all know. What we all know is often wrong.

One of the best examples of this came from Stephen Wolfram, the leader of the computational knowledge engine Wolfram Alpha.

So an important skill is being able to evaluate when ChatGPT, as a summary of our shared knowledge, is getting it wrong.

One of the key skills for using ChatGPT is evaluating its responses. There are three basic ways to do this.

1. Go outside ChatGPT to other resources (Wolfram Alpha is a good reference for anything computational, but it also has limitations as it is decidedly bought into the idea that the world can be described as a sequence of 0s and 1s.)

2. Have three way conversations with ChatGPT and your colleagues and learn to test responses together

3. Use ChatGPT to check itself by asking questions different ways, uncovering contradictions, and asking it to self assess (ChatGPT is quite good at this)

As simple rule of thumb …

When you think you have a definitive answer, ask more questions

Detecting Bias

I tend to think ChatGPT is correct, or at least headed in the right direction, when it agrees with me. Like all people, I am biased by my experiences and beliefs. Like all systems, ChatGPT is biased based on the underlying approach used to build the LLM and the data used to train it. Bias is part of who we are and the world we live in. The goal is not to eradicate bias but to detect it and be aware of it.

Can ChatGPT be used to detect and challenge my own bias?

Our own biases are usually invisible to us. Can we use ChatGPT to challenge ourselves and put our assumptions into question?

I think yes. Here is something I have been trying.

1. Take one of your strongly held beliefs and ask ChatGPT for evidence to support the belief.
   
   ”Does cognitive diversity lead to improved performance?”
   ”When does cognitive diversity lead to improved performance?”
   ”How does cognitive diversity lead to improved performance?”
   

2. Ask the inverse.
   
   ”Does cognitive diversity lead to degraded performance?”
   ”When does cognitive diversity lead to degraded performance?”
   ”How does cognitive diversity lead to degraded performance?”
   

3. Combine the answers to the two sets of questions in ChatGPT and ask for a summary, a synthesis, and for contradictions.

How can I detect and account for ChatGPT’s biases?

There is a lot of research going on into the biases built into ChatGPT. As a ChatGPT user you need to keep up with this research. It is also research into the biases built into Internet content (ChatGPT is capturing and distilling biases that already exist).

One interesting example of this is David Rozado’s analysis of the political bias of ChatGPT and his extension of that work. There are lessons to be learned here from the general approach. Take a number of standard tests on orientation and apply them to ChatGPT and see what results.

This is a quickly moving area of research so we all need to devote some time each week to updating our understanding of bias in LLMs. Remember that these are not the biases of people at Open.ai or its researchers, but rather the bias of the vast organization of human knowledge, experience and belief that has been captured in the Internet.

ChatGPT and Generative Thinking

ChatGPT will challenge our critical thinking skills as we learn to construct effective prompts, evaluate responses and detect our own and system bias.

Can it also support generative thinking, the ability to come up with new ideas and explore them?

One of the concerns about LLMs is that by relying on them we will limit our ability to imagine things outside of the model. The model will come to constrain what is possible. (That is to say, the models will be Barnesian performative, if we behave in the way a model suggests we should behave the model becomes a better and better model of reality.)

This is a real concern, but at the same time LLMs are combinatorial and probabilistic and so can generate trillions of different combinations of text. They are like Borges Library of Babel, or perhaps a version of everything, everywhere all at once. How can we leverage this and develop our own generative thinking skills?

I am just starting to explore this by applying concept blending to the construction of prompts.

Concept blending is a formal approach developed by Mark Turner and many associates to better understand how we think, understand and come up with new ideas. See the website Blending and Conceptual Integration.

Try the following …

1. Ask ChatGPT to summarize two different domains that you are interested in but do not see as being particularly closely related.

2. Combine the two ChatGPT responses into one prompt and input that.

3. Ask ChatGPT to summarize, synthesize and find the contradictions in the result it creates (remembering to explore a number of iterations on both the inputs and the outputs).

4. Draw out the concept blend yourself using a standard concept blend diagram.

5. Look for places where you could apply this concept blend in your own work and life.

Developing applications for ChatGPT

We are in very early days in understanding the power of LLMs (Large Language Models) and conversational interfaces. There are many uses to be explored and exploring these applications is one of the best ways to learn.

A simple example. I am writing a novel with one of my granddaughters, The Enchanted Library. We are experimenting with using ChatGPT to write the dialog for one of the characters. We are beginning by writing some dialog ourselves, to establish a tone an habits of speech, and using this to ‘train’ ChatGPT. Will this work? We have no idea, but we expect to learn by trying to find out.

Josh Bersin, the HR pundit, has a useful post Understanding Chat-GPT, And Why It’s Even Bigger Than You Think that looks at some of the ways ChatGPT and similar systems can be used in HR. He is advocating training language models on smaller, more focussed, sets of data that have been pre screened by humans. For example, one could train a model that only uses peer-reviewed journal articles. Will this give better results? Well, it will likely give more predictable results, if that is what you are looking for (and it is in certain use cases, like user support or other rules based behavior, it is not what is wanted for creative applications).

There will be a new discipline of selecting and training language models on different document sets and applications that make this easy to do. Ibbaka will be seeing if we can apply this in areas like skill description and connection or value proposition generation and testing.

Which leads to an interesting question around how LLMs will evolve. There are several possibilities to imagine.

• LLMs will converge and a best model will emerge

• LLMs will diverge and different models will evolve in different directions creating many different niches

• LLMs will modularize and it will be possible to split, combine, invert, replace modules (or communities in Social Network Analysis approaches)

Probably all three of these evolutionary paths will be explored and applied to different use cases.

Keep track of the emerging use cases for ChatGPT and add your own.