Prompt engineering¶

Estimated time to read: 12 minutes

Prompt¶

The prompt is any input or query that we are doing against an LLM. When using the GPT models or any other similar LLM models, you can use a wide variety of prompts, and the versatility of the model means that there are practically no limitations on the types of prompts you can use. However, they can be broadly categorised into a few types.

What is Prompt Engineering¶

We need to think that prompt engineering is a new way to build applications and utilise the power of AI and LLM in our day-to-day activities. We can instruct an LLM to pivot a table in a document or provide a CSV or JSON file and instruct the model to proceed with classifications, translations, summarisation or anything else you can imagine. For example, we can easily create our own Google Translate competitor or a plugin to check our documents or emails for syntax errors and sentiment.

Prompt engineering workflow is designing, testing, and refining the initial input—or "prompt"—given to a machine learning model, especially a language model like GPT.

The prompt engineering is not limited to the input we provide to a model in terms of receiving the desirable quality output. It also defines the adjustments some models need for outstanding performance. That is easier when you are engineering a prompt for a specific service. However, it is becoming challenging for generic-purpose applications like the ChatGPT. Some of the adjustments are:

Mode: This refers to the mode of operation for the model. For instance, it could be a 'chat' mode, where the AI generates responses as if participating in a conversation, or a 'completion' mode, where the AI generates text that completes a given prompt or even a 'Code' mode where the AI generates only code.

Model: This refers to the specific version of the GPT model you're using. For instance, you might use GPT-3, GPT-4, GPTj, NeoGPT, or a fine-tuned variant of these models. Each model has different capabilities and characteristics.

Temperature: This parameter controls the randomness of the AI's output. A higher temperature (closer to 1) makes the output more random, and the model becomes more creative, sometimes with limited training in the topic the model is hallucinating. A lower temperature (closer to 0) makes the output more deterministic and focused.

Maximum length: This parameter controls the maximum length of the generated output, measured in tokens. If you set this to a low number, the AI will generate shorter responses, while a high number allows longer responses. That also helps reduce the cost and response time in specific use cases and types of prompting (see below).

Stop Sequences: These are specific strings of text that, when encountered, signal the AI to stop generating further output. This can be useful for controlling the length and structure of the AI's responses.

Top P (also known as nucleus sampling): This parameter is an alternative to temperature for controlling randomness based on probabilities. A value between 0 and 1 determines the cumulative probability from which the AI selects the next token. A lower value makes the output more focused and deterministic, while a higher value allows for more diversity.

Frequency Penalty: This parameter penalises tokens generated too frequently, which can help increase the output's diversity.

Presence Penalty: This parameter penalises new tokens not present in the input, which can help the AI stay more focused on the given prompt.

Best of: This parameter controls how often the AI should generate output for each prompt and select the best one according to certain criteria.

Inject Start Text and Inject Restart Text: These parameters control whether and how to include specific text strings at the beginning or middle of the AI's output. For example, we can have a pre-defined role for the model (see below).

Show Probabilities: This parameter controls whether to display the probabilities associated with each token in the AI's output. This can be useful for understanding the AI's decision-making process, debugging your prompts, or/and understanding if you need to finetune the model with more content or add embeddings.

The Prompt engineers use the same workflow as any continuous improvement system like DevOps.

This process is crucial for several reasons:

Model Direction¶

Crafting a well-phrased prompt is akin to giving specific instructions to the model. The more precise and targeted your prompt is, the better the AI can generate the desired output. For example, if you want the model to write about the impact of climate change on agriculture, the prompt should reflect that directly. Instead of a broad prompt like "Tell me about climate change", a more directed prompt like "Discuss the impact of climate change on global agriculture" helps guide the model's response in the desired direction. The temperature and top-p parameters can influence the direction of the model's responses. A lower temperature or top-p value can make the output more focused and deterministic, which can be helpful when you need the model to stay closely aligned with the given prompt.

Improving Accuracy¶

Thoughtful, prompt design can reduce the model's tendency to generate incorrect or nonsensical outputs. By specifying the format of the expected answer, providing additional context, or asking the model to think step by step, you can guide the model towards more accurate responses. For instance, if you're looking for a summary of a complex concept, a prompt like "Explain the concept of quantum mechanics in simple terms suitable for a high school student" can lead to a more concise and understandable response. Adjusting parameters such as temperature and top-p can also help improve accuracy. A lower temperature or top-p value can make the model's outputs more conservative and less likely to include incorrect or nonsensical information.

Model Efficiency¶

Effective prompts can help obtain the required output in fewer tokens, saving processing time and computational resources. For instance, if you're looking for a concise answer, a prompt like "In one sentence, describe the process of photosynthesis" encourages the model to be brief and to the point. The maximum length parameter directly influences the efficiency of the model. If you set a lower maximum length, the model will generate shorter responses, saving processing time and computational resources.

Ethical Considerations¶

Given that AI models can sometimes reflect the biases in the data they were trained on, prompt engineering can be a useful tool to manage potential biases and prevent the generation of inappropriate content. Prompts can be designed to explicitly ask the model to consider multiple perspectives, to provide balanced information, or to avoid certain topics entirely. For example, a prompt like "Provide a balanced and unbiased overview of the pros and cons of nuclear energy" encourages a more fair and comprehensive response. Parameters such as the frequency and presence penalties can be used to manage potential biases. The frequency penalty can help increase the output's diversity and prevent the overuse of certain phrases. In contrast, the presence penalty can help the model to stay more focused on the given prompt and avoid potentially inappropriate content.

Customising Model Behavior¶

While adding new embeddings or fine-tuning the parameters of a model during training can require substantial computational resources, prompt engineering offers a more accessible way to customise the model's behaviour post-training. This makes prompt engineering a crucial skill for anyone working with AI language models. Each parameter available for adjustment, including temperature, top-p, maximum length, frequency and presence penalties, and others, provides a way to customise the model's behaviour. Even after model training, carefully tuning these parameters allows you to influence the model's output in various ways, such as making it more or less verbose, more or less random, more or less biased towards certain phrases, etc.

Why that is important¶

While prompt engineering involves crafting the right input to the model, it also includes adjusting various parameters to fine-tune the model's responses after training. This combination of well-designed prompts and carefully tuned parameters can greatly enhance the performance and usefulness of an LLM model. In the context of GPT and similar models, prompt engineering is particularly important due to the large number of outputs the model can generate.

These models are trained on a diverse range of internet text. Still, they lack specific knowledge, beliefs, or intentions, so the prompt is their only guide for generating output. By learning to engineer prompts effectively, users can get more useful and accurate outputs from the model.

Also, the prompt engineering process guides us to find the right model for our application or how to fine-tune an existing model to fit better to our needs. For example, we maybe need a faster response from the model or a more cost-effective model based on the use case. Limiting the model's output to Yes or No will save us a lot of tokens. Another example will be when we have a project based on the English language. In that case, we don't need a model trained in multiple languages. Choosing the right model will give us better response time, faster finetuning, and probably will be more cost-effective. Furthermore, it could guide us in improving an existing model in terms of fine-tuning or adding/removing embeddings for better results for our application.

Prompts by type¶

I will try to categorise the prompts based on types. However, we do not have to be limited to that. With prompt engineering, you need to be creative and test, test, test.

Completion Prompts¶

These are the most straightforward type of prompts where you provide the beginning of a text and ask the model to generate the rest. For example, "Once upon a time, in a land far, far away," and the model will complete the story.

Question Prompts¶

These prompts are formed as questions to get specific information from the model. For example, "Who was the 16^th President of the United States?" Instructional Prompts: These prompts are framed as instructions or tasks for the model to perform. For example, "Translate the following English text to French: 'Hello, how are you?'"

Conditional Prompts¶

These prompts include a condition that the model needs to consider when generating the response. For example, "Write a story about a knight, but make it humorous."

Predictions¶

You ask the model to predict something based on a given context. For example, "What will happen if global warming continues at its current rate?"

Explanation¶

You ask the model to explain a concept, phenomenon, or process. For example, "Explain how photosynthesis works."

Opinion¶

You ask the model to generate an opinion on a particular topic. Remember, though, that AI doesn't have opinions; it generates responses based on patterns in the data it was trained on. For example, "What's your opinion on the importance of renewable energy?"

Scenario¶

You provide a scenario or context and ask the model to generate a response or continuation. For example, "Imagine you are a detective solving a mystery. What do you do?"

Comparative¶

You ask the model to compare two or more things. For example, "Compare and contrast cats and dogs."

Feedback¶

You ask the model to provide feedback on a text or concept. For example, "What could improve this essay on the Civil War?"

Ordering¶

You ask the model to order or arrange things based on certain criteria. For example, "Arrange these historical events in chronological order: The invention of the telephone, the first moon landing, the signing of the Declaration of Independence."

Binary¶

You ask a question with a yes/no or true/false answer. For example, "Is the sky blue?"

Fill-in-the-blank¶

You provide a sentence with a missing word or phrase and ask the model to fill in the blank. For example, "The capital of France is _____."

Multiple-Choice¶

You provide a question or statement and several answer options and ask the model to select the most appropriate one. For example, "Which of the following is not a mammal: a) Dog b) Whale c) Shark d) Elephant."

Open-Ended¶

You ask a question that doesn't have a specific or narrow answer, allowing the model to generate a more creative or extensive response. For example, "What are some potential effects of artificial intelligence on society?"

Step-by-step instructions for complex tasks¶

These prompts are helpful when dealing with complicated tasks or processes. They break the task into smaller, step-by-step instructions, enabling the AI to focus on each step and provide accurate and detailed information. For instance, instead of asking, "How do I build a website?" you could ask, "What are the first three steps in building a website from scratch?"​​.

Role-playing for creative scenarios¶

This involves leveraging the power of role-playing to explore creative scenarios or brainstorm ideas. By assigning roles to yourself and the AI, you can create engaging and imaginative interactions that stimulate the AI’s creativity and generate unique insights. For example, you could use a prompt like, "You are a renowned chef, and I am your apprentice. Teach me how to prepare a gourmet pasta dish with a unique twist"​​.

Iterative refinement of responses¶

If the initial response from the AI doesn’t quite meet your expectations, you can iterate and refine the response. You can ask the AI to provide more information, rephrase its answer, or explore a different perspective. This iterative process can help you fine-tune the output until it aligns with your requirements. An example prompt could be, "That’s a good start, but can you please explain the concept in simpler terms and provide a real-world example?"​​.

Instruct-style prompts¶

These prompts involve asking the AI to think through a problem or question step by step before providing an answer. This approach encourages the AI to consider multiple aspects of the issue and generate a more comprehensive and well-thought-out response. For example, you could use a prompt like, "Think through the advantages and disadvantages of solar energy, and then provide a balanced assessment of its potential as a renewable energy source"​.

Combination of multiple types at one.¶

Usually, we use more than one of the above technics to achieve what we want. For example, you could use something like that. "Act as a Spanish teacher. You are preparing the exam paper for kids in 10^th grade. Provide me with 10 questions based on the Spanish grammar that the kids need to answer. Also, provide me separately the answers."

I hope that helps a few of you to start with Prompt engineering.