Image created by Author with DALL•E 3
Prompt engineering, like language models themselves, has come a long way in the past 12 months. It was only a little over a year ago that ChatGPT burst onto the scene and threw everyone’s fears and hopes for AI into a supercharged pressure cooker, accelerating both AI doomsday and savior stories almost overnight. Certainly, prompt engineering existed long before ChatGPT, but the range of ever-changing techniques we use for eliciting desired responses from the plethora of language models that now invade our lives has really come into its own alongside the rise of ChatGPT. Five years ago with the unveiling of the original GPT we joked about how “prompt engineer” could one day become a job title; today, prompt engineers are one of the hottest tech (or tech adjacent) careers out there.
Prompt engineering is the process of structuring text that can be interpreted and understood by a generative AI model. A prompt is natural language text describing the task that an AI should perform.
From the “Prompt engineering” Wikipedia entry
Hype aside, prompt engineering is now an integral part of the lives of those interacting with LLMs on a regular basis. If you are reading this, there’s a good chance this describes you, or describes the direction that your career may be taking. For those looking to get an idea of what prompt engineering is, and — crucially — what the current prompt strategy landscape looks like, this article is for you.
Let’s start with the basics. This article, Prompt Engineering for Effective Interaction with ChatGPT, on Machine Learning Mastery covers the prompt engineering foundational concepts. Specifically, topics introduced include:
- Principles of Prompting, outlining several foundational techniques to remember in the process of prompt optimization
- Basic Prompt Engineering, such as prompt wording, succinctness, and positive and negative prompting
- Advanced Prompt Engineering Strategies, including one-shot and multi-shot prompting, Chain-of-Thought prompting, self-criticism, and iterative prompting
- Collaborative Power Tips for recognizing and fostering a collaborative atmosphere with ChatGPT to lead to further success
Prompt engineering is the most crucial aspect of utilizing LLMs effectively and is a powerful tool for customizing the interactions with ChatGPT. It involves crafting clear and specific instructions or queries to elicit the desired responses from the language model. By carefully constructing prompts, users can guide ChatGPT’s output toward their intended goals and ensure more accurate and useful responses.
From the Machine Learning Mastery article “Prompt Engineering for Effective Interaction with ChatGPT“
Once you have covered the basics, and have a taste for what prompt engineering is and some of the most useful current techniques, you can move on to mastering some of those techniques.
The following KDnuggets articles are each an overview of a single commonplace prompt engineering technique. There is a logical progression in the complexity of these techniques, so starting from the top and working down would be the best approach.
Each article contains an overview of the academic paper in which the technique was first proposed. You can read the explanation of the technique, see how it relates to others, and find examples of its implementation all within the article, and if you are then interested in reading or browsing the paper it is linked to from within as well.
Unraveling the Power of Chain-of-Thought Prompting in Large Language Models
This article delves into the concept of Chain-of-Thought (CoT) prompting, a technique that enhances the reasoning capabilities of large language models (LLMs). It discusses the principles behind CoT prompting, its application, and its impact on the performance of LLMs.
Exploring Tree of Thought Prompting: How AI Can Learn to Reason Through Search
New approach represents problem-solving as search over reasoning steps for large language models, allowing strategic exploration and planning beyond left-to-right decoding. This improves performance on challenges like math puzzles and creative writing, and enhances interpretability and applicability of LLMs.
Automating the Chain of Thought: How AI Can Prompt Itself to Reason
Auto-CoT prompting method has LLMs automatically generate their own demonstrations to prompt complex reasoning, using diversity-based sampling and zero-shot generation, reducing human effort in creating prompts. Experiments show it matches performance of manual prompting across reasoning tasks.
Parallel Processing in Prompt Engineering: The Skeleton-of-Thought Technique
Explore how the Skeleton-of-Thought prompt engineering technique enhances generative AI by reducing latency, offering structured output, and optimizing projects.
Unlocking GPT-4 Summarization with Chain of Density Prompting
Unlock the power of GPT-4 summarization with Chain of Density (CoD), a technique that attempts to balance information density for high-quality summaries.
Unlocking Reliable Generations through Chain-of-Verification: A Leap in Prompt Engineering
Explore the Chain-of-Verification prompt engineering method, an important step towards reducing hallucinations in large language models, ensuring reliable and factual AI responses.
Graph of Thoughts: A New Paradigm for Elaborate Problem-Solving in Large Language Models
Discover how Graph of Thoughts aims to revolutionize prompt engineering, and LLMs more broadly, enabling more flexible and human-like problem-solving.
Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models
Thought Propagation is a prompt engineering technique that instructs LLMs to identify and tackle a series of problems that are similar to the original query, and then use the solutions to these similar problems to either directly generate a new answer or formulate a detailed action plan that refines the original solution.
While the above should get you to a spot where you can begin engineering effective prompts, the following resources may provide some additional depth and/or alternative views that you might find helpful.
Mastering Generative AI and Prompt Engineering: A Practical Guide for Data Scientists [eBook] from Data Science Horizons
The ebook provides an in-depth understanding of generative AI and prompt engineering, covering key concepts, best practices, and real-world applications. You’ll gain insights into popular AI models, learn the process of designing effective prompts, and explore the ethical considerations surrounding these technologies. Furthermore, the book includes case studies demonstrating practical applications across different industries.
Mastering Generative AI Text Prompts [eBook] from Data Science Horizons
Whether you’re a writer seeking inspiration, a content creator aiming for efficiency, an educator passionate about knowledge sharing, or a professional in need of specialized applications, Mastering Generative AI Text Prompts is your go-to resource. By the end of this guide, you’ll be equipped to harness the power of generative AI, enhancing your creativity, optimizing your workflow, and solving a wide range of problems.
The Psychology of Prompt Engineering [eBook] from Data Science Horizons
Our ebook is packed with captivating insights and practical strategies, covering a wide range of topics such as understanding human cognition and AI models, psychological principles of effective prompts, designing prompts with cognitive principles in mind, evaluating and optimizing prompts, and integrating psychological principles into your workflow. We’ve also included real-world case studies of successful prompt engineering examples, as well as an exploration of the future of prompt engineering, psychology, and the value of interdisciplinary collaboration.
Prompt Engineering Guide from DAIR.AI
Prompt engineering is a relatively new discipline for developing and optimizing prompts to efficiently use language models (LMs) for a wide variety of applications and research topics. Prompt engineering skills help to better understand the capabilities and limitations of large language models (LLMs).
Prompt Engineering Guide from Learn Prompting
Generative AI is the world’s hottest buzzword, and we have created the most comprehensive (and free) guide on how to use it. This course is tailored to non-technical readers, who may not have even heard of AI, making it the perfect starting point if you are new to Generative AI and Prompt Engineering. Technical readers will find valuable insights within our later modules.
Prompt engineering is a must-have skill for both AI engineers and LLM power users. Beyond this, prompt engineering has flourished into an AI niche career in its own right. There is no telling what the exact role for prompt engineering — or if dedicated prompt engineer roles will continue to be sought after AI professionals — but one thing is clear: knowledge of prompt engineering will never be held against you. By following the steps in this article, you should now have a great foundation to engineering your own high-performance prompts.
Who knows? Maybe you’re the next AI whisperer.
Matthew Mayo (@mattmayo13) holds a Master’s degree in computer science and a graduate diploma in data mining. As Editor-in-Chief of KDnuggets, Matthew aims to make complex data science concepts accessible. His professional interests include natural language processing, machine learning algorithms, and exploring emerging AI. He is driven by a mission to democratize knowledge in the data science community. Matthew has been coding since he was 6 years old.
