Skip to content

Langchain Prompting with python

Introduction

In today's rapidly evolving landscape of artificial intelligence and natural language processing, Large Language Models (LLMs) like OpenAI's GPT series have become pivotal in developing intelligent applications. LangChain is a framework designed to simplify the integration of LLMs into your applications by providing tools for prompt management, chaining, and more.

This comprehensive guide delves deep into the art and science of prompting in LangChain with python, aiming to equip you with the knowledge and skills to craft effective prompts and integrate them seamlessly with Ollama and python.

Understanding Prompts

A prompt is the input text given to a language model to elicit a response. It serves as the starting point for the model's text generation process. In the context of LLMs, the quality and structure of the prompt significantly influence the output.

For example, if you input is :

Explain the theory of natural language processing in simple terms please.

The LLM will generate a response based on this prompt. Before diving into the topic, it's helpful to understand some key terms related to large language models (LLMs):

  • Zero-shot learning: This technique involves using an LLM without any specific examples, relying on its pre-trained pattern recognition abilities like your beloved generalist OpenAI LLM 🤓
  • Few-shot learning: In this approach, you provide a small set of concrete examples for the LLM to learn from, enabling it to understand how to perform a task.
  • Fine-tuning: This method involves taking an existing pre-trained model and re-training it on new data to adapt it to a specific task or domain.

In this post we will be focusing only on Zero-shot learning LLM, to be precise we will be using the llama3.2 model from Ollama server.

The Importance of Prompt Engineering

Prompt Engineering is the practice of designing and refining prompts to obtain the desired output from a language model. As LLMs do not possess consciousness or understanding, the way you phrase your prompts guides their responses.

Effective prompt engineering can:

  • Improve the relevance and accuracy of the model's output.
  • Reduce ambiguity and misinterpretation.
  • Enable the model to perform complex tasks by providing clear instructions.

LangChain Prompt Templates

In LangChain, a Prompt Template is a reusable and parameterized way to generate prompts. It allows you to define a prompt with placeholders for variables, which can be filled in dynamically during runtime.

Components of a Prompt Template

A prompt template can include:

  • Instructions: Guidelines for the language model on how to formulate the response.
  • Variable Placeholders: Dynamic parts of the prompt that will be replaced with actual values.
  • Examples: Sample inputs and outputs to guide the model.
  • Questions: The core query or task you want the model to address.

Creating Prompt Templates with Variable Inputs

Prompt templates can be tailored with variables to make them dynamic and versatile in order to reuse them as you want.

from langchain import PromptTemplate

# Define a prompt template with a single variable
prompt_template = """
You are a helpful assistant.
Question: {question}
Answer in simple terms.
"""

prompt = PromptTemplate.from_template(template=prompt_template)

# Format the prompt with an actual question
formatted_prompt = prompt.format(question="What is the capital of France?")

print(formatted_prompt)

We can observe this kind of output : 

You are a helpful assistant.
Question: What is the capital of France?
Answer in simple terms.

Multiple Variable Prompts

Example:

from langchain import PromptTemplate

# Define a prompt template with multiple variables
prompt_template = """
You are a language translator.
Translate the following sentence into {language}:
"{sentence}"
"""

prompt = PromptTemplate(
    template=prompt_template,
    input_variables=["sentence", "language"]
)

# Format the prompt with actual values
formatted_prompt = prompt.format(
    sentence="Hello, how are you?",
    language="Spanish"
)

print(formatted_prompt)

We can observe this kind of output : 

You are a language translator.
Translate the following sentence into Spanish:
"Hello, how are you?"

Integrating prompts with Ollama LLM

To get responses from our LLM using LangChain, we need to pass the formatted prompt to the model. Before diving into langchain make sure to have :

  • Installed LangChain and the required dependencies:

pip install langchain langchain_ollama
- Set up your Ollama server according to the official documentation qnd it's running

Example:

from langchain_ollama import ChatOllama
from decouple import config  # For loading environment variables

# Instantiate the OpenAI LLM
llm = ChatOllama(
    base_url="http://0.0.0.0:11434",
    model="llama3.2", 
    callback_manager = CallbackManager([StreamingStdOutCallbackHandler()]))

# Define the prompt template
prompt_template = """
You are a knowledgeable historian.
Question: {question}
Provide a detailed answer.
"""

prompt = PromptTemplate.from_template(template=prompt_template)

# Format the prompt
formatted_prompt = prompt.format(question="Who was Alexander the Great?")

# Get the model's prediction
response = llm.predict(formatted_prompt)

print(response)

Efficient Prompt Generation Techniques

To optimize your LLM workflows, it's a good practice to create prompt templates that are both reusable and adaptable like below.

def create_prompt(question, role="expert", style="detailed"):
    prompt_template = f"""
    You are a {role}.
    Question: {{question}}
    Provide a {style} answer.
    """
    prompt = PromptTemplate.from_template(template=prompt_template)
    return prompt.format(question=question)

# Usage
formatted_prompt = create_prompt(
    question="What are the health benefits of green tea?",
    role="nutritionist",
    style="comprehensive"
)

response = llm.predict(formatted_prompt)
print(response)

Few-Shot Prompting

Few-Shot Prompting is just the action to provide examples within the prompt to guide the model.

prompt_template = """
Translate the following English sentences to French:

English: "Good morning."
French: "Bonjour."

English: "How are you?"
French: "Comment ça va?"

English: "{sentence}"
French:
"""

prompt = PromptTemplate.from_template(template=prompt_template)
formatted_prompt = prompt.format(sentence="I love programming.")

response = llm.predict(formatted_prompt)
print(response)

This approach can be both effective and limiting, depending on the model and its strengths.

When is few-shot prompting effective

  • Well-defined tasks: Few-shot prompting works well for tasks with clear, well-defined objectives, such as text classification, sentiment analysis, or question-answering.
  • Domain adaptation: When adapting a model to a new domain or task, providing relevant examples within the prompt can help the LLM quickly learn the new context and improve performance.
  • Small-scale datasets: For small-scale datasets where labeled data is scarce, few-shot prompting can be an efficient way to fine-tune a model without requiring large amounts of training data.

When might few-shot prompting not be effective

  • Complex tasks: Few-shot prompting may struggle with complex tasks that require a deeper understanding of language nuances, such as natural language generation, creative writing, or multi-step reasoning.
  • Unstructured domains: When dealing with unstructured or open-ended domains where examples are hard to define and provide, few-shot prompting might not be effective in capturing the underlying patterns and relationships.
  • Highly nuanced tasks: Few-shot prompting may not perform well on tasks that require a deep understanding of subtle language cues, context-dependent idioms, or cultural nuances.

In summary we can say few-shot prompting is a useful technique for guiding LLMs when working with well-defined tasks, small-scale datasets, and domain adaptation. However, it's essential to carefully evaluate the strengths and limitations of this approach in each specific scenario, considering the complexity of the task, the quality of the examples, and the model's ability to generalize and adapt.

Chain-of-Thought

A chain of thought is a logical sequence of thinking that involves the generation, evaluation, and refinement of ideas. In the context of conversational AI, a chain of thought refers to a series of connected thoughts generated by a model in response to a user's input. It's encourage the model to reason through a problem step-by-step.

Let's take a look at the example below: 

prompt_template = """
Solve the following problem step-by-step:

Question: {question}
Answer:
"""

prompt = PromptTemplate.from_template(template=prompt_template)
formatted_prompt = prompt.format(question="If a train travels at 60 mph for 2 hours, how far does it go?")

response = llm.predict(formatted_prompt)
print(response)

Traditional prompting typically involves providing a specific question or topic and expecting the AI to respond with an answer or fact. While this can be effective for simple queries, it has limitations

In contrast, a chain of thought approach offers several advantages:

  • Deeper understanding: By generating a series of connected ideas, the model can develop a deeper understanding of the user's intent and provide more nuanced responses.
  • Contextual relevance: The model can better capture the context of the conversation, leading to more relevant and accurate responses.
  • Increased creativity: The chain of thought approach allows for the exploration of multiple hypotheses, increasing the potential for creative and innovative responses.

Chains of Chains : build a structured course plan

In this section let's explore the SequentialChain and SimpleMemory LangChain class in order to build a structured course plan with multiples chains called SequentialChain and be a prompt 🥷

Adding SimpleMemory class to our script will enable a better memory to our Ollama LLM

from langchain.prompts import PromptTemplate
from langchain.chains.sequential import SequentialChain
from langchain.chains.llm import LLMChain
from langchain_ollama import ChatOllama
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler 
from langchain.memory import SimpleMemory
import logging

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logging.info("Script started")

# Initialize Ollama LLM
logging.info("Initializing ChatOllama LLM")

# Initialize the Ollama LLM
llm = ChatOllama(
    base_url="http://0.0.0.0:11434",
    model="llama3.2", 
    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()])
)

# Step 1: Process Additional Context and Course Theme
logging.info("Step 1: Setting up course context prompt template")
context_template = """
You are designing a course titled "{course_title}" aimed at {target_audience}.

Additional Information:
{additional_info}

Learning Objectives:
{learning_objectives}

Topics:
{topics}

Provide a concise summary of how the additional information, topics, and learning objectives can be integrated into the course structure.
"""

context_prompt = PromptTemplate(
    input_variables=["course_title", "target_audience", "additional_info", "learning_objectives", "topics"],
    template=context_template
)

context_chain = LLMChain(
    llm=llm,
    prompt=context_prompt,
    output_key="context_summary"
)

# Step 2: Generate the Outline
logging.info("Step 2: Setting up outline prompt template")
outline_template = """
Using the topics and context summary below, create a detailed outline for the course titled "{course_title}".

Target Audience:
{target_audience}

Topics:
{topics}

Context Summary:
{context_summary}

Outline Structure:
1. Course Introduction: Brief overview of course goals and structure.
2. Module Introduction: Summarize each module and its learning objectives.
3. Main Content: Present a breakdown of each module, detailing topics covered, key activities, and practical exercises.
4. Assessments: Define types of assessments to evaluate learners’ understanding.
5. Conclusion: Summary of course takeaways and final assignments or assessments.

Do not label the sections; instead, present the outline in a cohesive, instructional format.
"""

outline_prompt = PromptTemplate(
    input_variables=["course_title", "target_audience", "topics", "context_summary"],
    template=outline_template
)

outline_chain = LLMChain(
    llm=llm,
    prompt=outline_prompt,
    output_key="outline"
)

# Step 3: Write the Course Content
logging.info("Step 3: Write the Course Content")
course_content_template = """
Write engaging content for the course titled "{course_title}", aimed at {target_audience}, with approximately {word_count} words for each module, based on the following outline and topics.

Outline:
{outline}

Topics:
{topics}

Guidelines:
- Ensure content is practical and informative, providing actionable insights on each topic.
- Structure content to suit the needs of {target_audience}, with clear explanations, examples, and exercises.
- Avoid jargon where possible and ensure that all explanations are accessible and engaging.
- Include key takeaways for each module.
"""

course_content_prompt = PromptTemplate(
    input_variables=["course_title", "target_audience", "word_count", "outline", "topics"],
    template=course_content_template
)

course_content_chain = LLMChain(
    llm=llm,
    prompt=course_content_prompt,
    output_key="course_content"
)

# Step 4: Combine the chains into a SequentialChain to write the course
logging.info("Step 4: Combine the chains into a SequentialChain and generate course content")
memory = SimpleMemory()
overall_chain = SequentialChain(
    chains=[context_chain, outline_chain, course_content_chain],
    input_variables=["course_title", "target_audience", "additional_info", "learning_objectives", "topics", "word_count"],
    output_variables=["context_summary", "outline", "course_content"],
    memory=memory,
    verbose=True
)

# Prepare input variables
course_info = {
    'title': 'Introduction to Python Programming',
    'audience': 'beginners with no prior programming experience'
}

learning_objectives = """
- Understand the basics of Python syntax and structure.
- Develop problem-solving skills through coding.
- Build foundational skills for data manipulation and basic automation.
"""

topics = "variables, data types, control structures, functions, basic I/O, and error handling"
word_count = 500

additional_info = "Focus on practical examples, real-world applications, and hands-on exercises to reinforce learning."

# Create the callback manager
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])

# Run the overall chain
result = overall_chain(
    {
        "course_title": course_info['title'],
        "target_audience": course_info['audience'],
        "additional_info": additional_info,
        "learning_objectives": learning_objectives,
        "topics": topics,
        "word_count": word_count,
    },
    callbacks=callback_manager
)

# Print the final course content
print("\nFinal Course Content:\n")
print(result['course_content'])

This code sets up a LangChain based mini pipeline for generating a structured course content. It has four main steps: defining the course context, creating an outline, writing detailed content, and combining everything into a cohesive course plan.

  • Step 1: Course Context (context_chain):
    • Uses a PromptTemplate to create a summary of the course context. The prompt asks for the course title, target audience, additional details, learning objectives, and topics.
    • The LLMChain (context_chain) sends this information to the model to generate a "context summary," a concise overview of the course's purpose and structure.
  • Step 2: Course Outline (outline_chain):
    • Creates another prompt template for generating a course outline based on the context summary from Step 1.
    • The outline follows a structured format, including an introduction, module breakdowns, key activities, assessments, and a conclusion.
    • outline_chain generates this detailed course outline in a format suitable for learners.
  • Step 3: Course Content (course_content_chain):
    • This step generates detailed content for each module based on the outline and topics.
    • The prompt encourages practical, accessible, and engaging content to match the audience's needs.
    • The course_content_chain runs this prompt to create full-length content with examples, exercises, and takeaways.
  • Step 4: Sequential Execution (overall_chain):
    • Combines all three chains (context_chain, outline_chain, and course_content_chain) into a SequentialChain.
    • The SequentialChain ensures each step flows into the next, with outputs (like the context summary) being passed along to the following step.
    • overall_chain runs through all steps sequentially to produce the final course content.
  • Input Preparation and Execution:
    • Sets up input variables, such as the course title, audience, learning objectives, topics, and word count.
    • Passes these inputs into the overall_chain and executes it to generate the full course content.

In order to see the chains reasoning in action you can add pass langchain in debug mode by adding this line of code a the beginning of your script. langchain.debug = True

Best Practices

  • Be Clear and Specific: Ambiguity leads to unpredictable outputs.
  • Provide Context: Use system prompts to set the role or behavior of the assistant.
  • Use Variable Placeholders: For dynamic content, always use variables in your prompt templates.
  • Test and Iterate: Experiment with different phrasings and structures to achieve optimal results.
  • Limit Prompt Length: Be concise to reduce token usage and improve performance.

Wrap it up

Hope this article show you how prompt engineering is a crucial skill when working with language models. We only seen the tip of the LangChain iceberg, but as you can see it's a good start if you are looking a simple framework for managing prompts and integrating them with LLMs. It's also available in javascript, hope you've learn a things or two 😎

More ressources