How to properly work with prompts

[MikePendo]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

python
from transformers import  pipeline
from langchain_huggingface import HuggingFacePipeline
from langchain.prompts import PromptTemplate
#TextGenerationPipeline
model = pipeline("text-generation",
                 model="meta-llama/Llama-3.2-1B", 
                 max_new_tokens=64,
                 temperature=0.2,
                 device="cpu",
                 truncation=True)

llm = HuggingFacePipeline(pipeline=model)

template = PromptTemplate(
    input_variables=["question"],
    # template="You are a helpful assistant. Answer the following question: {question} \nAnswer:"
template="You are a helpful assistant. Answer the following question: {question} "
)

# Create the LLMChain
chain = template | llm

question = input("\nEnter question:\n")

# Execute the summarization chain
summary = chain.invoke({"question": question})
print("\n🔹 **Generated Summary:**")
print(summary)

Description

I am very new to langchain and trying understand how to properly work with prompts to get good brief answers with small local models from hugging face.

I am trying to use "meta-llama/Llama-3.2-1B" with very simple prompts and simple question like: what is capital of France?

Here is my prompt

template = PromptTemplate(
    input_variables=["question"],
    # template="You are a helpful assistant. Answer the following question: {question} \nAnswer:"
template="You are a helpful assistant. Answer the following question: {question} "
)

The following:
template="You are a helpful assistant. Answer the following question: {question} \nAnswer:"
gives the answer:
You are a helpful assistant. Answer the following question: What is the capital of France?
Answer: Paris
Explanation: The capital of France is Paris. The capital of France is Paris. The capital of France is Paris. The capital of France is Paris. The capital of France is Paris. The capital of France is Paris. The capital of France is Paris. The capital of France is Paris. The capital of France

if I use :
template="You are a helpful assistant. Answer the following question: {question} "
gives the answer
You are a helpful assistant. Answer the following question: What is the capital of France? 1. Paris 2. Lyon 3. Marseille 4. Toulouse 5. Nice 6. Strasbourg 7. Bordeaux 8. Nantes 9. Lille 10. Montpellier 11. Rennes 12. Toulon 13. Tours 14. Angers

I am not sure how that small nuance like Answer: in the template has such impact?
I have tried to use the same model with Ollama and it works as expected.

The first answer is better but still not brief an precise as I would like also there is a lot of redundant context around.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 24.3.0: Thu Jan 2 20:24:16 PST 2025; root:xnu-11215.81.4~3/RELEASE_ARM64_T6000
Python Version: 3.11.6 (main, Mar 17 2024, 14:51:29) [Clang 15.0.0 (clang-1500.1.0.2.5)]

Package Information

langchain_core: 0.3.47
langchain: 0.3.18
langsmith: 0.3.18
langchain_huggingface: 0.1.2
langchain_sample: Installed. No version info available.
langchain_text_splitters: 0.3.7
langgraph_agent: Installed. No version info available.

Optional packages not installed

langserve

Other Dependencies

aiohttp<4.0.0,>=3.8.3: Installed. No version info available.
async-timeout<5.0.0,>=4.0.0;: Installed. No version info available.
httpx: 0.28.1
huggingface-hub: 0.28.1
jsonpatch<2.0,>=1.33: Installed. No version info available.
langchain-anthropic;: Installed. No version info available.
langchain-aws;: Installed. No version info available.
langchain-cohere;: Installed. No version info available.
langchain-community;: Installed. No version info available.
langchain-core<1.0.0,>=0.3.34: Installed. No version info available.
langchain-core<1.0.0,>=0.3.45: Installed. No version info available.
langchain-deepseek;: Installed. No version info available.
langchain-fireworks;: Installed. No version info available.
langchain-google-genai;: Installed. No version info available.
langchain-google-vertexai;: Installed. No version info available.
langchain-groq;: Installed. No version info available.
langchain-huggingface;: Installed. No version info available.
langchain-mistralai;: Installed. No version info available.
langchain-ollama;: Installed. No version info available.
langchain-openai;: Installed. No version info available.
langchain-text-splitters<1.0.0,>=0.3.6: Installed. No version info available.
langchain-together;: Installed. No version info available.
langsmith-pyo3: Installed. No version info available.
langsmith<0.4,>=0.1.125: Installed. No version info available.
langsmith<0.4,>=0.1.17: Installed. No version info available.
numpy<2,>=1.26.4;: Installed. No version info available.
numpy<3,>=1.26.2;: Installed. No version info available.
openai-agents: Installed. No version info available.
opentelemetry-api: Installed. No version info available.
opentelemetry-exporter-otlp-proto-http: Installed. No version info available.
opentelemetry-sdk: Installed. No version info available.
orjson: 3.10.15
packaging: 24.2
packaging<25,>=23.2: Installed. No version info available.
pydantic: 2.10.6
pydantic<3.0.0,>=2.5.2;: Installed. No version info available.
pydantic<3.0.0,>=2.7.4: Installed. No version info available.
pydantic<3.0.0,>=2.7.4;: Installed. No version info available.
pytest: Installed. No version info available.
PyYAML>=5.3: Installed. No version info available.
requests: 2.32.3
requests-toolbelt: 1.0.0
requests<3,>=2: Installed. No version info available.
rich: Installed. No version info available.
sentence-transformers: 3.4.1
SQLAlchemy<3,>=1.4: Installed. No version info available.
tenacity!=8.4.0,<10,>=8.1.0: Installed. No version info available.
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
tokenizers: 0.21.1
transformers: 4.48.3
typing-extensions>=4.7: Installed. No version info available.
zstandard: 0.23.0

[adocherty]

Hi,

I think that LangChain is not the best place to have this question answered. It seems to be a question abou tthe model rather than the LangChain framework.

I have tried to use the same model with Ollama and it works as expected.

Is the model that you are using exactly the same (the same size and quantization)? Are other model parameters the same? Also, models are statistical if temperature is not zero, so running them multiple times may affect this.

Two resrouces you might like to investigate. If you are interested in getting output that meets specific format expectations, look at structured outputs:
https://python.langchain.com/docs/how_to/structured_output/
https://medium.com/@docherty/mastering-structured-output-in-llms-revisiting-langchain-and-json-structured-outputs-d95dfc286045

If you are interested in how to write a prompt, look at prompt engineering guides, like this one:
https://www.promptingguide.ai/

Hope this helps.

[MikePendo]

Hi,
I am not sure what to say. I tried OllamaLLM and ChatOllama and they worked as expected, without repeating or stuck in the middle, I would assume that HuggingFacePipeline might has some issue with those models.