LangChain: A Framework for LLM Applications

A direct model API call gets you one request and one response. Real applications need more: fetching relevant documents first, calling tools, carrying conversation state, chaining several model calls, streaming output, retrying failures, and swapping providers. LangChain supplies standard components for those pieces so you are not rebuilding the same plumbing each time.

The framework has a reputation for over-abstraction, some of it earned by earlier versions. The current composition model is thinner than it was, but the underlying judgement call has not changed: a framework pays for itself in proportion to how many of its pieces you actually use.

What it composes

The building blocks are small and fit together in one direction. A prompt template turns variables into messages. A model takes messages and returns a response. An output parser turns that response into a typed structure. A retriever takes a query and returns documents. Tools are functions the model may call. Memory carries state across turns.

Composition chains them:

chain = prompt | model | parser
answer = chain.invoke({"question": "What changed in the refund policy?"})

Building on that interface gives you streaming, batching, and async for free across every step, which is genuinely useful — those three are tedious to implement consistently by hand, and they are where a lot of the framework's real value sits.

The retrieval path, stage by stage

Retrieval-augmented generation is the most common reason teams adopt a framework, and it is worth seeing as six distinct stages, because when the answers are bad the cause is almost never the model.

Load documents from their source. Split them into chunks — this is the stage that most determines quality, and the common failure is chunking by character count so that a table, a code block, or a numbered procedure is cut in half. Split on structure first, then size, and overlap so context spanning a boundary survives. Embed each chunk into a vector. Store those vectors in an index. Retrieve the nearest chunks for a query. Generate an answer grounded in what came back.

Debug in that order and from the retrieval end. Print the chunks the retriever returned before blaming the prompt: if the answer is not in them, no amount of prompt engineering will produce it, and the fix is in splitting, embedding, or the number of results requested. Pure vector search also misses exact identifiers — error codes, SKUs, names — so hybrid search combining keyword and vector matching is usually a meaningful improvement over either alone.

Tool calling and where agents come in

The second reason teams adopt a framework is tool calling: letting the model decide to invoke a function, then feeding the result back. The framework handles the mechanics — turning a Python function's signature and docstring into the schema the model sees, parsing the call it returns, executing it, and appending the result to the conversation.

That plumbing is genuinely repetitive, and getting it right by hand for several providers is tedious. What the framework cannot do is make tool calling reliable, and it is worth being clear about where the difficulty actually sits. Models pick the wrong tool, invent arguments, call the same tool repeatedly, or stop early. The fixes are all on your side: fewer tools rather than more, descriptions written for the model rather than for a developer, arguments validated before execution, and a hard cap on loop iterations so a confused agent cannot run indefinitely.

The same caution applies to the agent abstractions built on top. A loop that plans, calls tools, observes, and repeats is easy to start and hard to make dependable, and the failure modes — a wrong step early that everything after inherits, cost that scales with iterations rather than requests — are properties of the architecture rather than of the framework. Start with a fixed sequence of steps you control, and move to a model-directed loop only when the task genuinely cannot be expressed as one.

Framework or direct call

Reach for the framework when you need several pieces together: retrieval plus tools plus multi-step flow, streaming and async across all of them, or genuine provider portability. The plumbing is the hard part, and it is already written.

Call the API directly when the application is a prompt and a response, when you need exact control of the request payload, or when the team will spend more time reading framework source to understand behaviour than the plumbing would have taken to write. For a single-purpose feature, twenty lines of direct HTTP is often the more maintainable choice, and it stays that way.

The cost is real and worth naming. Debugging crosses layers you did not write. Version churn has been significant. The exact prompt sent to the model is one indirection away from your code. And a thin wrapper around a well-documented API can make it harder to consult that API's own documentation.

Mitigations if you do adopt it: turn on tracing so you can see the actual prompts and intermediate steps, pin versions, keep your business logic outside the chain rather than inside prompt templates, and be willing to drop to a direct call for the one step where you need precise control.

Common mistakes

Adopting the framework before a multi-step use case exists. Complexity without the matching benefit.

Debugging RAG from the prompt end. Inspect the retrieved chunks first; that is where the fault usually is.

Splitting on character count alone. Cutting tables and procedures in half is the leading cause of poor retrieval.

Running without tracing. You cannot fix what you cannot see, and the framework hides the request by design.