Guide Decision Guide

RAG vs fine-tuning: the answer is a decision, not a debate.

It's the most common architecture question in enterprise AI, and it's usually asked backwards — starting from the technique instead of the problem. Here's the version I walk clients through: RAG changes what the model knows at answer time. Fine-tuning changes how the model behaves. Once you frame it that way, most decisions make themselves.

Knowledge RAG’s home turf
Behavior Fine-tuning’s home turf
Both What mature systems often run

Start with what each actually does. RAG (retrieval-augmented generation) fetches relevant passages from your corpus at query time and grounds the answer in them. The model's weights never change; its knowledge is whatever the retriever finds. That means updates are instant (fix the document, fix the answer), every claim can carry a citation, and access controls can be enforced per-query. Its weakness is that it's only as good as retrieval — miss the right passage and the model improvises.

Fine-tuning continues training the model on your examples, changing the weights themselves. It's how you get consistent format, voice, and judgment on narrow tasks — classification at scale, extraction into your exact schema, output in your house style. Its weaknesses mirror RAG's strengths: knowledge baked into weights goes stale, can't cite a source, can't respect per-user permissions, and updating means retraining. Fine-tuning for knowledge is how you get a confidently outdated model.

The decision rules I actually use

  • Facts that change, need citations, or carry permissions → RAG. Policies, contracts, product docs, tickets — anything where "show your source" or "who can see this" matters.
  • Form, tone, and narrow judgment at volume → fine-tuning. Classify these 50 ways, extract into this schema, write like our brand — where examples exist by the thousand and correctness is stylistic or structural.
  • Both → grounded knowledge plus tuned behavior. A support system that retrieves the policy (RAG) and answers in your voice at your length (tuned) is the standard mature architecture.

Cost tips the tie. RAG's spend is infrastructure and retrieval quality — an index, a pipeline, and per-query tokens. Fine-tuning's is data curation (thousands of clean examples are the real price tag), training runs, and re-training whenever things drift. When uncertain, start with RAG: you'll need clean data anyway, you get citations for free, and if answers still miss on style or structure, you've just discovered your fine-tuning spec at almost no wasted cost.

Fine-tuning teaches the model to speak your language. RAG hands it your library card. Most enterprises need the library card first.

Four questions that settle it.

Answer these honestly and the architecture picks itself.
01
Does the answer need a source?
If users, auditors, or lawyers will ask "says who?" — that’s retrieval. Weights can’t cite.
02
Does the knowledge change?
Weekly policy updates mean RAG. Retraining a model to learn Tuesday’s price list is a very expensive way to be wrong by Thursday.
03
Is the skill narrow and voluminous?
Thousands of consistent examples of one task — classify, extract, reformat — is exactly what fine-tuning is for.
04
Do permissions differ by user?
Per-user access control only works at retrieval time. A fine-tuned model tells everyone everything it learned.
RAGFine-tuningHybridCitationsCost modelDecision framework

Questions, answered.

The ones every buyer asks first.
Is fine-tuning cheaper than RAG?

Usually not once you price the data work. Training runs are cheap; curating thousands of clean, representative examples is not — and it recurs every time behavior drifts. RAG costs more in infrastructure up front but updates by editing documents. For knowledge use cases, RAG is almost always the better unit economics.

Does fine-tuning stop hallucinations?

No — it shapes style and task behavior, not truthfulness, and tuning knowledge into weights can make hallucinations more confident, not rarer. Grounding answers in retrieved sources with citations is the anti-hallucination mechanism that actually audits.

What about long context windows — do they replace RAG?

For small, stable corpora, sometimes — you can stuff the documents into the prompt and skip the index. At enterprise scale (millions of documents, per-user permissions, per-query costs), retrieval remains the economical and governable architecture. Long context changes chunking strategy more than it changes the decision.

When do you genuinely need both?

When knowledge and behavior both matter: a claims assistant that retrieves policy language (RAG) but must output decisions in a strict schema and regulatory register (tuned); a support bot grounded in docs but tuned to your brand voice and length. Build the RAG layer first, then tune on the transcripts it generates.

Related work.

These usually travel together.
Service / 02
RAG systems
Retrieval pipelines that put your institutional knowledge to work. Grounded, source-cited answers on your own data — without leaking it.
Explore
Service / 11
LLM consulting
Claude, GPT, Gemini, or open-source — model selection, integration, and production tuning by someone who has shipped all of them.
Explore
Service / 05
Knowledge systems
Your documents, wikis, tickets, and contracts turned into one queryable, source-cited system that finally answers back — with the receipts.
Explore
Service / 15
AI evaluation & QA
Evals, golden test sets, and regression gates — so you know your AI works before your customers find out it doesn’t.
Explore
Service / 17
AI cost optimization
Token spend, model routing, and caching tuned until the unit economics work — often 60–80% off the monthly bill.
Explore
Service / 18
Data readiness
The unglamorous work that decides whether AI works at all: access, structure, quality, and permissions sorted before the build.
Explore

Still arguing about it internally? Send me the use case.

One call and you'll have the architecture answer with reasons attached — and if it's 'both,' the order to build them in.

Let's talk

Markets served.

Remote-first across the United States and internationally — including these markets.

New York City, New York (NY)

Los Angeles, California (CA)

Chicago, Illinois (IL)

Houston, Texas (TX)

Phoenix, Arizona (AZ)

Philadelphia, Pennsylvania (PA)

San Antonio, Texas (TX)

San Diego, California (CA)

Dallas, Texas (TX)

San Jose, California (CA)

Austin, Texas (TX)

Jacksonville, Florida (FL)

Fort Worth, Texas (TX)

Columbus, Ohio (OH)

Charlotte, North Carolina (NC)

Indianapolis, Indiana (IN)

San Francisco, California (CA)

Seattle, Washington (WA)

Denver, Colorado (CO)

Washington, District of Columbia (DC)

Boston, Massachusetts (MA)

El Paso, Texas (TX)

Nashville, Tennessee (TN)

Detroit, Michigan (MI)

Oklahoma City, Oklahoma (OK)

Portland, Oregon (OR)

Las Vegas, Nevada (NV)

Memphis, Tennessee (TN)

Louisville, Kentucky (KY)

Baltimore, Maryland (MD)

Milwaukee, Wisconsin (WI)

Albuquerque, New Mexico (NM)

Tucson, Arizona (AZ)

Fresno, California (CA)

Sacramento, California (CA)

Kansas City, Missouri (MO)

Atlanta, Georgia (GA)

Miami, Florida (FL)

Colorado Springs, Colorado (CO)

Raleigh, North Carolina (NC)

Omaha, Nebraska (NE)

Long Beach, California (CA)

Virginia Beach, Virginia (VA)

Oakland, California (CA)

Minneapolis, Minnesota (MN)

Tulsa, Oklahoma (OK)

Arlington, Texas (TX)

New Orleans, Louisiana (LA)

Wichita, Kansas (KS)

Cleveland, Ohio (OH)

Tampa, Florida (FL)

Bakersfield, California (CA)

Aurora, Colorado (CO)

Honolulu, Hawaii (HI)

Anaheim, California (CA)

Santa Ana, California (CA)

Corpus Christi, Texas (TX)

Riverside, California (CA)

Lexington, Kentucky (KY)

St. Louis, Missouri (MO)

Stockton, California (CA)

Pittsburgh, Pennsylvania (PA)

Saint Paul, Minnesota (MN)

Cincinnati, Ohio (OH)

Greensboro, North Carolina (NC)

Anchorage, Alaska (AK)

Plano, Texas (TX)

Lincoln, Nebraska (NE)

Orlando, Florida (FL)

Irvine, California (CA)

Newark, New Jersey (NJ)

Toledo, Ohio (OH)

Durham, North Carolina (NC)

Chula Vista, California (CA)

Fort Wayne, Indiana (IN)

Jersey City, New Jersey (NJ)

St. Petersburg, Florida (FL)

Laredo, Texas (TX)

Madison, Wisconsin (WI)

Chandler, Arizona (AZ)

Buffalo, New York (NY)

Lubbock, Texas (TX)

Scottsdale, Arizona (AZ)

Reno, Nevada (NV)

Glendale, Arizona (AZ)

Gilbert, Arizona (AZ)

Winston-Salem, North Carolina (NC)

North Las Vegas, Nevada (NV)

Norfolk, Virginia (VA)

Chesapeake, Virginia (VA)

Fremont, California (CA)

Garland, Texas (TX)

Richmond, Virginia (VA)

Baton Rouge, Louisiana (LA)

Boise, Idaho (ID)

San Bernardino, California (CA)

Spokane, Washington (WA)

Des Moines, Iowa (IA)

Modesto, California (CA)

Birmingham, Alabama (AL)

Tacoma, Washington (WA)

Fontana, California (CA)

Oxnard, California (CA)

Fayetteville, North Carolina (NC)

Huntsville, Alabama (AL)

Moreno Valley, California (CA)

Rochester, New York (NY)

Glendale, California (CA)

Yonkers, New York (NY)

Augusta, Georgia (GA)

Amarillo, Texas (TX)

Little Rock, Arkansas (AR)

Akron, Ohio (OH)

Shreveport, Louisiana (LA)

Grand Rapids, Michigan (MI)

Mobile, Alabama (AL)

Salt Lake City, Utah (UT)

Huntsville, Texas (TX)

Tallahassee, Florida (FL)

Overland Park, Kansas (KS)

Knoxville, Tennessee (TN)

Worcester, Massachusetts (MA)

Brownsville, Texas (TX)

New Port Richey, Florida (FL)

Jackson, Mississippi (MS)

Providence, Rhode Island (RI)

Fort Lauderdale, Florida (FL)

Sioux Falls, South Dakota (SD)

Tempe, Arizona (AZ)

Cape Coral, Florida (FL)

Springfield, Missouri (MO)

Pembroke Pines, Florida (FL)

Eugene, Oregon (OR)

Peoria, Arizona (AZ)

Corona, California (CA)

Lancaster, California (CA)

Rockford, Illinois (IL)

Salinas, California (CA)

Palmdale, California (CA)

Springfield, Massachusetts (MA)

Charleston, South Carolina (SC)

Duluth, Minnesota (MN)

London, England (ENG)

Dublin, Ireland (IRE)