Run a Four-Week AI Practice Cycle

The four-week cycle prevents consumption without evidence

A common failure: you consume massive amounts of AI content (courses, papers, videos, tutorials) and at the end have no evidence that anything changed. You watched 20 hours of tutorials. You read 4 papers. You attended 3 webinars. But can you build anything? Can you explain it? Can you improve it? The four-week cycle prevents this by forcing each week to produce tangible evidence.

The cycle works because it separates learning (understanding) from delivery (proving it works) into distinct, measurable weeks. Each week has one job. Each week produces something you can point to.

Week 1: Orientation — establish a baseline

Purpose: Take a snapshot of your current state before you start learning. Define terms, list constraints, and run one realistic baseline task. You're not learning yet; you're marking the starting line.

The baseline task: Spend 2-3 hours attempting to do the thing your outcome requires, without instruction or a course. This is uncomfortable on purpose.

Example (LLM code generation): Your outcome is "I will use an LLM to write and test Python functions." Your Week 1 baseline: spend 2 hours trying to get Claude (or your LLM of choice) to write a function that sorts a list of dictionaries by a field, including unit tests. Don't follow a tutorial. Just try. Save:

1. Your first prompt and Claude's response
2. Two failed attempts and why they failed
3. A note: "What felt unclear to me?"

After 2 hours, stop. This is your baseline artifact. You'll compare Week 4 against it.

Example (RAG system): Your outcome is "I will build a Q&A chatbot using retrieval-augmented generation." Week 1 baseline: spend 2 hours trying to build a very simple RAG system without following a course. Use whatever tools you think might work. Get something running, even if broken. Save:

1. Your architecture diagram (even if wrong)
2. The code you wrote
3. Results: Did it answer a test question correctly?
4. A note: "What do I not understand about RAG?"

This baseline is your evidence that you started as a beginner at this specific task.

Why Week 1 baseline matters

A baseline serves three purposes:

1. Reality check: You discover what you don't know before the course claims to teach it. If you can't get a single function from an LLM in 2 hours, the course has to teach you why, not just how.

2. Motivation later: In Week 4, when you're shipping your artifact, you compare it to Week 1. The gap is visible and real.

3. Honest assessment: You're not guessing at your skill level; you've tested it.

Week 2: Guided repetition — learn by following and logging

Purpose: Follow a structured course, tutorial, or worked example step-by-step. But don't just copy. Maintain a decision log that captures every choice the instructor makes and every choice you make differently.

The decision log: A document with three columns: (1) What the instructor did, (2) What I did, (3) Why I chose differently and what changed.

Example (code generation):

# Decision Log: LLM Code Generation (Week 2)

| Instructor's choice | My choice | Why I chose differently | Result |
|---|---|---|---|
| System prompt: "You are a Python expert. Write clean, well-tested code." | System prompt: "You are a Python expert. Write code optimized for readability. Include docstrings. Write unit tests." | I wanted to emphasize documentation and testing explicitly, because the first attempt ignored edge cases. | The generated code was more defensive. It included checks for empty lists and None values. Quality went from "works" to "works + handles edge cases." |
| Temperature: 0.7 | Temperature: 0 | I wanted deterministic output for reproducibility. This is a code generation task, not creative writing. | Generation time was identical. Outputs were identical across 3 runs at temp=0, but varied at 0.7. Trade-off: no randomness = more predictable for testing. |
| Prompt format: Direct instruction "Write a function that does X" | Prompt format: Specification format "Function name: X. Inputs: [types]. Outputs: [types]. Requirements: [list]. Edge cases: [list]." | Structured input seemed more precise than narrative. | Output quality improved. The function handled 3 edge cases I listed but hadn't mentioned in the narrative prompt. This taught me: *structure in the input leads to structure in the output.* |

Key insight: Explicit, structured prompts beat narrative prompts for code generation by ~40% success rate (illustrative estimate, 5-sample experiment).

This log is more valuable than the code you produce. It shows you were thinking, not just copying.

Example (RAG system):

# Decision Log: RAG Chatbot (Week 2)

| Instructor's choice | My choice | Why | Result |
|---|---|---|---|
| Vector DB: Pinecone | Vector DB: Chroma (local) | I wanted to iterate fast without paying. Course used Pinecone, but for a prototype, local is fine. | Chroma works fine for 1000 docs. Takes ~500ms to retrieve, vs Pinecone's ~50ms. Trade-off: speed vs. free local iteration. |
| Chunk size: 1000 tokens | Chunk size: 500 tokens | Smaller chunks might be more specific to queries. | Smaller chunks improved retrieval precision (measuring against a 10-question test set): 8/10 correct at 500-token chunks vs 6/10 at 1000-token chunks. Latency nearly identical. |
| Embedding model: OpenAI | Embedding model: all-MiniLM-L6-v2 (open source) | Cost, and testing open-source alternatives. | The open model is 99% as good as OpenAI on my test set. Cost is $0 vs. ~$0.01 per call. For prototyping, open is better. |

The log forces you to notice choices, not just follow steps.

Week 3: Variation — run isolation experiments

Purpose: Change one variable at a time, measure the difference. This is where learning locks in, because you're running small, controlled experiments instead of following instructions.

Pick one dimension to vary. Examples:

• Model variation (code generation): Keep your prompts constant. Generate code 5 times with Claude, then 5 times with GPT-4. Compare: correctness, latency, token count, code style.
• Prompt variation (RAG): Keep the model and retrieval constant. Try 3 different prompt structures: (1) basic "answer this", (2) few-shot with examples, (3) chain-of-thought. Measure: accuracy on a 10-question test set.
• Chunk size variation (RAG): Keep everything else the same. Test chunk sizes 256, 512, 1024. Measure: retrieval precision and latency.

Document your experiments:

# Week 3 Experiments: Prompt Structure in RAG

**Hypothesis:** A chain-of-thought prompt ("think step by step before answering") improves accuracy compared to a simple prompt.

**Test set:** 10 questions from our internal docs

**Experiment 1: Basic prompt**
Prompt: "Answer this question based on the retrieved documents: {question}"
Results: 7/10 correct. Average response time: 800ms.
Failure examples: "How do I reset my password?" — answered with generic advice, not our specific process.

**Experiment 2: Few-shot prompt**
Prompt: "Here are 2 examples: [Q1: answered correctly], [Q2: answered correctly]. Now answer: {question}"
Results: 9/10 correct. Average response time: 1200ms.
Failure examples: One edge case where the LLM over-applied the example format.

**Experiment 3: Chain-of-thought prompt**
Prompt: "Think step-by-step. First, identify what the question is asking. Second, find the most relevant documents. Third, extract the answer. Answer: {question}"
Results: 9/10 correct. Average response time: 1100ms.

**Conclusion:** Few-shot and chain-of-thought are tied at 9/10. Few-shot is slightly faster (1200ms vs 1100ms for CoT, difference within variance). For our use case, few-shot wins because it's simpler to maintain (fewer lines in the prompt).

**Trade-off:** 2 more correct answers (7 vs 9) costs ~40% more latency (800ms to 1200ms). Worth it for accuracy.

This is real experimentation. You're discovering tradeoffs, not just following instructions.

Week 4: Delivery — build and ship something real

Purpose: Create a small artifact another person can inspect, use, or critique. It doesn't have to be polished. It has to be real and it has to work.

For code: Push a GitHub repo with a README, code, tests, and a decision document.

For creative work: Produce the 10 mockups, write the brief, or generate the asset set. Share it with your audience (colleague, team, target user).

For research: Write a 1000-1500 word summary of what you learned, with citations.

Example code artifact structure:

rag-chatbot-prototype/
├── README.md (how to run it, what it does, known limitations)
├── requirements.txt (dependencies)
├── src/
│   ├── embedder.py (embedding + chunking logic)
│   ├── retriever.py (vector DB querying)
│   ├── generator.py (LLM prompt + response)
│   └── app.py (main entry point)
├── tests/
│   └── test_questions.py (10 test questions + expected answers)
├── data/
│   └── sample_docs.txt (the documents being retrieved)
└── LEARNING_LOG.md (Week 1 baseline, Week 2 decisions, Week 3 experiments, Week 4 results)

The LEARNING_LOG.md is as important as the code. It shows your thinking.

Week 4 result evaluation against your rubric

In Week 1, you wrote a rubric with 3-4 criteria. Now score your Week 4 artifact against the same rubric:

Example rubric for a RAG chatbot:

| Criterion | Week 1 baseline score | Week 4 artifact score | Evidence | |---|---|---|---| | Correctness: Answers test questions without hallucinating | 1/10 (couldn't build anything in Week 1) | 9/10 (9 of 10 test questions answered correctly) | Test log in tests/test_questions.py | | Explainability: I can explain how the system works without looking at code | 2/10 (vague ideas about RAG) | 8/10 (can explain retrieval → ranking → generation pipeline, document my chunking strategy) | LEARNING_LOG.md Week 3 experiments | | Reproducibility: Someone else can run it and get the same results | 0/10 (didn't exist) | 8/10 (README + code, runs on laptop with python app.py, but requires API key) | README instructions, requirements.txt |

The delta shows real progress. Week 1 to Week 4: +8 on correctness, +6 on explainability, +8 on reproducibility.

When to extend beyond four weeks

Do not extend out of habit. Extend only when you can name a specific, blockable uncertainty that the current cycle hasn't answered.

Vague reasons to extend (don't do this):

• "I want to learn more."
• "The topic is interesting, let me go deeper."
• "I feel like I should spend more time."

Specific reasons to extend (do this):

• "My Week 4 artifact answers 80% of questions correctly, but I hallucinates 20%. I need a second week to test three mitigation strategies: (1) adding reranking, (2) limiting context window, (3) using a different embedding model."
• "I built the chatbot, but latency is 3 seconds and my requirement is <1 second. I need to measure the bottleneck and test optimizations."
• "The code works, but I don't understand why few-shot prompting improved accuracy. I want to read the paper on in-context learning and reproduce the benchmark."

Each of these is specific and blockable. You can measure success: "Does reranking help?" "Is latency <1s now?" "Can I explain few-shot learning?"

When you do extend, the new cycle inherits your rubric. You're not restarting; you're deepening. Your Week 4 artifact from Cycle 1 becomes the starting point for Cycle 2 Week 1.

The compound effect of stacking cycles

| Cycle | Topic | Week 4 artifact | Skill progression | |---|---|---|---| | Cycle 1 | RAG basics | Working chatbot (80% accuracy) | Can build and retrieve | | Cycle 2 | RAG evaluation & optimization | Same chatbot, <1s latency, 95% accuracy | Can measure and improve | | Cycle 3 | Multi-step reasoning | Chatbot that breaks complex questions into steps | Can architect more sophisticated flows | | Cycle 4 | Cost optimization | Same system, 70% cost reduction | Can balance quality + cost |

By month 4, you've shipped 4 artifacts and you have real perspective on which topics move the needle for your goals. You've learned more by doing four focused cycles than by spending 12 weeks trying to master one topic.

The evidence trail: your most valuable learning artifact

By Week 4, your evidence trail is a complete record of what you tried, what worked, why, and what changed. This evidence trail is:

• A portfolio piece (if you're job hunting: "I shipped a RAG chatbot with a documented learning process")
• A teaching tool (if you're explaining to a colleague: "Here's what we tried, here's why it worked")
• A debugging reference (if you hit the same problem in 6 months: "I remember investigating this")

Most learners throw away their learning logs. Save them. They're worth more than the final artifact, because they show thinking, not just code.

Common mistake

Do not confuse "consuming content" with "completing a learning cycle." You can watch 20 hours of tutorials, read 10 papers, attend 5 webinars, and at the end have completed zero weeks of real practice. Watching is input. Building is output. A 2-hour tutorial followed by a closed-book attempt to rebuild it from memory + a 15-minute decision log comparing your output to the tutorial = one session of a real learning cycle.

A 20-hour course with no artifact at the end is 20 hours of input with zero evidence of learning.

Each week must produce something tangible: a baseline artifact (Week 1), a decision log (Week 2), an experiment report (Week 3), a shipped artifact + rubric evaluation (Week 4). If it doesn't, you're still in consumption mode. Stop and ask: Am I building, or just watching?

The learning velocity gain from structured cycles

Research on deliberate practice (Ericsson et al., 1993) shows that structure accelerates learning. A learner who completes one structured 4-week cycle with evidence trails learns more than a learner who passively consumes for 4 weeks. The difference is striking: illustrative estimate, based on retention research, shows 70-80% retention after one month with structured cycles vs. 15-25% retention with passive consumption.

By cycle 3 (week 12), a structured learner has shipped 3 artifacts and understands the field from multiple angles. A passive learner has watched 20 hours of content and remembers fragments. The structured learner is ready to innovate; the passive learner is still trying to remember concepts.