Research = uncertainty. Unlike building a house, you don’t know all the materials or obstacles ahead.

Visible progress tracker Have a simple Kanban board (Notion/Trello) with columns: Month → Week → Day → Done. This makes milestones not just a plan, but a visual progress bar.

Define Minimum Viable Output (MVO) Ask: What’s the smallest version of this milestone that proves progress? Example: For Collect → don’t wait until you’ve read 50 blog posts. Ten high-quality ones may be enough to proceed.

Without milestones → work naturally keeps inflating.

A milestone introduces artificial scarcity: “I only have 2 weeks to gather advice → better prioritize the most important sources.”

“Keep reading until I know enough” → “By end of Week 2, I’ll have 20 summarized advices.” “Keep synthesizing until it feels right” → “By end of Week 4, I’ll have a draft, even if messy.”

Now, even though the universe of possible advice is infinite, you’ve forced closure at each step.

Creates time pressure → reduces endless wandering. Builds momentum by giving you small wins. Forces trade-offs: if time is almost up, you’ll cut distractions more aggressively.

derailing

Exploration Log → Any recurring themes worth folding into (1) later?

prioritization

Step 4: Tight Execution Loops for (1)

Example: Friday afternoons = serendipity time.

Rapid

Step 1: Define a Sharp Focus Box Write down your main goal in one crisp sentence: “Collect and synthesize research advice on problem-driven research to develop my own meta research style.” Break this into milestones (e.g. “Collect 20 pieces of advice → Cluster key themes → Draft my meta approach”). Keep this Focus Box visible in your workspace (sticky note, Notion top bar). This serves as your anchor whenever FOMO tries to pull you away.

A good “bit to flip” has 3 properties: Universality: shared by many current methods. Constraining: it actually limits progress toward capabilities. Replaceability: you can imagine an alternative (not just removing, but replacing with something better). Bad flips: assumptions that were helpful approximations without being real bottlenecks → flipping them just adds complexity.

3. Build Trust in Your Evaluation Harness If you never rerun baselines, how do you know: your code, logging, and evaluation are sound? improvements aren’t due to hidden differences in implementation? By reproducing others’ results in your own harness, you ensure a fair playing field for comparing your new method. 👉 Otherwise, reviewers (and yourself) can’t be sure whether gains are real or artifacts.

Papers tend to report best-case or average numbers, not the messy reality: instability across seeds, catastrophic drops mid-training, reproducibility quirks. By rerunning, you see the variance, fragility, and edge cases that are invisible in benchmark tables. 👉 Schulman’s insight about instability in policy gradients (leading to TRPO) only came from seeing the failures himself, not from reported scores.

Different methods break in different ways. Running several gives you a menu of failure patterns: instability, sample inefficiency, poor generalization, etc. These failures help surface the real subproblems you need to tackle.

5. De-Risk Your Research Jumping straight into your own method is high-risk — it may fail for reasons you could’ve predicted by simply running known baselines. Testing others first is a low-risk learning stage that grounds your work in reality before you spend months on a new algorithm. 👉 It’s like scouting the terrain before building your own path.

Running others’ methods forces you to internalize their assumptions, strengths, and weaknesses. Often, you notice small tweaks that make a huge difference, or you realize why a method doesn’t scale to your problem. Schulman explicitly said: testing many methods was a way of discovering what properties mattered most (stability, variance reduction, etc.). 👉 This hands-on process sharpens your taste and informs your own design choices.

Lets you add comments anywhere on a webpage, including ChatGPT.

Highlight text on ChatGPT responses.

Add sticky notes/comments