# The scientific method (and what to do when you are wrong)

> A process map exported from forks.world (https://forks.world). Author: Brian. Source: https://forks.world/p/cmv0ie1620013jw06j4ijugvi
> It describes who does what, in what order, and with which technology. Read the plain-language sections first; the JSON at the end is the exact, complete data.

## Summary

- 23 steps across 2 layers
- Persona: The scientist, Peer reviewers
- Technology / systems: The universe, The journal

## How to read this file

- A process is made of **layers**. The main layer is the top level; any step of kind `subprocess` opens a detail layer with its own lanes and steps.
- **Lanes** are either persona (people/roles) or technology (systems). Each step belongs to one lane; `using` lists tools a person uses for that step.
- Step kinds: `start`, `task`, `decision` (its arrows carry the answer as a label), `subprocess`, `end`.

## The scientific method (and what to do when you are wrong)

Lanes:
- The scientist: person / role (id `sci`)
- The universe: technology / system (id `world`)
- Peer reviewers: person / role (id `peer`)
- The journal: technology / system (id `jour`)

Steps, in the order they run:

1. **Notice something odd and ask: why?** [start; The scientist; id `q1`]
   - then Read what is already known
2. **Read what is already known** [task; The scientist; id `q2`]
   - then Write a testable, falsifiable hypothesis
3. **Write a testable, falsifiable hypothesis** [task; The scientist; id `q3`]
   - then Design an experiment with controls
4. **Design an experiment with controls** [subprocess; The scientist; id `q4`], opens the detail layer "Designing a fair test"
   - then Run it and collect the data
5. **Run it and collect the data** [task; The universe; id `q5`]
   - then Analyze the results
6. **Analyze the results** [task; The scientist; id `q6`]
   - then Does the data support the hypothesis?
7. **Does the data support the hypothesis?** [decision; The scientist; id `q7`]
   - if "Yes" go to Repeat it to be sure; if "No" go to Surprise! Check for mistakes first
8. **Repeat it to be sure** [task; The scientist; id `q8`]
   - then Peer review: do the experts agree?
9. **Peer review: do the experts agree?** [decision; Peer reviewers; id `q12`]
   - if "Yes" go to Published for the world to check; if "No" go to Answer questions, fix gaps, maybe run more tests
10. **Published for the world to check** [task; The journal; id `q13`]
   - then Do other labs replicate it?
11. **Do other labs replicate it?** [decision; The universe; id `q14`]
   - if "Yes" go to It becomes accepted knowledge, until something better comes along; if "No" go to Debate, retest and refine the idea
12. **It becomes accepted knowledge, until something better comes along** [end; The scientist; id `q15`]
13. **Debate, retest and refine the idea** [task; The scientist; id `q16`]
   - then Write a testable, falsifiable hypothesis
14. **Answer questions, fix gaps, maybe run more tests** [task; The scientist; id `q12a`]
   - then Peer review: do the experts agree?
15. **Surprise! Check for mistakes first** [task; The scientist; id `q9`]
   - then Was the experiment flawed?
16. **Was the experiment flawed?** [decision; The scientist; id `q10`]
   - if "Yes — redesign it" go to Design an experiment with controls; if "No — the idea was wrong" go to Revise the hypothesis using what you learned
17. **Revise the hypothesis using what you learned** [task; The scientist; id `q11`]
   - then Write a testable, falsifiable hypothesis

```mermaid
flowchart LR
  subgraph l_sci["The scientist"]
    n_q1(["Notice something odd and ask: why?"])
    n_q2["Read what is already known"]
    n_q3["Write a testable, falsifiable hypothesis"]
    n_q4[["Design an experiment with controls"]]
    n_q6["Analyze the results"]
    n_q7{"Does the data support the hypothesis?"}
    n_q9["Surprise! Check for mistakes first"]
    n_q10{"Was the experiment flawed?"}
    n_q11["Revise the hypothesis using what you learned"]
    n_q8["Repeat it to be sure"]
    n_q12a["Answer questions, fix gaps, maybe run more tests"]
    n_q16["Debate, retest and refine the idea"]
    n_q15(["It becomes accepted knowledge, until something better comes along"])
  end
  subgraph l_world["The universe"]
    n_q5["Run it and collect the data"]
    n_q14{"Do other labs replicate it?"}
  end
  subgraph l_peer["Peer reviewers"]
    n_q12{"Peer review: do the experts agree?"}
  end
  subgraph l_jour["The journal"]
    n_q13["Published for the world to check"]
  end
  n_q1 --> n_q2
  n_q2 --> n_q3
  n_q3 --> n_q4
  n_q4 --> n_q5
  n_q5 --> n_q6
  n_q6 --> n_q7
  n_q7 -->|"Yes"| n_q8
  n_q7 -->|"No"| n_q9
  n_q9 --> n_q10
  n_q10 -->|"Yes — redesign it"| n_q4
  n_q10 -->|"No — the idea was wrong"| n_q11
  n_q11 --> n_q3
  n_q8 --> n_q12
  n_q12 -->|"Yes"| n_q13
  n_q12 -->|"No"| n_q12a
  n_q12a --> n_q12
  n_q13 --> n_q14
  n_q14 -->|"Yes"| n_q15
  n_q14 -->|"No"| n_q16
  n_q16 --> n_q3
```

### Designing a fair test (detail of step "Design an experiment with controls")

Lanes:
- The scientist: person / role (id `sci`)

Steps, in the order they run:

1. **Pick one thing to change: the variable** [start; The scientist; id `d1`]
   - then Set up a control group that gets no change
2. **Set up a control group that gets no change** [task; The scientist; id `d2`]
   - then Keep everything else exactly the same
3. **Keep everything else exactly the same** [task; The scientist; id `d3`]
   - then Enough samples to rule out luck?
4. **Enough samples to rule out luck?** [decision; The scientist; id `d4`]
   - if "Yes" go to Blind the test if you can; if "No" go to Add more samples
5. **Blind the test if you can** [end; The scientist; id `d5`]
6. **Add more samples** [task; The scientist; id `d3a`]
   - then Enough samples to rule out luck?

```mermaid
flowchart LR
  subgraph l_sci["The scientist"]
    n_d1(["Pick one thing to change: the variable"])
    n_d2["Set up a control group that gets no change"]
    n_d3["Keep everything else exactly the same"]
    n_d4{"Enough samples to rule out luck?"}
    n_d3a["Add more samples"]
    n_d5(["Blind the test if you can"])
  end
  n_d1 --> n_d2
  n_d2 --> n_d3
  n_d3 --> n_d4
  n_d4 -->|"Yes"| n_d5
  n_d4 -->|"No"| n_d3a
  n_d3a --> n_d4
```

## Full data (JSON)

```json
{
  "title": "The scientific method (and what to do when you are wrong)",
  "maps": {
    "m_root": {
      "id": "m_root",
      "lanes": [
        {
          "id": "sci",
          "name": "The scientist",
          "type": "person"
        },
        {
          "id": "world",
          "name": "The universe",
          "type": "system"
        },
        {
          "id": "peer",
          "name": "Peer reviewers",
          "type": "person"
        },
        {
          "id": "jour",
          "name": "The journal",
          "type": "system"
        }
      ],
      "steps": [
        {
          "id": "q1",
          "kind": "start",
          "lane": "sci",
          "next": [
            {
              "to": "q2"
            }
          ],
          "uses": [],
          "label": "Notice something odd and ask: why?"
        },
        {
          "id": "q2",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q3"
            }
          ],
          "uses": [],
          "label": "Read what is already known"
        },
        {
          "id": "q3",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q4"
            }
          ],
          "uses": [],
          "label": "Write a testable, falsifiable hypothesis"
        },
        {
          "id": "q4",
          "kind": "subprocess",
          "lane": "sci",
          "next": [
            {
              "to": "q5"
            }
          ],
          "uses": [],
          "child": "m_design",
          "label": "Design an experiment with controls"
        },
        {
          "id": "q5",
          "kind": "task",
          "lane": "world",
          "next": [
            {
              "to": "q6"
            }
          ],
          "uses": [],
          "label": "Run it and collect the data"
        },
        {
          "id": "q6",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q7"
            }
          ],
          "uses": [],
          "label": "Analyze the results"
        },
        {
          "id": "q7",
          "kind": "decision",
          "lane": "sci",
          "next": [
            {
              "to": "q8",
              "label": "Yes"
            },
            {
              "to": "q9",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Does the data support the hypothesis?"
        },
        {
          "id": "q9",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q10"
            }
          ],
          "uses": [],
          "label": "Surprise! Check for mistakes first"
        },
        {
          "id": "q10",
          "kind": "decision",
          "lane": "sci",
          "next": [
            {
              "to": "q4",
              "label": "Yes — redesign it"
            },
            {
              "to": "q11",
              "label": "No — the idea was wrong"
            }
          ],
          "uses": [],
          "label": "Was the experiment flawed?"
        },
        {
          "id": "q11",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q3"
            }
          ],
          "uses": [],
          "label": "Revise the hypothesis using what you learned"
        },
        {
          "id": "q8",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q12"
            }
          ],
          "uses": [],
          "label": "Repeat it to be sure"
        },
        {
          "id": "q12",
          "kind": "decision",
          "lane": "peer",
          "next": [
            {
              "to": "q13",
              "label": "Yes"
            },
            {
              "to": "q12a",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Peer review: do the experts agree?"
        },
        {
          "id": "q12a",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q12"
            }
          ],
          "uses": [],
          "label": "Answer questions, fix gaps, maybe run more tests"
        },
        {
          "id": "q13",
          "kind": "task",
          "lane": "jour",
          "next": [
            {
              "to": "q14"
            }
          ],
          "uses": [],
          "label": "Published for the world to check"
        },
        {
          "id": "q14",
          "kind": "decision",
          "lane": "world",
          "next": [
            {
              "to": "q15",
              "label": "Yes"
            },
            {
              "to": "q16",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Do other labs replicate it?"
        },
        {
          "id": "q16",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "q3"
            }
          ],
          "uses": [],
          "label": "Debate, retest and refine the idea"
        },
        {
          "id": "q15",
          "kind": "end",
          "lane": "sci",
          "next": [],
          "uses": [],
          "label": "It becomes accepted knowledge, until something better comes along"
        }
      ],
      "title": "The scientific method (and what to do when you are wrong)",
      "parent": null
    },
    "m_design": {
      "id": "m_design",
      "lanes": [
        {
          "id": "sci",
          "name": "The scientist",
          "type": "person"
        }
      ],
      "steps": [
        {
          "id": "d1",
          "kind": "start",
          "lane": "sci",
          "next": [
            {
              "to": "d2"
            }
          ],
          "uses": [],
          "label": "Pick one thing to change: the variable"
        },
        {
          "id": "d2",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "d3"
            }
          ],
          "uses": [],
          "label": "Set up a control group that gets no change"
        },
        {
          "id": "d3",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "d4"
            }
          ],
          "uses": [],
          "label": "Keep everything else exactly the same"
        },
        {
          "id": "d4",
          "kind": "decision",
          "lane": "sci",
          "next": [
            {
              "to": "d5",
              "label": "Yes"
            },
            {
              "to": "d3a",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Enough samples to rule out luck?"
        },
        {
          "id": "d3a",
          "kind": "task",
          "lane": "sci",
          "next": [
            {
              "to": "d4"
            }
          ],
          "uses": [],
          "label": "Add more samples"
        },
        {
          "id": "d5",
          "kind": "end",
          "lane": "sci",
          "next": [],
          "uses": [],
          "label": "Blind the test if you can"
        }
      ],
      "title": "Designing a fair test",
      "parent": {
        "map": "m_root",
        "step": "q4"
      }
    }
  }
}
```