# How a chatbot like ChatGPT or Claude is made

> A process map exported from forks.world (https://forks.world). Author: Brian. Source: https://forks.world/p/cmv0kppjs003jjn06e9er1os6
> It describes who does what, in what order, and with which technology. Read the plain-language sections first; the JSON at the end is the exact, complete data.

## Summary

- 25 steps across 4 layers
- Persona: The AI lab, Testers and red teams, Users
- Technology / systems: Training data, A giant computer cluster

## Pain points

Where this process hurts, worst first:

- **Collect text, code and images from the web and licensed sources** (How a chatbot like ChatGPT or Claude is made; painful): Contested: whether training on copyrighted work needs permission or payment
- **People use it, give thumbs up and down, and find new failures** (How a chatbot like ChatGPT or Claude is made; painful): Models can state falsehoods with total confidence

## How to read this file

- A process is made of **layers**. The main layer is the top level; any step of kind `subprocess` opens a detail layer with its own lanes and steps.
- **Lanes** are either persona (people/roles) or technology (systems). Each step belongs to one lane; `using` lists tools a person uses for that step.
- Step kinds: `start`, `task`, `decision` (its arrows carry the answer as a label), `subprocess`, `end`.

## How a chatbot like ChatGPT or Claude is made

Lanes:
- Training data: technology / system (id `data`)
- The AI lab: person / role (id `team`)
- A giant computer cluster: technology / system (id `gpu`)
- Testers and red teams: person / role (id `testers`)
- Users: person / role (id `users`)

Steps, in the order they run:

1. **Collect text, code and images from the web and licensed sources** [subprocess; The AI lab; id `l1`] ⚠ PAIN POINT (painful): Contested: whether training on copyrighted work needs permission or payment, opens the detail layer "Building the training set"
   - then Clean and filter the data
2. **Clean and filter the data** [task; The AI lab; id `l2`]
   - then Pretraining: predict the next word, trillions of times
3. **Pretraining: predict the next word, trillions of times** [subprocess; A giant computer cluster; id `l3`], opens the detail layer "Pretraining"
   - then Fine-tune on examples of good answers
4. **Fine-tune on examples of good answers** [subprocess; The AI lab; id `l4`], opens the detail layer "Teaching it to follow instructions"
   - then Humans rank the answers and the model learns what people prefer
5. **Humans rank the answers and the model learns what people prefer** [task; Testers and red teams; id `l5`]
   - then Safety testing: try to make it do harm
6. **Safety testing: try to make it do harm** [task; Testers and red teams; id `l6`]
   - then Does it pass the safety and quality tests?
7. **Does it pass the safety and quality tests?** [decision; The AI lab; id `l7`]
   - if "Yes" go to Release it to users; if "No" go to Fix, retrain and retest
8. **Release it to users** [task; Users; id `l8`]
   - then People use it, give thumbs up and down, and find new failures
9. **People use it, give thumbs up and down, and find new failures** [task; Users; id `l9`] ⚠ PAIN POINT (painful): Models can state falsehoods with total confidence
   - then Update and release the next version
10. **Update and release the next version** [task; The AI lab; id `l10`]
   - then Collect text, code and images from the web and licensed sources
11. **Fix, retrain and retest** [task; The AI lab; id `l6a`]
   - then Safety testing: try to make it do harm

```mermaid
flowchart LR
  subgraph l_team["The AI lab"]
    n_l1[["Collect text, code and images from the web and licensed sources"]]
    n_l2["Clean and filter the data"]
    n_l4[["Fine-tune on examples of good answers"]]
    n_l7{"Does it pass the safety and quality tests?"}
    n_l6a["Fix, retrain and retest"]
    n_l10["Update and release the next version"]
  end
  subgraph l_gpu["A giant computer cluster"]
    n_l3[["Pretraining: predict the next word, trillions of times"]]
  end
  subgraph l_testers["Testers and red teams"]
    n_l5["Humans rank the answers and the model learns what people prefer"]
    n_l6["Safety testing: try to make it do harm"]
  end
  subgraph l_users["Users"]
    n_l8["Release it to users"]
    n_l9["People use it, give thumbs up and down, and find new failures"]
  end
  n_l1 --> n_l2
  n_l2 --> n_l3
  n_l3 --> n_l4
  n_l4 --> n_l5
  n_l5 --> n_l6
  n_l6 --> n_l7
  n_l7 -->|"Yes"| n_l8
  n_l7 -->|"No"| n_l6a
  n_l6a --> n_l6
  n_l8 --> n_l9
  n_l9 --> n_l10
  n_l10 --> n_l1
```

### Building the training set (detail of step "Collect text, code and images from the web and licensed sources")

Lanes:
- The AI lab: person / role (id `team`)
- Training data: technology / system (id `data`)

Steps, in the order they run:

1. **Crawl public web pages** [start; The AI lab; id `d1`]
   - then Add books, code and licensed data
2. **Add books, code and licensed data** [task; The AI lab; id `d2`]
   - then Remove duplicates, spam and private information
3. **Remove duplicates, spam and private information** [task; The AI lab; id `d3`]
   - then Is the mix balanced enough?
4. **Is the mix balanced enough?** [decision; The AI lab; id `d4`]
   - if "Yes" go to The training set is ready; if "No" go to Remove duplicates, spam and private information
5. **The training set is ready** [end; Training data; id `d5`]

```mermaid
flowchart LR
  subgraph l_team["The AI lab"]
    n_d1(["Crawl public web pages"])
    n_d2["Add books, code and licensed data"]
    n_d3["Remove duplicates, spam and private information"]
    n_d4{"Is the mix balanced enough?"}
  end
  subgraph l_data["Training data"]
    n_d5(["The training set is ready"])
  end
  n_d1 --> n_d2
  n_d2 --> n_d3
  n_d3 --> n_d4
  n_d4 -->|"Yes"| n_d5
  n_d4 -->|"No"| n_d3
```

### Teaching it to follow instructions (detail of step "Fine-tune on examples of good answers")

Lanes:
- The AI lab: person / role (id `team`)
- A giant computer cluster: technology / system (id `gpu`)
- Testers and red teams: person / role (id `testers`)

Steps, in the order they run:

1. **Write or collect high-quality example conversations** [start; The AI lab; id `t1`]
   - then Train the model to answer like the examples
2. **Train the model to answer like the examples** [task; A giant computer cluster; id `t2`]
   - then Does it follow instructions?
3. **Does it follow instructions?** [decision; Testers and red teams; id `t3`]
   - if "Yes" go to Move on to preference training; if "No" go to Write or collect high-quality example conversations
4. **Move on to preference training** [end; The AI lab; id `t4`]

```mermaid
flowchart LR
  subgraph l_team["The AI lab"]
    n_t1(["Write or collect high-quality example conversations"])
    n_t4(["Move on to preference training"])
  end
  subgraph l_gpu["A giant computer cluster"]
    n_t2["Train the model to answer like the examples"]
  end
  subgraph l_testers["Testers and red teams"]
    n_t3{"Does it follow instructions?"}
  end
  n_t1 --> n_t2
  n_t2 --> n_t3
  n_t3 -->|"Yes"| n_t4
  n_t3 -->|"No"| n_t1
```

### Pretraining (detail of step "Pretraining: predict the next word, trillions of times")

Lanes:
- A giant computer cluster: technology / system (id `gpu`)

Steps, in the order they run:

1. **Show the model a sentence with the last word hidden** [start; A giant computer cluster; id `p1`]
   - then The model guesses the word
2. **The model guesses the word** [task; A giant computer cluster; id `p2`]
   - then Is the error low enough yet?
3. **Is the error low enough yet?** [decision; A giant computer cluster; id `p3`]
   - if "Yes" go to It has absorbed grammar, facts and patterns; if "No" go to Nudge billions of numbers (weights) to guess better
4. **It has absorbed grammar, facts and patterns** [end; A giant computer cluster; id `p6`]
5. **Nudge billions of numbers (weights) to guess better** [task; A giant computer cluster; id `p5`]
   - then Show the model a sentence with the last word hidden

```mermaid
flowchart LR
  subgraph l_gpu["A giant computer cluster"]
    n_p1(["Show the model a sentence with the last word hidden"])
    n_p2["The model guesses the word"]
    n_p3{"Is the error low enough yet?"}
    n_p5["Nudge billions of numbers (weights) to guess better"]
    n_p6(["It has absorbed grammar, facts and patterns"])
  end
  n_p1 --> n_p2
  n_p2 --> n_p3
  n_p3 -->|"Yes"| n_p6
  n_p3 -->|"No"| n_p5
  n_p5 --> n_p1
```

## Full data (JSON)

```json
{
  "title": "How a chatbot like ChatGPT or Claude is made",
  "maps": {
    "m_data": {
      "id": "m_data",
      "lanes": [
        {
          "id": "team",
          "name": "The AI lab",
          "type": "person"
        },
        {
          "id": "data",
          "name": "Training data",
          "type": "system"
        }
      ],
      "steps": [
        {
          "id": "d1",
          "kind": "start",
          "lane": "team",
          "next": [
            {
              "to": "d2"
            }
          ],
          "uses": [],
          "label": "Crawl public web pages"
        },
        {
          "id": "d2",
          "kind": "task",
          "lane": "team",
          "next": [
            {
              "to": "d3"
            }
          ],
          "uses": [],
          "label": "Add books, code and licensed data"
        },
        {
          "id": "d3",
          "kind": "task",
          "lane": "team",
          "next": [
            {
              "to": "d4"
            }
          ],
          "uses": [],
          "label": "Remove duplicates, spam and private information"
        },
        {
          "id": "d4",
          "kind": "decision",
          "lane": "team",
          "next": [
            {
              "to": "d5",
              "label": "Yes"
            },
            {
              "to": "d3",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Is the mix balanced enough?"
        },
        {
          "id": "d5",
          "kind": "end",
          "lane": "data",
          "next": [],
          "uses": [],
          "label": "The training set is ready"
        }
      ],
      "title": "Building the training set",
      "parent": {
        "map": "m_root",
        "step": "l1"
      }
    },
    "m_root": {
      "id": "m_root",
      "lanes": [
        {
          "id": "data",
          "name": "Training data",
          "type": "system"
        },
        {
          "id": "team",
          "name": "The AI lab",
          "type": "person"
        },
        {
          "id": "gpu",
          "name": "A giant computer cluster",
          "type": "system"
        },
        {
          "id": "testers",
          "name": "Testers and red teams",
          "type": "person"
        },
        {
          "id": "users",
          "name": "Users",
          "type": "person"
        }
      ],
      "steps": [
        {
          "id": "l1",
          "kind": "subprocess",
          "lane": "team",
          "next": [
            {
              "to": "l2"
            }
          ],
          "pain": {
            "note": "Contested: whether training on copyrighted work needs permission or payment",
            "level": 2
          },
          "uses": [],
          "child": "m_data",
          "label": "Collect text, code and images from the web and licensed sources"
        },
        {
          "id": "l2",
          "kind": "task",
          "lane": "team",
          "next": [
            {
              "to": "l3"
            }
          ],
          "uses": [],
          "label": "Clean and filter the data"
        },
        {
          "id": "l3",
          "kind": "subprocess",
          "lane": "gpu",
          "next": [
            {
              "to": "l4"
            }
          ],
          "uses": [],
          "child": "m_pretrain",
          "label": "Pretraining: predict the next word, trillions of times"
        },
        {
          "id": "l4",
          "kind": "subprocess",
          "lane": "team",
          "next": [
            {
              "to": "l5"
            }
          ],
          "uses": [],
          "child": "m_tune",
          "label": "Fine-tune on examples of good answers"
        },
        {
          "id": "l5",
          "kind": "task",
          "lane": "testers",
          "next": [
            {
              "to": "l6"
            }
          ],
          "uses": [],
          "label": "Humans rank the answers and the model learns what people prefer"
        },
        {
          "id": "l6",
          "kind": "task",
          "lane": "testers",
          "next": [
            {
              "to": "l7"
            }
          ],
          "uses": [],
          "label": "Safety testing: try to make it do harm"
        },
        {
          "id": "l7",
          "kind": "decision",
          "lane": "team",
          "next": [
            {
              "to": "l8",
              "label": "Yes"
            },
            {
              "to": "l6a",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Does it pass the safety and quality tests?"
        },
        {
          "id": "l6a",
          "kind": "task",
          "lane": "team",
          "next": [
            {
              "to": "l6"
            }
          ],
          "uses": [],
          "label": "Fix, retrain and retest"
        },
        {
          "id": "l8",
          "kind": "task",
          "lane": "users",
          "next": [
            {
              "to": "l9"
            }
          ],
          "uses": [],
          "label": "Release it to users"
        },
        {
          "id": "l9",
          "kind": "task",
          "lane": "users",
          "next": [
            {
              "to": "l10"
            }
          ],
          "pain": {
            "note": "Models can state falsehoods with total confidence",
            "level": 2
          },
          "uses": [],
          "label": "People use it, give thumbs up and down, and find new failures"
        },
        {
          "id": "l10",
          "kind": "task",
          "lane": "team",
          "next": [
            {
              "to": "l1"
            }
          ],
          "uses": [],
          "label": "Update and release the next version"
        }
      ],
      "title": "How a chatbot like ChatGPT or Claude is made",
      "parent": null
    },
    "m_tune": {
      "id": "m_tune",
      "lanes": [
        {
          "id": "team",
          "name": "The AI lab",
          "type": "person"
        },
        {
          "id": "gpu",
          "name": "A giant computer cluster",
          "type": "system"
        },
        {
          "id": "testers",
          "name": "Testers and red teams",
          "type": "person"
        }
      ],
      "steps": [
        {
          "id": "t1",
          "kind": "start",
          "lane": "team",
          "next": [
            {
              "to": "t2"
            }
          ],
          "uses": [],
          "label": "Write or collect high-quality example conversations"
        },
        {
          "id": "t2",
          "kind": "task",
          "lane": "gpu",
          "next": [
            {
              "to": "t3"
            }
          ],
          "uses": [],
          "label": "Train the model to answer like the examples"
        },
        {
          "id": "t3",
          "kind": "decision",
          "lane": "testers",
          "next": [
            {
              "to": "t4",
              "label": "Yes"
            },
            {
              "to": "t1",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Does it follow instructions?"
        },
        {
          "id": "t4",
          "kind": "end",
          "lane": "team",
          "next": [],
          "uses": [],
          "label": "Move on to preference training"
        }
      ],
      "title": "Teaching it to follow instructions",
      "parent": {
        "map": "m_root",
        "step": "l4"
      }
    },
    "m_pretrain": {
      "id": "m_pretrain",
      "lanes": [
        {
          "id": "gpu",
          "name": "A giant computer cluster",
          "type": "system"
        }
      ],
      "steps": [
        {
          "id": "p1",
          "kind": "start",
          "lane": "gpu",
          "next": [
            {
              "to": "p2"
            }
          ],
          "uses": [],
          "label": "Show the model a sentence with the last word hidden"
        },
        {
          "id": "p2",
          "kind": "task",
          "lane": "gpu",
          "next": [
            {
              "to": "p3"
            }
          ],
          "uses": [],
          "label": "The model guesses the word"
        },
        {
          "id": "p3",
          "kind": "decision",
          "lane": "gpu",
          "next": [
            {
              "to": "p6",
              "label": "Yes"
            },
            {
              "to": "p5",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Is the error low enough yet?"
        },
        {
          "id": "p5",
          "kind": "task",
          "lane": "gpu",
          "next": [
            {
              "to": "p1"
            }
          ],
          "uses": [],
          "label": "Nudge billions of numbers (weights) to guess better"
        },
        {
          "id": "p6",
          "kind": "end",
          "lane": "gpu",
          "next": [],
          "uses": [],
          "label": "It has absorbed grammar, facts and patterns"
        }
      ],
      "title": "Pretraining",
      "parent": {
        "map": "m_root",
        "step": "l3"
      }
    }
  }
}
```