# How a deepfake is made and caught

> A process map exported from forks.world (https://forks.world). Author: Brian. Source: https://forks.world/p/cmv0kpr3z003yjn06ou1iinlz
> It describes who does what, in what order, and with which technology. Read the plain-language sections first; the JSON at the end is the exact, complete data.

## Summary

- 17 steps across 2 layers
- Persona: The maker, A viewer
- Technology / systems: The AI model, A platform, Detection tools

## Pain points

Where this process hurts, worst first:

- **Post it** (How a deepfake is made and caught; painful): Used for satire and film, but also for fraud, harassment and political deception

## How to read this file

- A process is made of **layers**. The main layer is the top level; any step of kind `subprocess` opens a detail layer with its own lanes and steps.
- **Lanes** are either persona (people/roles) or technology (systems). Each step belongs to one lane; `using` lists tools a person uses for that step.
- Step kinds: `start`, `task`, `decision` (its arrows carry the answer as a label), `subprocess`, `end`.

## How a deepfake is made and caught

Lanes:
- The maker: person / role (id `maker`)
- The AI model: technology / system (id `model`)
- A platform: technology / system (id `platform`)
- Detection tools: technology / system (id `detect`)
- A viewer: person / role (id `viewer`)

Steps, in the order they run:

1. **Collect photos, video and audio of a target** [start; The maker; id `f1`]
   - then Train a model on their face or voice
2. **Train a model on their face or voice** [task; The AI model; id `f2`]
   - then Generate fake video or audio
3. **Generate fake video or audio** [task; The AI model; id `f3`]
   - then Polish: lighting, lip sync, compression
4. **Polish: lighting, lip sync, compression** [task; The maker; id `f4`]
   - then Post it
5. **Post it** [task; The maker; id `f5`] ⚠ PAIN POINT (painful): Used for satire and film, but also for fraud, harassment and political deception
   - then Does a label or a detector flag it?
6. **Does a label or a detector flag it?** [decision; A platform; id `f6`]
   - if "Yes" go to Label, limit or remove it; if "No" go to It spreads
7. **Label, limit or remove it** [task; A platform; id `f7`]
   - then Does the viewer check the source?
8. **Does the viewer check the source?** [decision; A viewer; id `f9`]
   - if "Yes" go to Look for provenance data or telltale glitches; if "No" go to Believes it and shares it
9. **Look for provenance data or telltale glitches** [subprocess; Detection tools; id `f10`], opens the detail layer "Spotting a fake"
   - then Fact-checkers and the person respond
10. **Fact-checkers and the person respond** [end; A platform; id `f12`]
11. **Believes it and shares it** [task; A viewer; id `f11`]
   - then Fact-checkers and the person respond
12. **It spreads** [task; A viewer; id `f8`]
   - then Does the viewer check the source?

```mermaid
flowchart LR
  subgraph l_maker["The maker"]
    n_f1(["Collect photos, video and audio of a target"])
    n_f4["Polish: lighting, lip sync, compression"]
    n_f5["Post it"]
  end
  subgraph l_model["The AI model"]
    n_f2["Train a model on their face or voice"]
    n_f3["Generate fake video or audio"]
  end
  subgraph l_platform["A platform"]
    n_f6{"Does a label or a detector flag it?"}
    n_f7["Label, limit or remove it"]
    n_f12(["Fact-checkers and the person respond"])
  end
  subgraph l_detect["Detection tools"]
    n_f10[["Look for provenance data or telltale glitches"]]
  end
  subgraph l_viewer["A viewer"]
    n_f8["It spreads"]
    n_f9{"Does the viewer check the source?"}
    n_f11["Believes it and shares it"]
  end
  n_f1 --> n_f2
  n_f2 --> n_f3
  n_f3 --> n_f4
  n_f4 --> n_f5
  n_f5 --> n_f6
  n_f6 -->|"Yes"| n_f7
  n_f6 -->|"No"| n_f8
  n_f7 --> n_f9
  n_f8 --> n_f9
  n_f9 -->|"Yes"| n_f10
  n_f9 -->|"No"| n_f11
  n_f10 --> n_f12
  n_f11 --> n_f12
```

### Spotting a fake (detail of step "Look for provenance data or telltale glitches")

Lanes:
- Detection tools: technology / system (id `detect`)

Steps, in the order they run:

1. **Does the file carry a signed record of where it came from?** [start; Detection tools; id `d1`]
   - if "Yes" go to Check that the record is genuine; if "No" go to Look for glitches: odd blinking, warped teeth, mismatched shadows
2. **Check that the record is genuine** [task; Detection tools; id `d2`]
   - then Run an AI detector, which is not reliable on its own
3. **Run an AI detector, which is not reliable on its own** [task; Detection tools; id `d4`]
   - then Find the original, and ask who gains from it
4. **Find the original, and ask who gains from it** [end; Detection tools; id `d5`]
5. **Look for glitches: odd blinking, warped teeth, mismatched shadows** [task; Detection tools; id `d3`]
   - then Run an AI detector, which is not reliable on its own

```mermaid
flowchart LR
  subgraph l_detect["Detection tools"]
    n_d1(["Does the file carry a signed record of where it came from?"])
    n_d2["Check that the record is genuine"]
    n_d3["Look for glitches: odd blinking, warped teeth, mismatched shadows"]
    n_d4["Run an AI detector, which is not reliable on its own"]
    n_d5(["Find the original, and ask who gains from it"])
  end
  n_d1 -->|"Yes"| n_d2
  n_d1 -->|"No"| n_d3
  n_d2 --> n_d4
  n_d3 --> n_d4
  n_d4 --> n_d5
```

## Full data (JSON)

```json
{
  "title": "How a deepfake is made and caught",
  "maps": {
    "m_root": {
      "id": "m_root",
      "lanes": [
        {
          "id": "maker",
          "name": "The maker",
          "type": "person"
        },
        {
          "id": "model",
          "name": "The AI model",
          "type": "system"
        },
        {
          "id": "platform",
          "name": "A platform",
          "type": "system"
        },
        {
          "id": "detect",
          "name": "Detection tools",
          "type": "system"
        },
        {
          "id": "viewer",
          "name": "A viewer",
          "type": "person"
        }
      ],
      "steps": [
        {
          "id": "f1",
          "kind": "start",
          "lane": "maker",
          "next": [
            {
              "to": "f2"
            }
          ],
          "uses": [],
          "label": "Collect photos, video and audio of a target"
        },
        {
          "id": "f2",
          "kind": "task",
          "lane": "model",
          "next": [
            {
              "to": "f3"
            }
          ],
          "uses": [],
          "label": "Train a model on their face or voice"
        },
        {
          "id": "f3",
          "kind": "task",
          "lane": "model",
          "next": [
            {
              "to": "f4"
            }
          ],
          "uses": [],
          "label": "Generate fake video or audio"
        },
        {
          "id": "f4",
          "kind": "task",
          "lane": "maker",
          "next": [
            {
              "to": "f5"
            }
          ],
          "uses": [],
          "label": "Polish: lighting, lip sync, compression"
        },
        {
          "id": "f5",
          "kind": "task",
          "lane": "maker",
          "next": [
            {
              "to": "f6"
            }
          ],
          "pain": {
            "note": "Used for satire and film, but also for fraud, harassment and political deception",
            "level": 2
          },
          "uses": [],
          "label": "Post it"
        },
        {
          "id": "f6",
          "kind": "decision",
          "lane": "platform",
          "next": [
            {
              "to": "f7",
              "label": "Yes"
            },
            {
              "to": "f8",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Does a label or a detector flag it?"
        },
        {
          "id": "f7",
          "kind": "task",
          "lane": "platform",
          "next": [
            {
              "to": "f9"
            }
          ],
          "uses": [],
          "label": "Label, limit or remove it"
        },
        {
          "id": "f8",
          "kind": "task",
          "lane": "viewer",
          "next": [
            {
              "to": "f9"
            }
          ],
          "uses": [],
          "label": "It spreads"
        },
        {
          "id": "f9",
          "kind": "decision",
          "lane": "viewer",
          "next": [
            {
              "to": "f10",
              "label": "Yes"
            },
            {
              "to": "f11",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Does the viewer check the source?"
        },
        {
          "id": "f10",
          "kind": "subprocess",
          "lane": "detect",
          "next": [
            {
              "to": "f12"
            }
          ],
          "uses": [],
          "child": "m_detect",
          "label": "Look for provenance data or telltale glitches"
        },
        {
          "id": "f11",
          "kind": "task",
          "lane": "viewer",
          "next": [
            {
              "to": "f12"
            }
          ],
          "uses": [],
          "label": "Believes it and shares it"
        },
        {
          "id": "f12",
          "kind": "end",
          "lane": "platform",
          "next": [],
          "uses": [],
          "label": "Fact-checkers and the person respond"
        }
      ],
      "title": "How a deepfake is made and caught",
      "parent": null
    },
    "m_detect": {
      "id": "m_detect",
      "lanes": [
        {
          "id": "detect",
          "name": "Detection tools",
          "type": "system"
        }
      ],
      "steps": [
        {
          "id": "d1",
          "kind": "start",
          "lane": "detect",
          "next": [
            {
              "to": "d2",
              "label": "Yes"
            },
            {
              "to": "d3",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Does the file carry a signed record of where it came from?"
        },
        {
          "id": "d2",
          "kind": "task",
          "lane": "detect",
          "next": [
            {
              "to": "d4"
            }
          ],
          "uses": [],
          "label": "Check that the record is genuine"
        },
        {
          "id": "d3",
          "kind": "task",
          "lane": "detect",
          "next": [
            {
              "to": "d4"
            }
          ],
          "uses": [],
          "label": "Look for glitches: odd blinking, warped teeth, mismatched shadows"
        },
        {
          "id": "d4",
          "kind": "task",
          "lane": "detect",
          "next": [
            {
              "to": "d5"
            }
          ],
          "uses": [],
          "label": "Run an AI detector, which is not reliable on its own"
        },
        {
          "id": "d5",
          "kind": "end",
          "lane": "detect",
          "next": [],
          "uses": [],
          "label": "Find the original, and ask who gains from it"
        }
      ],
      "title": "Spotting a fake",
      "parent": {
        "map": "m_root",
        "step": "f10"
      }
    }
  }
}
```