# How an election poll is made (and why it is sometimes wrong)

> A process map exported from forks.world (https://forks.world). Author: Brian. Source: https://forks.world/p/cmv0kpolx0039jn06iexv9x87
> It describes who does what, in what order, and with which technology. Read the plain-language sections first; the JSON at the end is the exact, complete data.

## Summary

- 17 steps across 2 layers
- Persona: Pollster, People surveyed, Media, Voters
- Technology / systems: The statistical model

## Pain points

Where this process hurts, worst first:

- **Most people do not: try again and find replacements** (How an election poll is made (and why it is sometimes wrong); painful): Response rates are very low

## How to read this file

- A process is made of **layers**. The main layer is the top level; any step of kind `subprocess` opens a detail layer with its own lanes and steps.
- **Lanes** are either persona (people/roles) or technology (systems). Each step belongs to one lane; `using` lists tools a person uses for that step.
- Step kinds: `start`, `task`, `decision` (its arrows carry the answer as a label), `subprocess`, `end`.

## How an election poll is made (and why it is sometimes wrong)

Lanes:
- Pollster: person / role (id `pollster`)
- People surveyed: person / role (id `people`)
- The statistical model: technology / system (id `model`)
- Media: person / role (id `media`)
- Voters: person / role (id `voters`)

Steps, in the order they run:

1. **Decide who to measure: all adults, registered or likely voters** [start; Pollster; id `p1`]
   - then Build a sample list: phones and online panels
2. **Build a sample list: phones and online panels** [task; Pollster; id `p2`]
   - then Contact people
3. **Contact people** [task; People surveyed; id `p3`]
   - then Do they respond?
4. **Do they respond?** [decision; People surveyed; id `p4`]
   - if "Yes" go to Weight the answers so the sample looks like the country; if "No" go to Most people do not: try again and find replacements
5. **Weight the answers so the sample looks like the country** [task; The statistical model; id `p5`]
   - then Estimate who will actually vote
6. **Estimate who will actually vote** [subprocess; The statistical model; id `p6`], opens the detail layer "Who is a likely voter?"
   - then Compute the margin of error
7. **Compute the margin of error** [task; The statistical model; id `p7`]
   - then Publish the poll and the headline
8. **Publish the poll and the headline** [task; Media; id `p8`]
   - then The election happens
9. **The election happens** [task; Voters; id `p9`]
   - then Was the poll close?
10. **Was the poll close?** [decision; Pollster; id `p10`]
   - if "Yes" go to Credit for accuracy, until next time; if "No" go to Find the miss: who did not answer, who changed their mind
11. **Credit for accuracy, until next time** [end; Pollster; id `p11`]
12. **Find the miss: who did not answer, who changed their mind** [task; Pollster; id `p12`]
   - then Decide who to measure: all adults, registered or likely voters
13. **Most people do not: try again and find replacements** [task; Pollster; id `p4a`] ⚠ PAIN POINT (painful): Response rates are very low
   - then Do they respond?

```mermaid
flowchart LR
  subgraph l_pollster["Pollster"]
    n_p1(["Decide who to measure: all adults, registered or likely voters"])
    n_p2["Build a sample list: phones and online panels"]
    n_p4a["Most people do not: try again and find replacements"]
    n_p10{"Was the poll close?"}
    n_p12["Find the miss: who did not answer, who changed their mind"]
    n_p11(["Credit for accuracy, until next time"])
  end
  subgraph l_people["People surveyed"]
    n_p3["Contact people"]
    n_p4{"Do they respond?"}
  end
  subgraph l_model["The statistical model"]
    n_p5["Weight the answers so the sample looks like the country"]
    n_p6[["Estimate who will actually vote"]]
    n_p7["Compute the margin of error"]
  end
  subgraph l_media["Media"]
    n_p8["Publish the poll and the headline"]
  end
  subgraph l_voters["Voters"]
    n_p9["The election happens"]
  end
  n_p1 --> n_p2
  n_p2 --> n_p3
  n_p3 --> n_p4
  n_p4 -->|"Yes"| n_p5
  n_p4 -->|"No"| n_p4a
  n_p4a --> n_p4
  n_p5 --> n_p6
  n_p6 --> n_p7
  n_p7 --> n_p8
  n_p8 --> n_p9
  n_p9 --> n_p10
  n_p10 -->|"Yes"| n_p11
  n_p10 -->|"No"| n_p12
  n_p12 --> n_p1
```

### Who is a likely voter? (detail of step "Estimate who will actually vote")

Lanes:
- The statistical model: technology / system (id `model`)

Steps, in the order they run:

1. **Ask: how likely are you to vote?** [start; The statistical model; id `l1`]
   - then Check past voting records
2. **Check past voting records** [task; The statistical model; id `l2`]
   - then Pick a cutoff
3. **Pick a cutoff** [task; The statistical model; id `l3`]
   - then Different cutoffs give different results
4. **Different cutoffs give different results** [end; The statistical model; id `l4`]

```mermaid
flowchart LR
  subgraph l_model["The statistical model"]
    n_l1(["Ask: how likely are you to vote?"])
    n_l2["Check past voting records"]
    n_l3["Pick a cutoff"]
    n_l4(["Different cutoffs give different results"])
  end
  n_l1 --> n_l2
  n_l2 --> n_l3
  n_l3 --> n_l4
```

## Full data (JSON)

```json
{
  "title": "How an election poll is made (and why it is sometimes wrong)",
  "maps": {
    "m_lv": {
      "id": "m_lv",
      "lanes": [
        {
          "id": "model",
          "name": "The statistical model",
          "type": "system"
        }
      ],
      "steps": [
        {
          "id": "l1",
          "kind": "start",
          "lane": "model",
          "next": [
            {
              "to": "l2"
            }
          ],
          "uses": [],
          "label": "Ask: how likely are you to vote?"
        },
        {
          "id": "l2",
          "kind": "task",
          "lane": "model",
          "next": [
            {
              "to": "l3"
            }
          ],
          "uses": [],
          "label": "Check past voting records"
        },
        {
          "id": "l3",
          "kind": "task",
          "lane": "model",
          "next": [
            {
              "to": "l4"
            }
          ],
          "uses": [],
          "label": "Pick a cutoff"
        },
        {
          "id": "l4",
          "kind": "end",
          "lane": "model",
          "next": [],
          "uses": [],
          "label": "Different cutoffs give different results"
        }
      ],
      "title": "Who is a likely voter?",
      "parent": {
        "map": "m_root",
        "step": "p6"
      }
    },
    "m_root": {
      "id": "m_root",
      "lanes": [
        {
          "id": "pollster",
          "name": "Pollster",
          "type": "person"
        },
        {
          "id": "people",
          "name": "People surveyed",
          "type": "person"
        },
        {
          "id": "model",
          "name": "The statistical model",
          "type": "system"
        },
        {
          "id": "media",
          "name": "Media",
          "type": "person"
        },
        {
          "id": "voters",
          "name": "Voters",
          "type": "person"
        }
      ],
      "steps": [
        {
          "id": "p1",
          "kind": "start",
          "lane": "pollster",
          "next": [
            {
              "to": "p2"
            }
          ],
          "uses": [],
          "label": "Decide who to measure: all adults, registered or likely voters"
        },
        {
          "id": "p2",
          "kind": "task",
          "lane": "pollster",
          "next": [
            {
              "to": "p3"
            }
          ],
          "uses": [],
          "label": "Build a sample list: phones and online panels"
        },
        {
          "id": "p3",
          "kind": "task",
          "lane": "people",
          "next": [
            {
              "to": "p4"
            }
          ],
          "uses": [],
          "label": "Contact people"
        },
        {
          "id": "p4",
          "kind": "decision",
          "lane": "people",
          "next": [
            {
              "to": "p5",
              "label": "Yes"
            },
            {
              "to": "p4a",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Do they respond?"
        },
        {
          "id": "p4a",
          "kind": "task",
          "lane": "pollster",
          "next": [
            {
              "to": "p4"
            }
          ],
          "pain": {
            "note": "Response rates are very low",
            "level": 2
          },
          "uses": [],
          "label": "Most people do not: try again and find replacements"
        },
        {
          "id": "p5",
          "kind": "task",
          "lane": "model",
          "next": [
            {
              "to": "p6"
            }
          ],
          "uses": [],
          "label": "Weight the answers so the sample looks like the country"
        },
        {
          "id": "p6",
          "kind": "subprocess",
          "lane": "model",
          "next": [
            {
              "to": "p7"
            }
          ],
          "uses": [],
          "child": "m_lv",
          "label": "Estimate who will actually vote"
        },
        {
          "id": "p7",
          "kind": "task",
          "lane": "model",
          "next": [
            {
              "to": "p8"
            }
          ],
          "uses": [],
          "label": "Compute the margin of error"
        },
        {
          "id": "p8",
          "kind": "task",
          "lane": "media",
          "next": [
            {
              "to": "p9"
            }
          ],
          "uses": [],
          "label": "Publish the poll and the headline"
        },
        {
          "id": "p9",
          "kind": "task",
          "lane": "voters",
          "next": [
            {
              "to": "p10"
            }
          ],
          "uses": [],
          "label": "The election happens"
        },
        {
          "id": "p10",
          "kind": "decision",
          "lane": "pollster",
          "next": [
            {
              "to": "p11",
              "label": "Yes"
            },
            {
              "to": "p12",
              "label": "No"
            }
          ],
          "uses": [],
          "label": "Was the poll close?"
        },
        {
          "id": "p12",
          "kind": "task",
          "lane": "pollster",
          "next": [
            {
              "to": "p1"
            }
          ],
          "uses": [],
          "label": "Find the miss: who did not answer, who changed their mind"
        },
        {
          "id": "p11",
          "kind": "end",
          "lane": "pollster",
          "next": [],
          "uses": [],
          "label": "Credit for accuracy, until next time"
        }
      ],
      "title": "How an election poll is made (and why it is sometimes wrong)",
      "parent": null
    }
  }
}
```