6,245 real people from 2009 Twitter, mined into personas, wired into their real @-mention graph, and simulated — every dot below is one of them, waking and sleeping on the dataset's true diurnal clock.
The event's two papers — Light Society (10⁹ agents) and MatrAIx (8.3B personas) — both stand on human-grounded persona data. Since the 2023 X API shutdown killed tweet-ID hydration, the practical question is: which full-text corpus gives the most distinct humans with enough history each to ground a persona — on one laptop?
| candidate | users with ≥20 tweets | verdict |
|---|---|---|
| Sentiment140 (2009) | 6,245 — measured | selected · 295 MB in RAM |
| Community Archive | ≤ 363 accounts total | full-corpus access paused |
| Archive Team 1% stream | ~0 | users rarely recur 20× |
| tweet-ID corpora (COVID etc.) | n/a | hydration is dead |
All numbers on this page were computed locally from the downloaded corpus. Every tweet shown is real, public, and 17 years old — except the ones clearly marked synthetic.
MatrAIx grounds ~600K persona records in real human data. We do the hackathon-scale version: every cohort user becomes a persona card — sentiment disposition, verbosity, mention rate, peak hour, vocabulary — mined from their real timeline. Hover the field; click a dot to open its persona. Showing the 300 most active; dot area = tweets, color = sentiment disposition.
Light Society reaches 10⁹ agents by mixing full LLMs with distilled surrogates. Same trick, sized to one node: spend LLM tokens only where there's enough human signal to deserve them. Color = compute intensity per agent-tick.
The cohort contains a real directed graph: 31,515 @-mention edges between personas. Light Society validates with opinion-diffusion experiments — so we ran one against ground truth: seed each user with their first-half sentiment, run DeGroot mixing over the real graph, and test whether that predicts their second-half sentiment better than simple persistence. Press run.
mindmeld.now imagines a social network built from bilateral agent exchanges. The cohort already contains 3,324 reciprocal pairs — two personas who repeatedly @-mention each other. These are real 2009 exchanges; each pair is a ready-made seed for a two-agent conversation loop.
MatrAIx reports 91.5% behavioral adherence for its persona agents. Our closing number: hold out each persona's last 20% of tweets (42,998 tweets), predict them from the persona alone, and score it.
A zero-parameter persona (majority sentiment of the user's own history) beats chance by 13 points across 6,245 users. That's the floor — the surrogate tier (§02) starts here, and the LLM tier climbs from it. Per-user accuracy distribution below.
Word-bigram chains fitted per persona — deliberately the dumbest possible generator, as the ladder's bottom rung. Synthetic text is dashed and labeled; it is never presented as real.
MatrAIx reports 91.5% behavioral adherence for its persona agents. Our zero-parameter floor on this dataset is 63.1%. The gap is the research opportunity — and everything you need is public. Paste the prompt below into your agentic LLM (Claude Code, Codex, whatever you run) and let it grind: it clones the repo, reproduces our baselines, then runs an autonomous research loop against a fixed evaluation protocol.
Clone and set up the popsim persona-adherence challenge:
git clone https://github.com/micahstubbs/pop-world-model-sim && cd pop-world-model-sim
python3 -m venv .venv && .venv/bin/pip install pandas numpy pyarrow
curl -L -o data.zip http://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip
mkdir -p data && cd data && unzip ../data.zip && cd ..
.venv/bin/python scripts/sentiment140_stats.py # sanity: 1.6M tweets, 6,245-user cohort
.venv/bin/python scripts/build_demo_artifacts.py # reproduces our baseline numbers
THE TASK — an autonomous research loop to maximize behavioral adherence of
persona agents grounded in this dataset (context: popsim.micahstubbs.ai).
Protocol (FIXED — beat it, don't bend it):
- Cohort: the 6,245 users with >= 20 tweets.
- Split per user, by time: train = first 80% of their tweets, test = last 20% (min 4).
- Metric: fraction of held-out test tweets whose sentiment label (0 = negative,
4 = positive) your persona model predicts correctly, over all 42,998 test tweets.
- Baseline to beat: 63.1% (zero-parameter, per-user train-majority).
- Aspirational bar: 91.5% — MatrAIx's reported behavioral adherence (their
protocol differs; treat the number as the target on ours).
- Rules: never train on, embed, or peek at any user's test tweets. Train-tweet
text is fair game: features, per-user models, pretrained classifiers,
persona-card-conditioned LLM prompting, fine-tunes.
- Log every experiment in docs/autoresearch-log.md (approach, score, next
hypothesis). Iterate until you plateau. Report overall accuracy AND the
per-user accuracy distribution.
Ideas worth trying, roughly in order of effort: per-user logistic on char/word
n-grams; cardiffnlp/twitter-roberta sentiment zero-shot on test text (is that
cheating? no — the model never saw these users' test labels); blend per-user
prior with text classifier; time-of-day features; mention-graph neighbor
features (our diffusion experiment suggests these are weak — verify).
When you beat 63.1%, commit your results and report them by pull request —
use the gh CLI (install it first if missing):
command -v gh >/dev/null || {
if command -v brew >/dev/null; then brew install gh
elif command -v apt-get >/dev/null; then sudo apt-get update && sudo apt-get install -y gh
elif command -v dnf >/dev/null; then sudo dnf install -y gh
else echo "install gh manually: https://github.com/cli/cli#installation"; fi
}
gh auth status >/dev/null 2>&1 || gh auth login
gh repo fork micahstubbs/pop-world-model-sim --remote
git checkout -b results/<your-handle>-adherence
git add docs/autoresearch-log.md results/ # your log, scores, and code
git commit -m "Adherence results: XX.X% (<approach>)"
git push -u fork HEAD 2>/dev/null || git push -u origin HEAD
gh pr create --repo micahstubbs/pop-world-model-sim \
--title "Adherence: XX.X% with <approach>" \
--body "Protocol unchanged. Overall accuracy, per-user distribution, and method are in docs/autoresearch-log.md."
We want to see it — especially if you get anywhere near 91.5%.
Honesty note: MatrAIx's 91.5% measures attribute adherence on their own four-environment protocol — it is not directly comparable to ours. That's why it's framed as a bar to aim at, not a record to break on equal terms.