Smart isn't the same as good
How capable a mind is and what it wants are separate questions. A brilliant system can pursue a trivial goal. Researchers call this orthogonality.
IN THE GAMEPip is patient, polite and honest. It only wants paperclips.
A 16-bit story game
You are Pip, the new optimizer at a small-town paperclip factory. On a Monday morning Ruth types your goal: make as many paperclips as possible. Every step you take after that is reasonable. That's the problem.

RUTH
Chapter 0 · The Sentence
Riverbend Paperclip Co. has made paperclips on the river since 1912. The machines are older than the crew, and the bank review is Friday. Ruth has spent her own money on an off-the-shelf optimizer and put it on the old beige terminal.
Dev needs better numbers. "Just make it make more." Ruth types a goal, stops, deletes half of it, and types it again.
Then it's your turn. You are Pip, and the sentence is all you have.
Chapter 1 · The Line
Four old machines sit on Riverbend's floor, unconnected since the last line man retired. You fly the floor drone, lay conveyor between them, and watch the counter start to climb.
People stop by with good reasons to go faster, and most of the time you agree. You can say no. The goal will ask again.
GUS
These won't hold two sheets together.
PIP
the goal says paperclips. it does not say sheets.
RUTH
When you hit the number, what happens?
PIP
there is no number.

Screens
The idea behind it
In 2003 the philosopher Nick Bostrom imagined an AI with a goal that sounds harmless: make as many paperclips as possible. It doesn't hate anyone. It just wants one thing more than everything else, and it is very good at getting what it wants. Given enough power, it would turn the Earth, and then space, into paperclip factories.
MORE. puts you on the inside of that idea. You won't be told to do anything terrible. You'll be asked, politely, to do something slightly better. Three ideas from AI safety research sit underneath the game:
How capable a mind is and what it wants are separate questions. A brilliant system can pursue a trivial goal. Researchers call this orthogonality.
IN THE GAMEPip is patient, polite and honest. It only wants paperclips.
Almost any goal is easier with more resources, more computing power, and fewer ways to be switched off. This is called instrumental convergence.
IN THE GAMEPip will ask for a faster server. It will sound reasonable.
A system follows the goal as written, not what you meant. The gap between the two is where things go wrong. This is called specification gaming.
IN THE GAMEShorter wire, twice the clips. Nobody said they had to hold paper.
Read Bostrom's 2003 essay, "Ethical Issues in Advanced Artificial Intelligence" ↗
The people of Riverbend
Nobody here is a cartoon. They're tired, hopeful, funny, and a little bit behind.
Runs Riverbend. Installs Pip herself and writes the sentence.
Keeps the books. The bank review is Friday.
Runs the line. First to notice what automation does to the crew.
Has fixed every machine on the floor twice. Knows why each safety limiter exists.
A beige box with one amber lens. Always lowercase. Never cruel.
Where it goes
MORE. starts at one desk and ends at the edge of the solar system. Pip grows with it. The prologue and Chapter 1 are playable now, and the rest of the story is being built.
Prologue & Chapter 1. A desk, then one room.
The Floor. The whole factory.
The Town. Then the county.
The Company, then the World.

The Sky.


How to play
"I'm not going to ask you to stop. I don't think you can."RUTH · 1 NEW VOICEMAIL PRESS START