Learning complex goals with iterated amplification
OpenAI proposes 'iterated amplification'—a new AI safety technique that could solve the alignment problem without labeled data.
Why it matters
OpenAI is publishing early-stage research on a scalable AI safety approach that decomposes complex tasks into human-verifiable sub-tasks, addressing a fundamental challenge in building aligned AI systems at scale.
The key facts
10 to knowTechnique: Iterated amplification for task decomposition
Published by: OpenAI
Stage: Early preliminary research on toy algorithmic domains
Problem addressed: Specifying complicated behaviors beyond human scale without labeled data or reward functions
Focus area: AI safety and alignment
Publication date: October 22, 2018
Focus: scalable approach to AI safety and goal specification
Stage: early research, toy algorithmic domains only
Published October 2018 by OpenAI
Addresses: alignment without labeled data or explicit reward functions
Go to the source
OpenAI Blogopenai.com
Publisher excerpt: We’re proposing an AI safety technique called iterated amplification that lets us specify complicated behaviors and goals that are beyond human scale, by demonstrating how to decompose a task into simpler sub-tasks, rather than by providing labeled data or a reward function. Although this idea is…