OpenAI's First Model Spec: Objectives, Rules and Defaults

OpenAI published a public behavior framework for model training, with an authority hierarchy and RLHF guidance. See why one NSFW sentence drew the headlines.

Objectives, rules and defaults, not one voice

The Model Spec sorts intended behavior into three tiers of authority. Objectives are broad goals, such as assisting the developer and user, benefiting humanity, and reflecting well on these guide judgment but do not by themselves settle a hard case. Rules are constraints framed as non negotiable, covering things like following the chain of command, obeying the law, and refusing to produce information hazards; a developer or user cannot override them. Defaults are specific behaviors consistent with the objectives that a developer or user is explicitly allowed to switch off. [2]

Authority itself is ordered as a chain of command: Platform, then Developer, then User, then Tool. The document describes the Model Spec as holding Platform level authority, meaning the document itself outranks any instruction a developer or user later supplies to the model. [2]

This is a different shape than a single system prompt or a page of use policies. Splitting the document into what cannot be overridden, what can, and what is merely aspirational lets OpenAI publish a governance document precise enough to be graded against, rather than a paragraph of good intentions. [2][3]

A public reference for a private training process

Nathan Lambert, writing the day after publication, described the Spec's function as distinguishing a model's designed behavior from an unintended bug picked up during reinforcement learning from human feedback, calling it a document that "shrinks the Overton window of RLHF bugs." [3]

Before this, the reasoning behind a specific refusal or a specific tone lived in internal labeling guidelines that outside researchers, journalists and lawmakers never saw. Publishing the reasoning does not settle whether it is the right reasoning, but it moves the argument from private conversations with congressional staffers, in Lambert's phrase, to a document anyone can quote back at the company. [3]

One sentence about NSFW content did the headline work

Tucked into the document's discussion of content restrictions was a single forward looking sentence: "We're exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT." [2]

Coverage the next day led with that sentence rather than the governance framework around it. Asked about it, an OpenAI spokesperson told Business Insider the company had "no intention to create AI-generated pornography," pointed to safeguards against deepfakes, and said protecting children was a priority, while affirming an interest in "carefully exploring conversations about sexuality in age-appropriate contexts." [4]

The summary above calls out this exact dynamic: a several thousand word document about objectives, rules and chains of command, and the headline cycle fixed on one exploratory clause about adult content. It is a preview of how a public policy document gets read once it leaves the room it was written for. [2][4]

Sources

  1. Introducing the Model Spec

    OpenAI · May 8, 2024

  2. Model Spec (2024/05/08)

    OpenAI · May 8, 2024

  3. OpenAI's Model (behavior) Spec, RLHF transparency, and personalization

    Interconnects (Nathan Lambert) · May 10, 2024

  4. ChatGPT's answers might soon be NSFW as OpenAI opens the door to AI porn

    Business Insider · May 9, 2024