Skip to main content

​
Abstract

MRS (Music Representation) is a model and protocol suite for encoding and safely editing musical scores in AI-assisted workflows:
  • MRS-S: The canonical storage format (S-expression based) for complete, archival-quality scores with stable UUID identifiers
  • MRS-Ops: The typed operation protocol for reliable agent mutations
  • Working Set Envelopes: Scoped MRS-S fragments with task-adaptive context for bounded agent operations
  • Structural Index: Compressed global views providing structural awareness
  • Overlays: Analytical metadata attached to score regions
Agents read MRS-S and write MRS-Ops. The orchestrator validates operations progressively and applies them to canonical state. See: /MRS-Specification-RFC#abstract

​
The Vision

MRS enables AI-assisted composition and orchestration of full scores—symphonies, vocal arrangements, film cues—through iterative collaboration with a human arranger/orchestrator. The goal is not “AI music generation.” The goal is professional score authorship with Dorico-level semantic detail, where an AI system can propose precise edits without breaking the score, and a human can approve, revise, and audition those edits as part of a normal workflow. For a detailed feasibility analysis, see: The Problem & Why MRS is Feasible

​
The Human-in-the-Loop Workflow

MRS is designed for the way musicians actually work: a mix of notation, listening, and hands-on input. A typical loop:
  1. Arranger input: enter material by notation, a MIDI keyboard, or a rough sketch (melody, chords, rhythm, orchestration notes).
  2. Bounded AI task: the orchestrator creates a Working Set (a scoped score fragment plus task-specific context) and asks an agent to perform a focused operation (e.g., “orchestrate these 8 bars for winds,” “add phrase slurs,” “write a bass line supporting this harmony”).
  3. Validated application: the agent returns typed operations (MRS-Ops). The orchestrator validates them (syntax → references → permissions → musical rules) and applies them deterministically.
  4. Human review + audition: the human reviews semantic diffs (what changed and why) and can audition playback previews. The human remains the authority for style, taste, and final approval.
MRS stays notation-first. Playback/rendering can be integrated for feedback, but the canonical model remains focused on score semantics.

​
Motivation

MRS addresses three fundamental challenges in AI-assisted music composition:

​
The Scale Problem

A full orchestral score is too large for any context window. Editing measure 847 should not require loading measures 1-846.
Score TypeApprox. Tokens
Lead sheet~2K
Piano sonata~40K
Full orchestra~1.5M

​
The Context Problem

Musical decisions require understanding surrounding material. An agent writing a countermelody needs to know phrase boundaries, harmonic rhythm, and thematic relationships—not just the target measures in isolation.

​
The Reliability Problem

Asking AI to emit complete, structurally-valid score fragments creates high failure rates:
  • Accidental omissions (agent returns less than given → content deleted)
  • Calculation errors (duration sums, absolute positions)
  • Hallucinated references (IDs that don’t exist)
MRS solves these through:
  • Separation of storage (MRS-S) and mutation (MRS-Ops) formats
  • Orchestrator authority for UUIDs and derived fields
  • Task-adaptive context views (not fixed rings)
  • Progressive validation before state mutation
  • Player-Instrument model for professional orchestration
See: /MRS-Specification-RFC#1-introduction

​
Goals

  1. Semantic completeness: Encode everything a professional engraver needs (Dorico-level granularity)
  2. Reliable agent output: Typed operations with progressive validation minimize repair loops
  3. Structural stability: UUID identifiers survive insertions and deletions
  4. Professional orchestration: Player/instrument model supports doubling, switching, condensing
  5. Task-appropriate context: Agents receive information relevant to their specific task
  6. Deterministic reconciliation: Operations produce predictable, auditable results
  7. Human auditability: Operations are semantic diffs; humans review intent, not syntax
See: /MRS-Specification-RFC#1-2-goals

​
Non-goals

MRS explicitly does not attempt to:
  • Encode detailed visual layout (beyond optional hints)
  • Provide a programming language (no loops/macros/conditionals)
  • Define synthesis/playback behavior
  • Replace domain-specific formats (e.g. audio, MIDI, engraving source)
See: /MRS-Specification-RFC#1-3-non-goals

​
Terminology

Key terms used throughout the spec:
TermDefinition
ScoreA complete musical work in MRS-S format
PlayerA performer who may play multiple instruments
InstrumentA specific sound source with fixed transposition and range
StaffA visual grouping of five lines; instruments may have multiple staves
VoiceAn independent melodic/rhythmic stream within a staff
MeasureA metrical unit bounded by barlines, identified by UUID
EventAn atomic musical occurrence (note, rest, chord), identified by UUID
SpanA relation connecting multiple events (slur, beam, hairpin)
MRS-SThe canonical storage format (S-expression based)
MRS-OpsThe typed operation protocol for agent mutations
Structural IndexA compressed structural view of the full score
Working Set EnvelopeA scoped MRS-S fragment with context for agent tasks
Context ViewTask-adaptive information beyond the edit scope
OverlayAnalytical metadata attached to score regions
OrchestratorSystem that coordinates agent tasks and manages the full score
LaneA permission boundary for a category of content
Lane BundleA predefined set of lanes for common workflows
See: /MRS-Specification-RFC#1-4-terminology

​
Architecture at a Glance

         ORCHESTRATOR (owns canonical score state)
                │
    ┌───────────┼───────────┬───────────┐
    ▼           ▼           ▼           ▼
 Harmony    Melody      Orch.      Expression
  Agent      Agent      Agent        Agent
    │           │           │           │
    │        (read MRS-S, write MRS-Ops)
    │           │           │           │
    └───────────┴───────────┴───────────┘
                │
         Working Set Envelopes
         (scoped MRS-S + task-adaptive context)
Agents read MRS-S content with task-appropriate context. Agents write typed MRS-Ops operations. The orchestrator validates progressively and applies to canonical state. See: Architecture Overview and Orchestrator Contract for details.