GravityLLM vs General LLMs: Why a Specialized Model Wins at Immersive Audio Positioning
Spatial9 Engineering Team · · 14 min read
Key takeaways
- GravityLLM is trained only on spatial placement, using positions professional engineers chose in real immersive sessions.
- It answers with structured, section-timed 3D scenes that render straight to ADM, Eclipsa Audio (IAMF), binaural and Ambisonics.
- General LLMs such as ChatGPT, Claude, Gemini, Grok, Llama, Qwen, Kimi and DeepSeek give advice in prose; GravityLLM gives the mix plan.
- For video, GravityLLM moves sound with tracked on-screen objects instead of describing the scene in words.
- Assistants and GravityLLM work best together: the assistant talks, GravityLLM places, connected through MCP and A2A.
- The patent-pending Intent Fidelity Score (IFS) scores placement, brief alignment and format compliance together, and agrees with expert engineers at r = 0.89.
The problem: placement is a decision, not a paragraph
Ask a general assistant "where should the backing vocals go in a Dolby Atmos mix of this ballad?" and you get a thoughtful paragraph: keep the lead centred, spread the harmonies, add some height to the pads. All of it sounds reasonable. None of it is a mix.
An immersive mix needs numbers for every source: azimuth, elevation, distance, width, and how each of those changes over time, locked to the song's sections and tempo. It needs those numbers to be consistent across a whole album and safe on headphones, soundbars and 7.1.4 rooms. That is a narrow, precise task, and narrow, precise tasks are where specialized models shine.
What GravityLLM is
GravityLLM is a specialized language model built for spatial and immersive audio. It takes two inputs, a short brief ("intimate, vocal front and centre, open the chorus upward") and an analysis of the material (stems, instrument types, genre, sections, tempo, and for video the objects on screen), and returns a scene description: a position and motion plan for every source.
How GravityLLM was trained
General models learn by predicting the next word across trillions of words of web text. GravityLLM learns from a much smaller but far more relevant source: real immersive sessions.
- Real sessions as examples. Professional studios contribute stems, the spatial positions their engineers chose, and genre metadata. Each example pairs the material with what a skilled engineer actually did: where the lead vocal sat, how a synth pad rose in the chorus, how wide the drums were.
- Language tied to space. Every example carries a human brief. That is how the model learns that "intimate" pulls sources closer and narrower, that "concert hall" adds distance and height, and that "front centre" means a precise spot rather than a feeling.
- Structured storage. Positions, instrument types, genre classes and movement patterns are stored as structured 3D data, so the model learns geometry, not descriptions of geometry.
- Supervised training loop. The model predicts a position for every source, the prediction is compared with the engineer's choice, and the size of the gap (the loss) drives a small correction. This repeats across many sessions.
- Held-out validation. Sessions the model never trains on are used to measure how it handles new music. The version kept is the one that performs best on those unseen sessions, not the one that memorizes the training set.
A general model has never been graded on whether its backing vocals landed 30 degrees left at 15 degrees elevation. GravityLLM has been graded on exactly that, over and over.
Why specialized beats general for this job
The wider AI field has been moving the same way. Small, focused models trained on high-quality domain data regularly match or beat much larger general models on their own task, while being faster and cheaper to run. Code models, medical models and legal models all followed this path. Spatial audio is a textbook case, for five reasons.
- The right training signal. Web text contains plenty of opinions about mixing and almost no ground-truth coordinates from real immersive sessions. GravityLLM's training data is ground truth.
- The right output. GravityLLM answers in a fixed structure that a renderer can use immediately. General models answer in prose, or in JSON they invent on the spot that may not match any renderer.
- Consistency. A focused model gives stable answers for the same brief, which matters when you deliver a twelve-track album rather than one song.
- Physical and format awareness. Positions stay inside valid ranges, the lead stays anchored, low-frequency sources are not flung overhead, and motion respects tempo and sections.
- Speed and cost. A small specialized model runs fast enough to sit inside a production pipeline, a DAW workflow or a live room, not only in a chat window.
Side by side: GravityLLM and general LLMs
| What matters for placement | GravityLLM | General LLMs (ChatGPT, Claude, Gemini, Grok, Llama, Mistral) | General LLMs (DeepSeek, Qwen, Kimi, GLM, ERNIE) |
|---|---|---|---|
| Trained on | Real immersive sessions with engineer-chosen positions | Broad web, books and code | Broad web, books and code, strong Chinese-language coverage |
| Main output | Structured scene: azimuth, elevation, distance, width, motion per source | Prose, or improvised JSON on request | Prose, or improvised JSON on request |
| Hears or analyses the stems | Uses a dedicated audio analysis of stems, sections and tempo | Not by default; works from your description | Not by default; works from your description |
| Motion over time | Section-aware curves locked to the tempo map | Usually described in words | Usually described in words |
| Same brief, same song | Stable, repeatable scenes | Answers vary from run to run | Answers vary from run to run |
| Video objects | Places sound where the object is on screen and follows it | Can describe a scene, not track sources in 3D | Can describe a scene, not track sources in 3D |
| Delivery | Renders to ADM, Eclipsa Audio (IAMF), binaural and Ambisonics | Needs a separate tool and an engineer | Needs a separate tool and an engineer |
| Best use | Placing and moving every source | Brainstorming, writing, explaining | Brainstorming, writing, explaining |
General models are excellent at what they were built for. Many of them, including Claude, ChatGPT and Gemini, can already call Spatial9 through MCP, so the two work best together: the assistant handles the conversation and GravityLLM handles the geometry.
Example: the same request, two kinds of answer
Brief: "Pop ballad, 72 BPM. Intimate verses, open the last chorus upward. Keep the lead vocal still."
A general assistant typically answers like this:
Keep the lead vocal in the centre. Pan the backing vocals left and right and
maybe raise them a little in the chorus. Put the piano slightly wide, strings
behind the listener for an enveloping feel, and add some height to the pads
in the final chorus for lift.GravityLLM answers with a scene a renderer can use directly (shortened):
{
"sources": [
{ "id": "lead_vocal", "azimuth": 0, "elevation": 0, "distance": 0.9, "width": 0.10, "anchor": true },
{ "id": "bvox_l", "azimuth": -32, "elevation": 8, "distance": 1.4, "width": 0.25,
"motion": [{ "beat": 192, "elevation": 8 }, { "beat": 200, "elevation": 28 }] },
{ "id": "bvox_r", "azimuth": 32, "elevation": 8, "distance": 1.4, "width": 0.25,
"motion": [{ "beat": 192, "elevation": 8 }, { "beat": 200, "elevation": 28 }] },
{ "id": "piano", "azimuth": -18, "elevation": 0, "distance": 1.2, "width": 0.45 },
{ "id": "strings", "azimuth": 110, "elevation": 12, "distance": 2.2, "width": 0.80 },
{ "id": "pad", "azimuth": 180, "elevation": 20, "distance": 2.5, "width": 1.00,
"motion": [{ "beat": 192, "elevation": 20 }, { "beat": 208, "elevation": 55 }] }
],
"sections": [{ "label": "Final chorus", "start_beat": 192, "end_beat": 224 }]
}The first answer is advice that an engineer still has to turn into a mix. The second answer is the mix plan, timed to the song and ready to render and review.
Video: placing sound on moving objects
For video, the gap widens. Sound has to follow a car crossing the frame, a speaker turning, or a crowd behind the camera. GravityLLM receives the tracked objects from the video analysis and returns positions that move with them, frame-aligned. A general model can describe the scene in words ("the car moves left to right") but does not output a time-aligned 3D path for each sound source.
The Intent Fidelity Score (IFS): measuring what a great mix actually is
Most spatial audio quality checks look at one thing at a time. A localization test asks whether a sound landed where it was supposed to. A compliance check asks whether the file will render. Neither one asks the question that matters most to an artist or a director: did the mix deliver what I asked for? A scene can pass every technical check and still miss the brief. It can match the brief and still blur on headphones. GravityLLM is built and judged on a single score that covers all three at once: the Intent Fidelity Score (IFS), a Spatial9 innovation with a patent pending. It was presented in a peer-reviewed paper at the AES International Conference on Machine Learning and Artificial Intelligence for Audio in London in September 2025.
The three parts of IFS
IFS combines three measurements into one number:
IFS = 0.5 x PLA + 0.3 x CBA + 0.2 x FCC| Part | What it measures | How it is measured |
|---|---|---|
| PLA, Psychoacoustic Localization Accuracy | Whether each sound is heard exactly where it was meant to be | The distance between where a source is perceived and where it should be, scored against the largest possible error and confirmed in ITU-R BS.1116-3 listening tests with 15 trained listeners |
| CBA, Creative Brief Alignment | Whether the scene means what the brief said, such as "a ghostly voice drifting upward from the left rear" | Semantic comparison between the creative brief and the spatial metadata the model produced, using embeddings designed for language and spatial audio |
| FCC, Format Compliance Checks | Whether every rule of the target format is respected | Automated checks on every constraint. One failed rule sets FCC to zero, so a scene that would break on delivery can never score well |
The weights were not picked by hand. They were tuned by grid search against the ratings of 22 professional engineers, so the score rewards the same things experienced ears reward: accurate localization first, then faithful intent, with format compliance treated as a strict gate.
IFS agrees with expert ears
A quality score is only useful if it agrees with the people who judge mixes for a living. In the published validation, IFS matched expert judgement far more closely than the usual single-purpose measures:
| Measure | Agreement with expert ratings (Pearson's r) |
|---|---|
| Intent Fidelity Score (IFS) | 0.89 |
| Format compliance score alone | 0.71 |
| Localization error rate alone | 0.62 |
On the same work, GravityLLM reached a Psychoacoustic Localization Accuracy of 0.82, up 29%, 98% format compliance, up 41%, and a Creative Brief Alignment of 0.76.
Why IFS raises quality
Because GravityLLM is optimized and checked against IFS, every scene has to pass three tests at once, not one.
- Placements you can actually hear. PLA is grounded in psychoacoustics and HRTF-based localization, the way human ears and head locate sound. A position that looks right on a 3D graph but smears on headphones scores poorly, so it is not what GravityLLM delivers.
- The brief survives the mix. CBA makes the artist's words part of the score. "Intimate", "encircling" or "rising overhead" are checked against the result rather than left to interpretation.
- Nothing breaks on delivery. Because FCC is a strict gate, a scene that violates a format rule fails outright. Dolby Atmos bed limits, Eclipsa Audio height channels and object rules are enforced before anyone presses export.
Why IFS fights fatigue
Fatigue shows up in two places in immersive work, and IFS targets both.
Listener fatigue. Sources that wander without reason, images that blur between speakers, and elevation the ear cannot resolve make the brain work harder to build a stable picture. Over a full album or a feature film, that extra effort turns into tiring, harsh listening. By rewarding precise, perceptually stable localization, IFS steers GravityLLM toward scenes that feel natural and effortless, so you can listen for hours.
Engineer fatigue. Industry research on the mental load of spatial audio production describes positional revision loops: mixes adjusted again and again to reconcile what was asked for with what the format allows. Manual object positioning averages around 5.7 hours per project. When intent, perception and compliance are scored together from the first pass, the first scene is much closer to the final one. That means fewer revision rounds, fewer late-night listening sessions, and fresher ears for the creative decisions that matter.
Why it matters
As immersive audio spreads across music, film, games and VR, the volume of content far outpaces the number of trained immersive engineers. A system that is fast but sloppy creates more work, not less. IFS is what lets GravityLLM be both fast and trustworthy: every placement is measured against the brief, against human hearing and against the format, the same three things a senior engineer checks by instinct.
How we test placement quality
We hold GravityLLM to measurable standards on sessions it has never seen, and we invite anyone to run the same tests against any model.
| Measure | What it checks |
|---|---|
| Angular error | Degrees between the model's position and the engineer's position for each source |
| Anchor stability | Whether sources that must stay still (often the lead vocal) actually stay still |
| Schema validity | Whether every answer is a complete, renderable scene with values in range |
| Section timing | Whether motion starts and ends on the right sections and beats |
| Run-to-run consistency | How much the scene changes when the same brief is asked again |
| Listening review | Engineers rating the rendered result on headphones and speakers |
| Intent Fidelity Score | One combined score of localization, brief alignment and format compliance, tuned against expert ratings |
The last row matters most. A low error number means the model is close to its examples; only listening proves a mix works musically. Every Spatial9 scene is meant to be heard and approved by a person before delivery.
Why this matters for the wider AI world
The first wave of AI was about one model that does everything. The next wave is about the right model for each job, connected by open protocols. Assistants coordinate, specialized models do the precise work, and standards such as MCP and A2A link them. GravityLLM is Spatial9's specialist for space: a focused model that any assistant or agent can call when the question is "where should this sound be?"
You can try it today with your own files, from your AI assistant through Spatial9 MCP, from your own agents through A2A, or from your DAW through the Gravity Data Feed. Developers can read how plugins connect in the AI-era audio plugin guide, the spatial inference docs explain the model in depth, and the Intent Fidelity Score explained shows how every scene is judged.
FAQ
What is GravityLLM?
GravityLLM is the Spatial9 AI model for spatial and immersive audio. It turns a short brief and an analysis of your stems or video into a position and motion plan for every source, which Spatial9 renders to ADM, Eclipsa Audio (IAMF), binaural and Ambisonics.
Is GravityLLM better than ChatGPT, Claude or Gemini?
For general writing, reasoning and conversation, no; those models are built for that. For deciding where sounds should sit and move in a 3D mix, GravityLLM is the better tool, because it was trained on real immersive sessions and answers with structured, renderable positions.
How does GravityLLM compare with Chinese models such as Qwen, Kimi or DeepSeek?
The same way it compares with other general models. Qwen, Kimi, DeepSeek, GLM and ERNIE are strong general models, but they are not trained on engineer-chosen spatial positions and do not output section-timed 3D scenes for a renderer.
Can I use GravityLLM together with my AI assistant?
Yes. Add Spatial9 to ChatGPT, Claude, Gemini or any MCP-capable app, and the assistant can call Spatial9 tools that use GravityLLM for placement.
Does GravityLLM work for video?
Yes. It places and moves sound to follow tracked objects on screen, so sound comes from where the action is.
What is the Intent Fidelity Score (IFS)?
IFS is Spatial9's patent-pending quality score for spatial audio. It combines how accurately each source is heard where it belongs, how closely the scene matches the creative brief, and whether every format rule is respected. In peer-reviewed validation it agreed with expert engineers at r = 0.89, far above localization error (0.62) or compliance scores (0.71) alone.
How does IFS reduce listening fatigue?
IFS rewards stable, perceptually precise placement and penalizes positions the ear cannot resolve cleanly. That avoids the blurred images and pointless movement that make immersive listening tiring, and it reduces the revision loops that wear engineers down.
Do I still need an engineer?
GravityLLM proposes the scene; a person should always listen and approve it. Many teams use it to get to a strong first immersive mix fast, then refine by ear.
Sources
- Spatial9 documentation, spatial inference and GravityLLM training: /docs/spatial-inference
- "Leveraging Large Language Models for Intelligent Audio Object Positioning in Spatial Audio Ecosystems," AES International Conference on Machine Learning and Artificial Intelligence for Audio, London, September 2025: https://www.aes.org/e-lib
- ITU-R BS.1116-3, methods for the subjective assessment of small impairments in audio systems: https://www.itu.int/rec/R-REC-BS.1116
- Spatial9 and AudioShake, Marcos Valle in 3D case study: https://www.audioshake.ai/case-studies/marcos-valle-in-3d-with-spatial9
- Model Context Protocol specification: https://modelcontextprotocol.io/specification
- Agent2Agent (A2A) protocol: https://a2a-protocol.org
- ITU-R BS.2076 Audio Definition Model: https://www.itu.int/rec/R-REC-BS.2076
- Alliance for Open Media, IAMF specification: https://aomediacodec.github.io/iamf/