Teaching a language model to write music

What the model writes

The model writes ABC notation, a text format for notes, rhythm, and instrument parts. The browser can turn that notation into sound. For example, a prompt might ask for 80 bars in G major, with a trumpet opening alone and piano and bass joining later.

I can check some of these requirements directly: does each bar have the right number of beats? Are all the instrument parts there? Does each part stop at the requested length? A music verifier supplies the score, with extra checks for these structural requirements.

Listen to what changed

I picked three examples to compare, including one where the score got worse. They are selected examples rather than a random sample. You can listen to both versions and open the prompt and ABC notation below.

How I built the dataset

I started with music prompts from XiaomiMiMo’s MiMo-V2.6-RL-oss release. The project split contains 806 training prompts, 105 validation prompts, and 89 test prompts. These are the splits I used for this project.

I used gpt-6.1-sol as the teacher, with low reasoning effort, a fresh context for each prompt, and no tools. Each training prompt contributed one final answer. An answer had to earn a positive score and pass the syntax, rhythm, voice-count, bar-count, and no-repeat checks. That left 607 accepted answers and 199 rejected ones.

Rejected answers are saved with their failure reasons for inspection. Only the accepted answers went into training. Passing these checks still does not tell me whether a piece sounds good to a listener.

How I trained the student

The student is Qwen/Qwen3.8-27B. During training, it learned to predict the teacher’s answer one token at a time. Prompt and padding tokens are masked, so only the answer contributes to the loss.

The training setup is QLoRA: keep the base model frozen in 4-bit NF4 and train small LoRA adapters alongside it, using BF16 computation. This reduces the memory needed for training. The file I save afterward is a LoRA adapter containing the learned changes.

This is response distillation through supervised fine-tuning. The targets are the teacher’s text, without logits or reinforcement learning updates. The verifier helped me choose training examples and evaluate the results.

Base weights

4-bit NF4 · double quantization · BF16 compute

Adapter

Rank 16 · alpha 32 · dropout 0.05

Optimization

3 epochs · 114 updates · learning rate 1e-4

Batching

Microbatch 1 · accumulation 16

Context

11,008 padded tokens · gradient checkpointing

Three epochs on one H100 produced three saved adapters to compare with the base model.

Training infrastructure and evaluation

Training and evaluation ran as separate Modal jobs, each with one H100. Once training finished, the evaluation worker loaded the saved adapters and generated new compositions.

Train → save adapters → evaluate. Separate single-GPU jobs, without distributed training.

Where vLLM fits

vLLM generates the evaluation answers. It loads the base model and adapters and batches several requests on the GPU. My evaluation code sends the prompts, checks the generated ABC, and saves the scores.

For this run, I used vLLM 0.31.0 with FP8 per-tensor quantization and BF16 dtype. I set the context limit to 12,288 tokens, the batched-token limit to 4,096, and allowed up to 64 sequences at once. Prefix caching was enabled, though its speedup was not measured separately.

Training used a 4-bit base; inference used an FP8 version of the same source model with the saved adapter. Evaluation settings stayed the same across checkpoints.

What improved after training

The base model and all three checkpoints were tested on the same 105 validation prompts, with greedy decoding, seed 42, and an 8,192-token output limit. A guard stopped generations that broke its structural rules. Those failures stay in the results with zero reward.

By epoch 3, the average reward was 3.31 times the base score. But only 15.2% of answers passed every check. You can also see the pass count drop from 10 to 9 at epoch 2 even though the average score increased. That is why I track both numbers.

This is a single run, without repeated seeds or a human listening study. These results show improvement on this validation set and verifier; they do not prove that the music sounds better.

The model still loses count

At the final checkpoint, 61 answers still scored zero. The length guard stopped 52 of them, and the other nine had invalid bar durations. Keeping track of the structure over a long piece was still a problem.

The guard has a limitation too: it checks completed lines. If the model keeps writing one unfinished line, that check can miss it until the token limit. Increasing the output limit would not solve that.

Next, I want to try having the model plan the bars and voices before writing the notes, then check the structure as it generates. I would test that on the same validation prompts to see whether it actually reduces these failures.

Training and inference metrics

Student reward uses 105 validation prompts. Teacher reward is the running mean across 607 accepted training answers (final mean 0.746); these are different datasets, not a head-to-head comparison. Inference charts use five-minute sample averages; throughput is tokens per second.

Training was the short part

The final trainer reported 40m 03s. Across both recorded training worker invocations, total worker time was 1h 22m 55s. The inference worker ran for 4h 35m 33s, including roughly six minutes of startup.

Training hardware

1 × H100 per invocation

Inference hardware

1 × H100 · up to 64 sequences

Training peak allocation

44.77 GiB CUDA memory

Worker time scope

Model loading, waiting, and work inside the invocation

The timers cover different things. The trainer reports time inside training. The worker times also include loading and waiting after the function starts, but exclude earlier container provisioning. The inference time covers the whole evaluation job, including more than the final 105 validation answers.

I keep the cost breakdown and runtime records separate so I can compare the same scope of work. Training-loop time alone does not account for everything the GPU worker was doing.

Try it yourself

The final distilled model is on Hugging Face. It is the epoch-3 LoRA adapter from step 114, the checkpoint behind the results above. Load it alongside Qwen/Qwen3.8-27B; the model card includes the base revision, loading example, training settings, and evaluation limitations.

The repository includes collection, verification, training, evaluation, and playback code. The dataset includes the accepted answers, rejected answers with reasons, and validation and test prompts. You can load it with:

from datasets import load_dataset

data = load_dataset("suunny/music-teacher-distillation")
print(data)  # Inspect the published splits and schema

Start with the run instructions for environment setup and commands. Review the dataset card and its licensing notes before reuse; the published license does not override third-party or provider terms.

For me, the main result is that 607 checked examples were enough to improve the score, but not enough to make the model reliable. Before scaling up, I want to fix the bar-count and rhythm failures and see whether more pieces pass the full set of checks.