Flowers, from scratch
A small reproduction of Hugging Face's train to paint with code. At first the model could not call the library correctly; after a prompt fix and 20 GRPO-style updates on 2× H100, scored by HPSv3, it beat its untrained self on 22 of 24 held-out pairs.
Held-out, same seed: untrained → trained
8 flowers × 3 seeds; half were never seen in training. Click a painting for its code.
| Untrained | After 20 updates | |
|---|---|---|
| Mean reward | 0.060 | 0.424 |
| Valid code | 10 / 24 | 23 / 24 |
| Paired wins | – | 22 / 24 |
Caveats: the evaluation score is the training reward, and the paintings converged on one composition.
Cute cats, from scratch: an honest failure
My theme, my 30 training and 10 secret subjects, my ratings. The untrained model only copied the example cat, so I taught it first (SFT on 84 sketches I approved), then ran 35 RL steps on a reward checked against my taste. The numbers rose; my blind ratings did not. Across 169 paintings from five variants I loved one, and my own ratings of the same paintings agreed only half the time. Dictating thousands of coordinates without seeing the picture hit a ceiling.
Held-out subjects, same seed
| Blind, strict mean (0–2) | Round 1 | Round 2 |
|---|---|---|
| Original Qwen3-8B | 0.47 | 0.30 |
| v2 SFT + RL | 0.47 | 0.30 |
| v2 SFT | 0.07 | – |
| RL only, no SFT | – | 0.07 |
| v2.1 (no example, KL to SFT) | – | 0.10 |
The sketches it learned from (ones I loved)
The cat kit: compose, don't dictate
I wrote a small watercolour library that draws cats, props and scenes well every time. The model now writes about ten lines that compose the picture. I checked the ceiling first (I loved 30 of 50 kit paintings), taught it 255 compositions I rated, then ran 31 RL steps rewarding image quality and "shows this exact subject".
// what the model writes
paper(); weather('snow');
cat(300, 500, 1, 'sit', 'tabby', 'up');
wear('scarf', '#d95f6a');
Held-out subjects: v2's best against v3
| Blind round 3 · 127 paintings | Love | Okay | No | Strict mean |
|---|---|---|---|---|
| v2 best | 1 | 10 | 27 | 0.30 |
| Untrained + kit | 13 | 1 | 0 | 0.68* |
| v3 SFT | 28 | 7 | 1 | 1.57 |
| v3 SFT + RL | 33 | 4 | 2 | 1.75 |
* Only 14 of 40 rendered: one kit effect crashed the headless renderer (fixed since). I loved 13 of the 14.
The kit's ceiling: compositions I wrote and loved
How one training step works
- Promptsubject and rules, plus the kit's functions in v3
- Qwen3-8B + LoRAeight programs per subject, sampled at temperature 1
- Safety gatean allowlist of drawing calls, numbers and strings; anything else scores 0
- Sandboxed Chromiump5.js and p5.brush render a seeded 600 × 600 painting
- RewardHPSv3 image quality, plus a VLM "shows the subject" check in v2 and v3
- Updateadvantage = reward − group mean; nudge the LoRA toward the better ones
Spend on GPUs from SF Compute: v1 $24.50 · v2 $42.54 · v2.1 and control $28.77 · v3 $23.33.