Training AI to Paint with Code

Surya Narreddi trained a language model to create watercolor-style images by writing JavaScript code, using reinforcement learning. The key insight: instead of prompting for images, the model generates editable code, allowing granular changes. The project tackles the challenge of applying RL to subjective tasks by designing a reward system based on pairwise comparisons against a hand-curated reference pool. This approach outperformed a complex multi-signal rubric, leading to faster convergence and more concise code. The article details the training loop, reward function evolution, and system prompt optimization, offering a practical framework for RL on creative tasks.