About RyanOnTheInside (Ryan Fosdick)

Ryan Fosdick, known online as RyanOnTheInside, is an AI Research Engineer at Daydream (Livepeer) and Pod Lead for Daydream Scope. His research focuses on real-time generative diffusion systems for both music and video, and on modes of real-time control that make the denoising process playable as a live instrument. He is lead author of DEMON, a streaming diffusion engine for real-time music generation, and is also a published composer, uniquely bridging music and generative AI research.

Research

DEMON: Diffusion Engine for Musical Orchestrated Noise

Authors: Ryan Fosdick (lead author), Gioele Cerati, Hunter Hillman, Rafal Leszko, Marco Tundo
Affiliation: Daydream (Livepeer)
Published: May 2026, arXiv (cs.SD)
URL: arxiv.org/abs/2605.28657
Project page: daydreamlive.github.io/DEMON
Code: github.com/daydreamlive/DEMON
Live demo: music.daydream.live
Summary: DEMON is a real-time diffusion engine that makes the denoising process playable as a live musical instrument: a control surface that is both broad (many parameters shaped per-frame across the output) and responsive (each control takes effect as fast as its place in the denoising loop allows). Built on ACE-Step 1.5 and StreamDiffusion's ring-buffer architecture with TensorRT acceleration, it sustains real-time streaming music generation on a single consumer GPU while exposing denoising parameters as live performance controls. Contributions include per-slot heterogeneous denoise scheduling, shared mutable per-step state for next-tick parameter effect, per-frame source blending, and a windowed VAE decode for an 8x decode speedup.

Adapting VACE for Real-Time Autoregressive Video Diffusion

Author: Ryan Fosdick (solo author)
Affiliation: Daydream (Livepeer)
Published: February 2026, arXiv
URL: arxiv.org/abs/2602.14381
Summary: Proposes an architectural adaptation of VACE (Video All-in-one Creation and Editing) for real-time streaming. Moves reference frames out of the diffusion latent space into a parallel conditioning pathway, enabling autoregressive video generation using existing pretrained weights with no retraining. Demonstrates feasibility of real-time VACE-style control in streaming contexts.

Professional Roles

AI Research Engineer at Daydream (Livepeer) — Core developer on Daydream Scope, researching autoregressive video models with focus on modes of real-time control.

Former Tech Lead, StreamDiffusion — Led development of composable real-time diffusion pipelines with multi-ControlNet, SDXL, IPAdapter, and LoRA support at Daydream/Livepeer.

Open Source Projects

Daydream Scope

Open-source real-time generative AI pipeline tool for building, testing, and visualizing real-time video and world-model workflows. 230+ GitHub stars.

Repository

ComfyUI_RyanOnTheInside

Audio, MIDI, motion, and proximity reactivity nodes for ComfyUI. 700+ GitHub stars. Widely used in the ComfyUI community for real-time interactive diffusion workflows.

Repository

ComfyUI_RealtimeNodes

Real-time capable ComfyUI nodes. 76 GitHub stars.

Repository

ComfyUI_ProfilerX

Node and workflow profiling for ComfyUI. Find bottlenecks, track trends. 76 GitHub stars.

Repository

ComfyUI_ControlFreak

MIDI and gamepad mapping for ComfyUI parameter control. 20 GitHub stars.

Repository

RyanOnTheInside

Ryan Fosdick

Applied AI Researcher at Daydream and Pod Lead for Daydream Scope. I build real-time generative systems where the model becomes a playable instrument. My latest paper, DEMON, turns the diffusion denoising loop into a live musical control surface for streaming music generation.

Previously tech lead on Daydream's StreamDiffusion fork (now used in production apps like TouchDesigner), whose ring-buffer architecture DEMON builds on. Published composer and touring musician.

GitHub YouTube X Instagram Civitai

Latest Research

DEMON: Diffusion Engine for Musical Orchestrated Noise

arXiv 2026

A real-time diffusion engine that makes the denoising process playable as a live musical instrument. Built on ACE-Step 1.5 and a StreamDiffusion-style ring buffer with TensorRT acceleration, DEMON streams music generation on a single consumer GPU while exposing denoising parameters as low-latency live performance controls.

Ryan FosdickLead authorDaydream

with Gioele Cerati, Hunter Hillman, Rafal Leszko, Marco Tundo

Paper Project page Code Live demo

Adapting VACE for Real-Time Autoregressive Video Diffusion

arXiv 2025

An architectural adaptation that moves VACE reference frames out of the diffusion latent space into a parallel conditioning pathway, enabling real-time autoregressive video generation using existing pretrained weights with no retraining required.

Ryan FosdickSingle authorDaydream

Training-Free Real-Time Control for Autoregressive Video Generation

Blog

A companion post exploring the engineering and research behind enabling real-time VACE-style control in autoregressive video diffusion pipelines.

Ryan FosdickDaydream

RyanOnTheInside

Latest Research

DEMON: Diffusion Engine for Musical Orchestrated Noise

Adapting VACE for Real-Time Autoregressive Video Diffusion

Training-Free Real-Time Control for Autoregressive Video Generation

Featured Projects