Rasterpiece: A cross-modal framework for real-time image sonification, sound synthesis, and multimedia art

This paper introduces Rasterpiece, a multimedia environment for direct real-time sound synthesis from images based on the method of raster scanning combining scanning variables (such as sample rate and probe size) with image processing filters. Rasterpiece enables cross-modal integration of image and sound. By providing an arbitrary number of image windows, each with independent control of sonification parameters and image processing, Rasterpiece supports simultaneous playback, mixing, and recording of multiple sonification results. Applications include diagnostic exploration of visual data, creative sound design and composition, and real-time multimedia performance. Special attention is paid to its use as a sound particle design tool for pulsar synthesis.