Spire Muse: A Virtual Musical Partner for Creative Brainstorming
Spire Muse is co-creative musical agent that engages in different kinds of interactive behaviors and stylistic responses. The software utilizes corpora of solo instrumental performances encoded as self-organized maps and outputs slices of the corpora as concatenated, remodelled audio sequences. Transitions between the agent’s behaviors are partially automated, and the interface enables the negotiation of these transitions through feedback buttons that signal approval, force reversions to previous behaviors, or request change. Styles of musical responses are embedded in a pre-trained latent space, emergent in the interaction, and may be influenced through the weighting of rhythmic, spectral, harmonic and melodic features. The training and run-time modules utilize the MASOM agent architecture.
Spire Muse is designed to encourage creative exploration and defer cognitive deliberation. The musical response type is embedded in the latent memory space and activated through interactive behaviors ranging from highly reactive to the user's input to largely independent. The musical direction can be influenced indirectly through weighting rhythmic, spectral, melodic and harmonic features.
"Spire Muse: A Virtual Musical Partner for Creative Brainstorming" was awarded the best paper award at the 2021 International Conference on New Interfaces for Musical Expression (NIME). The paper is available here as part of the conference's proceedings, and a brief demonstration of the project for the conference is included below.
Video demos
The following videos are demos of Spire Muse in the different behavioral modes.
Video 1: Free improv in shadowing mode. The agent is trying to follow the user's input. Due to sparsities in certain areas of the corpus (such as with the harmonics in the beginning of the video), the agent's responses can be quite diverging. Yet, it maintains a degree of coherence. In this demo, the agent's responses are excited by input that is deliberately trying to "trick" it into difficult territory.
Video 2: Demo of Spire Muse in mirroring mode. The agent listens to the user's input on a phrase level and attempts to mirror back its own version of what it "heard".
Video 3: Demo of Spire Muse in coupling mode. The slight lag in the agent's response is due to the screen recording software and Max competing for processing power.
Video 4: One of the earliest modifications of MASOM in use by Bernt Isak Wærstad (guitar, electronics) at Sentralen, Oslo - October 2020.