JUST TYPED ALONG WITH THE SESSION SO EVERYTHING IS STILL PRETTY MESSY, NO LINKS INCLUDED AND POSSIBLE FAULTS ARE MY OWN RATHER THAN THE SPEAKERS,…
* New Interfaces for Popular Music Performance
Roger Dannenberg
- using computers to augment music performance: tape music, click tracks, computer accompaniment, interactive improvisation systems, new instruments, popular music is relatively unexplored: too common? too technical? huge potential market, some very challenging problems.
- Popular music: characteristics: general a steady tempo, generally fixed score, somewhat flexible structure, drums, guitar, keyboards are not playing fixed parts. This genre is interesting because: there is enough structure to be predictable and understandable, there is enough variation to require realtime interaction.
- How can computers augment performance of popular music: play additional parts, assist performers with technical parts, create new musical material, assist with rehearsals, assist with sound reinforcement and digital audio effects.
- Research framework: where is the beat? beat tapping interfaces, tempo and phase estimation, merging multiple sources of information. Also data preparation: editing, mixing, annotation of scores with performance info. Also: Where are we in the song? listening to chord changes, melody spotting, interfaces to cue score location, visual display integrated with musical notation. Sound generation: realtime time stretching for synchronization, sound synthesis from scores and lead-sheets, computer assisted sound reinforcement/mixing.
- Online adaptation: coordination of note attack times, learning beat phase relationships, dynamics / balance /mixing, learning charts from rehearsal
- Prototype: Goal: to augment horn section in rock band, demands: control by one musician, high quality, time scaling for tempo adaptation, tap interface to get tempo, extra taps to get phase and cue entrances.
* Towards Rhythmic Analysis of Human Motion using Acceleration-Onset Times
Eric Lee, Urs Enke, Leo de Jong
- Motivation: music is a rhythmitization of sound: whether this is true or not, rhythm is at least important
- Rhythm pattern: a repeating series of accentuated impulses separated by time intervals.
- Goal: attach accelerometers to people and extract rhythmic notions from the input data
- Related work: dance-movement analysis: Paradiso’s “DanceShoe”, Griffith’s “LifeFoot”, automatic beat detection tools.
- We try to extract beat information from accelerometers, so no audio data analysis for beat extraction. We want to do this all in real time.
- The Algorithm: sensor signal, movement detection, impulse sequence generation from movement detection, interval analysis, frequency analysis –> data fusion from the two analyses, impulse folding from impulse sequence and pattern structure, impulse clustering –> rhythm
* nJam user experiments: enabling NMP (networked music performance) from milliseconds to seconds
Nicolas Bouillot
In the remote real-time musical interaction domain, the end- to-end latency is a well known problem. Today, the main explored approach is to keep it below the musician perception threshold. In this paper, we experiment another approach, where end-to-end delays rise to several seconds but computed in a controlled (and synchronized) way depending on the musical pieces structure. We implement a prototype called nJam and perform user experiments to show how this new kind of interactivity breaks the actual end-to-end latency bounds while being user friendly.
* Ashitaka: an audiovisual instrument
Niall Moody, Dr. Nick Fells, Dr. Nicholas Bailey
project website
- Aim is to create an audiovisual instrument whose output is perceived as an audiovisual whole, with audio and visuals not easily separated
- based on Michel Chion’s AudioVision: sound on screen” (synchresis)
- synchresis is based on motion: objects that we can see that are moving mostly make a sound.
- metaphor based mappings and perception based mappings
- sound and image influence each other
- performer’s gestures are mapped to the audiovisual parameters
- X3D is a virtual world file format, the successor to VRML
- interface is based on claw-based gestures
- sensors: stretch, twist, 4x force sensors, accelerometers
- visual: a system of gravity objects
- audio: Tao physical modeling language, single string model as primary synthesis method
* Percussion instruments using realtime convolution: Physical controllers
Roberto Aimi
- Physical controllers: drum pad, frame drum, brushes, bass drum, cymbals
- convolution latency overcome by increasing FFT window size –> effective delay of eg 64 samples
- Using real modified percussion instruments as controllers for other sounds
- using non-linear waveshaping in order to emulate for example the non-linear properties of a real cymbal
