Paper Session 6 – Systems and Standards

JUST TYPED ALONG WITH THE SESSION SO EVERYTHING IS STILL PRETTY MESSY, NO LINKS INCLUDED AND POSSIBLE FAULTS ARE MY OWN RATHER THAN THE SPEAKERS,…

    * New Musical Interfaces in Context: Sonic Interaction Design in the Urban Setting
Karmen Franinovic, Yon Visell
- recorded sounds are played back by spinning black dishes
- work in public space,.. ot’s about interaction design
- social listening, exploration about what such a system is about, negotiate with strangers about what is the soundscape
- interactive installation in public space: various users, platforms for discovery of public space, turning production and consumption into a more creative space, people discover new and unintended experiences
- design by discovery (urban probes)
- these instruments are an alternative media to let passers by appropriate public space
- expanded performance
- social and political questions: it’s about sociallity as expanded (musical) performance
- see also situationists movements: citizen is pushed into a creative environment
- If you want a lot of people make something that is visually striking
- Questions of context: sound is an ecologically sensitive medium, performance, sensory perception, timbre perception are cultural acts, musicology offers cultural settings for this type of art

    * Extended Applications of the Wireless Sensor Array (WISEAR)
David Topper, Virginia Center for Computer Music
- genreal idea: using Linux based SBCs in performance, robotics, etc.
- linux is close to OS X
- linux is open source
- Desire for something more flexible / universal /easier to use than BASIC stamps & PIC chips
- first generation WISEAR: built on TS-5500 board from technologis Systems (www.embeddedarm.com
- second generation board: less power hungry: 6 sensors, data transmission via off the shelf wireless
- But still too many problems: now we’re moving to Gumstix (www.gumstix.com)
- Gumstix: built in audio, ADC, DIO, GPIO, modular platform with extensive features: usb, bluetooth, ct, etc; 600mhz, wiki docs, better user base, very small

* CELERITAS: Wearable Wireless System
Giuseppe Torre, Mikael Fernstrom, Brendan O’Flynn
- a wearable wireless sensor unit for the purpose of live dance performance
- main tasks: low latency (-15ms with eight nodes), low costs, allowing solo and group performances, usability (programmable by composer and choreographer), reliability (last through a whole performance)
- signal path: 25 mm WIMU, base station, serial port, computer (driver), max/msp object (external),mapping, av output
- measures: 50 x 25 x 25 mm, weight 30g, 3.4 volt lithium-ion rechargeable battery (approx 3h life)
- FIFO: first in, first out
- driver is compiled for win and apple and runs with max/msp and pd
- objective: user friendly interface
- to create a 3d virtual interface instrument around he body of the dancer. For this we need the exact position of each node, now they only relative position.
- Negative: strong jitter which is solved by averaging in max/msp
- results: acceleration, speed, distance, orientation, angles, quaternion, etc.

    * Defining a control standard for easily integrating haptic virtual environments with existing audio / visual systems
Stephen Sinclair, Marcelo Wanderley (input devices and music interaction laboratory, McGill University, Montreal)
- opening up the ability in order to easily play with haptic systems (force feedback systems)
- force feedback haptics, virtual environments, communication
- sensors: force-feedback using motors = sense of tough (feels as if you can feel the touch of an object in virtual space)
- haptic devices: SensAble Phantom Omni, Force Dimension Omega, Novint Falcon (now approx $200!!!!!) , ERGOS, MPBT Freedom 6S
- previous works: virtual instruments: ANCROE: CORDIS ANIMA, Mulder er al: virtual musical instruments, Verplank: The Plank; Gillespie, etc
- Integrating VR with Audio: many libraries support graphics but not sound
- ideal: physical object of an object and then extract audio output
- should be realistic and accurate, energy preserving, computationally demanding, difficult to integrate, impose a synthesis model
- Asyncronous Architecture: phsical dynamics, haptic device, audio synthesis, audio output (haptic rate ~100Hz)
- DIMPLE: Dynamically Interactive Musically Physical Environment (download url is in the paper) = opensource

    * Chroma Palette: Chromatic Maps of Sound As Granular Synthesis Interface
Justin Donaldson, Ian Knopke, Chris Raphael  (Indiana University School of Informatics)
- Chromapalate: granular synthesis interface
- Granular Synthesis: Gabor, Roads, Truax, Xenakis
- selection of grains, density, pitch, length, envelope, combine and re-synthesis
- grains form 1-50ms
- nwe way to sort out the sounds: Chromatic characteristics of the individual grains (what pitch do they represent?)
- use of FFT analysis and Aygmented Chroma
- Converting the Chroma dat into a 2d space
- Now you can use the maps for granular synthesis (for grain selection)
- Client side: interface is coded in Flash, only grain index and onset information need to be sent to the synthesis engine
- issues: MDS can involve error in the representation, it can only handle a few thousand grains per map

Paper Session 4 – Timing, Motion and Rhythm

JUST TYPED ALONG WITH THE SESSION SO EVERYTHING IS STILL PRETTY MESSY, NO LINKS INCLUDED AND POSSIBLE FAULTS ARE MY OWN RATHER THAN THE SPEAKERS,…

* New Interfaces for Popular Music Performance
Roger Dannenberg
- using computers to augment music performance: tape music, click tracks, computer accompaniment, interactive improvisation systems, new instruments, popular music is relatively unexplored: too common? too technical? huge potential market, some very challenging problems.
- Popular music: characteristics: general a steady tempo, generally fixed score, somewhat flexible structure, drums, guitar, keyboards are not playing fixed parts. This genre is interesting because: there is enough structure to be predictable and understandable, there is enough variation to require realtime interaction.
- How can computers augment performance of popular music: play additional parts, assist performers with technical parts, create new musical material, assist with rehearsals, assist with sound reinforcement and digital audio effects.
- Research framework: where is the beat? beat tapping interfaces, tempo and phase estimation, merging multiple sources of information. Also data preparation: editing, mixing, annotation of scores with performance info. Also: Where are we in the song? listening to chord changes, melody spotting, interfaces to cue score location, visual display integrated with musical notation. Sound generation: realtime time stretching for synchronization, sound synthesis from scores and lead-sheets, computer assisted sound reinforcement/mixing.
- Online adaptation: coordination of note attack times, learning beat phase relationships, dynamics / balance /mixing, learning charts from rehearsal
- Prototype: Goal: to augment horn section in rock band, demands: control by one musician, high quality, time scaling for tempo adaptation, tap interface to get tempo, extra taps to get phase and cue entrances.

* Towards Rhythmic Analysis of Human Motion using Acceleration-Onset Times
Eric Lee, Urs Enke, Leo de Jong
- Motivation: music is a rhythmitization of sound: whether this is true or not, rhythm is at least important
- Rhythm pattern: a repeating series of accentuated impulses separated by time intervals.
- Goal: attach accelerometers to people and extract rhythmic notions from the input data
- Related work: dance-movement analysis: Paradiso’s “DanceShoe”, Griffith’s “LifeFoot”, automatic beat detection tools.
- We try to extract beat information from accelerometers, so no audio data analysis for beat extraction. We want to do this all in real time.
- The Algorithm: sensor signal, movement detection, impulse sequence generation from movement detection, interval analysis, frequency analysis –> data fusion from the two analyses, impulse folding from impulse sequence and pattern structure, impulse clustering –> rhythm

* nJam user experiments: enabling NMP (networked music performance) from milliseconds to seconds
Nicolas Bouillot
In the remote real-time musical interaction domain, the end- to-end latency is a well known problem. Today, the main explored approach is to keep it below the musician perception threshold. In this paper, we experiment another approach, where end-to-end delays rise to several seconds but computed in a controlled (and synchronized) way depending on the musical pieces structure. We implement a prototype called nJam and perform user experiments to show how this new kind of interactivity breaks the actual end-to-end latency bounds while being user friendly.

* Ashitaka: an audiovisual instrument
Niall Moody, Dr. Nick Fells, Dr. Nicholas Bailey
project website
- Aim is to create an audiovisual instrument whose output is perceived as an audiovisual whole, with audio and visuals not easily separated
- based on Michel Chion’s AudioVision: sound on screen” (synchresis)
- synchresis is based on motion: objects that we can see that are moving mostly make a sound.
- metaphor based mappings and perception based mappings
- sound and image influence each other
- performer’s gestures are mapped to the audiovisual parameters
- X3D is a virtual world file format, the successor to VRML
- interface is based on claw-based gestures
- sensors: stretch, twist, 4x force sensors, accelerometers
- visual: a system of gravity objects
- audio: Tao physical modeling language, single string model as primary synthesis method

* Percussion instruments using realtime convolution: Physical controllers
Roberto Aimi
- Physical controllers: drum pad, frame drum, brushes, bass drum, cymbals
- convolution latency overcome by increasing FFT window size –> effective delay of eg 64 samples
- Using real modified percussion instruments as controllers for other sounds
- using non-linear waveshaping in order to emulate for example the non-linear properties of a real cymbal

Session 3 – Philosophical, Historical and Pedagogical Issues

JUST TYPED ALONG WITH THE SESSION SO EVERYTHING IS STILL PRETTY MESSY, NO LINKS INCLUDED AND POSSIBLE FAULTS ARE MY OWN RATHER THAN THE SPEAKERS,…

* Erkki Kurenniemi’s Electronic Musical Instruments of the 1960’s and 1970’s
Mikko Ojanen, Jari Suominen, Titti Kallio
MISSED

* The Acoustic, the Digital and the Body: A Survey on Musical Instruments
Thor Magnusson, Enrike Hurtado Mendieta
- University of Sussex, Huddersfield University
- research of interactive modes in musical software
- free and opensource software is on the website
- propagating opensource software and the sharing of knowledge
- Research focus: dual semiotic stance of the user of the software
- designer of creative software has to be aware of this fact
- What does it mean to be a consumer of musical software
- does it make sense to talk about “software interpretation”?
- environment in supercollider: various instruments
- you have the ability to code in realtime
- interested in the software as the “neuro instrument” of the software
- people can create and modify their instruments
- what is a digital instrument? –> people have different opinions, here we did not try to define it
- focus: control interaction, instrument entropy, affordance and constraints, creativity: the epistemic tool as the prime mover
- phenomenological: based on experience
- Participants: mailing lists for audio programming languages, conservatories, universities and orchestras
- 210 replies of which 9 were female
- linux (45), osx (88), windows (105), average age 45(!)
- acoustic instruments positive: tactile feedback, limitations are inspiring, traditions and legacy, depth, instrument becomes 2nd nature, embodied experience, no latency, easy to express mood, extrovert state when playing
- acoustic instruments negative: lacking in range, no editing, no memory or intelligence, prone to cliche playing, too much tradition, no experimentation in design, inflexible, less microtonalities or tunings, no inharmonic spectra
- digital instruments positive: free from traditions, experimental, any sound and interface, freedom in mapping, designed for specific needs, automation and intelligence, good for composing, easier to get into, not as limited to tonal music
- digital instruments negative: lacking in substance no legacy, no haptic feedback, latency, disembodied experience, lacking social conventions, slave to the historic, limitation to the acoustic, introvert state when playing, no haptic feedback
- Conclusion: people work with the best of both worlds, design around the constraints of each, digital is playing and composition in once, the entropy of acoustic instruments is important, open-source: people express the importance in freedom in expression, open standards, etc etc…..

* Ten Years of Tablet Musical Interfaces
Michael Zbyszynski, Matthew Wright, Ali Momeni
- UC Berkeley Centre for new Music
- interfaces with a future, interfaces with a past: “towards a theoretical formulation of the Long New”
- What makes a good musical controller: advantages of standard controllers: low cost and availability, leading to,.. redundancy and replaceability / General characteristics: high resolution output data, fine temporal accuracy, multiple axes of control.
- Tactile reference: the player touches it (haptic feedback), spatial coordinates are absolute, leverages fine motor control and writing/drawing experience, leaves the other hand free to do something else
- Precedents: many, including: Xenakis UPIC Sstem, Boie/Mathews/Schloss Radio Drum
- opensoundcontrol.org (OSC wraper between controllers and sound maker processes. This facilitates a max patch to document this and opens up for a wrapper to change and remap things
- Matt Wright: interactive instruments, he makes performance templates from samples of recordings of real players
- Wacom objects for Max/MSP
- Ali Momeni: parameter interpolation space, multi-touch sensors (wacom and contact mic),
- temporal resolution and accuracy leaves something to be desired

* Expression and Its Discontents: Toward an Ecology of Musical Creation
Michael Gurevich, Jeffrey Treviño
- dominant model of creation –> ecological model of creation
The Dominant model of creation in the NIME discourse:
- Defining creation through expression: “deviation” and ” deformation”: implied determinate artistic content. Is there something else that needs to be expressed through music than sounds? Composer: creation, performer: expression. Where is the interpretation.
- Are there expressive interfaces? In the NIME discourse we are locating expression in the interface / quantifying expression. Is expression a unified flow of quantities.
- “Performance communicate musical expression to listeners by a process of coding. Listeners receive musical expression by decoding” (Poepel)
- Experimentalism as non-expressive creation “the shortest path between two people is not a straight line” (Earle Brown)
- The medium as active participant
- Relationships between composers, performance and listeners: any form of configuration may exist. It’s also about context and surrounding elements and influences, economics, authorship, etc. Of paramount importance is that expression is an option.
- Applications: imitation of expression by machines, interfaces/mappings to facilitate traditional expression, develop new expressive cues within the text/act model. We should question expression as the goal.

* Live Coding Practice
Nick Collins (Click Nollins?)
- makes powerpoint presentation on the fly,.. not using: he uses supercollider instead to generate a clock for how much time he has left. Then generates a random numberin order to show how many quotes he will give. What is live coding? defenition: when musicians and artists can express themselves immediately through a program. “Humans make dynamic cogs within the threads of rule systems which rewrite themselves”
- online live coding performance video: “Study in Keith”
- www.toplap.org
- In the paper he goes through pedagogical issues and suggest practice and exercises.
- What about live coding together with live musicians? Live Coding cards can inform musicians what to do.
- Is live coding a scene by now?
- skill acquisition of violin: 3 hours a day for 10 years,… how much do you have to practice to become a live coder?
- ” Teach yourself programming in 10 years” = website
- question: where do you draw the line between what you’ll prepare at home and what you do live? What is considered cheating in the scene?

* Natural Interfaces for Musical Expression: Physiphones as primordial Infra-Instruments
Steve Mann
- Hydraulophones as Physiphones
- organolog (ethnomusicology): strings, percussion, wind: strings and percussion are more similar to each other than to wind)
- Geaphones make sound from matter in its solid state, wind: make sound from matter in gas-like sound
- idiophone (3d solid), membranophone (2d solid), chordophone (1d solid), Aerophone (gas), Electrophone (informatic), note again that the first three are more similar to each other than the last two.
- there also can be a physics based organology: solid (1, 2 and 3d), liquid, gas, plasma, informatic: Greek names correspond to earth, water, wind, fire and Quintessence (idea)
- So what is the complete orchestra?
- presents the “self-cleaning keyboard” a water-based flute in a public park
- presents the “global village fountain and immersive multimedia”: physiphones, electric xylophones, hyper-hydraulophone, cyborg instruments.
- presents “splash page”: a waterfall musical instrument
- The KEY to good music is to PLAY in the water

* Wireless sensor interface and gesture-follower for music pedagogy
Frederic Bevilacqua, Fabrice Guedy, Emmanuel Flety, Nicolas Leroy, Norbert Schnell
- Wireless Interface: requirements: compact size and weight, handheld interfaces, augmented string interfaces
- Custum XBee digitizer: 6 inputs, 10bits, 5ms / Ethernet receiver/base untit / OSC compliant data / Li-Po battery
- 5 output sensor (sparkfun combo board)
- Parallel use of units is possible
- Examples: dance (it’s wearable), augmented instruments
- gesture follower: analyze data in real time and compare it to already learned gestures,.. the device learns these gestures (you’ll need a “training fase”)
- The program reacts only on gestural changes so ths means you can do a particular gesture slower or faster and possibly speeding a coupled sound up or down.,… or control any other parameter
- This device can be used for training conductors
- Preliminary studies encouraging: scenarios stimulating the interaction between theory and practice, etc.
- gesture follower: free download at http://ftm.ircam.fr

NIME07 Paper Session 2 – Instrument Design

JUST TYPED ALONG WITH THE SESSION SO EVERYTHING IS STILL PRETTY MESSY, NO LINKS INCLUDED AND POSSIBLE FAULTS ARE MY OWN RATHER THAN THE SPEAKERS,…

Paper Session 2 – Instrument Design

* The Multimodal Music Stand
Dan Overholt, Lance Putnam, John Thompson
- Made for multimodal musical performance. Made for the untethered performance gestures that are not directly controlling the other instrument they are playing. Use of different sensors
- generalized approach towards instr augmentation
- Capture expressive gestures and map them to synthesis parameters
- multimodal
- video camera, mic, 4 E-field sensors
- Background: instrument for expressive control, Augmented music stands, gestures in music, score following
- Computer Vision Techniques: flute segmentation algorithm, gaze detection using the Viola-Jones face detector, nod detection using LK pyramidal optical flow
- Multimodal detection layer, Sound Synthesis
- Future Goals: Incorporate more features in gestural control and recognition, Expressive gesture tracking,

* The T-Stick: from Musical Interface to Musical Instrument
Joseph Malloch, Marcelo Wanderley
- Other stick controllers: Sweatstick, interval stick/talking stick, musicPole
- T-Stick motivation: to create a family of DMInstr, to create a robust physical interface
- A family of DMi’s: context in pedagogy, familiarity in performance (also for the audience to understand what you are doing), fits into traditional performance aesthetics
- Metaphor: a vibrating string or bar, like any object that would make sound acoustically,.. so you can swing, throw, beat, shake, etc
- It’s not a physical model
- Any user could pick it up and have a model in their mind what it’s about,.. later they can become better at the instrument and learn
- Goals: to create a multi-touch sensor surface, to make it more robust, to make a model of a real vibrating object incl excitation and damping etc.
- Multi-touch sensing: an array of discrete capacitive sensors
- 3 axis accelerometer, pressure sensors, contact microphone inside
- cover: to add mechanical strength: shrink-tubing
- Performances: DMI Seminar; McGill Digital Orchestra Project
- From Interface to Instrument: must be extremely robust, many hours of private practice, simple to operate, hide the sensors, hide the tech, methaphors pulls it together, multiple performers make a better context for pedagogy

* The Thummer Mapping Project (ThuMP)
Garth Paine, Ian Stevenson, Angela Pearce
Marcs, Comarts,
- The freedom Thummer: how can we turn this into a musical interface?
- Design Paradigms: Design for EMinstr are often based on reductionist models of user interaction and sound synthesis / Derived from research in the fields of human comp interaction, industrial design and digital signal processing./ lacking musical context
- How many discrete control parameters do trained acoustic musicians normally exercise in a performance? How are these related to the produced sounds?
- Pressure, Speed, Angle, Position: these are the most important control elements of acoustic instruents: how do we translate them to interfaces for electronic instruments?

* HandSketch Bi-Manual Controller: Investigation on Expressive Control Issues of an Augmented…
Nicolas D’Alessandro, Thierry Dutoit
- Context of research: Realtime CALM, nime06 Paris
- From there: realtime control of voice features, dimension based study of expressivity (singing synthesis), intuitive hand-based control of voice textures:
- Voice, Quality, Control
- Voice: VQCLib, Quality:RAMCESS, Control:HandSketch
- Pen-based control (Kyma System) Pushing forward writing skills with pen-based gestures
- Does RT singing synthesis need precise and complex control? –> vibrato is complex, automatic production is difficult to make sounding natural, realtime CALM sounded god at nime06
- pitch, vocal effort, tenseness
- Now start with control space of realtime improvements in precision,ergonomics and codification
- mapped to angle, pressure and radius
- voice contains articulations impossible to do with shapes (intensity and/or pitch sensors, consonants, etc) solution: make all controllers asymmetric, use position based approach, NPH techniques, etc.
- Work with a FSR (force sensing resistor) network
- mapping strategies: direct, modal (overall control space deformation), spectral (links between dactoylemes and phonemes)
- All in one tablet based musical instrument

* Mobile Clavier: A New Music Keyboard for Flexible Key Transposition
Yoshinari Takegawa, Tsutomu Terada, Masahiko Tsukamoto
- Requirement: musical performance need to show off their virtuosity so they want a mobile (small) keyboard
- porblem: piano’s are too large and heavy to use portably
- Mobile Clavier with Key Transpose cause problems and cause mistake because the keyboard layout becomes unclear
- Adding additional black keys in between two adjacent white keys can solve the issue.

NIME07 Paper Session 1 – Controllers and Physical Models

JUST TYPED ALONG WITH THE SESSION SO EVERYTHING IS STILL PRETTY MESSY, NO LINKS INCLUDED AND POSSIBLE FAULTS ARE MY OWN RATHER THAN THE SPEAKERS,…

Controlling a Physical Model with a 2D Force Matrix
Randy Jones, Andrew Schloss

Mystic center. First goal was intimate control of percussion synthesis. A musical instrument should be alive in the hands of the musician. What comes between the notes in addition to the notes itself. Other goal is about sonic exploration.

2d waveguide mesh was first described by van Duyne & Smith. WaveguideMesh is implemented as 3×3 convolution in a Max/msp/jitter object.

2d Force Matrix takes care of the input to the waveguide mesh. excitation as damping at the same time. Two sources of data go into the matrix. Surface Data as well as multitouch data from multitouch controllers. There is a continuous sampling of surface pressure. Spatially as well as temporally.

Concerts: Schloss, Duran, Mitri Trio: EMF, Real Art Ways & Schloss, Neto, Mitri Trio: CCRNA

Current Goals: Realistic filtering (nonlinear hammer), more aspects of drum modeling, increase controller sampling rate.

PHYSMISM: A control interface for creative exploration of physical models
Niels Boettcher, Steven Gelineck, Stefania Serafin

Medialogi, Aalborg university, Copenhagen, Denmark

Motivation: physical models are oldschool and they sound like shit,.. it was a challenge

What are the possibilities and boundaries,.. focus on completely new sounds

Design criteria: many different models, replica models, sounding like original, extended replica models, hybrid models (models + instrunments), physical interface, unusual interface that the audience can understand, interface should be musically and have a lot of possibilities

4 different models implemented: flute model, PHISM model, Friction model, drum model

crank: prticle model a crank controlling a PHISM particle model: rotation speed is mount of beams
drumpads, flute like interface controlling tube/string model (extended Karplus Strong)
Slider – Friction: horizontal and vertical slider with pressure sensor controlling friction model

2 control stations with 4 parameters, patching system combining the 4 different modes: you can take the output of one model into the other model

mini sequencer that is not innovative at all

Physical models and musical controllers – designing a novel electronic percussion instrument
Katarzyna Chuchacz, Sile O’Modhrain, Roger Woods

Sonic Arts Research Centre, Queen’s University in Belfast

Existing Eletronic Percussion Instruments: Buchla Thunder, Korg Wavedrum, ETabla
Limitations: complexity, extend of control,.. especially difficult are the modeling of large size instruments and the nonlinear sounds

Creation of realtime plate based electronic percussion instruments
high quality sound, range of modeled resonators

Finite difference schemes: problems: huge computational requirement, memory access: possibilities of real-tie performance. Possible are the recreation of large instruments

Solution: FPGA Hardware Implementation: possible to program the architecture of your system: full flexibility
Why? –> more processing power, parallelism in the algorithm, higher memory access bandwidth, flexibility in terms of onterfacing to a range of sensors.
Now it runs “faster than realtime”

Parameter Space: grid size, Plate size , sample frequency

Parameters mapping: hardware mapping parameters, sound synthesis parameters

Sound world of the model
opportunity to drive the model in a number of ways: many parameters fully open,… possibility to go beyond the constraints existing in acoustic systems

design approach is based on connecting to the sample of the model and creating a successful interface

What are the range of techniques of a real percussion player?

future work concentrated on observation of real percussionists, sensor system specifications should follow from this

A Force Sensitive Multi-touch Array Supporting Multiple 2-D Control Structures
David Wessel, Rimas Avizienis Avizienis, Matthew Wright

Gestures and Signals: very high rate of motion capture if you work with percussion sensors. Interlinks VersaPad Semi-conductive Touch-pad.

Multi-touch is the big rage,. we should get the data-rate up in order to be able to process multi-touch high definition controllers.

Most compact layout to get all fingers on the pad. Not really multi-touch but several versapads next to each other. New one has 32 pads.

Data acquisition hardware” daughter-boards consist of 4 to 6 sensors each, analog conditioning, multichannel A/D

72 or 96 variables: most efficient way is to use just 72 audio channels and only convert them if necessary.

Only a midi input, no output. Just to turn the midi into sample signals of the audio.

Yes: reading 147456000 bits per sec is cheaper than demultiplexing, up-sampling and converting to floating-point on the host CPU.

pressure profiles of short taps that percussionists use: 9ms, 14 ms, 18ms, etc.

Pressure profiles of short taps asks for substantially different attacks and curves even in the first milliseconds.

Zstretch: A Stretchy Fabric Music Controller
Angela Chang, Hiroshi Ishii, Joe Paradiso

Starting point from our hand” our hands posess rich capabilities of interacting with materials

Related works: most lack haptic feedback which alters the control loop.

Musical fabrics: mostly about localised places for touching the fabric rather than supporting the many gestures of our hands.

They should support 0 to 20 Newtons: that means it should be robust / Haptic expression / the fabric should guide the interaction

Resistive strech sensors are sown into the lycra fabric

Mechanical: a tabletop frame that holds te fabric but allows access to all sides of the fabric

Robustness issues: noise from mechanical contacts, drift of threads resistance over time, bouncebacks after a hard pull, fabric fatigue from wear and tear.

Software Mapping playback speed (pitch) and volume,.. interrupting zing noise and volume control of it (later was considered to be annoying)

Conclusion: scalable, no electronics in interaction, supports the interaction with the hands, it’s about material properties. Now better mappings and better materials

Oculog: Playing with Eye Movements
Juno Kim, Greg Schiemer, Terumi Narushima

From the Sonic Arts Research Network Faculty of Creative arts, University of Wollongong, Australia

Initially for clinical use, adapted as an expressive performance interface. First performance will be held in July 2007

Interface: firewire camera on snow goggles. Camera mounted to capture the eye movements. Up to 120 fps, in performance 30 fps,

control either voluntary or involuntary. Eye movement is mapped to MIDI, implemented using STK.

5 channels of information: horizontal position, vertical position, etc… CHECK

Active listening to a virtual orchestra through an expressive gestural interface: The Orchestra…
Antonio Camurri, Corrado Canepa, Gualtiero Volpe

University of Genova,InfoMus Lab

Embodied Active Listening: enabling to interactively operate on musical content by modifying it in realtime

full body movement and gesture

focus on: high level expressive qualities of movement and gesture, cross- and multi-modal techniques\\the result is embodied control of….

actively explore the orchestral play.

Multitrack Audio inputs. You can operate on each single channel with realtime mixing.

input with video camera and other possible sensors,… than tracking and extraction of specific features, modes for interaction with space and possible visual feedback

interaction with space: 2d potential functions superimposed onto physical space, single instruments and each function applied to individual instruments. You can change the parameters of the functions in real time in the space

Public Installations: they are aiming for a natural as possible interface (ambient design, disappearance of technology for non-expert users)