MUSIC-GA 2199-002 (5547)
New Media Creation Workshop:
Gesture, Sound, and Image Interactions

Instructor: Jaime Oliver La Rosa
Meeting time & place: MON 3-5pm, Room 365 (24 Waverly)

Syllabus

Install Pd and Gem using these instructions.

Week 1: Introduction to GEM, OpenGL and GEOS. Patches from todays session are here. If you’d like to learn I’d recommend these exercises. Here’s my clock solution.

Week 2: More about Geos; Alpha channel; Single and Double buffering; Pixels, loading, storing and displaying images and films into memory. Here are the patches of the day.

Week 3: PART 1) Pd Review: counters & until; sending&receiving; abstractions&variables; PART 2) Tracking McLaren. Here are the patches of the day. Also, here‘s a small exercise I forgot to send last week in video… I generated it with the [env~] object and a sound file.

Week 4:  More image manipulation, traversing images with pix_coordinate, and strategies for blob tracking. Here are the patches of the day.

Week 5: Connecting Audio and Video patches on multiple pd instances and threading; video tracking and continuity. Here are the patches of the day. The exercise for next week is to think about potential mappings that are more interesting than the one we saw in class and potentially get one of them going in a patch…

Week 6: Video synthesis, audio control. Patches here.

Week 7: Rokeby Discussion, and Latency papers by Levitin, Maki Patola, Wright&Wessel, plus visual overriding and audiovisual fusion by McGurk and Schutz. Reading of Part 1 of Audiovision by M. Chion. Readings sent by email.

Week 8: GLSL shaders and Cyrille Henry’s Egregore. Patches here. Paper by M. Schebella. Collection of GLSL shaders here.

Week 9: Projecting onto unusual surfaces. Particle system generation and sonification. Patches here.

Week 10: pix_snap, iterations and recursions. Patches here.

Week 11: Video tracking: Subtracting background and previous frame. Video tracking strategy in the MANO Cotroller. Files here.

Week 12: Hidden Histories of graphic to music conversions with Norman McLaren, Percy Grainger, Daphne Oram, Iannis Xenakis (the unhidden), and CLAEM’s Fernando Von Reichenbach and Pedro Carievschy. notes and files here. More info on Dal Farra’s catalog of electronic music in Latin America here.