Kadenze: Creative Programming for Audiovisual Art

 with  Mick Grierson

Creative Programming for Audiovisual Art aims to enhance student skills and experience in the development of software for the creation and manipulation of sounds and images, both in real and non-real time. The course explores the intersection and similarities between sound and image signal processing through the exploration of combined audiovisual approaches. As such it is suitable for those with a strong interest in creative technologies from either an engineering or creative arts background.

This course extends the principles of creative engineering for use in arts, games, and more general interaction scenarios so that students can develop their own projects through the use of computational approaches to audiovisual processing. The lessons will be taught using Javascript. It is recommended that students have some experience with using Processing and some background in digital audio and/or digital image manipulation before taking this course.

Materials on how to use Javascript will be made available, and we will be using bespoke tools and libraries to make this a lot easier. At first we will use ready made code to apply creative techniques. Following this, you’ll be expected to develop your own code based on what you’ve learned. Throughout this process, you should pick up some great skills.

We will spend the first few sessions exploring Digital Audio Signal Processing. This will cover synthesis, sequencing, filtering, sample loading and playback, panning and rudimentary analysis. Following this we’ll be looking at audiovisual interaction using video and 3D graphics.


Session 1: Introduction To The Course: What Is Creative Programming? 
Introduction to Digital Audio in C++. In this session we will get started using an IDE, and then download the current version of Maximilian. We’ll be going through some of the examples, and building some really basic music software to get us started. Some of the most important things you will learn in this session include how simple waveforms can be used to create more complex ones, and how signals can be used to make sound, or to control sound, depending on how those signals are scaled.
Session 2: Sound Synthesis And Control 
Building on the previous session's introduction, we will be learning more about waveforms and their construction, and looking at additive synthesis. After this we’ll be exploring how we can use simple statements to make an efficient metronome object that can be used to make things happen. At the end of the session, you will have all built synthesizers in C++ and made them play things that sound like music.
Session 3: Real-Time Recursive Filters, LFOs, And Sample Playback 
In this session we will learn how to use Maximilian’s built-in filters to create a subtractive synthesizer and an LFO, and get some insight into how these work. We will also learn how to load in a sample and play it back.
Session 4: Building A Drum Machine/Sample Playback Device, Manipulating Samples, And Using Envelopes 
During the first half of the lesson we will be working to create a drum machine using sample players and sequencer skills from the previous session. Following this we will be looking at various ways of manipulating sample playback to create interesting effects, and learn how to use basic envelope generators to control parameters. We will also look at an advanced example demonstrating how to create a polyphonic music device.
Session 5: Audio Visual Composition: Contemporary Work By Audiovisual Creatives 
In this session we will look at the work of contemporary AV artists and talk about what inspires them, showing examples of their work and discussing what approaches they take to audiovisual composition. We will also briefly introduce openFrameworks, and begin to explore the fundamentals of algorithmic computer graphics design in C++.
Session 6: Using OpenFrameworks With Maximilian 
This session will be a basic introduction on how to use openFrameworks in more detail, and how to augment it with Maximilian. We will look at the main openFrameworks methods, and demonstrate how to incorporate your existing audio DSP code and projects into openFrameworks to make interactive music applications that use graphics, mouse and keyboard interaction.
Session 7: Audiovisual Synthesis Using OpenFrameworks 
This session will look at how to get basic information from our sound algorithms and use it to map graphical parameters to make basic audiovisual synthesizers. We will explore a range of generative drawing techniques including the geometric systems, non-linear systems and noise. We will also look at controlling generative drawing through audio, and using graphics parameters to control sound.
Session 8: VJ/DJ Techniques And Audiovisual Sampling 
In this session we will be using audio signals to control aspects of a video signal using sampled footage and sound files. This means loading a video, understanding how to control a video, and then understanding how to map audio signals into a range that make them useful for video processing.
Session 9: Audio Features And Their Uses: FFT/IFFT/MFCC/Convolution 
In this session we’ll be visualizing audio features using Maximilian’s FFT libs in openFrameworks. Here we can do great effects with not much coding, but getting started is a headache, and the FFT is always tough to conceptualize.
Session 10: Video Manipulation And Analysis 
In this session we will look at an example that allows you to edit the pixels from a movie or a webcam, to create a background subtraction system for doing basic homegrown computer vision. This project also demonstrates how to load a char buffer into a sound object and play it like a sample. Lots of fun I’m sure you will agree. We will also introduce frag shaders as a mechanism for manipulating images.
3 Student
Cost Free Online Course
Subject Digital Media
Provider Kadenze
Language English
Certificates Paid Certificate Available
Calendar 10 weeks long
+ Add to My Courses
Learn Digital Marketing

Gain real-world experience running live campaigns as you learn from top experts.

Become a Data Scientist

Learn Python & R at your own pace. Start now for free!

FAQ View All
What are MOOCs?
MOOCs stand for Massive Open Online Courses. These are free online courses from universities around the world (eg. Stanford Harvard MIT) offered to anyone with an internet connection.
How do I register?
To register for a course, click on "Go to Class" button on the course page. This will take you to the providers website where you can register for the course.
How do these MOOCs or free online courses work?
MOOCs are designed for an online audience, teaching primarily through short (5-20 min.) pre recorded video lectures, that you watch on weekly schedule when convenient for you.  They also have student discussion forums, homework/assignments, and online quizzes or exams.

3 reviews for Kadenze's Creative Programming for Audiovisual Art

Write a review
a month ago
profile picture
Anonymous partially completed this course.
Unfortunately this course is a struggle to get through. The videos are out of date in comparison to the "codecircle" site that Mick uses for the coding, so even in the first video's you'll fail to get sound generated as codecircle is now on a secure HTTPS connection and the maxiLib library is being served from a non-s Read More
Unfortunately this course is a struggle to get through. The videos are out of date in comparison to the "codecircle" site that Mick uses for the coding, so even in the first video's you'll fail to get sound generated as codecircle is now on a secure HTTPS connection and the maxiLib library is being served from a non-secure HTTP connection, which the browser will not permit - It requires some fiddling around to get the library working. Even if you achieve that, Mick refers to code examples that he's created on codecircle, but if you are lucky enough to find them still there, they are mostly not in the initial state that he started with them in the videos, so it's hard to follow along; not to mention that Mick's maxiLib library, whilst available on GitHub, lacks any form of documentation to describe the APIs available, so it's either guesswork or digging in to the details of his Javascript code to try and figure it out. On the plus side, the general principles of generating sounds and the theory of it are good, but I've had to give up after getting to session 6 and getting sick of the lack of supporting files, such as the sample files that he refers to - which others have also complained are missing. It's more work trying to work around and debug your way through this course than it's worth. It's a shame as I was looking forward to the creative graphical side of the course, and it seems pointless trying to do that now. I'm sure the course would be good if it were re-video'd and the core material were re-created to make it available to use.
Was this review helpful to you? YES | NO
3 months ago
profile picture
Anonymous partially completed this course.
I've liked the course and the teacher, the only thing that I would improve is the software used. The JS library is not packaged and available through npm, therefore, it is complicated to integrate it in projects more complex than the sketches done in the exercises.

For the rest, I would suggest this course to anyone who wants to know more about sound and reactive graphics
Was this review helpful to you? YES | NO
a year ago
Leon Fedden is taking this course right now.
Brilliant course and instructors; a great introduction to sonic and graphical art via the computer. Would recommend it to anyone thinking of learning more about how to programmatically generate audio or visuals
Was this review helpful to you? YES | NO

Class Central

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free