Have you ever wanted to build a new musical instrument that responded to your gestures by making sound? Or create live visuals to accompany a dancer? Or create an interactive art installation that reacts to the movements or actions of an audience? If so, take this course!
In this course, students will learn fundamental machine learning techniques that can be used to make sense of human gesture, musical audio, and other real-time data. The focus will be on learning about algorithms, software tools, and best practices that can be immediately employed in creating new real-time systems in the arts.
Specific topics of discussion include:
• What is machine learning?
• Common types of machine learning for making sense of human actions and sensor data, with a focus on classification, regression, and segmentation
• The “machine learning pipeline”: understanding how signals, features, algorithms, and models fit together, and how to select and configure each part of this pipeline to get good analysis results
• Off-the-shelf tools for machine learning (e.g., Wekinator, Weka, GestureFollower)
• Feature extraction and analysis techniques that are well-suited for music, dance, gaming, and visual art, especially for human motion analysis and audio analysis
• How to connect your machine learning tools to common digital arts tools such as Max/MSP, PD, ChucK, Processing, Unity 3D, SuperCollider, OpenFrameworks
• Introduction to cheap & easy sensing technologies that can be used as inputs to machine learning systems (e.g., Kinect, computer vision, hardware sensors, gaming controllers)
Session 1: Introduction What is machine learning? And what is it good for?
Session 2: Classification This session will cover fundamentals, how to use Wekinator for classification, and an introduction to classification algorithms: kNN, Decision trees, AdaBoost, SVM.
Session 3: Regression In this session we will discuss the fundamentals of regression, using Wekinator for regression, and neural networks for more complex types of models.
Session 4: Dynamic Time Warping In this session you will learn what dynamic time warping is and what it's useful for, as well as how to use Wekinator for dynamic time warping.
Session 5: Sensors & Features Part I: Basic Signal Processing For Learning This session will cover retrieving data from devices: Streaming data vs events; Smoothing noisy signals; Throttling, downsampling, and upsampling; First and second order differences; Buffering & chunking.
Session 6: Sensors & Features Part II: Intro To A Few Fun/Popular Types Of Sensors & Sensing Systems This session will introduce Kinect, Leap, and basic physical computing sensors such as accelerometers, gyros, FSRs, ultrasonic distance sensors, and photosensors.
Session 7: Wrap Up This session will provide a wrap up for the course, and will discuss practical tools, books, and resources students can access for furthering their work in this field.
MOOCs stand for Massive Open Online Courses. These arefree online courses from universities around the world (eg. StanfordHarvardMIT) offered to anyone with an internet connection.
How do I register?
To register for a course, click on "Go to Class" button on the course page. This will take you to the providers website where you can register for the course.
How do these MOOCs or free online courses work?
MOOCs are designed for an online audience, teaching primarily through short (5-20 min.) pre recorded video lectures, that you watch on weekly schedule when convenient for you. They also have student discussion forums, homework/assignments, and online quizzes or exams.
Mike O'connorcompleted this course, spending 12 hours a week on it and found the course difficulty to be medium.
Terrific class for a person looking to bring interactivity to music or visual art. It's also a great introduction to machine learning that goes deep enough to give you an understanding of the tools without taking you ALL the way into a very deep subject.
Rebecca Fiebrink is not only a well-known authority in this field, she also oversees the development of the open source machine-learning tools that you will learn how to use in conjunction with your own art or music.
In my case I learned how to transform a multi-variable stream of electric-power monitoring data into inputs for an Ableton Live composition. I'm planning to add a stream of weather data and another stream of live game-cameras (using photo-recognition) to the mix. This class gave me the confidence to take on that great hack. :-)
For me, a dream course which puts together some long standing areas of interest. Pragmatically, this course gives you the tools to introduce meaningful gestural control or input to digital music (my interest) as well as a range of other applications which emerge in the course and from the forums.
The software tools provided are accessible with good quality free options as well as well-known paid-for options. The teaching material is top-notch and the most exciting part for me is the way that different machine learning approaches are illustrated in a very accessible way, and I have done the straight math versions too ;)
The course-specific Wekinator tool is really useful and lets you hook up whicheve bits of code you can already work with to each other: that bit is pure genius.
The class is very lightweight, yet gives a solid understanding of how one can apply physic-based models to generate natural looking sound effects. I appreciate that choice of programming language, because the class listeners don't have to waste their time developing building blocks from scratch. I also liked authentic environment used by the lecturer as well as clear and noiseless picture and audio of the lectures. I recommend this class to anyone interested in game development or procedural content generation.
UPD: sorry, this review is for another course from Kadenze.
Simply the best and most inspiring introduction to ML that exists out there. Rebecca manages to take creative students all through the landscape, starting from scratch and giving a hands-on experience that enables newbies to experiment creatively from the outset.
I've given the link to several of my students, and I'm happy to say that the course has been a seminal turnaround point for several of them and their later studio practice as graduates.
I had alot of the suggested equipment so working on this class was straightforward. I appreciate that we focused more on training and use vs writing direct code, while still providing access to the code. It's somewhat of a challenge at first but once you get there it gets fun.
Brilliant. I learned a lot and after that course I started to dig a lot deeper into Machine Learning.
For me personally with a background in informatics the first two sessions started a bit slow but at session three it finally got the pace I enjoyed. But given that this course should reach a broad audience this isn't really a negative point.
This course was super inspiring and open minding , as a musician I had so many great things to take from this course, and ever since I took it I try and incorporate Machine learning in my practices. great quality and great lecturer. Highly recommended
Great course, very helpful and inspirational. I can recommend this course for anyone wanting to get into machine learning, particularly if you're interested in performance / realtime aspects of the field.
Fantastic course - giving artists and musicians the skills to dig into the variety of powerful machine-leaning techniques. Rebecca Fiebrink is a brilliant teacher, clear and entertaining in complex matters - I told my own students to take this class during summer.. .
This course gives an excellent introduction to machine learning, from an arts perspective. It gives you the ability to explore tools and concepts, hands on, learning by doing. It makes Machine Learning accessible and points the way to possibilities.
I like this course. What I learned from this course has taught me how to use my computer in a different way. Using instruments to make sound and how to translate to from computer to music is interesting. Want more classes in this manner.
This was an excellent course - clear, authoritative and makes very complex topics understandable without dumbing them down. It also fills an important gap in the literature by focusing on artistic / interactive uses of ML. Bravo !
I found the course easy to follow and the material relevant to what I wanted to learn. I also liked how easy the software tools were to use. I tried other courses like this one but this was definitely the best.