12/24/2021 1:49:24 PM
New Scientist (paywall) -- Sixty hours of footage were recorded but most of the audio was captured by a single microphone that picked up the musicians' instruments in a noisy jumble rather than a carefully crafted mix. It also recorded background noise and chatter, which made much of the footage unusable.
The team scoured academic papers on using AI to separate audio sources but realized that none of the previous research would work for a music documentary. They consulted with Paris Smaragdis at the University of Illinois Urbana-Champaign and started to create a neural network called MAL (machine assisted learning) and a set of training data that was higher quality than datasets used in academic experiments.