Putting it all together

The goal of this chapter is to provide a fully annotated and functional script for training a vocals separation model using nussl and Scaper, putting together everything that we’ve seen in this tutorial thus far. So that this part runs in reasonable time, we’ll set up our model training code so that it overfits to a small amount of data, and then show the output of the model on that data. We’ll also give instructions on how to scale your experiment code up so that it’s a full MUSDB separation experiment.

We’ll have to introduce a few concepts in nussl that hasn’t been covered yet that will make our lives easier. Alright, let’s get started!

%%capture
!pip install scaper
!pip install nussl
!pip install git+https://github.com/source-separation/tutorial

Getting the data

The first concept we’ll want to be familiar with is that of data transforms. nussl provides a transforms API for audio, much like the one found in torchvision does for image data. Remember all that data code we built up in the previous section? Let’s get it back, this time by just importing it from the common module that comes with this tutorial:

%%capture
from common import data, viz
import nussl
# Prepare MUSDB
data.prepare_musdb('~/.nussl/tutorial/')

The next bit of code initializes a Scaper object with all the bells and whistles that were introduced in the last section, then wraps it in a nussl OnTheFly dataset. First, we should set our STFTParams to what we’ll be using throughout this notebook:

stft_params = nussl.STFTParams(window_length=512, hop_length=128, window_type='sqrt_hann')
fg_path = "~/.nussl/tutorial/train"
train_data = data.on_the_fly(stft_params, transform=None, fg_path=fg_path, num_mixtures=1000, coherent_prob=1.0)

Let’s take a look at a single item from the dataset:

item = train_data[0]
viz.show_sources(item['sources'])
../_images/putting_it_all_together_7_0.png