Exploring Griffin–Lim

Tacotron predicts how strong each frequency should be, but leaves out phase. Phase says where each wave is in its cycle. Without it, the spectrogram cannot give us the original sound directly.

Griffin–Lim fills in that missing piece by making a guess, turning it into audio, measuring the audio again, and keeping the new phase. This small NumPy implementation lets me hear that loop.

Listen

Each reconstruction starts with the same frequency magnitudes from a synthetic vowel-like sound. The first clip is the original.

The loop

The short-time Fourier transform (STFT) cuts audio into overlapping windows and runs an FFT on each one. Its result has magnitude and phase. Here we keep the original magnitude and throw away the original phase.

phase = np.exp(2j * np.pi * rng.random(magnitude.shape))

for _ in range(iterations):
    audio = istft(magnitude * phase, length, n_fft, win_length, hop_length)
    rebuilt = stft(audio, n_fft, win_length, hop_length)
    phase = rebuilt / np.maximum(np.abs(rebuilt), 1e-12)

return istft(magnitude * phase, length, n_fft, win_length, hop_length)

The inverse STFT overlaps the little audio windows and divides by their accumulated window weights. That keeps the reconstruction from getting louder where windows overlap. The full script has both transforms, WAV reading and writing, and the generated example.

What changed?

I compared the magnitude of each reconstructed clip with the magnitude we started with. Lower is closer.

Each round makes the audio's measured spectrum fit the target better. It still does not recover the original phase, so the result need not sound identical to the source.

The example uses Tacotron's 24 kHz sample rate, 50 ms window, 12.5 ms hop, and 2048-point FFT. Tacotron also raises predicted magnitudes to the power of 1.2 before reconstruction; I left that out here so the loop works against the exact magnitudes used in the comparison.

Run it

With NumPy installed, run python3 griffin_lim.py to regenerate these clips. To try your own recording, pass --input voice.wav --output out. The input needs to be a mono, 16-bit PCM WAV file.