[Idea] Compute key similarity over the log-scale Mel spectrogram
Author: ggerganovCreated Oct 2, 2022Updated Oct 2, 2022
Labelsenhancement
Currently, we compute the cross-correlation between time-domain key waveforms to determine how similar 2 keys are. Instead, we can compute the similarity metric over the Mel spectrograms of the signals. The Mel spectrogram seems to be the go-to choice for audio representation in modern state-of-the-art speech recognition algorithms, so why not give it a try in keytap.
Here is a sample implementation to compute the log-scaled Mel spectrogram of an audio, that I recently did for the whisper.cpp project:
Source: ggerganov/kbd-audio