YAMNet Does Not Detect Specific Sounds

Description

We are using the YAMNet model for real-time environmental sound detection.

We tested the same type of dog barking audio in two implementations:

• Android: Kotlin implementation
• Windows: Electron application using @mediapipe/tasks-audio

On the Android/Kotlin implementation, YAMNet correctly identifies the dog barking sound with the specific Dog bark classification.

However, when using YAMNet through @mediapipe/tasks-audio in our Electron application on Windows, the same type of dog barking sound is not classified specifically. Instead, it is usually classified with generic labels such as:

Animal
Wild animals

For example, we see results such as:

[Noise Check] Generic Top: Animal (Score: 0.67)
[Noise Check] Generic Top: Wild animals (Score: 0.59)
[Noise Check] Generic Top: Animal (Score: 0.33)

Expected behavior

The Electron implementation should produce a similar specific classification to the Android/Kotlin implementation when detecting the same dog barking sound.

Actual behavior

The Android/Kotlin implementation detects Dog bark, while the Electron/Windows implementation tends to classify the same type of sound as the more generic Animal or Wild animals categories.

Questions

1. Why could YAMNet produce a specific Dog bark prediction in the Android/Kotlin implementation but only a generic Animal prediction in the Electron/Windows implementation?
2. Could this difference be caused by differences in the audio input pipeline between Android/Kotlin and Web/Electron, such as sample rate, resampling, buffering, audio frame size, channel configuration, or audio format?
3. Are there any recommended audio preprocessing or configuration steps for @mediapipe/tasks-audio in Electron to ensure that the input is equivalent to the Android implementation?
4. Is there a recommended way to verify that the same audio samples are being passed to the YAMNet model on both platforms?
5. What would be the recommended approach to ensure that the Electron/Windows implementation produces the correct and specific classifications, consistent with the Android/Kotlin implementation?

Environment

Android

• Language: Kotlin
• Model: YAMNet
• Input: Microphone audio

Windows

• Application: Electron
• Language: JavaScript
• Package: @mediapipe/tasks-audio
• Model: YAMNet
• Input: Microphone audio