The problem you will solve
A learner wants to understand when a small audio classifier distinguishes speech, music and everyday sounds. Build an explorer for short permitted clips with category scores and timestamps. This is sound classification rather than transcription, and it must not be presented as a reliable emergency alarm, surveillance or medical monitoring system.
Before you start
Audio sample rates, basic Python arrays and classification metrics.
Suggested tech stack
Data plan
Record your own non-sensitive sounds or use clips with explicit reuse permission. Choose three categories supported by the selected model, add quiet and mixed-sound cases, and keep variants of each original clip in one evaluation group.
Hardware and cost
Start with a supported small MediaPipe audio model on a compatible CPU platform. Follow its sample-rate and format requirements; performance varies by runtime and hardware. A file-upload demo avoids continuous microphone collection and requires no paid model API.
Define the input and output
Use this illustrative contract as a starting point, then specify required fields, validation and error responses in your own implementation. It is not a complete API schema.
Input: permitted audio file + declared sample rate
Output: {"segments":[{"start_ms":0,"labels":[{"name":"Speech","score":0.61}]}],"decision":"uncertain"}
Expose only categories supported by the selected model.Your five build milestones
Finish and test one milestone before extending the system. Keep an experiment log with the decision, the alternative you rejected and the evidence behind the choice.
Pick supported categories
Inspect the selected model's labels and define three categories for the experiment. Document how silence, mixed sounds and an unknown category should be shown rather than forcing every clip into a chosen class.
Validate audio inputs
Check file duration, format and sample rate before inference. Resample only through a documented path and keep original files separate so preprocessing changes can be evaluated.
Run timestamped inference
Apply the classifier to valid clips and retain per-chunk category scores. Explain that a chunk-level prediction need not describe every sound across the full recording.
Choose a cautious decision rule
Tune a threshold and any temporal smoothing on development clips. Keep an uncertain output and test overlapping sounds instead of treating the largest score as a guaranteed correct label.
Build the error explorer
Show permitted playback, chunk timings and reference labels side by side. Publish a confusion matrix and examples of noise-induced errors, with controls to clear uploaded files.
How to evaluate your project
Set your acceptance criteria before looking at the final test results. The checks below are proposed experiments, not measured outcomes or a guarantee that a particular model will pass.
- Report per-category precision/recall and macro-F1 on held-out original recordings, stating how chunk predictions become clip decisions.
- Measure unknown/uncertain behaviour on silence and mixed sounds separately from the three known categories.
- Compare clean and noisy variants without counting them as independent original recordings; report latency and test-set size.
Save your test cases, run command, dependency versions and results together. Compare on identical inputs, retain failures in the denominator and report sample sizes. If a change improves one metric but harms another, explain which trade-off fits your use case.
Failure modes worth demonstrating
- Training or tuning on augmented copies of the final test recordings.
- Presenting a non-calibrated score as certainty or deploying a demo as a safety-critical alerting system.
Optional extension
Add an optional, explicitly started local microphone mode with a visible listening indicator and an immediate stop control.
Turn the build into portfolio evidence
Your repository should let another person understand the problem and reproduce the core evaluation without needing your private credentials.
- A README with problem, intended user, architecture, setup commands and a small input/output example.
- A permitted sample dataset, its origin or generator, a frozen test split and a single evaluation command.
- A results table comparing your baseline with your final approach, including failures and sample counts.
- A short demo or screenshots, a limitations section and one decision you would change with more time.
- Environment-variable names without secret values, a dependency list and attribution/license notices for any reused code or datasets.
Resume framing
Describe this as a personal or educational project. State what you implemented, how you tested it and what your measured results were. Replace placeholders with your own evidence; do not claim employer impact, production traffic or accuracy you did not measure.
Built environmental sound classification explorer using Python, MediaPipe, NumPy; evaluated against [baseline] on [N held-out cases], documented [measured result] and [important limitation].Be ready to explain your decisions
- How is sound classification different from speech transcription?
- How did you aggregate chunk predictions without hiding overlapping sounds?
Answer with a concrete failure from your implementation, the options you considered, your measured evidence and the limitation that remains.
Google documentation and GitHub references
The project scope, milestones and evaluation plans above are original AIJobFever educational material. These official Google resources offer background or implementation examples; they are not finished implementations of this blueprint. No Google endorsement is implied. Review repository licenses, supported platforms and current costs before reusing code or provisioning services.
Our editorial policy