Service-metric anomaly alert dashboard

The problem you will solve

A fictional service has regular traffic cycles, occasional metric failures and rare injected incidents. Build an anomaly dashboard that helps a reviewer inspect suspicious periods without treating every unusual point as an outage. Unlike demand forecasting, success here depends on useful alerts and low alert fatigue, not prediction accuracy alone.

Before you start

Time-series indexing, rolling statistics, validation splits and basic dashboards.

Suggested tech stack

• Python
• pandas
• NumPy
• SQL
• Streamlit

Data plan

Generate several weeks of synthetic request counts, latency and error rates with a reproducible seed. Inject spikes, level shifts and missing intervals, keeping an incident-label file unavailable to detector fitting. Explicitly label all charts synthetic.

Hardware and cost

A robust rolling-statistic baseline runs on CPU with local files. BigQuery ML is an optional Google Cloud implementation path with billable processing; use bounded queries and a spending plan rather than implying the project is automatically free.

Define the input and output

Use this illustrative contract as a starting point, then specify required fields, validation and error responses in your own implementation. It is not a complete API schema.

Input: timestamp, request_count, error_rate, p95_latency_ms
Output: {"incident_group":"demo-3","start":"...","signal":"error_rate","reason":"above expected range","acknowledged":false}

Your five build milestones

Finish and test one milestone before extending the system. Keep an experiment log with the decision, the alternative you rejected and the evidence behind the choice.

1. Set the alerting objective

   Define which simulated incidents matter and a maximum tolerable alert volume. Distinguish a missing metric from a measured zero so collection failures do not masquerade as healthy traffic.

2. Create a robust baseline

   Estimate normal behaviour using earlier observations only, with weekday or time-of-day patterns where needed. Keep the labeled incident intervals separate from fitting and threshold selection.

3. Group useful notifications

   Convert point anomalies into incident groups with a documented cooldown. Show expected range and observed values, and suppress duplicate notifications without hiding a continuing issue.

4. Evaluate in time order

   Tune thresholds on development periods, then replay a later held-out interval. Include quiet weeks and data gaps, not only obvious injected spikes that make the detector look strong.

5. Build the review dashboard

   Display the signal, incident grouping and reviewer acknowledgment separately. Save a false-alarm gallery and explain why anomaly scores alone do not establish business impact or root cause.

How to evaluate your project

Set your acceptance criteria before looking at the final test results. The checks below are proposed experiments, not measured outcomes or a guarantee that a particular model will pass.

• Report incident-level precision and recall with your incident-matching window, rather than only point-level scores.
• Measure false alerts per day and delay from incident start to first alert on the held-out replay.
• Show results for spikes, persistent shifts and missing-data periods separately, with counts and threshold settings.

Save your test cases, run command, dependency versions and results together. Compare on identical inputs, retain failures in the denominator and report sample sizes. If a change improves one metric but harms another, explain which trade-off fits your use case.

Failure modes worth demonstrating

• Using future observations to estimate an earlier expected range.
• Counting hundreds of alerts for one incident as successful detections or assuming unusual behaviour is necessarily harmful.

Optional extension

Add a reviewer feedback log and assess threshold changes on a separate replay before updating the default detector.

Turn the build into portfolio evidence

Your repository should let another person understand the problem and reproduce the core evaluation without needing your private credentials.

• A README with problem, intended user, architecture, setup commands and a small input/output example.
• A permitted sample dataset, its origin or generator, a frozen test split and a single evaluation command.
• A results table comparing your baseline with your final approach, including failures and sample counts.
• A short demo or screenshots, a limitations section and one decision you would change with more time.
• Environment-variable names without secret values, a dependency list and attribution/license notices for any reused code or datasets.

Resume framing

Describe this as a personal or educational project. State what you implemented, how you tested it and what your measured results were. Replace placeholders with your own evidence; do not claim employer impact, production traffic or accuracy you did not measure.

Built service-metric anomaly alert dashboard using Python, pandas, NumPy; evaluated against [baseline] on [N held-out cases], documented [measured result] and [important limitation].

Be ready to explain your decisions

• Why can better point-level accuracy produce worse alerting?
• How would you separate a collection problem from a service incident?

Answer with a concrete failure from your implementation, the options you considered, your measured evidence and the limitation that remains.

Google documentation and GitHub references

The project scope, milestones and evaluation plans above are original AIJobFever educational material. These official Google resources offer background or implementation examples; they are not finished implementations of this blueprint. No Google endorsement is implied. Review repository licenses, supported platforms and current costs before reusing code or provisioning services.

• Google Cloud: anomaly detection overview ↗