Hand-gesture presentation controller

The problem you will solve

A presenter wants an optional hands-free way to navigate a demonstration deck. Build a gesture controller that acts only after a stable intentional gesture, rather than triggering on every camera frame. This is a convenience prototype, not a certified accessibility device or a control system for safety-critical actions.

Before you start

Browser events, asynchronous JavaScript, basic camera permissions and simple state management.

Suggested tech stack

• JavaScript
• MediaPipe
• Webcam
• HTML
• CSS

Data plan

Create a consented test protocol using your own hands and a short non-sensitive slide deck. Record lighting, camera distance and intended gesture. Prefer logging predicted labels and timings instead of storing camera footage.

Hardware and cost

Try the supported MediaPipe web runtime on a compatible browser with a small camera resolution. Actual frame rate depends on the device. Keep processing local where configured, provide a stop-camera control and do not promise universal hardware support.

Define the input and output

Use this illustrative contract as a starting point, then specify required fields, validation and error responses in your own implementation. It is not a complete API schema.

Input: live camera frame + explicitly enabled controller
Output: {"gesture":"Open_Palm","stable_ms":800,"action":"next_slide"}
States: disabled / ready / holding / cooldown; keyboard navigation always remains available.

Your five build milestones

Finish and test one milestone before extending the system. Keep an experiment log with the decision, the alternative you rejected and the evidence behind the choice.

1. Define deliberate actions

   Choose two supported gesture labels and map them to next and previous slide. Specify a hold duration, cooldown and a neutral reset condition before wiring any slide actions.

2. Start and stop safely

   Ask for camera access only after a user action. Display whether recognition is active, handle permission denial and stop media tracks when disabled or leaving the demo.

3. Recognize without side effects

   Run the gesture recognizer and show predicted labels before connecting controls. Test framing and lighting, and separate an absent hand from an unrecognized gesture.

4. Add the action state machine

   Require stable predictions across time and permit one navigation event per accepted gesture. Reset on neutral input and keep the keyboard buttons working when recognition stalls.

5. Run a usability drill

   Use a declared set of intended gestures and ordinary non-command movements. Log accidental activations, missed commands and reaction time, then publish the limitations and a local-only privacy explanation.

How to evaluate your project

Set your acceptance criteria before looking at the final test results. The checks below are proposed experiments, not measured outcomes or a guarantee that a particular model will pass.

• Count unintended slide changes during neutral movements as well as correct responses to deliberate gestures.
• Measure median command delay and missed commands under the same lighting and distance conditions.
• Verify camera stop, permission denial and keyboard fallback behaviour; report participant and trial counts without generalizing beyond them.

Save your test cases, run command, dependency versions and results together. Compare on identical inputs, retain failures in the denominator and report sample sizes. If a change improves one metric but harms another, explain which trade-off fits your use case.

Failure modes worth demonstrating

• Advancing one slide per frame while a gesture is held.
• Keeping the camera active after disabling controls or excluding users who cannot perform the selected gestures.

Optional extension

Add a calibration screen for hold duration and cooldown, with settings that can be restored to safe defaults.

Turn the build into portfolio evidence

Your repository should let another person understand the problem and reproduce the core evaluation without needing your private credentials.

• A README with problem, intended user, architecture, setup commands and a small input/output example.
• A permitted sample dataset, its origin or generator, a frozen test split and a single evaluation command.
• A results table comparing your baseline with your final approach, including failures and sample counts.
• A short demo or screenshots, a limitations section and one decision you would change with more time.
• Environment-variable names without secret values, a dependency list and attribution/license notices for any reused code or datasets.

Resume framing

Describe this as a personal or educational project. State what you implemented, how you tested it and what your measured results were. Replace placeholders with your own evidence; do not claim employer impact, production traffic or accuracy you did not measure.

Built hand-gesture presentation controller using JavaScript, MediaPipe, Webcam; evaluated against [baseline] on [N held-out cases], documented [measured result] and [important limitation].

Be ready to explain your decisions

• Why does gesture recognition need a separate action state machine?
• How would you reduce accidental activation without making controls frustratingly slow?

Answer with a concrete failure from your implementation, the options you considered, your measured evidence and the limitation that remains.

Google documentation and GitHub references

The project scope, milestones and evaluation plans above are original AIJobFever educational material. These official Google resources offer background or implementation examples; they are not finished implementations of this blueprint. No Google endorsement is implied. Review repository licenses, supported platforms and current costs before reusing code or provisioning services.

• Google AI Edge: gesture recognition guide and examples ↗