Engineer, Staff - IEIOT AI Systems

What matters most

• NPU and DSP Inference Optimization

  The core of this role is optimizing AI/ML models specifically for Hexagon NPU and DSP architectures. You must demonstrate the ability to implement and tune deep learning operators on SIMD/VLIW architectures to ensure high-performance execution on Snapdragon chipsets.

• Model Quantization and Toolchain Management

  You will be responsible for managing quantization schemes and utilizing internal toolchains to control model loading, execution, and scheduling. This requires a deep understanding of the trade-offs between model precision and hardware performance.

• System-Level Concurrent Execution Analysis

  The role requires performing system-level analysis for concurrent ML model execution and daisy-chaining models (e.g., Vision AI, Gen AI, and VLAs). You must be able to evaluate how multiple models interact within the constraints of a system-on-chip micro-architecture.

Likely challenges to think through

• Hardware-Software Co-Design Complexity

  Bridging the gap between high-level ML frameworks like PyTorch/TensorFlow and low-level hardware constraints (SIMD/VLIW) is difficult. Prepare by discussing how you have previously mapped abstract graph operations to specific hardware primitives to resolve performance bottlenecks.

• Resource-Constrained Optimization

  Deploying LLMs and VLAs on-device requires balancing accuracy with strict memory and power budgets. Be prepared to explain your methodology for selecting quantization strategies that maintain model utility while meeting the stringent latency requirements of IoT and mobile platforms.

Topics to prepare

• AI Model Quantization and Optimization

  Review the trade-offs between different quantization schemes (e.g., INT8 vs. FP16) and their impact on model accuracy and latency. Practice identifying graph optimization techniques such as operator fusion and constant folding, which are critical for deploying models on resource-constrained NPU and DSP architectures.

• System-on-Chip (SoC) Micro-architecture

  Study the interaction between CPUs, GPUs, NPUs, and DSPs in modern SoC designs. Focus on how data movement and memory bandwidth constraints affect concurrent ML model execution and daisy-chaining, as this is essential for optimizing on-device IoT and Vision AI applications.

Skills to revise

• Embedded C/C++ and Assembly Programming

  Practice writing high-performance code for SIMD and VLIW architectures. Focus on manual memory management and low-level optimization techniques, as these are required for implementing deep learning operators directly on NPU or DSP hardware.

• Inference Frameworks (TensorFlow/TFLite/PyTorch/ONNX)

  Develop proficiency in converting and deploying models across these frameworks. Specifically, practice using tools to inspect model graphs and customize execution runtimes to ensure compatibility with specialized hardware accelerators.

• Review NPU and DSP Architecture Fundamentals

  Prepare to discuss how deep learning operators are mapped to SIMD/VLIW architectures. Focus your study on how Hexagon NPU and DSP architectures handle parallel execution and memory constraints for on-device AI.

• Prepare a Technical Walkthrough of Model Optimization

  Select a project where you performed model quantization or graph optimization. Be ready to explain the trade-offs between precision (e.g., INT8 vs FP16) and performance, and how you utilized frameworks like TFLite or ONNX to achieve these results.

• Demonstrate Embedded Systems Proficiency

  Review your experience with C/C++ and Assembly in resource-constrained environments. Prepare examples of how you have optimized code for specific hardware targets, focusing on latency reduction and efficient resource scheduling.

• Analyze Concurrent Model Execution Strategies

  Develop a conceptual approach for daisy-chaining multiple AI models (e.g., Vision AI or VLA) on a single System-on-Chip. Be prepared to discuss how you manage memory, power, and scheduling when multiple models compete for NPU/DSP resources.

• Formulate Targeted Questions for the Team

  Prepare questions regarding the current challenges in deploying GenAI on IoT devices, the specific toolchains used for model loading and scheduling, and how the team balances performance requirements across different Snapdragon chipset components.