Member of Technical Staff - Inference Systems

What matters most

• Build trustworthy benchmarks

  The role designs suites covering inference performance, model quality, and knowledge evaluation across hardware targets. Results must be explainable, independently verifiable, and reproducible for internal teams and partners.

• Prove model-port correctness

  Port models such as LFM2 across runtimes and platforms, checking outputs and end-to-end correctness. Multiple LFM2 ports are a stated first-year goal.

• Extend the production inference layer

  Maintain and modify the engine layer built on llama.cpp, ONNX, and MLX as research introduces new architectures. The posting calls for framework-internals experience and performance-sensitive C++ and Python.

• Deliver evidence-based partner evaluations

  Evaluate external solutions against the benchmarks, identify gaps, and communicate findings to external engineering teams. Completing multiple partner verifications end-to-end is a stated first-year goal.

Likely challenges to think through

• Keeping cross-platform comparisons fair

  Preparation hypothesis: hardware and runtime differences could make a faster result misleading if model quality or test conditions change. Prepare to explain how you would control configurations, validate measurements, and document results so others can reproduce them.

• Balancing optimizations against correctness

  Preparation hypothesis: quantization, decoding, or memory-layout changes may improve latency or memory use while altering outputs. Prepare a validation approach that compares intermediate and end-to-end results across runtimes and defines how discrepancies would be investigated.

• Assessing unfamiliar partner solutions

  Preparation hypothesis: limited familiarity with an external tool may conflict with the need for a rigorous, timely evaluation. Prepare to describe how you would establish a comparable baseline, test claims against the benchmark suite, and present gaps with evidence rather than assumptions.

Topics to prepare

• Benchmark methodology

  Review how to control workload, hardware, model settings, and measurement conditions. Exercise: critique a hypothetical comparison for confounders and specify what another team would need to reproduce it. The posting requires sound methodology and independently verifiable results.

• Inference tradeoffs

  Explain how quantization, decoding strategies, and memory layout can interact; sketch which performance and output changes you would check after changing one. These fundamentals are explicitly required.

• Evaluation coverage

  Draft a conceptual test matrix separating inference performance, model quality, and knowledge evaluation across hardware targets. This reflects the benchmark suite the role would build.

• Model-port correctness

  Outline an end-to-end comparison between a reference model and a port, including how you would investigate output discrepancies. Porting models such as LFM2 and proving correctness are stated responsibilities.

• Deployment constraints

  Compare how latency and memory priorities might change between data-center accelerators and on-device hardware. This is preparation for the posting’s range of targets; edge-inference familiarity is nice-to-have, not required.

Skills to revise

• Modify an inference framework

  As a practice exercise, trace an inference path in one of llama.cpp, ONNX Runtime, or MLX; make a small change and test its behavior. The must-have experience goes beyond basic usage into framework internals and modification.

• Build a reproducible benchmark pipeline

  Implement a small pipeline that records model settings, hardware, workload, repeated measurements, and validation checks, then produces results another person can rerun. Designing and building such pipelines is explicitly required.

• Profile performance-sensitive C++ and Python

  Measure a small inference-related operation, identify a bottleneck, change its implementation, and compare performance and outputs before and after. Strong C++ and Python in performance-sensitive contexts are must-haves.

• Verify a cross-runtime port

  As practice, run the same available model and inputs in two of the named runtimes, compare intermediate or final outputs, and document discrepancies. Cross-runtime porting and numerical checks are nice-to-have experience and central to the stated work.

• Present a technical verification

  Write a short findings report for a hypothetical partner solution: benchmark conditions, results, gaps, and evidence supporting each conclusion. External partner verification and clear communication are stated responsibilities.

• Prepare a framework-internals walkthrough

  The posting requires work beyond basic use of an inference framework. If you have a relevant example, prepare a concise walkthrough of a llama.cpp, ONNX Runtime, MLX, or comparable modification: the code changed, why it changed, and how you tested it. Identify your C++ and Python contributions where applicable.

• Draft a reproducible benchmark plan

  As a preparation exercise, outline a suite for inference performance, model quality, and knowledge evaluation across hardware targets. Specify workloads, measurement conditions, validation checks, recorded versions, and the artifacts another team would need to reproduce and explain the results.

• Rehearse a model-port correctness review

  The role involves porting models such as LFM2 across runtimes. Using a port you can discuss, or an explicitly hypothetical one, prepare a comparison plan for intermediate and end-to-end outputs, numerical differences, failure diagnosis, and evidence that the port is correct.

• Work through an inference tradeoff exercise

  Prepare a short design sketch showing how quantization, decoding strategy, and memory layout could affect latency, memory use, and output quality on a data-center versus on-device target. State what you would measure rather than assuming one configuration wins.

• Write a sample partner-verification memo

  The posting calls for evaluating external solutions and clearly reporting gaps. Draft a hypothetical one-page memo that compares a partner result with a controlled benchmark, flags missing evidence or methodology differences, and lists checks needed before drawing a conclusion.

• Bring targeted questions for the team

  Ask which hardware targets and runtimes are priorities, how port correctness is established, what makes benchmark results independently verifiable, and how research, product, and external engineering teams use the suite. These clarify the posted work without assuming undocumented processes.