Member of Technical Staff - Post-Training Engineer

What matters most

• Ship post-training improvements end to end

  The role owns data curation, SFT or preference optimization on open-weight models, evaluation, and production deployment. The posting calls for a production-model improvement within two weeks and a measurable lift in one modality by day 60.

• Turn difficult inputs into useful training data

  Handwritten multilingual deeds, skewed scans, regional-dialect calls, maps, and imagery are central to Terra’s data. The role includes designing annotation taxonomies and converting production transcripts and failures into curated training data.

• Make agent answers evidence-grounded

  Terra needs trained tool use, multi-step reasoning, code-switching, and decisions about when to answer or escalate. Structured outputs, constrained decoding, and citations are priorities because lenders and families act on its judgments.

• Own one modality while shaping broader direction

  Document understanding is the current focus; voice agents, video, and geospatial reasoning are on the near-term roadmap. By day 60, the engineer is expected to own a document VLM, voice, or agent-behavior track; by day 90, recommend an evidence-backed structural post-training direction.

• Use evaluations to gate releases

  The posting calls for modality- and agent-specific evaluations, error analysis, and regional quality floors. Preparation should center on how an evaluation is defined before training and how regressions affect a release decision.

Likely challenges to think through

• Helpful answers versus unsupported certainty

  Likely challenge: an agent may produce a fluent answer when the underlying evidence is incomplete or conflicting. Prepare a concrete approach to checking citations against source material, training escalation behavior, and measuring unsupported claims alongside answer quality.

• Overall gains versus regional regressions

  Likely challenge: improvements on common documents or languages could hide failures in another region or input type. Prepare a stratified evaluation plan with regional floors, failure analysis, and explicit release criteria rather than relying on one aggregate score.

• Fast delivery versus dependable evidence

  Likely challenge: the two-week production improvement and day-60 quality lift leave limited time for broad experimentation. Prepare to explain how to select a narrow failure mode, establish a baseline, run reproducible training, and validate a measurable improvement before proposing a larger day-90 bet.

Topics to prepare

• SFT versus preference optimization

  Compare when you would use SFT or DPO/GRPO-class methods to improve an open-weight model on difficult property documents. Sketch the training data, baseline evaluation and likely failure modes for each. The role covers both approaches and asks for evidence-backed post-training decisions.

• Document and non-text distribution shifts

  Map how handwriting, six languages, skewed scans and survey maps could break document understanding; then choose one non-text modality you know well and outline its distinct failure modes. Document understanding is the current focus, while the role requires depth in at least one modality beyond text.

• Agent decisions and evidence grounding

  For a hypothetical property question with conflicting document evidence, diagram when an agent should use a tool, answer with citations or escalate. The posting calls for multi-step agent behavior and provenance-grounded answers; this is a preparation exercise, not a stated decision policy.

• Evaluation and release gates

  Draft a scorecard that separates document accuracy, unsupported claims, agent decisions and regional performance; specify what regression would block release. The posting explicitly calls for modality- and behavior-specific evaluations with regional quality floors.

• Data taxonomy and post-training strategy

  Classify a small set of hypothetical transcript and document failures into annotation categories, then argue which category to address first and how you would measure a lift. The role turns production failures into training data and expects a measurable improvement and an evidence-backed structural recommendation.

Skills to revise

• Run a reproducible fine-tuning experiment

  Using permitted sample data, rehearse a small open-weight SFT run in PyTorch and the Hugging Face training ecosystem. Record data version, configuration, GPU usage, experiment results and a repeatable evaluation. Personally running and shipping such work is an explicit requirement; this exercise does not substitute for shipped experience.

• Build a failure-to-data pipeline

  Write Python code that takes synthetic document or call failures through categorization, review and train/evaluation splits, with checks for duplicates and leakage. Data-curation pipelines and reproducible training are required for the role.

• Implement an evaluation comparison

  Create a small evaluation script that reports results by language or region and error category, then compare a baseline with a candidate and explain one regression. This practices the posting’s evaluation-driven error analysis and release-gate expectations.

• Prototype provenance-grounded generation

  Define a structured answer schema with evidence references, generate a few synthetic property-answer cases using constrained decoding, and test missing or unsupported citations. Schema design and constrained decoding are named responsibilities.

• Rehearse the end-to-end systems work sample

  Choose one posted track—document VLMs, voice or agent behavior—and prepare a concise data → training → evaluation gates → serving design with a measurable success criterion and deployment risks. The advertised work sample and systems round explicitly probe this end-to-end reasoning.

• Select an end-to-end post-training project

  For the advertised deep-dive, prepare a walkthrough of an open-weight SFT or preference-optimization job you personally ran and shipped, if available: data choices, training setup, evaluation findings, production result, and what you would redo. Do not imply hands-on work you did not do.

• Document your strongest non-text modality

  Prepare concrete evidence from vision-language, speech, video, or geospatial work, if available. Show the failure cases, data-curation decisions, PyTorch/Hugging Face and GPU workflow, experiment tracking, and steps used to reproduce results.

• Draft a Terra work-sample approach

  For the advertised work sample, outline a data strategy, training plan, and success measures in your strongest modality. Document understanding is a current focus; voice agents, video, and geospatial reasoning are described as near-term work.

• Design evidence-grounded behavior

  As a preparation exercise, sketch an output schema that links property claims to source evidence, plus examples of when an agent should use a tool, answer, or escalate. Explain how you would test unsupported claims and code-switching failures.

• Rehearse the full post-training loop

  Practice explaining curation → training → evaluation gates → serving for Terra's multimodal agents. Include modality-specific tests, regional quality floors, regression analysis, and how you would respond to a failed release gate.

• Outline a first-90-days plan

  Prepare a tentative plan aligned with the posting: investigate real failures and target an early production improvement, own one model surface and measurable modality track, then recommend a structural training direction supported by evidence. Label proposed actions as your approach, not established company plans.

• Bring questions for the employer

  Ask which model surface needs attention first, how evidence and regional quality thresholds are defined, what data can be used in the work sample, and what would count as a meaningful early production improvement.