New opportunity

Research Engineer — Agent Architectures (Coding & Autonomous Systems)

AryaXAI · Mumbai, India

About this role

Research agent architectures using coding agents as the primary testbed. The agents work on customer repositories, verify their own changes and produce records of their actions, often within customer networks without internet access. The role targets reliable performance from small models under practical latency and cost constraints.

Responsibilities

Design and test agent harnesses, including tool surfaces, context policies, control topology, verification and inference-time compute allocation. Determine when deterministic program analysis should replace model inference.
Develop memory schemas outside the context window, compaction and retrieval policies suited to closed-world environments, and ways to measure whether an agent improves on a repository over time.
Build evaluations from real repository history with executable verification. Assess long-horizon work, control variance and contamination, and attribute failures to system components.
Use eval traces to isolate failures, test architectural hypotheses through controlled experiments and ablations, and check whether gains hold across repositories and model sizes. Produce papers, benchmarks, released tooling or production architecture as appropriate. Collaborate with alignment and interpretability researchers on post-training and behavioral evaluation.

Requirements

Five or more years in research engineering, systems or ML infrastructure, including at least two years on agentic or LLM-based systems. A PhD in ML, programming languages, systems or a related field, or equivalent industrial research work, counts toward this experience.
Experience building agents for real workloads, creating an evaluation dataset or harness used by others, and owning a system from design through production behavior. Provide a public research or technical record, such as first-author publications, a maintained open-source project, a benchmark or a substantial writeup.
Strong Python, code intelligence and program analysis, scalable containerized execution, sandboxing, observability and debugging. Understand agent frameworks, tool-use under partial failure, SWE-bench-class evaluation and noisy multi-run scoring. Be conversant with post-training, distillation, inference-time scaling, and quantization and serving trade-offs.

Useful but not required: developer tooling or IDE internals, language servers, refactoring engines, compiler work, large monorepo or legacy modernization, RL infrastructure, and production alignment, interpretability or safety tooling. The posting lists the position as full-time and remote in Mumbai, India.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

AI agentsCoding agentsAgent orchestrationLLMsPythonLangGraphLangChainSemantic KernelReActTool-use protocolsEvaluation harnessesSWE-benchStatistical analysisProgram analysisASTstree-sitterStatic analysisDataflow analysisCode searchSymbol indexingCall graphsCodemodsDockerSandboxingDistributed executionOpenTelemetryDistributed tracingStructured loggingSFTDPO schema

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Agent harness architecture
The role designs tool surfaces, context policies, control topology, verification and inference-time compute allocation. It also calls for deciding when deterministic program analysis is preferable to model inference.
Repository memory and code intelligence
The posting emphasizes state outside the context window, compaction, closed-world retrieval and measuring improvement on a repository over time. Repository-scale indexing and program analysis support that work.
Credible evaluations at scale
A core duty is building tasks from real repository history with executable checks, then assessing long-horizon work, noisy multi-run results, variance, contamination and component-level failures. The role also calls for containerized execution at volume.
Experimental research that changes production behavior
The work starts with failures in eval traces and tests architectural explanations through controlled experiments and ablations across repositories and model sizes. Outputs may be papers, benchmarks, released tooling or shipped architecture; demonstrated ownership through production behavior is required.

Likely challenges to think through

Reliability within tight budgets
Likely preparation challenge: balance verification and additional inference against latency and cost while targeting small models. Prepare a concrete decision framework for when to spend model calls, use deterministic analysis or stop and report uncertainty.
Working inside constrained customer environments
Likely preparation challenge: agents may run without internet egress on large, older, thinly tested repositories. Prepare to explain how local indexing, dependency management, sandboxing and executable checks could work under those constraints without assuming comprehensive tests.
Separating real gains from evaluation artifacts
Likely preparation challenge: a higher score may reflect run variance, contamination or a favorable repository rather than a better architecture. Prepare an ablation and multi-run evaluation plan with trace-based failure attribution and checks across repositories and model sizes.
Trusting actions rather than reported traces
Likely preparation challenge: an agent's account of its work may not match its actual tool actions, especially with retries, timeouts or non-idempotent operations. Prepare an approach to replayable logs and artifact inspection that can establish what changed and what verification ran.

Practice your answers

Technical practice

How would you choose between a single agent loop and decomposed control for a coding agent?

What to coverSuggested preparation: compare coordination overhead, error propagation and opportunities for verification under latency and cost limits. Explain how you would test the choice on repository tasks rather than assume one topology is better.

When should program analysis replace model inference in a repository-scale agent?

What to coverSuggested preparation: discuss an index using symbols, dependencies or call graphs, and where deterministic results are more reliable than model guesses. Address stale indexes, language coverage and the cost of building or updating the index.

How would you verify an agent’s change in a large, thinly tested repository?

What to coverSuggested preparation: distinguish executable checks from evidence that a change meets the objective. Explain how you would record unresolved uncertainty when available tests cannot establish correctness.

What should an agent retain outside its context window while working repeatedly on one repository?

What to coverSuggested preparation: propose an execution-state schema and a retrieval policy that work without internet access. Explain what compaction might discard and how you would measure whether retained memory improves later work.

How would you build coding-agent evaluation tasks from repository history?

What to coverSuggested preparation: describe task selection, reproducible repository states and executable verification. Address how you would prevent information from a later change or solution from leaking into the task.

How would you score noisy, partially checkable long-horizon agent runs?

What to coverSuggested preparation: explain what pass@k, best-of-N and majority-vote scores reveal or conceal across repeated trials. Include how you would report variance and separate verified outcomes from work that cannot be fully checked.

How would you determine which system component caused an evaluation failure?

What to coverSuggested preparation: use structured logs, traces and replay to distinguish what the agent did from what its account claims it did. Describe a controlled ablation and how you would check that any gain persists across repositories and model sizes.

How should an agent handle a tool timeout after a potentially non-idempotent action?

What to coverSuggested preparation: discuss how to establish the action’s actual state before retrying, and how to preserve that state in the agent’s record. Compare safe recovery options when a tool response is missing or incomplete.

How would you run thousands of coding-agent trials inside customer-network constraints?

What to coverSuggested preparation: address sandbox isolation, containerized execution, dependencies and artifacts in an on-premise or air-gapped setting. Explain the trade-off between trial throughput, reproducibility and containment.

Where would you spend inference-time compute when targeting small models?

What to coverSuggested preparation: compare additional model calls with better tools, context selection or deterministic checks under latency and cost budgets. Explain how post-training, distillation or serving choices could change that architectural decision without assuming training alone solves it.

Behavioural practice

When have you owned a system from design through production behavior?

What to coverSuggested preparation: use a real situation, your decisions and actions, and the observed result. Distinguish your direct contribution from the team’s, including what production behavior changed your original design.

Tell me about a difficult failure you traced through an agent or distributed system.

What to coverSuggested preparation: describe the situation, the evidence you gathered and the steps that isolated the cause. Close with the result and what you changed to make a similar failure easier to detect or recover from.

When did an apparent research improvement turn out to be an evaluation artifact?

What to coverSuggested preparation: give a real example of the initial result, the check or controlled experiment you ran, and what it revealed. Explain the resulting decision, even if it meant discarding a promising approach.

Describe a time you collaborated across systems and model-behavior research.

What to coverSuggested preparation: choose a real situation where collaborators interpreted evidence differently, then explain your actions and the outcome. Focus on how you reconciled observed behavior with traces or other measurements rather than claiming agreement was automatic.

When have you made a design decision under competing reliability, cost and safety constraints?

What to coverSuggested preparation: set out the loosely specified situation, the alternatives you considered and the decision you owned. State the observed result and any constraint or risk that remained unresolved.

Sharpen your knowledge

Topics to prepare

Coding-agent harness design
Suggested revision exercise: compare a single-loop agent with a decomposed design for one repository change. Specify tool surfaces, context policy, verification steps and inference-time compute limits. The posting makes these harness choices central to the role.
Memory outside the context window
Suggested revision exercise: design an execution-state schema and a compaction policy, then identify what information compaction could destroy and how retrieval would work without internet access. The posting explicitly calls for memory suited to closed-world repositories.
Repository-history evaluation
Suggested revision exercise: outline how to turn a historical repository change into a task with executable verification, including a plan for partially checkable, long-horizon outcomes. This reflects the posting’s evaluation work.
Noisy scores and credible gains
Suggested revision exercise: use hypothetical multi-run results to explain when pass@k, best-of-N and majority vote mislead; propose controls for variance and contamination. The posting asks for statistical treatment and gains that hold across repositories and model sizes.
Program analysis versus model inference
Suggested revision exercise: map a code-change workflow to ASTs, symbol indexes, code search and dependency analysis, marking decisions better handled deterministically. The role explicitly asks where program analysis should replace model inference.
Small-model and deployment trade-offs
Suggested revision exercise: reason through how post-training, distillation, inference-time scaling, quantisation and serving affect a coding agent’s latency, cost and reliability. The posting targets small models under practical budgets and expects fluency, not training specialization.

Skills to revise

Implement fault-tolerant tool use
Suggested revision exercise: build a small Python agent harness that records tool calls and handles a timeout, retry and non-idempotent action without silently duplicating work. The posting requires advanced Python and orchestration under partial failure.
Build a repository-scale code index
Suggested revision exercise: index symbols and references in a sample repository using tree-sitter or ASTs, then explain update costs and limits of the index. The posting asks for code intelligence and the ability to defend an index design.
Run isolated, verifiable trials
Suggested revision exercise: specify a Docker-based task runner that executes a repository change and its checks in isolation, preserving artifacts needed for replay. The posting calls for sandboxed, containerised evaluation at scale, including air-gapped settings.
Analyze architectural ablations
Suggested revision exercise: define a baseline, change one harness component, repeat runs and compare outcomes across repositories and model sizes. The posting emphasizes controlled experiments rather than prompt tuning.
Debug from traces and observed effects
Suggested revision exercise: instrument an agent run with structured logs and OpenTelemetry traces, then classify a failed change by component and check the repository outcome against the agent’s account. The posting requires end-to-end debugging and attention to what the agent actually did.

Your preparation checklist

Select a project walkthrough
Prepare an account of a real agent or ML system you owned, if applicable: workload, your design decisions, production behavior, a failure you debugged, and measured outcomes. Identify a public paper, repository, benchmark, or technical writeup the employer can review.
Sketch a coding-agent harness
Draw a proposed agent loop for changing a repository and verifying the result. Explain tool permissions, context selection, retries and non-idempotent actions, and where deterministic program analysis should replace model inference.
Defend a repository intelligence and memory design
Prepare a design exercise covering symbol indexing or code search, execution state outside the context window, compaction losses, and retrieval without internet access. State how you would test whether the agent improves on a repository over time.
Specify a repository-history evaluation
Draft a task-construction and scoring plan using historical changes and executable checks. Include sandboxed runs, partially checkable long-horizon tasks, contamination controls, multi-run variance, and limits of pass@k or best-of-N.
Walk through an architectural experiment
Choose an illustrative agent failure and prepare a trace-to-hypothesis-to-ablation walkthrough. Specify component-level failure attribution and how you would check that a gain holds across repositories and model sizes.
Explain execution and debugging trade-offs
Bring a concrete design for scalable containerized trials in an on-premise or air-gapped setting, with artifact management, structured logs, traces, and replay. Be ready to discuss latency, cost, reliability, and sandbox boundaries.
Prepare a small-model trade-off discussion
Outline when you would change the harness versus use post-training, distillation, or more inference-time compute. Explain how quantization and serving choices could affect the role’s small-model latency and cost targets.
Write questions for the research team
Ask how they verify agent actions beyond self-reported traces, which long-horizon outcomes matter most, and how they decide between publishable benchmarks or tooling and architecture that must remain inside customer networks.

Availability last checked: 5 Nov 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…