Software Engineer, AI/ML

What matters most

• Build a safe agent runtime

  The role owns planning, tool use, orchestration, and guardrails for agents taking actions on customer data. Prepare to explain architectural choices for controlling and tracing those actions.

• Make customer playbooks usable

  Kya calls for memory and retrieval systems that help agents learn each customer's playbooks. Focus preparation on how retrieved context supports useful actions and how its quality can be assessed.

• Own model orchestration in production

  The engineer will integrate LLMs, including Claude and OpenAI, and manage prompting, context, routing, cost, and latency. Prepare to discuss trade-offs behind routing and context decisions rather than model choice alone.

• Measure improvement and catch regressions

  The posting calls for evaluation harnesses and observability that show whether agents are improving and detect regressions before customers do. Prepare a concrete approach to evaluating agent actions over time.

• Ship and evolve the platform end to end

  This founding engineer will make architectural decisions, ship APIs, pipelines, and infrastructure across customers' software stacks, and use Kya to inform platform improvements. Prepare to connect production ownership with fast iteration and feedback from founders and customer-facing colleagues.

Likely challenges to think through

• Agent autonomy versus safe action

  Likely preparation challenge, not a stated problem: more capable tool use can increase the risk of an incorrect action on customer data. Be ready to reason through permission boundaries, checks before consequential actions, and how to retain useful automation.

• Useful memory versus trustworthy context

  Likely preparation challenge: playbooks and retrieved information may be incomplete or stale, while agents still need relevant context to act. Prepare to explain how you would test retrieval quality and handle uncertainty before an agent uses that context.

• Rapid shipping versus production reliability

  Likely preparation challenge: a short idea-to-agent loop must coexist with reliable APIs, pipelines, and infrastructure at the scale described in the posting. Prepare an example of what you would validate before release, what you would monitor afterward, and how cost or latency would affect the design.

Topics to prepare

• Agent planning, tool use, and safety

  Suggested exercise: diagram how an agent plans an onboarding task, selects a tool, checks permissions, and handles a failed or unsafe action. Relevant because the role designs agent orchestration and guardrails for actions on customer data.

• Customer playbook memory and retrieval

  Suggested exercise: compare ways to store and retrieve account-specific playbooks, including how to prevent one customer's information from appearing in another's results. Relevant because the role owns agent memory and retrieval; RAG fluency is explicitly requested.

• LLM orchestration tradeoffs

  Suggested exercise: explain how you would choose prompts, context, and model routing for a task while weighing quality, latency, and cost. Relevant because the posting names Claude and OpenAI integration and assigns ownership of these production tradeoffs.

• Agent evaluation and observability

  Suggested exercise: define measures for successful, safe customer-account actions and a method for detecting regressions before release. Relevant because the role builds evaluation and observability systems, and evaluation fluency is required.

• Production architecture

  Suggested exercise: sketch the APIs, data pipelines, and distributed components behind an agent action; explain failure handling and reliability tradeoffs. Relevant because the role ships production services at scale and calls for strong fundamentals in these areas.

Skills to revise

• Implement a guarded agent action

  Suggested exercise: build a small agent flow that calls a mock customer-data tool, checks authorization before acting, and records the outcome. This rehearses the posted responsibility for tool use and safe action; the mock flow is a preparation exercise, not a specified employer test.

• Build a playbook retrieval prototype

  Suggested exercise: index sample playbook entries for two fictional accounts, retrieve relevant passages for an onboarding question, and test account isolation. This practices the required RAG and retrieval skills and the role's memory-layer work.

• Orchestrate an LLM request

  Suggested exercise: implement a small request flow with context selection and a configurable model choice, then record response time and estimated usage cost. This maps to the role's LLM API, context-engineering, routing, latency, and cost responsibilities.

• Create an evaluation loop

  Suggested exercise: assemble a small set of agent scenarios with expected outcomes, run candidate changes against them, and inspect failures alongside action logs. This practices building the evaluation and observability systems described in the posting.

• Ship a resilient agent-facing API

  Suggested exercise: expose a mock agent action through an API and test retries, duplicate requests, and partial pipeline failures. This practices the production API, pipeline, and reliability work explicitly assigned to the role.

• Prepare a production project walkthrough

  Select a real LLM- or ML-powered feature you shipped, if available. Prepare a concise account of your ownership, architecture, users, deployment, outcomes and tradeoffs; Kya asks for hands-on delivery to real users and end-to-end ownership.

• Sketch a safe agent runtime

  As a practice design exercise, diagram planning, tool use and orchestration for an onboarding agent. Explain how you would authorize actions on customer data, handle failures and prevent unsafe actions.

• Map a playbook retrieval flow

  Prepare a hypothetical design showing how customer playbooks enter a memory and retrieval layer, how relevant context reaches an agent, and how you would test retrieval quality. Kya lists this layer as a core responsibility.

• Compare model-routing choices

  Make a short decision table for prompting, context and routing across LLM APIs, including Claude and OpenAI. State how you would measure quality, latency and cost rather than assuming one model is always best.

• Draft an evaluation and observability plan

  Propose example tasks, success and safety measures, regression checks and production signals for an agent taking customer-account actions. Label the plan as your proposal; the role calls for evaluation systems that detect regressions.

• Gather evidence of production engineering judgment

  Prepare concrete examples, where available, of API design, data pipelines or distributed systems work. For each, note a reliability or correctness decision, a latency or cost tradeoff, and what changed after shipping.

• Write questions for the founders and customer-facing team

  Ask how they assess agent improvement, what customer-data actions need the strongest guardrails, which integrations matter most, and how feedback from using Kya informs platform changes. These are questions to investigate, not assumed company practices.