Senior AI Engineer

What matters most

• Own client engagements from discovery to production

  Embed with private equity and venture capital teams, scope high-leverage agent use cases, set technical direction, and ship and harden systems in client environments.

• Build grounded, tool-using agents

  Design production-grade, multi-step agents using Anthropic and OpenAI APIs, with retrieval over each client’s documents and data.

• Make agent quality measurable in production

  Build the supporting vector search and data pipelines, evaluation harnesses that measure agent quality, and production observability.

• Handle enterprise integration and security

  Connect agents to existing systems through APIs and authentication such as SSO, SAML, and OAuth while meeting client security and compliance requirements.

• Exercise senior engineering judgment

  Evaluate AI-native tools against traditional systems, choose what to use, establish engineering standards, and mentor other forward-deployed engineers.

Likely challenges to think through

• Scoping value amid ambiguity

  Likely preparation challenge, not a reported employer problem: an embedded engagement may require choosing a useful first agent while resisting a scope too broad to harden. Prepare to explain how you would validate a use case, define success, and stage delivery through production.

• Balancing agent capability with reliability

  Likely preparation challenge: multi-step tool use and retrieval can make an agent more useful but harder to evaluate and troubleshoot. Prepare a concrete approach to testing agent quality, tracing failures, and using production observations to guide changes.

• Reconciling access with security

  Likely preparation challenge: access to client systems and documents makes agents useful, but enterprise authentication and compliance constrain that access. Prepare to discuss how you would work through API, identity, and data-permission decisions before deployment.

• Choosing tools without overcomplicating delivery

  Likely preparation challenge: AI-native and traditional options may offer different integration effort, control, and maintainability. Prepare to explain a decision framework tied to client needs and how you would turn that choice into standards other engineers can follow.

Topics to prepare

• Agent orchestration and tool use

  The posting calls for multi-step agents using tools, prompt engineering, and Anthropic and OpenAI LLM APIs. Suggested exercise: diagram an agent workflow and explain tool selection, stopping conditions, and how you would handle a failed tool call.

• Retrieval and grounding

  The role requires retrieval over client documents and data, plus vector search. Suggested exercise: compare indexing, retrieval, and grounding choices for a hypothetical document collection; explain how you would detect unsupported answers.

• Agent quality and production operations

  The posting explicitly names evaluation harnesses, production observability, and hardening. Suggested exercise: define quality tests and operational signals for a hypothetical deployed agent, separating answer quality from system reliability.

• Enterprise integration and security

  The role involves REST and GraphQL APIs, SSO, SAML, OAuth, IAM, and client security and compliance requirements. Suggested exercise: sketch an integration boundary and explain how identity, permissions, and data access would be enforced.

• Forward-deployed technical leadership

  The engineer will scope client use cases, set technical direction, evaluate tools against traditional systems, and mentor engineers. Suggested exercise: outline how you would turn an ambiguous client request into a scoped use case, delivery plan, and defensible build-versus-buy decision.

Skills to revise

• Production coding in Python and TypeScript

  Strong Python and TypeScript and a history of shipping production systems are explicit requirements. Suggested exercise: implement a small API-backed workflow across the two languages, then add error handling and tests.

• Implement a tool-using agent

  The posting calls for hands-on LLM applications and agents using Anthropic and OpenAI APIs. Suggested exercise: build a small multi-step tool workflow with one of the named APIs and test successful, failed, and irrelevant tool responses.

• Build a retrieval pipeline

  Data pipelines, retrieval-augmented generation, and vector databases such as Pinecone, Weaviate, or pgvector are named. Suggested exercise: ingest a small synthetic document set into one named vector database, retrieve passages, and use them to ground answers.

• Run agent evaluations and inspect behavior

  Evaluation harnesses and production observability are stated responsibilities. Suggested exercise: create a small test set, measure answer quality and tool-call failures, and record enough execution detail to investigate a bad result.

• Connect an agent to an enterprise-style API

  The posting emphasizes REST and GraphQL integrations alongside enterprise authentication. Suggested exercise: connect a prototype to a mock REST or GraphQL service and demonstrate permission checks and handling of expired or denied access; this is practice, not a stated employer project.

• Prepare a discovery brief

  The role involves scoping agent use cases with private equity and venture capital teams. Draft a hypothetical client brief with discovery questions, a prioritized use case, success measures, and a path from pilot to production; label the scenario as hypothetical.

• Build a production-project walkthrough

  Choose a real system you helped ship, if available. Prepare a concise account of your Python and TypeScript contributions, technical decisions, deployment, and outcomes, with dates and evidence that clarify your production-engineering experience.

• Sketch an agent architecture

  For a hypothetical client-document workflow, draw a multi-step agent using tools, an Anthropic or OpenAI API, retrieval, a data pipeline, and vector search. Explain grounding, access boundaries, and failure handling as proposed design choices.

• Define an evaluation and monitoring plan

  Prepare a sample evaluation set and measures for retrieval and agent quality, plus a production observability plan. Explain how you would investigate failures and decide whether a change is ready to deploy.

• Map enterprise integration risks

  Draft an integration checklist covering REST or GraphQL APIs, SSO, SAML or OAuth, IAM, and security and compliance review. Be ready to distinguish integrations you have implemented from ones you would approach as a design exercise.

• Explain a technical leadership decision

  Prepare one example of choosing between an AI-native tool and a traditional system, including trade-offs and evidence. Add how you set an engineering standard or mentored engineers, if you have a relevant example.

• Bring questions about delivery expectations

  Ask how Dealstitch selects client use cases, defines production success, divides responsibility with client teams, and handles security approval and on-site work in New York.