New opportunity

Machine Learning

Cantor Fitzgerald Technology Markets LLC · New York, United States

About this role

Cantor Fitzgerald Technology Markets LLC seeks a Machine Learning Engineer to build AI-driven solutions for a high-volume financial services business. The role focuses on creating, testing, and operationalizing LLM applications with attention to performance, reliability, and responsible AI.

Responsibilities

Design and implement production LLM features; build pipelines for structured and unstructured data; and write clean, tested Python code and reusable libraries.
Develop prompts, tool-calling workflows, and retrieval pipelines. Build evaluation suites and test sets, define success metrics, review failures, and improve prompts or retrieval. Diagnose and mitigate hallucination, latency, and cost issues.
Monitor AI services through logging and alerting, contribute to responsible-AI guardrails and human-in-the-loop processes, and document designs, experiments, and findings. Collaborate with product, engineering, and business teams.

Qualifications

Bachelor’s degree in computer science, machine learning, mathematics, physics, statistics, or econometrics, or equivalent practical experience. Experience contributing to production or production-like software through work, internships, research, open source, or substantial personal projects.
Strong Python programming; experience with web services, data integrations, testing, logging, and basic monitoring across diverse data types. Hands-on use of LLM tools or frameworks for prompting, structured outputs, tool calling, retrieval, or multi-step workflows, including awareness of failure modes and experience evaluating LLM applications.
Grounding in machine learning, statistics, and experimental design; ability to interpret technical papers and documentation; strong communication and cross-functional collaboration. Interest in responsible financial-services AI, including privacy, security, human review, and appropriate automation. Familiarity with cloud deployment, containers, and modern release pipelines.

The position is full-time in New York, NY. Listed compensation is $140,000–$160,000; the posting does not specify a pay period.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

PythonMachine LearningLarge Language Models (LLMs)Prompt EngineeringTool CallingRetrievalData PipelinesStructured DataUnstructured DataWeb ServicesData IntegrationSoftware TestingLLM EvaluationExperimental DesignStatisticsMonitoringLoggingAlertingCloud DeploymentContainersRelease PipelinesResponsible AICommunication

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Production-ready LLM features
The role calls for LLM features in production systems, clean and tested Python, reusable libraries, and integration with web services. Prepare to explain how you would take a feature from prototype to a maintainable service.
Prompts, tools, and retrieval
The posting names prompts, tool-calling workflows, retrieval pipelines, and structured outputs. Prepare to describe how these components fit together and where each can fail.
Evidence-based iteration
Building test sets, defining success metrics, reviewing failures, and improving prompts or retrieval are explicit duties. Prepare an example of an evaluation approach that supports a clear decision about what to change.
Reliable and responsible operation
The role includes monitoring, logging, alerting, responsible-AI guardrails, human review, and documentation. Prepare to discuss how an AI service would be observed, reviewed, and explained to product, engineering, and business partners.

Likely challenges to think through

Balancing answer quality, speed, and cost
A likely design challenge is reducing hallucinations without making retrieval or model calls too slow or expensive; the posting explicitly names all three concerns. Prepare to compare possible mitigations using quality metrics alongside latency and cost.
Evaluating across diverse data
Because the role covers structured and unstructured data pipelines, a plausible difficulty is building test sets that reveal different retrieval and integration failures. Prepare to explain how you would select representative cases, inspect failures, and avoid relying on one aggregate score.
Choosing when to automate
The posting emphasizes privacy, security, human review, and appropriate automation in financial services. A likely trade-off is workflow efficiency versus the risk of an incorrect or sensitive output. Prepare to describe decision points for guardrails, escalation, and human review without assuming the company has a particular policy.

Practice your answers

Technical practice

How would you design and test a Python web service that exposes an LLM-driven feature?

What to coverThe posting calls for production LLM features, web services, and maintainable, tested Python. Suggested answer focus: explain service boundaries, reusable components, tests for expected and failure cases, and how you would check reliability before release.

How would you build a pipeline that handles both structured and unstructured data for an LLM application?

What to coverThe role includes building and maintaining pipelines for both data types. Suggested answer focus: describe ingestion, validation, transformation, and how you would detect data-quality failures.

How would you design a prompt and tool-calling workflow that produces a structured output?

What to coverThe posting names prompting, structured outputs, tool calling, and multi-step workflows. Suggested answer focus: cover output validation, tool-use boundaries, and recovery when a tool fails or returns unexpected data.

How would you evaluate and improve a retrieval pipeline?

What to coverThe role includes retrieval pipelines and iteration on prompts or retrieval after reviewing failures. Suggested answer focus: distinguish retrieval failures from generation failures, then explain what evidence would justify changing each component.

How would you build a test set and choose success metrics for an LLM feature?

What to coverThe posting calls for evaluation suites, metrics, failure analysis, and grounding in statistics and experimental design. Suggested answer focus: explain representative test cases, a baseline, measurable outcomes, and how you would avoid drawing conclusions from too few examples.

How would you diagnose and mitigate hallucinations in an LLM application?

What to coverDiagnosing hallucination is an explicit responsibility. Suggested answer focus: explain how you would identify unsupported claims, trace possible causes in prompts or retrieval, and measure whether a mitigation improves results.

How would you investigate an LLM service that meets quality targets but has unacceptable latency or cost?

What to coverThe posting explicitly names latency and cost alongside performance. Suggested answer focus: identify where time and expense arise, propose changes, and describe how you would check their effect on answer quality.

What would you log, monitor, and alert on for an operational LLM service?

What to coverThe role includes monitoring, logging, and alerting for AI services. Suggested answer focus: cover service health and application-quality signals, alert thresholds, and how to make failures diagnosable without exposing sensitive data.

Where would you place guardrails and human review in a financial-services AI workflow?

What to coverThe posting emphasizes privacy, security, responsible-AI guardrails, human review, and appropriate automation. Suggested answer focus: explain how you would identify higher-risk outputs, limit exposure of sensitive information, and define when a person should review a result.

How would you prepare an LLM feature for deployment and subsequent releases?

What to coverThe posting asks for familiarity with cloud deployment, containers, and modern release pipelines. Suggested answer focus: discuss reproducible packaging, automated checks, deployment verification, and a way to detect and respond to regressions.

Behavioural practice

Tell me about a time you aligned technical work with product, engineering, and business needs.

What to coverThe role calls for collaboration with all three groups. Suggested preparation: use a real situation, your actions to clarify differing priorities, and the resulting decision or outcome.

Describe a time you diagnosed a failure in production or production-like software.

What to coverThe posting values production-like contributions, failure analysis, and reliability. Suggested preparation: explain the situation, the evidence and actions you used to isolate the cause, and the result, including what remained unresolved if applicable.

Tell me about a time evaluation results changed your technical approach.

What to coverThe role includes defining metrics, reviewing failures, and iterating on prompts or retrieval. Suggested preparation: give the situation, how you assessed the evidence and changed course, and the measured result.

Describe a time you raised a privacy, security, or automation concern.

What to coverThe posting emphasizes responsible AI and appropriate human review in financial services. Suggested preparation: describe the situation, how you communicated the concern and proposed a response, and what resulted; a relevant example need not have involved an LLM.

Tell me about a time you documented a design or experiment so others could use its findings.

What to coverDocumenting designs, experiments, and findings for internal knowledge sharing is a stated responsibility. Suggested preparation: cover the situation, what you recorded and how you shared it, and the resulting effect on others' work.

Sharpen your knowledge

Topics to prepare

LLM failure modes and trade-offs
Posting: The role calls for diagnosing hallucination, latency, and cost issues. Suggested revision exercise: For a hypothetical LLM feature, list likely causes of each issue, a metric to detect it, and one mitigation with its trade-off.
Evaluation and experimental design
Posting: The role requires success metrics, test sets, failure analysis, and grounding in statistics and experimental design. Suggested revision exercise: Design a comparison of two prompt versions, specifying test-set selection, success metrics, and how you would interpret uncertain results.
Retrieval and data grounding
Posting: The role includes retrieval pipelines and both structured and unstructured data. Suggested revision exercise: Diagram how a question retrieves evidence from each data type, then identify where missing or irrelevant evidence could lead to a bad answer.
Responsible AI in financial services
Posting: Privacy, security, human review, appropriate automation, and responsible-AI guardrails are named. Suggested revision exercise: Take a hypothetical sensitive request and define what the system should answer, refuse, or route for human review.
Production service reliability
Posting: LLM features must run in production with monitoring, logging, and alerting. Suggested revision exercise: Sketch the signals and alert conditions that would reveal rising errors, latency, or failed tool calls.

Skills to revise

Testable Python implementation
Posting: Strong Python, reusable libraries, and clean, tested code are required. Suggested revision exercise: Implement a small request-processing module with input validation and tests for normal, malformed, and failed-service responses.
Prompt and tool-call workflow construction
Posting: Hands-on prompting, structured outputs, tool calling, and multi-step workflows are specified. Suggested revision exercise: Build a small workflow that requests a structured result, invokes a mock tool when needed, and handles invalid output or tool failure.
LLM evaluation and iteration
Posting: The role builds evaluation suites, reviews failures, and improves prompts or retrieval. Suggested revision exercise: Assemble a small test set, score two workflow versions against a stated metric, categorize failures, and document one targeted revision.
Structured and unstructured data integration
Posting: The role builds pipelines across both data types and integrates with web services. Suggested revision exercise: Create a small pipeline that validates records and text documents from a mock service, then records rejected inputs for inspection.
Logging, monitoring, and alerting
Posting: AI-service logging, basic monitoring, and alerting are explicit. Suggested revision exercise: Add request-level logs and counters to the mock service, and specify alerts for error rate and latency without logging sensitive input content.
Deployment and release basics
Posting: Familiarity with cloud deployment, containers, and modern release pipelines is requested. Suggested revision exercise: Explain how you would package the mock service in a container, run its tests before release, and verify service health after deployment; no particular platform is specified.

Your preparation checklist

Prepare a production-style LLM project walkthrough
The role calls for production or production-like software. Prepare a concise walkthrough of a project you can discuss, covering the problem, your contribution, architecture, tested Python code, data integrations, and what you would improve.
Practice a retrieval and tool-calling design exercise
Sketch an LLM feature that handles structured and unstructured data. Show the pipeline, prompt and structured-output design, tool boundaries, and tests; treat the scenario as interview practice, not an assumed company project.
Build an evaluation plan
The posting specifies test sets, success metrics, failure review, and iteration. Prepare a sample evaluation table with representative cases, measurable criteria, and a plan for improving prompts or retrieval after failures.
Explain performance and reliability tradeoffs
Prepare a short example of how you would diagnose hallucinations, latency, and cost, then describe what you would log, monitor, and alert on for an AI service.
Show your experimental reasoning
Revisit an ML or LLM experiment you can discuss. Be ready to explain the baseline, metric choice, statistical or experimental-design considerations, results, and limits of the conclusion.
Draft a responsible-AI review
For the practice design, identify privacy and security risks, where human review belongs, and which actions should not be automated without safeguards. These are concerns named in the posting, not a claim about the employer’s specific controls.
Prepare collaboration examples and employer questions
Outline one example of communicating technical findings to product, engineering, or business partners. Ask how this team defines LLM success, reviews failures, handles human-in-the-loop decisions, and releases and monitors features.

Availability last checked: Nov 4, 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in United States. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…