New opportunity

Data Scientist

Middesk · San Francisco; New York, United States

About this role

Middesk seeks a hands-on Data Scientist to build data-driven fraud and risk systems for its business identity and onboarding products. The role focuses on applying practical machine learning and data techniques to messy, real-world problems rather than inventing new algorithms. The systems support KYB, trust and safety, and compliance workflows.

Responsibilities

Design and ship production systems that detect and prevent fraud. Address extreme class imbalance, sparse signals, evolving adversarial behavior, and limited ground truth.
Use graph-based approaches and entity resolution to uncover connections and improve risk detection.
Generate better features and labels using heuristics, weak supervision, and AI tools, including LLMs where appropriate.
Partner with engineering to develop systems for feature generation, model training, and production deployment across multiple use cases.

Requirements

5+ years of experience in fraud, risk, or trust and safety, including work on real-world fraud or abuse problems.
Experience deploying models or data-driven systems for external-facing products, and a strong foundation in applied machine learning or data systems.
Ability to handle classification problems involving imbalanced data, sparse signals, and changing patterns, while balancing speed, accuracy, and maintainability.

Preferred

Familiarity with knowledge graphs, network analysis, or entity linking is strongly preferred.

The role is hybrid, with an expectation of two days per week in Middesk’s San Francisco or New York office. Candidates should live within commutable distance. The stated salary range is $175,000–$210,000 per year, with equity and benefits; actual compensation depends on factors including experience, qualifications, skills, and location.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

Fraud detectionRisk assessmentTrust and safetyKnow Your Business (KYB)Applied machine learningClassificationGraph analysisKnowledge graphsNetwork analysisEntity resolutionEntity linkingWeak supervisionFeature engineeringData labelingModel trainingModel deploymentLarge language models (LLMs)

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Ship fraud and risk systems
Source: The role designs and ships production systems to detect and prevent fraud across KYB, trust and safety, and compliance workflows. Preparation: Be ready to explain how a data-driven system moved from a risk signal to an external-facing product.
Handle difficult fraud data
Source: The posting emphasizes extreme class imbalance, sparse signals, evolving adversarial behavior, and limited ground truth. Preparation: Explain how you would evaluate a classifier and revisit its performance as patterns change.
Use relationships to improve detection
Source: Graph-based approaches and entity resolution are duties; knowledge graphs, network analysis, or entity linking are strongly preferred. Preparation: Work through how linking business entities might reveal useful connections, including how to assess mistaken links.
Improve features and labels pragmatically
Source: The role uses heuristics, weak supervision, and AI tools, including LLMs where appropriate, to generate features and labels. Preparation: Be ready to distinguish useful added signal from noisy or misleading labels.
Build repeatable production infrastructure
Source: The role partners with engineering on feature generation, model training, and deployment across multiple use cases, while balancing speed, accuracy, and maintainability. Preparation: Explain the design choices that make a system reliable enough to ship and practical to evolve.

Likely challenges to think through

Finding fraud without overwhelming legitimate businesses
Preparation hypothesis, not a confirmed Middesk problem: Rare fraud and sparse signals can make more aggressive detection increase false positives in onboarding. Prepare to discuss threshold and evaluation choices in terms of both detected fraud and impact on legitimate businesses.
Trusting connections between messy records
Preparation hypothesis, not a confirmed Middesk problem: Entity resolution could expose hidden networks, but incorrect matches could spread misleading risk signals. Prepare to explain how you would validate links and decide when relational evidence is strong enough to use.
Keeping a shipped system useful as behavior changes
Preparation hypothesis, not a confirmed Middesk problem: Adversarial behavior and limited ground truth can make initial labels and model results less dependable over time. Prepare to discuss how you would detect changing performance and update features or labels without sacrificing maintainability.

Practice your answers

Technical practice

How would you design a production fraud-detection system for business onboarding?

What to coverThe posting calls for systems that detect and prevent fraud across KYB, trust and safety, and compliance workflows. Preparation suggestion: explain how you would define the decision, establish a practical baseline, integrate available signals, and evaluate the effects on fraud prevention and legitimate onboarding.

How would you evaluate a classifier when fraud is extremely rare?

What to coverThe role explicitly involves extreme class imbalance. Preparation suggestion: discuss precision and recall at useful decision thresholds, the costs of missed fraud and false positives, and why overall accuracy could be misleading.

What would you do when onboarding data is sparse or unreliable?

What to coverThe posting highlights sparse signals and messy real-world data. Preparation suggestion: describe how you would check data quality, distinguish missing information from negative evidence, and choose a robust fallback when confidence is low.

How would you develop usable fraud labels with limited ground truth?

What to coverThe role calls for better labels using heuristics and weak supervision despite limited ground truth. Preparation suggestion: explain how you would combine imperfect signals, audit a sample of labels, and guard against leakage or systematic labeling errors.

When would an LLM help with feature or label generation?

What to coverThe posting mentions AI tools, including LLMs where appropriate, for improving features and labels. Preparation suggestion: identify a bounded use case and explain how you would test output quality, handle uncertainty, and compare it with a simpler heuristic.

How would you resolve business entities across inconsistent records?

What to coverEntity resolution is an explicit part of the work. Preparation suggestion: discuss candidate matching, evidence for and against a match, and how false merges versus missed links could affect risk decisions.

How could a graph reveal risk that individual business records miss?

What to coverThe posting calls for graph-based approaches to uncover hidden connections, with network analysis or knowledge graphs strongly preferred. Preparation suggestion: explain what entities and relationships you would represent, how you would derive a useful signal, and how you would check for spurious connections.

How would you respond when fraud patterns change after deployment?

What to coverEvolving adversarial behavior and changing classification patterns are stated challenges. Preparation suggestion: describe what you would monitor, how you would investigate a shift, and how you would update a system without overreacting to noisy evidence.

How would you structure feature generation and model deployment across use cases?

What to coverThe role involves partnering with engineering on feature generation, training, and production deployment across multiple use cases. Preparation suggestion: address consistency between training and production data, reusable components, and safeguards for reliable releases.

How would you choose between a simple heuristic and a more complex model?

What to coverThe posting emphasizes practical techniques and balancing speed, accuracy, and maintainability. Preparation suggestion: compare the options using available evidence, operational cost, and their effects on fraud detection and onboarding rather than assuming complexity is better.

Behavioural practice

Tell me about a time you partnered with engineering to ship a data-driven system.

What to coverThe role explicitly involves engineering collaboration and production deployment. Preparation suggestion: use situation, action, and result to show how you divided responsibilities, resolved an implementation constraint, and established whether the shipped system worked.

Describe a time messy data changed your approach to a risk problem.

What to coverThe posting stresses unreliable real-world data and limited ground truth. Preparation suggestion: use situation, action, and result to explain what you discovered, how you adapted the analysis or system, and what you could verify afterward.

Give an example of responding to changing fraud or abuse behavior.

What to coverUnderstanding bad actors and evolving adversarial behavior is central to the role. Preparation suggestion: use situation, action, and result to describe the observed change, your investigation, the response, and any remaining uncertainty.

Tell me about a time you made a pragmatic speed-versus-quality trade-off.

What to coverThe posting explicitly values impact while balancing speed, accuracy, and maintainability. Preparation suggestion: use situation, action, and result to explain the alternatives, why you chose one, and how you checked its consequences.

Describe a time you influenced the design of a system beyond your own implementation.

What to coverThe role is described as hands-on, with broad influence over data-driven system design and scale. Preparation suggestion: use situation, action, and result to show how you made a technical case, incorporated others’ input, and measured the outcome without overstating your ownership.

Sharpen your knowledge

Topics to prepare

Fraud risk in business onboarding
The posting centers on fraud prevention across KYB, trust and safety, and compliance workflows. As preparation, sketch a hypothetical onboarding case and explain how a suspicious business connection could affect a risk decision.
Classification with scarce evidence
The role explicitly involves extreme class imbalance, sparse signals, and limited ground truth. Revise how these conditions affect evaluation and decisions by explaining which errors you would examine in a hypothetical rare-fraud case.
Changing adversarial behavior
The posting calls out evolving adversarial behavior and changing patterns. As preparation, describe how a once-useful fraud signal could become unreliable and what evidence would prompt you to revisit it.
Relational risk signals
Graph-based approaches and entity resolution are part of the work; familiarity with knowledge graphs, network analysis, or entity linking is strongly preferred. Revise by explaining when a shared business attribute indicates a meaningful connection rather than a coincidental match.
Labels and pragmatic production tradeoffs
The role calls for heuristics, weak supervision, and AI tools, including LLMs where appropriate, while balancing speed, accuracy, and maintainability. Compare two hypothetical labeling approaches and explain how uncertain labels would affect a production decision.

Skills to revise

Evaluate a rare-fraud classifier
For a hypothetical imbalanced dataset, draft a review showing false positives, missed fraud, and performance across sparse-signal cases. This practices the applied classification judgment the posting requires; the exercise is a suggestion, not a stated interview task.
Build an entity-resolution example
Create a small hypothetical set of business records, propose matching rules, and trace which linked entities warrant investigation versus review for false matches. This practices the graph and entity-resolution work described in the posting.
Prototype feature and label rules
Write a few illustrative fraud feature definitions and heuristic labeling rules, then flag ambiguous cases for review. This rehearses the posting’s feature-generation and weak-supervision responsibilities without assuming a particular tool or data source.
Outline a production handoff
Draw a concise flow from feature generation through model training to deployment, noting where engineering collaboration, data failures, and maintainability decisions arise. The posting explicitly calls for shipping external-facing systems and scaling this infrastructure; the diagram is a preparation suggestion.

Your preparation checklist

Select a fraud or risk project
The role calls for real-world fraud, risk, or trust-and-safety experience. If available, prepare a one-page account of a relevant problem, your decisions, and measurable outcomes.
Build a production-system walkthrough
The posting emphasizes systems shipped for external-facing products. Prepare a diagram showing data inputs, feature generation, training, deployment, engineering collaboration, and what you would change.
Rehearse an imbalanced-classification case
Work through a hypothetical KYB risk decision with sparse signals and limited ground truth. Explain validation, threshold trade-offs, and how you would detect changing fraud patterns; present these as proposed approaches, not Middesk practices.
Sketch a relationship-based detection approach
The role favors graph and entity-resolution techniques. Draw a small example linking businesses and related entities, then explain match uncertainty, useful network signals, and potential false connections.
Prepare a labeling and feature plan
Outline when you would use heuristics, weak supervision, or an LLM to propose labels or features, and how you would check their quality before production use.
Formulate questions about the role
Ask which onboarding or KYB risk decisions are the initial priority, where ground truth is hardest to obtain, and how data science and engineering divide responsibility for production systems.

Availability last checked: Nov 4, 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in United States. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…