Skip to content
yainier.com

Bringing AI Features to a Production SaaS

Yainier Martínez Ruben

Bringing AI Features to a Production SaaS

Adding AI to a demo is easy. Adding it to a product that customers use every day to run their business is a different story. Working on AI-driven features for a CRM platform taught me that the model is usually the smallest part of the problem. Here is what matters.

Start with the workflow, not the model

The best AI features remove friction from something users already do: summarizing long email threads, extracting data from documents, suggesting the next action on a deal, drafting a reply. Before choosing any model, ask:

  • What task takes users the most time today?
  • What does a “good” result look like, and who can judge it?
  • What happens if the AI gets it wrong?

If you can’t answer the last question, you are not ready to ship.

Design for being wrong

Language models are probabilistic. Your UX and your architecture must assume errors will happen:

  • Keep a human in the loop for actions with consequences. The AI proposes, the user confirms.
  • Show sources whenever the answer is based on data (emails, records, documents) so users can verify it.
  • Validate structured output. If you ask the model for JSON, parse it against a schema and retry or fall back when it doesn’t match.
const DealSummary = z.object({
  summary: z.string(),
  nextSteps: z.array(z.string()),
  risk: z.enum(["low", "medium", "high"]),
});

const result = DealSummary.safeParse(JSON.parse(modelOutput));
if (!result.success) {
  return fallbackSummary(deal);
}

Context is everything

Most of the quality comes from what you send to the model, not from the model itself. Invest in:

  • Retrieval: fetch only the relevant records instead of dumping everything into the prompt. Search engines like ElasticSearch or vector stores are your friends here.
  • Permissions: the AI must never see data the current user isn’t allowed to see. Apply the same authorization rules you use everywhere else.
  • Clear instructions: versioned prompts, stored alongside the code and reviewed like code.

Measure before and after

“It feels better” is not a metric. Build a small evaluation set of real (anonymized) cases with expected results and run it every time you change a prompt or a model. Track in production:

  • Acceptance rate of suggestions.
  • How often users edit or discard the output.
  • Latency and cost per request.

Keep costs and latency under control

AI calls are slower and more expensive than a typical API call. Some techniques that help:

  • Cache results for identical or very similar inputs.
  • Use the smallest model that meets the quality bar for each task.
  • Stream responses so users see progress immediately.
  • Run heavy jobs asynchronously in queues instead of blocking the request.

Conclusion

Successful AI features are built with the same discipline as any other part of the product: a clear user problem, solid data access, validation, observability and iteration. The model is a powerful component, but it is still just a component.

Hire Me