Skip to content
MiraiTech
All work

Case study · AI

Bilingual support agent

The agent replies in whichever language the customer writes in. It answers from the retailer's policy and product documents and cites them. When it cannot resolve a case, it hands over to a person with a structured summary instead of a raw transcript.

Year
2025
Duration
4 months
Type
AI
Status
In production
Bilingual support agent, main screen

Results

  • 71% of tier-1 tickets resolved without staff
  • 1.9 s median time to first token
  • Handoffs in either language with nothing retyped

The problem

A Japanese retailer with a growing overseas customer base had one small team answering LINE and web chat in two languages. Tier-1 questions (order status, returns, size guides, store hours) made up about 70% of volume, and every one still needed a person. English tickets waited hours longer than Japanese ones. Earlier chatbots had failed because they answered confidently from stale FAQ pages and lost the context when a ticket was handed to staff.

What we built

We built an agent that detects the language of each message, answers from a versioned knowledge base of policies and product data with citations, and takes real actions such as order lookups and returns through typed tools. When confidence is low or the customer asks for a person, it hands off to Zendesk with the language, the intent, what has already been checked and a suggested reply. Staff never have to ask the customer to repeat themselves.

Language detection per message
Each message is classified as JP or EN on its own, so customers who switch languages get a reply in the one they just used. Tone presets cover Japanese keigo and casual English.
Answers with citations
Hybrid pgvector and BM25 retrieval over policy documents and the product catalog. Every answer links to its source paragraph, and a nightly freshness check flags stale documents.
Typed action tools
Order status, returns and store hours run as Pydantic-validated tools against the retailer's APIs. Idempotency keys make sure a retried message never creates a second return.
Handoff to staff
Escalations arrive in Zendesk with the language, intent, verified facts and a draft reply. Staff accept, edit or reject the draft, and rejections go into the eval set.
Agent console
A Next.js console shows live conversations, retrieval traces and confidence scores. The support lead can pause the agent on any topic with one click.

How it works

One support agent for every channel. It picks the language per message, cites its sources, acts through typed tools and escalates with a structured handoff.

  1. 01

    Channel adapter

    LINE, web chat and email messages normalized into one conversation model

  2. 02

    Language and intent

    Claude classifies language and intent per message; conversation state stored in Redis

  3. 03

    Retrieve or act

    Hybrid pgvector + BM25 retrieval with citations, or typed tool calls to the order and return APIs

  4. 04

    Confidence gate

    Low confidence or a request for a person → Zendesk handoff with a draft reply

  5. 05

    Reply and learn

    Reply streamed to the channel; staff edits and rejections flow back into the eval set

Screens

Bilingual support agent, screen 1
1 / 2

Decisions

LangGraphAI / ML
The support flow has explicit states (classify, retrieve, act, confirm, escalate), and the agent must never start a return without a confirmation step. With an explicit graph we could prove that. With a free-form ReAct loop we could not.
Claude Sonnet 4AI / ML
Of the models we tested on 400 real tickets, it wrote the best Japanese and kept keigo the most consistent. It was also the most reliable at declining to answer when retrieval found nothing relevant.
pgvectorData
The knowledge base is under 50k chunks and already sat next to the order data in Postgres. Keeping retrieval in the same database made citations, versioning and the nightly freshness checks simpler.
LINE Messaging APIBackend
LINE is the main support channel for Japanese customers. The channel adapter lets one agent graph serve LINE, web chat and email with no channel-specific prompts.

What went wrong

Customers who switch languages

Problem
Overseas customers living in Japan often mixed languages in one conversation: a Japanese product name inside an English sentence, or a switch to Japanese for an address. Detecting the language once per conversation produced a reply in the wrong language about 18% of the time.
Fix
We moved detection to each message, with a short rolling context, and made the reply follow the language of the customer's latest message. We also built an eval set of 120 mixed-language exchanges from real tickets.
Result
Wrong-language replies fell from 18% to under 2%. Satisfaction on English tickets matched Japanese tickets for the first time.

Citing outdated policies

Problem
The retailer updated its return policy every season, but old PDF versions stayed in the document store. The agent quoted them accurately and still gave wrong answers, because the cited document had been superseded.
Fix
We added versioning with effective dates to the ingestion pipeline. A nightly job flags chunks whose source has a newer version, and retrieval only returns policy that is currently in effect. Superseded chunks are kept for audit but never cited.
Result
Policy-related escalations fell 64% the following quarter. The support lead can see which policy version answered any ticket.

Team and stack

Hiroshi led the architecture, designed the LangGraph conversation graph and built the web chat UI and agent console. Kenta built the retrieval and escalation services and the LINE adapter. Hiroki built knowledge-base ingestion and versioning.

  • LangGraph
  • Claude Sonnet 4
  • Next.js 15
  • FastAPI
  • pgvector
  • LINE Messaging API
  • Redis
  • Zendesk API