Skip to content
MiraiTech
All work

Case study · AI

Bilingual meeting notes

Built for teams that switch between Japanese and English in the same call. It transcribes with speaker labels, summarizes in both languages, pulls out decisions and action items with owners and due dates, and sends them to the tools the team already uses.

Year
2024
Duration
3 months
Type
AI
Status
In production
Bilingual meeting notes, main screen

Results

  • Transcripts in Japanese and English
  • Action items in Slack within 2 minutes
  • About 6 hours a week saved per project manager

The problem

A Tokyo product team ran every meeting in a mix of Japanese and English. Whoever was least busy took notes, action items lived in three places, and English-speaking members regularly missed decisions made in a Japanese side conversation. Existing meeting assistants transcribed one language well and the other badly, and their summaries read as if written for a different company.

What we built

We built a bot that joins the call and transcribes it with Whisper v3, with speaker labels and language detection per segment. Claude then extracts decisions, action items with an owner and due date, open questions and a summary in both languages, constrained by PydanticAI schemas. Results are posted to the right Slack channel and Notion page within two minutes, and every item links back to the transcript segment it came from.

Bilingual transcription
Whisper v3 detects the language of each segment, so switching between JP and EN mid-meeting works. Speaker diarization makes sure action items are attributed to the right person.
Typed extraction
Decisions, action items and questions are extracted into typed PydanticAI models. The owner must be an attendee and due dates must be real dates, so nothing vague reaches Slack.
Summaries in both languages
Each meeting gets a Japanese and an English summary. Both are generated from the same structured items instead of being translated from each other, so they say the same thing.
Links to the source
Each item links to the transcript segment and audio timestamp behind it. Arguments about who said what are settled in seconds.
Slack and Notion sync
Action items are posted to the channel the meeting was scheduled from and synced to a Notion database. Marking an item done in one place updates the other.

How it works

Transcription, then structured extraction, then delivery in two languages. Every item can be traced to its source audio.

  1. 01

    Capture

    Bot joins Google Meet or Zoom → audio streamed to self-hosted Whisper v3

  2. 02

    Transcribe

    Speaker-labeled, language-tagged segments stored in Neon Postgres + pgvector

  3. 03

    Extract

    Claude + PydanticAI produce typed decisions, actions, questions and a summary in both languages

  4. 04

    Validate

    Schema checks (owner, dates, duplicates) decide what is allowed to leave the system

  5. 05

    Deliver

    Slack post + Notion sync within 2 minutes; completion status syncs both ways

Screens

Bilingual meeting notes, screen 1
1 / 2

Decisions

Whisper v3AI / ML
It had the best accuracy we measured on Japanese business speech, and we could host it ourselves. The team required that audio never leave their own infrastructure.
PydanticAIAI / ML
Extraction output has to be usable by code. Typed models with validation (the owner is an attendee, the due date parses) let us reject a malformed item and retry instead of posting junk to Slack.
Claude 3.5 SonnetAI / ML
Good at finding decisions in polite Japanese phrasing where the conclusion is never stated outright, which was the hardest part of the problem.
pgvectorData
Transcript segments are embedded, so users can ask "what did we decide about pricing?" across months of meetings and get answers that cite the exact segment.

What went wrong

Decisions nobody says out loud

Problem
In Japanese meetings, agreement is often signaled by silence or a soft "let's go in that direction" instead of an explicit decision. Early extraction missed about half of the real decisions because no sentence stated them.
Fix
We reframed the extraction prompt around "what would a participant believe was agreed?", added few-shot examples from the team's own meetings and added a confidence field. Soft decisions are posted as "implied: confirm?" instead of being dropped.
Result
Measured against human notes, decision recall rose from 52% to 91%, and implied decisions are marked for confirmation.

Action items without owners

Problem
About a third of extracted action items had no clear owner, and posting them to Slack created noise that nobody acted on.
Fix
We made owner a required field and validate it against the attendee list. When the model cannot determine an owner, the item goes to the meeting organizer as a single "needs owner" thread instead of to the channel.
Result
Action items completed within a week rose from 44% to 78%, and the complaints about channel noise stopped.

Team and stack

Hiroshi designed the extraction pipeline and the PydanticAI schemas, and built the meeting review UI. Kenta built the Claude extraction service and the Slack and Notion integrations. Hiroki built transcript storage and the pgvector search layer.

  • Whisper v3
  • Claude 3.5 Sonnet
  • PydanticAI
  • Next.js 15
  • FastAPI
  • pgvector
  • Neon Postgres
  • Slack API