Case study · AI
Bilingual meeting notes
Built for teams that switch between Japanese and English in the same call. It transcribes with speaker labels, summarizes in both languages, pulls out decisions and action items with owners and due dates, and sends them to the tools the team already uses.
- Year
- 2024
- Duration
- 3 months
- Type
- AI
- Status
- In production

Results
- Transcripts in Japanese and English
- Action items in Slack within 2 minutes
- About 6 hours a week saved per project manager
The problem
A Tokyo product team ran every meeting in a mix of Japanese and English. Whoever was least busy took notes, action items lived in three places, and English-speaking members regularly missed decisions made in a Japanese side conversation. Existing meeting assistants transcribed one language well and the other badly, and their summaries read as if written for a different company.
What we built
We built a bot that joins the call and transcribes it with Whisper v3, with speaker labels and language detection per segment. Claude then extracts decisions, action items with an owner and due date, open questions and a summary in both languages, constrained by PydanticAI schemas. Results are posted to the right Slack channel and Notion page within two minutes, and every item links back to the transcript segment it came from.
- Bilingual transcription
- Whisper v3 detects the language of each segment, so switching between JP and EN mid-meeting works. Speaker diarization makes sure action items are attributed to the right person.
- Typed extraction
- Decisions, action items and questions are extracted into typed PydanticAI models. The owner must be an attendee and due dates must be real dates, so nothing vague reaches Slack.
- Summaries in both languages
- Each meeting gets a Japanese and an English summary. Both are generated from the same structured items instead of being translated from each other, so they say the same thing.
- Links to the source
- Each item links to the transcript segment and audio timestamp behind it. Arguments about who said what are settled in seconds.
- Slack and Notion sync
- Action items are posted to the channel the meeting was scheduled from and synced to a Notion database. Marking an item done in one place updates the other.
How it works
Transcription, then structured extraction, then delivery in two languages. Every item can be traced to its source audio.
- 01
Capture
Bot joins Google Meet or Zoom → audio streamed to self-hosted Whisper v3
- 02
Transcribe
Speaker-labeled, language-tagged segments stored in Neon Postgres + pgvector
- 03
Extract
Claude + PydanticAI produce typed decisions, actions, questions and a summary in both languages
- 04
Validate
Schema checks (owner, dates, duplicates) decide what is allowed to leave the system
- 05
Deliver
Slack post + Notion sync within 2 minutes; completion status syncs both ways
Screens
Decisions
- Whisper v3AI / ML
- It had the best accuracy we measured on Japanese business speech, and we could host it ourselves. The team required that audio never leave their own infrastructure.
- PydanticAIAI / ML
- Extraction output has to be usable by code. Typed models with validation (the owner is an attendee, the due date parses) let us reject a malformed item and retry instead of posting junk to Slack.
- Claude 3.5 SonnetAI / ML
- Good at finding decisions in polite Japanese phrasing where the conclusion is never stated outright, which was the hardest part of the problem.
- pgvectorData
- Transcript segments are embedded, so users can ask "what did we decide about pricing?" across months of meetings and get answers that cite the exact segment.
What went wrong
Decisions nobody says out loud
- Problem
- In Japanese meetings, agreement is often signaled by silence or a soft "let's go in that direction" instead of an explicit decision. Early extraction missed about half of the real decisions because no sentence stated them.
- Fix
- We reframed the extraction prompt around "what would a participant believe was agreed?", added few-shot examples from the team's own meetings and added a confidence field. Soft decisions are posted as "implied: confirm?" instead of being dropped.
- Result
- Measured against human notes, decision recall rose from 52% to 91%, and implied decisions are marked for confirmation.
Action items without owners
- Problem
- About a third of extracted action items had no clear owner, and posting them to Slack created noise that nobody acted on.
- Fix
- We made owner a required field and validate it against the attendee list. When the model cannot determine an owner, the item goes to the meeting organizer as a single "needs owner" thread instead of to the channel.
- Result
- Action items completed within a week rose from 44% to 78%, and the complaints about channel noise stopped.
Team and stack
Hiroshi designed the extraction pipeline and the PydanticAI schemas, and built the meeting review UI. Kenta built the Claude extraction service and the Slack and Notion integrations. Hiroki built transcript storage and the pgvector search layer.
- Whisper v3
- Claude 3.5 Sonnet
- PydanticAI
- Next.js 15
- FastAPI
- pgvector
- Neon Postgres
- Slack API