🎯 Project Overview
Patient Voice is an AI-powered voice chatbot built for the Heidi Health Hackathon 2025 to transform patient triage and medical data collection.
Problem Statement
Clinicians spend excessive time on initial patient consultations and routing. Patients often don't know which specialist they need. Manual data collection is time-consuming and error-prone.
Solution
- Conducts natural voice conversations with patients
- Intelligently routes to appropriate consultation types
- Collects comprehensive patient data automatically
- Checks for drug contraindications
- Exports structured data to Google Sheets
- Suggests appropriate GP clinical templates
85%
Classification Accuracy
🤖 AI-First Classification System
Patient Voice uses a hybrid ensemble classifier that combines keyword matching with advanced AI semantic analysis to automatically suggest the most appropriate GP clinical template for each consultation.
Classification Method
Ensemble Weighting:
- 80% AI Classification (Primary Decision Maker)
- Uses GPT-4o for semantic understanding
- Analyzes conversation context and flow
- Understands medical terminology in context
- Differentiates similar conditions (e.g., "6-week baby check" vs "general well-child visit")
- 20% Keyword Matching (Supporting Role)
- Rule-based matching on medical keywords
- Position-weighted (keywords in first 20% get 1.5x boost)
- Frequency-based scoring
- Provides baseline signal for AI
How It Works
1. Patient conversation transcript → Template Classifier
2. Keyword Matcher analyzes medical terms → Keyword Scores (0.0-1.0)
3. AI Classifier uses GPT-4o semantic analysis → AI Scores (0.0-1.0)
4. Ensemble Scorer combines: Final = (0.2 × Keywords) + (0.8 × AI)
5. Confidence Verifier checks score:
- HIGH (≥75%): Auto-select, no review needed
- MEDIUM (50-74%): Suggest with indicator
- LOW (30-49%): Flag for manual review
- VERY_LOW (<30%): Require manual categorization
6. Return top 3 predictions with reasoning
Fuzzy Template Matching
The AI classifier includes intelligent fuzzy matching to handle natural language variations:
- AI says "Vaccination Record" → Maps to "Vaccine Admin Note" ✓
- AI says "Specialist Referral" → Maps to "GP Letter with Summary" ✓
- AI says "Well Child Check" → Maps to "6 week infant check" ✓
📋 Complete GP Template List
The classifier supports 22 specialized GP clinical templates covering a comprehensive range of primary care consultations:
Mother of all GP Templates
Problem-oriented consultation with full SOAP structure
Patient Explainer Letter
Simple language summary for patients
6 week infant check
Postnatal baby developmental assessment
Consult/Visit Summary for the patient
Brief consultation summary
Contraception counselling
Family planning consultations
Dysmennorhoea/Menorrhagia
Period pain and heavy bleeding
GP Chronic Condition Management Plan
Long-term condition care plans
GP Chronic Disease Management Review
Ongoing disease review appointments
GP Letter ADHD
ADHD assessment and referral letters
GP Letter with Summary
Specialist/imaging referral letters
History and Physical Examination Note GP
Complete H&P documentation
Iron Infusion Consent
Procedure consent documentation
Mental Health Note
Psychiatric and counseling sessions
Mirena insertion consent
IUD procedure consent
Multi Issue Consultation
Multiple concerns in one visit
Psychiatry Follow Up
Mental health follow-up appointments
SMART Goals note GP Management Plan
Goal-setting care plans
SOAP Note for the GP
Internal GP documentation (no external follow-up)
Skin Check Note
Dermatological assessments
Vaccine Admin Note
Vaccination and immunization records
Well Child Check
General pediatric wellness visits
Well Child Visit
Child health assessments
⚙️ Technology Stack
AI & Machine Learning
OpenAI GPT-4o
Whisper STT
TTS API
Scikit-learn
Pandas
NumPy
Backend
Python 3.11+
FastAPI
Uvicorn
Pydantic
SQLite
Frontend
HTML5
CSS3
Vanilla JavaScript
WebRTC
Web Audio API
Deployment
Railway (Backend)
Cloudflare Pages (Frontend)
Docker
GitHub Actions
Integrations
Google Sheets API
Google Service Accounts
🌟 Key Features
For Patients
- Natural Voice Interface - Speak naturally, no forms to fill
- Multi-Language Support - Whisper handles 50+ languages
- Empathetic Conversations - AI trained on medical communication
- Fast Triage - Get routed to the right specialist quickly
- Privacy-Focused - Secure data handling with consent tracking
For Clinicians
- Structured Data Export - Google Sheets with all patient information
- Smart Template Routing - AI recommends appropriate GP templates (85% accuracy)
- Drug Interaction Alerts - Automatic contraindication screening
- Data Analytics Ready - Clean, structured data for analysis
- Multiple Conversation Flows - Specialized paths for different health focuses
- Live Transcription - Real-time speech-to-text for consultations
- Doctor Style Templates - 4 note styles (Brief, Goldilocks, Detailed, Super Detailed)
🚀 How to Use Patient Voice
Main Conversation Flow
- Click "Start Conversation" on the home page
- Allow microphone permissions when prompted
- Speak naturally about your health concern
- AI will guide you through relevant questions
- Receive treatment recommendations and next steps
- Export consultation data to Google Sheets
Template Classifier
- Navigate to "📋 Template Classifier"
- Paste a medical consultation transcript
- Click "Classify Template"
- View AI's top 3 template suggestions with confidence scores
- Review reasoning and alternative templates
Live Transcription
- Navigate to "🎙️ Live Transcription"
- Click "Start Recording"
- Speak or conduct consultation
- View real-time transcription
- Stop recording and export transcript
Doctor Notes
- Navigate to "📝 Doctor Notes"
- Select conversation from database
- Choose doctor style (Brief/Goldilocks/Detailed/Super Detailed)
- View generated clinical note
- Compare styles side-by-side
- Copy or export final note
🏗️ Deployment Architecture
Production Setup
- Frontend: Cloudflare Pages (
https://healthconnect.vip)
- Backend: Railway (
https://paitientvoice-production.up.railway.app)
- Database: SQLite (embedded in backend)
- Data Export: Google Sheets via Service Account
Environment Variables (Railway)
OPENAI_API_KEY=sk-proj-... # OpenAI API key
OPENAI_MODEL=gpt-4o # GPT model
GOOGLE_SHEETS_SPREADSHEET_ID=1GaqBi... # Google Sheets ID
ALLOWED_ORIGINS=https://healthconnect.vip # CORS origins
KEYWORD_WEIGHT=0.20 # Classifier keyword weight
AI_WEIGHT=0.80 # Classifier AI weight
GitHub Repository
Repository: fivelidz/paitientvoice
Main Branch: Deployed to production automatically
CI/CD: Railway auto-deploys on push to main
🏆 Heidi Health Hackathon 2025
Patient Voice was built as a submission for the Heidi Health Hackathon 2025, focusing on innovative AI solutions for healthcare efficiency and patient experience.
Hackathon Goals Addressed
- Reduce Clinician Workload - Automated triage and data collection saves time
- Improve Patient Experience - Natural voice interface eliminates form fatigue
- Enhance Data Quality - Structured AI-guided conversations ensure completeness
- Intelligent Routing - AI determines appropriate specialist or template
- Safety First - Drug contraindication checks built-in
Innovation Highlights
- AI-First Classification - 85% accurate template suggestion with 80/20 ensemble
- Voice-Native Design - Complete conversation flow via voice
- Semantic Understanding - GPT-4o contextual analysis beyond keywords
- Flexible Documentation - 4 doctor note styles for personalization
- Production-Ready - Fully deployed and accessible at healthconnect.vip