AutoCPT

AutoCPT automates medical billing by extracting CPT codes from doctor-patient talks, seamless fracture detection, and generating bills via Capital One API—saving time and reducing errors.

Comment

Inspiration

Doctors spend 16 hours a week on burdensome paperwork, time that can be better spent by the patient's side delivering quality care. Additionally, almost $88 billion dollars are spent on incorrect insurance billing. Thus, we came up with AutoCPT.

AutoCPT is an assistant came from the need to streamline medical coding, and does so by identifying CPT (Current Procedural Terminology) codes from live speech. In medical settings, documentation can be time-consuming, and errors in coding can lead to financial and administrative inefficiencies. By leveraging speech recognition and AI, we aimed to create an accessible, efficient, and accurate way to generate CPT codes in real-time.

What it does

AutoCPT listens to live speech, transcribes medical conversations, and extracts the most relevant CPT codes. This helps healthcare professionals quickly document procedures and consultations with improved accuracy and efficiency.

How we built it

Tech Stack

  • Frontend: React Native (Expo), TypeScript
  • Backend: Flask, Python
  • Speech Recognition: @react-native-voice/voice
  • AI & NLP Processing: Groq API -Object Detection: Ultralytics and OpenCV
  • State Management: React Hooks (useState, useEffect)
  • Data Handling: Axios for API requests, WebSockets for real-time communication

Development Process

  1. Speech Recognition Setup: Configured @react-native-voice/voice to transcribe real-time speech into text.
  2. Flask Backend: Implemented a WebSocket-based server to process and store CPT code history.
  3. Groq API Integration: Sent transcriptions to the API for CPT code extraction and formatted responses.
  4. Fracture Detection: Trained a Yolov11 model on a fracture dataset and used OpenCV to run detection on uploaded images of patient x-rays.
  5. User Interface Design: Built an accessible UI with live visualization, buttons, and a collapsible sidebar.
  6. Error Handling & Optimization: Reduced redundancy in transcriptions, limited API responses to relevant codes, and fixed UI inconsistencies.
  7. Testing & Debugging: Simulated real-world medical conversations to refine accuracy and performance.

Challenges we ran into

1. Duplicate Word Detection in Transcriptions

  • Initially, words were repeated multiple times due to the interim results of speech recognition.
  • Solution: Implemented logic to only append finalized words to the transcript.

2. Too Many Irrelevant CPT Codes

  • The API sometimes returned an excessive number of CPT codes.
  • Solution: Adjusted the prompt to restrict responses to 3-5 highly relevant codes.

3. Environment Variable Issues

  • .env was not always properly loaded in React Native.
  • Solution: Used process.env.EXPO_PUBLIC_GROQ_API_KEY for compatibility and debugging.

4. Expo & Dependency Issues

  • Encountered dependency conflicts with Expo, Node.js versions, and Metro bundler.
  • Solution: Upgraded Node.js, cleared caches (expo start -c), and reinstalled dependencies.

5. Integration of Backend into the App

  • Initially, the backend was running separately from the mobile app.
  • Solution: Embedded the Flask server inside the app and managed requests efficiently.

Accomplishments that we're proud of

  • Successfully implemented real-time speech-to-text conversion and AI-powered CPT code extraction.
  • Optimized UI/UX for smooth user interaction and accessibility.
  • Reduced redundant transcriptions and improved API efficiency.
  • Overcame major technical challenges related to React Native, Expo, and backend integration.
  • Integrated fracture detection into the front end

What we learned

  • Speech Recognition Optimization: Managing interim and final results effectively to reduce duplication.
  • AI Prompt Engineering: Structuring API prompts to yield more accurate and relevant CPT codes.
  • React Native Debugging: Handling Expo environment variables and dependency conflicts.
  • Backend Optimization: Improving WebSocket-based communication and response handling.
  • Medical Coding Applications: Understanding how CPT codes are structured and assigned in real-world medical workflows.

What's next for AutoCPT

  • Improved Accuracy: Fine-tune the speech recognition model for better medical terminology handling.
  • Offline Mode: Implement local processing for CPT code extraction without internet dependency.
  • Customization: Allow users to input specialty-specific CPT codes for tailored results.
  • Multilingual Support: Expand support for additional languages beyond English.
  • Integration with EHR Systems: Enable seamless integration with electronic health record (EHR) platforms for automatic documentation.
Share this project:

Updates