Athena
Written on April 1st , 2023 by Frank Hsiung
Inspiration
Athena is inspired by existing digital assistants like Alexa, which serve much like companions to older patients, and the need for increased communication between patients in geriatric care and their service providers. What if patients could have a fun conversation to share their healthcare needs and concerns, on their own schedule? Look no further than Athena. It’s all too common that a patient may decide to stop taking medication for one reason or another, or concerns arise that patients may not desire to ask professionals about. Athena acts as a way to answer these queries and deliver updates to health care providers
What it does
Your 24-7 Medical Consultant
Athena provides a suite of conversational AI assistants for patients utilizing home health services, allowing service providers to stay up to date with their patients’ needs and well-being. Whenever you don’t feel right, Athena will provide you with the most alleviating and considerate response to your health!
Expression-inducing customization
Research shows that the better patients feel during communication, the more efficient the healthcare invention will be. So we further boost our user experience by providing different customized selections of personalities and tones of the person you’re talking to and also tailor the tone based on your personal demographics. Our text-to-voice features make users feel even more at ease sharing their symptoms.
Personalized Tracking Report
Last but not least, we personalized the query history and build a personalized analysis report for every user. We track the concerns and worries of users and do emotion detection and extract top words used for their discomfort. We then convert that into a readable, diagnosis-friendly chart for further usage from experts. That way, healthcare staff no longer needs to spend time updating patients about their recent problems.
We have a simple prebuilt UI for patients, but we also provide detailed API documentation, allowing healthcare providers or other vendors to build their own solutions around Athena or integrate data from existing products into this system.
How we built it
We used an API written in Python with FastAPI, and Firebase for authentication and data persistence. We use OpenAI’s API for gpt-3.5-turbo, the same large language model behind ChatGPT. With some carefully crafted prompts, we are able to provide potential solutions for low severity health issues, while referring users to medical providers for more serious needs.
For a demo of integrating with other data providers, we’ve integrated with the Fitbit API, so we have detailed information on heart rate data and calories burned
For the frontend, we used Next.js with Chart.js to display data in an interactive way for the service provider dashboard.
Challenges we ran into
- Originally, we had planned on using a MySQL database running in Google Cloud for storing user data. After running into issues with our deployment, we made the decision to switch over to Firestore.
- The credit from OpenAI only applying to one project and restrictived
- Getting the audio recording to transfer from react-media-recorder to the whisper
Accomplishments that we’re proud of
-
Speech to text using Whisper and Text to speech using Azure Text-to-speech
- Speech to text with OpenAI Whisper
- We found that OpenAI’s Whisper model for speech to text was more accurate than existing solutions. In combination with TTS, patients can have a fully verbal conversation with Athena.
- prompt engineering
- data analysis using Text Rank
- We implement a well-known PageRank algorithm to extract keywords from users’ queries and make the report informative and clean.
- Implementing Roberta for sentiment report
What we learned
- Learned how to integrating between services, from Python and Flask backend to Next.js front end
- learned more about challenges associated w home health/geriatric care thanks to axxess
- making api docs for easy future development
What’s next for Athena
- Response streaming
- We want to stream responses from the LLM to the client, as this would enable a much faster, real time chatting experience for patients.
- Automatic alerting
- Right now, the product will refer patients to healthcare providers for any issues that should require human attention. However, there are some queries that are very serious and require immediate response from family members and/or healthcare professionals. Using tools like Twilio, we can automatically dispatch SMS messages, phone calls, and/or emails to designated emergency contacts when we detect messages that exceed predetermined risk factor.
- Finetuning
- We want to fine tune the model using definitions of conditions and symptoms that are known to be accurate, from well-renowned sources such as the Mayo Clinic, American Heart Association, and the Health in Aging Foundation. While we haven’t ran into any issues in our testing, finetuning would reduce the risk of incorrect suggestions being returned to a patient, which could have potentially devastating impacts on a patient.
