FairTeach

Learn how to become a kinder, less discriminatory and more attentive instructor with FairTeach, a social coaching platform powered by Hume AI.

Comment

Inspiration

Facing implicit bias during office hours can significantly impact student experiences and outcomes. As members of various marginalized communities, we have each personally felt the negative effects of these biases and know we are not alone in our struggles. However, identifying our own implicit biases, which often arise from deeply ingrained systemic prejudices, can be challenging. Receiving feedback on how these biases may influence our behaviors, especially in educational settings, is crucial for fostering a respectful and conducive learning environment. Ensuring that all students feel supported, regardless of their identity, is essential for creating a space where learning can blossom.

What it does

Goal:

  • Our product FairTeach provides feedback on TA performances during online office hours with a focus on analyzing implicit biases and potential prejudices.

Overview:

  • Students provide consent for video recordings and fill out an info form on their external identity characteristics (gender, race, age) before the meeting.
  • FairTeach takes in the recording of the office hours meeting.
  • The TA’s speech is isolated from the recording’s audio.
  • The TA's isolated speech is inputted into Hume for analysis, which is then converted into an individual report displaying key statistics on the TA's reactions and behavior patterns during the meeting.
  • Individual reports are consolidated into a general report that highlights trends and changes in emotions and responses to various communities over time.
  • Various general reports of each team member are compiled into a comprehensive “company” report to offer insights for creating specialized educational training and workshops (these statistics are kept confidential, so no individual team member is singled out)

How we built it

  • Voice isolation script:
    • We utilized a trained speaker diarization model from pyannote and ran our model on the specified audio file from the mp3 file recording of our online meeting.
    • Then we extracted the audio of the specified TA speaker using pydub and exported it as a new audio file to run Hume analysis on.
  • Hume analysis:
    • We used Hume’s Prosody model to analyze .wav audio files and produce emotion scores based on tone, inflection, and content.
      • We also created an API key and sent a request to the expression measurement REST API.
    • We next flagged specific emotions that Hume generated that were most detrimental to establishing a productive learning environment (anger, contempt, doubt, etc.).
    • Finally, we extracted the most prevalent emotions from the analysis and published this data, along with new statistics, by identifying and highlighting the highly present flagged emotions on our website.
  • Website:
    • We first created a multi-layer outline of our website using Figma.
    • We then used Javascript and HTML to code our website based on the baseline templates where we visualized statistics that were collected from the API.

Challenges we ran into

  • We needed to find a way to distinguish different voices (student or TA).
    • We also had to also find a way to automate the process of isolating the audio file
    • Solution: We developed a diarization Python script using HuggingFace & PyTorch
  • The audio file had to be a certain file size of around 200kb for Hume’s (Empathic Voice Interface (EVI) endpoint to run on it, but the diarization process changed the file size significantly.
    • Only one audio was effectively working on Evi, and any other isolated audios were causing the service to time out.
    • Solution: We utilized a different endpoint (Expression Measurement REST API).
  • We were not as familiar with front-end design or development, so we spent a lot of time researching and enhancing our web development skills.
  • We also had to spend significant time planning out a creative, but still user-friendly website.

Accomplishments that we're proud of

  • We developed a fully functional website powered by Node.js.
  • We also created a script utilizing training models to isolate the distinct speeches of each speaker.
  • We extracted specific results of the most importance from the overall analysis.
  • We built a clean, intuitive website with a functional signup/login feature.

What we learned

  • We gained a better comprehension of training models and running them on real data.
  • We learned how to make REST API requests and collect/reformat resultant data.
  • We also learned how to use Figma to prototype website designs.
  • We explored the customer service side of a business through our website development and identified key challenges and solutions that companies face when implementing their services.

What's next for FairTeach

  • We would like to leverage AI technology to provide recommendations for expressing dialogue in a more considerate and professional manner for future conversations.
  • We also aim to provide Hume analysis and insights on visual input, in addition to audio, to further identify biases.
  • We could also implement real-time analysis and suggestions for dialogue during meetings to ensure a safe and civil environment at all times.

Built With

Share this project:

Updates