MINDcraft

• 

  MINDcraft!!!!!!!!!

• 

• 

Inspiration

WE LOVE MINECRAFT! WE LOVE BIOTECH! LET'S COMBINE THE TWO! also mindcraft is a good name

What it does

PLAY MINECRAFT WITH YOUR MIND!

We extract real-time EEG signals using a Brain-Computer Interface (BCI) paired with motor imagery to control movement and jumping in Minecraft. (think about moving to move!) We also extract accelerometer data from an IMU for 'punching' and 'placing' blocks, and EMG data for panning the camera in minecraft!

How we built it

The MIND:

• We collect 5 mins of data where our subject thinks of either relaxing, moving forward, or jumping.
• We then train a lightweight deep learning model to classify the three states. The model includes convolutional neural networks and an attention module.
• We also apply signal processing techniques such as bandpass filtering, normalization, and common average referencing.
• We then run the deep learning model in real-time, allowing us to move forward and jump in Minecraft!

The BODY:

• We collected 100 actions for 5 unique classes, arm raise up and down, supinating and pronating the wrist to fully utilize the IMU and EMG sensors to represent other minecraft actions such as block mining, placing, and turning pov left and right.
• We trained a SVM classification for the model and achieved 98.5% accuracy on test dataset and around 75% accuracy on inference
• Calibrated for each user for better results

Challenges we ran into

• broken hardware
• noisy signals/data
• so much training
• data collection
• mapping onto Minecraft

Accomplishments that we're proud of

• it kinda works!

What's next for MINDcraft

-BIG THINGS!

Built With

• python