LeLamp James | Devpost

Inspiration

One of the main functions of the human is to learn.

Humans' places of residence are full of household items with their own distinct functions. An example is the lamp, which allows for light to shine in our shelters.

Why not combine the light of humanity, learning, with the light of our houses?

We present LeLamp James, the lamp that teaches you math.

What it does

Full workflow:

Show the lamp's camera your math question (ideally on paper)
Tell the lamp you need "Help"
Your math problem will be calculated
The lamp will respond with your answer, providing easy math practice and assistance

How we built it

Using LeLamp's codebase, we made use of their FastAPI backend for speech and camera commands.

*Gemini: * Our reasoning layer.

Gemini...

Analyzed a real-world frame captured by the lamp
Interpreted the presence of handwritten/printed numbers
Then interpreted the math expression using its RAG capabilities
Returned a reliable answer in natural language

Why Gemini? Gemini was built for multimodal reasoning. We could directly go from image to meaning, while minimizing the need for OCR pipelines - we used it to do what it does best: saving time and operational efficiency.

Challenges we ran into

Familiarization with LeLamp's codebase.

Accomplishments that we're proud of

For a lot of us, completing our first hardware hack. Also, hours of debugging the device, and eventually winning and achieving our goal project.

What's next for LeLamp James

Integrating CV with a user interface.

Other domains:

Using the camera to detect unknown entries into rooms
Games, such as fitness tracking games
An IOS application to give users summaries on their learning patterns and experience