NYSee | Devpost

🧠 About NYSee

💡 Inspiration

With the theme of urban life, we focused on how chaotic and visually dense cities like NYC can be. Navigation depends heavily on reading signs, signals, and surroundings—something visually impaired individuals can’t always rely on, nor can they always depend on others for help.

NYSee aims to make users more independent by turning visual information into clear, spoken guidance. During testing, our app detected text like “Greenwich Village Storm District”—something none of us even noticed—highlighting how much important information can be missed.

⚙️ What it does

NYSee is a web app that converts surroundings into audio:

📸 Image analysis (Google Vision API) Detects objects + extracts key text and reads it aloud
📍 Location feature (NYC Open Data) Finds the nearest accessible pedestrian signal (APS) and tells the user:
- distance
- direction
- intersection

🏗️ How we built it

Frontend: HTML, CSS, JavaScript (accessible UI + speech synthesis)
Backend: FastAPI (Python)
APIs: Google Cloud Vision + NYC Open Data

We added logic to filter noisy OCR results and convert them into natural, readable speech.

🧩 Challenges

Filtering messy OCR output
Making responses sound natural instead of robotic
Handling inconsistent NYC dataset fields
Balancing detail vs. information overload

🚀 Future potential

NYSee can expand using more NYC Open Data, such as:

311 reports (broken signals, hazards)
street lighting conditions
terrain and obstacles

Many accessibility tools rely on maps, which can be difficult to interpret. NYSee instead focuses on real-time, spoken understanding of the environment.

❤️ Why it matters

NYSee turns a visually overwhelming city into something you can hear, helping users navigate with more confidence and independence.

Learning Experience

Most of our team members weren't familiar with the hackathon environment as well as full stack applications, so we prioritized learning things like full stack architecture, using APIs, using datasets, vanillaJS, etc. It was a lot of fun!