Inspiration
As millennials and members of an increasingly tech-oriented world, we are victims of mass consumption. Resources such as youtube provide us with endless amounts of video to watch, and most of our interaction occurs with videos and not real people. We believe that there exists a tangible link between people and themselves when they are captured on video- they are same people, after all. Thus, we created Viewr, which provides us with an analysis of predetermined traits for a speaker in a video. Viewr takes a video and brings up real-world analysis of it- to make the video become something more.
What it does
Viewr allows the the user to understand their video holistically. It shifts the focus away from the spoken word and onto the visual cues. Users are prompted to input videos of their choosing in order to be analyzed. Our analysis software returns with 6 normalized quantities: Happiness, sadness, surprise, calm, disgust, and anger. These are displayed on a graph that shows the change in time, as well as another dynamic graph that displays their values over the course of the video. In order to perform the analysis of the video, the video is decomposed into frames (a video is just an series of frames). These frames are analyzed on the individual level and then consolidated for our final product.
How I built it
Our application uses a number of different languages and frameworks. The backend is done as a Flask application coded using Python. Tensorflow is used to train the neural net. On the front end, the landing page and animations are developed with javascript, HTML, CSS (skeleton). The images are extrapolated from the video, they are processed and refined. The images are compared to others that were used to train the machine, and then the results are all displayed on a responsive graph.
Challenges I ran into
The issues came mainly with our larger undertakings. During the File Upload process, for example, we had to create a new file and source for every upload and make it unique using the hash value for the object. This meant that each upload has its own location for it to be processed from. We also had to examine the source code for the terminal library in order to run the neural network from the command line. This had to be modified in order to run it from the file, which we successfully were able to do after an in-depth analysis of how the process works.
Accomplishments that I'm proud of
We are very proud of maximizing what little we gave ourselves. Although a video may seem like a lot of information, in truth, it is nothing more than a repeated collection of pixels. We could have done other things, such as create a transcript of the video, but our ultimate aim was to push the limits of images and what we can do with them. We are very proud that we were able to do this.
We are also proud that we did not use an API for the creation of this project. This means that our project is not dependent on any third party. We are able to run this completely offline. On the note of independence, we are also proud of the fact that this works on any video. We had considered YouTube but opted against doing so due to copyright restrictions.
What I learned
We learned how to become more and more efficient as we work. We constantly had new ambitions and more ideas to add the project, and ultimately we were able to complete a lot of these goals. This happened because we prioritized working on the project, and we worked from the bottom up. We first made a minimum viable product and kept expanding it all the way until we reached the final, complete product. We also learned that it is not possible to do everything, but we take pride in what we were able to do.
What's next for Viewr
Next for our product is expansion beyond videos. We believe that aside from some minor qualities of the images (such as brightness), we have maximize the utility of these images. Videos have two functional parts, auditory information and visual information, and we hope to take advantage of the auditory information and use in conjunction with Viewr. We firmly believe that Viewr is just a foundation for the analysis of mass media, and we will be working on a better future by using and expanding Wave.
Log in or sign up for Devpost to join the conversation.