UsefulScore

Ever wondered what classes you should take? How are you suppose to know whether the stuff the classes teach will be useful later on? Check the UsefulScore for any classes you might be interested in.

Comment

Inspiration

The inspiration for this project was as a solution to a personal problem. Post graduating and entering the job search, it came to my realization a lot of the things we studied for classes, ended up not actually being things relevant to working in the industry. While the most popular skills required by jobs within the field, were not taught or even introduced in school. So for future students, and university faculty hoping to make improvements to their program, a tool like UsefulScore would be extremely helpful.

What it does

The project is a combination of a few different tools, that when combined, allow users to be able to easily determine how useful the available courses are that their school offers. It scrapes Google for job postings and collects the information of each post, it parses the html page of a school course catalog to collect the information on courses offered, and lastly it provides a way for users to quickly measure the usefulness of a course they may be interested in, and how important the skills taught are to getting a job.

How we built it

  • Python+Selenium to automatically collect all of the data on jobs posted on Google, creating a table with every posts unique ID, the company that posted, the job role or title, and the job description
  • Python+BeautifulSoup to parse the html code of a schools course catalog page, creating a table of every course and the corresponding description
  • Python+mySQL.connector to store all of the information collected on jobs and courses into one mySQL database
  • Python+Flask to create the interface that allows users to interact with the collected data, and get specific results based on their personal plans

Challenges we ran into

Some of the early, minor problems we ran into include trouble successfully connecting to and inserting the data on jobs and courses into the mySQL database, and then reformatting any uncleaned and raw data into tidy and readable data. The biggest challenge was determining how to develop the connection between the front end as it's used, and how it retrieves and outputs the correct data from the mySQL database.

Accomplishments that we're proud of

Participating in my first Hackathon. As someone who had never done real work with SQL, the filling of the database, and then querying the database while performing calculations to determine the UsefulScores of courses and skills was a pretty big accomplishment. Although the Flask app was not visually appealing, just the creation of it at all was an accomplishment.

What we learned

A lot. It was our first experience working with a lot of the tools used for the project. Learned how to both move data from external sources into a SQL database, and retrieve that data to be used. We learned that creating a Django Rest API framework to allow for the interaction between an interface and database, is not something you do with zero front end, Django, or any API experience on a time constraint. A Python Flask app can perform similar functions with a much quicker learning curve to implement.

What's next for UsefulScore

The first thing that should be done for UsefulScore to get it to the point that it is a tool that people would want to use, is working on the visual aspects. One tool I wanted to implement was Github Actions to automate a schedule that runs the webscraper on daily or weekly intervals, so that the scores are always relevant to what employers are actually asking for at any point. Including a section that allows to provide the URL to their own university's course catalog, instead of having it hardcoded to one fixed course catalog, would allow for a much larger population to be able to use it.

Share this project:

Updates