Effect Notebooks

Effect Notebooks lets you do data science work in Effect Force. The campaign allows you to upload data, and gives workers and environment to analyze and build scripts to clean your data to your specs

Comment

Inspiration

When learning about the Effect Network, I noticed that many of the use cases for Effect Force were for classifying data manually with the intent of using that data for machine learning. Having done some data science work myself, I thought about how most of the time is spent on cleaning data. If only there was a way to automate the formatting of data... Effect Network seemed like the perfect platform to implement such an idea. And with that I created Effect Notebooks.

What it does

Effect Notebooks is a campaign where workers can earn EFX tokens for writing code that will automate the cleaning of data, the way a task specifies. From the app side of things, you can upload a CSV and then give an example of how the data should be reformatted. Efffect Notebooks then takes this and builds a jupyter notebook file with the data preloaded for the worker. All the worker has to do is fill in the blanks of the cleanData function to earn tokens.

How we built it

I build Effect Notebooks using the Effect Force API, a local jupyter server, and a local flask server that helps persist and manage the file data

Challenges we ran into

The user interface is pretty minimal and could be cleaned up a bit. Learning how to use the Effect Force api took some time, but the examples of the website were helpful.

PLEASE NOTE: The task and the task creator are dependent on local APIs. You must be running these locally, as outlined in the README in order for the task to render correctly for you on the Effect Force Page. If you do not have these running you will not see the jupyter environment in the task page

Accomplishments that we're proud of

I'm proud of getting a real campaign up and being able to show a proof of concept of my idea.

What we learned

I learned a lot about IFrames and all of the security policies that web devs should know for security purposes. I also learned a lot about the Effect Network and how campaigns/tasks work.

What's next for Effect Notebooks

There is a ton of potential for more notebook based use cases outside of cleaning data. This was a proof of concept use case for the purpose of the hackathon, but there are many other ways we could improve the development environment to add more complex work to the Effect Network

Built With

Share this project:

Updates