Seinfeld-bot

• 

  Final Poster

Final Writeup https://docs.google.com/document/d/1k7Cqnd1XMru3xUEPJ_N-ydo7xYg9wq3OWivsf-qLSfo/edit?usp=sharing

Checkin 2 Reflection: https://docs.google.com/document/d/1GTIOf4JtZ0E3qoXGyc5Pcq6Ke_wkgiif9SS15ohGv8U/edit?usp=sharing

Checkin 1 Proposal: Seinfeld-bot Logan Bauman (lbauman), Ben Fiske (bfiske1)

Introduction: Of all the abstract concepts humans have, humor is one of the hardest to grasp, even for people. Being funny is a gift, and creating new and original jokes is a skill that few humans possess. In a world where content consumption is at an all time high, the demand for humor is equally massive. A natural question to ask is whether or not we can use techniques from deep learning to write original jokes using a large data set of jokes. We want to create a model that, given a large data set of jokes, can reproduce jokes similar to the ones in the data set, but with sufficient variety to continue to be of interest for a long time. Because of this structure, this is an unsupervised method of learning.

Related Work: This is a problem that has been tried a few times from what we can tell, but none have been able to emulate anything that imitates human produced humor. There are blog posts (https://towardsdatascience.com/can-a-robot-make-you-laugh-teaching-an-ai-to-tell-jokes-815f1e1e689c) that mostly use language models, some of them very advanced like GPT2, that they train on a data set of jokes. A very powerful language model will be able to produce text that semantically looks like a joke, but we hope that adding an adversarial model will force our LSTM to actively produce something that is funny.

Data: A critical component of achieving success in this project is having a large and robust data-set. Luckily, there are many large datasets available on the internet. This one (https://www.kaggle.com/abhinavmoudgil95/short-jokes) containing over 200k jokes is good for our purpose. However, we want to make this problem a little simpler, so we plan on filtering out the jokes that are above a certain length.

Methodology: We plan on using a GAN model to solve this problem. We plan on using an LSTM to generate the jokes, and use an attention based feed forward model as our discriminator. We plan on training the model by producing a batch of samples with the generator, then mixing them with a batch of real samples, and then running the whole batch through the discriminator, alternating every batch whether we optimize the discriminator or the generator. We hope that our generator will eventually converge to something that produces varied and sensical jokes. If this doesn’t work, we could also try using just an LSTM language model that predicts the next word in a given sequence that we train on the joke data set, in a similar way we did on the LM homework.

Metrics: To test this model, we want to have it produce a large amount of jokes and evaluate them by surveying a number of fellow students to determine how funny they are in comparison to the human-created jokes represented in our training data. Our goal is to have the network able to produce sentences that are coherent, while also being funny, or at least making an honest attempt at humor. We also want the network to produce a diverse sample of jokes, and not just converge to a model that produces the same output and fall victim to mode collapse. Our base goal is to be able to produce sentences that have some semblance of a joke. However, our target is to be able to produce samples that have the structure of a joke, and are considered by the humans in our sample to be equally humorous as compared with human written jokes. If we are able to implement all of this, an awesome reach goal would be to add a feature that generates a joke based on keyword(s), or even a longer input such as a headline or factoid.

Ethics: One serious concern about this project is the potential bias in our dataset. It’s common knowledge that jokes often reflect harmful and dangerous biases that exist. Racist and sexist jokes are common, and there is a high chance that in the dataset containing over 200k jokes, there are some harmful and offensive jokes. We will need to be careful in the way that we preprocess our data and filter out jokes to make sure that our model isn’t learning to tell offensive jokes. We also consider the broader ethical questions that arise out of having deep learning algorithms creating content for people to consume. If our model could be used to create an abundance of jokes and memes that people spend a lot of time looking at, there is a serious question about if we want people looking at artificially created jokes all day, and if that is beneficial to society. We want to make sure our model is actually contributing something valuable and constructive to society.

Division of Labor: As the preprocessing for this project is minimal, the bulk of the work will be in actually coding and building the architecture for our network. We plan on working on these parts together, as the intricacies of the network are complicated and having each other to lean on and help each other out will be invaluable in getting our network to work. While we plan on working together, our model is naturally split into generator and discriminator, so it will make natural sense to have one of us focus more on one, and the other focus more on the other. Right now, we think Logan will be mostly responsible for the generator, while Ben will work more on the discriminator.

Built With

• python