We are often asked about examples of red teaming and bias bounties. Imagine if we did a red teaming exercise for cookies – yes, cookies. A red teaming cookie event could involve participants of any background or skill level identifying the basics of the situation: does the cookie taste good or bad? Is the cookie too hard? Was it baked correctly or does it fall apart when it’s picked up? A cookie bias bounty goes deeper and would involve people with more skill or knowledge to identify what went wrong. In this case, someone who works in the bakery might realize that salt was used instead of sugar because of a mislabelled sugar container. Likewise, red teaming is the best starting point to identify potential issues with AI / ML models or systems. Bias bounties then go further to explain the problem and can result in mitigations or solutions.