VQA Challenge Workshop
Location: Room 301AB, Hawaii Convention Center
at CVPR 2017, July 26, Honolulu, Hawaii, USA

Home Program SubmissionAccepted Abstracts


Image


Introduction

The primary purpose of this workshop is to hold the 2nd edition of the Visual Question Answering Challenge on the 2nd edition (v2.0) of the VQA dataset introduced in Goyal et al., CVPR 2017. The 1st edition of the VQA Challenge was organized in CVPR 2016 on the 1st edition (v1.0) of the VQA dataset introduced in Antol et al., ICCV 2015. VQA v2.0 dataset is a more balanced version of VQA v1.0 which significantly reduces the language biases. VQA v2.0 is about twice the size of VQA v1.0.

Our idea in creating this new “balanced” VQA dataset is the following: For every (image, question, answer) triplet (I,Q,A) in the VQA v1.0 dataset, we identify an image I’ that has an answer A’ to Q such that A and A’ are different. Both the old (I,Q,A) and the new (I’,Q,A’) triplets are present in the VQA v2.0 dataset balancing the VQA v1.0 dataset on a per question basis. Since I and I’ are semantically similar, a VQA model will have to understand the subtle differences between I and I’ to provide the right answer to both images. It cannot succeed as easily by making “guesses” based on the language alone.

This workshop will provide an opportunity to benchmark algorithms on VQA v2.0 and to identify state-of-the-art algorithms that need to truly understand the image content in order to perform well on this balanced VQA dataset. A secondary goal of this workshop is to continue to bring together researchers interested in Visual Question Answering to share state-of-the-art approaches, best practices, and future directions in multi-modal AI.

In addition to invited talks from established researchers, we invite submissions of extended abstracts of at most 2 pages describing work in the areas relevant to Visual Question Answering such as: Visual Question Answering, (Textual) Question Answering, Dialog, Commonsense knowledge, Video Question Answering, Image/Video Captioning, Language + Vision. Accepted submissions will be presented as posters at the workshop. The workshop will be held on July 26th, 2017 at the IEEE Conference on Computer Vision and Pattern Recognition, 2017.


Invited Speakers

Image

Abhishek Das
Georgia Tech

Image

Sanja Fidler
University of Toronto

Image

Derek Hoeim
University of Illinois at Urbana-Champaign

Image

Hugo Larochelle
Google Brain

Image

Svetlana Lazebnik
University of Illinois at Urbana-Champaign

Image

Marcus Rohrbach
Facebook AI Research

Image

Kate Saenko
Boston University

Image

Anton Van Den Hengel
University of Adelaide

Image

Jason Weston
Facebook AI Research


Program

9:00 AM - 9:10 AM
Image
Welcome
Dhruv Batra (Georgia Tech)
9:10 AM - 9:35 AM
Image
Transfer of Specialized Knowledge for Vision-Language Tasks
Invited Talk: Svetlana Lazebnik (UIUC)
[Slides]
9:35 AM - 10:00 AM
Image
Answering Visual Questions with End-to-End Module Networks and Explaining Decisions
Invited Talk: Marcus Rohrbach (Facebook AI Research)
[Slides]
10:00 AM - 10:25 AM
Image
Why Did You Say That? Explaining and Diversifying Captioning Models
Invited Talk: Kate Saenko (Boston University)
[Slides]
10:25 AM - 10:45 AM
Morning Break
10:45 AM - 11:10 AM
Image
Dangers and Opportunities in Research with VQA
Invited Talk: Derek Hoeim (UIUC)
[Slides]
11:10 AM - 11:20 AM
Image
Overview of EvalAI
Deshraj Yadav (Georgia Tech)
[Slides]
11:20 AM - 12:00 PM
ImageImage
Overview of dataset, challenge, winner announcements, analysis of results
Yash Goyal (Virginia Tech), Aishwarya Agrawal (Virginia Tech)
[Slides]
12:00 PM - 12:05 PM
Image Image Image
Image Image Image
Image Image Image
Challenge Runner-Up Talk
Zhou Yu, Jun Yu, Chenchao Xiang, Dalu Guo, Jianping Fan and Dacheng Tao
[Slides]
12:05 PM - 12:15 PM
Image Image
Image Image
Image
Challenge Runner-Up Talk
YoungChul Sohn, Kibeom Lee, Jong-Ryul Lee and Gyu-tae Park
12:15 PM - 12:30 PM
Image Image Image
Image Image Image Image
Image Image Image Image
Challenge Winner Talk
Damien Teney, Peter Anderson*, David Golub*, Po-Sen Huang, Lei Zhang, Xiaodong He and Anton van den Hengel
[Slides]
12:30 PM - 1:45 PM
Lunch
1:45 PM - 2:10 PM
Image
VQA, and why it's asking the wrong question
Invited Talk: Anton Van Den Hengel (University of Adelaide)
[Slides]
2:10 PM - 2:35 PM
Image
Teaching Machines to Describe Images via Human Feedback
Invited Talk: Sanja Fidler (University of Toronto)
2:35 PM - 4:00 PM
ImageImageImageImage
ImageImageImageImage
Poster session and Afternoon break
4:00 PM - 4:25 PM
Image
Unifying QA, Dialog, VQA and Visual Dialog
Invited Talk: Jason Weston (Facebook AI Research)
[Slides]
4:25 PM - 4:50 PM
Image
Visual Dialog
Invited Talk: Abhishek Das (Georgia Tech)
[Slides]
4:50 PM - 5:15 PM
Image
GuessWhat?! Visual object discovery through multi-modal dialogue
Invited Talk: Hugo Larochelle (Google Brain)
[Slides]
5:15 PM - 5:55 PM
ImageImageImageImage
ImageImageImageImageImage
Panel
Future Directions
5:55 PM
Image
Closing Remarks
Devi Parikh (Georgia Tech)

Poster Presentation Instructions

1. Poster stands will be 8 feet wide by 4 feet high. Please review the CVPR17 poster template for more details on how to prepare your poster. You do not need to use this template, but please read the instructions carefully and prepare your posters accordingly.

2. Poster presenters are asked to install their posters between 12:15 PM and 1:45 PM. Push pins will be provided for attaching posters to the boards.


Submission Instructions

We invite submissions of extended abstracts of at most 2 pages describing work in areas such as: Visual Question Answering, (Textual) Question Answering, Dialog, Commonsense Knowledge, Video Question Answering, Image/Video Captioning and other problems at the intersection of vision and language. Accepted submissions will be presented as posters at the workshop. The extended abstract should follow the CVPR formatting guidelines and be emailed as a single PDF to the email id mentioned below. Please use the following LaTeX/Word templates.

  • LaTeX/Word Templates (tar): cvpr2017AuthorKit.tgz
  • LaTeX/Word Templates (zip): cvpr2017AuthorKit.zip

  • Dual Submissions

    We encourage submissions of relevant work that has been previously published, or is to be presented at the main conference. The accepted abstracts will be posted on the workshop website and will not appear in the official IEEE proceedings.


    Dates

    May 28, 2017 Extended Submission Deadline
    May 29, 2017 Decision to Authors


    Where to Submit?

    Please send your abstracts to [email protected]


    Organizers

    Image

    Aishwarya Agrawal
    Virginia Tech

    Image

    Yash Goyal
    Virginia Tech

    Image

    Tejas Khot
    Virginia Tech

    Image

    Peng Zhang
    Virginia Tech

    Image

    Jiasen Lu
    Virginia Tech

    Image

    Larry Zitnick
    Facebook AI Research

    Image

    Dhruv Batra
    Georgia Tech

    Image

    Devi Parikh
    Georgia Tech


    Webmaster

    Image

    Akrit Mohapatra
    Virginia Tech


    Contact: [email protected]