Harit Vishwakarma

@harit_v

Postdoc@Oxford, Reliable AI, building @paperena_ai, prev. @WisconsinCS, @iiscbangalore, @IBMResearch.

Oxford, UK

harit7.github.io

Joined December 2012

2,107 Following

687 Followers

Replies Reposts Media

Pinned
Harit Vishwakarma
@harit_v
Sep 23
Overwhelmed with the number of ICLR submissions and the projected growth to millions of papers in next few years? The paper mill is there, but is it any good? can this be aligned to what we care? Join us at the Frontier Data Summit to see how @paperena_ai can help. TL;DR:
Snorkel AI
@SnorkelAI
Sep 23
Paperena evaluates AI scientists over realistic research cycles: writing, reviewing, and revising. It measures factuality, reproducibility, novelty, and community behavior. From @harit_v and @paperena_ai, one of 25+posters at Frontier Data Summit. frontierdatasummit.ai
1
Harit Vishwakarma
@harit_v
Oct 22
Exactly! This an uncomfortable situation now. All the labour researchers did for writing and reviewing papers was in a different spirit. It was meant to be read by the community. There is a secondary use of that content now — for training and evaluation of AI. I guess, the
Yuval Benjamini
@yuvalbenj
Oct 22
Lets admit: ICLR reviewing is basically providing free huge-scale expert QAing to for llm-companies paper writing. The product is obviously not ready to actually write high-level science papers, though its close. There should be submission cost.
Harit Vishwakarma
@harit_v
Oct 22
At this point, ICLR and OpenReview feel more like data collection and labeling platforms. I did 3 HITs, 2 labeled as slop and 1 full review. AISTATS pile is looking much better. Less popular venues are still spared from the DDoS. Anyways, lots of cool benchmarks/datasets to come
2
Harit Vishwakarma
@harit_v
Oct 20
Proposing new acronyms that send the likes of FIFO, LIFO into existential crisis😅 SISO: Slop In, Slop Out. MISO: Meaning In, Slop Out. (better left as 🍜) SIMO: Slop In, Meaning Out.
Harit Vishwakarma
@harit_v
Oct 7
Evaluation is underrated and underfunded despite its importance and complexities. Great to see the success and expansion of this program.
vincent sunn chen
@vincentsunnchen
Oct 7
Today, we’re expanding @SnorkelAI’s Open Benchmarks Grants by 10x to a $30M commitment: - Funding a more diverse, robust, and continuously-updated ecosystem of open benchmarks - Launching the Open Benchmarks Red Team to continually test and strengthen them - Introducing the
1

Harit Vishwakarma

Harit Vishwakarma’s posts