Jonathan Liu

Peer review remains a primary signal of research quality in science, yet its outcome is widely considered noisy and unpredictable. How much of the acceptance decision is recoverable from the paper alone? We find that much of it is. We construct a balanced ICLR benchmark with equal accepts and rejects and fine-tune a 7B vision-language model, PaperLens, that reaches 70.9% accuracy on ICLR 2025, suggesting that papers carry substantial signal about their eventual acceptance. More importantly, the predicted acceptance probability can be trusted: when the model is $X$% confident, its accuracy is close to $X$ %. We find that processing paper pages as images with a vision-language model outperforms tokenizing the text with a language model, suggesting that visual presentation carries acceptance signal beyond written content. Frontier general-purpose models, by contrast, accept nearly every paper when prompted to review. By integrating PaperLens' calibrated predictions into these models, we produce reviews grounded in accurate acceptance probabilities, closing the loop between generating useful feedback and predicting the review outcome.

We present a parameter-efficient Diffusion Transformer (DiT) for generating 200bp cell-type-specific regulatory DNA sequences. By replacing the U-Net backbone of DNA-Diffusion with a transformer denoiser equipped with a 2D CNN input encoder, our model matches the U-Net's best validation loss in 13 epochs (60× fewer) and converges 39% lower, while reducing memorization from 5.3% to 1.7% of generated sequences aligning to training data via BLAT. Ablations show the CNN encoder is essential: without it, validation loss increases 70% regardless of positional embedding choice. We further apply DDPO finetuning using Enformer as a reward model, achieving a 38× improvement in predicted regulatory activity. Cross-validation against DRAKES on an independent prediction task confirms that improvements reflect genuine regulatory signal rather than reward model overfitting.

Recent research has shown that hallucinations, omissions, and biases are prevalent in everyday use-cases of LLMs. However, chatbots used in medical contexts must provide consistent advice in situations where non-medical factors are involved, such as when demographic information is present. In order to understand the conditions under which medical chatbots fail to perform as expected, we develop an infrastructure that 1) automatically generates queries to probe LLMs and 2) evaluates answers to these queries using multiple LLM-as-a-judge setups and prompts. For 1), our prompt creation pipeline samples the space of patient demographics, histories, disorders, and writing styles to create realistic questions that we subsequently use to prompt LLMs. In 2), our evaluation pipeline provides hallucination and omission detection using LLM-as-a-judge as well as agentic workflows, in addition to LLM-as-a-judge treatment category detectors. As a baseline study, we perform two case studies on inter-LLM agreement and the impact of varying the answering and evaluation LLMs. We find that LLM annotators exhibit low agreement scores (average Cohen's Kappa=0.118), and only specific (answering, evaluation) LLM pairs yield statistically significant differences across writing styles, genders, and races. We recommend that studies using LLM evaluation use multiple LLMs as evaluators in order to avoid arriving at statistically significant but non-generalizable results, particularly in the absence of ground-truth data. We also suggest publishing inter-LLM agreement metrics for transparency. Our code and dataset are available here: https://github.com/BBN-E/medic-neurips-2025-demo.

We introduce Infinigen-Sim, a toolkit for generating realistic, procedurally generated articulated assets for robotics simulation. We include procedural generators for 12 common articulated object categories along with high-level utilities for use creating custom articulated assets in Blender. We also provide an export pipeline to integrate the resulting assets along with their physical properties into common robotics simulators. Experiments show that assets sampled from these generators are useful for movable object segmentation, training generalizable reinforcement learning policies, and sim-to-real transfer of imitation learning policies.