Do papers submitted later receive lower review scores?
Every October, machine learning PhD students all over the world celebrate the ICLR deadline. It is now Thanksgiving season and the reviews are in.
In order to procrastinate on dealing with the reviews, we1 have analyzed which properties of papers, other than their scientific content, correlate with review scores.
ICLR is a top-tier machine learning conference that conducts its process on the OpenReview platform and releases all reviews and other data in real time. Paper Copilot has been doing a great job at tracking broad statistics for years. In this post, we ask questions that serious people like the maintainers of Paper Copilot did not dare to ask.
Submitting your abstract early correlates with better scores
ICLR has an abstract submission deadline and a paper submission deadline a few days apart; for the former, only the title, set of authors, and the abstract (summary of the paper) are required. The paper submission IDs are ordered by the time their abstracts were submitted.
The papers whose abstracts were submitted closer to the abstract submission deadline have a lower average final score.
Pre-rebuttal scores from 20262 show a similar trend:
This is not a novel finding, and it makes sense: if a paper is submitted close to the deadline, it is possible that the authors were not certain the paper will be ready in time. This effect increases with time, but is extremely strong for papers whose abstracts are submitted minutes before the deadline (the last point on both graphs).
More authors corrrelates with a higher score
The number of authors is positively correlated with the score. This might be a novel trend: 2024 has an even higher positive correlation, but 2023 has correlation 0.05, and 2022 has no statistically significant correlation. We don’t have the deanonymized scores for 2026 yet.
Longer titles receive slightly lower scores
There is a slight negative correlation of the length of the title with the score; the correlation reproduces across 2022-2026.
The number of authors per paper
and the number of words in the title are increasing
The same analysis for the number of words in the abstract and in the review text did not show any interesting trends.
As an aside, the growth of the number of paper submissions (and thus the number of people doing some sort of research) in machine learning is fascinating.
ICLR 2026 has close to 20,000 submissions; compared to 5000 in 2023. This has some implications on the expected impact of an average paper which we will not spell out here.
Bonus: Sentence case in titles is becoming less popular
There are two ways you could capitalize the title of a paper:
The first way is becoming increasingly uncommon:
Not sure what to do with this information yet, but you heard it here first.
“We” is short for “Claude 4.5 Sonnet and myself”.
In this post, we consider reviews for ICLR 2022-2026. The review system has changed for ICLR 2026, in that the “borderline accept” score had the numeric value 6 in 2022-2025, and is 5 for ICLR 2026. A naive adjustment would be to just subtract 1 from the scores for ICLR 2026, but there were other changes to the scoring scale and I don’t know if such an adjustment would be principled.