2 Matching Annotations
- Apr 2024
-
arxiv.org arxiv.org
-
However, it is unclear how meaningful the notion of "zero-shot" generalization is for such multimodal models, as it is not known to what extent their pretraining datasets encompass the downstream concepts targeted for during "zero-shot" evaluation.
What seems zero-shot performance by an LLM may well be illusionary as it is unclear what was in training data.
-
We consistently find that, far from exhibiting "zero-shot" generalization, multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance
Exponential increase in training data is needed for linear improvements in zero-shot results of LLMs. This implies a very near, more or less now, brick wall in improvement.
Tags
Annotators
URL
-