1 Matching Annotations
  1. Apr 2024
    1. However, it is unclear how meaningful the notion of "zero-shot" generalization is for such multimodal models, as it is not known to what extent their pretraining datasets encompass the downstream concepts targeted for during "zero-shot" evaluation.

      What seems zero-shot performance by an LLM may well be illusionary as it is unclear what was in training data.