Having described the NAs themselves, Lett moves on to discussseveral important reliability and validity issues that should be ofconcern to needs analysts everywhere, but which have very rarelybeen discussed in the NA literature. These include the use ofconvenience samples of subject matter experts, the lack of readyexternal criteria for assessing the validity of analyses, the lack ofindependence of proficiency level ratings obtained via the collabora-tive group process, possible response bias, and the halo effect. Heidentifies potential solutions to several of the problems, e.g., stratifiedrandom sampling, use of surrogate or partial test-retest and modifiedsplit-half procedures for improving reliability, and relating DLIgraduates' language proficiency ratings to supervisors' field reportson their subsequent job performance in predictive validity studies. Hepoints out, however, that the increased costs in time and personnel,among other problems, would often render them inadequate, or insome cases preclude their adoption altogether.
example of NA (à voir)