2 Matching Annotations
 Mar 2023

h.readthedocs.io h.readthedocs.io

With respect to the predictive text portion of ChatGPT, a good nontechnical (nonmathematical) description of a related mathematical model is described in chapter 3 of:
Pierce, John Robinson. An Introduction to Information Theory: Symbols, Signals and Noise. Second, Revised. Dover Books on Mathematics. 1961. Reprint, Mineola, N.Y: Dover Publications, Inc., 1980. https://www.amazon.com/IntroductionInformationTheorySymbolsMathematics/dp/0486240614.

 Feb 2023

wordcraftwritersworkshop.appspot.com wordcraftwritersworkshop.appspot.com

The application is powered by LaMDA, one of the latest generation of large language models. At its core, LaMDA is a simple machine — it's trained to predict the most likely next word given a textual prompt. But because the model is so large and has been trained on a massive amount of text, it's able to learn higherlevel concepts.
Is LaMDA really able to "learn higherlevel concepts" or is it just a large, straightforward information theoreticbased prediction engine?
