which scheme to use; tokenization is learned from text data based on an algorithm that I will describe later in this post. In OpenAI’s GPT models, most English words are represented using one or a small number of tokens.

Tokenization is the process of converting written language into integers (an integer is a whole number). You might think that tokenization is as simple as a=1, b=2, c=3, etc. That is one way to tokenize text, but it turns out that it’s not the only way — and certainly not the most optimal way. For example, the figure below shows three ways to represent the word “Hello” using integers.

Language models like ChatGPT and Claude don’t work with text. That’s because LLMs are based on mathematics, and math works on numbers, not on words. Therefore, LLMs need to work on numbers.

Educational material on large language models (LLMs) comes in one of two categories: (1) high-level overviews that avoid details; or (2) highly technical tutorials written for computer scientists.

ut the LLM cannot backtrack once it has gone this far. It must go on, each of its own words are clues on how to proceed. What awards might a computer scientist professor have one? A lot of people talk about the “Turing Award” in computer science so that might be a good guess based on the words so far. Except in reality I have not won any such award.

Large Language Models are just guessing what you want to hear. They often come up with something that we want to hear based on the clues we leave. The better the clues we leave, the more likely we are to get what we want, such as a right answer to a question. But it is never guaranteed.

Large Language Models (LLMs) are fancy artificial neural networks. But you don’t have time to learn the math or engineering. Unfortunately, a lot of primers will throw up diagrams like this:

Large Language Models are a type of artificial neural network called a Transformer. It is going to go through the process of transforming your words into… something. Before we get to what that something is, let’s walk through the transformation process. There are basically two transformations that occur over and over again:

I’m not saying that a Large Language Model does sentence diagramming. But that intuition will sort of work. The LLM is looking at all pairs of words and assessing if they are relevant to each other or not. And if they are relevant to the missing next word, then they contribute more heavily to the next word guess.

series of tubes,” in the immortal words of the Alaska senator Ted Stevens, went online in the late nineteen-sixties, though “tubes” exaggerates its concreteness. Technically, the internet is a protocol: a set of rules that let computers send and receive data over various networks by breaking it into “packets.”

In the beginning, the internet was without form, and void, and data trickled through the ports of the routers

Young people don’t understand what it took to make the web,” he said. “It took companies giving up their patent rights, it took individuals giving up their time and energy, it took bright people giving up their ideas for the sake of a common idea.

Still, his creation keeps growing, absorbing our reality in the process. If you’re reading this online, Berners-Lee wrote the hypertext markup language (HTML) that your browser is interpreting.

the World Wide Web, in 1989, but people informed of this often respond with a joke: Wasn’t that Al Gore?

In 2004, after losing a vote, a group of browser developers who wanted to keep improving HTML formed a rival standards body. Berners-Lee considered the move a power grab, describing it as “the first real blow to the integrity of the World Wide Web.”

the web’s design supple amid exponential growth, even when that clashed with demands for more features. The lodestar was Berners-Lee’s “principle of least power,” which dictated a minimal architecture. “

He bought domain names for everyone in the family and encouraged my early experiments in programming. At recess in middle school, while others played soccer or traded Yu-Gi-Oh! cards, I pored over tomes on HTML, JavaScript, and PHP—which paid off, socially, when I built a proxy server to let classmates access banned Flash games.

at linked them together. On August 6, 1991, the web’s first page, http://info.cern.ch, went online, introducing itself as “a wide-area hypermedia information retrieval

HTML, the language of web pages; HTTP, the protocol that governed their transmission; and URLs, the addresses that linked them together. On August 6, 1991, the web’s first page, http://info.cern.ch, went online, introducing itself as “a wide-area hypermedia information retrieval initiative aiming to give universal access to a large universe of documents.” Soon enough, there would be porn.

When you do actions in Git, nearly all of them only add data to the Git database. It is hard to get the system to do anything that is not undoable or to make it erase data in any way. As with any VCS, you can lose or mess up changes you haven’t committed yet, but after you commit a snapshot into Git, it is very difficult to lose, especially if you regularly push your database to another repository.

Figure 4. Storing data as changes to a base version of each file Git doesn’t think of or store its data this way. Instead, Git thinks of its data more like a series of snapshots of a miniature filesystem. With Git, every time you commit, or save the state of your project, Git basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot. To be efficient, if files have not changed, Git doesn’t store the file again, just a link to the previous identical file it has already stored. Git thinks about its data more like a stream of snapshots

So, what is Git in a nutshell? This is an important section to absorb, because if you understand what Git is and the fundamentals of how it works, then using Git effectively will probably be much easier for you. As you learn Git, try to clear your mind of the things you may know about other VCSs, such as CVS, Subversion or Perforce — doing so will help you avoid subtle confusion when using the tool. Even though Git’s user interface is fairly similar to these other VCSs, Git stores and thinks about information in a very different way, and understanding these differences will help you avoid becoming confused while using it.