he most convincing historical argument revolves around preferential attachment, or rich-get-richer. In a friendship network, this means that those who have a lot of friends are at a higher risk of making even more friends; they have a larger circle that can, in turn, introduce them to even more people. This positive feedback loops continues until a few people dominate the landscape in terms of connectivity. This is also true of citation networks, in that articles that already are highly cited are more likely to be seen by other scholars, and cited again in turn.

A historian may wish to see the evolution of transitivity across a social network to find the relative importance of introductions in forming social bonds.

Unsurprisingly, in the early modern Republic of Letters, we see this as well. The probability that a person will correspond with another in the network increases if they have a history of previous contact.

For the historian, it is unlikely that the diameter will be more useful than the average path length on most occasions.

the vital connective tissue

There is a tendency when using graphs to become smitten with one’s own data. Even though a graph of a few hundred nodes quickly becomes unreadable, it is often satisfying for the creator because the resulting figure is elegant and complex and may be subjectively beautiful, and the notion that the creator’s data is ‘complex’ fits just fine with the creator’s own interpretation of it. Graphs have a tendency of making a data set look sophisticated and important, without having solved the problem of enlightening the viewer.[13]

Besides this special case of two opposing colors, it is best to avoid using hue to represent quantitative data.

While my description of the chart does describe the trends accurately, it does not convey the sheer magnitude of difference between earlier and later years as covered by dissertations, nor does it mention the sudden drop in dissertations covering periods after 1970.

an additional layer in the hermeneutic process of hypothesis formation

exactly what data are available and how they interconnect.

Networks, blunt methods for considering connections,do not provide a way to reflect on the quality of

trans-generational and transnational exchange

For example, Correspondence Courseincludes no letters to or from Austrian performance artist and filmmaker, Valie Export, although the two artists met in 1970, but Export ismentioned several times by Schneemann.25

What we really want here of course is a visualization that combines all the things, but I’ve resisted creating one for now. The complex historical questions of who gets counted when we count in histories of women’s liberation exists because data reduces people’s lived experiences to columns on a spreadsheet.

I share this not to reveal my own sloppy data, but to highlight the difficulties of doing this kind of visualization.

We cannot rely only on the computer-driven groups to use in analyzing texts.  The next step is to look at the texts that contain repeating word patterns and conduct a close reading to see what we can learn about the topic. Plotting the topic over time enables us to locate trends in how important the topic was to the author, or when we compare them with other authors, we can investigate differences in the ways that two authors valued these topics or the different ways that they expressed themselves.

What topic modeling can offer a historian is an objective snapshot of the content of the collection.

Even more significantly, topic modeling allows us a glimpse not only into Martha’s tangible world (such as weather or housework topics), but also into her abstract world.

Yet this pattern bolsters the argument made by Ulrich in A Midwife’s Tale, in which she points out that the first half of the diary was “written when her family’s productive power was at its height.” (285) As her children married and moved into different households, and her own husband experienced mounting legal and financial troubles, her daily burdens around the house increased. Topic modeling allows us to quantify and visualize this pattern, a pattern not immediately visible to a human reader.

In essence, topic modeling accurately recognized, in a mere 55 words (many abbreviated into a jumbled shorthand), the dominant theme of that entry:

Instead, the program is only concerned with how the words are used in the text, and specifically what words tend to be used similarly.

MALLET generated a list of thirty topics comprised of twenty words each, which I then labeled with a descriptive title.

it worked. Beautifully

topic modeling, a method of computational linguistics that attempts to find words that frequently appear together within a text and then group them into clusters.

EMOTION words

When its entry scores are aggregated into months of the year, it shows exactly what one would expect over the course of a typical year:

When you encounter someone else’s topic model, do not accept at first glance. Rather, to understand the potentials and pitfalls, you must be aware of how the tools work and their limitations.

As you read that essay, consider for yourself the choices we have made in how we perform the topic model, and in how we visualize the results. How justified are we in those choices?

We mention this here to highlight the speed with which the digital landscape of tools can change. When we initially wrote about Paper Machines, we were able to topic model and visualize John Adams diaries, scraping the page itself using Zotero. When we revisited that workflow a few months later, given changes that we had made to our own machines (updating software, moving folders around and so on, and changes to the underlying html of the John Adams Diaries website), it –our workflow- no longer worked! Working with digital tools can sometimes make it necessary to not update your software! Rather, keep in mind which tools work with what versions of other supporting pieces.

Different tools give different results even if they are all still ‘topic modeling’ writ large. This is a useful way to further understand how these algorithms shape our research, and is a useful reminder to always be up front about the tools that you are using in your research.

Because the STMT is most useful for us as historians when we have documents with dates, let us consider the workflow for getting a digitized diary off a website and into the STMT. You may wish to skip this section; you can obtain the scraped data csv from our website.

Available in a Google Code repository at https://code.google.com/p/topic-modeling-tool/, the GTMT provides quick and easy topic model generation and navigation.

our suspicions are confirmed:

remove stopwords, normalize text by standardizing case, and tweak your iterations, size of topic descriptors, and the threshold at which you want to cut topics off

In fact there is a danger in using topic models as historical evidence; they are configurable and ambiguous enough that no matter what you are looking for, you just might find it. Remember, a topic model is in essence a statistical model that describes the way that topics are formed. It might not be the right model for your corpus. It is however a starting point, and the topics that it finds (or fails to find) should become a lens through which you look at your material, reading closely to understand this productive failure. Ideally, you would then re-run the model, tweaking it so that it better describes the kind of structure you believe exists.

Leave a comment on paragraph 66 0 There is a fundamental difficulty however. When we began looking at the Gettysburg Address, Hollis was instructed to look for two topics that we had already named ‘war’ and ‘governance’. When the computer looks for two topics, it does not know beforehand that there are two topics present, let alone what they might mean in human terms. In fact, we as the investigators have to tell the computer ‘look for two topics in this corpus of material’, at which point the machine will duly find two topics. At the moment of writing, there is no easily-instantiated method to automatically determine the ‘best’ number of topics in a corpus, although this will no doubt be resolved. For the time being, the investigator has to try out a number of different scenarios to find out what’s best. This is not a bad thing, as it forces the investigator continually to confront (or even, close-read) the data, the model, and the patterns that might be emerging.

In fact there is a danger in using topic models as historical evidence; they are configurable and ambiguous enough that no matter what you are looking for, you just might find it.

it is possible to decompose from the entire collection of words the original distributions held in those bags and buckets

f you are a literary scholar, you will understand what a ‘topic’ might mean perhaps rather differently than how a librarian might understand it,

it is possible to decompose from the entire collection of words the original distributions held in those bags and buckets.

What is a topic, anyway?

Topic modeling can offer us some new groupings of documents that we might have overlooked, and it will give us the capacity to analyze Sanger’s rhetoric over time, looking for key changes. An example might be the belief among women’s historians that Sanger abandoned her feminist rationales for birth control in the late 1910s and early 1920s as she sought support from experts in the fields of medicine, social work and eugenics.

The most descriptive label I could assign this topic would be EMOTION – a tricky and elusive concept for humans to analyze, much less computers. Yet MALLET did a largely impressive job in identifying when Ballard was discussing her emotional state. How does this topic appear over the course of the diary?

What topic modeling can offer a historian is an objective snapshot of the content of the collection.  Rather than relying on our own readings of documents to combine them together into subject categories, we look instead to the words that appear together most frequently and then label those words in ways that make sense to us. 

Sometimes it might be necessary to break one of these csv files into separate text files, in order to do the next stage of your analysis. For instance, imagine that you had scraped John Adams’ diaries into a single csv file, but what you really wanted to know was the way his views on governance changed over time. This might be a question for which topic modeling (see chapter four) could be well suited; many topic modeling tools require each document (here, a diary entry) to be in its own text file.

Breaking a CSV file into separate txt files

Yet we believe that for all the importance of Big Data, it does not offer any change to the fundamental questions of historical knowing facing historians.

Yet with such a visualization the main downside becomes clear: we lose context. Who are the protagonists? Who are the villains? As adjectives are separated from other concepts, we lose the ability to derive meaning. For example, a politician speaks of “taxes” frequently: but from a word cloud, it is difficult to learn whether they are positive or negative references. With these shortcomings in mind, however, historians can find utility in word clouds.

But the changing words are useful.

they are a very useful entryway into the world of basic text mining.

Having large datasets

word cloud. In brief, they are generated through the following process. First, a computer program takes a text and counts how frequent each word is. In many cases, it will normalize the text to some degree, or at least give the user options: if “racing” appears 80 times and “Racing

It also represents the inversion of the traditional historical process: rather than looking at documents that we think may be important to our project and pre-existing thesis, we are looking at documents more generally to see what they might be about. With Big Data, it is sometimes important to let the sources speak to you, rather than looking at them with pre-conceptions of what you might find.

In brief, they are generated through the following process. First, a computer program takes a text and counts how frequent each word is. In many cases, it will normalize the text to some degree, or at least give the user options: if “racing” appears 80 times and “Racing” appears 5 times, you may want it to register as a total of 85 times to that term.

One in particular that Stray mentions is called ‘Tabula’, which can be used to extract tables of information from PDFs, such as may be found in census documents.

The vocabulary of regular expressions is pretty large, but there are many cheat sheets for regex online (one that we sometimes use is http://regexlib.com/CheatSheet.aspx. Another good one is at http://docs.activestate.com/komodo/4.4/regex-intro.html)

Regular expressions can be mixed, so if you wanted to find words only matching “cat”, no matter where in the sentence, you’d search for ¶ 19 Leave a comment on paragraph 19 0 \bcat\b ¶ 20 Leave a comment on paragraph 20 0 which would find every instance. And, because all regular expressions can be mixed, if you searched for ¶ 21 Leave a comment on paragraph 21 0 \bcat|dog\b

The astute reader will have noticed a problem with the instructions above; simply replacing every instance of “dog” or “cat” with “animal” is bound to create problems. Simple searches don’t differentiate between letters and spaces, so every time “cat” or “dog” appear within words, they’ll also be replaced with “animal”. “catch” will become “animalch”; “dogma” will become “animalma”; “certificate” will become “certifianimale”. In this case, the solution appears simple; put a space before and after your search query, so now it reads: ¶ 9 Leave a comment on paragraph 9 0 dog | cat ¶ 10 Leave a comment on paragraph 10 0 With the spaces, “animal” replace “dog” or “cat” only in those instances where they’re definitely complete words; that is, when they’re separated by spaces.

When you type the vertical bar on your keyboard (it looks like |, shift+backslash on windows keyboards), which means ‘or’ in regular expressions. So, if your query is dog|cat and you press ‘find’, it will show you the first time either dog or cat appears in your text. Open up a new file in your editor and write some words that include ‘dog’ and ‘cat’ and try it out.

In addition to the basics provided here, you will also be able to simply search regular expression libraries online: for example, if you want to find all postal codes, you can search “regular expression Canadian postal code” and learn what ‘formula’ to search for to find them

Regular expressions can often be used right inside the ‘Find and Replace’ box in many text and document editors, such as Notepad++ on Windows, or TextWrangler on OS X. You cannot use regex with Microsoft Word, however!

a regular expression is just a way of looking through texts to locate patterns. A regular expression can help you find every line that begins with a number, or every instance of an email address, or whenever a word is used even if there are slight variations in how it’s spelled. As long as you can describe the pattern you’re looking for, regular expressions can help you find it. Once you’ve found your patterns, they can then help you manipulate your text so that it fits just what you need.

McGill University servers

load text or pdf files into the system

The other possibilities are even more exciting. The Concordance Plot traces where various keywords appear in files, which can be useful to see the overall density of a certain term. For example, in the below visualization of newspaper articles, we trace when frequent media references to ‘community’ in the old Internet website GeoCities declined (figure 3.6)

import files

visible data work and what remains as invisible labor.

The history and development of how we became data subjects in cultures of data are just now beginning to be told by historians and sociologists.

"era of data" with sensor networks and 5thgeneration mobile networks, as personal computing devices and internet saturation become tighter and tighter in our homes, institutions, and public spaces.

. But as the following contributions show, data is often characterized as a natural resource out in the wild to be discovered and sourced, with all the requisite taming of nature, including the frontierism metaphors of cowboys, lumberjacks, and gold rush miners.

Creators' agency over data that has been created, collected, and managed is thus central to understanding the datafication of culture. Indeed, calls for agency over our individual data footprints and practices of collection exist, but more methods for understanding how individuals are interpolated through sociomaterial practices of data collection as subjects are still needed.17

But we know that data is not just given, it is taken up by people and given forms, standards, names, putting it into relationships with cultural practices.4Data is not nature, waiting to be tamed; it is always already a cultural product.5

it provides a quite quick and relatively painless way to get a broad sense of what is going on within your documents.

it provides a quite quick and relatively painless way to get a broad sense of what is going on within your documents.

Thus, you could use a spreadsheet program to create bar charts of the count of documents with an ‘architecture’ tag or a ‘union history’ tag, or ‘children’, ‘women’, ‘agriculture,’ etc. We might wonder how plaques concerned with ‘children’, ‘women’, ‘agriculture’, ‘industry’, etc might be grouped, so we could use Overview’s search function to identify these plaques by search for a word or phrase, and applying that word or phrase as a tag to everything that is found. One could then visually explore the way various tags correspond with particular folders of similar documents.[5]

Why does this division exist? That would be an interesting question to explore.

Why does this division exist? That would be an interesting question to explore.

The decision by Mario Gonzalez illustrates how individuals create their own life-narrative

However, having the opportunity to build up a greater understanding of an individual’s lifestyle and their social networks brings with it ethical concerns. These concerns are increased by their involvement in a criminal activity. Disclosing the identity of a ‘forgotten’ individual and their connection to crime has the potential to cause harm. Therefore, careful thought is needed about how this evidence is presented when sharing research findings, and in this digital age this may not just be in the form of a printed academic journal.

This study wanted to consider the period from a new perspective – that of the consumer. Of interest was:

Graphic designers intricate two page spreads are chopped in half when digitisation occurs at the page level. Layouts that carefully juxtaposed articles, poems, and photographs on a page are eradicated when individual items are digitised as separate files. An artist’s delicate work loses nuances when contrast levels are set to prioritise text legibility.

I adopt instead the more specific term, ‘digital archival environment’, to describe accessing online digitised surrogates of materials taken from archives.

How can a researcher determine whether individuals who appear in a digital archival environment have consented to this and what controls should they look for to mitigate this increased exposure? Researchers can check for copyright information, investigate searchability settings, and look for procedures that enable individuals to have materials taken down.

comes to questions of privacy or evolving identities

n this chapter we explore how history-making practices of radical and lesbian feminists offer a model of cultural history preservation and transmis-sion for those of us who create digital resources.

the macroscope makes it easier to grasp the incredibly large. It does so through a process of compression, by selectively reducing complexity until once-obscure patterns and relationships become clear. Often, macroscopes produce textual abstractions or data visualizations in lieu of direct images.[1]

we believe that we are now on the cusp of a third revolution in computational history.

big is in the eye of the beholder