The first few months of working full-time on Computational Dostoevsky as a postdoc have been very busy, and we as a team have managed to get quite a lot done. Read on to see what we’ve been up to.

Project re-organization

With the help of the wonderful Joey Takeda, we’ve redone much of the structuring and data organization of our project. This includes:

Merging all the previously separate repositories into one comprehensive repository. Alongside being simply more aesthetically pleasing (who doesn’t love a well-organized file system?), this makes it infinitely easier to process and manipulate our files. For example, if we want to extract all instances of speech (more on this later) in our encoded novels, we can now run the code that does this over all the novels at once, rather than one at a time. Or, if we want to run diagnostics over our files (more on this later, too) we can do so quickly and accurately.

Creating our own project schema. Up until now we’ve been validating (i.e. making sure our encoding is correct) our project with the standard TEI schema. We’ve now put together our own schema, which is preferable for three reasons. First, it is composed of only those TEI elements and attributes that we use in our own project. This minimalism constrains the possibilities of encoding, thereby ensuring our encoding decisions are standardized across the corpus. Second, we are able to modify parts of the TEI to fit our own needs better. As one small example, we can require a small subset of values (patronymic, diminutive, etc.) when encoding a character’s name, something which the general TEI schemas do not do. If the encoding of a name does not meet our schema’s requirements, we receive a validation error and can fix the mistake from there. Third, the creation of our own schema (particularly in what’s called an ODD file) enables us to easily document our encoding practices for the public to see.

Running diagnostics over our corpus. A schema can’t (or not easily) catch every error. As one example, since our project is encoded by humans who occasionally make mistakes, there are numerous small typos in the xml:ids of characters in the novels. (An xml:id is basically just a unique identifier for a character. #afk, for instance, is the xml:id for Alyosha Fyodorovich Karamazov.) Sometimes #afk is entered mistakenly as #akf, which seems like a small thing on the surface, but since computers can only know what we say and not what we mean, a mistake like this can cause problems if we want to extract data tied to Alyosha’s xml:id – a processor won’t know that the typo #akf is supposed to be Alyosha. Our diagnostics code now quickly scans the entire corpus and alerts us to any xml:id that isn’t defined in our project’s dictionary of characters, which is where we tie xml:ids to character names.

Expanding extraction of speech data. As you might have read in the previous blog post, Kate and Katia have been working on networks of speech in the novels. Joey and I spent a lot of time refining the project code to extract precisely those kinds of speech that Kate and Katia are currently interested in. Critically, this revamped code eliminates much possibility for error during the transfer of our data from our novels into Gephi, the software we use for network visualization. Previously, we were manually filtering out speech data we didn’t need, which, long story short, got real messy real quick. Now our extraction process is much more computationally selective from the outset, which cuts down manual filtering (and thus, mistakes) to essentially zero.

Standardizing speech. Speaking of speech data, one of my biggest projects over the last few months has been to standardize speech encoding across the corpus, particularly in really tricky group scenes like gambling halls, revolutionary conspiracy meetings, trials, drunken escapades, funerals, body disposals… What? it is Dostoevsky we’re talking about. In the past, and despite our best efforts, one result of novels being encoded by different people was that these different people understood speech differently, which was reflected in differing encoding practices. This was especially the case in group scenes.

Consider a scenario where Prince Myshkin is talking to everyone (and thus not to anyone in particular) on Lebedev’s veranda. Different people may have encoded this in the following ways:

<said aloud="true" direct="true" who="#mysh" toWhom="">[insert speech here]</said>

That is, Myshkin is speaking, but technically not to anyone.

<said aloud="true" direct="true" who="#mysh" toWhom="#lebed">[insert speech here]</said>

That is, Myshkin is speaking only to one person (here, Lebedev). This was very common when a small portion of the larger speech act was directed to one person in particular. Such as (and I’m making this example up… kind of): “Myshkin turned to Lebedev and said, ‘Why do you act like this? Why does Lebedev act like this, everyone?’” Obviously, the example of encoding given above would ignore that Myshkin, in addition to talking to Lebedev, is also talking to whoever else is present on the veranda.

<said aloud="true" direct="true" who="#mysh" toWhom="#lebed #agla #rogozh #gavr">[insert speech here]</said>

That is, Myshkin is speaking simultaneously to Lebedev, Aglaya, Rogozhin, and Ganya. This gets us a lot closer to reality, but only when Myshkin is indeed addressing all of these people together. What happens, similar to the example above, if Myshkin switches addressees throughout his speech act? (Again, making this up) “‘Aglaya, I love you, but I really love Nastasya Filippovna. Yes, that’s right Rogozhin, I said that. Lebedev, don’t you dare gossip about this! Ganya, why are you so pale??’” Though this presents as one instance of speech (it’s all contained in one quotation), it really is separate instances of speech to separate people. The encoding above doesn’t reflect that.

<said aloud="true" direct="true" who="#mysh" toWhom="#verandagroup">[insert speech here]</said>

That is, everyone Myshkin speaks to on the veranda is lumped together into one group, which we define at the end of the text in the “. This technically could work if Myshkin is genuinely addressing the group simultaneously as a whole – “‘I don’t feel well!’ Myshkin exclaimed to everyone.” – but it isn’t very useful in network analysis because it obscures the connection Myshkin has to individual people. And, of course, it wouldn’t capture the scenario above where Myshkin switches addressees in one and the same speech act.

So, what is the new process like?

Well, if there is speech legitimately simultaneously to everyone in a group, it will look like one of the examples above:

<said aloud="true" direct="true" who="#mysh" toWhom="#lebed #agla #rogozh #gavr">[insert speech here]</said>

But to capture the flow of speech that switches addressees, we’ll do the following:

<said aloud="true" direct="true" who="#mysh" toWhom="#agla">Aglaya, I love you, but I really love Nastasya Filippovna.</said> <said aloud="true" direct="true" who="#mysh" toWhom="#rogozh">Yes, that’s right Rogozhin, I said that.</said> <said aloud="true" direct="true" who="#mysh" toWhom="#lebed">Lebedev, don’t you dare gossip about this!</said> <said aloud="true" direct="true" who="#mysh" toWhom="#gavr">Ganya, why are you so pale??</said>

This way of encoding is much more intricate, which makes our speech networks more accurate.

Lastly, much previous encoding of speech in group scenes included only main characters, or, at best, only characters that spoke, whether major or minor. For example, in The Adolescent in the scene where Arkady goes to Dergachev’s and unsuccessfully reveals his “idea,” our previous encoding left out three characters who were physically present in the group but never spoke: the sister of Dergachev’s wife; a relative of Dergachev’s wife; and a man from the people. For a complete network analysis, these characters should be represented as addressees of speech in that group setting.

Putting all of this together, the work we’ve been doing in the last few months has made our project more organized, easier to manage, and more accurate. And with the work we’ve done with our schema, we’ve laid a good foundation for producing documentation of our project for external interest.

Some literary musings

Tracking speech in the above way opened up an interesting facet of Dostoevsky’s texts for me that I had never noticed before: the small, seemingly insignificant movements of characters in group scenes. Why does Lebedev’s daughter Vera leave the conversation on the veranda to work in the kitchen? Or rather, what is the function of that movement? Why did Dostoevsky include that? It seems completely irrelevant to the rest of the scene. Or, in the scene in The Adolescent I mentioned above, why narrate in one tiny sentence that Dergachev’s wife had to leave the conversation to care for her baby? In The Brothers Karamazov characters pop in and out of the Mokroye spree and Mitya’s interrogations there almost every other paragraph. Perhaps more interestingly, while the removal of these characters from the scenes is usually explicitly pointed out by the narrator, their reappearances generally aren’t. Dergachev’s wife, after attending the baby, materializes in the doorway at a certain point in the conversation. We only know this because she suddenly speaks, not because the narrator said she returned. The same happens with Vera, and many of the characters in Mokroye. Perhaps the unannounced returns of characters who suddenly speak as if they’ve been they’re all along just eavesdropping is one small way Dostoevsky creates feelings of discomfort or madness or chaos?

Another thing that has acually been an interest of mine for a while now, but which was brought back into view while working on re-encoding speech: all the different kinds of quotations in Dostoevsky’s novels. In 2024 I delivered a conference paper that detailed how, using text encoding as an aid, I identified and catalogued the appearances of 11 distinct types of quoted speech in the first 70 pages or so of The Brothers Karamazov. I got distracted after that presentation by the small matter of completing my dissertation, but now that I’m working on Computational Dostoevsky full time, and as I’ve been revisiting speech intensely over the past few months, I’ve encountered many more types of quotation throughout all Dostoevsky’s major novels. Quotation has long been an object of study in Dostoevsky scholarship, of course, but the kind of data cataloguing that text encoding is capable of opens up the possibility of exploring quotation much more thoroughly. Perhaps this can be a project for the future.

That’s all for now! Stay tuned for more updates.