How to find patterns in millions of words?

I’ve asked Claude to read my entire archive: hundreds of thousands of words in the posts and comments on social media. And 1.2 million words published on my website, or approximately 15 novels.

I can’t possibly read it all myself.

What I’m looking for is not the frequency of any given word, because that yields words like: well, even, asked, kind, best, love, different.

What I want are words that have high use at specific periods, indicating some kind of focus.


From social posts

Word When Posts
residence 2018, 23, 24 16 posts
pilgrimage 2020-22 19 posts
postcards 2020-23 13 posts
psalms 2022-25 19 posts
magnolia 2023-25 22 posts
burgh 2025-26 40 posts
multitudes 2026 28 posts

Claude is going to build me a verbal mind map, with a branching structure. Branches get labelled in my own words, lifted from the posts.

The map will be built as an OPML file to save in an app called Zavala. Over time, I’ll be able to build new more specific maps that link from one to another.

A few findings already: a large chunk of the writing on my website is archived newspaper and magazine stories – things I was sent to write about, may not have chosen to write about. Between them these account for nearly half a million words.

Forty-seven per cent of the articles on my website have no keywords, and 29 per cent belong to no category, so the nature of what they contain is a mystery.

***

I know this is faulty, because there has been no mention at all of two recent obsessions: speccy and micro-memoir. But I want to keep posting about the process of analysing the archive while it’s happening. Will follow up about those later.

Meanwhile: a cluster I’d forgotten entirely. In 2015, within months of starting a documentary about the Rejection Game – four players on stage, one of whom must be socially rejected by the others, and it mustn’t be you – I wrote Why apologise, if they won’t?, What do other people think of you?, Did anybody ask for your opinion?, How to say ‘No, thank you’, Talking about people as if they weren’t there, and What if nobody joins in?

Every one of them uncategorised.

***