5,117 public AI conversations, analyzed
Among 5,117 public AI chats, six in ten show the model running tools, and 20 contain suspected credentials pasted by a person.
The numbers
5,117
public conversations read
A convenience sample of chats people chose to publish
61.0%
show the model running tools
3,120 of 5,117
54,143
tool runs in all
A shell led, with 14,963 runs
Show all figures
20
contain suspected credentials pasted by a person
20 of 5,117, 0.39%. Detected by pattern; not tested
77%
of the chats with tools started them before message three
2,398 of 3,120
2,536
messages in the longest chat
Over 44 days
Every public chat, one square each
- 3,120 show the model running tools 61.0%
- 20 contain suspected credentials pasted by a person 0.39%
Positions do not identify conversations. Tool use and suspected exposures are not mutually exclusive.
From links found to chats read
- 7,272 share links found
- 5,117 still worked and were read
- 2,155 links did not open
People share their AI chats in public: to show a colleague, to attach to a bug report, to say “look what I built.” I wanted to know what is in those chats: what people hand to the AI once it does real work.
I publish aggregate counts, with no names, links to individual chats or credential strings.
What I found
- Agents are ordinary in these chats. 61.0% of the conversations (3,120 of 5,117) contain the model running tools, 54,143 runs in all. The most used tool is a shell, with 14,963 runs.
- Tool use begins early. The median first tool call is in message 1, and in 77% of the conversations where the model ran tools (2,398 of 3,120) it started before message three.
- About 1 in 256 chats (20 of 5,117, or 0.39%) contains a suspected credential pasted by a person. The strings matched credential patterns, but I did not test whether they worked. This is the observed rate in this collected sample; selection into public sharing does not establish a lower bound for all AI use.
- The first scanner pass found 72 matches, including 48 placeholders in the model’s own text. The final result counts 20 distinct conversations. The record does not yet explain which matches were excluded or merged into the same conversation. Until that is documented, treat the count of 20 as provisional.
- The platform strips what the shell ran from a shared chat, but it leaves the chat text as it is. That text is where the suspected credentials appear. My reading is that hiding actions after the fact protects the transcript but not the system. I did not measure that.
- Most conversations go past one question and answer. 63% of the conversations have more than two messages, 308 reach 50 messages or more (294 pass 50), and the longest has 2,536 messages over 44 days.
- The sample is multilingual. 4,190 of the 5,117 are in English, and 20 languages show up in total.
How it was made
I collected conversations that people had already shared through public links. The links came from public places such as code search, forum posts and web indexes. Scripts read the text of every live one and counted the patterns. The corpus was captured on 26 July 2026. Of 7,272 share links found, 5,117 still worked, leaving 2,155 unavailable links, or 29.6%. The counts alone do not establish why those links were unavailable.
The scanner counted strings matching credential patterns. None were tested against a service. This is a convenience sample of chats people chose to publish, so every rate means “among the collected public shares,” not among all AI use. The AI helped me collect, parse and draft. I set the question.
What can be checked
The saved scripts and outputs cover several collection snapshots. The exact set of conversations behind the 5,117-chat report has not been reconciled with those files. These are the reported counts, not the result of a fresh analysis of the transcripts.
The aggregate counts and Python check let you recalculate the percentages. The download notes explain what each count means and what remains unresolved. They contain no chat text or identifiers. Checking this arithmetic does not reproduce the collection or validate the scanner.
Sources
- My own collection of public chats. I publish no links to individual conversations.
- The question
- As people hand AI real work, what are they actually handing it?
- How it was made
- My own field study of 5,117 public Claude conversations, collected and analyzed with AI help.