# 5,117 public AI conversations, analyzed

People share their AI chats in public: to show a colleague, to attach to a bug report, to say "look what I built." I wanted to know what is in those chats: what people hand to the AI once it does real work.

I publish aggregate counts, with no names, links to individual chats or credential strings.

## What I found
- Agents are ordinary in these chats. 61.0% of the conversations (3,120 of 5,117) contain the model running tools, 54,143 runs in all. The most used tool is a shell, with 14,963 runs.
- Tool use begins early. The median first tool call is in message 1, and in 77% of the conversations where the model ran tools (2,398 of 3,120) it started before message three.
- About 1 in 256 chats (20 of 5,117, or 0.39%) contains a suspected credential pasted by a person. The strings matched credential patterns, but I did not test whether they worked. This is the observed rate in this collected sample; selection into public sharing does not establish a lower bound for all AI use.
- The first scanner pass found 72 matches, including 48 placeholders in the model's own text. The final result counts 20 distinct conversations. The record does not yet explain which matches were excluded or merged into the same conversation. Until that is documented, treat the count of 20 as provisional.
- The platform strips what the shell ran from a shared chat, but it leaves the chat text as it is. That text is where the suspected credentials appear. My reading is that hiding actions after the fact protects the transcript but not the system. I did not measure that.
- Most conversations go past one question and answer. 63% of the conversations have more than two messages, 308 reach 50 messages or more (294 pass 50), and the longest has 2,536 messages over 44 days.
- The sample is multilingual. 4,190 of the 5,117 are in English, and 20 languages show up in total.

## How it was made
I collected conversations that people had already shared through public links. The links came from public places such as code search, forum posts and web indexes. Scripts read the text of every live one and counted the patterns. The corpus was captured on 26 July 2026. Of 7,272 share links found, 5,117 still worked, leaving 2,155 unavailable links, or 29.6%. The counts alone do not establish why those links were unavailable.

The scanner counted strings matching credential patterns. None were tested against a service. This is a convenience sample of chats people chose to publish, so every rate means "among the collected public shares," not among all AI use. The AI helped me collect, parse and draft. I set the question.

## What can be checked
The saved scripts and outputs cover several collection snapshots. The exact set of conversations behind the 5,117-chat report has not been reconciled with those files. These are the reported counts, not the result of a fresh analysis of the transcripts.

The [aggregate counts](https://peycheff.com/research/public-ai-chats/aggregates.json) and [Python check](https://peycheff.com/research/public-ai-chats/check.py) let you recalculate the percentages. The [download notes](https://peycheff.com/research/public-ai-chats/README.md) explain what each count means and what remains unresolved. They contain no chat text or identifiers. Checking this arithmetic does not reproduce the collection or validate the scanner.

## Sources
- My own collection of public chats. I publish no links to individual conversations.
