How many samples does a voice profile need before it stabilises?
SGR-003, conducted 2026-09-20.
Question: As pieces are added to a voice profile, when do its attributes stop moving, and when does it start telling its author from a stranger?
Sample
Great Expectations (Gutenberg #1400), Pride and Prejudice (#1342) and Emerson's Essays (#16643), body text only, cut at paragraph boundaries into pieces of about 800 words (600 to 1,400 accepted). With a fixed seed (20260920), 15 training pieces and 4 held-out pieces were drawn per author; the training sets are nested, so the 1-piece profile's sample is inside the 2-piece profile's, and so on. Public-domain prose was used so the study is reproducible from the source text and needs nobody's consent.
- Body text between the Gutenberg start and end markers; chapter headings, illustrations and paragraphs under 12 words dropped
- Held-out pieces never appear in any training set, for any author
- One extraction per profile, one score per held-out piece; nothing re-run or chosen
Method
Profiles: POST /v1/profiles with the first k training pieces (k = 1, 2, 3, 5, 8, 15), status polled to complete. Extraction is the same two-pass pipeline every user gets; extractions are rate-limited to five an hour, so the run spans several hours.
Same-author score: POST /v1/voice-match for each of the author's four held-out pieces against the k-piece profile; the mean is reported.
Other-author score: the same call for the other two authors' eight held-out pieces; the mean is reported. The gap (same minus other) is how well the profile tells its author from a stranger.
Attribute agreement: each k-piece profile's enumerated attributes (complexity, clause ordering, humour, expressiveness, opening style, argument structure, ellipsis habit, list preference, connective preference, specificity, rhythm, paragraph length, power-sentence position) compared with the author's 15-piece profile; the share that agree is reported. Numeric attributes are reported as the absolute difference from the 15-piece value.
Confidence: the confidence_score the extraction itself returns with each profile.
Runner: scripts/experiments/run-stability-study.mjs in the ScriptGrain repository; data file linked below.
Variables measured
- Voice match of held-out pieces by the same author (0 to 1, profile-calibrated)
- Voice match of held-out pieces by the other two authors (0 to 1)
- Discrimination gap (same minus other)
- Share of enumerated attributes agreeing with the 15-piece profile
- Absolute drift of numeric attributes from the 15-piece profile
- Extraction confidence score (0 to 1)
- Words in the training set
Findings
- The score barely cares how many pieces built the profile (0.86 at 1 piece, 0.86 at 3, 0.86 at 15): Mean voice match of each author's four held-out pieces against their own profile, averaged over three authors, did not move as pieces were added: 0.86 at one piece, 0.86 at three, 0.86 at fifteen. A single 800-word piece already pins the counted habits (sentence length, contractions, commas, pronouns) closely enough for the scorer to read the author's other work as on voice.
- What more pieces buy is a profile that stops moving (sentence length 4.9 words off at 1 piece, 1.2 at 3; contractions 3.6 per 1,000 off, then 0.8): Measured against each author's fifteen-piece profile, the one-piece profile's average sentence length was off by 4.9 words, contractions by 3.6 per 1,000 and formality by 0.9 points on the 0 to 10 scale. At three pieces those errors were 1.2 words, 0.8 per 1,000 and 0.3; from five pieces on, under two words, under one contraction per 1,000 and about 0.1 on formality. The product's recommendation of three or more pieces and about 3,000 words is where the attributes settle, and it holds on this corpus.
- The extraction's own confidence tracks sample size (0.67 at 1 piece, 0.79 at 3, 0.87 at 15): The confidence_score returned with each profile rose with the pieces behind it: 0.67 on average at one piece, 0.78 at two, 0.79 at three, 0.80 at five, 0.86 at eight, 0.87 at fifteen. It is a fair guide to how much a profile should be trusted, and it is the number the API returns for free.
- Telling one author from another is where the method is weakest here (gap 0.06 to 0.10; Dickens 0.17, Emerson 0.01): The gap between an author's own held-out pieces and the other two authors' averaged 0.06 to 0.10 at every sample count, with no trend. Dickens separated best (0.17 from five pieces), Emerson hardly at all (0.01 at fifteen: his profile scored Dickens and Austen as high as it scored him). All three are formal, dry, flowing nineteenth-century voices at formality 7 to 9 with almost no contractions, so the counted habits the score leans on are shared; the scorer measures distance on those habits and reads similar registers as similar. A modern conversational writer scored against any of these profiles would sit far lower.
- Labelled attributes are the least stable part of a profile (72% to 87% agreement with the 15-piece profile): The share of thirteen enumerated attributes (humour register, argument structure, rhythm, clause ordering and the rest) that matched the fifteen-piece profile's labels ran 82% at one piece, 72% at two, 82% at three, 74% at five and 87% at eight, with Emerson as low as 46% at two pieces. These labels are a model's reading of the samples and flip between extractions more than the counted numbers move; eight pieces or more is where they settle, and the scorer already gives adjacent labels partial credit for this reason.
Results by pieces in the profile (means over three authors)
| Pieces | Words (mean) | Same-author match | Other-author match | Gap | Confidence | Attributes agreeing with 15-piece profile |
|---|---|---|---|---|---|---|
| 1 | 863 | 0.86 | 0.76 | 0.10 | 0.67 | 82% |
| 2 | 1718 | 0.86 | 0.80 | 0.06 | 0.78 | 72% |
| 3 | 2589 | 0.86 | 0.78 | 0.08 | 0.79 | 82% |
| 5 | 4317 | 0.86 | 0.75 | 0.10 | 0.80 | 74% |
| 8 | 6986 | 0.86 | 0.77 | 0.09 | 0.86 | 87% |
| 15 | 13089 | 0.86 | 0.78 | 0.08 | 0.87 | reference |
Results by author
| Author | Pieces | Words | Same-author match | Other-author match | Gap | Confidence | Attributes agreeing |
|---|---|---|---|---|---|---|---|
| Dickens | 1 | 819 | 0.84 | 0.79 | 0.05 | 0.60 | 85% |
| Dickens | 2 | 1632 | 0.85 | 0.79 | 0.07 | 0.78 | 85% |
| Dickens | 3 | 2450 | 0.86 | 0.77 | 0.10 | 0.78 | 85% |
| Dickens | 5 | 4154 | 0.86 | 0.70 | 0.17 | 0.72 | 69% |
| Dickens | 8 | 6644 | 0.88 | 0.72 | 0.16 | 0.88 | 92% |
| Dickens | 15 | 12661 | 0.89 | 0.71 | 0.17 | 0.90 | reference |
| Austen | 1 | 864 | 0.90 | 0.75 | 0.15 | 0.70 | 85% |
| Austen | 2 | 1804 | 0.88 | 0.81 | 0.07 | 0.82 | 85% |
| Austen | 3 | 2782 | 0.89 | 0.80 | 0.09 | 0.80 | 100% |
| Austen | 5 | 4478 | 0.87 | 0.81 | 0.06 | 0.86 | 92% |
| Austen | 8 | 7269 | 0.88 | 0.80 | 0.08 | 0.88 | 92% |
| Austen | 15 | 13046 | 0.87 | 0.81 | 0.06 | 0.88 | reference |
| Emerson | 1 | 906 | 0.84 | 0.74 | 0.09 | 0.70 | 77% |
| Emerson | 2 | 1717 | 0.84 | 0.79 | 0.05 | 0.74 | 46% |
| Emerson | 3 | 2535 | 0.84 | 0.77 | 0.07 | 0.78 | 62% |
| Emerson | 5 | 4320 | 0.84 | 0.75 | 0.09 | 0.82 | 62% |
| Emerson | 8 | 7045 | 0.83 | 0.78 | 0.04 | 0.82 | 77% |
| Emerson | 15 | 13560 | 0.83 | 0.83 | 0.01 | 0.83 | reference |
How far the numeric attributes sit from the 15-piece profile (mean absolute difference)
| Attribute | 1 piece | 2 | 3 | 5 | 8 |
|---|---|---|---|---|---|
| Average sentence length (words) | 4.90 | 1.95 | 1.17 | 1.55 | 1.93 |
| Contractions per 1,000 | 3.60 | 1.80 | 0.82 | 0.28 | 0.62 |
| Formality (0 to 10) | 0.93 | 0.43 | 0.27 | 0.07 | 0.12 |
| Commas per sentence | 0.50 | 0.70 | 0.23 | 0.43 | 0.22 |
What the numbers say
The question was how many pieces a profile needs before it stabilises, and the answer splits in two. The score stabilises immediately: one piece is enough for the author's other writing to read as on voice, because the features the scorer counts are already present in 800 words. The profile itself takes longer. Its numbers move by several words of sentence length and several contractions per thousand between one piece and three, then settle; its labels keep flipping until about eight. Three pieces and about 3,000 words, the product's standing recommendation, is where a profile stops being a reading of one piece and becomes a reading of a writer.
The uncomfortable number is the gap. On three literary voices that share a register, the profile tells its author from the others by 0.06 to 0.10 on average, and Emerson's not at all. That is a property of the corpus and of what the score measures: distance on habits, and these three share the habits. It is also a limit to state plainly: a voice-match score says how far a piece sits from a profile, and two writers who sit close to each other will score close to each other's profiles. The discrimination a real user sees is between their own conversational newsletter and a model's default register, which is a much larger distance than Dickens to Austen.
Three authors, one draw of pieces, one extraction each: read the curves, not the decimals. Every profile, every score and the piece hashes are in the data file, and the runner and the source texts are public, so the whole study can be re-run.
Limitations
- Three authors, one draw of pieces each, one extraction per profile: the direction of the curves is the finding, not the decimals. A second draw would move individual cells.
- All three authors are nineteenth-century literary prose, which the scorer separates less sharply than it separates a newsletter from a legal notice; the other-author baseline is therefore a hard test, and a modern business writer's profile would likely show a larger gap.
- The judged attributes come from a model reading the samples, so some run-to-run variation in labels is inherent and is part of what this study measures.
- Pieces of about 800 words are longer than a typical LinkedIn post and shorter than a typical blog post; the piece count is the variable studied, and the word count scales with it.
- The author of this study builds the product. Nothing was blind; nothing was re-run or selected.
Reproducing this
- Data file: /research/sgr-003-profile-stability.json (every profile's attributes, every score, the piece hashes and word counts, the seed).
- Runner: scripts/experiments/run-stability-study.mjs (public repository); the texts are Project Gutenberg #1400, #1342 and #16643.
- Free instruments: the off-voice check on this site scores any two pieces the same way.