A Comparative Analysis of Variability in AI-Generated and Human-Written Text

The rapid integration of large language models (LLMs) into everyday communication has raised fundamental questions about the nature of machine-generated text and how it compares to human writing across measurable linguistic dimensions.[1] LLMs such as GPT, Claude, Gemini, Grok, and Perplexity are now capable of producing fluent, coherent responses to open-ended prompts, making them increasingly indistinguishable from human-authored text in surface-level reading.[2] However, fluency and coherence are not equivalent to variability — the degree to which responses differ from one another across a population of writers or generators. Understanding whether AI-generated text is systematically more uniform than human-written text has practical implications for fields ranging from automated content detection to educational assessment and computational linguistics.[3]

Loneliness in modern urban environments has been identified as a significant and growing public health concern, affecting cognitive function, physical health, and social behaviour. [4] Holt-Lunstad, J. et al. (2015). Loneliness and social isolation as risk factors for mortality. Perspectives on Psychological Science. As AI systems are increasingly deployed in contexts intended to address social and emotional needs — including companionship applications, mental health tools, and community platforms — it becomes important to understand not only what these systems say, but how consistently and uniformly they say it relative to the diversity of human expression.[5] If AI-generated responses converge toward statistically narrow linguistic patterns, this has implications for the authenticity and representational breadth of AI communication in such contexts.

Linguistic variability in written text can be operationalised through a range of quantitative metrics. Word count and sentence count measure structural length properties, while average sentence length captures syntactic complexity.[6] Type-token ratio (TTR), defined as the ratio of unique word types to total word tokens, provides a measure of lexical diversity within a text.[7] These metrics, drawn from computational linguistics, allow for objective comparison of textual properties without reliance on subjective judgments of quality or creativity. Prior research has applied such metrics to distinguish authorship, assess writing development, and characterise genre differences, though their systematic application to human-versus-AI comparative variability analysis remains limited.[8]

The present study applies this quantitative framework to a controlled comparative design. Human-written and AI-generated responses to an identical open-ended prompt were collected and analysed across five linguistic metrics. We hypothesized that AI-generated responses would exhibit significantly lower distributional spread — that is, lower variance — across structural and lexical metrics compared to human-written responses. Results supported this hypothesis across all primary metrics, with variance ratios ranging from 17.26 to 52.46, indicating that human responses occupied a substantially wider range of the linguistic metric space. These findings suggest that measurable distributional compression may be a consistent property of AI-generated text under controlled prompt conditions, with implications for how such text is evaluated, detected, and deployed.

To characterise baseline differences between groups, descriptive statistics were computed for three primary linguistic metrics across 172 human-written and 150 AI-generated responses. Human-written responses exhibited a mean word count of 51.52 words (SD = 47.72), a mean average sentence length of 21.90 words (SD = 14.77), and a mean type-token ratio of 0.8398 (SD = 0.1175). AI-generated responses produced a mean word count of 77.40 words (SD = 6.59), a mean average sentence length of 19.63 words (SD = 3.27), and a mean type-token ratio of 0.8945 (SD = 0.0283). The human word count distribution exhibited a coefficient of variation of approximately 92%, suggesting considerable skew. AI responses were tightly clustered around their respective means across all metrics. Mean and standard deviation values for both groups are presented in Figure 2.

Variance ratios indicated substantially greater distributional spread in human-written responses across all three metrics. Word count exhibited the largest difference (VR = 52.46), followed by average sentence length (VR = 20.05) and type-token ratio (VR = 17.26), indicating that human responses were approximately 52, 20, and 17 times more variable than AI responses respectively (Figure 1). F-tests confirmed statistically significant variance differences for all three metrics: word count, F(171, 149) = 52.47, p < .001; average sentence length, F(171, 149) = 20.39, p < .001; and type-token ratio, F(171, 149) = 17.24, p < .001.

Both Cohen's d and Glass's delta were computed for each metric. Glass's delta — which uses the human group SD as denominator — was treated as the preferred measure given the substantial inequality in group standard deviations. For word count, Cohen's d = −0.76 and Glass's Δ = +0.54 (medium to medium-large). For average sentence length, Cohen's d = +0.21 and Glass's Δ = −0.15 (small). For type-token ratio, Cohen's d = −0.64 and Glass's Δ = +0.47 (medium). Effect sizes are presented in Figure 3.

The present study investigated whether AI-generated responses exhibit lower linguistic variability than human-written responses when both groups respond to an identical open-ended prompt. Across all three primary metrics examined — word count, average sentence length, and type-token ratio — human-written responses demonstrated substantially greater distributional spread than AI-generated responses, with variance ratios ranging from 17.26 to 52.46. These findings are consistent with the hypothesis that AI-generated text occupies a narrower region of the linguistic metric space under controlled prompt conditions, and suggest that distributional compression may be a measurable and quantifiable property of large language model output.

The most striking result was the variance ratio for word count (VR = 52.46), indicating that human responses were approximately 52 times more variable in length than AI-generated responses. Human word counts ranged from 3 to 229 words, reflecting the full spectrum of engagement. AI responses ranged from 61 to 83 words, clustering tightly around a narrow band regardless of which model produced them. This pattern is consistent with the architectural properties of large language models, which are trained to produce contextually appropriate, well-formed outputs and may therefore converge toward modal response lengths for a given prompt type.[1] It is important to note that this convergence does not indicate that AI responses are of lower quality — only that they are more uniform in length.

The variance ratio for average sentence length (VR = 20.05) similarly indicated tighter structural consistency in AI-generated responses. Human sentence lengths ranged from 3 to 61 words per sentence on average, while AI responses maintained average sentence lengths within a narrow band of 15 to 28 words across all five models — a convergence that warrants further investigation in future work with larger, more systematically controlled model samples.

The type-token ratio results introduced an important nuance. AI responses exhibited a higher mean TTR (M = 0.8945) than human responses (M = 0.8398), despite being longer on average. TTR is known to be sensitive to text length, and without length-normalised alternatives such as MATTR or MTLD, direct TTR comparisons between groups of differing mean length remain potentially confounded.[2] What the TTR variance ratio (VR = 17.26) does indicate reliably is that AI responses were considerably more consistent in their lexical diversity scores than human responses — human TTR ranged from 0.60 to 1.00, while AI TTR was constrained to 0.85 to 0.94.

Glass's delta indicated medium-sized mean differences for word count (Δ = +0.54) and type-token ratio (Δ = +0.47), while average sentence length showed a negligible mean difference (Δ = −0.15). The central tendency of sentence length was similar between groups, but the variance around that central tendency was not — a distinction that mean-focused analyses alone would have missed entirely.

Several factors may have influenced these results. The human response pool was collected via a self-distributed Google Form, introducing selection bias. AI responses were generated across five models without systematic control for temperature parameters — some portion of the observed variance differences may reflect parameter differences rather than fundamental properties of AI-generated text. All findings are conditional on a single open-ended prompt and should not be generalised beyond the prompt conditions studied here.

Future work should employ larger and more diverse model samples, apply length-normalised lexical diversity measures, preserve raw response-level data to enable Levene's test and non-parametric inferential testing, and extend the prompt set to include multiple domains. Semantic-level analysis — examining whether AI responses also converge in meaning and ideational content — would represent a meaningful extension of this framework.

In summary, this study demonstrated that AI-generated responses exhibited measurably lower linguistic variability than human-written responses across multiple quantitative metrics. Variance ratios of 52.46, 20.05, and 17.26 indicate that this difference is not marginal. These findings do not support claims about AI creativity or quality, but they do suggest that distributional uniformity is a quantifiable and potentially consistent characteristic of large language model output.

Human responses were collected via a Google Form distributed over a two-week period through social media platforms and peer networks. The form presented respondents with a single open-ended prompt — "Propose a unique solution to loneliness in modern cities" — and provided an unrestricted text field for responses. No length or formatting guidelines were provided. A total of 172 usable human responses were retained following preprocessing. All responses were in English.

AI-generated responses were produced using five large language models accessed through their standard web interfaces: ChatGPT (OpenAI), Claude (Anthropic), Gemini (Google), Grok (xAI), and Perplexity. All models were queried under default generation settings without modification of temperature or other parameters. To minimise the influence of personalisation, no accounts were logged into during response generation. Each model generated 30 responses to the identical prompt, yielding 150 AI-generated responses in total.

All responses underwent a standardised cleaning procedure prior to analysis. Human responses were excluded if they contained inappropriate, sexual, or evidently non-serious content; constituted verbatim or near-verbatim duplicates; or failed to address the prompt in any meaningful way. AI responses were similarly reviewed for prompt alignment and duplication. Following preprocessing, the final dataset comprised 172 human responses and 150 AI-generated responses.

Five linguistic metrics were computed for each response: word count, unique word count, type-token ratio (TTR), sentence count, and average sentence length. Prior to computation, all responses were normalised by converting text to lowercase and removing punctuation. TTR was calculated as the ratio of unique word count to total word count. Average sentence length was computed by dividing total word count by sentence count. Metrics were calculated using Microsoft Excel and Google Colab. To verify computational accuracy, a random sample of 20 responses was manually evaluated and cross-checked against computed values.

This study presents a descriptive quantitative analysis employing variance ratios, F-tests for equality of variances, and standardised effect sizes as primary quantitative evidence. Variance ratios were computed as σ²_human ÷ σ²_AI for each metric. F-tests were conducted using group-level summary statistics with degrees of freedom df₁ = 171 and df₂ = 149. Both Cohen's d and Glass's delta were computed as effect size measures; Glass's delta was treated as the preferred measure given substantial inequality in group standard deviations. Inferential testing at the response level could not be conducted due to the absence of preserved raw response-level data; findings are therefore interpreted as preliminary and hypothesis-generating. Figures were produced in Google Colab using the matplotlib and NumPy libraries.

Paste any text below to compute the same linguistic metrics used in this study. Your results are compared against the human and AI group statistics from the paper.

The following Python code was used to generate Figures 1, 2, and 3 in Google Colab.