LLM pipelines · Language analysis

Fair Tales — LLM language analysis of children's books

Using GPT-4o for complex language analysis across a hand-built corpus of ~200 Early Years storybooks — character extraction, speaker attribution and gendered-language analysis, engineered with task decomposition, in-context learning and unit testing.

A Fair Tales reading session: a woman reads a monster picture-book aloud to young children in a bright classroom.

The question

Who gets to be the hero in the books we read to our youngest children? Created by Tamzin Kitby and funded by the University of Bristol's Jean Golding Institute (Seedcorn and Follow-on awards), Fair Tales turns that question about fairness on the bookshelf into measurable evidence: a hand-built corpus of around 200 Early Years storybooks, analysed for who appears, who leads and how the language treats them. The headline finding: male characters outnumber female more than two to one — and are far more likely to lead the story than watch from the sidelines.

The LLM methodology

The language analysis uses GPT-4o for complex NLP across the corpus: character extraction, speaker attribution, gendered-language analysis and a picture-book take on the Bechdel test. The interesting part is not that an LLM was used — it's how.

The analysis was decomposed into subtasks, each with its own prompt, data schema and test cases. Prompts were developed iteratively with in-context learning: when the model failed on an edge case — speech split by a non-speech clause, say — a worked example of exactly that case was added to the prompt, fixing the behaviour on subsequent texts.

Every subtask was unit-tested during prompt development, and outputs were manually validated against data sampled from the corpus. It is software-engineering discipline applied to LLM pipelines — the difference between a demo and a method you can publish against.

Beyond the analysis

The methodology is written up as a chapter in the Edward Elgar Handbook of Mixed Methods Research in Education — 'AI and cloud data collection as tools for mixed methods research: a case study on gender in young children's story books' — with the methods code public on GitHub.

Around the research sits public engagement: a Streamlit data-entry app lets anyone photograph, transcribe and tag a book into the dataset, and partnerships with independent Bristol bookshops turned the findings into counter-stereotypical reading lists.