Semantic Scoring

Semantic scoring measures originality as the distance between a prompt and response in word-embedding space. It’s a well-validated approach for divergent thinking research, though for most use cases we now recommend Ocsai LLM-based scoring, which achieves higher agreement with human raters (r = .81).

Semantic scoring is hosted on Hugging Face Spaces. First load may take a moment while the Space wakes up.

Cite

Dumas, D., Organisciak, P., & Doherty, M. (2021). Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods. Psychology of Aesthetics, Creativity, and the Arts, 15(4), 645–663. https://doi.org/10.1037/aca0000319