About

Credits

OCS was developed by Peter Organisciak, sharing the outputs of work performed with Denis Dumas and Selcuk Acar. OCS was first developed with a seed grant from the University of Denver. Child-specific scoring models and large-language model scoring was supported by IES #R305A200199 as part of the MOTES project (Acar, Dumas, and Organisciak).

The OCS programmatic API was made possible by support from Pier Luc de Chantal. Thank you to a number of scholars for finding bugs or suggesting improvements, including: Pier Luc de Chantal, Katalin Grajzel, and Milan Kovacevic.

Are you looking to cite OCS? References are on the bottom of this page or any of the scoring pages for the associated publication.

Do you have questions or bugs? Contact us! For debugging, files or snippets of files causing issues are especially useful.

We also have an API underlying the system. Read the API documentation. The API can allow you to use our scoring in your own systems - e.g. Qualtrics, or R, or SASS, etc.

How does it work?

Explanations of how the systems work are on the scoring pages. See the Ocsai (LLM Scoring) and Semantic Scoring pages for details.

References

Since the AI and semantic scoring modules are different, they have different citations associated with them. You can find the citations on their respective pages. If you want to cite the overall system, you can use the following:

Organisciak, P., Dumas, D., Acar, S., and de Chantal, P. L. (2025). Open Creativity Scoring [Computer software]. Denver, CO: University of Denver. https://openscoring.du.edu.

LLM Scoring

To cite the work which introduced automated scoring of divergent thinking with large language models:

Organisciak, P., Acar, S., Dumas, D., & Berthiaume, K. (2023). Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models. Thinking Skills and Creativity, 49, 101356. https://doi.org/10.1016/j.tsc.2023.101356

Semantic Scoring

To cite the psychometric work on which the semantic scoring system is directly based:

Dumas, D., Organisciak, P., & Doherty, M. (2021). Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods. Psychology of Aesthetics, Creativity, and the Arts, 15(4), 645–663. https://doi.org/10.1037/aca0000319

To cite the GLoVe 840B text-mining model that this semantic scoring system builds on:

Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543).