1887
  1. Home
  2. e-Journals
  3. The Mental Lexicon
  4. Volume 19, Issue 3
  5. Article
Volume 19, Issue 3
  • ISSN 1871-1340
  • E-ISSN: 1871-1375
EPUB
PDF

Abstract

Abstract

Measures of orthographic typicality have long been studied as predictors of lexical access. The best-known orthographic typicality measure is ( or ), the number of words that are one letter different, by substitution, from the target word. A more recent related measure of orthographic typicality is orthographic Levenshtein distance 20 (), the average Levenshtein orthographic edit distance of a target word from its 20 closest neighbours (Yarkoni, Balota, and Yap, 2008). Both measures have been implicated in lexical access. In this paper, we propose and assess a family of measures of word form similarity we call . These measures are based on Shannon entropy (Shannon, 1948), which has a long history of being considered psychologically relevant. Orthographic uncertainty measures are superior to ON and OLD20 at predicting lexical decision and naming reaction times and accuracies. They are also superior to the older measures insofar as they are naturally tied to the widely-accepted quantification using Shannon Entropy of the psychological functions of familiarity, uncertainty, learnability, and representational and computational efficiency.

© John Benjamins Publishing Company
Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/ml.24006.wes
2025-02-06
2026-04-17

  • From This Site

    /content/journals/10.1075/ml.24006.wes
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

/deliver/fulltext/ml.24006.wes.html?itemId=/content/journals/10.1075/ml.24006.wes&mimeType=html&fmt=ahah

References

  1. Akaike, H.
    (1973) Information theory and an extension of the maximum likelihood principle. InB. N. Petrov & F. Caski (Eds.), Proceedings of the Second International Symposium on Information Theory (pp.267–281). Budapest: Akademiai Kiado.
     [Google Scholar]
  2. (1974) A new look at the statistical model identification. IEEE Transactions on Automatic Control, 191, 716–723. 10.1109/TAC.1974.1100705
     https://doi.org/10.1109/TAC.1974.1100705 [Google Scholar]
  3. Andrews, S.
    (1989) Frequency and neighborhood effects on lexical access: Activation or search?Journal of Experimental Psychology: Learning, Memory, and Cognition, 151, 802–814.
     [Google Scholar]
  4. (1992) Frequency and neighborhood effects on lexical access: Lexical similarity or orthographic redundancy?Journal of Experimental Psychology: Learning, Memory, and Cognition, 181, 234–254.
     [Google Scholar]
  5. (1997) The effect of orthographic similarity on lexical retrieval: Resolving neighborhood conflicts. Psychonomic Bulletin & Review, 4(4), 439–461. 10.3758/BF03214334
     https://doi.org/10.3758/BF03214334 [Google Scholar]
  6. Assink, E. M., Kattenberg, G., & Wortmann, C.
    (1998) Exploring the boundaries of sublexical word identification units: The use of onsets and rimes and reading ability. Journal of psycholinguistic research, 271, 639–659. 10.1023/A:1023279921733
     https://doi.org/10.1023/A:1023279921733 [Google Scholar]
  7. Balota, D. A., Yap, M. J., Hutchison, K. A., Cortese, M. J., Kessler, B., Loftis, B., Neely, J., Nelson, D. L., Simpson, G. B., & Treiman, R.
    (2007) The English lexicon project. Behavior Research Methods, 39(3), 445–459. 10.3758/BF03193014
     https://doi.org/10.3758/BF03193014 [Google Scholar]
  8. Bateson, G.
    (1972) Steps to an ecology of mind. New York, NT: Ballantine.
     [Google Scholar]
  9. Bentz, C., Alikaniotis, D., Cysouw, M., & Ferrer-i-Cancho, R.
    (2017) The entropy of words — Learnability and expressivity across more than 1000 languages. Entropy, 19(6), 275. 10.3390/e19060275
     https://doi.org/10.3390/e19060275 [Google Scholar]
  10. Berlyne, D. E.
    (1971) Aesthetics and Psychobiology. New York: Appleton-Century-Crofts.
     [Google Scholar]
  11. Blais, C., Fiset, D., Arguin, M., Jolicoeur, P., Bub, D., & Gosselin, F.
    (2009) Reading between eye saccades. PLoS One, 4(7), e6448. 10.1371/journal.pone.0006448
     https://doi.org/10.1371/journal.pone.0006448 [Google Scholar]
  12. Blevins, J. P.
    (2013) The information-theoretic turn. Psihologija, 46(4), 355–375. 10.2298/PSI1304355B
     https://doi.org/10.2298/PSI1304355B [Google Scholar]
  13. Booth, J. R., & Perfetti, C. A.
    (2002) Onset and rime structure influences naming but not early word identification in children and adults. Scientific Studies of Reading, 6(1), 1–23. 10.1207/S1532799XSSR0601_01
     https://doi.org/10.1207/S1532799XSSR0601_01 [Google Scholar]
  14. Bowey, J. A.
    (1990) Orthographic onsets and rimes as functional units of reading. Memory & Cognition, 18(4), 419–427. 10.3758/BF03197130
     https://doi.org/10.3758/BF03197130 [Google Scholar]
  15. Carhart-Harris, R. L., Leech, R., Hellyer, P. J., Shanahan, M., Feilding, A., Tagliazucchi, E., Chialvo, D. R., & Nutt, D.
    (2014) The entropic brain: a theory of conscious states informed by neuroimaging research with psychedelic drugs. Frontiers in human neuroscience, 81, 20. 10.3389/fnhum.2014.00020
     https://doi.org/10.3389/fnhum.2014.00020 [Google Scholar]
  16. Carreiras, M., Perea, M., & Grainger, J.
    (1997) Effects of the orthographic neighborhood in visual word recognition: Cross-task comparisons. Journal of experimental psychology: learning, memory, and cognition, 23(4), 857.
     [Google Scholar]
  17. Chaitin, G. J.
    (1975) A theory of program size formally identical to information theory. Journal of the ACM, 22(3), 329–340. 10.1145/321892.321894
     https://doi.org/10.1145/321892.321894 [Google Scholar]
  18. Chen, Q., & Mirman, D.
    (2012) Competition and cooperation among similar representations: toward a unified account of facilitative and inhibitory effects of lexical neighbors. Psychological review, 119(2), 417. 10.1037/a0027175
     https://doi.org/10.1037/a0027175 [Google Scholar]
  19. Coltheart, M., Davelaar, E., Jonasson, J., & Besner, D.
    (1977) Access to the internal lexicon. InS. Dornic (Ed.), Attention and performance VI: Proceedings of the Sixth International Symposium on Attention and Performance, Stockholm, Sweden, July 28-August 1, 1975. Hillsdale, N.J: Lawrence Erlbaum.
     [Google Scholar]
  20. Dahan, D., Magnuson, J. S., & Tanenhaus, M. K.
    (2001) Time course of frequency effects in spoken-word recognition: Evidence from eye movements. Cognitive psychology, 42(4), 317–367. 10.1006/cogp.2001.0750
     https://doi.org/10.1006/cogp.2001.0750 [Google Scholar]
  21. Davis, C. J., Perea, M., & Acha, J.
    (2009) Re (de) fining the orthographic neighborhood: The role of addition and deletion neighbors in lexical decision and reading. Journal of Experimental Psychology: Human Perception and Performance, 35(5), 1550.
     [Google Scholar]
  22. del Prado Martın, F. M., Kostić, A., & Baayen, R. H.
    (2004) Putting the bits together: An information theoretical perspective on morphological processing. Cognition, 94(1), 1–18. 10.1016/j.cognition.2003.10.015
     https://doi.org/10.1016/j.cognition.2003.10.015 [Google Scholar]
  23. Duñabeitia, J. A., & Vidal-Abarca, E.
    (2008) Children like dense neighborhoods: Orthographic neighborhood density effects in novel readers. Spanish Journal of Psychology, 11(1), 26. 10.1017/S113874160000408X
     https://doi.org/10.1017/S113874160000408X [Google Scholar]
  24. Dye, M., Johns, B. T., Jones, M. N., & Ramscar, M.
    (2016) The structure of names in memory: Deviations from uniform entropy impair memory for linguistic sequences. InA. Papafragou, D. Grodner, D. Mirman, & J. C. Trueswell (Eds.), Proceedings of the 38th Annual Conference of the Cognitive Science Society (pp.1763–1768). Austin, TX: Cognitive Science Society.
     [Google Scholar]
  25. Dye, M., Milin, P., Futrell, R. & Ramscar, M.
    (2017) A functional theory of gender paradigms. In: Kiefer, F., Blevins, J. P. and Bartos, H., (eds.) Perspectives on Morphological Structure: Data and Analyses. Brill, Leiden, pp.212–239. 10.1163/9789004342934_011
     https://doi.org/10.1163/9789004342934_011 [Google Scholar]
  26. Gibson, E., Futrell, R., Piantadosi, S. P., Dautriche, I., Mahowald, K., Bergen, L., & Levy, R.
    (2019) How efficiency shapes human language. Trends in cognitive sciences, 23(5), 389–407. 10.1016/j.tics.2019.02.003
     https://doi.org/10.1016/j.tics.2019.02.003 [Google Scholar]
  27. Grainger, J., & Jacobs, A. M.
    (1993) Masked partial-word priming in visual word recognition: Effects of positional letter frequency. Journal of Experimental Psychology-Human Perception and Performance, 19(5), 951–964. 10.1037/0096‑1523.19.5.951
     https://doi.org/10.1037/0096-1523.19.5.951 [Google Scholar]
  28. (1996) Orthographic processing in visual word recognition: a multiple read-out model. Psychological Review, 103(3), 518. 10.1037/0033‑295X.103.3.518
     https://doi.org/10.1037/0033-295X.103.3.518 [Google Scholar]
  29. Hale, J.
    (2016) Information‐theoretical complexity metrics. Language and Linguistics Compass, 10(9), 397–412. 10.1111/lnc3.12196
     https://doi.org/10.1111/lnc3.12196 [Google Scholar]
  30. Harati, P., Westbury, C., & Kiaee, M.
    (2021) Evaluating the predication model of metaphor comprehension: Using word2vec to model best/worst quality judgments of 622 novel metaphors. Behavior Research Methods, 53(5), 2214–2225. 10.3758/s13428‑021‑01558‑w
     https://doi.org/10.3758/s13428-021-01558-w [Google Scholar]
  31. Hastie, T. J. & Tibshirani, R. J.
    (1990) Generalized Additive Models. Chapman & Hall/CRC.
     [Google Scholar]
  32. Heilbron, M., Armeni, K., Schoffelen, J. M., Hagoort, P., & De Lange, F. P.
    (2022) A hierarchy of linguistic predictions during natural language comprehension. Proceedings of the National Academy of Sciences, 119(32), e2201968119. 10.1073/pnas.2201968119
     https://doi.org/10.1073/pnas.2201968119 [Google Scholar]
  33. Hirsh, J. B., Mar, R. A., & Peterson, J. B.
    (2012) Psychological entropy: a framework for understanding uncertainty-related anxiety. Psychological review, 119(2), 304–320. 10.1037/a0026767
     https://doi.org/10.1037/a0026767 [Google Scholar]
  34. Hollis, G.
    (2018) Scoring best-worst data in unbalanced many-item designs, with applications to crowdsourcing semantic judgments. Behavior Research Methods, 50(2), 711–729. 10.3758/s13428‑017‑0898‑2
     https://doi.org/10.3758/s13428-017-0898-2 [Google Scholar]
  35. (2020) The role of number of items per trial in best–worst scaling experiments. Behavior Research Methods, 52(2), 694–722. 10.3758/s13428‑019‑01270‑w
     https://doi.org/10.3758/s13428-019-01270-w [Google Scholar]
  36. Hollis, G., & Westbury, C.
    (2006) NUANCE: Naturalistic University of Alberta nonlinear correlation explorer. Behavior Research Methods, 38(1), 8–23. 10.3758/BF03192745
     https://doi.org/10.3758/BF03192745 [Google Scholar]
  37. (2018) When is best-worst best? A comparison of best-worst scaling, numeric estimation, and rating scales for collection of semantic norms. Behavior Research Methods, 501, 115–133. 10.3758/s13428‑017‑1009‑0
     https://doi.org/10.3758/s13428-017-1009-0 [Google Scholar]
  38. Hollis, G., Westbury, C. F., & Peterson, J. B.
    (2006) NUANCE 3.0: Using genetic programming to model variable relationships. Behavior research methods, 38(2), 218–228. 10.3758/BF03192772
     https://doi.org/10.3758/BF03192772 [Google Scholar]
  39. Huntsman, L. A., & Lima, S. D.
    (1996) Orthographic neighborhood structure and lexical access. Journal of Psycholinguistic Research, 25(3), 417–429. 10.1007/BF01727000
     https://doi.org/10.1007/BF01727000 [Google Scholar]
  40. Keuleers, E.
    (2013) vwr r package (v. 3). Downloaded from: https://rdocumentation.org/packages/vwr/versions/.3.0
  41. Keuleers, E., Lacey, P., Rastle, K., & Brysbaert, M.
    (2012) The British Lexicon Project: lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 44(1), 287–304. 10.3758/s13428‑011‑0118‑4
     https://doi.org/10.3758/s13428-011-0118-4 [Google Scholar]
  42. Kiritchenko, S., & Mohammad, S. M.
    (2016) Capturing reliable fine-grained sentiment associations by crowdsourcing and best-worst scaling. San Diego: Paper presented atthe 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL). 10.18653/v1/N16‑1095
     https://doi.org/10.18653/v1/N16-1095 [Google Scholar]
  43. Levenshtein, V. I.
    (1966) Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 101, 707.
     [Google Scholar]
  44. Louviere, J. J., Flynn, T. N., & Marley, A. A. J.
    (2015) Best-worst scaling: Theory, methods and applications. Cambridge: Cambridge University Press. 10.1017/CBO9781107337855
     https://doi.org/10.1017/CBO9781107337855 [Google Scholar]
  45. Luce, R. D.
    (2003) Whatever happened to information theory in psychology?. Review of General Psychology, 7(2), 183–188. 10.1037/1089‑2680.7.2.183
     https://doi.org/10.1037/1089-2680.7.2.183 [Google Scholar]
  46. Luthra, S., You, H., Rueckl, J. G., & Magnuson, J. S.
    (2020) Friends in Low-Entropy Places: Orthographic Neighbor Effects on Visual Word Identification Differ Across Letter Positions. Cognitive Science, 44(12), e12917. 10.1111/cogs.12917
     https://doi.org/10.1111/cogs.12917 [Google Scholar]
  47. McClelland, J. L., & Rumelhart, D. E.
    (1981) An interactive activation model of context effects in letter perception: I. An account of basic findings. Psychological review, 88(5), 375. 10.1037/0033‑295X.88.5.375
     https://doi.org/10.1037/0033-295X.88.5.375 [Google Scholar]
  48. Milin, P., Kuperman, V., Kostic, A., & Baayen, R. H.
    (2009) Paradigms bit by bit: An information theoretic approach to the processing of paradigmatic structure in inflection and derivation. Analogy in grammar: Form and acquisition, 3811, 214–252. 10.1093/acprof:oso/9780199547548.003.0010
     https://doi.org/10.1093/acprof:oso/9780199547548.003.0010 [Google Scholar]
  49. Miller, R. R., Barnet, R. C., & Grahame, N. J.
    (1995) Assessment of the Rescorla-Wagner model. Psychological Bulletin, 117(3), 363. 10.1037/0033‑2909.117.3.363
     https://doi.org/10.1037/0033-2909.117.3.363 [Google Scholar]
  50. O’Regan, J. K., Lévy-Schoen, A., Pynte, J., & Brugaillière, B.
    (1984) Convenient fixation location within isolated words of different length and structure. Journal of Experimental Psychology: Human Perception and Performance, 10(3), 393.
     [Google Scholar]
  51. Peereman, R., & Content, A.
    (1997) Orthographic and phonological neighborhoods in naming: Not all neighbors are equally influential in orthographic space. Journal of Memory and language, 37(3), 382–410. 10.1006/jmla.1997.2516
     https://doi.org/10.1006/jmla.1997.2516 [Google Scholar]
  52. Perea, M., & Rosa, E.
    (2000) The effects of orthographic neighborhood in reading and laboratory word identification tasks: A review. Psicológica, 21(2), 327–340.
     [Google Scholar]
  53. Piantadosi, S. T., Tily, H., & Gibson, E.
    (2011) Word lengths are optimized for efficient communication. Proceedings of the National Academy of Sciences, 108(9), 3526–3529. 10.1073/pnas.1012551108
     https://doi.org/10.1073/pnas.1012551108 [Google Scholar]
  54. Pothos, E.
    (2010) An entropy model for artificial grammar learning. Frontiers in psychology, 11, 16. 10.3389/fpsyg.2010.00016
     https://doi.org/10.3389/fpsyg.2010.00016 [Google Scholar]
  55. Ramscar, M., Dye, M., & Klein, J.
    (2013) Children value informativity over logic in word learning. Psychological science, 24(6), 1017–1023. 10.1177/0956797612460691
     https://doi.org/10.1177/0956797612460691 [Google Scholar]
  56. Ramscar, M., & Port, R. F.
    (2016) How spoken languages work in the absence of an inventory of discrete units. Language Sciences, 531, 58–74. 10.1016/j.langsci.2015.08.002
     https://doi.org/10.1016/j.langsci.2015.08.002 [Google Scholar]
  57. Rayner, K., & Kaiser, J. S.
    (1975) Reading mutilated text. Journal of Educational Psychology, 671, 301–306. 10.1037/h0077015
     https://doi.org/10.1037/h0077015 [Google Scholar]
  58. Rayner, K., White, S. J., Johnson, R. L., & Liversedge, S. P.
    (2006) Raeding wrods with jubmled lettres: There is a cost. Psychological Science, 171, 192–193. 10.1111/j.1467‑9280.2006.01684.x
     https://doi.org/10.1111/j.1467-9280.2006.01684.x [Google Scholar]
  59. Rescorla, R.
    (1988) Pavlovian conditioning: it’s not what you think it is. American Psychologist, 431, 151–160. 10.1037/0003‑066X.43.3.151
     https://doi.org/10.1037/0003-066X.43.3.151 [Google Scholar]
  60. Rumelhart, D. E., & McClelland, J. L.
    (1982) An interactive activation model of context effects in letter perception: II. The contextual enhancement effect and some tests and extensions of the model. Psychological review, 89(1), 60. 10.1037/0033‑295X.89.1.60
     https://doi.org/10.1037/0033-295X.89.1.60 [Google Scholar]
  61. Shannon, C. E.
    (1948) A note on the concept of entropy. Bell System Technical Journal, 271, 379–423. 10.1002/j.1538‑7305.1948.tb01338.x
     https://doi.org/10.1002/j.1538-7305.1948.tb01338.x [Google Scholar]
  62. Sears, C. R., Hino, Y., & Lupker, S. J.
    (1995) Neighborhood size and neighborhood frequency effects in word recognition. Journal of Experimental Psychology: Human Perception and Performance, 21(4), 876.
     [Google Scholar]
  63. Shannon, C. E.
    (1948) A Mathematical Theory of Communication. Bell System Technical Journal, 271, 379–423 & 623–656. 10.1002/j.1538‑7305.1948.tb01338.x
     https://doi.org/10.1002/j.1538-7305.1948.tb01338.x [Google Scholar]
  64. (1956) The bandwagon. IRE Transactions on Information Theory, 2(1), 3. 10.1109/TIT.1956.1056774
     https://doi.org/10.1109/TIT.1956.1056774 [Google Scholar]
  65. Shaoul, C. & Westbury, C.
    (2006) USENET Orthographic Frequencies for 111,627 English Words (2005–2006) Edmonton, AB: University of Alberta (downloaded fromwww.psych.ualberta.ca/~westburylab/downloads/wlfreq.download.html)
  66. Siakaluk, P. D., Sears, C. R., & Lupker, S. J.
    (2002) Orthographic neighborhood effects in lexical decision: The effects of nonword orthographic neighborhood size. Journal of Experimental Psychology: Human Perception and Performance, 28(3), 661.
     [Google Scholar]
  67. Sutton, R., & Barto, A.
    (1998) Reinforcement learning. Cambridge, MA: MIT Press.
     [Google Scholar]
  68. Takahashi, T.
    (2013) A psychophysical theory of Shannon entropy. Neuroendocrinology Letters, 34(7), 615–617.
     [Google Scholar]
  69. Treiman, R.
    (1985) Onsets and rimes as units of spoken syllables: Evidence from children. Journal of experimental child psychology, 39(1), 161–181. 10.1016/0022‑0965(85)90034‑7
     https://doi.org/10.1016/0022-0965(85)90034-7 [Google Scholar]
  70. (1986) The division between onsets and rimes in English syllables. Journal of Memory and Language, 25(4), 476–491. 10.1016/0749‑596X(86)90039‑2
     https://doi.org/10.1016/0749-596X(86)90039-2 [Google Scholar]
  71. Treiman, R., Fowler, C. A., Gross, J., Berch, D., & Weatherston, S.
    (1995) Syllable structure or word structure? Evidence for onset and rime units with disyllabic and trisyllabic stimuli. Journal of Memory and Language, 34(1), 132–155. 10.1006/jmla.1995.1007
     https://doi.org/10.1006/jmla.1995.1007 [Google Scholar]
  72. Westbury, C. F., & Hollis, G.
    (2007) Putting Humpty together again: Synthetic approaches to nonlinear variable effects underlying lexical access. InG. Jarema & G. Libben (Eds.), The Mental Lexicon: Core perspectives (pp.7–30). Bingley: Emerald. 10.1163/9780080548692_003
     https://doi.org/10.1163/9780080548692_003 [Google Scholar]
  73. Westbury, C., Shaoul, C., Moroschan, G., & Ramscar, M.
    (2016) Telling the world’s least funny jokes: On the quantification of humor as entropy. Journal of Memory and Language, 861, 141–156. 10.1016/j.jml.2015.09.001
     https://doi.org/10.1016/j.jml.2015.09.001 [Google Scholar]
  74. Westbury, C., Yang, M., and Anderson, K.
    (2024) The Principal Components of Meaning, Revisited. [Accepted for publication inPsychonomic Bulletin & Review]
     [Google Scholar]
  75. Wood, S. N.
    (2011) Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. Journal of the Royal Statistical Society (B), 73(1), 3–36. 10.1111/j.1467‑9868.2010.00749.x
     https://doi.org/10.1111/j.1467-9868.2010.00749.x [Google Scholar]
  76. Yarkoni, T., Balota, D., & Yap, M.
    (2008) Moving beyond Coltheart’s N: A new measure of orthographic similarity. Psychonomic Bulletin & Review, 15(5), 971–979. 10.3758/PBR.15.5.971
     https://doi.org/10.3758/PBR.15.5.971 [Google Scholar]
  77. Zhang, J. W., & Wang, Q. H.
    (2010) The orthographic neighborhood effect in word recognition. Advances in Psychological Science, 18(06), 892–899.
     [Google Scholar]
  78. Ziegler, J. C., & Perry, C.
    (1998) No more problems in Coltheart’s neighborhood: Resolving neighborhood conflicts in the lexical decision task. Cognition, 68(2), B53–B62. 10.1016/S0010‑0277(98)00047‑X
     https://doi.org/10.1016/S0010-0277(98)00047-X [Google Scholar]
  79. Zipf, G. K.
    (1936) The Psychobiology of Language. Houghton-Mifflin: New York, NY.
     [Google Scholar]
/content/journals/10.1075/ml.24006.wes
Loading
Orthographic uncertainty
The Mental Lexicon 19, 439 (2024); https://doi.org/10.1075/ml.24006.wes
/content/journals/10.1075/ml.24006.wes
/content/journals/10.1075/ml.24006.wes
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): Coltheart’s N; Levenshtein distance; lexical access; OLD20; ON; orthographic neighbourhood; orthography; reading; word typicality

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error