Volume 22, Issue 2
  • ISSN 1572-0373
  • E-ISSN: 1572-0381



In this paper, we explore interaction history as a particular source of pressure for achieving emergent compositional communication in multi-agent systems. We propose a training regime implementing template transfer, the idea of carrying over learned biases across contexts. In the presented method, a sender-receiver dyad is first trained with a disentangled pair of objectives, and then the receiver is transferred to train a new sender with a standard objective. Unlike other methods (e.g. the obverter algorithm), the template transfer approach does not require imposing inductive biases on the architecture of the agents. We experimentally show the emergence of compositional communication using topographical similarity, zero-shot generalization and context-independence as evaluation metrics. The presented approach is connected to an important line of work in semiotics and developmental psycholinguistics: it supports a conjecture that compositional communication is scaffolded on simpler communication protocols.

Available under the CC BY 4.0 license.

Article metrics loading...

Loading full text...

Full text loading...



  1. Barrett, J., Skyrms, B., and Cochran, C.
    (2018) Hierarchical Models for the Evolution of Compositional Language. Institute for Mathematical Behavioral Sciences Technical Report MBS 18-03.
    [Google Scholar]
  2. Barrett, J. A. and Skyrms, B.
    (2017) Self-assembling Games. The British Journal for the Philosophy of Science, 68(2):329–353. 10.1093/bjps/axv043
    https://doi.org/10.1093/bjps/axv043 [Google Scholar]
  3. Barrett, J. A., Skyrms, B., and Mohseni, A.
    (2019) Self-Assembling Networks. The British Journal for the Philosophy of Science, 70(1):301–325. 10.1093/bjps/axx039
    https://doi.org/10.1093/bjps/axx039 [Google Scholar]
  4. Batali, J.
    (1998) Computational simulations of the emergence of grammar. Approach to the Evolution of Language, pages405–426.
    [Google Scholar]
  5. Bogin, B., Geva, M., and Berant, J.
    (2018) Emergence of Communication in an Interactive World with Consistent Speakers. arXiv:1809.00549 [cs]. arXiv: 1809.00549.
    [Google Scholar]
  6. Bouchacourt, D. and Baroni, M.
    (2018) How agents see things: On visual representations in an emergent language game. arXiv:1808.10696 [cs]. arXiv: 1808.10696. 10.18653/v1/D18‑1119
    https://doi.org/10.18653/v1/D18-1119 [Google Scholar]
  7. Brighton, H.
    (2002) Compositional Syntax From Cultural Transmission. Artificial Life, 8(1):25–54. 10.1162/106454602753694756
    https://doi.org/10.1162/106454602753694756 [Google Scholar]
  8. Brighton, H. and Kirby, S.
    (2006) Understanding Linguistic Evolution by Visualizing the Emergence of Topographic Mappings. Artificial Life, 12(2):229–242. 10.1162/artl.2006.12.2.229
    https://doi.org/10.1162/artl.2006.12.2.229 [Google Scholar]
  9. Brown, P. F., Della-Pietra, S. A., Della-Pietra, V. J., and Mercer, R. L.
    (1993) The mathematics of statistical machine translation. Computational Linguistics, 19(2):263–313.
    [Google Scholar]
  10. Bruner, J. S.
    (1981) Intention in the structure of action and interaction. Advances in Infancy Research, 11:41–56.
    [Google Scholar]
  11. (1983) Child’s talk: learning to use language. W.W. Norton, New York, 1st ed edition.
    [Google Scholar]
  12. Cangelosi, A.
    (2001) Evolution of communication and language using signals, symbols, and words. IEEE Transactions on Evolutionary Computation, 5(2):93–101. 10.1109/4235.918429
    https://doi.org/10.1109/4235.918429 [Google Scholar]
  13. Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E., and Baroni, M.
    (2020) Compositionality and Generalization In Emergent Languages. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages4427–4442, Online. Association for Computational Linguistics. 10.18653/v1/2020.acl‑main.407
    https://doi.org/10.18653/v1/2020.acl-main.407 [Google Scholar]
  14. Choi, E., Lazaridou, A., and de Freitas, N.
    (2018) Compositional Obverter Communication Learning From Raw Visual Input. ICLR 2018. arXiv: 1804.02341.
    [Google Scholar]
  15. Cogswell, M., Lu, J., Lee, S., Parikh, D., and Batra, D.
    (2019) Emergence of Compositional Language with Deep Generational Transmission. ArXiv, abs/1904.09067.
    [Google Scholar]
  16. Cowley, S. J.
    editor (2011) Distributed language. Number v. 34 in Benjamins current topics. John Benjamins Pub. Co, Amsterdam; Philadelphia. OCLC: ocn741355729. 10.1075/bct.34
    https://doi.org/10.1075/bct.34 [Google Scholar]
  17. Das, A., Kottur, S., Moura, J. M. F., Lee, S., and Batra, D.
    (2017) Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 20171. arXiv: 1703.06585. 10.1109/ICCV.2017.321
    https://doi.org/10.1109/ICCV.2017.321 [Google Scholar]
  18. De Beule, J. and Bergen, B. K.
    (2006) On the emergence of compositionality. InThe Evolution of Language, pages35–42, Rome, Italy. World scientific. 10.1142/9789812774262_0005
    https://doi.org/10.1142/9789812774262_0005 [Google Scholar]
  19. Deacon, T. W.
    (1998) The symbolic species: the co-evolution of language and the brain. Norton, New York, NY, norton paperback edition. OCLC: 254499872.
    [Google Scholar]
  20. Elman, J. L.
    (1990) Finding Structure in Time. Cognitive Science, 14(2):179–211. 10.1207/s15516709cog1402_1
    https://doi.org/10.1207/s15516709cog1402_1 [Google Scholar]
  21. Foerster, J. N., Assael, Y. M., de Freitas, N., and Whiteson, S.
    (2016) Learning to Communicate with Deep Multi-Agent Reinforcement Learning. NIPS’16 Proceedings of the 30th International Conference on Neural Information Processing Systems. arXiv: 1605.06676.
    [Google Scholar]
  22. Gong, T., Shuai, L., and Zhang, M.
    (2014) Modelling language evolution: Examples and predictions. Physics of Life Reviews, 11(2):280–302. 10.1016/j.plrev.2013.11.009
    https://doi.org/10.1016/j.plrev.2013.11.009 [Google Scholar]
  23. Goodfellow, I., Bengio, Y., and Courville, A.
    (2016) Deep learning. Adaptive computation and machine learning. The MIT Press, Cambridge, Massachusetts.
    [Google Scholar]
  24. Grouchy, P., D’Eleuterio, G. M. T., Christiansen, M. H., and Lipson, H.
    (2016) On The Evolutionary Origin of Symbolic Communication. Scientific Reports, 6:34615. 10.1038/srep34615
    https://doi.org/10.1038/srep34615 [Google Scholar]
  25. Jaques, N., Lazaridou, A., Hughes, E., Gulcehre, C., Ortega, P. A., Strouse, D. J., Leibo, J. Z., and de Freitas, N.
    (2018) Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning. arXiv:1810.08647 [cs, stat]. arXiv: 1810.08647.
    [Google Scholar]
  26. Kharitonov, E. and Baroni, M.
    (2020) Emergent Language Generalization and Acquisition Speed are not tied to Compositionality. arXiv:2004.03420 [cs]. arXiv: 2004.03420. 10.18653/v1/2020.blackboxnlp‑1.2
    https://doi.org/10.18653/v1/2020.blackboxnlp-1.2 [Google Scholar]
  27. Kharitonov, E., Chaabouni, R., Bouchacourt, D., and Baroni, M.
    (2019) EGG: a toolkit for research on Emergence of lanGuage in Games. arXiv:1907.00852 [cs]. arXiv: 1907.00852. 10.18653/v1/D19‑3010
    https://doi.org/10.18653/v1/D19-3010 [Google Scholar]
  28. Kingma, D. P. and Ba, J.
    (2014) Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs]. arXiv: 1412.6980.
    [Google Scholar]
  29. Kirby, S.
    (2001) Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity. IEEE Transactions on Evolutionary Computation, 5(2):102–110. 10.1109/4235.918430
    https://doi.org/10.1109/4235.918430 [Google Scholar]
  30. Kirby, S., Cornish, H., and Smith, K.
    (2008) Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language. Proceedings of the National Academy of Sciences, 105(31):10681–10686. 10.1073/pnas.0707835105
    https://doi.org/10.1073/pnas.0707835105 [Google Scholar]
  31. Korbak, T., Zubek, J., and Raczaszek-Leonardi, J.
    (2020) Measuring non-trivial compositionality in emergent communication. arXiv preprint arXiv:2010.15058.
    [Google Scholar]
  32. Kottur, S., Moura, J. M. F., Lee, S., and Batra, D.
    (2017) Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog. arXiv:1706.08502 [cs]. arXiv: 1706.08502. 10.18653/v1/D17‑1321
    https://doi.org/10.18653/v1/D17-1321 [Google Scholar]
  33. Kriegeskorte, N.
    (2008) Representational similarity analysis – connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience. 10.3389/neuro.06.004.2008
    https://doi.org/10.3389/neuro.06.004.2008 [Google Scholar]
  34. Kuciński, L., Korbak, T., Kolodziej, P., and Miloś, P.
    (2021) Catalytic role of noise and necessity of inductive biases in the emergence of compositional communication. InAdvances in Neural Information Processing Systems.
    [Google Scholar]
  35. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J.
    (2016) Building Machines That Learn and Think Like People. arXiv:1604.00289 [cs, stat]. arXiv: 1604.00289.
    [Google Scholar]
  36. Lazaridou, A., Hermann, K. M., Tuyls, K., and Clark, S.
    (2018) Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input. arXiv:1804.03984 [cs]. arXiv: 1804.03984.
    [Google Scholar]
  37. Lazaridou, A., Peysakhovich, A., and Baroni, M.
    (2016) Multi-Agent Cooperation and the Emergence of (Natural) Language. arXiv:1612.07182 [cs]. arXiv: 1612.07182.
    [Google Scholar]
  38. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P.
    (1998) Gradient-based learning applied to document recognition. InPROCEEDINGS OF THE IEEE, pages2278–2324. 10.1109/5.726791
    https://doi.org/10.1109/5.726791 [Google Scholar]
  39. Levenshtein, V. I.
    (1966) Binary Codes Capable of Correcting Deletions, Insertions and Reversals. Soviet Physics Doklady, 101:707.
    [Google Scholar]
  40. Lewis, D. K.
    (1969) Convention: a philosophical study. Blackwell, Oxford, nachdr. edition. OCLC: 837747718.
    [Google Scholar]
  41. Li, F. and Bowling, M.
    (2019) Ease-of-Teaching and Language Structure from Emergent Communication. arXiv:1906.02403 [cs]. arXiv: 1906.02403.
    [Google Scholar]
  42. Lowe, R., Foerster, J., Boureau, Y.-L., Pineau, J., and Dauphin, Y.
    (2019) On the Pitfalls of Measuring Emergent Communication. arXiv:1903.05168 [cs, stat]. arXiv: 1903.05168.
    [Google Scholar]
  43. Nomikou, I., Koke, M., and Rohlfing, K. J.
    (2017) Verbs in Mothers’ Input to Six-Month-Olds: Synchrony between Presentation, Meaning, and Actions Is Related to Later Verb Acquisition. Brain Sciences, 7(12):52. 10.3390/brainsci7050052
    https://doi.org/10.3390/brainsci7050052 [Google Scholar]
  44. Nowak, M. A. and Krakauer, D. C.
    (1999) The evolution of language. Proceedings of the National Academy of Sciences, 96(14):8028–8033. Publisher: National Academy of Sciences Section: Biological Sciences. 10.1073/pnas.96.14.8028
    https://doi.org/10.1073/pnas.96.14.8028 [Google Scholar]
  45. Oliphant, M. and Batali, J.
    (1997) Learning and the Emergence of Coordinated Communication. Center for Research on Language Newsletter, 111.
    [Google Scholar]
  46. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A.
    (2017) Automatic differentiation in PyTorch.
    [Google Scholar]
  47. Peirce, C. S.
    (1998) Collected papers of Charles Sanders Peirce. Thoemmes Press, Bristol, England. OCLC: ocm39692049.
    [Google Scholar]
  48. Raczaszek-Leonardi, J.
    (2012) Language as a System of Replicable Constraints. InLAWS, LANGUAGE and LIFE, volume71, pages295–333. Springer Netherlands, Dordrecht. 10.1007/978‑94‑007‑5161‑3_19
    https://doi.org/10.1007/978-94-007-5161-3_19 [Google Scholar]
  49. Raczaszek-Leonardi, J., Nomikou, I., Rohlfing, K. J., and Deacon, T. W.
    (2018) Language Development From an Ecological Perspective: Ecologically Valid Ways to Abstract Symbols. Ecological Psychology, 30(1):39–73. 10.1080/10407413.2017.1410387
    https://doi.org/10.1080/10407413.2017.1410387 [Google Scholar]
  50. Rainey, H. J., Zuberbühler, K., and Slater, P. J. B.
    (2004) Hornbills can distinguish between primate alarm calls. Proceedings of the Royal Society of London. Series B: Biological Sciences, 271(1540):755–759. 10.1098/rspb.2003.2619
    https://doi.org/10.1098/rspb.2003.2619 [Google Scholar]
  51. Rumelhart, D. E., Hinton, G. E., and Williams, R. J.
    (1986) Learning representations by back-propagating errors. Nature, 323(6088):533–536. 10.1038/323533a0
    https://doi.org/10.1038/323533a0 [Google Scholar]
  52. Ryan, M. and Rand, S.
    (1993) Sexual selection and signal evolution: the ghost of biases past. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 340(1292):187–195. 10.1098/rstb.1993.0057
    https://doi.org/10.1098/rstb.1993.0057 [Google Scholar]
  53. Skyrms, B.
    (2010) Signals: evolution, learning, & information. Oxford University Press, Oxford; New York. OCLC: ocn477256653. 10.1093/acprof:oso/9780199580828.001.0001
    https://doi.org/10.1093/acprof:oso/9780199580828.001.0001 [Google Scholar]
  54. Steinert-Threlkeld, S.
    (2020) Towards the Emergence of Non-trivial Compositionaliy. Philosophy of Science. 10.1086/710628
    https://doi.org/10.1086/710628 [Google Scholar]
  55. Stern, D. N.
    (1974) The Goal and Structure of Mother-Infant Play. Journal of the American Academy of Child Psychiatry, 13(3):402–421. 10.1016/S0002‑7138(09)61348‑0
    https://doi.org/10.1016/S0002-7138(09)61348-0 [Google Scholar]
  56. Tomasello, M., Carpenter, M., Call, J., Behne, T., and Moll, H.
    (2005) Understanding and sharing intentions: The origins of cultural cognition. Behavioral and Brain Sciences, 28(5):675–691. 10.1017/S0140525X05000129
    https://doi.org/10.1017/S0140525X05000129 [Google Scholar]
  57. Wittgenstein, L.
    (1953) Philosophical Investigations. Basil Blackwell, Oxford.
    [Google Scholar]
  58. Yamashita, Y. and Tani, J.
    (2008) Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment. PLoS Computational Biology, 4(11):e1000220. 10.1371/journal.pcbi.1000220
    https://doi.org/10.1371/journal.pcbi.1000220 [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error