Volume 17, Issue 2
  • ISSN 1572-0373
  • E-ISSN: 1572-0381
Buy:$35.00 + Taxes


The development of reasoning systems exploiting expert knowledge from interactions with humans is a non-trivial problem, particularly when considering how the information can be coded in the knowledge representation. For example, in human development, the acquisition of knowledge at one level requires the consolidation of knowledge from lower levels. How is the accumulated experience structured to allow the individual to apply knowledge to new situations, allowing reasoning and adaptation? We investigate how this can be done automatically by an iCub that interacts with humans to acquire knowledge via demonstration. Once consolidated, this knowledge is used in further acquisitions of experience concerning preconditions and consequences of actions. Finally, this knowledge is translated into rules that allow reasoning and planning for novel problem solving, including a Tower of Hanoi scenario. We thus demonstrate proof of concept for an interaction system that uses knowledge acquired from human interactions to reason about new situations.


Article metrics loading...

Loading full text...

Full text loading...


  1. Broz, F. , Nehaniv, C. L. , Kose-Bagci, H. , & Dautenhahn, K.
    (2012) Interaction Histories and Short Term Memory: Enactive Development of Turn-taking Behaviors in a Childlike Humanoid Robot. Computing Research Repository. Artificial Intelligence; Adaptation and Self-Organizing Systems.
    [Google Scholar]
  2. Cangelosi, A. , Metta, G. , Sagerer, G. , Nolfi, S. , Nehaniv, C. , Fischer, K. , … Zeschel, A.
    (2010) Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics. IEEE Transactions on Autonomous Mental Development, 2(1), 167–195. doi: 10.1109/TAMD.2010.2053034
    https://doi.org/10.1109/TAMD.2010.2053034 [Google Scholar]
  3. Carey, S.
    (2009) The Origin of Concepts. Oxford University Press. doi: 10.1093/acprof:oso/9780195367638.001.0001
    https://doi.org/10.1093/acprof:oso/9780195367638.001.0001 [Google Scholar]
  4. Carey, S. , & Xu, F.
    (2001) Infants’ knowledge of objects: beyond object files and object tracking. Cognition, 80(1–2), 179–213. doi: 10.1016/S0010‑0277(00)00154‑2
    https://doi.org/10.1016/S0010-0277(00)00154-2 [Google Scholar]
  5. Cheng, P. W.
    (1996) From Covariation to Causation: A Causal Power Theory. Psychological Review, 104(1), 367–405.
    [Google Scholar]
  6. Crangle, C. , & Suppes, P.
    (1994) Language and Learning for Robots. Center for the Study of Language and Information.
    [Google Scholar]
  7. Demiris, Y. , & Khadhouri, B.
    (2006) Content-based control of goal-directed attention during human action perception. InROMAN 2006 – The 15th IEEE International Symposium on Robot and Human Interactive Communication (pp.226–231). IEEE. doi: 10.1109/ROMAN.2006.314422
    https://doi.org/10.1109/ROMAN.2006.314422 [Google Scholar]
  8. Dindo, H. , & Schillaci, G.
    (2010) An adaptive probabilistic approach to goal-level imitation learning. IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 – Conference Proceedings, 4452–4457. doi: 10.1109/IROS.2010.5654298
    https://doi.org/10.1109/IROS.2010.5654298 [Google Scholar]
  9. Dominey, P. F.
    (2003) Learning Grammatical Constructions in a Miniature Language from Narrated Video Events. In25th Annual Meeting of the Cognition Science Society, Boston.
    [Google Scholar]
  10. Dominey, P. F. , & Dodane, C.
    (2004) Indeterminacy in language acquisition: the role of child directed speech and joint attention. Journal of Neurolinguistics, 17(2–3), 121–145. doi: 10.1016/S0911‑6044(03)00056‑3
    https://doi.org/10.1016/S0911-6044(03)00056-3 [Google Scholar]
  11. Dominey, P. F. , Mallet, A. , & Yoshida, E.
    (2007a) Progress in Programming the HRP-2 Humanoid Using Spoken Language. InProceedings 2007 IEEE International Conference on Robotics and Automation (pp.2169–2174). IEEE.
    [Google Scholar]
  12. (2007b) Real-time cooperative behavior acquisition by a humanoid apprentice. In2007 7th IEEE-RAS International Conference on Humanoid Robots (pp.270–275). IEEE.
    [Google Scholar]
  13. (2009) Real-time spoken-language programming for cooperative interaction with a humanoid apprentice. International Journal of Humanoid Robotics, 6(1), 147–171. doi: 10.1142/S0219843609001711
    https://doi.org/10.1142/S0219843609001711 [Google Scholar]
  14. Dore, A. , Cattoni, A. F. , & Regazzoni, C. S.
    (2010) Interaction Modeling and Prediction in Smart Spaces: A Bio-Inspired Approach Based on Autobiographical Memory. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 40(1), 1191–1205. doi: 10.1109/TSMCA.2010.2052600
    https://doi.org/10.1109/TSMCA.2010.2052600 [Google Scholar]
  15. Doshi, F. , & Roy, N.
    (2008) Spoken language interaction with model uncertainty: an adaptive human – robot interaction system. Connection Science, 20(1), 299–318. doi: 10.1080/09540090802413145
    https://doi.org/10.1080/09540090802413145 [Google Scholar]
  16. Fern, A. , Givan, R. , & Siskind, J. M.
    (2002) Specific-to-General Learning for Temporal Events with Application to Learning Event Definitions from Video. J. Artificial Intelligence Res., 103–129.
    [Google Scholar]
  17. G. L. Drescher.
    (1991) Made-up minds: a constructivist approach to artificial intelligence. MIT press.
    [Google Scholar]
  18. Gergely, G. , & Csibra, G.
    (2003) Teleological reasoning in infancy: the naı̈ve theory of rational action. Trends in Cognitive Sciences, 7(1), 287–292. doi: 10.1016/S1364‑6613(03)00128‑1
    https://doi.org/10.1016/S1364-6613(03)00128-1 [Google Scholar]
  19. (2005) The social construction of the cultural mind: Imitative learning as a mechanism of human pedagogy. Interaction Studies, 6(1), 463–481.
    [Google Scholar]
  20. (2006) Sylvia’s recipe: The role of imitation and pedagogy in the transmission of cultural knowledge. Roots of Human Sociality: Culture, Cognition, and Human Interaction, 229–255.
    [Google Scholar]
  21. Gergely, G. , Nádasdy, Z. , Csibra, G. , & Bíró, S.
    (1995) Taking the intentional stance at 12 months of age. Cognition, 56(1), 165–193. doi: 10.1016/0010‑0277(95)00661‑H
    https://doi.org/10.1016/0010-0277(95)00661-H [Google Scholar]
  22. Gori, I. , Pattacini, U. , Nori, F. , Metta, G. , & Sandini, G.
    (2012) DForC: A real-time method for reaching, tracking and obstacle avoidance in humanoid robots. In2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012) (pp.544–551). IEEE.
    [Google Scholar]
  23. Gorniak, P. , & Roy, D.
    (2004) Grounded Semantic Composition for Visual Scenes. Journal of Artificial Intelligence Research, 21, 429–470.
    [Google Scholar]
  24. Guez, A. , Silver, D. , & Dayan, P.
    (2013) Scalable and efficient bayes-adaptive reinforcement learning based on Monte-Carlo tree search. Journal of Artificial Intelligence Research, 48, 841–883.
    [Google Scholar]
  25. Hayes-Roth, F.
    (1997) June15. Artificial Intelligence: What Works and What Doesn’t?AI Magazine.
    [Google Scholar]
  26. Helmert, M.
    (2009) Concise finite-domain representations for PDDL planning tasks. Artificial Intelligence, 173(5–6), 503–535. doi: 10.1016/j.artint.2008.10.013
    https://doi.org/10.1016/j.artint.2008.10.013 [Google Scholar]
  27. Ho, W. C. , Dautenhahn, K. , Lim, M. Y. , Vargas, P. A. , Aylett, R. , & Enz, S.
    (2009) An initial memory model for virtual and robot companions supporting migration and long-term interaction. InRO-MAN 2009 – The 18th IEEE International Symposium on Robot and Human Interactive Communication (pp.277–284). IEEE. doi: 10.1109/ROMAN.2009.5326204
    https://doi.org/10.1109/ROMAN.2009.5326204 [Google Scholar]
  28. Kalkan, S. , Dag, N. , Yürüten, O. , Borghi, A. M. , & Sahin, E.
    (2014) Verb concepts from affordances. Interaction Studies Journal, 15(1), 1–37. doi: 10.1075/is.15.1.01kal
    https://doi.org/10.1075/is.15.1.01kal [Google Scholar]
  29. Kaplan, F. , & Hafner, V. V.
    (2006) The challenges of joint attention. Interaction Studies, 7(1), 135–169.
    [Google Scholar]
  30. Kulick, J. , Toussaint, M. , Lang, T. , Lopes, M. , Sud-ouest, I. B. , & Bat, A.
    (2013) Active Learning for Teaching a Robot Grounded Relational Symbols at Stuttgart. InProceedings of theTwenty-Third international joint conference on Artificial Intelligence.
    [Google Scholar]
  31. Lallee, S. , Lemaignan, S. , Lenz, A. , Melhuish, C. , Natale, L. , Skachek, S. , … Dominey, P. F.
    (2010) Towards a platform-independent cooperative human-robot interaction system: I. Perception. In2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp.4444–4451). IEEE.
    [Google Scholar]
  32. Lallee, S. , Madden, C. , Hoen, M. , & Dominey, P. F.
    (2010) Linking language with embodied and teleological representations of action for humanoid cognition. Frontiers in Neurorobotics, 4, 8.
    [Google Scholar]
  33. Lallee, S. , Pattacini, U. , Boucher, J. D. , Lemaignan, S. , Lenz, A. , Melhuish, C. , … Dominey, P. F.
    (2011) Towards a platform-independent cooperative human-robot interaction system: II. Perception, execution and imitation of goal directed actions. In2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp.2895–2902). IEEE.
    [Google Scholar]
  34. Lallee, S. , Pattacini, U. , Lemaignan, S. , Lenz, A. , Melhuish, C. , Natale, L. , … Dominey, P. F.
    (2012) Towards a Platform-Independent Cooperative Human Robot Interaction System: III An Architecture for Learning and Executing Actions and Shared Plans. IEEE Transactions on Autonomous Mental Development, 4(1), 239–253. doi: 10.1109/TAMD.2012.2199754
    https://doi.org/10.1109/TAMD.2012.2199754 [Google Scholar]
  35. Lallee, S. , Warneken, F. , & Dominey, P.
    (2009) Learning to collaborate by observation. InEpirob, Venice, Italy.
  36. Lauria, S. , Bugmann, G. , Kyriacou, T. , & Klein, E.
    (2002) Mobile robot programming using natural language. Robotics and Autonomous Systems, 38(3–4), 171–181. doi: 10.1016/S0921‑8890(02)00166‑5
    https://doi.org/10.1016/S0921-8890(02)00166-5 [Google Scholar]
  37. Lévi-Strauss, C.
    (1979) Myth and Meaning. Routledge.
    [Google Scholar]
  38. Martinez, D. , Alenya, G. , & Torras, C.
    (2015) Relational Reinforcement Learning with Guided Demonstrations. Artificial Intelligence, (Corrected Proof, 19 February 2015). doi: 10.1016/j.artint.2015.02.006
    https://doi.org/10.1016/j.artint.2015.02.006 [Google Scholar]
  39. McDermott, D. , Malik, G. , Adele, H. , Craig, K. , Ashwin, R. , Manuela, V. , … David, W.
    (1998) PDDL – The Planning Domain Definition Language. Intech. report CVC TR–98-003/DCS TR–1165, Yale Center for Computational Vision and Control, Yale Univ., New Haven, Conn..
    [Google Scholar]
  40. McGuire, P. , Fritsch, J. , Steil, J. J. , Rothling, F. , Fink, G. A. , Wachsmuth, S. , … Ritter, H.
    (2002) Multi-modal human-machine communication for instructing robot grasping tasks. InIEEE/RSJ International Conference on Intelligent Robots and System (Vol.2, pp.1082–1088). IEEE.
    [Google Scholar]
  41. Metta, G. , Sandini, G. , Vernon, D. , Natale, L. , & Nori, F.
    (2008) The iCub humanoid robot. InProceedings of the 8th Workshop on Performance Metrics for Intelligent Systems – PerMIS ’08 (p.50). New York, New York, USA: ACM Press. doi: 10.1145/1774674.1774683
    https://doi.org/10.1145/1774674.1774683 [Google Scholar]
  42. Mirza, N. A. , Nehaniv, C. L. , Dautenhahn, K. , & Boekhorst, R.
    (2008) Developing social action capabilities in a humanoid robot using an interaction history architecture. InHumanoids 2008 – 8th IEEE-RAS International Conference on Humanoid Robots (pp.609–616). IEEE.
    [Google Scholar]
  43. Ognibene, D. , Chinellato, E. , Sarabia, M. , & Demiris, Y.
    (2013) Contextual action recognition and target localization with an active allocation of attention on a humanoid robot. Bioinspiration & Biomimetics, 8(1), 035002.
    [Google Scholar]
  44. Pasula, H. M. , Zettlemoyer, L. S. , & Kaelbling, L. P.
    (2007) Learning symbolic models of stochastic domains. Journal of Artificial Intelligence Research, 29, 309–352.
    [Google Scholar]
  45. Payne, J. D. , & Nadel, L.
    (2004) Sleep, dreams, and memory consolidation: the role of the stress hormone cortisol. Learning & Memory (Cold Spring Harbor, N.Y.), 11(1), 671–8.
  46. Petit, M. , Lallee, S. , Boucher, J.-D. , Pointeau, G. , Cheminade, P. , Ognibene, D. , … Dominey, P. F.
    (2013) The Coordinating Role of Language in Real-Time Multimodal Learning of Cooperative Tasks. IEEE Transactions on Autonomous Mental Development, 5(1), 3–17. doi: 10.1109/TAMD.2012.2209880
    https://doi.org/10.1109/TAMD.2012.2209880 [Google Scholar]
  47. Pointeau, G. , Petit, M. , & Dominey, P. F.
    (2013) Embodied Simulation Based on Autobiographical Memory. InBiomimetic and Biohybrid Systems (Vol.8064, pp.240–250). Berlin, Heidelberg: Springer Berlin Heidelberg. doi: 10.1007/978‑3‑642‑39802‑5_21
    https://doi.org/10.1007/978-3-642-39802-5_21 [Google Scholar]
  48. (2013) Robot Learning Rules of Games by Extraction of Intrinsic Properties. InACHI 2013, The Sixth International Conference on Advances in Computer-Human Interactions (pp.109–116).
    [Google Scholar]
  49. (2014) Successive Developmental Levels of Autobiographical Memory for Learning Through Social Interaction. IEEE Transactions on Autonomous Mental Development, 6(1), 200–212. doi: 10.1109/TAMD.2014.2307342
    https://doi.org/10.1109/TAMD.2014.2307342 [Google Scholar]
  50. Rader, N. de V. , & Zukow-Goldring, P.
    (2012) Caregivers’ gestures direct infant attention during early word learning: the importance of dynamic synchrony. Language Sciences, 34(1), 559–568. doi: 10.1016/j.langsci.2012.03.011
    https://doi.org/10.1016/j.langsci.2012.03.011 [Google Scholar]
  51. Ross, S. , & Pineau, J.
    (2008) Online Planning Algorithms for POMDPs. The Journal of Artificial Intelligence Research, 32(1), 663–704.
    [Google Scholar]
  52. Roy, D. K.
    (2002a) Learning visually grounded words and syntax for a scene description task. Computer Speech & Language, 16(3–4), 353–385. doi: 10.1016/S0885‑2308(02)00024‑4
    https://doi.org/10.1016/S0885-2308(02)00024-4 [Google Scholar]
  53. (2002b) Learning words from sights and sounds: a computational model. Cognitive Science, 26(1), 113–146. doi: 10.1207/s15516709cog2601_4
    https://doi.org/10.1207/s15516709cog2601_4 [Google Scholar]
  54. Siskind, J. M.
    (1996) A computational study of cross-situational techniques for learning word-to-meaning mappings. Cognition, 61(1–2), 39–91. doi: 10.1016/S0010‑0277(96)00728‑7
    https://doi.org/10.1016/S0010-0277(96)00728-7 [Google Scholar]
  55. Sutton, S. , Ronald A., C. , & Jacques De Villiers , Johan Schalkwyk , Pieter JE Vermeulen , Michael W. Macon, Y. Y. et al.
    (1998) Universal speech tools: the CSLU toolkit. InICSLP, vol.98 (pp.3221–3224).
    [Google Scholar]
  56. Takács, B. , & Demiris, Y.
    (2008) Balancing Spectral Clustering for Segmenting Spatio-temporal Observations of Multi-agent Systems. In2008 Eighth IEEE International Conference on Data Mining (pp.580–587). IEEE.
    [Google Scholar]
  57. Wang, X.
    (1994) Learning Planning Operators by Observation and Practice. InProceedings of theSecond International Conference on AI Planning Systems, AIPS-94 (pp.335–340). Chicago, IL.
    [Google Scholar]
  58. Welke, K. , Kaiser, P. , Kozlov, A. , Adermann, N. , Asfour, T. , Lewis, M. , & Steedman, M.
    (2013) Grounded Spatial Symbols for Task Planning Based on Experience. In13th International Conference on Humanoid Robots (Humanoids). IEEE/RAS.
    [Google Scholar]
  59. Wood, R. , Baxter, P. , & and Belpaeme, T.
    (2011) A review of long-term memory in natural and synthetic systems. Adaptive Behavior, 20(1), 81–103.
    [Google Scholar]
  60. Zacks, J. M. , Speer, N. K. , Swallow, K. M. , Braver, T. S. , & Reynolds, J. R.
    (2007) Event perception: A mind-brain perspective. Psychological bulletin2, 133, 273. doi: 10.1037/0033‑2909.133.2.273
    https://doi.org/10.1037/0033-2909.133.2.273 [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error