submitted to Epirob 2011

Mechanisms leading to tool use:

A longitudinal study in human infants


J. Kevin O'Regan

Laboratoire Psychologie de la Perception - UMR 8158 CNRS

UniversitŽ Paris Descartes

Paris, France

 

http://www.kevin-oregan.net

Lauriane Rat-Fischer

Laboratoire Psychologie de la Perception - UMR 8158 CNRS

UniversitŽ Paris Descartes

Paris, France

 

lratfischer@gmail.com

Jacqueline Fagard

Laboratoire Psychologie de la Perception - UMR 8158 CNRS

UniversitŽ Paris Descartes

Paris, France

 

jacqueline.fagard@parisdescartes.fr

 

NB: Video extracts can be found in the links accompanying each figure. Additional videos illustrating the article can be found on http://www.kevin-oregan.net/OReganEpirob2011


Abstract—We studied four infants aged 12 to 18 months in the progression of their ability to retrieve an out-of-reach toy using a rake-like tool.  We hypothesize that a simple "adaptive curiosity" mechanism might explain their behaviour in the early phases of development of this skill, but that in the later phases a second-order mechanism is necessary to account for what seems to be a more deliberative, goal-directed form of means-end behaviour involving hierarchically planning a sequence of actions. Videos are available for prospective robotic modelers.

Index Terms— goal-directed, human infants, means-end behaviour, planning, tool use.

I.    INTRODUCTION

It might be thought that it should be easy to learn to use a tool like a rake to retrieve an object that is out of reach. Surprisingly however, such behaviour is rarely seen in non human primates and then only within some social contexts [1], requires painstaking training to be learned in laboratory settings [2], and is only accomplished by human infants over a long period of learning culminating after the end of the second year of life [3-5].

The difficulty of acquiring the concept of extending one's reach is surprising, and the fact that most animals simply cannot acquire it at all as a concept, and need extensive conditioning to learn to do it raises the question of where exactly the difficulty lies.

As a first step to answer this question we examined how four infants aged 12 to 18 months progressed in their learning of this ability. We tried to extract a few principles which would explain how they progressed, and which could potentially be implemented in a computational model that might be of relevance to tool learning in robots.

II.    Method

We used a 25 cm long rake-like T-shaped tool constructed out of white cardboard. The objects to be attained by infants were small toys that we had previously evaluated as being particularly desirable. The toy was always placed too far away from the infant to be grasped directly. Four infants were observed in 4 to 6 sessions (depending on the infant) between the ages of 12 and 18 months. The sessions lasted between 20 and 45 minutes, and involved five conditions: toy attached to the rake part of the tool (C1), toy inside/against the rake part of the tool (C2), toy inside the tool but not against it (C3), and toy to the side of the tool (C4) (Fig. 1). A fifth condition was sometimes presented with the tool given in hand (C5). Trials were given in order of difficulty. When the child failed to use the tool, demonstrations were provided by parent or experimenter.

 

C1             C2               C3                   C4

Figure 1. C1: Toy attached to tool; C2: toy inside, touching; C3: toy inside not touching; C4: toy to the side of tool

III.    Results

We observed that the process of learning to extend reach with the tool extends over many months, with only a gradual change in the way the child uses the tool. This may seem surprising, given how obvious the task seems to adults.

An exception is condition C1, which is quite distinct from the other conditions. When the toy is obviously connected to the tool, even the youngest children we studied seem to consider the tool and toy as a single, composite object. They pick up the tool immediately and move it expertly in what seems like an intentional way so as to bring the part bearing the desired object into reach (Fig. 2). On the other hand the other conditions, where the toy is not attached to the tool, pose great problems to the infants.

Thus in conditions C2 and C3, successes seem mainly to be what we call "contingent", that is, they come from the simple fact that taking hold of the tool will often, by virtue of the toy's position with respect to the tool, move the toy and bring it into reach. Evidence that children do not understand why they succeed comes from the fact that they often start by pointing at the toy and ignoring the tool, and succeed only at a later moment when they loose interest in the toy, take the tool and thereby accidentally move the toy. Furthermore, when they do take the tool, they often do so without looking at the toy: they only look at the toy when motion of the tool makes it move, thereby attracting their attention. Another indication that children don't understand use of the tool comes from the fact that when the toy does not come all the way toward them, they do not try again to rake it closer, but point to it or beg for it with their hand, forgetting the tool. "Success” in these conditions depended on the relationship between the hand which grasped the tool and the side of the toy:  given the natural tendency to make centripetal arm motion, the toy was much more likely to come along when the right hand grasped the tool if the toy was on the left side of the tool).

 

13mo_C1_ImmediatelyOrientingToolWhenObjectAttached_child4_Fig2.mpg

Figure 2. Child at 12 months easily orienting tool to remove attached toy

 

Compared to conditions C1-C3, condition C4 is a critical condition of this experiment. Here the toy is further away from the tool and so cannot by brought closer by contingency. This condition is not accomplished efficiently until after 18 months. The same applies to condition C5, where the tool is given directly to the child.

In order to get insight into the mechanisms underlying these behaviours we attempted to code them in a way that allowed us to infer what the infant was attending to, and what was its goal, intention or plan, if any.

A.    Coding of behaviors

We first defined all the 'elementary' behaviours that occurred, of which there were 15 (e.g. "grasps tool", "points to toy", etc.). In any particular trial, these behaviours could occur in different combinations. We catalogued all these combinations and found that there were 28 distinct combinations over all 255 trials of the experiment. We then classified these behaviour combinations in categories that suggest the infant's intentions and where he was attending. For instance when an infant points toward the toy after refusing the tool, he clearly shows his desire for the toy and his lack of understanding of the usefulness of the tool. Or when he rakes with the tool on the table and puts it into his mouth without looking at the toy, which may even be accidentally brought closer, he clearly shows his interest in the tool. Such considerations gave rise to the following categories.

 

 

N: Not interested

1.      Grasps tool, gets rid of it, stops being interested (tool is grasped here without being the focus of attention)

2.      Looks at toy, looks at tool, looks at adult, doesn’t do anything

3.      Refusal (indicated by child shaking its head, or turning away from the table)

The first category ("Not Interested": behaviours 1-3) involved cases where the child was motivated neither to pick up the toy nor the tool, perhaps because the child was tired or discouraged. A cause of discouragement may sometimes have been that the child knew from previous trials that he would not be able to obtain the desired object. This was particularly the case in conditions C4 and C5, when the infants were in the youngest age groups. There were only 13 cases of Not Interested in all (5%).

 

 13mo_TslashT_C3_RakesToolOnTable_GivesToolToMother_child1_Fig3_Fig5.mpg 

16mo_TslashT_C4_PointsToToyToolInOtherHand_child1_Fig3.mpg

Figure 3. Behaviours 10 and 5: Raking for the sake of raking, and reaching for toy while forgetting tool in hand) at stage (T/T)

 

(T/T): Interested in the toy or in the tool, but not in connecting the two

4.      Points to toy and refuses the tool

5.      Points to toy, then grasps tool, points again toward toy with other hand

6.      Points to toy, hand on tool more or less by chance, grasps tool, hits tool on table

7.      Points to toy, hand on tool more or less by chance, forgets object, grasps tool, plays with it and then lets go of it

8.      Points to toy, then grasps tool (by chance or encouragement), swipes table with tool

9.      Looks at toy, then grasps tool, interested in tool only (puts into mouth or rubs, swipes, hits, etc. on table)

10.    Grasps tool, interested in tool only (puts into mouth or rubs, swipes, hits, etc. on table)

11.    Grasps tool, swipes table with it and sweeps toy away by accident

12.    Grasps tool, plays with it, then points to toy with empty hand

13.    Grasps tool, rakes with it (or lifts it), toy does not come, doesn’t try again to retrieve toy

Behaviours 4-13 are behaviours where the child's attention is focused either on the toy, or on the tool, or alternates between the two. Children may point to the toy (behaviours 4-7), or grasp the tool (behaviours 8-12), or one after the other. If they grasp the tool, they may manipulate it and test its affordances as an object in itself (they may put it in their mouth, rake it on the table, strike it on the table, try to rip it apart, etc.). But there will be no tool-toy connection except by contingency, by chance, or accidentally (behaviour 14). If there is an alternation between attending to toy and tool, then this alternation occurs simply because attention has faded on one and passed to another: there is no evidence of sequential planning. Examples are shown in Fig. 3.

 

A: Ambiguous cases

14.    Points to toy, hand on tool more or less by chance, grasps tool, rakes with it, toy comes possibly by contingency

15.    Points to toy then grasps tool encouraged by experimenter and brings the toy to hand possibly by contingency

16.    Looks at toy, grasps tool spontaneously, rakes tool, retrieves toy possibly by contingency

17.    Grasps tool, rakes tool spontaneously, retrieves toy possibly by contingency

Behaviours 14-17 are ambiguous and difficult to interpret. These are behaviours occurring almost only in conditions C2 and C3 where it is not clear whether the child is interested in the tool, the toy, or the connection between tool and toy. He may or may not grasp the toy. The situation is ambiguous because we don’t know whether or not retrieval occurred through contingency, that is, through the fact that pulling the tool causes it to rake in the toy simply because of where the toy is lying.

 

 12mo_TplusT_C4_HitsToyWithTool_child3_Fig4.mpg

18mo_TplusT_C4_PushesToyWithTool_child1_Fig4.mpg

Figure 4. Behaviours 18 and 19: banging the toy and pushing the toy at stage (T+T)

 

T+T: Interested in the connection between tool and toy (but not for retrieval)

18.    Points to toy, then grasps tool and pushes toy with it

19.    Grasps tool, touches, pushes, or hits toy with tool

Behaviours 18-19 concern cases where the child systematically and repetitively brings the tool to bear on the toy, for example by hitting it (Fig. 4). But the child seems not to be doing this in order to obtain the toy, since he will not systematically pick it up when it comes to hand (which may happen after repeated hits). This suggests that he is interested not in the tool itself or the toy itself, but in their connection or the way they interact. Said in another way, in this category of behaviours, the child is not planning a sequence of activities with the goal of obtaining the toy. He is simply exploring the way tool and toy interact (12% of all behaviors).

 

13mo_TslashT_C3_RakesToolOnTable_GivesToolToMother_child1_Fig3_Fig5.mpg 

17mo_ME_C4_PointsToToyWithTool_child2_Fig5.mpg

Figure 5. Behaviours 21 and 22: Giving tool to adult and pointing tool at toy at stage M-E (beginning of planned sequences of action).

 

M-E (Means-End): Beginning of planned sequences of action to obtain the toy

20.    Grasps tool, gets rid of it, points to toy

21.    Looks at object, grasps tool and gives tool to adult

22.    Points to toy, grasps tool, points to toy with tool

Contrary to the previous categories, here the child is able to hold the goal of obtaining the toy in mind and briefly divert his attention sufficiently to define the subgoal of picking up the tool to attain the goal. The child may actually cast the tool aside so it is not in the way; he may give the tool to the adult, or reach for the toy with the tool (Fig. 5). The child clearly wants the toy, and has the intuition that it is through doing something with the tool that he will attain this aim. Thus, behaviours 20-22 are the first indication that the child can use a subgoal as the means to the end of retrieving the toy.

 

R1: Interested in the tool for Retrieval: Trial and error, difficult or half-success, or only after encouragement or demonstration

23.    Grasps tool, moves tool, tries to bring back toy, failure

24.    Grasps tool, retrieves toy after demonstration

25.    Grasps tool after being encouraged to, moves tool and retrieves toy with it

26.    Grasps tool, awkward movements to bring toy to hand, success

27.    Grasps tool, moves tool, retrieves toy after several attempts

In behaviours 23-27 the child clearly has the goal of obtaining the toy, and has realized that it is through some kind of raking motions that this can be done. But the child has not yet perfected the exact motion that is necessary (Fig. 6). Note that behavior 24 occurred extremely rarely: out of 31 demonstration altogether, only 2 were followed by non- ambiguous retrieval, one at 14 months and one at 18 months, both in condition C5.

 

18mo_R1_C5_TriesToBringBackToyFailure_child1_Fig6.mpg 

17mo_R1_C5_AwkwardMovements_child3_Fig6.mpg

Figure 6. Behaviours 23 and 26: Retrieving inexpertly at stage R1

 

R2: Interested in the tool for Retrieval: Intentional mature success

28.    Grasps tool, moves tool to retrieve toy and plays with it

In behaviour 28 the child not only has the means-end knowledge, but has also mastered the particular raking movement that is necessary to retrieve the toy.

 

Figure 7. Incidence of behaviours corresponding to three categories as a function of age for C4 and C5: T/T & T+T corresponds to cases where the child is exploring the affordances of either the tool, the toy, or their combination. Means-End corresponds to behaviours involving a sequential activity where the toy is a goal, and the tool is the means to attain it, though never with success. R1 & R2 correspond to increasing degrees of proficiency at retrieving the toy with obvious understanding of the notion of tool.

 

B.    Behaviour categories as a function of age

The different categories of behaviour we have defined correspond to progressively more sophisticated cognitive capacities of the child. Figure 7 shows that Categories (T/T) and (T+T), where the child is exploring affordances of either toy, tool or their combination, are predominant up to about age 16 months, and then diminish. The means-end category M-E seems to be sporadically present after 13 months. The categories R1 and R2, corresponding to more or less efficient retrieval of the toy, start taking over at 18 months. In these analyses we have included only conditions C4 and C5, since condition C1 (toy attached) is always succeeded and conditions C2 and C3 give rise to behaviours in the Ambiguous category where little information can be extracted about the child's intentions. Note that with only four infants whose development was quite variable the data should be interpreted with caution. A cross-sectional study with a larger number of infants is in progress [6].

IV.    Possible mechanisms

We would like to envisage an account of these behaviours and their progression in terms of simple mechanisms that could conceivably be built into a robot.

A.    Adaptive curiosity

A first very basic mechanism could be an intrinsically motivated reinforcement learning algorithm such as "adaptive curiosity" suggested by Oudeyer and Kaplan [7], [8], and similar to related ideas implemented in a variety of robotic instantiations, for example [9-12]. The idea is that a useful strategy for an agent to learn autonomously is to act in a way such that the rate of information acquisition is maximal. The agent continually seeks new experiences, choosing as next action at every successive moment the action which is expected to fill out the currently most vacant parts of its knowledge structure.

From the psychological literature, a related idea for a basic autonomous learning mechanism might be the one supposed to be active in infant development at 4-8 months in Piaget's sensorimotor stage III, where with "secondary circular reactions" the child is assumed to be carrying out motor behaviours on objects to explore their sensory consequences. One could say that the child is exploring the affordances of objects. Piaget's idea differs from "adaptive curiosity" in not emphasizing maximum information gain at every moment.

In all such proposals the learning mechanism involved could be qualified as reactive rather than deliberative (in the terms of Sloman [13]), because it is based on the immediate situation. The agent or child's attention is focused only on one thing at a time, namely the thing that it is now involved in exploring. There is no long-term goal kept in mind, and no planning of a sequence of actions to attain it. There is only a continually evolving series of short-term goals designed to maximize information gain at the present moment. Such a mechanism is compatible with the finding that infants (and animals), when confronted with an unfamiliar object, may often appear to compulsively and automatically apply a whole battery of standard exploratory manipulations to it: putting the object in their mouth, shifting it from hand to hand, banging it on the table or the ground, exploring its articulations.

Operation of a learning mechanism such as adaptive curiosity starting from an age as early as 4-8 months may be what explains the ability of the infant to immediately solve the retrieval problem C1 in this experiment, i.e. when the toy is rigidly attached to the tool. From our results it seems that by age 12 months, the child already knows that composite rigid objects can be manipulated in ways that cause any subpart to be attained. This capacity is highly interesting in itself, and robotic simulations and investigations on younger children might profitably be devoted to confirm that this could be learned by a simple adaptive curiosity mechanism.

As concerns the other conditions C2-C5 of the present experiment, under the adaptive curiosity mechanism we can conceive that when confronted with the tool and the toy lying on the table, the child's attention will be attracted to one or the other, depending on its relative visual salience and novelty. The toy may often be the first object of attention (since the tool was purposefully fairly neutral in appearance), and the basic adaptive curiosity mechanism will lead the infant to attempt to obtain it, first by reaching and then often by pointing and begging. Pointing and begging can be considered methods of obtaining an object similar to grasping, but which involve the use of the adult as instrument instead of the hand [14]. Exploring the affordances of the pointing/begging act in order to explore the toy is thus one type of exploration activity which the child can engage in.

With little new information being acquired in this way, the child may rapidly tire of pointing and begging, and may shift his attention to the tool, the only object within reach. Here he may apply a battery of manipulations, for example finger it, put it in the mouth, pass it from hand to hand, wave it in the air. When the child's familiarity with such affordances is sufficient, the child may go back to pointing and begging for the toy, keeping the tool in hand (Fig. 3).

Orienting to the tool or orienting to the toy as just discussed are two behaviours which may occur singly or successively, and they correspond well to the behaviours we have categorized in class (T/T). An alternative class of behaviours is when the child's attention becomes loosened from the tool, and extends to focus on the effect that manipulating the tool can have on other objects and in particular on the toy. This corresponds to category (T+T), where the child is interested in the connection between tool and toy, but not in the goal of retrieving the toy. For example the child can bang the tool on the toy, or push the toy with it (Fig. 4). When doing this the child's purpose is presumably to explore the effects these actions produce  -- such an idea has also been proposed by [15]). But observing which actions produce which effects involves making a connection in one direction: from the action to the effect. It does not imply that if the child later wants to obtain an effect, it can find the necessary action. This would involve something like doing an inverse lookup, and may be difficult at this stage. This explains why seeing the toy come along with the tool by contingency does not help the child to understand the tool’s usefulness.

In conclusion at this point, the application of a simple autonomous learning mechanism with intrinsic motivation like adaptive curiosity is compatible with the observed behaviours that we have categorized in categories (T/T) and (T+T), where the child has a single focus of attention which may be the toy, the tool, or the interaction of toy and tool, but where in each case the child is not motivated by the overall goal of bringing the toy to hand, and is rather simply engaged in exploring the effects of its actions. There is no sequential planning, and no concept of the tool as a means for obtaining the toy.

A final note concerns the usefulness of demonstrations by the adult: at this stage, if the child is given a demonstration of how to obtain the toy, it cannot profit from this demonstration for two reasons: First, at this stage the child may not have the cognitive ability to put itself in the place of the demonstrator, and so it may not "see" the occurring action as a possible action it could engender itself. Second the child is only noting the relation in one direction, namely from action to effect, and so may not be able to invert the relation to find the necessary action. These two facts may explain the surprising inability of children at this stage to learn from demonstrations.

B.    A second order mechanism with planning and goals

Stages T/T and T+T seem to co-occur with the transition to stage M-E. In this, the child's behaviour can no longer simply be described as probing affordances in a purely reactive way. Instead, the child seems not only to have a goal, namely to obtain the toy, but it is also able to set this goal aside temporarily while it achieves a subgoal, for example to reject the tool, to give it to the adult so the adult can take the toy, or to seize the tool so as to point to the toy with it. Such behaviours clearly require sequential planning -- Bruner [16] refers to the notion of serially ordered constituents. Thus it can be said that the child is starting to display Means-End behaviour: the child has the intuition that the tool is the means with which to obtain the toy, even if at this first stage it does not know exactly how to proceed.

Can we explain the transition to such proto-deliberative, planning, goal-directed means-end behaviour by invoking the same "adaptive curiosity" mechanism as we invoked to explain behaviour in categories (T/T) and (T+T)? This is a question that must be posed to roboticists. Intuitively however it seems first that sequentiality is not something that is naturally part of an adaptive curiosity approach, since there action at any moment is determined only by the instantaneous expected information gain.

The notion of "goal", in the sense of something that guides a sequence of behaviours is something that also does not seem inherent in the simple adaptive curiosity approach. Adaptive curiosity leads to activity which can be described in terms of a goal such as "obtaining the toy", or "playing with the tool", but it is not clear how under a simple adaptive curiosity mechanism that goal could then be encapsulated and set aside while a different subgoal (e.g. setting aside the tool to more easily obtain the toy) is achieved. Some sort of hierarchical representation would seem to be required in the agent, as would also be the ability to sequentially order and prioritize actions.

Finally the notion of a means to obtain an end also is not naturally part of the immediate-reactive idea inherent in the adaptive curiosity approach.

From these considerations it seems that some higher order mechanism is needed, which allows conceptual manipulation of goals and subgoals and the ability to plan them sequentially. The notion of a "means to an end" might emerge from such a system. Perhaps, in a robotics framework, a system that might be able to achieve this would require a hierarchical intrinsic reinforcement learning scheme, e.g. [17], the ability to form qualitative hierarchies [18], or the ability make use of a longer temporal horizon than the mere instantaneous state of the system [19].

The idea that in order to explain the transition to Means-End behaviours a new mechanism might be necessary receives support from the developmental psychology literature, where it is known that during the second year there are important maturational changes which may underlie the appearance of planning and sequential behaviour [20]. These maturational changes may also be the cause for the sudden appearance of other capacities, both in language and in social cognition, which are peculiar to humans: the second year is the period when the word combinations start occurring in language acquisition, when mirror self-recognition appears, where children start doing pretend play [21], and where they start understanding the intentions of others [22]. Anecdotally, to illustrate the onset of a social, intentional aspect to the children's behaviour, it is worth noting that the older children in our experiment often seemed less interested in obtaining the toy than in playing the game of taking it and giving it back to the experimenter for the next trial.

Another interesting question concerns whether the transition to the means-end stage also corresponded to a transition in the type of pointing. One might expect that from the proto-imperative pointing at the younger ages in which the adult is considered an instrument to be "manipulated", the child graduates to a more declarative form of pointing involving shared intention: "wow that's interesting over there, why don't you have a look!" (c.f. however [23] and [24] who claim intention sharing occurs already at 12 months in pointing). Currently we have no way of testing this.

C.    Transition to efficient retrieval

Possessing the ability to sequence actions in pursuit of a goal is not sufficient to efficiently accomplish a task, since the actions must be appropriate. The latest stages in the progress of the infant in our experiment clearly corresponded to a stage where through trial and error they gradually adapted their skills so that they succeeded more and more easily in retrieving the toy. Here nothing more complicated than some form of reinforcement learning seems to be required to account for progress. Demonstrations by the adult become particularly useful to the child at this stage, because they can provide the child exactly what he is missing, namely better knowledge of the exact action to be performed. Furthermore, with the notion of goal and intentionality, the child is equipped to put himself in the shoes of the demonstrator and so make use of the information provided.

V.    CONCLUSION

We have examined the progressive acquisition of the ability to use a rake-like tool in four 12-18 month old infants. A first surprising result is the ease with which children accomplish the task when the toy to be attained is fixed to the tool. Our interpretation is that at 12 months children already have the notion of a composite object and know how to manipulate it to access its parts.

A second interesting fact is the great difficulty that children have in solving the problem when the toy is not fixed to the tool. They do actually succeed in this task when spatial proximity of toy and tool implies that moving the tool is likely to bring the toy closer, i.e. by simple contingency. However our analyses show that at the younger ages such successes do not correspond to real knowledge of the use of a tool as a means to an end. Thus, contrary to some claims in the developmental literature [3], we suggest that proximity itself may not be an essential factor determining real means-end understanding.

Our results further suggest that at first, children's performance is compatible with the operation of a simple learning algorithm with intrinsic motivation exemplified by the idea of "adaptive curiosity". Such an algorithm explains the purely reactive, exploratory behaviour displayed by infants up until after age 12 months. It also explains the very surprising inability of infants in their first sessions to learn from their own accidental successes: at this stage children are merely observing affordances, and cannot profit from observation of their successes to attain a goal: this would require having the notion of means and end.  Also explained is the fact that observation of adults performing the task likewise provides little help, since at this stage children may not have a sufficiently well developed ability to understand the intentions of others.

Finally we observe that, approaching age 18 months, children start displaying the ability to define a long-term goal that can be set briefly aside to accomplish intermediate subgoals. They start becoming able to sequence their behaviours, thereby setting the stage for use of the tool as the means to accomplish an end. It would seem that a simple adaptive curiosity algorithm would not be able to accomplish this, and that a hierarchically organized learning scheme would be necessary.

Acknowledgment

JKO'R acknowledges the support of ANR BINAAHR. LR-F acknowledges support from Fondation de France

References

[1]     E. E. Furlong, K. J. Boose, and S. T. Boysen, “Raking it in: the impact of enculturation on chimpanzee tool use,” Animal Cognition, vol. 11, no. 1, pp. 83-97, 2007.

[2]     H. Ishibashi, S. Hihara, and A. Iriki, “Acquisition and development of monkey tool-use: behavioral and kinematic analyses,” Canadian Journal of Physiology and Pharmacology, vol. 78, no. 11, pp. 958-966, 2000.

[3]     E. Bates, V. Carlson-Luden, and I. Bretherton, “Perceptual aspects of tool using in infancy*,” Infant Behavior and Development, vol. 3, p. 127–140, 1980.

[4]     L. van Leeuwen, A. Smitsman, and C. van Leeuwen, “Affordances, perceptual complexity, and the development of tool use,” Journal of Experimental Psychology, vol. 20, no. 1, p. 174–191, 1994.

[5]     R. Keen, “Learning without limits: From problem solving towards a unified theory of learning,” Annual Review of Psychology, vol. 62, pp. 1-21, 2011.

[6]     L. Rat-Fischer, J. K. O’Regan, and J. Fagard, “Tool-use learning mechanisms at the end of the second year in human infants,” presented at the First Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, Frankfurt am Main, 2011.

[7]     P.-Y. Oudeyer, F. Kaplan, and V. V. Hafner, “Intrinsic Motivation Systems for Autonomous Mental Development,” IEEE Transactions on Evolutionary Computation, vol. 11, no. 2, pp. 265-286, 2007.

[8]     F. Kaplan and P.-Y. Oudeyer, “Motivational principles for visual know-how development,” in Proceedings of the Third International Conference on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, vol. 101, L. Prince, H. Berthouze, D. Bullock, G. Stojanov, and C. Balkenius, Eds. Edinburgh, Scotland: Lund University Cognitive Studies, 2003, pp. 73-80.

[9]     J. Modayil and B. Kuipers, “The initial development of object knowledge by a learning robot,” Robotics and autonomous systems, vol. 56, no. 11, p. 879–890, 2008.

[10]   A. Stoytchev, “Behavior-grounded representation of tool affordances,” in IEEE International Conference on Robotics and Automation, 2005, vol. 3, p. 3060.

[11]   J. Marshall, D. Blank, and L. Meeden, “An emergent framework for self-motivation in developmental robotics,” in Proceedings of the 3rd international conference on development and learning (ICDL 2004), Salk Institute, San Diego, 2004, pp. 112-119.

[12]   J. Schmidhuber, “A possibility for implementing curiosity and boredom in model-building neural controllers,” in Proceedings of the first international conference on simulation of adaptive behavior on From animals to animats, Cambridge, MA, USA, 1990, p. 222–227.

[13]   A. Sloman, “Beyond shallow models of emotion,” Cognitive Processing: International Quarterly of Cognitive Science, vol. 2, no. 1, pp. 177-198, 2001.

[14]   E. Bates, L. Camaioni, and V. Volterra, “The acquisition of performatives prior to speech.,” Merrill-Palmer Quarterly: Journal of Developmental Psychology, vol. 21, no. 3, pp. 205-226, 1975.

[15]   J. J. Lockman, “A perception–action perspective on tool use development,” Child Development, vol. 71, no. 1, pp. 137-144, 2000.

[16]   J. S. Bruner, “Organization of early skilled action,” Child Development, vol. 44, no. 1, p. 1–11, 1973.

[17]   A. G. Barto, S. Singh, and N. Chentanez, “Intrinsically motivated learning of hierarchical collections of skills,” in Proceedings of the 3rd international conference on development and learning (ICDL 2004), Salk Institute, San Diego, 2004, pp. 112-119.

[18]   J. Mugan and B. Kuipers, “Autonomously learning an action hierarchy using a learned qualitative state representation,” in Proc. of the Int. Joint Conf. on Artificial Intelligence, 2009.

[19]   N. A. Mirza, C. L. Nehaniv, K. Dautenhahn, and R. te Boekhorst, “Grounded Sensorimotor Interaction Histories in an Information Theoretic Metric Space for Robot Ontogeny,” Adaptive Behavior, vol. 15, no. 2, pp. 167-187, 2007.

[20]   A. Diamond and P. S. Goldman-Rakic, “Comparison of human infants and rhesus monkeys on Piaget’s AB task: Evidence for dependence on dorsolateral prefrontal cortex,” Experimental Brain Research, vol. 74, no. 1, pp. 24-40, 1989.

[21]   L. Fenson, J. Kagan, R. B. Kearsley, and P. R. Zelazo, “The developmental progression of manipulative play in the first two years,” Child Development, vol. 47, no. 1, p. 232–236, 1976.

[22]   M. Tomasello and M. Carpenter, “Shared intentionality,” Developmental Science, vol. 10, no. 1, pp. 121-125, Jan. 2007.

[23]   M. Tomasello, M. Carpenter, and U. Liszkowski, “A New Look at Infant Pointing,” Child Development, vol. 78, no. 3, pp. 705-722, 2007.

[24]   J.-C. Gomez, “Pointing Behaviors in Apes and Human Infants: A Balanced Interpretation,” Child Development, vol. 78, no. 3, pp. 729-734, 2007.

 



Additional videos illustrating this article can be found on http://www.kevin-oregan.net/OReganEpirob2011