liu.seSearch for publications in DiVA
Change search
Link to record
Permanent link

Direct link
BETA
Lowe, Robert
Publications (5 of 5) Show all publications
Li, C., Lowe, R. & Ziemke, T. (2014). DMPs based CPG Actor Critic: A Method for Locomotion Learning.
Open this publication in new window or tab >>DMPs based CPG Actor Critic: A Method for Locomotion Learning
2014 (English)Manuscript (preprint) (Other academic)
Abstract [en]

In this article, a dynamic motor primitives (DMPs) based CPG-Actor-Critic is proposed to enable locomotion learning on a humanoid (the NAO robot) and a puppy robot (the ghostdog robot). In order to model two types of locomotion with one architecture, a novel application of an existing method to designa CPG architecture for learning locomotion. The method is to a) have an architectural base (4-cell CPG) and, b) have a learning component which is based on an existing method for designing DMPs. Learning locomotion here concerns gait emergence in relation to the robot’s body and prior knowledge. The focus of this article will be on two types of locomotion: crawling on ahumanoid and running on a puppy robot. On the two robots with two different morphologies, our method and architecture can make the robots learn by itself. We also compare the performance with respect to two state-of-the-art reinforcement learning algorithms with provided particular instantiations of ourDMPs-based CPG-Actor-Critic architecture. Finally, based on the analysis of the experimental results, a generic view/architecture for locomotion learning is discussed and introduced in the conclusion.

National Category
Computer Sciences
Identifiers
urn:nbn:se:liu:diva-106777 (URN)
Available from: 2014-05-22 Created: 2014-05-22 Last updated: 2018-10-08Bibliographically approved
Li, C., Lowe, R. & Ziemke, T. (2013). Humanoids learning to walk: a natural CPG-actor-critic architecture. Frontiers in Neurorobotics, 7(5)
Open this publication in new window or tab >>Humanoids learning to walk: a natural CPG-actor-critic architecture
2013 (English)In: Frontiers in Neurorobotics, ISSN 1662-5218, Vol. 7, no 5Article in journal (Refereed) Published
Abstract [en]

The identification of learning mechanisms for locomotion has been the subject of much research for some time but many challenges remain. Dynamic systems theory (DST) offers a novel approach to humanoid learning through environmental interaction. Reinforcement learning (RL) has offered a promising method to adaptively link the dynamic system to the environment it interacts with via a reward-based value system. In this paper, we propose a model that integrates the above perspectives and applies it to the case of a humanoid (NAO) robot learning to walk the ability of which emerges from its value-based interaction with the environment. In the model, a simplified central pattern generator (CPG) architecture inspired by neuroscientific research and DST is integrated with an actor-critic approach to RL (cpg-actor-critic). In the cpg-actor-critic architecture, least-square-temporal-difference based learning converges to the optimal solution quickly by using natural gradient learning and balancing exploration and exploitation. Futhermore, rather than using a traditional (designer-specified) reward it uses a dynamic value function as a stability indicator that adapts to the environment. The results obtained are analyzed using a novel DST-based embodied cognition approach. Learning to walk, from this perspective, is a process of integrating levels of sensorimotor activity and value.

Place, publisher, year, edition, pages
Frontiers Media S.A., 2013
Keywords
reinforcement learning, humanoid walking, central pattern generators, actor-critic, dynamical systems theory, embodied cognition, value system
National Category
Computer and Information Sciences
Research subject
Technology
Identifiers
urn:nbn:se:liu:diva-106770 (URN)10.3389/fnbot.2013.00005 (DOI)23675345 (PubMedID)
Available from: 2013-08-08 Created: 2014-05-22 Last updated: 2018-10-08Bibliographically approved
Li, C., Lowe, R. & Ziemke, T. (2012). Modelling Walking Behaviors Based on CPGs: A Simplified Bio-inspired Architecture. In: Tom Ziemke, Christian Balkenius, John Hallam (Ed.), From Animals to Animats 12: 12th International Conference on Simulation of Adaptive Behavior, SAB 2012Odense, Denamark, August 27-30, 2012. Paper presented at 12th International Conference on Simulation of Adaptive Behavior, SAB 2012, Odense, Denmark, August 27-30, 2012 (pp. 156-166). Berlin, Heidelberg: Springer
Open this publication in new window or tab >>Modelling Walking Behaviors Based on CPGs: A Simplified Bio-inspired Architecture
2012 (English)In: From Animals to Animats 12: 12th International Conference on Simulation of Adaptive Behavior, SAB 2012Odense, Denamark, August 27-30, 2012 / [ed] Tom Ziemke, Christian Balkenius, John Hallam, Berlin, Heidelberg: Springer , 2012, p. 156-166Conference paper, Published paper (Refereed)
Abstract [en]

In this article, we use a recurrent neural network including four-cell core architecture to model the walking gait and implement it with the simulated and physical NAO robot. Meanwhile, inspired by the biological CPG models, we propose a simplified CPG model which comprises motorneurons, interneurons, sensor neurons and the simplified spinal cord. Within this model, the CPGs do not directly output trajectories to the servo motors. Instead, they only work to maintain the phase relation among ipsilateral and contralateral limbs. The final output is dependent on the integration of CPG signals, outputs of interneurons, motor neurons and sensor neurons (sensory feedback).

Place, publisher, year, edition, pages
Berlin, Heidelberg: Springer, 2012
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 7426
Keywords
CPGs, the NAO robot, Interneuron, Motorneuron
National Category
Computer Sciences
Research subject
Technology
Identifiers
urn:nbn:se:liu:diva-106771 (URN)10.1007/978-3-642-33093-3_16 (DOI)2-s2.0-84866033575 (Scopus ID)978-3-642-33092-6 (ISBN)978-3-642-33093-3 (ISBN)
Conference
12th International Conference on Simulation of Adaptive Behavior, SAB 2012, Odense, Denmark, August 27-30, 2012
Available from: 2012-10-31 Created: 2014-05-22 Last updated: 2018-10-08Bibliographically approved
Li, C., Lowe, R., Duran, B. & Ziemke, T. (2011). Humanoids that crawl: Comparing gait performance of iCub and NAO using a CPG architecture. In: Shaozi Li, Ying Dai (Ed.), Proceedings - 2011 IEEE International Conference on Computer Science and Automation Engineering, CSAE 2011: . Paper presented at 2011 IEEE International Conference on Computer Science and Automation Engineering, CSAE 2011, 10 June 2011 through 12 June 2011, Shanghai (pp. 577-582). IEEE conference proceedings
Open this publication in new window or tab >>Humanoids that crawl: Comparing gait performance of iCub and NAO using a CPG architecture
2011 (English)In: Proceedings - 2011 IEEE International Conference on Computer Science and Automation Engineering, CSAE 2011 / [ed] Shaozi Li, Ying Dai, IEEE conference proceedings , 2011, p. 577-582Conference paper, Published paper (Refereed)
Abstract [en]

In this article, a generic CPG architecture is used to model infant crawling gaits and is implemented on the NAO robot platform. The CPG architecture is chosen via a systematic approach to designing CPG networks on the basis of group theory and dynamic systems theory. The NAO robot performance is compared to the iCub robot which has a different anatomical structure. Finally, the comparison of performance and NAO whole-body stability are assessed to show the adaptive property of the CPG architecture and the extent of its ability to transfer to different robot morphologies. © 2011 IEEE.

Place, publisher, year, edition, pages
IEEE conference proceedings, 2011
Series
Proceedings - 2011 IEEE International Conference on Computer Science and Automation Engineering, CSAE 2011 ; 4
Keywords
CPG, Crawling, iCub, Infant development, NAO, Algebra, Network architecture, Robots, System theory, Computer architecture
National Category
Engineering and Technology
Identifiers
urn:nbn:se:liu:diva-106772 (URN)10.1109/CSAE.2011.5952916 (DOI)2-s2.0-80051897170 (Scopus ID)9781424487257 (ISBN)
Conference
2011 IEEE International Conference on Computer Science and Automation Engineering, CSAE 2011, 10 June 2011 through 12 June 2011, Shanghai
Available from: 2013-02-19 Created: 2014-05-22 Last updated: 2018-10-08Bibliographically approved
Lee, G., Lowe, R. & Ziemke, T. (2011). Modelling Early Infant Walking: Testing a Generic CPG Architecture on the NAO Humanoid. In: IEEE International Conference on Development and Learning (ICDL), 2011: . Paper presented at The 2011 IEEE International Conference on Development and Learning, ICDL 2011; Frankfurt am Main; 24-27 August 2011, Category number CFP11294-ART; Code 87020 (pp. 1-6). IEEE conference proceedings
Open this publication in new window or tab >>Modelling Early Infant Walking: Testing a Generic CPG Architecture on the NAO Humanoid
2011 (English)In: IEEE International Conference on Development and Learning (ICDL), 2011, IEEE conference proceedings , 2011, p. 1-6Conference paper, Published paper (Refereed)
Abstract [en]

In this article, a simple CPG network is shown to model early infant walking, in particular the onset of independent walking. The difference between early infant walking and early adult walking is addressed with respect to the underlying neurophysiology and evaluated according to gait attributes. Based on this, we successfully model the early infant walking gait on the NAO robot and compare its motion dynamics and performance to those of infants. Our model is able to capture the core properties of early infant walking. We identify differences in the morphologies between the robot and infant and the effect of this on their respective performance. In conclusion, early infant walking can be seen to develop as a function of the CPG network and morphological characteristics.

Place, publisher, year, edition, pages
IEEE conference proceedings, 2011
Series
IEEE International Conference on Development and Learning (ICDL), ISSN 2161-9476 ; Vol. 2
Keywords
Early Walking, CPG, Morphology, Development, NAO
National Category
Computer Sciences
Identifiers
urn:nbn:se:liu:diva-106778 (URN)10.1109/DEVLRN.2011.6037318 (DOI)000297472300007 ()2-s2.0-80055009067 (Scopus ID)978-1-61284-989-8 (ISBN)
Conference
The 2011 IEEE International Conference on Development and Learning, ICDL 2011; Frankfurt am Main; 24-27 August 2011, Category number CFP11294-ART; Code 87020
Available from: 2013-02-18 Created: 2014-05-22 Last updated: 2018-10-08

Search in DiVA

Show all publications