liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
DMPs based CPG Actor Critic: A Method for Locomotion Learning
Interaction Lab University of Skövde Skövde, Sweden.
Interaction Lab University of Skövde Skövde, Sweden.
Interaction Lab University of Skövde Skövde, Sweden.
2014 (English)Manuscript (preprint) (Other academic)
Abstract [en]

In this article, a dynamic motor primitives (DMPs) based CPG-Actor-Critic is proposed to enable locomotion learning on a humanoid (the NAO robot) and a puppy robot (the ghostdog robot). In order to model two types of locomotion with one architecture, a novel application of an existing method to designa CPG architecture for learning locomotion. The method is to a) have an architectural base (4-cell CPG) and, b) have a learning component which is based on an existing method for designing DMPs. Learning locomotion here concerns gait emergence in relation to the robot’s body and prior knowledge. The focus of this article will be on two types of locomotion: crawling on ahumanoid and running on a puppy robot. On the two robots with two different morphologies, our method and architecture can make the robots learn by itself. We also compare the performance with respect to two state-of-the-art reinforcement learning algorithms with provided particular instantiations of ourDMPs-based CPG-Actor-Critic architecture. Finally, based on the analysis of the experimental results, a generic view/architecture for locomotion learning is discussed and introduced in the conclusion.

Place, publisher, year, edition, pages
2014.
National Category
Computer Science
Identifiers
URN: urn:nbn:se:liu:diva-106777OAI: oai:DiVA.org:liu-106777DiVA: diva2:718774
Available from: 2014-05-22 Created: 2014-05-22 Last updated: 2014-05-22Bibliographically approved
In thesis
1. Reinforcement Learning of Locomotion based on Central Pattern Generators
Open this publication in new window or tab >>Reinforcement Learning of Locomotion based on Central Pattern Generators
2014 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Locomotion learning for robotics is an interesting and challenging area in which the movement capabilities of animals have been deeply investigated and acquired knowledge has been transferred into modelling locomotion on robots. What modellers are required to understand is what structure can represent locomotor systems in different animals and how such animals develop various and dexterous locomotion capabilities. Notwithstanding the depth of research in the area, modelling locomotion requires a deep rethinking.

In this thesis, based on the umbrella of embodied cognition, a neural-body-environment interaction is emphasised and regarded as the solution to locomotion learning/development. Central pattern generators (CPGs) are introduced in the first part (Chapter 2) to generally interpret the mechanism of locomotor systems in animals. With a deep investigation on the structure of CPGs and inspiration from human infant development, a layered CPG architecture with baseline motion generation and dynamics adaptation interfaces are proposed. In the second part, reinforcement learning (RL) is elucidated as a good method for dealing with locomotion learning from the perspectives of psychology, neuroscience and robotics (Chapter 4). Several continuous-space RL techniques (e.g. episodic natural actor critic, policy learning by weighting explorations with returns, continuous action space learning automaton are introduced for practical use (Chapter 3). With the knowledge of CPGs and RL, the architecture and concept of CPG-Actor-Critic is constructed. Finally, experimental work based on published papers is highlighted in a path of my PhD research (Chapter 5). This includes the implementation of CPGs and the learning on the NAO robot for crawling and walking. The implementation is also extended to test the generalizability to different morphologies (the ghostdog robot). The contribution of this thesis is discussed from two angles: the investigation of the CPG architecture and the implementation (Chapter 6).

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2014. 71 p.
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1602
National Category
Computer Science
Identifiers
urn:nbn:se:liu:diva-105884 (URN)978-91-7519-313-7 (ISBN)
Public defence
2014-06-04, G110, hus G, Högskolan i Skövde, Skövde, 12:30 (English)
Opponent
Supervisors
Available from: 2014-05-22 Created: 2014-04-11 Last updated: 2014-05-22Bibliographically approved

Open Access in DiVA

No full text

Authority records BETA

Li, CaiLowe, RobertZiemke, Tom

Search in DiVA

By author/editor
Li, CaiLowe, RobertZiemke, Tom
Computer Science

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 155 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf