You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
rical730 9cdcf4314d
Version restriction in tutorial (#715)
5 days ago
.github update parl.maddpg without import gym (#208) 2 years ago
.teamcity add mock gym.py update continuous_wrapper add example test (#685) 2 months ago
benchmark add torch paddle dqn roadrunner (#695) 2 months ago
docs add paddle_based maddpg (#660) 3 months ago
evo_kit update evo_kit build.sh and readme (#418) 12 months ago
examples Version restriction in tutorial (#715) 5 days ago
papers Create ICLR_2021.md (#683) 2 months ago
parl update atari_wrappers (#702) 1 month ago
.copyright.hook chinese in doc (#266) 1 year ago
.gitignore implement of IMPALA with the newest parallel design (#60) 2 years ago
.pre-commit-config.yaml redesign basic class in PARL (#26) 3 years ago
.readthedocs.yml fix docs build fail (#637) 4 months ago
CMakeLists.txt unittest for paddle by default (#622) 3 months ago
LICENSE Initial commit 3 years ago
MANIFEST.in fix unittest bugs (#136) 2 years ago
README.cn.md release 2.0.2 (#678) 2 months ago
README.md release 2.0.2 (#678) 2 months ago
setup.py first PR of heartbeat refactor (#489) 9 months ago

README.md

PARL

English | 简体中文

Documentation Status Documentation Status Documentation Status Release

PARL is a flexible and high-efficient reinforcement learning framework.

About PARL

Features

Reproducible. We provide algorithms that stably reproduce the result of many influential reinforcement learning algorithms.

Large Scale. Ability to support high-performance parallelization of training with thousands of CPUs and multi-GPUs.

Reusable. Algorithms provided in the repository could be directly adapted to a new task by defining a forward network and training mechanism will be built automatically.

Extensible. Build new algorithms quickly by inheriting the abstract class in the framework.

Abstractions

abstractions PARL aims to build an agent for training algorithms to perform complex tasks. The main abstractions introduced by PARL that are used to build an agent recursively are the following:

Model

Model is abstracted to construct the forward network which defines a policy network or critic network given state as input.

Algorithm

Algorithm describes the mechanism to update parameters in Model and often contains at least one model.

Agent

Agent, a data bridge between the environment and the algorithm, is responsible for data I/O with the outside environment and describes data preprocessing before feeding data into the training process.

Note: For more information about base classes, please visit our tutorial and API documentation.

Parallelization

PARL provides a compact API for distributed training, allowing users to transfer the code into a parallelized version by simply adding a decorator. For more information about our APIs for parallel training, please visit our documentation.
Here is a Hello World example to demonstrate how easy it is to leverage outer computation resources.

#============Agent.py=================
@parl.remote_class
class Agent(object):

    def say_hello(self):
        print("Hello World!")

    def sum(self, a, b):
        return a+b

parl.connect('localhost:8037')
agent = Agent()
agent.say_hello()
ans = agent.sum(1,5) # it runs remotely, without consuming any local computation resources

Two steps to use outer computation resources:

  1. use the parl.remote_class to decorate a class at first, after which it is transferred to be a new class that can run in other CPUs or machines.
  2. call parl.connect to initialize parallel communication before creating an object. Calling any function of the objects does not consume local computation resources since they are executed elsewhere.
PARL As shown in the above figure, real actors (orange circle) are running at the cpu cluster, while the learner (blue circle) is running at the local gpu with several remote actors (yellow circle with dotted edge).

For users, they can write code in a simple way, just like writing multi-thread code, but with actors consuming remote resources. We have also provided examples of parallized algorithms like IMPALA, A2C. For more details in usage please refer to these examples.

Install:

Dependencies

  • Python 3.6+(Python 3.8+ is preferable for distributed training).
  • paddlepaddle>=2.0 (Optional, if you only want to use APIs related to parallelization alone)
pip install parl

Getting Started

Several-points to get you started:

For absolute beginners, we also provide an introductory course on reinforcement learning (RL) : ( Video | Code )

Examples

NeurlIPS2018 Half-Cheetah Breakout
NeurlIPS2018