OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. This is the
gym open-source library, which gives you access to an ever-growing variety of environments.
gym makes no assumptions about the structure of your agent, and is compatible with any numerical computation library, such as TensorFlow or Theano. You can use it from Python code, and soon from other languages.
If you're not sure where to start, we recommend beginning with the docs on our site.
Contents of this document
There are two basic concepts in reinforcement learning: the environment (namely, the outside world) and the agent (namely, the algorithm you are writing). The agent sends actions to the environment, and the environment replies with observations and rewards (that is, a score).
The core gym interface is Env, which is
the unified environment interface. There is no interface for agents;
that part is left to you. The following are the
Env methods you
- reset(self): Reset the environment's state. Returns observation.
- step(self, action): Step the environment by one timestep. Returns observation, action, reward, done.
- render(self, mode='human', close=False): Render one frame of the environment. The default mode will do something human friendly, such as pop up a window. Passing the close flag signals the renderer to close any such windows.
You can perform a minimal install of
git clone https://github.com/openai/gym.git cd gym pip install -e .
If you prefer, you can do a minimal install of the packaged version directly from PyPI:
pip install gym
You'll be able to run a few environments right away:
- classic_control (you'll need
pygletto render though)
We recommend playing with those environments at first, and then later installing the dependencies for the remaining environments.
Once you're ready to install everything, run
pip install -e '.[all]' (or
pip install 'gym[all]').
MuJoCo has a proprietary dependency we can't set up for you. Follow
mujoco-py package for help.
For the install to succeed, you'll need to have some system packages installed. We'll build out the list here over time; please let us know what you end up installing on your platform.
brew install cmake
On Ubuntu 14.04:
apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl
We currently support Linux and OS X running Python 2.7 or 3.5. Python 3 support should still be considered experimental -- if you find any bugs, please report them!
We will expand support to Windows based on demand. We will also soon ship a Docker container exposing the environments callable from any platform, for use with any non-Python framework, such as Torch.
pip install -e '.[all]', you'll need a semi-recent pip.
Please make sure your pip is at least at version
1.5.0. You can
upgrade using the following:
pip install --ignore-installed
pip. Alternatively, you can open setup.py and
install the dependencies by hand.
Installing dependencies for specific environments
If you'd like to install the dependencies for only specific environments, see setup.py. We maintain the lists of dependencies on a per-environment group basis.
The code for each environment group is housed in its own subdirectory gym/envs. The specification of each task is in gym/envs/__init__.py. It's worth browsing through both.
These are a variety of algorithmic tasks, such as learning to copy a sequence.
import gym env = gym.make('Copy-v0') env.reset() env.render()
The Atari environments are a variety of Atari video games. If you didn't do the full install, you can install dependencies via
pip install -e '.[atari]' (you'll need
cmake installed) and then get started as follow:
import gym env = gym.make('SpaceInvaders-v0') env.reset() env.render()
This will install
atari-py, which automatically compiles the Arcade Learning Environment. This can take quite a while (a few minutes on a decent laptop), so just be prepared.
The board game environments are a variety of board games. If you didn't do the full install, you can install dependencies via
pip install -e '.[board_game]' (you'll need
cmake installed) and then get started as follow:
import gym env = gym.make('Go9x9-v0') env.reset() env.render()
These are a variety of classic control tasks, which would appear in a typical reinforcement learning textbook. If you didn't do the full install, you will need to run
pip install -e '.[classic_control]' to enable rendering. You can get started with them via:
import gym env = gym.make('CartPole-v0') env.reset() env.render()
MuJoCo is a physics engine which can do
very detailed efficient simulations with contacts. It's not
open-source, so you'll have to follow the instructions in mujoco-py
to set it up. You'll have to also run
pip install -e '.[mujoco]' if you didn't do the full install.
import gym env = gym.make('Humanoid') env.reset() env.render()
Toy environments which are text-based. There's no extra dependency to install, so to get started, you can just do:
import gym env = gym.make('FrozenLake-v0') env.reset() env.render()
- Run examples/agents/random_agent.py to run an simple random agent and upload the results to the scoreboard.
- Run examples/agents/cem.py to run an actual learning agent (using the cross-entropy method) and upload the results to the scoreboard.
- Run examples/scripts/list_envs to generate a list of all environments. (You see also just browse the list on our site.
- Run examples/scripts/upload to upload the recorded output from
cem.py. Make sure to obtain an API key.
We are using nose2 for tests. You can run them via:
You can also run tests in a specific directory by using the
-s option, or by passing in the specific name of the test. See the nose2 docs for more details.