Using sample-factory at Hugging Face

sample-factory is a codebase for high throughput asynchronous reinforcement learning. It has integrations with the Hugging Face Hub to share models with evaluation results and training metrics.

Exploring sample-factory in the Hub

You can find sample-factory models by filtering at the left of the models page.

All models on the Hub come up with useful features:

1. An automatically generated model card with a description, a training configuration, and more.
2. Metadata tags that help for discoverability.
3. Evaluation results to compare with other models.
4. A video widget where you can watch your agent performing.

Install the library

To install the sample-factory library, you need to install the package:

pip install sample-factory

SF is known to work on Linux and MacOS. There is no Windows support at this time.

Loading models from the Hub

Using load_from_hub

To download a model from the Hugging Face Hub to use with Sample-Factory, use the load_from_hub script:

python -m sample_factory.huggingface.load_from_hub -r <HuggingFace_repo_id> -d <train_dir_path>

The command line arguments are:

• -r: The repo ID for the HF repository to download from. The repo ID should be in the format <username>/<repo_name>
• -d: An optional argument to specify the directory to save the experiment to. Defaults to ./train_dir which will save the repo to ./train_dir/<repo_name>

Download Model Repository Directly

Hugging Face repositories can be downloaded directly using git clone:

git clone git@hf.co:<Name of HuggingFace Repo> # example: git clone [email protected]:bigscience/bloom

Using Downloaded Models with Sample-Factory

After downloading the model, you can run the models in the repo with the enjoy script corresponding to your environment. For example, if you are downloading a mujoco-ant model, it can be run with:

python -m sf_examples.mujoco.enjoy_mujoco --algo=APPO --env=mujoco_ant --experiment=<repo_name> --train_dir=./train_dir

Note, you may have to specify the --train_dir if your local train_dir has a different path than the one in the cfg.json

Sharing your models

Using push_to_hub

If you want to upload without generating evaluation metrics or a replay video, you can use the push_to_hub script:

python -m sample_factory.huggingface.push_to_hub -r <hf_username>/<hf_repo_name> -d <experiment_dir_path>

The command line arguments are:

• -r: The repo_id to save on HF Hub. This is the same as hf_repository in the enjoy script and must be in the form <hf_username>/<hf_repo_name>
• -d: The full path to your experiment directory to upload

Using enjoy.py

You can upload your models to the Hub using your environment’s enjoy script with the --push_to_hub flag. Uploading using enjoy can also generate evaluation metrics and a replay video.

The evaluation metrics are generated by running your model on the specified environment for a number of episodes and reporting the mean and std reward of those runs.

Other relevant command line arguments are:

• --hf_repository: The repository to push to. Must be of the form <username>/<repo_name>. The model will be saved to https://huggingface.co/<username>/<repo_name>
• --max_num_episodes: Number of episodes to evaluate on before uploading. Used to generate evaluation metrics. It is recommended to use multiple episodes to generate an accurate mean and std.
• --max_num_frames: Number of frames to evaluate on before uploading. An alternative to max_num_episodes
• --no_render: A flag that disables rendering and showing the environment steps. It is recommended to set this flag to speed up the evaluation process.

You can also save a video of the model during evaluation to upload to the hub with the --save_video flag

• --video_frames: The number of frames to be rendered in the video. Defaults to -1 which renders an entire episode
• --video_name: The name of the video to save as. If None, will save to replay.mp4 in your experiment directory

For example:

python -m sf_examples.mujoco_examples.enjoy_mujoco --algo=APPO --env=mujoco_ant --experiment=<repo_name> --train_dir=./train_dir --max_num_episodes=10 --push_to_hub --hf_username=<username> --hf_repository=<hf_repo_name> --save_video --no_render