Roooooo ff4f1cce83 | 4 years ago | |
---|---|---|
virtualTB | 5 years ago | |
.gitignore | 5 years ago | |
ChangeLog.md | 5 years ago | |
README.md | 4 years ago | |
setup.py | 5 years ago |
This project provides VirtualTaobao simulators trained from the real-data of Taobao, one of the largest online retail platforms. In Taobao, when a customer entered some query, the recommondation system returns a list of items according to the query and the customer profile. The system is expected to return a good list such that customers will have high chances of clicking the items.
Using VirtualTaobao simulator, one can access a "live" environment just like the real Taobao environment. Virtual customers will be generated once at a time, the virtual customer starts a query, and the recommendation system needs to return a list of items. The virtual customer will decide if it would like to click the items in the list, similar to a real customer.
How VirtualTaobao was trained is described in
Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, and An-Xiang Zeng. Virtual-Taobao: Virtualizing real-world online retail environment for reinforcement learning. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19), Honolulu, HI, 2019.
We release in this repository a VirtualTaobao model for both the recommondation system research and the reinforcement learning research (see the supervised learning and reinforcement learning use cases below). Anyone can use this simulator freely, but should give proper credit to the above reference.
Currently, VirtualTaobao V0 model (VirtualTB-v0) is provided, which was trained from a middle-scaled anonymized Taobao dataset. More larger models will be released soon.
pip install -e .
Virtual Taobao simulates the customers, items, and recommendation system.
An interactive process between the system and a customer is as follows
In the above process, the model in step 3, is to be trained. The model inputs the features of the customer and the itemset, and outputs a 27-dimensional weight vector.
A data set is prepared at
virtualTB/SupervisedLearning/dataset.txt
Each line of the data set consists an instance of features, labels, and the number of clicks, separated by tab.
To train a model from the data set, the following codes give an demonstration using PyTorch
virtualTB/SupervisedLearning/main.py
which contains a full process from data set loading, model training, and model test.
Here is a simplest example of using VirtualTaobao as an environment for reinforcement learning. A random action is sampled every step to do the recommendation.
import gym
import virtualTB
env = gym.make('VirtualTB-v0')
print(env.action_space)
print(env.observation_space)
print(env.observation_space.low)
print(env.observation_space.high)
state = env.reset()
while True:
env.render()
action = env.action_space.sample()
state, reward, done, info = env.step(action)
if done: break
env.render()
As a more complete example using DDPG reinforcement learning algorithm and PyTorch is placed in
virtualTB/ReinforcementLearning/main.py
This project is an outcome of a joint work of Nanjing University and Alibaba Group, Inc.