Glad it's useful :)

There are two parts that come into mind when I think of OpenAI, the gym and the baselines.

It would be great to have samples from the baseline working in ue4 maybe use e.g. the deepq model for reinforcement learning. In the examples repository there is a qlearning branch found here (https://github.com/getnamo/tensorflow-ue4-examples/tree/qlearn) which has a basic pong map with qlearning. It should mostly work out of the box, but the model hasn't been successfully trained yet. Maybe using OpenAI there would be useful.

Regarding the gym, this is essentially their 'world' replacement which is functionally similar to what ue4 is, but generally for simpler games. As long as the gym can be installed on windows we should be able to use pip to get things working. I'm not sure how you'd like to use this part in ue4 though?

In terms of pip install, there are many ways to use it. The easiest is perhaps to just add a line in https://github.com/getnamo/tensorflow-ue4/blob/master/Content/Scripts/upymodule.json with your package name and version, when you re-launch your project it will automatically install all packages in that json file. The second way is to open your python console:

you can then just type python commands. There is a pip wrapper called upypip which uses commandline so it doesn't block the thread while installing to use it type e.g.

import upypip as pip

(nb: updated for current api)

then just type e.g. pip.list() to see all the installed dependencies. This is usually what it looks like by default with tensorflow installed

and then you can just type pip.install('gym') and it appears to resolve and install after about a minute

but trying to use it with e.g. an atari game example causes errors

fixing that by installing the atari pre-requisites didn't seem to work either

may need to follow this comment to get it to work properly openai/gym#11 (comment)

That should get you all the parts you need to figure out what you want working with tensorflow and openai in ue4 and how to approach it.

Closed, question is more suitable to https://github.com/getnamo/tensorflow-ue4-examples or the forum thread: https://forums.unrealengine.com/community/work-in-progress/1357673-tensorflow