[RLLib] Want to update PolicyClient and PolicyServerInput to the new API stack #46430
Labels
enhancement
Request for new feature and/or capability
P2
Important issue, but not time-critical
rllib
RLlib related issues
rllib-client-server
Issue related to RLlib's client/server API.
rllib-newstack
Description
My understanding is that
PolicyServerInput
andPolicyClient
don't work with the new API stack, which appears to be due to their use of the olderRolloutWorker
instead ofEnvRunner
s.The ability to hook in non-gym environments using this interface has been very valuable to me so far, and I'm interested in making an attempt to update these modules so that we can use the latest algorithms with them.
Currently, I have identified that mainly
PolicyServer
itself will need some updating, in addition toIOContext
(it currently only takes aRolloutWorker
in its__init__
method).My main question about this is, could I get some general guidance on how to update the modules in question without making breaking changes to other parts of the project? My main concern is that
IOContext
could touch other code, although I'm relatively sure thatPolicyServerInput
andPolicyClient
can be updated without causing issues elsewhere.Thank you in advance for considering my request!
Use case
The PolicyServerInput feature is really useful for cases in which wrapping the environment in a gym interface is difficult. It would be nice to make sure it gets updated to the new API stack so everyone can continue using it.
The text was updated successfully, but these errors were encountered: