Papers

Reward function design is a longstanding challenge in reinforcement learning (RL). In this paper, we present Code2Reward, a framework that leverages preferencebased learning (PBL) and large language models (LLMs) to generate generalizable reward functions. Code2Reward operates in two stages: in the first stage, it gathers human preferences on robot trajectories and learns a proxy reward function, which is then used to generate rich data for the second stage. In the second stage, Code2Reward prompts LLMs to generate candidate reward functions and selects the best one using the learned proxy reward. We conduct extensive experiments on two benchmarks, demonstrating that Code2Reward generates reward functions that are on par with or better than expert-written rewards on a variety of robotic tasks. You can find more information at https://code2reward.io/.

High-dimensional distributed quantile regression (QR) is studied in this paper. To overcome the non-smooth issue of the check loss function, a popular approach is to smooth it. However, the smoothed QR estimator and its inferential procedures require a large minimum local sample size. To address the problem, we propose a new estimator by combining the reversed smoothed check loss and ℓ1-penalization. Theoretically, in terms of estimation, we establish the minimax optimal convergence rate for the global estimator and the valid confidence interval for an individual coefficient. In terms of computation and communication, we show that the proposed iterative algorithm converges linearly for a fixed number of machines and requires only a logarithmic number of communication rounds. Additionally, our theoretical results hold under a weaker condition on the minimum local sample size. Numerical experiments corroborate our theoretical claims.