학술
Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise
arXiv Math
CC BY
이 매체는 공공·자유 라이선스로 본문을 직접 표시합니다.Abstract
This paper establishes, to our knowledge, the first almost sure convergence rate and the first maximal concentration bound with sub-Weibull tails for general contractive stochastic approximation algorithms with Markovian noise.
As a corollary, we also obtain convergence rates in $L^p$.
Key to our successes is a novel discretization of the mean ODE of stochastic approximation algorithms using intervals with diminishing (instead of constant) length.
Applications yield the first almost sure convergence rate for $Q$-learning without count-based learning rates and the first maximal concentration bound for off-policy temporal difference learning, both with Markovian samples.
관련 뉴스
'research' 카테고리 뉴스
Correction: Pupillary dynamics during hands-off L2 driving and transitions of control under high cognitive load
PLOS ONE
Correction: Ornamental motifs and visual concerns: Exploring aesthetic pleasure and conservation intention in architectural heritage renovation
PLOS ONE
Correction: Who is disrupting whom? Examining the nighttime activity patterns of humans and their dogs
PLOS ONE