Markov Chain Approximation of Sticky Diffusions and Hamilton-Jacobi-Bellman Equations on Networks

We propose a discrete Markov-chain approximation of diffusion processes on networks with both Kirchhoff and sticky vertex conditions.

Stickiness is modeled by a probabilistic residence mechanism at the vertex, while the motion along the edges follows an Euler-Maruyama-type update at the diffusive scale.

We prove that the associated time-interpolated chain converges in distribution to the limiting diffusion in the Skorokhod space using the Ethier-Kurtz framework.

Based on this construction, we derive a fully discrete semi-Lagrangian scheme for Hamilton-Jacobi-Bellman equations on networks and establish its convergence using viscosity solution techniques.