Jump-Diffusion Dynamics of Constrained SGD: Taming Transient Macroscopic Shocks via Linear Coupled Scheduling

Not scheduled
20m

Speaker

Mr Abdelouakil Fillali (Operationel Research)

Description

Traditional discrete-time approaches to constrained optimization via adap-
tive relaxed barriers impose a strictly positive lower bound on the barrier
parameter to restrict gradient variance. However, this prevalent restriction
introduces a structural bias that precludes achieving exact feasibility. Fur-
thermore, any theoretical attempt to extend classical continuous pure diffu-
sion models to achieve this exact feasibility by letting the barrier parameter
decay to zero invariably breaks down. In this paper, we overcome this funda-
mental limitation by allowing the barrier parameter to decay asymptotically
to zero, paired with a linearly coupled schedule where the learning rate de-
cays proportionally to the barrier parameter. Under this exact-feasibility
regime, the scale of discrete boundary updates remains macroscopic and fails
to vanish asymptotically, thereby violating the Uniform Asymptotic Negligi-
bility (UAN) condition. This dynamical behavior structurally invalidates
standard pure diffusion models and necessitates a transition to a Jump-
Diffusion framework. We derive a Stochastic Modified Equation utilizing
a compensated Poisson measure to rigorously capture these discrete bound-
ary collisions. By evaluating the finite-difference jump energy within the
integro-differential generator, our framework structurally bypasses the weak
approximation gap caused by severe gradient steepness at the boundaries. Crucially, our Foster-Lyapunov stability analysis proves that bounding this
energy injection requires the proposed linearly coupled scheduling to prevent
boundary breakdown and guarantee last-iterate self-stabilization. Finally, we
empirically reveal that in non-convex landscapes, these Poisson shocks act as
endogenous Lévy flights, bypassing Kramers’ law to enable polynomial-time
escape from flat saddle points.

Author

Mr Abdelouakil Fillali (Operationel Research)

Presentation materials

There are no materials yet.