Curator's Take
This article shows that the “grokning” transition and epoch‑wise double‑descent—well‑known quirks of large classical neural nets—also appear in tiny two‑qubit variational circuits, revealing that overparameterised quantum models can suddenly shift from memorising to genuinely generalising. By linking the late‑stage loss of test performance to an unchecked growth of the weight norm and offering a simple norm‑regularisation fix, the work provides one of the first concrete strategies for stabilising training as QML scales toward deeper, more expressive architectures. The findings suggest that many of the practical lessons learned in classical deep learning about depth, regularisation, and hyper‑parameter tuning will be directly relevant to future quantum‑hardware deployments.
— Mark Eatherly
Summary
Grokking, the delayed transition from memorization to generalization, is a fundamental phenomenon in gradient-based learning, yet its dynamics within variational quantum machine learning (QML) remain largely unexamined. In this work, we report the empirical observation of both the grokking transition and epoch-wise double descent in a two-qubit quantum neural network (QNN) under a complete parameterization of the SU(4) manifold. We demonstrate that overparameterization via increased circuit depth improves the probability of successful generalization. Notably, these architectures frequently exhibit an epoch-wise double descent in test error, degrading at a critical epoch before recovering into a generalizing state. Crucially, we identify a generalization decay in late-stage training, where the test error increases significantly despite a stagnant training loss. Bridging this behavior with algorithmic stability theory, our analysis reveals that this decay correlates with an unconstrained increase of the weight-norm, drifting away from sparse, phase-aligned harmonic solutions toward overfitted solutions in the Hilbert space. We analyze the underlying temporal dynamics of this transition, demonstrating how the onset of generalization is linked to optimization hyperparameters such as learning rate and weight decay. Finally, to mitigate late-stage decay, we introduce a weak explicit weight-norm regularization into the loss function. We demonstrate that this structural anchor stabilizes the post-grokking phase and permanently preserves generalization gains, providing a robust framework for training overparameterized quantum circuits.