Title: Delays, Detours, and Forks in the Road: Latent State Models of Training Dynamics Authors: Michael Y. Hu1 Angelica Chen1 Naomi Saphra1 Kyunghyun Cho Note: This paper seems cool, using older interpretable machine learning models, graphical models to understand what is going on inside a deep neural network

Link: https://arxiv.org/pdf/2308.09543.pdf

提出了新的 LocalNorm。既然Norm都玩得这么嗨了，看来接下来就可以研究小 GeneralizedNorm 或者 AnyRandomNorm 啥的了。。。[doge]

In assignment 1, function svm_loss_naive, there is a 1/2 term before the regularization loss. Where does this come from?

loss += 0.5 reg np.sum(W * W)