8.4.2 Bayesian Inference
One of the main differences between frequentist and Bayesian statistics is Bayesians assume the parameters themselves has a distribution and that previous information can help make predictions. Frequentists treat parameters as fixed and use repeated sampling/long running frequencies to make predictions
he regularization term is sometimes called the penalty term
Regularization is a technique that penalizes covariates when we have a lot of features. It either shrinks covariates or zeros out covariates completely. For smoother function we want a larger penalty term to shrink coefficients/
We assume that our data can be read by a computer, and represented ade-quately in a numerical format.
Understanding matrices as data structures makes linear algebra feel much more intuitive to me. Thinking of rows as observations and columns as variables helps explain why the column space represents all possible linear combinations of attributes, essentially the space where our predictions or projections live. It also connects nicely to concepts like eigenvalues and eigenvectors, where the directions of highest variance in the data correspond to the principal components or dominant eigenvectors of the covariance matrix.
“What dowe mean by good models?
I think what is meant by a good model also differs on what the goal of the model is. For functional modelling we might be more interested in the relationship between the attributes and how they relate to each other, while for prediction, having a model that can predict on unseen data would matter more than interpretability.