I would pass textual features through a language model encoder in order to represent them as dense vectors (using models such as word2vec, BERT, ...) and concatenate the numerical features to the resulting test embeddings.

Layer norm is more adapted when the batch size is small (because then the batch norm metrics are too noisy)

see

Given f,g the chain rule allows to calculate the derivative of f(g(x)): $$ \frac{df}{dx} = \frac{df}{dg} \frac{dg}{dx} $$

derivative of the exterior times derivative of the interior: $$ ( f(g(x) )' = f'(g(x))g'(x) $$

Collaborative filtering is the application of matrix factorization to identify the relationship between items’ and users’ entities

Using Kullback–Leibler divergence:

$$ D_{KL}(P, Q) = \sum P(x) log(\frac{P(x)}{Q(x)}) $$

It might be that comments are not independent with respect to time: spams are not uniformly distributed w.r.t time

Concept drift: distribution of the target changes

Predictors drift: distribution of the predictors changes

Frequentist: probabilities of events are derived from their observed frequencies of occurrence after an infinite number of trials (taking to the limit).

It calculates the probability that the experiment would have the same outcomes if you were to replicate the same conditions again. This model only uses data from the current experiment when evaluating outcomes.

Bayesian: interpret probability distributions as quantifying our uncertainty about the world. In particular, this means that we can now meaningfully talk about probability distributions of parameters, since even though the parameter is fixed, our knowledge of its true value may be limited.

We have to introduce the prior distribution into our analysis - this reflects our belief about the value of p before seeing the actual values of the Xi. The role of the prior is often criticised in the frequentist approach, as it is argued that it introduces subjectivity into the otherwise austere and object world of probability.

0 because X is continuous and the probability of a continuous random variable taking exactly a value is zero (integral of a continuous distribution from a to a is zero)

Consider that as a classification problem so use a regular classification metric such as F1 score

Using the elbow method: we plot sum of squared distances of samples to their closest cluster center versus number of clusters and look at inflection point

That the limit

$$ \frac{ f(x+h) - f(x) }{ h } $$ when h -> 0 exists for every x

Soient f une fonction convexe, (x1, … , xn) un n-uplet de réels appartenant à l'intervalle de définition de f et (λ1, … , λn) un n-uplet de réels positifs tels que:

$$ \sum_{i} \lambda_{i} = 1 $$

Alors

$$ f( \sum_{i} \lambda_{i} x_{i} ) \leq \sum_{i} \lambda_{i} f(x_{i} ) $$

Cause any gradient based algo is certain to find the global extremum at some point

relu activation function is not differentiable at 0

We use the fact that it is piecewise differentiable and connect the pieces derivatives by assuming a value of the derivative where it's not defined (such as relu'(0) = 0)

f(x) = abs(x) at 0

Rank(M) < dim(M) as first column vector and third one are linearly dependent, thus the determinant is 0

The determinant of a matrix is the factor by which areas are scaled by this matrix.

if matrices are linear transformations such that Au = v, matrix inverse A' of A is the transformation such that A'v = u

Only square matrices with non zero determinant have an inverse

Matrix inverse is unique only for bijections

$$ A \lambda (u + v) = \lambda Au + \lambda Av $$

it means that they are not proportional, i.e there is no number c such that u = cv

$$ v = \frac{u}{\mid \mid u \mid \mid} $$

it represents the product of the length of one vector and the projection of another on the former

dot product of two colinear vectors is the product of their length while the dot product of two orthogonal vectors is zero

Because we usually don't know $$ P(x,y) $$ so we take the average loss over the training set, called empirical risk

Expectation of the loss function

$$ R(h) = E[L(h(x), y)] = \int L(h(x), y) dP(x,y) $$