Well, at some level, every aspect of reality seems to be made of interconnected parts. Atoms and molecules are connected to each other with chemical bonds. Your neurons connect to each other through synapses, and the different parts of your brain connect to each other through groups of neurons interacting with each other. At a larger level, you are interconnected with other humans through social networks, and our economy is a global, interconnected trade network. The Earth’s food chain is an ecological network, and larger still, every object with mass in the universe is connected to every other object through a gravitational network.

6.2. Why embed networks?

rk 𝐀(𝑚)

orthogonal procrustes problem

Vertices

s

Our model could say that this distribution of a Gaussian, for example, in which case we would have to estimate the parameters of this Gaussian

We consider the distribution where our observations are sampled from as really anything

Gaussian

that

that means making a few assumptions about how the data we observed came to be

B

The first is given two graphs, and their corresponding latent positions, are their positions the same?

graphs

scans

there is a difference

thus

𝑟ℎ𝑜

𝑛𝑝𝑟ℎ𝑜=[100,100,100]=⎡⎣⎢⎢0.70.30.40.30.70.30.40.30.7⎤⎦⎥⎥=0.9

<AxesSubplot:title={'center':'A-B [Unshuffled]'}>

𝑎→𝑎a→aa \rightarrow a 𝑏→𝑏b→bb \rightarrow b 𝑐→𝑑c→dc \rightarrow d 𝑑→𝑐

choose a particular percentile, and divide it by 100100100 to obtain the quantile

it might in fact overfit the training data and model spurious noise, which raises the variance

′=12(𝐴+𝐴⊤)=12⎛⎝⎜⎜⎜⎡⎣⎢⎢⎢𝑎11⋮𝑎𝑛1...⋱...𝑎1𝑛⋮𝑎𝑛𝑛⎤⎦⎥⎥⎥+⎡⎣⎢⎢⎢𝑎11⋮𝑎1𝑛...⋱...𝑎𝑛1⋮𝑎𝑛𝑛⎤⎦⎥⎥⎥⎞⎠⎟⎟⎟=⎡⎣⎢⎢⎢12(𝑎11+𝑎11)⋮12(𝑎𝑛1+𝑎1𝑛)...⋱...12(𝑎1𝑛+𝑎𝑛1)⋮12(𝑎𝑛𝑛+𝑎𝑛𝑛)⎤⎦⎥⎥⎥=⎡⎣⎢⎢⎢𝑎11⋮12(𝑎𝑛1+𝑎1𝑛)...⋱...12(𝑎1𝑛+𝑎𝑛1)⋮𝑎𝑛𝑛⎤⎦⎥⎥⎥

𝑎𝑖𝑗

node 𝑖ii being stimulated leading to node 𝑗jj does not necessarily mean that node 𝑗jj being stimulated leads to node 𝑖ii being stimulated.

On wikipedia,

individual

purple

higher

euclidean

likelihood-maximization approach, or a bayes-optimal approach

What The Heck Is Th

text(".8", .25, .75) text(".8", .75, .25) text(".1", .25, .25) text(".1", .75, .75)

Next, we can select a different neighbor of node 𝑖ii, for which there are 𝑑𝑖−1di−1d_i - 1 total. This gives us a triplet consisting of node 𝑖ii, one of 𝑑𝑖did_i possible nodes, and one of 𝑑𝑖−1di−1d_i - 1 possible nodes, since there will exist at least two edges between them (one edge from node 𝑖ii to one of its 𝑑𝑖did_i neighbors, and the other edge from node 𝑖ii to one of its other 𝑑𝑖−1di−1d_i - 1 neighbors). Therefore, the number of open and closed triplets is the quantity ∑𝑖𝑑𝑖(𝑑𝑖−1)∑idi(di−1)\sum_i d_i (d_i - 1).

nodes from our example network. To do this, we will look only at the boroughs Staten Island, Manhattan, Brooklyn, and Queens. Our network looks like this:

Here, we cover some useful quantities we might want to compute about a network.

where 𝑎𝑖𝑗aija_{ij} takes the value of 111 if nodes 𝑖ii and 𝑗jj are connected, and the value 000 if nodes 𝑖ii and 𝑗jj are not connected.

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html warnings.warn(

summing all of the adjacencies corresponding to a potential edge incident a node 𝑖ii:

From the description above, we learned that every edge incident a node 𝑖ii will have 𝑎𝑖𝑗aija_{ij} take the value of one. Therefore,

, so we figured we would make it look extra special with a box.

For most purposes, we will largely be considered with binary networks, which are also more traditionally called

We will now discuss some properties about the network that somewhat simplify the possibilities for 𝐴AA.

To ease this discussion, we will begin with an example that we will use throughout this section.

simplify the possibilities for 𝐴AA.

Thus, 𝐴AA and 𝐵BB are said to be isomorphicisomorphic\textit{isomorphic}.

fig, axs = plt.subplots(1, 3, figsize=(20, 20)) heatmap(A, ax=axs[0], cbar=False, title = r'$A_T$') heatmap(B, ax=axs[1], cbar=False, title = r'$A_F$') heatmap(P@B@P.T, ax=axs[2], cbar=False, title = r'$A_F$

The permutation matrix

fig, axs = plt.subplots(1, 3, figsize=(20, 20)) heatmap(B, ax=axs[0], cbar=False, title = r'Original Matrix $B$') heatmap(P@B, ax=axs[1], cbar=False, title = r'Row Permutation $PB$:') heatmap(B@P.T, ax=axs[2], cbar=False, title = r'Row Permutation $BP^T$:')

where 𝐴=𝐵A=BA=B, that is, the networks are identical

As you can imagine, there are a very large number of these possible mappings

X v_

Using Graspologic¶

v_1

hat’s what the pseudoinverse does: it reverses what it can, and accepts that some info

v_est_proba = X @ v_1

plot_latents

# and an out-of-sample node with the same SBM call

first

arred sections, these sections will assume familiarity with more advanced mathematical and probability concepts. { requestKernel: true, binderOptions: { repo: "binder-examples/jupyter-stacks-datascience", ref: "master", }, codeMirrorConfig: { theme: "abcdef", mode: "python" }, kernelOptions: { kernelName: "python3", path: "./representations/ch5" }, predefinedOutput: true } kernelName = 'python3'

a posteriori Stochastic Block Model, Recap We just covered many details about how to perform statistical inference with a realization of a random network which we think can be well summarized by a Stochastic Block Model. For this reason, we will review some of the key things that were covered, to better put them in context: We learned that the Adjacency Spectral Embedding is a key algorithm for making sense of networks we believe may be realizations of networks which are well-summarized by Stochastic Block Models, as inference on the the estimated latent positions is key for learning about community assignments. We learned how unsupervised learning allows us to use the estimated latent positions to learn community assignments for nodes within our realization. We learned how to align the labels produced by our unsupervised learning technique with true labels in our network, using remap_labels. We learned how to produce community assignments, regardless of whether we know how many communities may be present in the first place. { requestKernel: true, binderOptions: { repo: "binder-examples/jupyter-stacks-datascience", ref: "master", }, codeMirrorConfig: { theme: "abcdef", mode: "python" }, kernelOptions: { kernelName: "python3", path: "./representations/ch6" }, predefinedOutput: true } kernelName = 'python3'

Unlike the SBM example, the scatter plots for the adjacency spectral embedding of a realization of an ER network no longer show the distinct separability into individual communities.

we would not expect the pairs plot to show discernable clusters.

histograms of the indicated values for the indicated dimension.

we will find reasonable “guesses” at community assignments further down the line.

One technique to do so that is particularly useful

it is critical to investigate the quality of an embedding

Simulated SBM($\pi$, B)"

Remember that as we learned in the single network models section, even though the communities eachh node is assigned to look obvious, this is an artifact of the ordering of the nodes.

⃗ π→\vec \pi and 𝐵BB

a similar example to the scenario we had above

Number of communities 𝐾KK is known

the approach we will take will be to use 𝐴AA to produce a best guess as to which community each node of 𝐴AA is from, and then use our best guesses as to which community each node is from to learn about 𝜋⃗ π→\vec \pi and 𝐵BB.

To estimate 𝜋⃗ π→\vec \pi and 𝐵BB,

l = RDPGEstimator(loops=False) # number of latent dimensions is not known model.fit(A) Phat = model.p_mat_

What if we did not know that 𝑑dd was 222 ahead of time

from graphbook_code import plot_latents fig, axs = plt.subplots(1, 2, figsize=(12, 6)) heatmap(Phat, vmin=0, vmax=1, font_scale=1.5, title="$\hat P_{RDPG}$", ax=axs[0]) heatmap(P, vmin=0, vmax=1, font_scale=1.5, title="$P_{RDPG}$", ax=axs[1]) fig;

We will evaluate the performance of the RDPG estimator agaigraspologic.plot the estimated probability matrix, 𝑃̂ =𝑋̂ 𝑋̂ ⊤P^=X^X^⊤\hat P = \hat X \hat X^\top, to the true probability matrix, 𝑃=𝑋𝑋⊤P=XX⊤P = XX^\top.

Let’s simulate an example network:

st 111) and when people are very far apart, we think that they will have a very low probability of being friends (almost 000). We define 𝑋XX to have rows given by: 𝑥⃗ 𝑖=⎡⎣⎢⎢(60−𝑖60)2(𝑖60)2⎤⎦⎥⎥

We define 𝑋XX to have rows given by: 𝑥⃗ 𝑖=⎡⎣⎢⎢(60−𝑖60)2(𝑖60

Let’s assume that we have 606060 people who live along a very long road that is 606060 miles long, and each person is 111 of a mile apart.

The ⋅̂ ⋅^\hat \cdot symbol just means that 𝑑̂ d^\hat d is an estimate of the number of latent dimensions 𝑑dd, and not necessarily the actual number of latent dimension

We might have a reasonable ability to “guess” what 𝑑dd is ahead of time, but this will often not be the case

he estimate of 𝑋XX is produced by using the Adjacency Spectral Embedding, by embedding the observed network 𝐴AA into 𝑑dd (if the number of latent dimensions is known) or 𝑑̂ d^\hat d (if the number of latent dimensions is not known) dimensions.

We estimate 𝑋XX extremely simply for a realization 𝐴AA of a random network 𝐴𝐴AA\pmb A which is characterized using the a priori Random Dot Product Graph.

That expression, it turns out, is a lot more complicated than what we had to deal with for the a priori Stochastic Block Model. Taking the log gives us that:

, in fact, additional steps on top of how we estimate parameters for an a priori Random Dot Product Graph (RDPG).

ℙ𝜃(𝐴)=∑𝜏⃗ ∈∏𝑘=1𝐾[𝜋𝑛𝑘𝑘⋅∏𝑘′=1𝐾𝑏𝑚𝑘′𝑘𝑘′𝑘(1−𝑏𝑘′𝑘)𝑛𝑘′𝑘−𝑚𝑘′𝑘]

original matrix B [[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12] [13 14 15 16]] row permutation: [[ 5 6 7 8] [ 9 10 11 12] [13 14 15 16] [ 1 2 3 4]] column permutation: [[ 2 3 4 1] [ 6 7 8 5] [10 11 12 9] [14 15 16 13]]

Due to the one-to-one nature of these matchings, they are also known as bijectionsbijections\textit{bijections}

𝑓(𝐴,𝐵)=0

As we can see they are very simple, and they are clearly equal to each other.

𝑓(𝐴,𝐵)=||𝐴−𝐵||2𝐹f(A,B)=||A−B||F2f(A, B) = ||A - B||_F^2

Stated another way, our observed network is assumed to be a realization of a governing random network. From now on, when we say the word network without the word random in front of it, we are referring to the realizations of random networks.

are binary

the diagonal is entirely 0

*

et’s try an example of an a priori RDPG. We will use the same example that we used in the single network models section, where we

𝑏𝑙′𝑙logℙ𝜃(𝐴)⇒𝑏∗𝑙′𝑙=0+∂∂𝑏𝑙′𝑙[𝑚𝑙′𝑙log𝑏𝑙′𝑙+(𝑛𝑙′𝑙−𝑚𝑙′𝑙)log(1−𝑏𝑙′𝑙)]=𝑚𝑙′𝑙𝑏𝑙′𝑙−𝑛𝑙′𝑙−𝑚𝑙′𝑙1−𝑏𝑙′𝑙=0=𝑚𝑙′𝑙𝑛𝑙′𝑙

∂∂𝑏𝑙′𝑙logℙ𝜃(𝐴)=∂∂𝑏𝑙′𝑙∑𝑘,𝑘′∈[𝐾]𝑚𝑘′𝑘log𝑏𝑘′𝑘+(𝑛𝑘′𝑘−𝑚𝑘′𝑘)log(1−𝑏𝑘′𝑘)=∑𝑘,𝑘′∈[𝐾]∂∂𝑏𝑙′𝑙[𝑚𝑘′𝑘log𝑏𝑘′𝑘+(𝑛𝑘′𝑘−𝑚𝑘′𝑘)log(1−𝑏𝑘′𝑘)]

ℙ𝜃(𝐴)=∏𝑘,𝑘′∈[𝐾]𝑏𝑚𝑘′𝑘𝑘′𝑘⋅(1−𝑏𝑘′𝑘)𝑛𝑘′𝑘−𝑚𝑘′𝑘Pθ(A)=∏k,k′∈[K]bk′kmk′k⋅(1−bk′k)nk′k−mk′k\begin{align*} \mathbb P_\theta(A) &= \prod_{k, k' \in [K]}b_{k'k}^{m_{k'k}} \cdot (1 - b_{k'k})^{n_{k'k - m_{k'k}}} \end{align*} where 𝑛𝑘′𝑘=∑𝑖<𝑗𝟙𝜏𝑖=𝑘𝟙𝜏𝑗=𝑘′nk′k=∑i<j1τi=k1τj=k′n_{k'k} = \sum_{i < j}\mathbb 1_{\tau_i = k}\mathbb 1_{\tau_j = k'} was the number of possible edges between nodes in community 𝑘kk and 𝑘′k′k', and 𝑚𝑘′𝑘=∑𝑖<𝑗𝟙𝜏𝑖=𝑘𝟙𝜏𝑗=𝑘′𝑎𝑖𝑗mk′k=∑i<j1τi=k1τj=k′aijm_{k'k} = \sum_{i < j}\mathbb 1_{\tau_i = k}\mathbb 1_{\tau_j = k'}a_{ij} was the number of edges in the realization 𝐴AA between nodes within communities 𝑘kk and 𝑘′k′k'.

ve, which we omit since it is rather mathematically tedious, we see that the second derivative at 𝑝∗p∗p^* is negative, so we indeed have found an estimate of the maximum, and will be denoted by 𝑝̂ p^\hat p. This gives that the Maximum Likelihood Estimate (or, the MLE, for short) of the probability 𝑝pp for a random network 𝐀A\mathbf A which is ER is: 𝑝̂ =𝑚(𝑛2)

xt, we take the derivative with respect to 𝑝pp, set equal to 000, and we

𝑑𝑑𝑝logℙ𝜃(𝐴)⇒𝑝∗=𝑚𝑝−(𝑛2)−𝑚1−𝑝=0=𝑚(𝑛2)

logℙ𝜃(𝐴)=log[𝑝𝑚⋅(1−𝑝)(𝑛2)−𝑚]=𝑚log𝑝+((𝑛2)−𝑚)log(1−𝑝)

𝑑𝑑𝑝logℙ(𝐱1=𝑥1,...,𝐱𝑛=𝑥𝑛;𝑝)⇒∑𝑛𝑖=1𝑥𝑖𝑝⇒(1−𝑝)∑𝑖=1𝑛𝑥𝑖∑𝑖=1𝑛𝑥𝑖−𝑝∑𝑖=1𝑛𝑥𝑖⇒𝑝∗=∑𝑛𝑖=1𝑥𝑖𝑝−𝑛−∑𝑛𝑖=1𝑥𝑖1−𝑝=0=𝑛−∑𝑛𝑖=1𝑥𝑖1−𝑝=𝑝(𝑛−∑𝑖=1𝑛𝑥𝑖)=𝑝𝑛−𝑝∑𝑖=1𝑛𝑥𝑖=1𝑛∑𝑖=1𝑛𝑥𝑖

ℙ𝜃(𝐱1=𝑥1,...,𝐱𝑛=𝑥𝑛;𝑝)=∏𝑖=1𝑛ℙ(𝐱𝑖=𝑥𝑖)=∏𝑖=1𝑛𝑝𝑥𝑖(1−𝑝)1−𝑥𝑖=𝑝∑𝑛𝑖=1𝑥𝑖(1−𝑝)𝑛−∑𝑛𝑖=1𝑥𝑖

along our assumed street

as described in the section on spectral embedding

an estimate of 𝑑

which is a real matrix with 𝑛nn rows (one for each node) and 𝑑dd columns (one for each latent dimension).

We estimate 𝑋XX extremely simply for a realization 𝐴AA of a random network 𝐴𝐴AA\pmb A which is characterized using the a priori Random Dot Product Graph.

Whereas the log of a product of terms is the sum of the logs of the terms, no such easy simplification exists for the log of a sum of terms. This means that we will have to get a bit creative here. Instead, we will turn first to the a priori Random Dot Product Graph, and then figure out how to estimate parameters from a a posteriori SBM using that.

ℙ𝜃(𝐴)=∑𝜏⃗ ∈∏𝑘=1𝐾[𝜋𝑛𝑘𝑘⋅∏𝑘′=1𝐾𝑏𝑚𝑘′𝑘𝑘′𝑘(1−𝑏𝑘′𝑘)𝑛𝑘′𝑘−𝑚𝑘′𝑘]Pθ(A)=∑τ→∈T∏k=1K[πknk⋅∏k′=1Kbk′kmk′k(1−bk′k)nk′k−mk′k]\begin{align*} \mathbb P_\theta(A) &= \sum_{\vec \tau \in \mathcal T} \prod_{k = 1}^K \left[\pi_k^{n_k}\cdot \prod_{k'=1}^K b_{k' k}^{m_{k' k}}(1 - b_{k' k})^{n_{k' k} - m_{k' k}}\right] \end{align*} That expression, it turns out, is a lot more complicated than what we had to deal with for the a priori Stochastic Block Model. Taking the log gives us that: logℙ𝜃(𝐴)=log(∑𝜏⃗ ∈∏𝑘=1𝐾[𝜋𝑛𝑘𝑘⋅∏𝑘′=1𝐾𝑏𝑚𝑘′𝑘𝑘′𝑘(1−𝑏𝑘′𝑘)𝑛𝑘′𝑘−𝑚𝑘′𝑘])

array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1], dtype=int32)

𝐴

𝐴

In particular, what this result says is that if we were to look at the sum of 𝐴AA expressed above, and only look at the sum of the first 𝑘kk of those terms, that rank 𝑘kk matrix 𝐴𝑘AkA_k is the most similar rank 𝑘kk matrix to 𝐴AA, according to the Frobenius norm.

What does this result say? This result expresses that the matrix 𝐴

Discussing the below,

Discussing the below, we will delve briefly into the concept of matrix rank, which we will define now.

s

What does 𝑈UU look like?

np.dot(U, np.dot(np.diag(s), Ut))))

Let’s illustrate

hat is, 𝐴∈ℝ𝑛×𝑛A∈Rn×nA \in \mathbb R^{n \times n}, and for any 𝑖,𝑗∈[𝑛]i,j∈[n]i, j \in [n], 𝑎𝑖𝑗=𝑎𝑗𝑖aij=ajia_{ij} = a_{ji}

we would recommend a Linear Algebra textbook [Trefethan, LADR].

This description of the SVD has been modified to fit our purposes: particularly, the description we provide applies only

Note that for these and successive sections, we will present a simplified, and non-rigorous, review of the SVD and many results that are important for developing intuition around this decomposition.

ax=adjplot(A[tuple([vtx_perm])] [:,vtx_perm], meta=meta, color="School", palette="Blues")

it will be obvious that the network will have a modular structure?

ting_context("talk", font_scale=1): ax = sns.heatmap(X, cmap="Purples", ax=ax, cbar_kws=dict(shrink=1), yticklabels=False, xticklabels=False, vmin=0, vmax=1) ax.set_title(titl

import matplotlib.pyplot as plt import seaborn as sns import numpy as np

Next, let’s plot what 𝜏⃗ τ→\vec \tau and 𝐵BB look like: import matplotlib.pyplot as plt import seaborn as sns import numpy as np import matplotlib def plot_tau(tau, title="", xlab="Node"): cmap = matplotlib.colors.ListedColormap(["skyblue", 'blue']) fig, ax = plt.subplots(figsize=(10,2)) with sns.plotting_context("talk", font_scale=1): ax = sns.heatmap((tau - 1).reshape((1,tau.shape[0])), cmap=cmap, ax=ax, cbar_kws=dict(shrink=1), yticklabels=False, xticklabels=False) ax.set_title(title) cbar = ax.collections[0].colorbar cbar.set_ticks([0.25, .75]) cbar.set_ticklabels(['School 1', 'School 2']) ax.set(xlabel=xlab) ax.set_xticks([.5,149.5,299.5]) ax.set_xticklabels(["1", "150", "300"]) cbar.ax.set_frame_on(True) return n = 300 # number of students # tau is a column vector of 150 1s followed by 50 2s # this vector gives the school each of the 300 students are from tau = np.vstack((np.ones((int(n/2),1)), np.full((int(n/2),1), 2))) plot_tau(tau, title="Tau, Node Assignment Vector", xlab="Student")

To describe the a priori SBM, we will use a latent variable model. To do so, we will assume there is some vector-valued random variable, 𝜏𝜏→ττ→\vec{\pmb \tau}, which we will call the node assignment vector. This random variable takes values 𝜏⃗ τ→\vec\tau which are in the space {1,...,𝐾}𝑛{1,...,K}n\{1,...,K\}^n. That means that each element of a realization 𝜏⃗ τ→\vec\tau takes one of 𝐾KK possible values. Each node receives a community assignment, so we say that 𝑖ii goes from 111 to 𝑛nn. Stated another way, each node 𝑖ii of our network receives an assignment 𝜏𝑖τi\tau_i to one of the 𝐾KK communities. This model is called the a priori SBM because we use it when we have a realization 𝜏⃗ τ→\vec\tau that we know ahead of time. In our social network example, for instance, 𝜏𝑖τi\tau_i would reflect that each student can attend one of two possible schools. For a single node 𝑖ii that is in community ℓℓ\ell, where ℓ∈{1,...,𝐾}ℓ∈{1,...,K}\ell \in \{1, ..., K\}, we write that 𝜏𝑖=ℓτi=ℓ\tau_i = \ell.

school 111 or school 222. Our network has 100100100 nodes, and each node represents a single student. The edges of this network represent whether a pair of students are friends. Intuitively, if two students go to the same school, it might make sense to say that they have a higher chance of being friends than if they do not go to the same school. If we were to try to characterize this network using an ER network, we would run into a problem very similar to when we tried to capture the two cluster coin flip example with only a single coin. Intuitively, there must be a better way! The Stochastic Block Model, or SBM, captures this idea by assigning each of the 𝑛nn nodes in the network to one of 𝐾KK communities. A community is a group of

If we were to try to characterize this network using an ER network

we would run into a problem very similar to when we tried to capture the two cluster coin flip example with only a single coin.

. Our

Any particular score m

So now we have a combined representation for our separate embeddings, but we have a new problem: our latent positions suddenly have way too many dimensions. In this example they have eight (the number of columns in our combined matrix), but remember that in general we’d have 𝑚×𝑑m×dm \times d. This somewhat defeats the purpose of an embedding: we took a bunch of high-dimensional objects and turned them all into a single high-dimensional object. Big whoop. We can’t see what our combined embedding look like in euclidean space, unless we can somehow visualize 𝑚×𝑑m×dm \times d dimensional space (hint: we can’t). We’d like to just have d dimensions - that was the whole point of using d components for each of our Adjacency Spectral Embeddings in the first place!

class Axes():

__drawArrow__

reflect

Formally

coordinates:

def plot_lp(X, title="", ylab="Student"):

So, intuitively, it s

𝑥⃗ 1=[10]x→1=[10]\vec x_1 = \begin{bmatrix}1 \\ 0\end{bmatrix},

each 𝑥⃗ 𝑖x→i\vec x_i

Let’s assume, for instance,

Code Examples

We write that X∈Rn×dX \in \mathbb R^{n \times d}, which means that it is a matrix with real values, nn rows, and dd columns.

What is so special about this formulation of the SBM problem?

𝐚𝑖𝑖=0aii=0\mathbf a_{ii} = 0.

𝜏𝑖=ℓτi=ℓ\tau_i = \ell and 𝜏𝑗=𝑘τj=k\tau_j = k, that 𝐚𝑖𝑗∼𝐵𝑒𝑟𝑛(𝑏ℓ𝑘)aij∼Bern(bℓk)\mathbf a_{ij} \sim Bern(b_{\ell k}).

represent each of the 300300300 students in our network

def plot_block(

Say we have 300300300 students, and we know that each student goes to one of two possible schools.

subgraph

by

In the first set of twenty coin flips, all of the coin flips are performed with the same coin. Stated another way, we have a single cluster, or a set of coin flips which are similar. On the other hand, in the second set of twenty coin flips, twenty of the coin flips are performed with a fair coin, and ten of the coin flips are performed with a different coin which is not fair. Here, we have two clusters of coin flips, those that occur with the first coin, and those that occur with the second coin. Since the first cluster of coin flips are with a fair coin, we expect that coin flips from the first cluster will not necessarily have an identical number of heads and tails, but at least a similar number of heads and tails. On the other hand, coin flips from the second cluster will tend to have more heads than tails. What does this example have to do with networks? In the above examples, the two sets of coin flips differ in the number of coins with different probabilities that we use for the example. The first example has only one unique coin, whereas the second example has two unique coins with different probabilities of heads or tails. If we were to assume that the second example had been performed with only a single coin when in reality it was performed with two different coins, we would be unable to capture that the second ten coin flips had a substantially different chance of landing on heads than the first ten coin flips. Just like coin flips can be performed with fundamentally different coins, the nodes of a network could also be fundamentally different. The way in which two nodes differ (or do not differ) sometimes holds value in determining the probability that an edge exists between them.

Imagine that we are flipping a fair single coin.

which had a different probability of seeing heads or tails

Code Examples

𝑗>𝑖j>ij > i

Basically, the approach is to look at each entry of 𝐴AA which can take different values, and multiply the total number of possibilities by 222 for every element which can take different values

The answer is to use something called combinatorics.

If 𝐴AA is 2×22×22 \times 2, there are (22)=1(22)=1\binom{2}{2} = 1 unique entry of 𝐴AA, which takes one of 222 values. There are 222 possible ways that 𝐴AA could look: [0110] or [0000][0110] or [0000]\begin{align*} \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\textrm{ or } \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix} \end{align*} If 𝐴AA is 3×33×33 \times 3, there are (32)=3×22=3(32)=3×22=3\binom{3}{2} = \frac{3 \times 2}{2} = 3 unique entries of 𝐴AA, each of which takes one of 222 values. There are 888 possible ways that 𝐴AA could look: ⎡⎣⎢⎢011101110⎤⎦⎥⎥ or ⎡⎣⎢⎢010101010⎤⎦⎥⎥ or ⎡⎣⎢⎢001001110⎤⎦⎥⎥ or ⎡⎣⎢⎢011100100⎤⎦⎥⎥ or ⎡⎣⎢⎢001000100⎤⎦⎥⎥ or ⎡⎣⎢⎢000001010⎤⎦⎥⎥ or ⎡⎣⎢⎢010100000⎤⎦⎥⎥ or ⎡⎣⎢⎢000000000⎤⎦⎥⎥

≜

n = 300 xs_1 = np.random.normal(size=n) pi = 0.5 ys = np.random.binomial(1, pi, size=n) xs_2 = np.random.normal(loc=ys*5) sex=["Male", "Female"]

by way of your node metadata,

defined

it has clearly delineated communities, which are the vertices that comprise the obvious “squares” in the above adjacency matrix.

vertices

We notice this from the fact that there are more connections between people from school 111 than from school 222.

=False) meta = pd.DataFrame( data = {"School": tau.reshape((n)).astype(int)} ) ax=adjplot(A, meta=meta, color="School", palette="Blues")

meta = pd.DataFrame( data = {"School": tau.reshape((n)).astype(int)} ) ax=adjplot(A, meta=meta, color="School", palette="Blues")

B = np.zeros((K, K)) B[0,0] = .5 B[0,1] = B[1,0] = .2 B[1,1] = .3

0.20.2.

school 222

111 of 222

M),

𝑖ii is male, is a realization of a Gaussian random variable with mean 000 and variance 111, and if 𝑖ii is female, is a realization of a Gaussian random variable with mean 555 and variance 111.

were 𝐸𝑅𝑛(𝑝)ERn(p)ER_n(p)?

For instance, if we see a half of the nodes have a very high degree, and the rest of the nodes with a much lower degree, we can reasonably conclude the network might be more complex than can be described by the ER model.

<ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑠−ℎ𝑖𝑔ℎ𝑙𝑖𝑔ℎ𝑡𝑐𝑙𝑎𝑠𝑠="ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑠−ℎ𝑖𝑔ℎ𝑙𝑖𝑔ℎ𝑡">𝜏⃗ </ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑠−ℎ𝑖𝑔ℎ𝑙𝑖𝑔ℎ𝑡>

The normal brains are all drawn from the same distribution, and the abnormal brains are also all drawn from the same distribution

The goal of MASE is to embed the networks into a single space, with each point in that space representing a single node

However, what you’d really like to do is combine them all into a single representation to learn from every network at once.

our

two possibilities

ubstituting symbol A from STIXNonUnicode

≜

see 𝑎𝑎aa\pmb a, a

let’s imagine we are tossing a coin 100 times, and we want to determine what the probability of the coin landing on heads is.