diff --git a/cards/communication/prep/index.html b/cards/communication/prep/index.html index ff12f25db..fc79e8526 100644 --- a/cards/communication/prep/index.html +++ b/cards/communication/prep/index.html @@ -25,8 +25,9 @@ #Soft Skills

PREP

PREP is a framework for making your point.

PREP: Point + Reason + Example + Point
  1. Point: Make a point; PREP is a good method.
  2. Reason: Give the reason; Because it has a clear logic.
  3. Example: Show examples; The famous XYZ did ABC then everyone was convinced.
  4. Point: State the point for a conclusion.

Planted: -by ;

Dynamic Backlinks:
PREP -
PREP is a framework for making your point.
Links to:
SCQ-A +by ;

Dynamic Backlinks to cards/communication/prep:
SCQ-A +
SCQ-A is a framework for problem solving.
WWH +
WWH WWH: What (happened) + Why (this happened) + How (to improve)
Links to:
SCQ-A
SCQ-A is a framework for problem solving.
WWH
WWH WWH: What (happened) + Why (this happened) + How (to improve)

L Ma (2021). 'PREP', Datumorphism, 01 April. Available at: https://datumorphism.leima.is/cards/communication/prep/.

diff --git a/cards/communication/scq-a/index.html b/cards/communication/scq-a/index.html index 751902fd7..4079e965e 100644 --- a/cards/communication/scq-a/index.html +++ b/cards/communication/scq-a/index.html @@ -26,8 +26,9 @@ #Soft Skills

SCQ-A

SCQ-A: Situation + Conflict + Question + Answer

SCA-A is a framework for problem-solving.

  1. Situation: background knowledge, set the stage
  2. Complications: what is happening
  3. Question: propose your hypothesis
  4. Answer: accept or reject the hypothesis

Planted: -by ;

Dynamic Backlinks:
SCQ-A -
SCQ-A is a framework for problem solving.
Links to:
PREP +by ;

Dynamic Backlinks to cards/communication/scq-a:
PREP +
PREP is a framework for making your point.
WWH +
WWH WWH: What (happened) + Why (this happened) + How (to improve)
Links to:
PREP
PREP is a framework for making your point.
WWH
WWH WWH: What (happened) + Why (this happened) + How (to improve)

L Ma (2021). 'SCQ-A', Datumorphism, 01 April. Available at: https://datumorphism.leima.is/cards/communication/scq-a/.

WWH

WWH: What (happened) + Why (this happened) + How (to improve)

Planted: -by ;

Dynamic Backlinks:
WWH -
WWH WWH: What (happened) + Why (this happened) + How (to improve)
Links to:
PREP +by ;

Dynamic Backlinks to cards/communication/wwh:
PREP +
PREP is a framework for making your point.
SCQ-A +
SCQ-A is a framework for problem solving.
Links to:
PREP
PREP is a framework for making your point.
SCQ-A
SCQ-A is a framework for problem solving.

L Ma (2021). 'WWH', Datumorphism, 01 April. Available at: https://datumorphism.leima.is/cards/communication/wwh/.

diff --git a/cards/forecasting/gnn-graph-structure-learning/index.html b/cards/forecasting/gnn-graph-structure-learning/index.html index 27c81de35..a39872dab 100644 --- a/cards/forecasting/gnn-graph-structure-learning/index.html +++ b/cards/forecasting/gnn-graph-structure-learning/index.html @@ -33,13 +33,8 @@  ↩︎ ↩︎

Planted: by ;

References:
  1. Wu2020 -Wu Z, Pan S, Long G, Jiang J, Chang X, Zhang C. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2005.11650
Similar Articles:
Mix-hop Propagation in GNN -
Mix-hop is a strategy to avoid oversmoothing in GNN
Time Convolution -
The temporal convolution is responsible for capturing temporal patterns in a sequence.
Empirical Correlation Coefficient (CORR) -
The Empirical Correlation Coefficient (CORR) is an evaluation metric in time series forecasting,1 -$$ …
Root Relative Squared Error (RSE) -
The Root Relative Squared Error (RSE) is an evaluation metric in time series forecasting,1 -$$ …
Dynamic Backlinks to cards/forecasting/gnn-graph-structure-learning:
Graph Structure Learning in GNN +
We can learn a graph structure without prior knowledge

L Ma (2022). 'Graph Structure Learning in GNN', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/forecasting/gnn-graph-structure-learning/.

diff --git a/cards/forecasting/gnn-mix-hop-propagation/index.html b/cards/forecasting/gnn-mix-hop-propagation/index.html index 0545f2f2d..426cdf6dc 100644 --- a/cards/forecasting/gnn-mix-hop-propagation/index.html +++ b/cards/forecasting/gnn-mix-hop-propagation/index.html @@ -33,13 +33,8 @@  ↩︎ ↩︎

Planted: by ;

References:
  1. Wu2020 -Wu Z, Pan S, Long G, Jiang J, Chang X, Zhang C. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2005.11650
Similar Articles:
Graph Structure Learning in GNN -
We can learn a graph structure without prior knowledge
Time Convolution -
The temporal convolution is responsible for capturing temporal patterns in a sequence.
Empirical Correlation Coefficient (CORR) -
The Empirical Correlation Coefficient (CORR) is an evaluation metric in time series forecasting,1 -$$ …
Root Relative Squared Error (RSE) -
The Root Relative Squared Error (RSE) is an evaluation metric in time series forecasting,1 -$$ …
Dynamic Backlinks to cards/forecasting/gnn-mix-hop-propagation:
Mix-hop Propagation in GNN +
Mix-hop is a strategy to avoid oversmoothing in GNN

L Ma (2022). 'Mix-hop Propagation in GNN', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/forecasting/gnn-mix-hop-propagation/.

diff --git a/cards/forecasting/prediction-space/index.html b/cards/forecasting/prediction-space/index.html index 34db9630c..3b2742d72 100644 --- a/cards/forecasting/prediction-space/index.html +++ b/cards/forecasting/prediction-space/index.html @@ -37,11 +37,9 @@  ↩︎

Planted: by ;

References:
  1. Gneiting2014 -Gneiting T, Katzfuss M. Probabilistic Forecasting. Annu Rev Stat Appl. 2014;1: 125–151. doi:10.1146/annurev-statistics-062713-085831
Similar Articles:
Continuous Ranked Probability Score - CRPS -
The Continuous Ranked Probability Score, known as CRPS, is a score to measure how a proposed …
Evaluating Time Series Models -
Evaluate time series models
Hidden Markov Model -
The hidden Markov model, HMM, is a type of [[State Space Models]] State Space Models The state space …
State Space Models -
The state space model is an important category of models for sequential data such as time series
Dynamic Backlinks to cards/forecasting/prediction-space:
Prediction Space in Forecasting +
In a forecasting problem, we have +$\mathcal P$, the priors, e.g., price and demand is negatively …

L Ma (2022). 'Prediction Space in Forecasting', Datumorphism, 04 April. Available at: https://datumorphism.leima.is/cards/forecasting/prediction-space/.

diff --git a/cards/forecasting/time-convolution/index.html b/cards/forecasting/time-convolution/index.html index 11cf3e830..d299bc14d 100644 --- a/cards/forecasting/time-convolution/index.html +++ b/cards/forecasting/time-convolution/index.html @@ -40,14 +40,8 @@ Planted: by ;

References:
  1. Wu2020 Wu Z, Pan S, Long G, Jiang J, Chang X, Zhang C. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2005.11650
  2. Lässig2021 -Lässig F. Temporal Convolutional Networks and Forecasting. In: Unit8 [Internet]. 6 Jul 2021 [cited 28 Nov 2022]. Available: https://unit8.com/resources/temporal-convolutional-networks-and-forecasting/
Similar Articles:
Empirical Correlation Coefficient (CORR) -
The Empirical Correlation Coefficient (CORR) is an evaluation metric in time series forecasting,1 -$$ …
Root Relative Squared Error (RSE) -
The Root Relative Squared Error (RSE) is an evaluation metric in time series forecasting,1 -$$ …
Prediction Space in Forecasting -
In a forecasting problem, we have -$\mathcal P$, the priors, e.g., price and demand is negatively …
Continuous Ranked Probability Score - CRPS -
The Continuous Ranked Probability Score, known as CRPS, is a score to measure how a proposed …
Additional Double Backet Links:
Convolutions Using Fourier Transform +Lässig F. Temporal Convolutional Networks and Forecasting. In: Unit8 [Internet]. 6 Jul 2021 [cited 28 Nov 2022]. Available: https://unit8.com/resources/temporal-convolutional-networks-and-forecasting/
Dynamic Backlinks to cards/forecasting/time-convolution:
Time Convolution +
The temporal convolution is responsible for capturing temporal patterns in a sequence.
Additional Double Backet Links:
Convolutions Using Fourier Transform
Convolution and Fourier transform
Dilated Convolution

Planted: by ;

References:
  1. wiki_adjacency_matrix -Contributors to Wikimedia projects. Adjacency matrix. In: Wikipedia [Internet]. 4 Oct 2021 [cited 4 Dec 2021]. Available: https://en.wikipedia.org/wiki/Adjacency_matrix#Undirected_graphs
Dynamic Backlinks:
Graph Adjacency Matrix -
A graph $\mathcal G$ can be represented with an adjacency matrix $\mathbf A$. There are some nice …
Links to:
What is Graph +Contributors to Wikimedia projects. Adjacency matrix. In: Wikipedia [Internet]. 4 Oct 2021 [cited 4 Dec 2021]. Available: https://en.wikipedia.org/wiki/Adjacency_matrix#Undirected_graphs
Dynamic Backlinks to cards/graph/graph-adjacency-matrix:
Graph Adjacency Matrix +
A graph $\mathcal G$ can be represented with an adjacency matrix $\mathbf A$. There are some nice …
Graph Convolution Operator +
For a given graph $\mathcal G$, we have an attribute on each node, denoted as $f_v$. All the node …
Graph Global Overlap Measure: Katz Index +
The Katz index is +$$ \mathbf S_{\text{Katz}}[u,v] = \sum_{i=1}^\infty \beta^i \mathbf A^i[u, v], $$ …
Graph Global Overlap Measure: Leicht-Holme-Newman Index +
The LHN index is a normalized similarity index. +From Katz Index to LHN Index [[Katz Index]] Graph …
Graph Global Overlap Measure: Random Walk Similarity +
Random Walk Construct a stochastic transfer matrix $P$ by normalizing the adjacency matrix $\mathbf …
Graph Laplacians +
Laplacian is a useful representation of graphs. The unnormalized Laplacian is +$$ \mathbf L = \mathbf …
Links to:
What is Graph
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, $$ \mathcal G = ( \mathcal …

Lei Ma (2021). 'Graph Adjacency Matrix', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-adjacency-matrix/.

diff --git a/cards/graph/graph-betweenness-centrality/index.html b/cards/graph/graph-betweenness-centrality/index.html index eaa1cf487..43804fa34 100644 --- a/cards/graph/graph-betweenness-centrality/index.html +++ b/cards/graph/graph-betweenness-centrality/index.html @@ -39,8 +39,8 @@ $$

where $\sigma_{u_su_t}(v)$ is the number of shortest path between $u_s$ and $u_t$, and passing through $u$, while $\sigma_{u_su_t}$ is the number of shortest path between $u_s$ and $u_t$.

A figure from wikipedia demonstrates this idea well. The nodes on the outreach have smaller betweenness centrality, while the nodes in the core have higher betweenness centrality.

Source: Wikipedia

Source: Wikipedia

Outreach and Core

It is almost like cheating using the work “outreach” and “core” here. They are somewhat defined by betweenness centrality.

Planted: by ;

References:
  1. BetweennessCentrality -Contributors to Wikimedia projects. Betweenness centrality. In: Wikipedia [Internet]. 21 Aug 2021 [cited 26 Sep 2021]. Available: https://en.wikipedia.org/wiki/Betweenness_centrality
Dynamic Backlinks:
Betweenness Centrality of a Graph -
Betweenness centrality of a node $v$ is measurement of how likely the shortest path between two …
Links to:
What is Graph +Contributors to Wikimedia projects. Betweenness centrality. In: Wikipedia [Internet]. 21 Aug 2021 [cited 26 Sep 2021]. Available: https://en.wikipedia.org/wiki/Betweenness_centrality
Dynamic Backlinks to cards/graph/graph-betweenness-centrality:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
Links to:
What is Graph
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, $$ \mathcal G = ( \mathcal …

L Ma (2021). 'Betweenness Centrality of a Graph', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-betweenness-centrality/.

Planted: -by ;

Dynamic Backlinks:
Graph Convolution Operator -
For a given graph $\mathcal G$, we have an attribute on each node, denoted as $f_v$. All the node …
Links to:
What is Graph +by ;

Dynamic Backlinks to cards/graph/graph-convolution-operator:
Graph Laplacians +
Laplacian is a useful representation of graphs. The unnormalized Laplacian is +$$ \mathbf L = \mathbf …
Links to:
What is Graph
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, $$ \mathcal G = ( \mathcal …
Graph Adjacency Matrix
A graph $\mathcal G$ can be represented with an adjacency matrix $\mathbf A$. There are some nice …
Convolutions Using Fourier Transform diff --git a/cards/graph/graph-cuts/index.html b/cards/graph/graph-cuts/index.html index 61ef751fe..4acb98138 100644 --- a/cards/graph/graph-cuts/index.html +++ b/cards/graph/graph-cuts/index.html @@ -87,8 +87,10 @@  ↩︎

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Cuts -
Cut For a subset of nodes $\mathcal A\subset \mathcal V$, the rest of nodes can be denoted as $\bar …
Dynamic Backlinks to cards/graph/graph-cuts:
Graphs Spectral Methods +
The [[Ratio Cut]] Graph Cuts Cut For a subset of nodes $\mathcal A\subset \mathcal V$, the rest of …
Graph Laplacians +
Laplacian is a useful representation of graphs. The unnormalized Laplacian is +$$ \mathbf L = \mathbf …

L Ma (2021). 'Graph Cuts', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-cuts/.

diff --git a/cards/graph/graph-eigenvector-centrality/index.html b/cards/graph/graph-eigenvector-centrality/index.html index d090b77a2..7e08edc81 100644 --- a/cards/graph/graph-eigenvector-centrality/index.html +++ b/cards/graph/graph-eigenvector-centrality/index.html @@ -45,9 +45,9 @@ $$

$e^{(0)} _ j$ is the probability of visiting node $j$, which is 1. $A_{ij}$ is the transfer probability from node $j$ to $i$. After this iteration, we get the number of visits on node $i$.

Repeat this, we have a vector that shows the number of visits on node $i$ after $t+1$ steps.

That being said, the eigenvector centrality is proportional to the likelihood of visiting the nodes after infinite steps of random walks on the graph.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Eigenvector Centrality of a Graph -
Given a graph with adjacency matrix $\mathbf A$, the eigenvector centrality is -$$ \mathbf e_u = …
Links to:
What is Graph +Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks to cards/graph/graph-eigenvector-centrality:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
Graph Global Overlap Measure: Random Walk Similarity +
Random Walk Construct a stochastic transfer matrix $P$ by normalizing the adjacency matrix $\mathbf …
Links to:
What is Graph
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, $$ \mathcal G = ( \mathcal …

L Ma (2021). 'Eigenvector Centrality of a Graph', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-eigenvector-centrality/.

References:
  1. Katz1953 Katz L. A new status index derived from sociometric analysis. Psychometrika. 1953;18: 39–43. doi:10.1007/BF02289026
  2. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Global Overlap Measure: Katz Index -
The Katz index is -$$ \mathbf S_{\text{Katz}}[u,v] = \sum_{i=1}^\infty \beta^i \mathbf A^i[u, v], $$ …
Links to:
Graph Adjacency Matrix +Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks to cards/graph/graph-global-overlap-katz-index:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
Graph Global Overlap Measure: Leicht-Holme-Newman Index +
The LHN index is a normalized similarity index. +From Katz Index to LHN Index [[Katz Index]] Graph …
Links to:
Graph Adjacency Matrix
A graph $\mathcal G$ can be represented with an adjacency matrix $\mathbf A$. There are some nice …

L Ma (2021). 'Graph Global Overlap Measure: Katz Index', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-global-overlap-katz-index/.

diff --git a/cards/graph/graph-global-overlap-leicht-holme-newman-similarity/index.html b/cards/graph/graph-global-overlap-leicht-holme-newman-similarity/index.html index 23ab74a58..2e9970366 100644 --- a/cards/graph/graph-global-overlap-leicht-holme-newman-similarity/index.html +++ b/cards/graph/graph-global-overlap-leicht-holme-newman-similarity/index.html @@ -52,9 +52,8 @@ Planted: by ;

References:
  1. Lue2010 Lu L, Zhou T. Link Prediction in Complex Networks: A Survey. arXiv [physics.soc-ph]. 2010. Available: http://arxiv.org/abs/1010.0725
  2. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Global Overlap Measure: Leicht-Holme-Newman Index -
The LHN index is a normalized similarity index. -From Katz Index to LHN Index [[Katz Index]] Graph …
Links to:
Graph Adjacency Matrix +Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks to cards/graph/graph-global-overlap-leicht-holme-newman-similarity:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
Links to:
Graph Adjacency Matrix
A graph $\mathcal G$ can be represented with an adjacency matrix $\mathbf A$. There are some nice …
Graph Global Overlap Measure: Katz Index
The Katz index is $$ \mathbf S_{\text{Katz}}[u,v] = \sum_{i=1}^\infty \beta^i \mathbf A^i[u, v], $$ …

L Ma diff --git a/cards/graph/graph-global-overlap-random-walk-similarity/index.html b/cards/graph/graph-global-overlap-random-walk-similarity/index.html index 20e8cf2e5..cb791bffb 100644 --- a/cards/graph/graph-global-overlap-random-walk-similarity/index.html +++ b/cards/graph/graph-global-overlap-random-walk-similarity/index.html @@ -56,8 +56,8 @@  ↩︎ ↩︎

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Global Overlap Measure: Random Walk Similarity -
Random Walk Construct a stochastic transfer matrix $P$ by normalizing the adjacency matrix $\mathbf …
Links to:
Graph Adjacency Matrix +Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks to cards/graph/graph-global-overlap-random-walk-similarity:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
Links to:
Graph Adjacency Matrix
A graph $\mathcal G$ can be represented with an adjacency matrix $\mathbf A$. There are some nice …
Eigenvector Centrality of a Graph
Given a graph with adjacency matrix $\mathbf A$, the eigenvector centrality is $$ \mathbf e_u = …

L Ma diff --git a/cards/graph/graph-laplacians/index.html b/cards/graph/graph-laplacians/index.html index e0a6ab806..1901fc87a 100644 --- a/cards/graph/graph-laplacians/index.html +++ b/cards/graph/graph-laplacians/index.html @@ -55,9 +55,11 @@ Planted: by ;

References:
  1. Hamilton2020 Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
  2. Li2014 -Li J, Guo J-M, Shiu WC. Bounds on normalized Laplacian eigenvalues of graphs. J Inequal Appl. 2014;2014: 1–8. doi:10.1186/1029-242X-2014-316
Dynamic Backlinks:
Graph Laplacians -
Laplacian is a useful representation of graphs. The unnormalized Laplacian is -$$ \mathbf L = \mathbf …
Links to:
Graph Adjacency Matrix +Li J, Guo J-M, Shiu WC. Bounds on normalized Laplacian eigenvalues of graphs. J Inequal Appl. 2014;2014: 1–8. doi:10.1186/1029-242X-2014-316
Dynamic Backlinks to cards/graph/graph-laplacians:
What is Graph +
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, +$$ \mathcal G = ( \mathcal …
Graphs Spectral Methods +
The [[Ratio Cut]] Graph Cuts Cut For a subset of nodes $\mathcal A\subset \mathcal V$, the rest of …
Graph Convolution Operator +
For a given graph $\mathcal G$, we have an attribute on each node, denoted as $f_v$. All the node …
Links to:
Graph Adjacency Matrix
A graph $\mathcal G$ can be represented with an adjacency matrix $\mathbf A$. There are some nice …
Graph Cuts
Cut For a subset of nodes $\mathcal A\subset \mathcal V$, the rest of nodes can be denoted as $\bar …
Convolutions Using Fourier Transform
Convolution and Fourier transform
Graph Convolution Operator diff --git a/cards/graph/graph-local-clustering-coefficient/index.html b/cards/graph/graph-local-clustering-coefficient/index.html index e6da855de..cc57ba7cc 100644 --- a/cards/graph/graph-local-clustering-coefficient/index.html +++ b/cards/graph/graph-local-clustering-coefficient/index.html @@ -35,8 +35,8 @@ $$

where $\color{red}{d_n \choose 2}$ means all the possible combinations of neighbor nodes, and $\mathcal N(u)$ is the set of nodes that are neighbor to $u$.

Closed Triangles

Ego Graph

Counting the closed triangles of the ego graph of a node and normalize it by the total possible number of triangles is also a measure of clustering coefficients.

  • If the ego graph of $u$ is fully connected, we have $c_u=1$;
  • If the ego graph of $u$ is a star, we have $c_u=0$.

This concept can be generalized from triangles to any type of motifs.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Clustering Coefficient -
Local Clustering Coefficient $$ c_u = \frac{ \lvert (v_1,v_2)\in \mathcal E: v_1, v_2 \in \mathcal …
Links to:
What is Graph +Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks to cards/graph/graph-local-clustering-coefficient:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
Links to:
What is Graph
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, $$ \mathcal G = ( \mathcal …

L Ma (2021). 'Graph Clustering Coefficient', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-local-clustering-coefficient/.

References:
  1. Adamic2003 Adamic LA, Adar E. Friends and neighbors on the Web. Soc Networks. 2003;25: 211–230. doi:10.1016/S0378-8733(03)00009-1
  2. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Local Overlap Measure: Adamic Adar Index -
The Adamic Adar (AA) index is1 -$$ \mathbf S_{\text{AA}}[v_1,v_2] = \sum_{u\in\mathcal N(u) \cap …
Dynamic Backlinks to cards/graph/graph-local-overlap-adamic-adar-index:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …

L Ma (2021). 'Graph Local Overlap Measure: Adamic Adar Index', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-local-overlap-adamic-adar-index/.

diff --git a/cards/graph/graph-local-overlap-resource-allocation-index/index.html b/cards/graph/graph-local-overlap-resource-allocation-index/index.html index 4e07079fb..d88e69b64 100644 --- a/cards/graph/graph-local-overlap-resource-allocation-index/index.html +++ b/cards/graph/graph-local-overlap-resource-allocation-index/index.html @@ -32,9 +32,8 @@ $$

where $d_u$ is the node degree of node $u$ and $\mathcal N(u)$ is the neighbor nodes of $u$.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Local Overlap Measure: Resource Allocation Index -
The Resource Allocation (RA) index is -$$ \mathbf S_{\text{RA}}[v_1,v_2] = \sum_{u\in\mathcal N(u) …
Dynamic Backlinks to cards/graph/graph-local-overlap-resource-allocation-index:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …

L Ma (2021). 'Graph Local Overlap Measure: Resource Allocation Index', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-local-overlap-resource-allocation-index/.

diff --git a/cards/graph/graph-local-overlap-salton-index/index.html b/cards/graph/graph-local-overlap-salton-index/index.html index 8b1feb65c..b6b8fc601 100644 --- a/cards/graph/graph-local-overlap-salton-index/index.html +++ b/cards/graph/graph-local-overlap-salton-index/index.html @@ -32,9 +32,8 @@ $$

where $d_u$ is the node degree of node $u$ and $\mathcal N(u)$ is the neighbor nodes of $u$.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Local Overlap Measure: Salton Index -
The Salton index is -$$ \mathbf S_{\text{Salton}}[u,v] = \frac{ 2\lvert \mathcal N (u) \cap \mathcal …
Dynamic Backlinks to cards/graph/graph-local-overlap-salton-index:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …

L Ma (2021). 'Graph Local Overlap Measure: Salton Index', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-local-overlap-salton-index/.

diff --git a/cards/graph/graph-local-overlap-sorensen-index/index.html b/cards/graph/graph-local-overlap-sorensen-index/index.html index 74150aaf0..49b98dddc 100644 --- a/cards/graph/graph-local-overlap-sorensen-index/index.html +++ b/cards/graph/graph-local-overlap-sorensen-index/index.html @@ -32,9 +32,8 @@ $$

where $d_u$ is the node degree of node $u$ and $\mathcal N(u)$ is the neighbor nodes of $u$.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Graph Local Overlap Measure: Sorensen Index -
The Sorensen index is -$$ \mathbf S_{\text{Sorensen}}[u,v] = \frac{ 2\lvert \mathcal N (u) \cap …
Dynamic Backlinks to cards/graph/graph-local-overlap-sorensen-index:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …

L Ma (2021). 'Graph Local Overlap Measure: Sorensen Index', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/graph-local-overlap-sorensen-index/.

diff --git a/cards/graph/graph-weisfeiler-lehman-kernel/index.html b/cards/graph/graph-weisfeiler-lehman-kernel/index.html index c73a3cbbc..06fa2b51c 100644 --- a/cards/graph/graph-weisfeiler-lehman-kernel/index.html +++ b/cards/graph/graph-weisfeiler-lehman-kernel/index.html @@ -37,9 +37,10 @@ Planted: by ;

References:
  1. Shervashidze2011 Shervashidze N, Schweitzer P, van Leeuwen EJ, Mehlhorn K, Borgwardt KM. Weisfeiler-Lehman Graph Kernels. J Mach Learn Res. 2011;12: 2539–2561. Available: https://dl.acm.org/doi/10.5555/1953048.2078187
  2. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Weisfeiler-Lehman Kernel +Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks to cards/graph/graph-weisfeiler-lehman-kernel:
Weisfeiler-Lehman Kernel
The Weisfeiler-Lehman kernel is an iterative integration of neighborhood information. -We initialize …
Links to:
What is Graph +We initialize …
Graph Isomorphism +
For two graphs, $\mathcal G$ and $\mathcal H$, the two graphs are isomorphism on the following …
Links to:
What is Graph
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, $$ \mathcal G = ( \mathcal …
Additional Double Backet Links:
Multiset, mset or bag

Heterophily is the tendency to differ from others. Heterophily on a graph is the tendency to connect to nodes that are different from itself, e.g., nodes with different attributes have higher probability of edge.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Heterophily on Graph -
Heterophily is the tendency to differ from others. Heterophily on a graph is the tendency to connect …
Dynamic Backlinks to cards/graph/heterophily:
Solving Problems on Graph +
Graphs can be used in many problem and there are many possible problems on graphs. We will mention a …

L Ma (2021). 'Heterophily on Graph', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/heterophily/.

diff --git a/cards/graph/homophily/index.html b/cards/graph/homophily/index.html index 88a4e78e0..50b804141 100644 --- a/cards/graph/homophily/index.html +++ b/cards/graph/homophily/index.html @@ -30,8 +30,8 @@  ↩︎

Planted: by ;

References:
  1. McPherson2001 -McPherson M, Smith-Lovin L, Cook JM. Birds of a Feather: Homophily in Social Networks. Annu Rev Sociol. 2001;27: 415–444. doi:10.1146/annurev.soc.27.1.415
Dynamic Backlinks:
Homophily on Graph -
Homophily is the principle that a contact between similar people occurs at ahigher rate than among …
Dynamic Backlinks to cards/graph/homophily:
Solving Problems on Graph +
Graphs can be used in many problem and there are many possible problems on graphs. We will mention a …

L Ma (2021). 'Homophily on Graph', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/homophily/.

diff --git a/cards/graph/node-degree/index.html b/cards/graph/node-degree/index.html index 6a7cd59ef..139907821 100644 --- a/cards/graph/node-degree/index.html +++ b/cards/graph/node-degree/index.html @@ -31,10 +31,8 @@ $$

where $A$ is the adjacency matrix.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Node Degree -
Node degree of a node $u$ -$$ d_u = \sum_{v\in \mathcal V} A[u,v], $$ -where $A$ is the adjacency …
Links to:
What is Graph +Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks to cards/graph/node-degree:
Statistics of Graphs +
Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
Links to:
What is Graph
Graph A graph $\mathcal G$ has nodes $\mathcal V$ and edges $\mathcal E$, $$ \mathcal G = ( \mathcal …

L Ma (2021). 'Node Degree', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/node-degree/.

Structural Equivalence means that nodes with similar neighborhood structures will share similar attributes.

Planted: by ;

References:
  1. Hamilton2020 -Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
Dynamic Backlinks:
Structural Equivalence on Graph -
Structural Equivalence means that nodes with similar neighborhood structures will share similar …
Dynamic Backlinks to cards/graph/structural-equivalence:
Solving Problems on Graph +
Graphs can be used in many problem and there are many possible problems on graphs. We will mention a …

L Ma (2021). 'Structural Equivalence on Graph', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/graph/structural-equivalence/.

diff --git a/cards/information/coding-theory-concepts/index.html b/cards/information/coding-theory-concepts/index.html index 78bb25eec..105ecb348 100644 --- a/cards/information/coding-theory-concepts/index.html +++ b/cards/information/coding-theory-concepts/index.html @@ -40,8 +40,14 @@ \mathcal H = - \sum_x p(x\in X) \log_2 p(x). $$

The Shannon source coding theorem says that for $N$ samples from the source, we can roughly compress it into $N\mathcal H$.

Planted: -by ;

References:
  1. Shannon’s Source Coding Theorem (Foundations of information theory: Part 3)
  2. Lecture 8: Source Coding Theorem, Huffman coding by Aarti Singh
  3. The Source Coding Theorem by Mario S. Alvim
Dynamic Backlinks:
Coding Theory Concepts -
The code function produces code words. The expected length of the code word is limited by the …

L Ma +by ;

References:
  1. Shannon’s Source Coding Theorem (Foundations of information theory: Part 3)
  2. Lecture 8: Source Coding Theorem, Huffman coding by Aarti Singh
  3. The Source Coding Theorem by Mario S. Alvim
Dynamic Backlinks to cards/information/coding-theory-concepts:
MaxEnt Model +
Maximum Entropy models makes least assumption about the data
Restricted Boltzmann Machine +
Introducing latent variables to Boltzmann machine and restrict the connections within groups.
MDL and Neural Networks +
Minimum Description Length ( [[MDL]] Minimum Description Length MDL is a measure of how well a model …
Fraser Information +
The Fraser information is +$$ I_F(\theta) = \int g(X) \ln f(X;\theta) , \mathrm d X. $$ +When …
Information Gain +
The information is a measurement of the entropy of the dataset.

L Ma (2021). 'Coding Theory Concepts', Datumorphism, 02 April. Available at: https://datumorphism.leima.is/cards/information/coding-theory-concepts/.

diff --git a/cards/information/cross-entropy/index.html b/cards/information/cross-entropy/index.html index 1d51b5c10..88ee81a5c 100644 --- a/cards/information/cross-entropy/index.html +++ b/cards/information/cross-entropy/index.html @@ -68,10 +68,8 @@ Planted: by ;

References:
  1. cross_entropy_wiki Contributors to Wikimedia projects. Cross entropy. In: Wikipedia [Internet]. 4 Jul 2021 [cited 4 Sep 2021]. Available: https://en.wikipedia.org/wiki/Cross_entropy
  2. Mehta2019 -Mehta P, Wang C-H, Day AGR, Richardson C, Bukov M, Fisher CK, et al. A high-bias, low-variance introduction to Machine Learning for physicists. Phys Rep. 2019;810: 1–124. doi:10.1016/j.physrep.2019.03.001
Dynamic Backlinks:
Cross Entropy -
Cross entropy is1 -$$ H(p, q) = \mathbb E_{p} \left[ -\log q \right]. $$ -Cross entropy $H(p, q)$ can …
Links to:
Shannon Entropy +Mehta P, Wang C-H, Day AGR, Richardson C, Bukov M, Fisher CK, et al. A high-bias, low-variance introduction to Machine Learning for physicists. Phys Rep. 2019;810: 1–124. doi:10.1016/j.physrep.2019.03.001
Dynamic Backlinks to cards/information/cross-entropy:
Logistic Regression +
logistics regression is a simple model for classification
Links to:
Shannon Entropy
Shannon entropy $S$ is the expectation of information content $I(X)=-\log \left(p\right)$1, …
KL Divergence
Kullback–Leibler divergence indicates the differences between two distributions
Logistic Regression
logistics regression is a simple model for classification

Lei Ma (2021). 'Cross Entropy', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/information/cross-entropy/.

diff --git a/cards/information/f-divergence/index.html b/cards/information/f-divergence/index.html index e761a1e01..10af95bbe 100644 --- a/cards/information/f-divergence/index.html +++ b/cards/information/f-divergence/index.html @@ -52,7 +52,9 @@ Planted: by ;

References:
  1. f-divergence_wiki Contributors to Wikimedia projects. F-divergence. In: Wikipedia [Internet]. 17 Jul 2021 [cited 4 Sep 2021]. Available: https://en.wikipedia.org/wiki/F-divergence
  2. Nowozin2016 -Nowozin S, Cseke B, Tomioka R. f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization. arXiv [stat.ML]. 2016. Available: http://arxiv.org/abs/1606.00709
Dynamic Backlinks:
f-Divergence +Nowozin S, Cseke B, Tomioka R. f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization. arXiv [stat.ML]. 2016. Available: http://arxiv.org/abs/1606.00709
Dynamic Backlinks to cards/information/f-divergence:
f-GAN +
The essence of [[GAN]] GAN The task of GAN is to generate features $X$ from some noise $\xi$ and …
infoGAN +
In GAN, the latent space input is usually random noise, e.g., Gaussian noise. The objective of …
f-Divergence
The f-divergence is defined as1 $$ \operatorname{D}_f = \int f\left(\frac{p}{q}\right) q\mathrm …
Links to:
KL Divergence
Kullback–Leibler divergence indicates the differences between two distributions

Lei Ma (2021). 'f-Divergence', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/information/f-divergence/.

Planted: -by ;

References:
  1. Ly A, Marsman M, Verhagen J, Grasman R, Wagenmakers E-J. A Tutorial on Fisher Information. arXiv [math.ST]. 2017. Available: http://arxiv.org/abs/1705.01064
  2. Fraser DAS. On Information in Statistics. aoms. 1965;36: 890–896. doi:10.1214/aoms/1177700061
Dynamic Backlinks:
Fisher Information -
Fisher information measures the second moment of the model sensitivity with respect to the …
Links to:
Fraser Information +by ;

References:
  1. Ly A, Marsman M, Verhagen J, Grasman R, Wagenmakers E-J. A Tutorial on Fisher Information. arXiv [math.ST]. 2017. Available: http://arxiv.org/abs/1705.01064
  2. Fraser DAS. On Information in Statistics. aoms. 1965;36: 890–896. doi:10.1214/aoms/1177700061
Dynamic Backlinks to cards/information/fisher-information:
Fraser Information +
The Fraser information is +$$ I_F(\theta) = \int g(X) \ln f(X;\theta) , \mathrm d X. $$ +When …
Links to:
Fraser Information
The Fraser information is $$ I_F(\theta) = \int g(X) \ln f(X;\theta) , \mathrm d X. $$ When …
Likelihood diff --git a/cards/information/fraser-information/index.html b/cards/information/fraser-information/index.html index 48fcf655d..94274ebf7 100644 --- a/cards/information/fraser-information/index.html +++ b/cards/information/fraser-information/index.html @@ -45,10 +45,9 @@ Kullback–Leibler divergence indicates the differences between two distributions 1.

  1. Fraser DAS. On Information in Statistics. aoms. 1965;36: 890–896. doi:10.1214/aoms/1177700061 ↩︎

Planted: -by ;

References:
  1. Fraser DAS. On Information in Statistics. aoms. 1965;36: 890–896. doi:10.1214/aoms/1177700061
Dynamic Backlinks:
Fraser Information -
The Fraser information is -$$ I_F(\theta) = \int g(X) \ln f(X;\theta) , \mathrm d X. $$ -When …
Links to:
Fisher Information +by ;

References:
  1. Fraser DAS. On Information in Statistics. aoms. 1965;36: 890–896. doi:10.1214/aoms/1177700061
Dynamic Backlinks to cards/information/fraser-information:
Explained Variation +
Using [[Fraser information]] Fraser Information The Fraser information is $$ I_F(\theta) = \int g(X) …
Fisher Information +
Fisher information measures the second moment of the model sensitivity with respect to the …
Links to:
Fisher Information
Fisher information measures the second moment of the model sensitivity with respect to the …
KL Divergence
Kullback–Leibler divergence indicates the differences between two distributions
Coding Theory Concepts
The code function produces code words. The expected length of the code word is limited by the …

Lei Ma (2021). 'Fraser Information', Datumorphism, 05 April. Available at: https://datumorphism.leima.is/cards/information/fraser-information/.

References:
  1. Liu2020 Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, et al. Self-supervised Learning: Generative or Contrastive. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2006.08218
  2. Latham2009 -Latham PE, Roudi Y. Mutual information. Scholarpedia. 2009;4. doi:10.4249/scholarpedia.1658
Dynamic Backlinks:
Mutual Information +Latham PE, Roudi Y. Mutual information. Scholarpedia. 2009;4. doi:10.4249/scholarpedia.1658
Dynamic Backlinks to cards/information/mutual-information:
Contrastive Model +
Contrastive models learn to compare1. Contrastive use special objective functions such as [[NCE]] …
Contrastive Model: Context-Instance +
In contrastive methods, we can manipulate the data to create data entries and infer the changes …
infoGAN +
In GAN, the latent space input is usually random noise, e.g., Gaussian noise. The objective of …
Deep Infomax +
Max Global Mutual Information +Why not just use the global mutual information of the input and …
Mutual Information
Mutual information is defined as $$ I(X;Y) = \mathbb E_{p_{XY}} \ln \frac{P_{XY}}{P_X P_Y}. $$ In …
Links to:
KL Divergence diff --git a/cards/information/shannon-entropy/index.html b/cards/information/shannon-entropy/index.html index e498f07a3..69b8bd40c 100644 --- a/cards/information/shannon-entropy/index.html +++ b/cards/information/shannon-entropy/index.html @@ -33,8 +33,10 @@  ↩︎

Planted: by ;

References:
  1. shannon_entropy_wiki -Contributors to Wikimedia projects. Entropy (information theory). In: Wikipedia [Internet]. 29 Aug 2021 [cited 4 Sep 2021]. Available: https://en.wikipedia.org/wiki/Entropy_(information_theory)
Dynamic Backlinks:
Shannon Entropy -
Shannon entropy $S$ is the expectation of information content $I(X)=-\log \left(p\right)$1, …

Lei Ma (2021). 'Shannon Entropy', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/information/shannon-entropy/.

Dynamic Backlinks to cards/information/shannon-entropy:
Cross Entropy +
Cross entropy is1 +$$ H(p, q) = \mathbb E_{p} \left[ -\log q \right]. $$ +Cross entropy $H(p, q)$ can …

Lei Ma (2021). 'Shannon Entropy', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/information/shannon-entropy/.

diff --git a/cards/machine-learning/datatypes/data-file-formats/index.html b/cards/machine-learning/datatypes/data-file-formats/index.html index 6fa7961a8..dad750178 100644 --- a/cards/machine-learning/datatypes/data-file-formats/index.html +++ b/cards/machine-learning/datatypes/data-file-formats/index.html @@ -31,7 +31,8 @@ #data-engineering #data-science

Data storage is diverse. For data on smaller scales, we are mostly dealing with some data files.

work_with_data_files

Efficiencies and Compressions

Parquet

Parquet is fast. But

  1. Don’t use json or list of json as columns. Convert them to strings or binary objects if it is really needed.

Planted: -by ;

References:
  1. Parquet Logical Type Definitions
  2. Working with Complex Data Formats with Structured Streaming in Apache Spark 2.1 Part 2 of Scalable Data @ Databricks
Links to:
Data Types +by ;

References:
  1. Parquet Logical Type Definitions
  2. Working with Complex Data Formats with Structured Streaming in Apache Spark 2.1 Part 2 of Scalable Data @ Databricks
Dynamic Backlinks to cards/machine-learning/datatypes/data-file-formats:
Data File Formats +
Data storage is diverse. For data on smaller scales, we are mostly dealing with some data files. …
Links to:
Data Types

LM (2021). 'Data File Formats', Datumorphism, 02 April. Available at: https://datumorphism.leima.is/cards/machine-learning/datatypes/data-file-formats/.

diff --git a/cards/machine-learning/datatypes/data-types/index.html b/cards/machine-learning/datatypes/data-types/index.html index e218b33fe..4b9527ab3 100644 --- a/cards/machine-learning/datatypes/data-types/index.html +++ b/cards/machine-learning/datatypes/data-types/index.html @@ -24,8 +24,8 @@ #Data #Data Types

Planted: -by ;

References:
  1. Level of Measurement
Dynamic Backlinks:
Data Types -

L Ma +by ;

References:
  1. Level of Measurement
Dynamic Backlinks to cards/machine-learning/datatypes/data-types:
Data File Formats +
Data storage is diverse. For data on smaller scales, we are mostly dealing with some data files. …

L Ma (2020). 'Data Types', Datumorphism, 01 April. Available at: https://datumorphism.leima.is/cards/machine-learning/datatypes/data-types/.

diff --git a/cards/machine-learning/embedding/continuous-bag-of-words/index.html b/cards/machine-learning/embedding/continuous-bag-of-words/index.html index ff64122db..256a7f55b 100644 --- a/cards/machine-learning/embedding/continuous-bag-of-words/index.html +++ b/cards/machine-learning/embedding/continuous-bag-of-words/index.html @@ -24,8 +24,9 @@ #Word2vec
Here we encode all words presented in the corpus to demostrate the idea of CBOW. In the real world, we might want to remove some certain words such as the.

We use the following quote by Ford in Westworld as an example.

I read a theory once that the human intellect is like peacock feathers. Just an extravagant display intended to attract a mate, just an elaborate mating ritual. But, of course, the peacock can barely fly. It lives in the dirt, pecking insects out of the muck, consoling itself with its great beauty.

The word intended is surrunded by extravagant display in the front and to attract after it. The task is to predict the middle word intended using the ‘history words’ extravagant display and to attract.

  • Input: extravagant, display, to, attract
  • Output: intended

In the bag-of-words model, the order of the words extravagant, display, to, attract doesn’t matter hence bag-of-words. [mikolov2013]

This makes it easier to represent the dataset:

Input (Context)Output (Center Word)
extravagantintended
displayintended
tointended
attractintended

To create a real dataset, we “slide” over all the words.

Input (Context)Output (Center Word)
readI
aI
Iread
aread
theoryread
Ia
reada
theorya
oncea
readtheory
atheory
oncetheory
……

It is not required to choose two words as history words and two words as future words. The number of words to choose is the window size.

Planted: by ;

References:
  1. mikolov2013 -Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781
  2. The Illustrated Word2vec
Dynamic Backlinks:
CBOW: Continuous Bag of Words -
Use the context to predict the center word
Dynamic Backlinks to cards/machine-learning/embedding/continuous-bag-of-words:
Word2vec +
Single-layer neural network creates embedding space
Negative Sampling +
negative sampling makes the calculations faster

L Ma (2020). 'CBOW: Continuous Bag of Words', Datumorphism, 01 April. Available at: https://datumorphism.leima.is/cards/machine-learning/embedding/continuous-bag-of-words/.

diff --git a/cards/machine-learning/embedding/continuous-skip-gram/index.html b/cards/machine-learning/embedding/continuous-skip-gram/index.html index 199dd5f93..61b416dcd 100644 --- a/cards/machine-learning/embedding/continuous-skip-gram/index.html +++ b/cards/machine-learning/embedding/continuous-skip-gram/index.html @@ -24,8 +24,9 @@ #Word2vec

We use the following quote by Ford in Westworld as an example.

I read a theory once that the human intellect is like peacock feathers. Just an extravagant display intended to attract a mate, just an elaborate mating ritual. But, of course, the peacock can barely fly. It lives in the dirt, pecking insects out of the muck, consoling itself with its great beauty.

The word intended is surrunded by extravagant display in the front and to attract after it. The task is to predict the probability of words around the middle word intended, which are the ‘history words’ extravagant, display and ‘future words’ to, attract in our case. [mikolov2013]

For this center word intended, we generate the following data.

Input (Center Word)Output (Context)
intendedextravagant
intendeddisplay
intendedto
intendedattract

We will build the following dataset using the sentence.

Input (Center Word)Output (Context)
Iread
Ia
readI
reada
readtheory
aI
aread
atheory
aonce
theoryread
theorya
theoryonce
……

It is not required to use two in the front and two words after the middle word. The number of words to choose is a hyperparameter to be decided.

Planted: by ;

References:
  1. mikolov2013 -Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781
  2. The Illustrated Word2vec
Dynamic Backlinks:
skipgram: Continuous skip-gram -
Use the center word to predict the context
Dynamic Backlinks to cards/machine-learning/embedding/continuous-skip-gram:
Word2vec +
Single-layer neural network creates embedding space
Negative Sampling +
negative sampling makes the calculations faster

L Ma (2020). 'skipgram: Continuous skip-gram', Datumorphism, 01 April. Available at: https://datumorphism.leima.is/cards/machine-learning/embedding/continuous-skip-gram/.

diff --git a/cards/machine-learning/embedding/negative-sampling/index.html b/cards/machine-learning/embedding/negative-sampling/index.html index f1b6d0e56..5a1f4917c 100644 --- a/cards/machine-learning/embedding/negative-sampling/index.html +++ b/cards/machine-learning/embedding/negative-sampling/index.html @@ -35,8 +35,8 @@ Planted: by ;

References:
  1. mikolov2013 Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781
  2. Goldberg2014 -Goldberg Y, Levy O. word2vec Explained: deriving Mikolov et al.’s negative-sampling word-embedding method. arXiv [cs.CL]. 2014. Available: http://arxiv.org/abs/1402.3722
  3. The Illustrated Word2vec
Dynamic Backlinks:
Negative Sampling -
negative sampling makes the calculations faster
Links to:
CBOW: Continuous Bag of Words +Goldberg Y, Levy O. word2vec Explained: deriving Mikolov et al.’s negative-sampling word-embedding method. arXiv [cs.CL]. 2014. Available: http://arxiv.org/abs/1402.3722
  • The Illustrated Word2vec
    • Dynamic Backlinks to cards/machine-learning/embedding/negative-sampling:
    Word2vec +
    Single-layer neural network creates embedding space
    Links to:
    CBOW: Continuous Bag of Words
    Use the context to predict the center word
    skipgram: Continuous skip-gram
    Use the center word to predict the context

    L Ma (2020). 'Negative Sampling', Datumorphism, 01 April. Available at: https://datumorphism.leima.is/cards/machine-learning/embedding/negative-sampling/.

    Planted: by ;

    References:
    1. Murphy2012 -Murphy, K. P. (2012). Probabilistic Machine Learning: An Introduction.
    Dynamic Backlinks:
    Cross Validation -
    Cross validation is a method to estimate the [[risk]] The Learning Problem The learning problem …
    Links to:
    The Learning Problem +Murphy, K. P. (2012). Probabilistic Machine Learning: An Introduction.
    Dynamic Backlinks to cards/machine-learning/learning-theories/cross-validation:
    Cross Validation +
    Cross validation is a method to estimate the [[risk]] The Learning Problem The learning problem …
    Measures of Generalizability +
    To measure the generalization, we define a generalization error, +$$ \begin{align} \mathcal G = …
    Links to:
    The Learning Problem
    The learning problem posed by Vapnik:1 Given a sample: $\{z_i\}$ in the probability space $Z$; …

    L Ma (2021). 'Cross Validation', Datumorphism, 05 April. Available at: https://datumorphism.leima.is/cards/machine-learning/learning-theories/cross-validation/.

    Planted: -by ;

    References:
    1. Murphy, K. P. (2012). Probabilistic Machine Learning: An Introduction.
    Dynamic Backlinks:
    ERM: Empirical Risk Minimization -
    In a [[learning problem]] The Learning Problem The learning problem posed by Vapnik:1 Given a …
    Links to:
    The Learning Problem +by ;

    References:
    1. Murphy, K. P. (2012). Probabilistic Machine Learning: An Introduction.
    Dynamic Backlinks to cards/machine-learning/learning-theories/empirical-risk-minimization:
    ERM: Empirical Risk Minimization +
    In a [[learning problem]] The Learning Problem The learning problem posed by Vapnik:1 Given a …
    Bias-Variance +
    Bias Variance Trade off is a key concept in statistical learning
    The Learning Problem +
    The learning problem posed by Vapnik:1 +Given a sample: $\{z_i\}$ in the probability space $Z$; …
    SRM: Structural Risk Minimization +
    [[ERM]] ERM: Empirical Risk Minimization In a [[learning problem]] The Learning Problem The learning …
    Links to:
    The Learning Problem
    The learning problem posed by Vapnik:1 Given a sample: $\{z_i\}$ in the probability space $Z$; …
    Likelihood
    Likelihood is not necessarily a pdf

    L Ma diff --git a/cards/machine-learning/learning-theories/induction-deduction-transduction/index.html b/cards/machine-learning/learning-theories/induction-deduction-transduction/index.html index ee5bd9fbc..baeb0df97 100644 --- a/cards/machine-learning/learning-theories/induction-deduction-transduction/index.html +++ b/cards/machine-learning/learning-theories/induction-deduction-transduction/index.html @@ -26,13 +26,8 @@ #Deduction #Transduction

    Planted: -by ;

    Similar Articles:
    Shatter -
    Given a set $\mathcal S$, and a class (collection of sets) $\mathcal H$. -For any subset of $\mathcal …
    Feasibility of Learning -
    Why is learning from data even possible? To discuss this problem, we need a framework for learning. …
    Noise Contrastive Estimation: NCE -
    Noise contrastive estimation (NCE) objective function is1 -$$ \mathcal L = \mathbb E_{x, x^{+}, …
    Cross Validation -
    Cross validation is a method to estimate the [[risk]] The Learning Problem The learning problem …

    L Ma +by ;

    Dynamic Backlinks to cards/machine-learning/learning-theories/induction-deduction-transduction:
    Induction, Deduction, and Transduction +

    L Ma (2022). 'Induction, Deduction, and Transduction', Datumorphism, 04 April. Available at: https://datumorphism.leima.is/cards/machine-learning/learning-theories/induction-deduction-transduction/.

    diff --git a/cards/machine-learning/learning-theories/learning-problem/index.html b/cards/machine-learning/learning-theories/learning-problem/index.html index 2f8e535fb..c7d3261c4 100644 --- a/cards/machine-learning/learning-theories/learning-problem/index.html +++ b/cards/machine-learning/learning-theories/learning-problem/index.html @@ -36,9 +36,11 @@  ↩︎

    Planted: by ;

    References:
    1. Vapnik2000 -Vladimir N. Vapnik. The Nature of Statistical Learning Theory. 2000. doi:10.1007/978-1-4757-3264-1
    Dynamic Backlinks:
    The Learning Problem -
    The learning problem posed by Vapnik:1 -Given a sample: $\{z_i\}$ in the probability space $Z$; …
    Links to:
    ERM: Empirical Risk Minimization +Vladimir N. Vapnik. The Nature of Statistical Learning Theory. 2000. doi:10.1007/978-1-4757-3264-1
    Dynamic Backlinks to cards/machine-learning/learning-theories/learning-problem:
    Machine Learning Overview +
    A brief overview of machine learning
    Cross Validation +
    Cross validation is a method to estimate the [[risk]] The Learning Problem The learning problem …
    ERM: Empirical Risk Minimization +
    In a [[learning problem]] The Learning Problem The learning problem posed by Vapnik:1 Given a …
    SRM: Structural Risk Minimization +
    [[ERM]] ERM: Empirical Risk Minimization In a [[learning problem]] The Learning Problem The learning …
    Links to:
    ERM: Empirical Risk Minimization
    In a [[learning problem]] The Learning Problem The learning problem posed by Vapnik:1 Given a …
    SRM: Structural Risk Minimization
    [[ERM]] ERM: Empirical Risk Minimization In a [[learning problem]] The Learning Problem The learning …
    VC Dimension

    Planted: by ;

    References:
    1. Liu2020 -Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, et al. Self-supervised Learning: Generative or Contrastive. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2006.08218
    Dynamic Backlinks:
    Noise Contrastive Estimation: NCE +Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, et al. Self-supervised Learning: Generative or Contrastive. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2006.08218
    Dynamic Backlinks to cards/machine-learning/learning-theories/noise-contrastive-estimation:
    Contrastive Model +
    Contrastive models learn to compare1. Contrastive use special objective functions such as [[NCE]] …
    Contrastive Model: Context-Instance +
    In contrastive methods, we can manipulate the data to create data entries and infer the changes …
    Noise Contrastive Estimation: NCE
    Noise contrastive estimation (NCE) objective function is1 $$ \mathcal L = \mathbb E_{x, x^{+}, …

    L Ma (2021). 'Noise Contrastive Estimation: NCE', Datumorphism, 08 April. Available at: https://datumorphism.leima.is/cards/machine-learning/learning-theories/noise-contrastive-estimation/.

    Planted: by ;

    References:
    1. shattered_set_wikipedia -Shattered Set @ Wikipedia
    Dynamic Backlinks:
    Shatter -
    Given a set $\mathcal S$, and a class (collection of sets) $\mathcal H$. -For any subset of $\mathcal …
    Links to:
    VC Dimension +Shattered Set @ Wikipedia
    Dynamic Backlinks to cards/machine-learning/learning-theories/set-shatter:
    VC Dimension +
    Two of the key elements in a learning problem are: +a set of hypothesis $\mathcal H$, and a set of …
    Links to:
    VC Dimension
    Two of the key elements in a learning problem are: a set of hypothesis $\mathcal H$, and a set of …

    L Ma (2021). 'Shatter', Datumorphism, 10 April. Available at: https://datumorphism.leima.is/cards/machine-learning/learning-theories/set-shatter/.

    Planted: -by ;

    References:
    1. Structural risk minimization @ Wikipedia
    2. Murphy, K. P. (2012). Probabilistic Machine Learning: An Introduction.
    Dynamic Backlinks:
    SRM: Structural Risk Minimization -
    [[ERM]] ERM: Empirical Risk Minimization In a [[learning problem]] The Learning Problem The learning …
    Links to:
    The Learning Problem +by ;

    References:
    1. Structural risk minimization @ Wikipedia
    2. Murphy, K. P. (2012). Probabilistic Machine Learning: An Introduction.
    Dynamic Backlinks to cards/machine-learning/learning-theories/structural-risk-minimization:
    SRM: Structural Risk Minimization +
    [[ERM]] ERM: Empirical Risk Minimization In a [[learning problem]] The Learning Problem The learning …
    The Learning Problem +
    The learning problem posed by Vapnik:1 +Given a sample: $\{z_i\}$ in the probability space $Z$; …
    Links to:
    The Learning Problem
    The learning problem posed by Vapnik:1 Given a sample: $\{z_i\}$ in the probability space $Z$; …
    ERM: Empirical Risk Minimization
    In a [[learning problem]] The Learning Problem The learning problem posed by Vapnik:1 Given a …

    L Ma diff --git a/cards/machine-learning/measurement/empirical-loss/index.html b/cards/machine-learning/measurement/empirical-loss/index.html index d160ba12d..ff93de1cf 100644 --- a/cards/machine-learning/measurement/empirical-loss/index.html +++ b/cards/machine-learning/measurement/empirical-loss/index.html @@ -28,8 +28,10 @@ \end{align} $$

    where $d(y_i, f(x_i))$ is the distance defined between $y_i$ and $f(x_i)$.

    Planted: -by ;

    Dynamic Backlinks:
    Empirical Loss -
    The loss calculated on all the data points
    Links to:
    Measures of Generalizability +by ;

    Dynamic Backlinks to cards/machine-learning/measurement/empirical-loss:
    Measures of Generalizability +
    To measure the generalization, we define a generalization error, +$$ \begin{align} \mathcal G = …
    Population Loss +
    The loss calculated on all the whole population
    Links to:
    Measures of Generalizability
    To measure the generalization, we define a generalization error, $$ \begin{align} \mathcal G = …
    Population Loss
    The loss calculated on all the whole population

    L Ma diff --git a/cards/machine-learning/measurement/gini-impurity/index.html b/cards/machine-learning/measurement/gini-impurity/index.html index 2fa6d37c5..b2b173688 100644 --- a/cards/machine-learning/measurement/gini-impurity/index.html +++ b/cards/machine-learning/measurement/gini-impurity/index.html @@ -34,8 +34,11 @@ G = p(0)(1-p(0)) + p(1)(1-p(1)) = 0.5 * 0.5+ 0.5*0.5 = 0.5. $$

    For data with two possible values $\{0,1\}$, the maximum Gini impurity is 0.25. The following chart shows all the possible values of the Gini impurity for two-value dataset.

    For data with three possible values, the Gini impurity is also visualized using the same chart given the condition that $p_3 = 1 - p_1 - p_2$.

    Planted: -by ;

    References:
    1. A Simple Explanation of Gini Impurity by Victor Zhou
    2. Shalev-Shwartz, S., & Ben-David, S. (2013). Understanding machine learning: From theory to algorithms. Understanding Machine Learning: From Theory to Algorithms.
    Supplementary:
    1. Code
    Dynamic Backlinks:
    Gini Impurity -
    The Gini impurity is a measurement of the impurity of a set.
    Links to:
    Decision Tree +by ;

    References:
    1. A Simple Explanation of Gini Impurity by Victor Zhou
    2. Shalev-Shwartz, S., & Ben-David, S. (2013). Understanding machine learning: From theory to algorithms. Understanding Machine Learning: From Theory to Algorithms.
    Supplementary:
    1. Code
    Dynamic Backlinks to cards/machine-learning/measurement/gini-impurity:
    Decision Tree +
    In this article, we will explain how decision trees work and build a tree by hand. +The code used in …
    Random Forest +
    random forest in machine learning
    Information Gain +
    The information is a measurement of the entropy of the dataset.
    Links to:
    Decision Tree
    In this article, we will explain how decision trees work and build a tree by hand. The code used in …
    Information Gain
    The information is a measurement of the entropy of the dataset.

    L Ma diff --git a/cards/machine-learning/measurement/hilbert-schmidt-independence-criterion/index.html b/cards/machine-learning/measurement/hilbert-schmidt-independence-criterion/index.html index 31a865c1e..11747aa4b 100644 --- a/cards/machine-learning/measurement/hilbert-schmidt-independence-criterion/index.html +++ b/cards/machine-learning/measurement/hilbert-schmidt-independence-criterion/index.html @@ -39,9 +39,8 @@ Useful when centering a vector around its mean .

    We can choose different kernel functions $k$ and $l$. For example, if $k$ and $l$ are linear kernels, we have $k(x, y) = l(x, y) = x \cdot y$. In this linear case, HSIC is simply $\parallel\operatorname{cov}(x^T,y^T) \parallel^2_{\text{Frobenius}}$.

    1. Gretton A, Bousquet O, Smola A, Schölkopf B. Measuring Statistical Dependence with Hilbert-Schmidt Norms. Algorithmic Learning Theory. Springer Berlin Heidelberg; 2005. pp. 63–77. doi:10.1007/11564089_7 ↩︎

    2. Kornblith S, Norouzi M, Lee H, Hinton G. Similarity of Neural Network Representations Revisited. arXiv [cs.LG]. 2019. Available: http://arxiv.org/abs/1905.00414 ↩︎

    Planted: -by ;

    Dynamic Backlinks:
    Hilbert-Schmidt Independence Criterion (HSIC) -
    Given two kernels of the feature representations $K=k(x,x)$ and $L=l(y,y)$, HSIC is defined as12 -$$ …
    Links to:
    Centering Matrix +by ;

    Dynamic Backlinks to cards/machine-learning/measurement/hilbert-schmidt-independence-criterion:
    Centered Kernel Alignment (CKA) +
    Centered Kernel Alignment (CKA) is a similarity metric designed to measure the similarity of between …
    Links to:
    Centering Matrix
    Useful when centering a vector around its mean

    L Ma (2021). 'Hilbert-Schmidt Independence Criterion (HSIC)', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/measurement/hilbert-schmidt-independence-criterion/.

    diff --git a/cards/machine-learning/measurement/information-gain/index.html b/cards/machine-learning/measurement/information-gain/index.html index ece3d734d..3516b815c 100644 --- a/cards/machine-learning/measurement/information-gain/index.html +++ b/cards/machine-learning/measurement/information-gain/index.html @@ -31,8 +31,11 @@ Gain = S_m - S'_m. $$

    Planted: -by ;

    References:
    1. Shalev-Shwartz, S., & Ben-David, S. (2013). Understanding machine learning: From theory to algorithms. Understanding Machine Learning: From Theory to Algorithms.
    Supplementary:
    1. Code
    Dynamic Backlinks:
    Information Gain -
    The information is a measurement of the entropy of the dataset.
    Links to:
    Decision Tree +by ;

    References:
    1. Shalev-Shwartz, S., & Ben-David, S. (2013). Understanding machine learning: From theory to algorithms. Understanding Machine Learning: From Theory to Algorithms.
    Supplementary:
    1. Code
    Dynamic Backlinks to cards/machine-learning/measurement/information-gain:
    Decision Tree +
    In this article, we will explain how decision trees work and build a tree by hand. +The code used in …
    Random Forest +
    random forest in machine learning
    Gini Impurity +
    The Gini impurity is a measurement of the impurity of a set.
    Links to:
    Decision Tree
    In this article, we will explain how decision trees work and build a tree by hand. The code used in …
    Gini Impurity
    The Gini impurity is a measurement of the impurity of a set.
    Coding Theory Concepts diff --git a/cards/machine-learning/measurement/population-loss/index.html b/cards/machine-learning/measurement/population-loss/index.html index c6d59ce32..9769fa3b6 100644 --- a/cards/machine-learning/measurement/population-loss/index.html +++ b/cards/machine-learning/measurement/population-loss/index.html @@ -28,8 +28,10 @@ \end{align} $$

    where $d(y, f(x))$ is the distance defined between $y$ and $f(x)$.

    Planted: -by ;

    Dynamic Backlinks:
    Population Loss -
    The loss calculated on all the whole population
    Links to:
    Measures of Generalizability +by ;

    Dynamic Backlinks to cards/machine-learning/measurement/population-loss:
    Measures of Generalizability +
    To measure the generalization, we define a generalization error, +$$ \begin{align} \mathcal G = …
    Empirical Loss +
    The loss calculated on all the data points
    Links to:
    Measures of Generalizability
    To measure the generalization, we define a generalization error, $$ \begin{align} \mathcal G = …
    Empirical Loss
    The loss calculated on all the data points

    L Ma diff --git a/cards/machine-learning/neural-networks/activation-bi-polar-sigmoid/index.html b/cards/machine-learning/neural-networks/activation-bi-polar-sigmoid/index.html index ec3ace8df..d2c7743a4 100644 --- a/cards/machine-learning/neural-networks/activation-bi-polar-sigmoid/index.html +++ b/cards/machine-learning/neural-networks/activation-bi-polar-sigmoid/index.html @@ -28,11 +28,8 @@ \sigma(x) = \frac{1-e^{-x}}{1+e^{-x}}. $$

    Visualization

    Bipolar Sigmoid

    Planted: -by ;

    Similar Articles:
    Conic Section Function -
    Conic Section Function and its properties
    ELU -
    ELU and its properties
    Hyperbolic Tanh -
    Tanh function and its properties
    Leaky ReLu -
    Leaky ReLu and its properties
    Links to:
    Artificial Neural Networks +by ;

    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-bi-polar-sigmoid:
    BiPolar Sigmoid +
    BiPolar sigmoid function and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'BiPolar Sigmoid', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-bi-polar-sigmoid/.

    diff --git a/cards/machine-learning/neural-networks/activation-conic-section-function/index.html b/cards/machine-learning/neural-networks/activation-conic-section-function/index.html index 2e01e0fde..71f3dcf93 100644 --- a/cards/machine-learning/neural-networks/activation-conic-section-function/index.html +++ b/cards/machine-learning/neural-networks/activation-conic-section-function/index.html @@ -29,11 +29,8 @@  ↩︎

    Planted: by ;

    References:
    1. Dorffner1994 -Dorffner G. UNIFIED FRAMEWORK FOR MLPs AND RBFNs: INTRODUCING CONIC SECTION FUNCTION NETWORKS. Cybern Syst. 1994;25: 511–554. doi:10.1080/01969729408902340
    2. Shenouda EAMA. A Quantitative Comparison of Different MLP Activation Functions in Classification. Advances in Neural Networks - ISNN 2006. Springer Berlin Heidelberg; 2006. pp. 849–857. doi:10.1007/11759966_125
    Similar Articles:
    BiPolar Sigmoid -
    BiPolar sigmoid function and its properties
    ELU -
    ELU and its properties
    Hyperbolic Tanh -
    Tanh function and its properties
    Leaky ReLu -
    Leaky ReLu and its properties
    Links to:
    Artificial Neural Networks +Dorffner G. UNIFIED FRAMEWORK FOR MLPs AND RBFNs: INTRODUCING CONIC SECTION FUNCTION NETWORKS. Cybern Syst. 1994;25: 511–554. doi:10.1080/01969729408902340
  • Shenouda EAMA. A Quantitative Comparison of Different MLP Activation Functions in Classification. Advances in Neural Networks - ISNN 2006. Springer Berlin Heidelberg; 2006. pp. 849–857. doi:10.1007/11759966_125
    • Dynamic Backlinks to cards/machine-learning/neural-networks/activation-conic-section-function:
    Conic Section Function +
    Conic Section Function and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'Conic Section Function', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-conic-section-function/.

    diff --git a/cards/machine-learning/neural-networks/activation-elu/index.html b/cards/machine-learning/neural-networks/activation-elu/index.html index 9d6fc73ba..917a23232 100644 --- a/cards/machine-learning/neural-networks/activation-elu/index.html +++ b/cards/machine-learning/neural-networks/activation-elu/index.html @@ -98,11 +98,8 @@ pass

    Planted: -by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Similar Articles:
    BiPolar Sigmoid -
    BiPolar sigmoid function and its properties
    Conic Section Function -
    Conic Section Function and its properties
    Hyperbolic Tanh -
    Tanh function and its properties
    Leaky ReLu -
    Leaky ReLu and its properties
    Links to:
    Artificial Neural Networks +by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-elu:
    ELU +
    ELU and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'ELU', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-elu/.

    diff --git a/cards/machine-learning/neural-networks/activation-hyperbolic-tangent/index.html b/cards/machine-learning/neural-networks/activation-hyperbolic-tangent/index.html index 432626047..2ba521b3c 100644 --- a/cards/machine-learning/neural-networks/activation-hyperbolic-tangent/index.html +++ b/cards/machine-learning/neural-networks/activation-hyperbolic-tangent/index.html @@ -28,11 +28,8 @@ \tanh(x) = \frac{\sinh(x)}{\cosh(x)} = \frac{e^{x} - e^{-x}}{e^x + e^{-x}} $$

    Hyperbolic tangent

    Planted: -by ;

    Similar Articles:
    BiPolar Sigmoid -
    BiPolar sigmoid function and its properties
    Conic Section Function -
    Conic Section Function and its properties
    ELU -
    ELU and its properties
    Leaky ReLu -
    Leaky ReLu and its properties
    Links to:
    Artificial Neural Networks +by ;

    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-hyperbolic-tangent:
    Hyperbolic Tanh +
    Tanh function and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'Hyperbolic Tanh', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-hyperbolic-tangent/.

    diff --git a/cards/machine-learning/neural-networks/activation-leaky-relu/index.html b/cards/machine-learning/neural-networks/activation-leaky-relu/index.html index ec532dba4..f8a087183 100644 --- a/cards/machine-learning/neural-networks/activation-leaky-relu/index.html +++ b/cards/machine-learning/neural-networks/activation-leaky-relu/index.html @@ -99,11 +99,8 @@ pass

    Planted: -by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Similar Articles:
    BiPolar Sigmoid -
    BiPolar sigmoid function and its properties
    Conic Section Function -
    Conic Section Function and its properties
    ELU -
    ELU and its properties
    Hyperbolic Tanh -
    Tanh function and its properties
    Links to:
    Artificial Neural Networks +by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-leaky-relu:
    Leaky ReLu +
    Leaky ReLu and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'Leaky ReLu', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-leaky-relu/.

    diff --git a/cards/machine-learning/neural-networks/activation-radial-basis-function/index.html b/cards/machine-learning/neural-networks/activation-radial-basis-function/index.html index a184038fb..00597975c 100644 --- a/cards/machine-learning/neural-networks/activation-radial-basis-function/index.html +++ b/cards/machine-learning/neural-networks/activation-radial-basis-function/index.html @@ -26,11 +26,8 @@ #Basics #Activation Function
    Two unnormalized Gaussian radial basis functions in one input dimension. The basis function centers are located at x1=0.75 and x2=3.25. Source Unnormalized Radial Basis Functions
    Hyperbolic tangent

    Two unnormalized Gaussian radial basis functions in one input dimension. The basis function centers are located at x1=0.75 and x2=3.25. Source Unnormalized Radial Basis Functions

    Planted: -by ;

    Similar Articles:
    BiPolar Sigmoid -
    BiPolar sigmoid function and its properties
    Conic Section Function -
    Conic Section Function and its properties
    ELU -
    ELU and its properties
    Hyperbolic Tanh -
    Tanh function and its properties
    Links to:
    Artificial Neural Networks +by ;

    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-radial-basis-function:
    Radial Basis Function +
    Radial Basis Function function and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'Radial Basis Function', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-radial-basis-function/.

    diff --git a/cards/machine-learning/neural-networks/activation-relu/index.html b/cards/machine-learning/neural-networks/activation-relu/index.html index 38e5a4872..6e04a4df1 100644 --- a/cards/machine-learning/neural-networks/activation-relu/index.html +++ b/cards/machine-learning/neural-networks/activation-relu/index.html @@ -97,11 +97,8 @@ pass

    Planted: -by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Similar Articles:
    BiPolar Sigmoid -
    BiPolar sigmoid function and its properties
    Conic Section Function -
    Conic Section Function and its properties
    ELU -
    ELU and its properties
    Hyperbolic Tanh -
    Tanh function and its properties
    Links to:
    Artificial Neural Networks +by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-relu:
    ReLu +
    Rectified Linear Unit, aka ReLu, and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'ReLu', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-relu/.

    diff --git a/cards/machine-learning/neural-networks/activation-swish/index.html b/cards/machine-learning/neural-networks/activation-swish/index.html index 911118f76..6067efe4b 100644 --- a/cards/machine-learning/neural-networks/activation-swish/index.html +++ b/cards/machine-learning/neural-networks/activation-swish/index.html @@ -99,11 +99,8 @@ pass

    Planted: -by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Similar Articles:
    BiPolar Sigmoid -
    BiPolar sigmoid function and its properties
    Conic Section Function -
    Conic Section Function and its properties
    ELU -
    ELU and its properties
    Hyperbolic Tanh -
    Tanh function and its properties
    Links to:
    Artificial Neural Networks +by ;

    References:
    1. Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.html
    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-swish:
    Swish +
    Swish and its properties
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron
    Additional Double Backet Links:
    Uni-Polar Sigmoid
    Uni-polar sigmoid function and its properties

    L Ma (2018). 'Swish', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-swish/.

    diff --git a/cards/machine-learning/neural-networks/activation-uni-polar-sigmoid/index.html b/cards/machine-learning/neural-networks/activation-uni-polar-sigmoid/index.html index 1486b362a..e14fd06bd 100644 --- a/cards/machine-learning/neural-networks/activation-uni-polar-sigmoid/index.html +++ b/cards/machine-learning/neural-networks/activation-uni-polar-sigmoid/index.html @@ -30,8 +30,9 @@ 1 - \sigma(x) = \sigma(-x). $$

    Planted: -by ;

    Dynamic Backlinks:
    Uni-Polar Sigmoid -
    Uni-polar sigmoid function and its properties
    Links to:
    Artificial Neural Networks +by ;

    Dynamic Backlinks to cards/machine-learning/neural-networks/activation-uni-polar-sigmoid:
    Uni-Polar Sigmoid +
    Uni-polar sigmoid function and its properties
    Logistic Regression +
    logistics regression is a simple model for classification
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2018). 'Uni-Polar Sigmoid', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/activation-uni-polar-sigmoid/.

    diff --git a/cards/machine-learning/neural-networks/mcculloch-pitts-model/index.html b/cards/machine-learning/neural-networks/mcculloch-pitts-model/index.html index 4dc9ad6bc..9eff9f187 100644 --- a/cards/machine-learning/neural-networks/mcculloch-pitts-model/index.html +++ b/cards/machine-learning/neural-networks/mcculloch-pitts-model/index.html @@ -27,8 +27,8 @@ y = \operatorname{sign}( w\cdot x - b). $$

    Since $w\cdot x - b = 0$ is a hyperplane, the McCulloch-Pitts model separates the state space using this hyperplane. The shift $b$ determines the interception, and $w$ decides the slope.

    Planted: -by ;

    References:
    1. Vladimir N. Vapnik. 2000. The Nature of Statistical Learning Theory.
    Dynamic Backlinks:
    McCulloch-Pitts Model -
    Artificial neuron that separates the state space
    Links to:
    Artificial Neural Networks +by ;

    References:
    1. Vladimir N. Vapnik. 2000. The Nature of Statistical Learning Theory.
    Dynamic Backlinks to cards/machine-learning/neural-networks/mcculloch-pitts-model:
    Rosenblatt's Perceptron +
    Connected perceptrons
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    LM (2021). 'McCulloch-Pitts Model', Datumorphism, 02 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/mcculloch-pitts-model/.

    diff --git a/cards/machine-learning/neural-networks/neural-networks-initialization/index.html b/cards/machine-learning/neural-networks/neural-networks-initialization/index.html index 4658a52b4..31588ff0b 100644 --- a/cards/machine-learning/neural-networks/neural-networks-initialization/index.html +++ b/cards/machine-learning/neural-networks/neural-networks-initialization/index.html @@ -57,8 +57,10 @@ by ;

    References:
    1. Lippe2020 Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io
    2. Ouannes P. How to initialize deep neural networks? Xavier and Kaiming initialization. In: Title [Internet]. 22 Mar 2019 [cited 24 Sep 2021]. Available: https://pouannes.github.io/blog/initialization
    3. Glorot2010 Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. Teh YW, Titterington M, editors. 2010;9: 249–256. Available: http://proceedings.mlr.press/v9/glorot10a.html
    4. Katanforoosh2019 -Katanforoosh & Kunin, "Initializing neural networks", deeplearning.ai, 2019.
    Dynamic Backlinks:
    Initialize Artificial Neural Networks -
    Initialize a neural network is important for the training and performance. Some initializations …
    Links to:
    Artificial Neural Networks +Katanforoosh & Kunin, "Initializing neural networks", deeplearning.ai, 2019.
    Dynamic Backlinks to cards/machine-learning/neural-networks/neural-networks-initialization:
    Learning Rate +
    Find a good learning rate
    PyTorch: Initialize Parameters +
    We can set the parameters in a for loop. We take some of the initialization methods from Lippe1. +To …
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron

    L Ma (2021). 'Initialize Artificial Neural Networks', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/machine-learning/neural-networks/neural-networks-initialization/.

    diff --git a/cards/machine-learning/practice/cuda-memory/index.html b/cards/machine-learning/practice/cuda-memory/index.html index 312f3ffc6..6c878f84e 100644 --- a/cards/machine-learning/practice/cuda-memory/index.html +++ b/cards/machine-learning/practice/cuda-memory/index.html @@ -35,8 +35,8 @@ Mohan A. Pipelining data processing and host-to-device data transfer. In: Telesens [Internet]. [cited 17 Oct 2022]. Available: https://www.telesens.co/2019/02/16/efficient-data-transfer-from-paged-memory-to-gpu-using-multi-threading/
  • Harris2012 Harris M. How to Optimize Data Transfers in CUDA C/C++. In: NVIDIA Technical Blog [Internet]. 5 Dec 2012 [cited 19 Oct 2022]. Available: https://developer.nvidia.com/blog/how-optimize-data-transfers-cuda-cc/
  • Contributors to Wikimedia projects. Memory paging. In: Wikipedia [Internet]. 7 Oct 2022 [cited 19 Oct 2022]. Available: https://en.wikipedia.org/wiki/Memory_paging
  • Computer Science. Segmented, Paged and Virtual Memory. YouTube. 2019. Available: https://www.youtube.com/watch?v=p9yZNLeOj4s
  • CoffeeBeforeArch. CUDA Crash Course (v2): Pinned Memory. YouTube. 2019. Available: https://www.youtube.com/watch?v=ShT7raBPP8k
  • pytorch-memory-pinning torch.utils.data — PyTorch 1.8.1 documentation. [cited 19 Oct 2022]. Available: https://pytorch.org/docs/1.8.1/data.html#memory-pinning
  • Mao L. Page-Locked Host Memory for Data Transfer. In: Lei Mao’s Log Book [Internet]. 26 Jun 2021 [cited 19 Oct 2022]. Available: https://leimao.github.io/blog/Page-Locked-Host-Memory-Data-Transfer/
  • smth-pinned-memory -Gao Y. What is the disadvantage of using pin_memory? In: PyTorch Forums [Internet]. 6 Apr 2017 [cited 19 Oct 2022]. Available: https://discuss.pytorch.org/t/what-is-the-disadvantage-of-using-pin-memory/1702
    • Dynamic Backlinks:
    CUDA Memory -
    Optimizing memory operations for CUDA

    L Ma (2022). 'CUDA Memory', Datumorphism, 10 April. Available at: https://datumorphism.leima.is/cards/machine-learning/practice/cuda-memory/.

    Dynamic Backlinks to cards/machine-learning/practice/cuda-memory:
    Pytorch Data Parallelism +
    Data parallelism in pytorch

    L Ma (2022). 'CUDA Memory', Datumorphism, 10 April. Available at: https://datumorphism.leima.is/cards/machine-learning/practice/cuda-memory/.

    diff --git a/cards/machine-learning/practice/learning-rate/index.html b/cards/machine-learning/practice/learning-rate/index.html index 4697a2a5a..1a070fb41 100644 --- a/cards/machine-learning/practice/learning-rate/index.html +++ b/cards/machine-learning/practice/learning-rate/index.html @@ -36,8 +36,8 @@ by ;

    References:
    1. Pointer2019 Pointer I. Programming PyTorch for deep learning: Creating and deploying deep learning applications. Sebastopol, CA: O’Reilly Media; 2019.
    2. fastaidocs Howard J, Thomas R. Hyperparam schedule. In: fastai [Internet]. [cited 30 Nov 2021]. Available: https://docs.fast.ai/callback.schedule.html#Learner.lr_find
    3. Howard&Gugger2020 -Howard J, Gugger S. Deep Learning for Coders with Fastai and PyTorch: AI Applications Without a PhD. O’Reilly Media, Incorporated; 2020. Available: https://www.oreilly.com/library/view/deep-learning-for/9781492045519/
    Dynamic Backlinks:
    Learning Rate -
    Find a good learning rate
    Links to:
    Artificial Neural Networks +Howard J, Gugger S. Deep Learning for Coders with Fastai and PyTorch: AI Applications Without a PhD. O’Reilly Media, Incorporated; 2020. Available: https://www.oreilly.com/library/view/deep-learning-for/9781492045519/
    Dynamic Backlinks to cards/machine-learning/practice/learning-rate:
    Differential Learning Rates in PyTorch +
    Using different learning rates in different layers of our artificial neural network.
    Links to:
    Artificial Neural Networks
    Simple artificial neural networks using multilayer perceptron
    Initialize Artificial Neural Networks
    Initialize a neural network is important for the training and performance. Some initializations …
    Additional Double Backet Links:
    Differential Learning Rates in PyTorch
    Using different learning rates in different layers of our artificial neural network.

    L Ma (2021). 'Learning Rate', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/machine-learning/practice/learning-rate/.

    diff --git a/cards/math/canonical-decomposition/index.html b/cards/math/canonical-decomposition/index.html index 13a1da58d..029a3aa8e 100644 --- a/cards/math/canonical-decomposition/index.html +++ b/cards/math/canonical-decomposition/index.html @@ -25,8 +25,8 @@ #Factorization #Linear Algebra

    I find this slide from Christoph Freudenthaler very useful.

    Canonical decomposition visualized by Christoph Freudenthaler

    Canonical decomposition visualized by Christoph Freudenthaler

    Planted: -by ;

    References:
    1. Matrix and Tensor Factorization from a Machine Learning Perspective
    Dynamic Backlinks:
    Canonical Decomposition -
    Canonical decomposition

    L Ma +by ;

    References:
    1. Matrix and Tensor Factorization from a Machine Learning Perspective
    Dynamic Backlinks to cards/math/canonical-decomposition:
    Tensor Factorization +
    A generalization of matrix factorization

    L Ma (2019). 'Canonical Decomposition', Datumorphism, 06 April. Available at: https://datumorphism.leima.is/cards/math/canonical-decomposition/.

    diff --git a/cards/math/cholesky-decomposition/index.html b/cards/math/cholesky-decomposition/index.html index e40121dbf..e5177e153 100644 --- a/cards/math/cholesky-decomposition/index.html +++ b/cards/math/cholesky-decomposition/index.html @@ -25,8 +25,8 @@ A = L L^T $$

    Planted: -by ;

    Dynamic Backlinks:
    Cholesky Decomposition -
    Decomposing a matrix into two

    L Ma +by ;

    Dynamic Backlinks to cards/math/cholesky-decomposition:
    Restricted Boltzmann Machine +
    Introducing latent variables to Boltzmann machine and restrict the connections within groups.

    L Ma (2019). 'Cholesky Decomposition', Datumorphism, 06 April. Available at: https://datumorphism.leima.is/cards/math/cholesky-decomposition/.

    diff --git a/cards/math/convolution-and-fourier-transform/index.html b/cards/math/convolution-and-fourier-transform/index.html index 701a0810f..a0d9c9939 100644 --- a/cards/math/convolution-and-fourier-transform/index.html +++ b/cards/math/convolution-and-fourier-transform/index.html @@ -48,8 +48,11 @@ \end{align} $$

    Planted: -by ;

    References:
    1. Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
    Dynamic Backlinks:
    Convolutions Using Fourier Transform -
    Convolution and Fourier transform

    L Ma +by ;

    References:
    1. Hamilton WL. Graph Representation Learning. Morgan & Claypool Publishers; 2020. pp. 1–159. doi:10.2200/S01045ED1V01Y202009AIM046
    Dynamic Backlinks to cards/math/convolution-and-fourier-transform:
    Convolutions Using Fourier Transform +
    Convolution and Fourier transform
    Graph Convolution Operator +
    For a given graph $\mathcal G$, we have an attribute on each node, denoted as $f_v$. All the node …
    Graph Laplacians +
    Laplacian is a useful representation of graphs. The unnormalized Laplacian is +$$ \mathbf L = \mathbf …

    L Ma (2021). 'Convolutions Using Fourier Transform', Datumorphism, 12 April. Available at: https://datumorphism.leima.is/cards/math/convolution-and-fourier-transform/.

    diff --git a/cards/math/convolution-dilated/index.html b/cards/math/convolution-dilated/index.html index 304c5f30a..6e6807e1f 100644 --- a/cards/math/convolution-dilated/index.html +++ b/cards/math/convolution-dilated/index.html @@ -40,9 +40,11 @@  ↩︎

    Planted: by ;

    References:
    1. Yu2015 -Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions. arXiv [cs.CV]. 2015. Available: http://arxiv.org/abs/1511.07122
    Similar Articles:
    Convolutions Using Fourier Transform -
    Convolution and Fourier transform
    Graph Convolution Operator -
    For a given graph $\mathcal G$, we have an attribute on each node, denoted as $f_v$. All the node …

    Lei Ma (2022). 'Dilated Convolution', Datumorphism, 08 April. Available at: https://datumorphism.leima.is/cards/math/convolution-dilated/.

    Dynamic Backlinks to cards/math/convolution-dilated:
    Dilated Convolution +
    For a convolution +$$ f*h(x) = \sum_{s+t=x} f(s) h(t), $$ +the dilated version of it is1 +$$ f*_l h(x) …

    Lei Ma (2022). 'Dilated Convolution', Datumorphism, 08 April. Available at: https://datumorphism.leima.is/cards/math/convolution-dilated/.

    diff --git a/cards/math/diagonalize-matrix/index.html b/cards/math/diagonalize-matrix/index.html index d1ab467cd..396761166 100644 --- a/cards/math/diagonalize-matrix/index.html +++ b/cards/math/diagonalize-matrix/index.html @@ -25,8 +25,9 @@ \mathbf A \mathbf x = \lambda \mathbf x, $$

    we know that the matrix $\mathbf A$ only scales the eigenvectors and no rotations. These directions are special to the matrix $\mathbf A$.

    1. Find the eigenvectors $\mathbf x_i$ of the matrix $\mathbf A$; If we find degerations, the matrix is not diagonalizable.
    2. Construct a matrix $\mathbf S = \begin{pmatrix} \mathbf x_1 & \mathbf x_2 & \cdots & \mathbf x_n \end{pmatrix}$;
    3. The matrix $\mathbf A$ is diagonalize using $\mathbf S^{-1} \mathbf A \mathbf S = \mathbf {A_D}$

    Planted: -by ;

    Dynamic Backlinks:
    Diagnolize Matrices -
    Diagnolizing a matrix is a transformation using its eigen space.
    Links to:
    Eigenvalues and Eigenvectors +by ;

    Dynamic Backlinks to cards/math/diagonalize-matrix:
    Eigenvalues and Eigenvectors +
    To find the eigenvectors $\mathbf x$ of a matrix $\mathbf A$, we construct the eigen equation +$$ …
    Links to:
    Eigenvalues and Eigenvectors
    To find the eigenvectors $\mathbf x$ of a matrix $\mathbf A$, we construct the eigen equation $$ …

    L Ma (2020). 'Diagnolize Matrices', Datumorphism, 03 April. Available at: https://datumorphism.leima.is/cards/math/diagonalize-matrix/.

    Planted: -by ;

    References:
    1. Eigenvectors and Eigenvalues @ Explained Visually
    Dynamic Backlinks:
    Eigenvalues and Eigenvectors -
    To find the eigenvectors $\mathbf x$ of a matrix $\mathbf A$, we construct the eigen equation -$$ …
    Links to:
    Diagnolize Matrices +by ;

    References:
    1. Eigenvectors and Eigenvalues @ Explained Visually
    Dynamic Backlinks to cards/math/eigendecomposition:
    NMF: Nonnegative Matrix Factorizatioin +
    Nonnegative Matrix Factorizatioin has a bright future
    Diagnolize Matrices +
    Diagnolizing a matrix is a transformation using its eigen space.
    Links to:
    Diagnolize Matrices
    Diagnolizing a matrix is a transformation using its eigen space.

    L Ma (2019). 'Eigenvalues and Eigenvectors', Datumorphism, 05 April. Available at: https://datumorphism.leima.is/cards/math/eigendecomposition/.

    diff --git a/cards/math/frobenius-distance/index.html b/cards/math/frobenius-distance/index.html index 513209ca5..b8f4714d6 100644 --- a/cards/math/frobenius-distance/index.html +++ b/cards/math/frobenius-distance/index.html @@ -27,8 +27,8 @@ \lVert X_{n}^{\phantom{n}k} - H_n^{\phantom{n}r} W_r^{\phantom{r}k} \rVert^2 \equiv \sum_{n,k} (X_{n}^{\phantom{n}k} - H_n^{\phantom{n}r} W_r^{\phantom{r}k})^2. $$

    Planted: -by ;

    References:
    1. Weisstein. Frobenius Norm. [cited 8 Nov 2021]. Available: https://mathworld.wolfram.com/FrobeniusNorm.html
    Dynamic Backlinks:
    Frobenius distance -
    Frobenius distance between the matrix $X_{n}^{\phantom{n}k}$ and $H_n^{\phantom{n}r} …

    L Ma +by ;

    References:
    1. Weisstein. Frobenius Norm. [cited 8 Nov 2021]. Available: https://mathworld.wolfram.com/FrobeniusNorm.html
    Dynamic Backlinks to cards/math/frobenius-distance:
    NMF: Nonnegative Matrix Factorizatioin +
    Nonnegative Matrix Factorizatioin has a bright future

    L Ma (2019). 'Frobenius distance', Datumorphism, 06 April. Available at: https://datumorphism.leima.is/cards/math/frobenius-distance/.

    diff --git a/cards/math/hubbard-stratonovich-identity/index.html b/cards/math/hubbard-stratonovich-identity/index.html index e166510cd..ee29c55ec 100644 --- a/cards/math/hubbard-stratonovich-identity/index.html +++ b/cards/math/hubbard-stratonovich-identity/index.html @@ -37,8 +37,8 @@  ↩︎

    Planted: by ;

    References:
    1. Hubbard1959 -Hubbard J. Calculation of Partition Functions. Physical Review Letters. 1959. pp. 77–78. doi:10.1103/physrevlett.3.77
    Dynamic Backlinks:
    The Hubbard-Stratonovich Identity -
    Very useful in calculating the partition function
    Dynamic Backlinks to cards/math/hubbard-stratonovich-identity:
    Restricted Boltzmann Machine +
    Introducing latent variables to Boltzmann machine and restrict the connections within groups.

    L Ma (2021). 'The Hubbard-Stratonovich Identity', Datumorphism, 06 April. Available at: https://datumorphism.leima.is/cards/math/hubbard-stratonovich-identity/.

    diff --git a/cards/math/jaccard-similarity/index.html b/cards/math/jaccard-similarity/index.html index 98da66888..2c32a4a20 100644 --- a/cards/math/jaccard-similarity/index.html +++ b/cards/math/jaccard-similarity/index.html @@ -41,8 +41,10 @@ $$

    Properties

    If the two sets are the same, $A=B$, we have $J(A,B)=1$ or $d_J(A,B)=0$. We have maximum similarity.

    If the two sets have nothing in common, we have $J(A,B)=0$ or $d_J(A,B)=1$. We have minimum similarity.

    Examples

    Word Set: (( sentenceOneWords ))
    Word Set: (( sentenceTwoWords ))
    Intersect: (( intersectWords ))
    Union: (( unionWords ))
    Jaccard Index: (( jaccardIndex ))
    Jaccard Distance: (( jaccardDistance ))

    Planted: -by ;

    References:
    1. Jaccard index
    Dynamic Backlinks:
    Jaccard Similarity -
    Jaccard index is the ratio of the size of the intersect of the set and the size of the union of the …

    L Ma +by ;

    References:
    1. Jaccard index
    Dynamic Backlinks to cards/math/jaccard-similarity:
    Statistics of Graphs +
    Local Statistics Node Degree Node Degree Node degree of a node $u$ $$ d_u = \sum_{v\in \mathcal V} …
    Cosine Similarity +
    As simple as the inner product of two vectors +$$ d_{cos} = \frac{\vec A}{\vert \vec A \vert} \cdot …

    L Ma (2019). 'Jaccard Similarity', Datumorphism, 05 April. Available at: https://datumorphism.leima.is/cards/math/jaccard-similarity/.

    diff --git a/cards/math/jensens-inequality/index.html b/cards/math/jensens-inequality/index.html index a55176ab1..b1393d304 100644 --- a/cards/math/jensens-inequality/index.html +++ b/cards/math/jensens-inequality/index.html @@ -29,10 +29,13 @@ f(\mathbb E(X)) \leq \mathbb E(f(X)) $$

    for a concave function $f(\cdot)$.

    Planted: -by ;

    References:
    1. Jensen's Inequality @ Wikipedia
    Dynamic Backlinks:
    Jensen's Inequality +by ;

    References:
    1. Jensen's Inequality @ Wikipedia
    Dynamic Backlinks to cards/math/jensens-inequality:
    Jensen's Inequality
    Jensen’s inequality shows that $$ f(\mathbb E(X)) \leq \mathbb E(f(X)) $$ -for a concave …
    Links to:
    Datumorphism +for a concave …
    Diffusion Models for Forecasting +
    Objective In a denoising diffusion model, given +an input $\mathbf x^0$ drawn from a complicated and …
    Evidence Lower Bound: ELBO +
    ELBO is an very important concept in variational methods
    Links to:
    Datumorphism

    LM (2021). 'Jensen's Inequality', Datumorphism, 04 April. Available at: https://datumorphism.leima.is/cards/math/jensens-inequality/.

    diff --git a/cards/math/level-set/index.html b/cards/math/level-set/index.html index 83a6150e5..ca93b5582 100644 --- a/cards/math/level-set/index.html +++ b/cards/math/level-set/index.html @@ -28,12 +28,8 @@  ↩︎

    Planted: by ;

    References:
    1. wiki-ls -Contributors to Wikimedia projects. Level set. In: Wikipedia [Internet]. 4 Nov 2022 [cited 12 Nov 2022]. Available: https://en.wikipedia.org/wiki/Level_set
    Similar Articles:
    Shatter -
    Given a set $\mathcal S$, and a class (collection of sets) $\mathcal H$. -For any subset of $\mathcal …
    Cosine Similarity -
    As simple as the inner product of two vectors -$$ d_{cos} = \frac{\vec A}{\vert \vec A \vert} \cdot …
    Jaccard Similarity -
    Jaccard index is the ratio of the size of the intersect of the set and the size of the union of the …
    Dynamic Backlinks to cards/math/level-set:
    Level Set +
    Level set can be used in ML

    L Ma (2022). 'Level Set', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/math/level-set/.

    diff --git a/cards/math/modes-and-slices-of-tensor/index.html b/cards/math/modes-and-slices-of-tensor/index.html index 007af075b..5d3506990 100644 --- a/cards/math/modes-and-slices-of-tensor/index.html +++ b/cards/math/modes-and-slices-of-tensor/index.html @@ -23,8 +23,8 @@ Current Note ID: The unique ID of this note.
    #Tensor
    Modes of a tensor
    Slices of a tensor

    Planted: -by ;

    References:
    1. Tensor Decompositions and Applications by Tamara G. Kolda and Brett W. Bader
    Dynamic Backlinks:
    Modes and Slices of Tensors -
    Simple decomposition of tensors

    L Ma +by ;

    References:
    1. Tensor Decompositions and Applications by Tamara G. Kolda and Brett W. Bader
    Dynamic Backlinks to cards/math/modes-and-slices-of-tensor:
    Tucker Decomposition +
    Tucker decomposition of a generalization of SVD to higher ranks

    L Ma (2019). 'Modes and Slices of Tensors', Datumorphism, 06 April. Available at: https://datumorphism.leima.is/cards/math/modes-and-slices-of-tensor/.

    diff --git a/cards/math/multiset-mset-bag/index.html b/cards/math/multiset-mset-bag/index.html index 42edf65b6..76118caf1 100644 --- a/cards/math/multiset-mset-bag/index.html +++ b/cards/math/multiset-mset-bag/index.html @@ -26,11 +26,9 @@ #Sets #Basics

    A bag is a set in which duplicate elements are allowed.

    An ordered bag is a list that we use in programming.

    Planted: -by ;

    References:
    1. Multiset @ Wikipedia
    Similar Articles:
    Conditional Probability Table -
    The conditional probability table is also called CPT
    Receiver Operating Characteristics: ROC -
    ROC is used to judging the performance of classifiers
    Covariance Matrix -
    Also known as the second central moment is a measurement of the spread.
    Data Types and Level of Measurement in Machine Learning -
    identifying the data types and level of measurement is important in data science

    LM (2020). 'Multiset, mset or bag', Datumorphism, 12 April. Available at: https://datumorphism.leima.is/cards/math/multiset-mset-bag/.

    Awesome Stuff

    Summarizations, workflows, experiences, fails, etc
    Awesome
    -#Fun
    Active

    I create workflows/checklists, and write about experiences here.

    Awesome (6 pages)

    +#Fun
    Active

    I create workflows/checklists, and write about experiences here.

    Awesome (6 pages)