updates?

content/CSE5313/CSE5313_L14.md

@@ -0,0 +1,297 @@
+# CSE5313 Coding and information theory for data science (Lecture 14)
+
+## The repair problem
+
+Main challenge:
+
+- Locality (number of contacted servers)
+- Bandwidth (number of bits transferred)
+
+From last lecture we build optimal Local Recoverable Codes (LRCs) for storage systems.
+
+Let $\mathcal{C} = [n, k]_q$ which is $r$-LRC, with minimum distance $d$.
+
+- Bound 1: $\frac{k}{n}\leq \frac{r}{r+1}$.
+- Bound 2: $d\leq n-k-\frac{k}{r}+2$.
+
+- Optimal LRC:
+- Let $\mathcal{A} = \{\alpha_1, \ldots, \alpha_n\}$, partition $\mathcal{A}$ to $\mathcal{A}_i$ for $i=1$ to $\frac{n}{r+1}$.
+- $g\in \mathbb{F}_q[x]$ is good if $\deg(g) = r+1$ and $g$ is constant on all $\mathcal{A}_i$'s.
+- $\mathcal{C} = \{f_a(\alpha_i)\}_{i=1}^{n}|a\in \mathbb{F}_q^k\}$, where
+  - $f_a(x) = \sum_{i=0}^{r-1} f_{a,i}(x)\cdot x^i$,
+  - $f_{a,i}(x) = \sum_{j=0}^{k/r-1} a_{i,j}\cdot g(x)^j$, where $g$ is a good polynomial.
+  - $g$ is a "good" polynomial.
+- $\dim \mathcal{C} = k$ and $d = n-k-\frac{k}{r}+2$.
+
+## Minimizing the repair bandwidth
+
+Goal: understand repair bandwidth.
+
+- What is the minimum repair bandwidth?
+- Is repair bandwidth in trade-off with other parameters?
+- Tool: The information flow graph.
+
+Spoiler alert:
+
+- Tradeoff: Storage Repair and bandwidth.
+- Codes which achieve an optimal tradeoff.
+
+### Information flow graph
+
+We can model the repair problem as a directed graph.
+
+- Source: System admin.
+- Sink: Data collector.
+- Nodes: Storage servers.
+  - Nodes leave/crash
+  - Newcomer replaces them $\to$ new nodes.
+- Edges: Represents transmission of information. (Number of $\mathbb{F}_q$ elements is weight.)
+
+Main observation:
+
+- $k$ elements from $\mathbb{F}_q$ must "flow" from the source (system admin) to the sink (data collector).
+- Any cut $(U,\overline{U})$ which separates source from sink must have capacity at least $k$.
+
+Roadmap:
+
+Information flow graph $\to$ Minimum cut analysis $\to$ Bound on file size $\to$ Storage/bandwidth tradeoff.
+
+#### Basic definitions for information flow graph
+
+> [!WARNING]
+>
+> This is not the same as definitions in linear codes. $k$ is not the dimension of the code and $d$ is not the minimum distance of the code for general cases.
+
+Parameters:
+
+- $n$ is the number of nodes in the initial system (before any node leaves/crashes).
+- $k$ is the number of nodes required to reconstruct the file $k$.
+- $d$ is the number of nodes required to repair a failed node.
+- $\alpha$ is the storage at each node.
+- $\beta$ is the edge capacity for repair.
+- $B$ is the file size.
+
+Goal: Find the trade off between $n,k,d,\alpha,\beta,B$ using min-cut analysis of the information flow graph.
+
+Initial system:
+
+We denote the system admin as $S$
+
+Sever as $1,2,\ldots,n$, each with edge capacity $\alpha$.
+
+For each new server:
+
+We have two nodes $in$ and $out$, the edge weight is $\alpha$.
+
+Connect to $d$ previous nodes $out$'s with edge capacity $\beta$.
+
+Data collector:connects to $k$ arbitrary nodes with each edge capacity $\alpha$.
+
+Observe that:
+
+- File size $B$.
+- Any cut separating $S$ form $DC$ must have capacity at least $B$.
+- Otherwise, two different files are indistinguishable to $DC$.
+
+#### Bound on bandwidth
+
+Claim: $mincut\geq \sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
+
+Intuition: Let $(U, \overline{U})$ be a cut separating $S$ form $DC$.
+
+- The DC contacts newest $k$ nodes, say $n_1,n_2,\ldots,n_k$.
+- The cut must decide if to cross $\alpha$ edges or $\beta$ edges.
+- At east $d-i$ edges to go to $U$.
+
+<details>
+
+<summary>Proof</summary>
+
+Let $(U, \overline{U})$ be a cut separating $S$ form $DC$, assuming $S\in U$ and $DC\in \overline{U}$.
+
+Every directed acyclic graph has **topological sort**.
+
+Let $x_{out}^1$ be the first $out$ node in $\overline{U}$. There are two cases:
+
+- $x_{in}^1\in U$. Then $\alpha$ edges must be crossed.
+- $x_{in}^1\in \overline{U}$. Then all $d$ incoming edges to $x_{in}^1$ must be crossed. (Otherwise, there exists an earlier $out$ node $x_{out}^j$ with $x_{in}^j\in U$, contradicting the topological sort.)
+
+So $x_{out}^1$ contributes at least $\min\{d\beta, \alpha\}$ to the cut capacity.
+
+Let $x_{out}^2$ be the second $out$ node in $\overline{U}$. There are two cases:
+
+- $x_{in}^2\in U$. Then $\alpha$ edges must be crossed.
+- $x_{in}^2\in \overline{U}$. Then at least $d-1$ incoming edges ($1$ edge may come from $x_{out}^1$) to $x_{in}^2$ must be crossed. (Otherwise, there exists an earlier $out$ node $x_{out}^j$ with $x_{in}^j\in U$, contradicting the topological sort.)
+
+So $x_{out}^2$ contributes at least $\min\{(d-1)\beta, \alpha\}$ to the cut capacity.
+
+By repeating this process, we can show that the minimum cut capacity is at least $\sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
+
+</details>
+
+#### Storage/bandwidth tradeoff
+
+Claim: There exists an information graph with $mincut = \sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
+
+Homework: Build this graph as follows:
+
+- Construct the initial graph with $n$ nodes and the system admin.
+- Add $n+k$ nodes, each node connects to the most recent $d$ nodes.
+- Find the minimum cut capacity.
+
+Corollary: $B\leq \sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
+
+#### Definition of regenerate code
+
+A code which attains $B=\sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$ is called a regenerate code.
+
+Goal: Find tradeoff between storage $\alpha$ to repair bandwidth $d\beta$.
+
+Let $\gamma = d\beta$, then $B \leq \sum_{i=0}^{k-1}\min\{(1-i/d)\gamma, \alpha\}$.
+
+Tool: Fix $\gamma$ and $d$, and minimize for $\alpha$ (not shown).
+
+Result: The storage/bandwidth tradeoff.
+
+- Each point on/above the line is feasible.
+- Points on the line = regenerating codes.
+- One endpoint: Minimum Bandwidth Regenerating (MBR) codes.
+- another endpoint: Minimum Storage Regenerating (MSR) codes
+
+![Storage/bandwidth tradeoff](https://notenextra.trance-0.com/CSE5313/Storage_bandwidth_tradeoff.png)
+
+For Minimum Storage Regenerating (MSR) codes, we have $\alpha = \frac{B}{k}$, $\beta = \frac{B}{k(d-k+1)}$
+
+For Minimum Bandwidth Regenerating (MBR) codes, we have $\alpha = \frac{dB}{kd-\frac{k(k-1)}{2}}$, $\beta = \frac{B}{kd-\frac{k(k-1)}{2}}$
+
+Notes:
+
+- In MSR $\alpha=B/k$, Data collector contacts $k$ nodes and downloads $B/k$ from each to reconstruct the file of size $B$, that is optimal.
+- In MBR $\beta=B/(kd-\frac{k(k-1)}{2})$, new comer download exactly what it stores, which is the same as replication. This has much smaller storage overhead in the replication.
+
+Regenerating codes, Magic #1:
+
+- MBR: Same repair-bandwidth as replication ($\alpha$), at lower storage costs.
+- MSR: Same reconstruction-bandwidth ($B/k$) as replication, at lower storage costs.
+
+Regenerating codes, Magic #2:
+
+- In MSR: $\gamma = d\beta = \frac{dB}{k(d-k+1)}$, $\alpha = \frac{B}{k}$
+- In MBR: $\gamma = d\beta = \frac{dB}{kd-\frac{k(k-1)}{2}}$, $\alpha = \frac{dB}{kd-\frac{k(k-1)}{2}}$
+- Both decreasing functions of $d$.
+- $\Rightarrow$ Less repair-bandwidth by contacting more nodes, minimized at $d = n - 1$.
+
+### Constructing Minimum bandwidth regenerating (MBR) codes from Minimum distance codes
+
+Observation: For MBR code with parameters $n, k, d$ and $\beta = 1$, one can construct MBR with parameters $n, k, d$ and any $\beta$.
+
+Next: Construct MBR for $[n, k, d = n - 1]$ and $\beta = 1$.
+
+In any MBR: $\alpha, \beta = \frac{dB}{kd-\frac{k(k-1)}{2}}, \frac{B}{kd-\frac{k(k-1)}{2}}$
+
+Specifically:
+
+- Storage $\alpha = d\beta = d = n - 1$.
+- File size $B = kd - \binom{k}{2}\beta = kd - \binom{k}{2}$
+
+Take an $[\binom{n}{2}, B]$ MDS code (e.g., Reed-Solomon).
+
+Need $q\geq \frac{n}{2}$.
+
+Consider a complete graph $K_n$ on $n$ nodes.
+
+- $\binom{n}{2}$ edges.
+- Place each codeword symbol on a distinct edge.
+- Storage server $i$ stores all codeword symbols adjacent with node $i$.
+  - $\alpha = n - 1$.
+
+#### Repairing on MBR codes
+
+New comer contacts each node $j\neq i$;
+
+And downloads the symbol on edge $(i,j)$.
+
+We get $\alpha=n-1=d\beta$ which is optimal.
+
+#### Reconstruction on MBR codes
+
+We use $[\binom{n}{2}, B]_q$ MDS code. So any $B$ symbols suffice to reconstruct the file.
+
+Any $t$ nodes have $\binom{t}{2}$ edges between them, and $(n-1)t-2\binom{t}{2}$ edges to other nodes.
+
+Overall $(n-1)t-\binom{t}{2}$. For $t=k$, we get $kd-\binom{k}{2}=B$.
+
+### Constructing Minimum bandwidth regenerating (MBR) codes from Product-Matrix MBR codes
+
+Recall: File size in MBR $B=kd-\binom{k}{2}=\binom{k+1}{2}+k(d-k)$.
+
+Step 1: Arrange the $B=\binom{k+1}{2}+k(d-k)$ symbols in a matrix $M$ follows:
+
+$$
+M=\begin{pmatrix}
+S & T\\
+T^T & 0
+\end{pmatrix}\in \mathbb{F}_q^{d\times d}
+$$
+
+- $S$ is a $k\times k$ symmetric matrix. contains $\binom{k+1}{2}$ symbols.
+- $T$ is a $k\times (d-k)$ matrix. contains $k(d-k)$ symbols.
+
+So there are $B$ elements overall.
+
+Step 2: Construct the encoding matrix $C=(\Psi,\Delta)\in \mathbb{F}_q^{n\times d}$
+
+$\Psi\in \mathbb{F}_q^{n\times k}$ such that
+
+- Any $k$ rows are linearly independent.
+- Example: Vandermonde matrix.
+
+$\Delta\in \mathbb{F}_q^{n\times (d-k)}$ such that
+
+- Any $d$ rows of $C$ are linearly independent.
+- Example: Complete $\Psi$ to a full $n\times d$ Vandermonde matrix.
+
+Step 3: Encoding of the data $M\in \mathbb{F}_q^{d\times d}$ using the encoding matrix $C\in \mathbb{F}_q^{n\times d}$.
+
+- Multiply $M$ by $C$.
+- Store the $i$ the row of $CM$ in the node $i$.
+- Note $CM=(\Psi,\Delta)M=(\Psi S+\Delta T, \Psi T)$
+
+#### Repairing on Product-Matrix MBR codes
+
+Assume node $i$ storing $c_iM$ is lost.
+
+Repair from (any) nodes $H = \{h_1, \ldots, h_d\}$.
+
+- Node $h_j$ stores $c_{h_j}M$.
+
+Newcomer contacts each $h_j$: “My name is $i$, and I’m lost.”
+
+Node $h_j$ sends $c_{h_j}M c_i^T$ (inner product).
+
+Newcomer assembles $C_H Mc_i^T$.
+
+$CH$ invertible by construction!
+
+- Recover $Mc_i^T$.
+
+- Recover $c_i^TM$ ($M$ is symmetric)
+
+#### Reconstruction on Product-Matrix MBR codes
+
+Data Collector (DC) contacts (any) nodes $D = \{d_1, \ldots, d_k\}$.
+
+- Node $d_j$ stores $c_{d_j}M$.
+
+Downloads $c_{d_j}M$ from node $d_j$.
+
+DC assembles $C_D M$.
+
+- Recall $CM=(\Psi S,\Delta)M=(\Psi S+\Delta T, \Psi T)$
+- $C_D M=(\Psi_D S,\Delta_D)M=(\Psi_D S+\Delta_D T, \Psi_D T)$
+
+$\Psi_D$ invertible by construction.
+
+- DC computes $\Psi_D^{-1}C_DM = (S+\Psi_D^{-1}\Delta_D^T, T)$
+- DC obtains $T$.
+- Subtracts $\Psi_D^{-1}\Delta_D T^T$ from $S+\Psi_D^{-1}\Delta_D T^T$ to obtain $S$.

content/CSE5313/_meta.js

@@ -16,4 +16,5 @@ export default {
     CSE5313_L11: "CSE5313 Coding and information theory for data science (Recitation 11)",
     CSE5313_L12: "CSE5313 Coding and information theory for data science (Lecture 12)",
     CSE5313_L13: "CSE5313 Coding and information theory for data science (Lecture 13)",
+    CSE5313_L14: "CSE5313 Coding and information theory for data science (Lecture 14)",
 }