# CSE5313 Coding and information theory for data science (Lecture 14)
## The repair problem
Main challenge:
- Locality (number of contacted servers)
- Bandwidth (number of bits transferred)
From last lecture we build optimal Local Recoverable Codes (LRCs) for storage systems.
Let $\mathcal{C} = [n, k]_q$ which is $r$-LRC, with minimum distance $d$.
- Bound 1: $\frac{k}{n}\leq \frac{r}{r+1}$.
- Bound 2: $d\leq n-k-\frac{k}{r}+2$.
- Optimal LRC:
- Let $\mathcal{A} = \{\alpha_1, \ldots, \alpha_n\}$, partition $\mathcal{A}$ to $\mathcal{A}_i$ for $i=1$ to $\frac{n}{r+1}$.
- $g\in \mathbb{F}_q[x]$ is good if $\deg(g) = r+1$ and $g$ is constant on all $\mathcal{A}_i$'s.
- $\mathcal{C} = \{f_a(\alpha_i)\}_{i=1}^{n}|a\in \mathbb{F}_q^k\}$, where
- $f_a(x) = \sum_{i=0}^{r-1} f_{a,i}(x)\cdot x^i$,
- $f_{a,i}(x) = \sum_{j=0}^{k/r-1} a_{i,j}\cdot g(x)^j$, where $g$ is a good polynomial.
- $g$ is a "good" polynomial.
- $\dim \mathcal{C} = k$ and $d = n-k-\frac{k}{r}+2$.
## Minimizing the repair bandwidth
Goal: understand repair bandwidth.
- What is the minimum repair bandwidth?
- Is repair bandwidth in trade-off with other parameters?
- Tool: The information flow graph.
Spoiler alert:
- Tradeoff: Storage Repair and bandwidth.
- Codes which achieve an optimal tradeoff.
### Information flow graph
We can model the repair problem as a directed graph.
- Source: System admin.
- Sink: Data collector.
- Nodes: Storage servers.
- Nodes leave/crash
- Newcomer replaces them $\to$ new nodes.
- Edges: Represents transmission of information. (Number of $\mathbb{F}_q$ elements is weight.)
Main observation:
- $k$ elements from $\mathbb{F}_q$ must "flow" from the source (system admin) to the sink (data collector).
- Any cut $(U,\overline{U})$ which separates source from sink must have capacity at least $k$.
Roadmap:
Information flow graph $\to$ Minimum cut analysis $\to$ Bound on file size $\to$ Storage/bandwidth tradeoff.
#### Basic definitions for information flow graph
> [!WARNING]
>
> This is not the same as definitions in linear codes. $k$ is not the dimension of the code and $d$ is not the minimum distance of the code for general cases.
Parameters:
- $n$ is the number of nodes in the initial system (before any node leaves/crashes).
- $k$ is the number of nodes required to reconstruct the file $k$.
- $d$ is the number of nodes required to repair a failed node.
- $\alpha$ is the storage at each node.
- $\beta$ is the edge capacity for repair.
- $B$ is the file size.
Goal: Find the trade off between $n,k,d,\alpha,\beta,B$ using min-cut analysis of the information flow graph.
Initial system:
We denote the system admin as $S$
Sever as $1,2,\ldots,n$, each with edge capacity $\alpha$.
For each new server:
We have two nodes $in$ and $out$, the edge weight is $\alpha$.
Connect to $d$ previous nodes $out$'s with edge capacity $\beta$.
Data collector:connects to $k$ arbitrary nodes with each edge capacity $\alpha$.
Observe that:
- File size $B$.
- Any cut separating $S$ form $DC$ must have capacity at least $B$.
- Otherwise, two different files are indistinguishable to $DC$.
#### Bound on bandwidth
Claim: $mincut\geq \sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
Intuition: Let $(U, \overline{U})$ be a cut separating $S$ form $DC$.
- The DC contacts newest $k$ nodes, say $n_1,n_2,\ldots,n_k$.
- The cut must decide if to cross $\alpha$ edges or $\beta$ edges.
- At east $d-i$ edges to go to $U$.
Proof
Let $(U, \overline{U})$ be a cut separating $S$ form $DC$, assuming $S\in U$ and $DC\in \overline{U}$.
Every directed acyclic graph has **topological sort**.
Let $x_{out}^1$ be the first $out$ node in $\overline{U}$. There are two cases:
- $x_{in}^1\in U$. Then $\alpha$ edges must be crossed.
- $x_{in}^1\in \overline{U}$. Then all $d$ incoming edges to $x_{in}^1$ must be crossed. (Otherwise, there exists an earlier $out$ node $x_{out}^j$ with $x_{in}^j\in U$, contradicting the topological sort.)
So $x_{out}^1$ contributes at least $\min\{d\beta, \alpha\}$ to the cut capacity.
Let $x_{out}^2$ be the second $out$ node in $\overline{U}$. There are two cases:
- $x_{in}^2\in U$. Then $\alpha$ edges must be crossed.
- $x_{in}^2\in \overline{U}$. Then at least $d-1$ incoming edges ($1$ edge may come from $x_{out}^1$) to $x_{in}^2$ must be crossed. (Otherwise, there exists an earlier $out$ node $x_{out}^j$ with $x_{in}^j\in U$, contradicting the topological sort.)
So $x_{out}^2$ contributes at least $\min\{(d-1)\beta, \alpha\}$ to the cut capacity.
By repeating this process, we can show that the minimum cut capacity is at least $\sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
#### Storage/bandwidth tradeoff
Claim: There exists an information graph with $mincut = \sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
Homework: Build this graph as follows:
- Construct the initial graph with $n$ nodes and the system admin.
- Add $n+k$ nodes, each node connects to the most recent $d$ nodes.
- Find the minimum cut capacity.
Corollary: $B\leq \sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$.
#### Definition of regenerate code
A code which attains $B=\sum_{i=1}^{k-1}\min\{(d-i)\beta, \alpha\}$ is called a regenerate code.
Goal: Find tradeoff between storage $\alpha$ to repair bandwidth $d\beta$.
Let $\gamma = d\beta$, then $B \leq \sum_{i=0}^{k-1}\min\{(1-i/d)\gamma, \alpha\}$.
Tool: Fix $\gamma$ and $d$, and minimize for $\alpha$ (not shown).
Result: The storage/bandwidth tradeoff.
- Each point on/above the line is feasible.
- Points on the line = regenerating codes.
- One endpoint: Minimum Bandwidth Regenerating (MBR) codes.
- another endpoint: Minimum Storage Regenerating (MSR) codes
For Minimum Storage Regenerating (MSR) codes, we have $\alpha = \frac{B}{k}$, $\beta = \frac{B}{k(d-k+1)}$
For Minimum Bandwidth Regenerating (MBR) codes, we have $\alpha = \frac{dB}{kd-\frac{k(k-1)}{2}}$, $\beta = \frac{B}{kd-\frac{k(k-1)}{2}}$
Notes:
- In MSR $\alpha=B/k$, Data collector contacts $k$ nodes and downloads $B/k$ from each to reconstruct the file of size $B$, that is optimal.
- In MBR $\beta=B/(kd-\frac{k(k-1)}{2})$, new comer download exactly what it stores, which is the same as replication. This has much smaller storage overhead in the replication.
Regenerating codes, Magic #1:
- MBR: Same repair-bandwidth as replication ($\alpha$), at lower storage costs.
- MSR: Same reconstruction-bandwidth ($B/k$) as replication, at lower storage costs.
Regenerating codes, Magic #2:
- In MSR: $\gamma = d\beta = \frac{dB}{k(d-k+1)}$, $\alpha = \frac{B}{k}$
- In MBR: $\gamma = d\beta = \frac{dB}{kd-\frac{k(k-1)}{2}}$, $\alpha = \frac{dB}{kd-\frac{k(k-1)}{2}}$
- Both decreasing functions of $d$.
- $\Rightarrow$ Less repair-bandwidth by contacting more nodes, minimized at $d = n - 1$.
### Constructing Minimum bandwidth regenerating (MBR) codes from Maximum distance separable (MDS) codes
Observation: For MBR code with parameters $n, k, d$ and $\beta = 1$, one can construct MBR with parameters $n, k, d$ and any $\beta$.
Next: Construct MBR for $[n, k, d = n - 1]$ and $\beta = 1$.
In any MBR: $\alpha, \beta = \frac{dB}{kd-\frac{k(k-1)}{2}}, \frac{B}{kd-\frac{k(k-1)}{2}}$
Specifically:
- Storage $\alpha = d\beta = d = n - 1$.
- File size $B = kd - \binom{k}{2}\beta = kd - \binom{k}{2}$
Take an $[\binom{n}{2}, B]$ MDS code (e.g., Reed-Solomon).
Need $q\geq \frac{n}{2}$.
Consider a complete graph $K_n$ on $n$ nodes.
- $\binom{n}{2}$ edges.
- Place each codeword symbol on a distinct edge.
- Storage server $i$ stores all codeword symbols adjacent with node $i$.
- $\alpha = n - 1$.
#### Repairing on MBR codes
New comer contacts each node $j\neq i$;
And downloads the symbol on edge $(i,j)$.
We get $\alpha=n-1=d\beta$ which is optimal.
#### Reconstruction on MBR codes
We use $[\binom{n}{2}, B]_q$ MDS code. So any $B$ symbols suffice to reconstruct the file.
Any $t$ nodes have $\binom{t}{2}$ edges between them, and $(n-1)t-2\binom{t}{2}$ edges to other nodes.
Overall $(n-1)t-\binom{t}{2}$. For $t=k$, we get $kd-\binom{k}{2}=B$.
### Constructing Minimum bandwidth regenerating (MBR) codes from Product-Matrix MBR codes
Recall: File size in MBR $B=kd-\binom{k}{2}=\binom{k+1}{2}+k(d-k)$.
Step 1: Arrange the $B=\binom{k+1}{2}+k(d-k)$ symbols in a matrix $M$ follows:
$$
M=\begin{pmatrix}
S & T\\
T^\top & 0
\end{pmatrix}\in \mathbb{F}_q^{d\times d}
$$
- $S$ is a $k\times k$ symmetric matrix. contains $\binom{k+1}{2}$ symbols.
- $T$ is a $k\times (d-k)$ matrix. contains $k(d-k)$ symbols.
So there are $B$ elements overall.
Step 2: Construct the encoding matrix $C=(\Psi,\Delta)\in \mathbb{F}_q^{n\times d}$
$\Psi\in \mathbb{F}_q^{n\times k}$ such that
- Any $k$ rows are linearly independent.
- Example: Vandermonde matrix.
$\Delta\in \mathbb{F}_q^{n\times (d-k)}$ such that
- Any $d$ rows of $C$ are linearly independent.
- Example: Complete $\Psi$ to a full $n\times d$ Vandermonde matrix.
Step 3: Encoding of the data $M\in \mathbb{F}_q^{d\times d}$ using the encoding matrix $C\in \mathbb{F}_q^{n\times d}$.
- Multiply $M$ by $C$.
- Store the $i$ the row of $CM$ in the node $i$.
- Note $CM=(\Psi,\Delta)M=(\Psi S+\Delta T, \Psi T)$
#### Repairing on Product-Matrix MBR codes
Assume node $i$ storing $c_iM$ is lost.
Repair from (any) nodes $H = \{h_1, \ldots, h_d\}$.
- Node $h_j$ stores $c_{h_j}M$.
Newcomer contacts each $h_j$: “My name is $i$, and I’m lost.”
Node $h_j$ sends $c_{h_j}M c_i^\top$ (inner product).
Newcomer assembles $C_H Mc_i^\top$.
$CH$ invertible by construction!
- Recover $Mc_i^\top$.
- Recover $c_i^\topM$ ($M$ is symmetric)
#### Reconstruction on Product-Matrix MBR codes
Data Collector (DC) contacts (any) nodes $D = \{d_1, \ldots, d_k\}$.
- Node $d_j$ stores $c_{d_j}M$.
Downloads $c_{d_j}M$ from node $d_j$.
DC assembles $C_D M$.
- Recall $CM=(\Psi S,\Delta)M=(\Psi S+\Delta T, \Psi T)$
- $C_D M=(\Psi_D S,\Delta_D)M=(\Psi_D S+\Delta_D T, \Psi_D T)$
$\Psi_D$ invertible by construction.
- DC computes $\Psi_D^{-1}C_DM = (S+\Psi_D^{-1}\Delta_D^\top, T)$
- DC obtains $T$.
- Subtracts $\Psi_D^{-1}\Delta_D T^\top$ from $S+\Psi_D^{-1}\Delta_D T^\top$ to obtain $S$.
Fill an example here please.