234 lines
6.5 KiB
Markdown
234 lines
6.5 KiB
Markdown
# CSE5313 Coding and information theory for data science (Lecture 12)
|
|
|
|
Challenge 1: Reconstruction
|
|
|
|
- Minimize reconstruction bandwidth.
|
|
|
|
Challenge 2: Repair
|
|
|
|
- Maintaining data consistency.
|
|
- Failed servers must be repaired:
|
|
- By contacting few other servers (locality, due to geographical constraints).
|
|
- By minimizing bandwidth.
|
|
|
|
Challenge 3: Storage overhead
|
|
|
|
- Minimize space consumption.
|
|
- Minimize redundancy.
|
|
|
|
## Code for storage systems
|
|
|
|
### Naive solution: Replication
|
|
|
|
Locality is 1 (by copying from another server).
|
|
|
|
This gives the optimal reconstruction bandwidth.
|
|
|
|
### Use codes to improve storage efficiency
|
|
|
|
Locality is $n-d+1$, high bandwidth.
|
|
|
|
#### Parity codes
|
|
|
|
Let $X_1,X_2,\ldots,X_n\in \mathbb{F}_2^t$ be the data blocks, take extra server to store the parity.
|
|
|
|
Reconstruction:
|
|
|
|
Optimal for reconstruction bandwidth. Only need $k$ servers to reconstruct the file.
|
|
|
|
Overhead:
|
|
|
|
Only need one additional server
|
|
|
|
Repair:
|
|
|
|
Any server failed, reconstruct from the other $n-d+1=n-2+1=n-1$ servers.
|
|
|
|
#### Reed-Solomon codes
|
|
|
|
Fragment the file $X = (X_1, \ldots, X_k)$.
|
|
|
|
Need $2^t\geq n$ servers to store the file.
|
|
|
|
Reconstruction:
|
|
|
|
Any $k$ servers can reconstruct the file.
|
|
|
|
Overhead:
|
|
|
|
Need $2^t\geq n$ servers to store the file.
|
|
|
|
Repair:
|
|
|
|
Worse, need all servers to reconstruct the file.
|
|
|
|
### New codes for storage systems
|
|
|
|
#### EVENODD code
|
|
|
|
- One of the first storage specific codes.
|
|
|
|
Can a xor only code be built that enables reconstruction if two disks are missing?
|
|
|
|
locality/bandwidth problem for next lecture.
|
|
|
|
For prime $m$, partition $X=(X_0,\ldots,X_{m-1})$ each $X_i$ with $m-1$ bits.
|
|
|
|
Store $Y_i=X_i$ in disks $0,1,\ldots,m-1$.
|
|
|
|
Add two redundant disks $Y_m,Y_{m+1}$.
|
|
|
|
- $(Y_m)_i$ is the parity of row $i$.
|
|
- $(Y_{m+1})_i$, first we defined $S=a_{0,4}+a_{1,3}+a_{2,2}+a_{3,1}$, then $(Y_{m+1})_i=S\oplus \sum_{j=0}^{m-2}a_{(i,j)\mod m,j}$
|
|
|
|
| $Y_0$ | $Y_1$ | $Y_2$ | $Y_3$ | $Y_4$ | $Y_5$ | $Y_6$ |
|
|
|-------|-------|-------|-------|-------|-------|-------|
|
|
| $1$ | $0$ | $1$ | $1$ | $0$ | $1$ | $0$ |
|
|
| $0$ | $1$ | $1$ | $0$ | $0$ | $0$ | $0$ |
|
|
| $1$ | $1$ | $0$ | $0$ | $0$ | $0$ | $1$ |
|
|
| $0$ | $1$ | $0$ | $1$ | $1$ | $1$ | $0$ |
|
|
|
|
Note that the $S$ diagonal can be extracted from $Y_m$ and $Y_{m+1}$.
|
|
|
|
$$
|
|
\sum_{j=0}^{m-2}(Y_m)_j\oplus \sum_{j=0}^{m-2}(Y_{m+1})_j=\sum_{j=1}^{m}S=S
|
|
$$
|
|
|
|
Goal: Reconstruct if any two disks are missing.
|
|
|
|
- If $Y_m, Y_{m+1}$ missing, nothing to do.
|
|
- If $Y_i, Y_{m+1}$ are missing for $i < m$, decode like a parity code.
|
|
- If $Y_i, Y_{m}$ are missing for $i < m$, similar, using diagonal parities.
|
|
|
|
The interesting case: $Y_i, Y_j$ are missing for $i,j < m$.
|
|
|
|
Using the skill you solve sudoku puzzles, we can find the missing values.
|
|
|
|
First we recover the $S$ diagonal from $Y_m$ and $Y_{m+1}$.
|
|
|
|
Then we solve for the row by $Y_m$ and the diagonal by $Y_{m+1}$.
|
|
|
|
<details>
|
|
<summary>Proof for why it always works</summary>
|
|
|
|
There are $m-1$ rows, $m$ including a ghost row with full $0$s.
|
|
|
|
$\mathbb{Z}_m$ is cyclic of prime size, any non-zero element is a generator.
|
|
|
|
When moving from diagonal to horizontal, we are moving some offset from the diagonal, which are always generator.
|
|
|
|
</details>
|
|
|
|
This is an example of array code:
|
|
|
|
The message $(X_0,X_1,\ldots,X_{m-1})$ is a matrix in $\mathbb{F}_2^{(m-1)\times m}$.
|
|
|
|
The codeword $(Y_0,Y_1,\ldots,Y_{m+1})$ is a matrix in $\mathbb{F}_2^{(m-1)\times (m+2)}$.
|
|
|
|
Encoding is done over $\mathbb{F}_q$.
|
|
|
|
## Locally Recoverable Codes
|
|
|
|
Locality: when a node $j$ fails,
|
|
|
|
- A newcomer node joins the system.
|
|
- The newcomer contacts a "small" number of helper nodes with the message "repairing $j$".
|
|
- Each of the helper nodes sends something to the newcomer.
|
|
- The newcomer aggregates the responses to find $Y_j$.
|
|
|
|
Notes:
|
|
|
|
- No adversarial behavior.
|
|
- No privacy issues.
|
|
- No concern about bandwidth (for now).
|
|
|
|
Research question:
|
|
|
|
- How small can the "small number of nodes" be?
|
|
- How does that affect the rate/minimum distance of the code?
|
|
- How to build codes with this capability?
|
|
|
|
### Definition of locally recoverable code
|
|
|
|
An $[n, k]_q$ code is called $r$-locally recoverable if
|
|
|
|
- every codeword symbol $y_j$ has a recovering set $R_j \subseteq [n] \setminus j$ ($[n]=\{1,2,\ldots,n\}$),
|
|
- such that $y_j$ is computable from $y_i$ for all $i \in R_j$.
|
|
- $|R_j| \leq r$ for every $j \in n$.
|
|
|
|
Notes:
|
|
|
|
- From $n-d+1$ nodes, we can reconstruct the entire file, always assume $k\leq n-d+1$.
|
|
- We want $r\ll n-d+1$.
|
|
- $R_j$ does not depend on $y_j$, nor on the codewords $y$, only on $j$. (Need to repair without knowing $y,y_j$.)
|
|
|
|
### Bounds for Locally Recoverable Codes
|
|
|
|
Let $\mathcal{C}$ be an $[n, k]_q$ code with $r$-locally recoverable, with minimum distance $d$.
|
|
|
|
Bound 1: $\frac{k}{n}\leq \frac{r}{r+1}$.
|
|
|
|
Bound 2: $d\leq n-k-\lceil\frac{k}{r}\rceil +2$.
|
|
|
|
Notes:
|
|
|
|
For $r=k$, bound 2 becomes $d\leq n-k+1$.
|
|
|
|
- The natural extension of singleton bound.
|
|
|
|
For $r=1$, bound 1 becomes $\frac{k}{n}\leq \frac{1}{2}$.
|
|
|
|
- The duplication code is trivial code for this bound
|
|
|
|
For $r=1$, bound 2 becomes $d\leq n-2k+2$.
|
|
|
|
- The duplication code is trivial code for this bound
|
|
|
|
### Bound 1
|
|
|
|
#### Turan's Lemma
|
|
|
|
Let $G$ be a graph with $n$ vertices. Then there exists an induced directed acyclic subgraph (DAG) of $G$ on at least $\frac{n}{1+\avg_i(d^{out}_i)}$ nodes, where $d^{out}_i$ is the out-degree of vertex $i$.
|
|
|
|
> Directed graphs have large acyclic subgraphs.
|
|
|
|
<details>
|
|
<summary>Proof via the probabilistic method</summary>
|
|
|
|
> Useful for showing the existence of a large acyclic subgraph, but not for finding it.
|
|
|
|
> [!TIP]
|
|
>
|
|
> Show that $\mathbb{E}[X]\geq something$, and therefore there exists $U_\pi$ with $|U_\pi|\geq something$, using pigeonhole principle.
|
|
|
|
For a permutation $\pi$ of $[n]$, define $U_\pi = \{\pi(i): i \in [n]\}$.
|
|
|
|
Let $i\in U_\pi$ if each of the $d_i^{out}$ outgoing edges from $i$ connect to a node $j$ with $\pi(j)>\pi(i)$.
|
|
|
|
In other words, we select a subset of nodes $U_\pi$ such that each node in $U_\pi$ has an outgoing edge to a node in $U_\pi$ with a larger index. All edges going to right.
|
|
|
|
This graph is clearly acyclic.
|
|
|
|
Choose $\pi$ at random and Let $X=|U_\pi|$ be a random variable.
|
|
|
|
Let $X_i$ be the indicator random variable for $i\in U_\pi$.
|
|
|
|
So $X=\sum_{i=1}^{n} X_i$.
|
|
|
|
Using linearity of expectation, we have
|
|
|
|
$$
|
|
E[X]=\sum_{i=1}^{n} E[X_i]
|
|
$$
|
|
|
|
$E[X_i]$ is the probability that $\pi$ places $i$ before any of its out-neighbors.
|
|
|
|
For each node, there are $(d_i^{out}+1)!$ ways to place the node and its out-neighbors.
|
|
|
|
For each node, there are $d_i^{out}!$ ways to place the out-neighbors.
|
|
|
|
So, $E[X_i]=\frac{d_i^{out}!}{(d_i^{out}+1)!}=\frac{1}{d_i^{out}+1}$.
|
|
|
|
Continue next time.
|
|
|
|
</details> |