# CSE5313 Coding and information theory for data science (Lecture 13)

## Recap from last lecture

Local recoverable codes:

An $[n, k]_q$ code is called $r$-locally recoverable if

- every codeword symbol $y_j$ has a recovering set $R_j \subseteq [n] \setminus j$ ($[n]=\{1,2,\ldots,n\}$),
- such that $y_j$ is computable from $y_i$ for all $i \in R_j$.
- $|R_j| \leq r$ for every $j \in n$.

### Bounds for locally recoverable codes

**Bound 1**: $\frac{k}{n}\leq \frac{r}{r+1}$.

**Bound 2**: $d\leq n-k-\lceil\frac{k}{r}\rceil +2$. ($r=k$ gives singleton bound)

#### Turan's Lemma

Let $G$ be a graph with $n$ vertices. Then there exists an induced directed acyclic subgraph (DAG) of $G$ on at least $\frac{n}{1+\operatorname{avg}_i(d^{out}_i)}$ nodes, where $d^{out}_i$ is the out-degree of vertex $i$.

Proof using probabilistic method.


<details>
<summary>Proof via the probabilistic method</summary>

> Useful for showing the existence of a large acyclic subgraph, but not for finding it.

> [!TIP]
>
> Show that $\mathbb{E}[X]\geq something$, and therefore there exists $U_\pi$ with $|U_\pi|\geq something$, using pigeonhole principle.

For a permutation $\pi$ of $[n]$, define $U_\pi = \{\pi(i): i \in [n]\}$.

Let $i\in U_\pi$ if each of the $d_i^{out}$ outgoing edges from $i$ connect to a node $j$ with $\pi(j)>\pi(i)$.

In other words, we select a subset of nodes $U_\pi$ such that each node in $U_\pi$ has an outgoing edge to a node in $U_\pi$ with a larger index. All edges going to right.

This graph is clearly acyclic.

Choose $\pi$ at random and Let $X=|U_\pi|$ be a random variable.

Let $X_i$ be the indicator random variable for $i\in U_\pi$.

So $X=\sum_{i=1}^{n} X_i$.

Using linearity of expectation, we have

$$
E[X]=\sum_{i=1}^{n} E[X_i]
$$

$E[X_i]$ is the probability that $\pi$ places $i$ before any of its out-neighbors.

For each node, there are $(d_i^{out}+1)!$ ways to place the node and its out-neighbors.

For each node, there are $d_i^{out}!$ ways to place the out-neighbors.

So, $E[X_i]=\frac{d_i^{out}!}{(d_i^{out}+1)!}=\frac{1}{d_i^{out}+1}$.

Since $X=\sum_{i=1}^{n} X_i$, we have

> [!TIP]
>
> Recall Arithmetic mean ($\frac{1}{n}\sum_{i=1}^{n} x_i$) is greater than or equal to harmonic mean ($\frac{n}{\sum_{i=1}^{n} \frac{1}{x_i}}$).

$$
\begin{aligned}
E[X]&=\sum_{i=1}^{n} E[X_i]\\
&=\sum_{i=1}^{n} \frac{1}{d_i^{out}+1}\\
&=\frac{1}{n}\sum_{i=1}^{n} \frac{1}{d_i^{out}+1}\\
&\geq \frac{n}{\frac{1}{n}(1+\sum_{i=1}^{n} d_i^{out})}\\
&=\frac{n}{1+\operatorname{avg}_i(d_i^{out})}\\
\end{aligned}
$$

</details>

## Locally recoverable codes

### Bound 1

$$
\frac{k}{n}\leq \frac{r}{r+1}
$$

<details>

<summary>Proof via Turan's Lemma</summary>

Let $G$ be a directed graph such that $i\to j$ is an edge if $j\in R_i$. ($j$ required to repair $i$)

$d_i^{out}\leq r$ for every $i$, so $\operatorname{avg}_i(d_i^{out})\leq r$.

By Turan's Lemma, there exists an induced directed acyclic subgraph (DAG) $G_U$ on $G$ with $U$ nodes such that $|U|\geq \frac{n}{1+\operatorname{avg}_i(d_i^{out})}\geq \frac{n}{1+r}$.

Since $G_U$ is a DAG, it is acyclic. So there exists a vertex $u_1\in U$ with no outgoing edges **inside** $G_U$. (leaf node in $G_U$)

Note that if we remove $u_1$ from $G_U$, the remaining graph is still a DAG.

Let $G_{U_1}$ be the induced graph on $U_1=U\setminus \{u_1\}$.

We can find a vertex $u_2\in U_1$ with no outgoing edges **inside** $G_{U_1}$.

We can continue this process until we have all the vertices in $U$ removed.

So all symbols in $U$ are redundant.

Note the number of redundant symbols $=n-k\geq \frac{n}{1+r}$.

So $k\leq \frac{rn}{r+1}$.

So $\frac{k}{n}\leq \frac{r}{r+1}$.

</details>

### Bound 2

$$
d\leq n-k-\lceil\frac{k}{r}\rceil +2
$$

#### Definition for code reduction

For $C=[n,k]_q$ code, let $I\subseteq [n]$ be a set of indices, e.g., $I=\{1,3,5,6,7,8\}$.

Let $C_I$ be the code obtained by deleting the symbols that is not in $I$. (only keep the symbols with indices in $I$)

Then $C_I$ is a code of length $|I|$, $C_I$ is generated by $G_I\in \mathbb{F}^{k\times |I|}_q$ (only keep the rows of $G$ with indices in $I$).

$|C_I|=q^{\operatorname{rank}(G_I)}$.

Note $G_I$ not necessarily of rank $k$.

#### Reduction lemma

Let $\mathcal{C}$ be an $[n, k]_q$ code. Then $d=n-\max\{|I|:C_I<q^k,I\subseteq [n]\}$.

<details>

<summary>Proof for reduction lemma</summary>

We proceed from two inequalities.

First $d\leq n-\max\{|I|:C_I<q^k,I\subseteq [n]\}$.

Let $I\subseteq [n]$ with $|C_I|\leq q^k$.

Then there exists $m_1,m_2\in \mathbb{F}_q^k$ such that $m_1G_I=m_2G_I$.

Let $y_1=m_1G_I$ and $y_2=m_2G_I$.

Then $y_1,y_2$ have at least $|I|$ entries in common.

So $d_H(y_1,y_2)\leq n-|I|$.

So $d\leq n-|I|$ for every $I$ such that $|C_I|\leq q^k$.

So $d\leq n-\max\{|I|:C_I<q^k,I\subseteq [n]\}$.

---

Now we show the other inequality.

$d\geq n-\max\{|I|:C_I<q^k,I\subseteq [n]\}$.

Let $y_1,y_2\in \mathcal{C}$ such that $d_H(y_1,y_2)=d$.

Let $J$ be the set on which $y_1$ and $y_2$ identical.

So $|J|=n-d$.

$|C_J|<q^k$ since $y_1$ and $y_2$ have at least $d$ entries in common.

So $\max\{|I|:C_I<q^k,I\subseteq [n]\}\geq n-d$. (by existence of $J$)

So $d\geq n-\max\{|I|:C_I<q^k,I\subseteq [n]\}$.

</details>

We prove bound 2 using the same subgraph from proof of bound 1.

<details>

<summary>Proof for bound 2</summary>

Let $G$ be a directed graph such that $i\to j$ is an edge if $j\in R_i$. ($j$ required to repair $i$)

$d_i^{out}\leq r$ for every $i$, so $\operatorname{avg}_i(d_i^{out})\leq r$.

By Turan's Lemma, there exists an induced directed acyclic subgraph (DAG) $G_U$ on $G$ with $U$ nodes such that $|U|\geq \frac{n}{1+\operatorname{avg}_i(d_i^{out})}\geq \frac{n}{1+r}$.

Let $U'\subset U$ with size $\lfloor\frac{k-1}{r}\rfloor$.

$U'$ exists since $\frac{n}{r+1}>\frac{k}{r}$ by bound 1, and $\frac{k}{r}>\lfloor\frac{k-1}{r}\rfloor$.

So $G_{U'}$ is also acyclic.

Let $N\subseteq [n]\setminus U'$ be the set of neighbors of $U'$ in $G_U$.

$|N|\leq r|U'|\leq k-1$.

Complete $n$ to be of the size $k-1$ by adding elements not in $U'$.

$|C_N|\leq q^{k-1}$

Also $|N\cup U'|=k-1+\lfloor\frac{k-1}{r}\rfloor$

Note that all nodes in $G_U$ can be recovered from nodes in $N$.

So $|C_{N\cup U'}|=|C_N|\leq q^{k-1}$.

Therefore, $\max\{|I|:C_I<q^k,I\subseteq [n]\}\geq |N\cup U'|=k-1+\lfloor\frac{k-1}{r}\rfloor$.

Using reduction lemma, we have $d= n-\max\{|I|:C_I<q^k,I\subseteq [n]\}\leq n-k-1-\lfloor\frac{k-1}{r}\rfloor+2=n-k-\lceil\frac{k}{r}\rceil +2$.

</details>

### Construction of locally recoverable codes

Recall Reed-Solomon code:

$\{f(a_1),f(a_2),\ldots,f(a_n)|f(x)\in \mathbb{F}_q^{k-1}[x]\}$.

- $\dim=k$
- $d=n-k+1$
- Need $n\geq q$.
- To encode $a=a_1,a_2,\ldots,a_n$, we need to evaluate $f_a(x)=\sum_{i=0}^{k-1}f_a(a_i)x^i$.

Assume $r+1|n$ and $r|k$.

Choose $A=\{\alpha_1,\alpha_2,\ldots,\alpha_n\}$

Partition $A$ into $\frac{n}{r+1}$ subsets of size $r+1$.

#### Definition for good polynomial

A polynomial $g$ over $\mathbb{F}_q$ is called good if

- $\deg(g)=r+1$
- $g$ is constant for every $A_i$., i.e., $\forall x,y\in A_i, g(x)=g(y)$.

To encode $a=(a_0,a_1,\ldots,a_n)$, we consider $a\in \mathbb{F}_q^{r\times (k/r)}$

Let $f_a(x)=\sum_{i=0}^{r-1} f_{a,i}(x)\cdot x^i$

$f_{a,i}(x)=\sum_{j=0}^{k/r-1} a_{i,j}\cdot g(x)^j$, where $g$ is a good polynomial.

<details>
<summary>Proof for valid codeword</summary

If $a\neq b$, then $(f_a(\alpha_i))_{i=1}^{n}\neq (f_b(\alpha_i))_{i=1}^{n}$.

- $\deg f_a = k-2$
- Maximum degree monomial in $f_a$:
$a_{r-1,k/r-1}\cdot g(x)^{k/r-1}\cdot x^{r-1}$.
- $\deg f_a \leq \frac{k}{r}-1+\frac{k}{r}-1=k-2$.

By Bound 1: $k-2\leq n-2$.

If $f_a(\alpha_i)_{i=1}^{n}=f_b(\alpha_i)_{i=1}^{n}$. then

- $f_a$ and $f_b$ agree on more points than their degree.
- $a=b$, a contradiction.

Corollary: $|C|=q^k$.

</details>

In other words,

$$
\mathcal{C}=\{(f_a(\alpha_i))_{i=1}^{n}|a\in \mathbb{F}_q^k\}
$$

For data $a\in \mathbb{F}_q^k$, the $i$th server stores $f_a(\alpha_i)$.

Note $\mathcal{C}$ is a linear code. and $\dim \mathcal{C}=k$.

Let $\alpha\in \{\alpha_1,\alpha_2,\ldots,\alpha_n\}$.

Suppose $\alpha\in A_1$.

Locality is ensured by recovering failed server $\alpha$ by $f_a(\alpha_i)$ from $\{f_a(\beta):\beta\in A_1\setminus \{\alpha\}\}$.

Note $g$ is constant on $A_1$.

Denote the constant by $f_i(A_1)$.

Let $\delta_1(x)=\sum_{i=0}^{r-1} f_{a,i}(A_1)x^i$.

$\delta_1(x)=f_a(x)$ for all $x\in A_1$.

#### Constructing good polynomial

Let $(\mathbb{F}_q^*,\cdot)$ be the multiplicative group of $\mathbb{F}_q$.

For a subgroup $H$ of this group, a multiplicative coset is $Ha=\{ha|h\in H,a\in \mathbb{F}_q^*\}$.

The family of all cosets of $H:\{Ha|a\in \mathbb{F}_q^*\}$.

This is a partition of $\mathbb{F}_q^*$.

#### Definition of annihilator

The annihilator of $H$ is $g(x)=\prod_{h\in H}(x-h)$.

Note that $g(h)=0$ for all $h\in H$.

#### Annihilator is a good polynomial

Fix $H$, consider the partition $\{Ha|a\in \mathbb{F}_q^*\}$ and let $g(x)=\prod_{a\in \mathbb{F}_q^*}(x-a)$. Then $g$ is a good polynomial.

<details>
<summary>Proof</summary>

Need to show if $x,y\in Ha$, then $g(x)=g(y)$.

Since $x\in Ha$, there exists $h',h''\in H$ such that $x=h'a$ and $y=h''a$.

So $g(x)=g(h'a)=\prod_{h\in H}(h'a-h)=(h')^{|H|}\prod_{h\in H}(a-h/h')$.

By Fermat's Little Theorem, $(h')^{|H|}\equiv 1$ for every $h'\in H$.

So $\{h/h'|h\in H\}=H$ for every $h'\in H$.

So $g(x)=(h')^{|H|}\prod_{h\in H}(a-h/h')=\prod_{h\in H}(a-h/h')=g(a)$.

Similarly, $g(y)=g(h''a)=\prod_{h\in H}(h''a-h)=(h'')^{|H|}\prod_{h\in H}(a-h/h'')=g(a)$.

</details>