# Coding and Information Theory Crash Course

## Encoding

Let $A,B$ be two finite sets with size $a,b$ respectively.

Let $S(A)=\bigcup_{r=1}^{\infty}A^r$ be the word semigroup generated by $A$.

A one-to-one mapping $f:A\to S(B)$ is called a code with message alphabet $A$ and encoded alphabet $B$.

Example:

- $A=$ RGB color space
- $B=\{0\sim 255\}$
- $f:A\to B^n$ is a code

For example, $f(white)=(255,255,255)$, $f(green)=(0,255,0)$

### Uniquely decipherable codes

A code $f:A\to S(B)$ is called uniquely decipherable if the extension code

$$
\tilde{f}:S(A)\to S(B)=f(a_1)f(a_2)\cdots f(a_n)
$$

is one-to-one.

Example:

- $A=\{a,b,c,d\}$
- $B=\{0,1\}$
- $f(a)=00$, $f(b)=01$, $f(c)=10$, $f(d)=11$

is uniquely decipherable.

- $f(a)=0$, $f(b)=1$, $f(c)=10$, $f(d)=11$

is not uniquely decipherable.

Since $\tilde{f}(ba)=10=\tilde{f}(c)$

#### Irreducible codes

A code $f:A\to S(B)$ is called irreducible if for any $x,y\in A$, $f(y)\neq f(x)w$ for some $w\in S(B)$.