63 lines
998 B
Markdown
63 lines
998 B
Markdown
# Coding and Information Theory Crash Course
|
|
|
|
## Encoding
|
|
|
|
Let $A,B$ be two finite sets with size $a,b$ respectively.
|
|
|
|
Let $S(A)=\bigcup_{r=1}^{\infty}A^r$ be the word semigroup generated by $A$.
|
|
|
|
A one-to-one mapping $f:A\to S(B)$ is called a code with message alphabet $A$ and encoded alphabet $B$.
|
|
|
|
Example:
|
|
|
|
- $A=$ RGB color space
|
|
- $B=\{0\sim 255\}$
|
|
- $f:A\to B^n$ is a code
|
|
|
|
For example, $f(white)=(255,255,255)$, $f(green)=(0,255,0)$
|
|
|
|
### Uniquely decipherable codes
|
|
|
|
A code $f:A\to S(B)$ is called uniquely decipherable if the extension code
|
|
|
|
$$
|
|
\tilde{f}:S(A)\to S(B)=f(a_1)f(a_2)\cdots f(a_n)
|
|
$$
|
|
|
|
is one-to-one.
|
|
|
|
Example:
|
|
|
|
- $A=\{a,b,c,d\}$
|
|
- $B=\{0,1\}$
|
|
- $f(a)=00$, $f(b)=01$, $f(c)=10$, $f(d)=11$
|
|
|
|
is uniquely decipherable.
|
|
|
|
- $f(a)=0$, $f(b)=1$, $f(c)=10$, $f(d)=11$
|
|
|
|
is not uniquely decipherable.
|
|
|
|
Since $\tilde{f}(ba)=10=\tilde{f}(c)$
|
|
|
|
#### Irreducible codes
|
|
|
|
A code $f:A\to S(B)$ is called irreducible if for any $x,y\in A$, $f(y)\neq f(x)w$ for some $w\in S(B)$.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|