NoteNextra-origin/pages/Swap/Math401/Math401_N3.md

Coding and Information Theory Crash Course

Encoding

Let A,B be two finite sets with size a,b respectively.

Let S(A)=\bigcup_{r=1}^{\infty}A^r be the word semigroup generated by A.

A one-to-one mapping f:A\to S(B) is called a code with message alphabet A and encoded alphabet B.

Example:

• A= RGB color space
• B=\{0\sim 255\}
• f:A\to B^n is a code

For example, f(white)=(255,255,255), f(green)=(0,255,0)

Uniquely decipherable codes

A code f:A\to S(B) is called uniquely decipherable if the extension code

\tilde{f}:S(A)\to S(B)=f(a_1)f(a_2)\cdots f(a_n)

is one-to-one.

Example:

• A=\{a,b,c,d\}
• B=\{0,1\}
• f(a)=00, f(b)=01, f(c)=10, f(d)=11

is uniquely decipherable.

• f(a)=0, f(b)=1, f(c)=10, f(d)=11

is not uniquely decipherable.

Since \tilde{f}(ba)=10=\tilde{f}(c)

Irreducible codes

A code f:A\to S(B) is called irreducible if for any x,y\in A, f(y)\neq f(x)w for some w\in S(B).