# CSE347 Analysis of Algorithms (Lecture 5)

## Takeaway from Bipartite Matching

- We saw how to solve a problem (bi-partite matching and others) by reducing it to another problem (maximum flow).
- In general, we can design an algorithm to map instances of a new problem to instances of known solvable problem (e.g., max-flow) to solve this new problem!
- Mapping from one problem to another which preserves solutions is called reduction.

## Reduction: Basic Ideas

Convert solutions to the known problem to the solutions to the new problem

- Instance of new problem
- Instance of known problem
- Solution of known problem
- Solution of new problem

## Reduction: Formal Definition

Problems $L,K$.

$L$ reduces to $K$ ($L\leq K$) if there is a mapping $\phi$ from **any** instance $l\in L$ to some instance $\phi(l)\in K'\subset K$, such that the solution for $\phi(l)$ yields a solution for $l$.

This means that **L is no harder than K**

### Using reduction to design algorithms

In the example of reduction to solve Bipartite Matching:

$L:$ Bipartite Matching

$K:$ Max-flow Problem

Efficiency:

1. Reduction: $\phi:l\to\phi(l)$ (Polynomial time reduction $\phi(l)$)
2. Solve prom $\phi(l)$ (Polynomial time to solve $poly(g)$)
3. Convert the solution for $\phi(l)$ to a solution to $l$ (Polynomial time to solve $poly(g)$)

### Efficient Reduction

A reduction $\phi:l\to\phi(l)$ is efficient ($L\leq p(k)$) if for any $l\in L$:

1. $\phi(l)$ is computable from $l$ in polynomial ($|l|$) time.
2. Solution to $l$ is computable from solution of $\phi(l)$ in polynomial ($|l|$) time.

We call $L$ is **poly-time reducible** to $K$, or $L$ poly-time
reduces to $K$.

### Which problem is harder?

Theorem: If $L\leq p(k)$ and there is a polynomial time algorithm to solve $K$, then there is a polynomial time algorithm to solve $L$.

Proof: Given an instance of $l\in L$ If we can convert the problem in polynomial time with respect to the original problem $l$.

1. Compute $\phi(l)$: $p(l)$
2. Solve $\phi(l)$: $p(\phi(l))$
3. Convert solution: $p(\phi(l))$

Total time: $p(l)+p(\phi(l))+p(\phi(l))=p(l)+p(\phi(l))$
Need to show: $|\phi(l)|=poly(|l|)$

Proof:

Since we can convert $\phi(l)$ in $p(l)$ time, and on every time step, (constant step) we can only write constant amount of data.

So $|\phi(l)|=poly(|l|)$

## Hardness Problems

Reductions show the relationship between problem hardness!

Question: Could you solve a problem in polynomial time?

Easy: polynomial time solution
Hard: No polynomial time solution (as far as we know)

### Types of Problems

Decision Problem: Yes/No answer

Examples: Subset sums

1. Is the there a flow of size $F$
2. Is there a shortest path of length $L$ from vertex $u$ to vertex $v$.
3. Given a set of intercal, can you schedule $k$ of them.

Optimization Problem: What is the value of an optimal feasible solution of a problem?

- Minimization: Minimize cost
  - min cut
  - minimal spanning tree
  - shortest path
- Maximization: Maximize profit
  - interval scheduling
  - maximum flow
  - maximum matching

#### Canonical Decision Problem

Does the instance $l\in L$ (an optimization problem) have a feasible solution with objective value $k$:

Objective value $\geq k$ (maximization) $\leq k$ (minimization)

$DL$ is the reduced Canonical Decision problem $L$

##### Hardness of Canonical Decision Problems

Lemma 1: $DL\leq p(L)$ ($DL$ is no harder than $L$)

Proof: Assume $L$ **maximization** problem $DL(l)$: does  have a solution $\geq k$.

Example: Does graph $G$ have flow $\geq k$.

Let $v^∗$  be the maximum objective on $l$ by solving $l$.

Let the instance of $DL:(l,k)$ and $l$ be the problem and $k$ be the objective

1. $l\to \phi(l)\in L$ (optimization problem) $\phi(l,k)=l$
2. Is $v^*(l)\geq k$? If so, return true, else return false.

Lemma 2: If $v^* =O(c^{|l|})$ for any constant $c$, then $L\leq p(DL)$.

Proof: First we could show  $L\leq DL$. Suppose maximization problem, canonical decision problem is is there a solution $\geq k$.

Naïve Linear Search: Ask $DL(l,k)$, if returns false, ask $DL(l,k+1)$ until returns true

Runtime: At most $k$ search to iterate all possibilities.

This is exponential! How to reduce it?

Our old friend Binary (exponential) Search is back!

You gets a no at some value: try power of 2 until you get a no, then do binary search

\# questions: $=log_2(v^*(l))=poly(l)$

Binary search in area: from last yes to first no.

Runtime: Binary search ($O(n)=\log(v^*(l))$)

### Reduction for Algorithm Design vs Hardness

For problems $L,K$

If $K$ is “easy” (exists a poly-time solution), then $L$ is also easy.

If $L$ is “hard” (no poly-time solution), then $k$ is also hard.

Every problem that we worked on so far, $K$ is “easy”, so we reduce from new problem  to known problem  (e.g., max-flow).

#### Reduction for Hardness: Independent Set (ISET)

Input: Given an undirected graph $G = (V,E)$, 

A subset of vertices $S\subset V$ is called an **independent set** if no two vertices of  are connected by an edge.

Problem: Does $G$ contain an independent set of size $\geq k$?

$ISET(G,k)$  returns true if $G$ contains an independent set of size $\geq k$, and false otherwise.

Algorithm? NO! We think that this is a hard problem.

A lot of people have tried and could not find a poly-time solution

### Example: Vertex Cover (VC)

Input: Given an undirected graph $G = (V,E)$

A subset of vertices $C\subset V$ is called a **vertex cover** if contains at least one end point of every edge.

Formally, for all edges $(u,v)\in E$, either $u\in C$, or $v\in C$.

Problem: $VC(G,j)$ returns true if has a vertex cover of size $\leq j$, and false otherwise (minimization problem)

Example:

#### How hard is Vertex Cover?

Claim:  $ISET\leq p(VC)$
Side Note: when we prove $VC$ is hard, we prove it is no easier than $ISET$.

DO NOT: $VC\leq p(ISET)$

Proof: Show that $G=(V,E)$ has an independent set of $k$ **if and only if** the same graph (not always!) has a vertex cover of size $|V|-k$.

Map:

$$
ISET(G,k)\to VC(g,|v|-k)
$$

$G'=G$

##### Proof of reduction: Direction 1

Claim 1: $ISET$ of size $k\to$ $VC$ of size $|V|-k$

Proof: Assume $G$ has an $ISET$ of size $k:S$, consider $C = V-S,|C|=|V|-k$

Claim: $C$ is a vertex cover

##### Proof of reduction: Direction 2


Claim 2: $VC$ of size $|V|-k\to ISET$ of size $k$

Proof: Assume $G$ has an $VC$ of size $|V| −k:C$, consider $S = V − C, |S| =k$

Claim: $S$ is an independent set

### What does poly-time mean?

Algorithm runs in time polynomial to input size.

- If the input has  items, algorithm runs in $\Theta(n^c)$ for any constant  is poly-time.
  - Examples: intervals to schedule, number of integers to sort, # vertices + # edges in a graph
- Numerical Value (Integer $n$), what is the input size?
  - Examples: weights, capacity, total time, flow constraints
  - It is not straightforward!

### Real time complexity of F-F?

In class: $O(F( |V| + |E|))$

- $|V| + |E|$ = this much space to represent the graph
- $F$ : size of the maximum flow.

If every edge has capacity , then $F = O(CE)$
Running time:$O(C|E|(|V|  + |E| )))$

### What is the actual input size?

Each edge ($|E|$ edges):

- 2 vertices: $|V|$ distinct symbol, $\log |V|$ bits per symbol
- 1 capacity: $\log C$

Size of graph:

- $O(|E|(|V| + \log C))$
  - $p( |E| , |V| , \log C)$

Running time:

- $P( |E| , |V| , |C| )$
  - Exponential if is exponential in $|V|+|E|$

### Pseudo-polynomial

Naïve Ford-Fulkerson is bad!

Problem ’s inputs contain some numerical values, say $|W|$. We need only log  bits to store . If algorithms runs in $p(W)$, then it is exponential, or **pseudopolynomial**.

In homework, you improved F-F to make it work in
$p( |V| ,|E|  , \log C)$, to make it a real polynomial algorithm.

## Conclusion: Reductions

- Reduction
  - Construction of mapping with runtime
  - Bidirectional proof
- Efficient Reduction $L\leq p(K)$
  - Which problem is harder?
  - If $L$ is hard, then $K$ is hard. $\to$ Used to show hardness
  - If $K$ is easy, then $L$ is easy. $\to$ Used for design algorithms
- Canonical Decision Problem
  - Reduction to and from the optimization problem
- Reduction for hardness
  - Independent Set$leq p$ Vertex Cover

## On class

Reduction: $V^* = O(c^k)$

OPT: Find max flow of at least one instance $(G,s,t)$

DEC: Is there a flow of size $pK$, given $G,s,t  \implies$ the instance is defined by the tuple $(G,s,t,k)$

Yes, if there exists one
No, otherwise

Forget about F-F and assume that you have an oracle that solves the decision problem.

First solution (the naive solution): iterate over $k = 1, 2, \dots$ until the oracle returns false and the last one returns true would be the max flow.

Time complexity: $K\cdot X$, where $X$ is the time complexity of the oracle
Input size: $poly(||V|,|E|, |E|log(max-capacity))$, and $V^* \leq \sum$ capacities

A better solution: do a binary search. If there is no upper bound, we use exponential binary search instead. Then,

$$
\begin{aligned}
log(V^*) &\leq X\cdot log(\sum capacities)\\
&\leq X\cdot log(|E|\cdot maxCapacity)\\
&\leq X\cdot (log(|E| + log(maxCapacity)))
\end{aligned}
$$
As $\log(maxCapacity)$ is linear in the size of the input, the running time is polynomial to the solution of the original problem.

Assume that ISET is a hard problem, i.e. we don't know of any polynomial time solution. We want to show that vertex cover is also a hard problem here:

$ISET \leq_{p} VC$

1. Given an instance of ISET, construct an instance of VC
2. Show that the construction can be done in polynomial time
3. Show that if the ISET instance is true than the CV instance is true
4. Show that if the VC instance is true then the ISET instance is true.

> ISET: given $(G,K)$, is there a set of vertices that do not share edges of size $K$  
> VC: given $(G,K)$, is there a set of vertices that cover all edges of size $K$

1. Given $l: (G,K)$ being an instance of ISET, we construct $\phi(l): (G',K')$ as an instance of VC. $\phi(l): (G, |V|-K), \textup{i.e., } G' = G \cup K' = |V| - K$
2. It is obvious that it is a polynomial time construction since copying the graph is linear, in the size of the graph and the subtraction of integers is constant time.

**Direction 1**: ISET of size k $\implies$ VC of size $|V| - K$ Assume that ISET(G,K) returns true, show that $VC(G, |V|-K)$ returns true

Let $S$ be an independent set of size $K$ and $C = V-S$

We claim that $C$ is a vertex cover of size $|V|-K$

Proof: 

We proceed by contradiction. Assume that $C$ is NOT a vertex cover, and it means that there is an edge $(u,v)$ such that $u\notin c , v\notin C$. And it implies that $u\in S , v\in S$, which contradicts with the assumption that S is an independent set.
Therefore, $c$ is an vertex cover

**Direction 2**: VC of size $|V|-K \implies$ ISET of size $K$

Let $C$ be a vertex cover of size $|V|-K$ , let $s = |v| - c$

We claim that $S$ is an independent set of size $K$.

Again, assume, for the sake of contradiction, that $S$ is not an independent set. And we get

$\exists (u,v) \textup{such that } u\in S, v \in S$

$u,v \notin C$

$C \textup{ is not a vertex cover}$

And this is a contradiction with our assumption.