upgrade structures and migrate to nextra v4
This commit is contained in:
Zheyuan Wu
341
content/CSE347/CSE347_L5.md
Normal file
341
content/CSE347/CSE347_L5.md
Normal file
@@ -0,0 +1,341 @@
|
||||
# Lecture 5
|
||||
|
||||
## Takeaway from Bipartite Matching
|
||||
|
||||
- We saw how to solve a problem (bi-partite matching and others) by reducing it to another problem (maximum flow).
|
||||
- In general, we can design an algorithm to map instances of a new problem to instances of known solvable problem (e.g., max-flow) to solve this new problem!
|
||||
- Mapping from one problem to another which preserves solutions is called reduction.
|
||||
|
||||
## Reduction: Basic Ideas
|
||||
|
||||
Convert solutions to the known problem to the solutions to the new problem
|
||||
|
||||
- Instance of new problem
|
||||
- Instance of known problem
|
||||
- Solution of known problem
|
||||
- Solution of new problem
|
||||
|
||||
## Reduction: Formal Definition
|
||||
|
||||
Problems $L,K$.
|
||||
|
||||
$L$ reduces to $K$ ($L\leq K$) if there is a mapping $\phi$ from **any** instance $l\in L$ to some instance $\phi(l)\in K'\subset K$, such that the solution for $\phi(l)$ yields a solution for $l$.
|
||||
|
||||
This means that **L is no harder than K**
|
||||
|
||||
### Using reduction to design algorithms
|
||||
|
||||
In the example of reduction to solve Bipartite Matching:
|
||||
|
||||
$L:$ Bipartite Matching
|
||||
|
||||
$K:$ Max-flow Problem
|
||||
|
||||
Efficiency:
|
||||
|
||||
1. Reduction: $\phi:l\to\phi(l)$ (Polynomial time reduction $\phi(l)$)
|
||||
2. Solve prom $\phi(l)$ (Polynomial time to solve $poly(g)$)
|
||||
3. Convert the solution for $\phi(l)$ to a solution to $l$ (Polynomial time to solve $poly(g)$)
|
||||
|
||||
### Efficient Reduction
|
||||
|
||||
A reduction $\phi:l\to\phi(l)$ is efficient ($L\leq p(k)$) if for any $l\in L$:
|
||||
|
||||
1. $\phi(l)$ is computable from $l$ in polynomial ($|l|$) time.
|
||||
2. Solution to $l$ is computable from solution of $\phi(l)$ in polynomial ($|l|$) time.
|
||||
|
||||
We call $L$ is **poly-time reducible** to $K$, or $L$ poly-time
|
||||
reduces to $K$.
|
||||
|
||||
### Which problem is harder?
|
||||
|
||||
Theorem: If $L\leq p(k)$ and there is a polynomial time algorithm to solve $K$, then there is a polynomial time algorithm to solve $L$.
|
||||
|
||||
Proof: Given an instance of $l\in L$ If we can convert the problem in polynomial time with respect to the original problem $l$.
|
||||
|
||||
1. Compute $\phi(l)$: $p(l)$
|
||||
2. Solve $\phi(l)$: $p(\phi(l))$
|
||||
3. Convert solution: $p(\phi(l))$
|
||||
|
||||
Total time: $p(l)+p(\phi(l))+p(\phi(l))=p(l)+p(\phi(l))$
|
||||
Need to show: $|\phi(l)|=poly(|l|)$
|
||||
|
||||
Proof:
|
||||
|
||||
Since we can convert $\phi(l)$ in $p(l)$ time, and on every time step, (constant step) we can only write constant amount of data.
|
||||
|
||||
So $|\phi(l)|=poly(|l|)$
|
||||
|
||||
## Hardness Problems
|
||||
|
||||
Reductions show the relationship between problem hardness!
|
||||
|
||||
Question: Could you solve a problem in polynomial time?
|
||||
|
||||
Easy: polynomial time solution
|
||||
Hard: No polynomial time solution (as far as we know)
|
||||
|
||||
### Types of Problems
|
||||
|
||||
Decision Problem: Yes/No answer
|
||||
|
||||
Examples: Subset sums
|
||||
|
||||
1. Is the there a flow of size $F$
|
||||
2. Is there a shortest path of length $L$ from vertex $u$ to vertex $v$.
|
||||
3. Given a set of intercal, can you schedule $k$ of them.
|
||||
|
||||
Optimization Problem: What is the value of an optimal feasible solution of a problem?
|
||||
|
||||
- Minimization: Minimize cost
|
||||
- min cut
|
||||
- minimal spanning tree
|
||||
- shortest path
|
||||
- Maximization: Maximize profit
|
||||
- interval scheduling
|
||||
- maximum flow
|
||||
- maximum matching
|
||||
|
||||
#### Canonical Decision Problem
|
||||
|
||||
Does the instance $l\in L$ (an optimization problem) have a feasible solution with objective value $k$:
|
||||
|
||||
Objective value $\geq k$ (maximization) $\leq k$ (minimization)
|
||||
|
||||
$DL$ is the reduced Canonical Decision problem $L$
|
||||
|
||||
##### Hardness of Canonical Decision Problems
|
||||
|
||||
Lemma 1: $DL\leq p(L)$ ($DL$ is no harder than $L$)
|
||||
|
||||
Proof: Assume $L$ **maximization** problem $DL(l)$: does have a solution $\geq k$.
|
||||
|
||||
Example: Does graph $G$ have flow $\geq k$.
|
||||
|
||||
Let $v^∗$ be the maximum objective on $l$ by solving $l$.
|
||||
|
||||
Let the instance of $DL:(l,k)$ and $l$ be the problem and $k$ be the objective
|
||||
|
||||
1. $l\to \phi(l)\in L$ (optimization problem) $\phi(l,k)=l$
|
||||
2. Is $v^*(l)\geq k$? If so, return true, else return false.
|
||||
|
||||
Lemma 2: If $v^* =O(c^{|l|})$ for any constant $c$, then $L\leq p(DL)$.
|
||||
|
||||
Proof: First we could show $L\leq DL$. Suppose maximization problem, canonical decision problem is is there a solution $\geq k$.
|
||||
|
||||
Naïve Linear Search: Ask $DL(l,k)$, if returns false, ask $DL(l,k+1)$ until returns true
|
||||
|
||||
Runtime: At most $k$ search to iterate all possibilities.
|
||||
|
||||
This is exponential! How to reduce it?
|
||||
|
||||
Our old friend Binary (exponential) Search is back!
|
||||
|
||||
You gets a no at some value: try power of 2 until you get a no, then do binary search
|
||||
|
||||
\# questions: $=log_2(v^*(l))=poly(l)$
|
||||
|
||||
Binary search in area: from last yes to first no.
|
||||
|
||||
Runtime: Binary search ($O(n)=\log(v^*(l))$)
|
||||
|
||||
### Reduction for Algorithm Design vs Hardness
|
||||
|
||||
For problems $L,K$
|
||||
|
||||
If $K$ is “easy” (exists a poly-time solution), then $L$ is also easy.
|
||||
|
||||
If $L$ is “hard” (no poly-time solution), then $k$ is also hard.
|
||||
|
||||
Every problem that we worked on so far, $K$ is “easy”, so we reduce from new problem to known problem (e.g., max-flow).
|
||||
|
||||
#### Reduction for Hardness: Independent Set (ISET)
|
||||
|
||||
Input: Given an undirected graph $G = (V,E)$,
|
||||
|
||||
A subset of vertices $S\subset V$ is called an **independent set** if no two vertices of are connected by an edge.
|
||||
|
||||
Problem: Does $G$ contain an independent set of size $\geq k$?
|
||||
|
||||
$ISET(G,k)$ returns true if $G$ contains an independent set of size $\geq k$, and false otherwise.
|
||||
|
||||
Algorithm? NO! We think that this is a hard problem.
|
||||
|
||||
A lot of pQEDle have tried and could not find a poly-time solution
|
||||
|
||||
### Example: Vertex Cover (VC)
|
||||
|
||||
Input: Given an undirected graph $G = (V,E)$
|
||||
|
||||
A subset of vertices $C\subset V$ is called a **vertex cover** if contains at least one end point of every edge.
|
||||
|
||||
Formally, for all edges $(u,v)\in E$, either $u\in C$, or $v\in C$.
|
||||
|
||||
Problem: $VC(G,j)$ returns true if has a vertex cover of size $\leq j$, and false otherwise (minimization problem)
|
||||
|
||||
Example:
|
||||
|
||||
#### How hard is Vertex Cover?
|
||||
|
||||
Claim: $ISET\leq p(VC)$
|
||||
Side Note: when we prove $VC$ is hard, we prove it is no easier than $ISET$.
|
||||
|
||||
DO NOT: $VC\leq p(ISET)$
|
||||
|
||||
Proof: Show that $G=(V,E)$ has an independent set of $k$ **if and only if** the same graph (not always!) has a vertex cover of size $|V|-k$.
|
||||
|
||||
Map:
|
||||
|
||||
$$
|
||||
ISET(G,k)\to VC(g,|v|-k)
|
||||
$$
|
||||
|
||||
$G'=G$
|
||||
|
||||
##### Proof of reduction: Direction 1
|
||||
|
||||
Claim 1: $ISET$ of size $k\to$ $VC$ of size $|V|-k$
|
||||
|
||||
Proof: Assume $G$ has an $ISET$ of size $k:S$, consider $C = V-S,|C|=|V|-k$
|
||||
|
||||
Claim: $C$ is a vertex cover
|
||||
|
||||
##### Proof of reduction: Direction 2
|
||||
|
||||
|
||||
Claim 2: $VC$ of size $|V|-k\to ISET$ of size $k$
|
||||
|
||||
Proof: Assume $G$ has an $VC$ of size $|V| −k:C$, consider $S = V − C, |S| =k$
|
||||
|
||||
Claim: $S$ is an independent set
|
||||
|
||||
### What does poly-time mean?
|
||||
|
||||
Algorithm runs in time polynomial to input size.
|
||||
|
||||
- If the input has items, algorithm runs in $\Theta(n^c)$ for any constant is poly-time.
|
||||
- Examples: intervals to schedule, number of integers to sort, # vertices + # edges in a graph
|
||||
- Numerical Value (Integer $n$), what is the input size?
|
||||
- Examples: weights, capacity, total time, flow constraints
|
||||
- It is not straightforward!
|
||||
|
||||
### Real time complexity of F-F?
|
||||
|
||||
In class: $O(F( |V| + |E|))$
|
||||
|
||||
- $|V| + |E|$ = this much space to represent the graph
|
||||
- $F$ : size of the maximum flow.
|
||||
|
||||
If every edge has capacity , then $F = O(CE)$
|
||||
Running time:$O(C|E|(|V| + |E| )))$
|
||||
|
||||
### What is the actual input size?
|
||||
|
||||
Each edge ($|E|$ edges):
|
||||
|
||||
- 2 vertices: $|V|$ distinct symbol, $\log |V|$ bits per symbol
|
||||
- 1 capacity: $\log C$
|
||||
|
||||
Size of graph:
|
||||
|
||||
- $O(|E|(|V| + \log C))$
|
||||
- $p( |E| , |V| , \log C)$
|
||||
|
||||
Running time:
|
||||
|
||||
- $P( |E| , |V| , |C| )$
|
||||
- Exponential if is exponential in $|V|+|E|$
|
||||
|
||||
### Pseudo-polynomial
|
||||
|
||||
Naïve Ford-Fulkerson is bad!
|
||||
|
||||
Problem ’s inputs contain some numerical values, say $|W|$. We need only log bits to store . If algorithms runs in $p(W)$, then it is exponential, or **pseudopolynomial**.
|
||||
|
||||
In homework, you improved F-F to make it work in
|
||||
$p( |V| ,|E| , \log C)$, to make it a real polynomial algorithm.
|
||||
|
||||
## Conclusion: Reductions
|
||||
|
||||
- Reduction
|
||||
- Construction of mapping with runtime
|
||||
- Bidirectional proof
|
||||
- Efficient Reduction $L\leq p(K)$
|
||||
- Which problem is harder?
|
||||
- If $L$ is hard, then $K$ is hard. $\to$ Used to show hardness
|
||||
- If $K$ is easy, then $L$ is easy. $\to$ Used for design algorithms
|
||||
- Canonical Decision Problem
|
||||
- Reduction to and from the optimization problem
|
||||
- Reduction for hardness
|
||||
- Independent Set$leq p$ Vertex Cover
|
||||
|
||||
## On class
|
||||
|
||||
Reduction: $V^* = O(c^k)$
|
||||
|
||||
OPT: Find max flow of at least one instance $(G,s,t)$
|
||||
|
||||
DEC: Is there a flow of size $pK$, given $G,s,t \implies$ the instance is defined by the tuple $(G,s,t,k)$
|
||||
|
||||
Yes, if there exists one
|
||||
No, otherwise
|
||||
|
||||
Forget about F-F and assume that you have an oracle that solves the decision problem.
|
||||
|
||||
First solution (the naive solution): iterate over $k = 1, 2, \dots$ until the oracle returns false and the last one returns true would be the max flow.
|
||||
|
||||
Time complexity: $K\cdot X$, where $X$ is the time complexity of the oracle
|
||||
Input size: $poly(||V|,|E|, |E|log(max-capacity))$, and $V^* \leq \sum$ capacities
|
||||
|
||||
A better solution: do a binary search. If there is no upper bound, we use exponential binary search instead. Then,
|
||||
|
||||
$$
|
||||
\begin{aligned}
|
||||
log(V^*) &\leq X\cdot log(\sum capacities)\\
|
||||
&\leq X\cdot log(|E|\cdot maxCapacity)\\
|
||||
&\leq X\cdot (log(|E| + log(maxCapacity)))
|
||||
\end{aligned}
|
||||
$$
|
||||
As $\log(maxCapacity)$ is linear in the size of the input, the running time is polynomial to the solution of the original problem.
|
||||
|
||||
Assume that ISET is a hard problem, i.e. we don't know of any polynomial time solution. We want to show that vertex cover is also a hard problem here:
|
||||
|
||||
$ISET \leq_{p} VC$
|
||||
|
||||
1. Given an instance of ISET, construct an instance of VC
|
||||
2. Show that the construction can be done in polynomial time
|
||||
3. Show that if the ISET instance is true than the CV instance is true
|
||||
4. Show that if the VC instance is true then the ISET instance is true.
|
||||
|
||||
> ISET: given $(G,K)$, is there a set of vertices that do not share edges of size $K$
|
||||
> VC: given $(G,K)$, is there a set of vertices that cover all edges of size $K$
|
||||
|
||||
1. Given $l: (G,K)$ being an instance of ISET, we construct $\phi(l): (G',K')$ as an instance of VC. $\phi(l): (G, |V|-K), \textup{i.e., } G' = G \cup K' = |V| - K$
|
||||
2. It is obvious that it is a polynomial time construction since copying the graph is linear, in the size of the graph and the subtraction of integers is constant time.
|
||||
|
||||
**Direction 1**: ISET of size k $\implies$ VC of size $|V| - K$ Assume that ISET(G,K) returns true, show that $VC(G, |V|-K)$ returns true
|
||||
|
||||
Let $S$ be an independent set of size $K$ and $C = V-S$
|
||||
|
||||
We claim that $C$ is a vertex cover of size $|V|-K$
|
||||
|
||||
Proof:
|
||||
|
||||
We proceed by contradiction. Assume that $C$ is NOT a vertex cover, and it means that there is an edge $(u,v)$ such that $u\notin c , v\notin C$. And it implies that $u\in S , v\in S$, which contradicts with the assumption that S is an independent set.
|
||||
Therefore, $c$ is an vertex cover
|
||||
|
||||
**Direction 2**: VC of size $|V|-K \implies$ ISET of size $K$
|
||||
|
||||
Let $C$ be a vertex cover of size $|V|-K$ , let $s = |v| - c$
|
||||
|
||||
We claim that $S$ is an independent set of size $K$.
|
||||
|
||||
Again, assume, for the sake of contradiction, that $S$ is not an independent set. And we get
|
||||
|
||||
$\exists (u,v) \textup{such that } u\in S, v \in S$
|
||||
|
||||
$u,v \notin C$
|
||||
|
||||
$C \textup{ is not a vertex cover}$
|
||||
|
||||
And this is a contradiction with our assumption.
|
||||
Reference in New Issue
Block a user