reduction

Recall definition: $A$ is mapping reducible to $B$ means there is a computable function $f : \Sigma^* \to \Sigma^*$ such that for all strings $x$ in $\Sigma^*$, \[x \in A \qquad \qquad \text{if and only if} \qquad \qquad f(x) \in B.\] Notation: when $A$ is mapping reducible to $B$, we write $A \leq_m B$.

Theorem (Sipser 5.23): If $A \leq_m B$ and $A$ is undecidable, then $B$ is undecidable.

Halting problem \[HALT_{TM} = \{ \langle M, w \rangle \mid \text{$M$ is a Turing machine, $w$ is a string, and $M$ halts on $w$} \}\]

We will define a computable function that witnesses the mapping reduction $A_{TM} \leq_m HALT_{TM}$.

Define $F: \Sigma^* \to \Sigma^*$ by \[F(x) = \begin{cases} const_{out} \qquad &\text{if $x \neq \langle M,w \rangle$ for any Turing machine $M$ and string $w$ over the alphabet of $M$} \\ \langle M', w \rangle \qquad & \text{if $x = \langle M, w \rangle$ for some Turing machine $M$ and string $w$ over the alphabet of $M$.} \end{cases}\] where $const_{out} = \langle \includegraphics[width=1.5in]{../../resources/machines/Lect22TM1.png} , \varepsilon \rangle$ and $M'$ is a Turing machine that computes like $M$ except, if the computation ever were to go to a reject state, $M'$ loops instead.

$F( \langle \includegraphics[width=1.5in]{../../resources/machines/Lect22TM1.png} , 001 \rangle)$ =

$F( \langle \includegraphics[width=2.5in]{../../resources/machines/Lect22TM2.png} , 1 \rangle)$ =

To use this function to prove that $A_{TM} \leq_m HALT_{TM}$, we need two claims:

Week9 wednesday

Recall: $A$ is mapping reducible to $B$, written $A \leq_m B$, means there is a computable function $f : \Sigma^* \to \Sigma^*$ such that for all strings $x$ in $\Sigma^*$, \[x \in A \qquad \qquad \text{if and only if} \qquad \qquad f(x) \in B.\]

Theorem (Sipser 5.28): If $A \leq_m B$ and $B$ is recognizable, then $A$ is recognizable.

Corollary: If $A \leq_m B$ and $A$ is unrecognizable, then $B$ is unrecognizable.

(i) To prove that a recognizable language $R$ is undecidable, prove that $A_{TM} \leq_m R$.

(ii) To prove that a co-recognizable language $U$ is undecidable, prove that $\overline{A_{TM}} \leq_m U$, i.e. that $A_{TM} \leq_m \overline{U}$.

\[E_{TM} = \{ \langle M \rangle \mid \text{$M$ is a Turing machine and $L(M) = \emptyset$} \}\]

$\overline{E_{TM}}$ is decidable / undecidable and recognizable / unrecognizable .

Proof: Need computable function $F: \Sigma^* \to \Sigma^*$ such that $x \in A_{TM}$ iff $F(x) \notin E_{TM}$. Define

Input string	Output string
$\langle M, w \rangle$ where $w \in L(M)$

$\langle M, w \rangle$ where $w \notin L(M)$

$x$ not encoding any pair of TM and string

\[EQ_{TM} = \{ \langle M, M' \rangle \mid \text{$M$ and $M'$ are both Turing machines and $L(M) =L(M')$} \}\]

$\overline{EQ_{TM}}$ is decidable / undecidable and recognizable / unrecognizable .

To prove, show that $\underline{\phantom{\hspace{1.6in}}} \leq_m EQ_{TM}$ and that $\underline{\phantom{\hspace{1.6in}}} \leq_m \overline{EQ_{TM}}$.

Input string	Output string
$\langle M, w \rangle$ where $M$ halts on $w$

$\langle M, w \rangle$ where $M$ loops on $w$

$x$ not encoding any pair of TM and string

Week9 friday

In practice, computers (and Turing machines) don’t have infinite tape, and we can’t afford to wait unboundedly long for an answer. “Decidable" isn’t good enough - we want “Efficiently decidable".

For a given algorithm working on a given input, how long do we need to wait for an answer? How does the running time depend on the input in the worst-case? average-case? We expect to have to spend more time on computations with larger inputs.

Definition (Sipser 7.1): For $M$ a deterministic decider, its running time is the function $f: \mathbb{N} \to \mathbb{N}$ given by \[f(n) = \text{max number of steps $M$ takes before halting, over all inputs of length $n$}\]

Definition (Sipser 7.7): For each function $t(n)$, the time complexity class $TIME(t(n))$, is defined by \[TIME( t(n)) = \{ L \mid \text{$L$ is decidable by a Turing machine with running time in $O(t(n))$} \}\]

Definition (Sipser 7.12) : $P$ is the class of languages that are decidable in polynomial time on a deterministic 1-tape Turing machine \[P = \bigcup_k TIME(n^k)\]

Theorem (Sipser 7.8): Let $t(n)$ be a function with $t(n) \geq n$. Then every $t(n)$ time deterministic multitape Turing machine has an equivalent $O(t^2(n))$ time deterministic 1-tape Turing machine.

Definition (Sipser 7.9): For $N$ a nodeterministic decider. The running time of $N$ is the function $f: \mathbb{N} \to \mathbb{N}$ given by \[f(n) = \text{max number of steps $N$ takes on any branch before halting, over all inputs of length $n$}\]

Definition (Sipser 7.21): For each function $t(n)$, the nondeterministic time complexity class $NTIME(t(n))$, is defined by \[NTIME( t(n)) = \{ L \mid \text{$L$ is decidable by a nondeterministic Turing machine with running time in $O(t(n))$} \}\] \[NP = \bigcup_k NTIME(n^k)\]

Can’t use nondeterminism; Can use multiple tapes; Often need to be “more clever” than naïve / brute force approach \[PATH = \{\langle G,s,t\rangle \mid \textrm{$G$ is digraph with $n$ nodes there is path from s to t}\}\] Use breadth first search to show in $P$ \[RELPRIME = \{ \langle x,y\rangle \mid \textrm{$x$ and $y$ are relatively prime integers}\}\] Use Euclidean Algorithm to show in $P$ \[L(G) = \{w \mid \textrm{$w$ is generated by $G$}\}\] (where $G$ is a context-free grammar). Use dynamic programming to show in $P$.

“Verifiable" i.e. NP, Can be decided by a nondeterministic TM in polynomial time, best known deterministic solution may be brute-force, solution can be verified by a deterministic TM in polynomial time.

\[HAMPATH = \{\langle G,s,t \rangle \mid \textrm{$G$ is digraph with $n$ nodes, there is path from $s$ to $t$ that goes through every node exactly once}\}\] \[VERTEX-COVER = \{ \langle G,k\rangle \mid \textrm{$G$ is an undirected graph with $n$ nodes that has a $k$-node vertex cover}\}\] \[CLIQUE = \{ \langle G,k\rangle \mid \textrm{$G$ is an undirected graph with $n$ nodes that has a $k$-clique}\}\] \[SAT =\{ \langle X \rangle \mid \textrm{$X$ is a satisfiable Boolean formula with $n$ variables}\}\]

Week8 monday

Proof: Suppose towards a contradiction that there is a Turing machine that decides $A_{TM}$. We call this presumed machine $M_{ATM}$.

Theorem (Sipser Theorem 4.22): A language is Turing-decidable if and only if both it and its complement are Turing-recognizable.

Proof, first direction: Suppose language $L$ is Turing-decidable. WTS that both it and its complement are Turing-recognizable.

Proof, second direction: Suppose language $L$ is Turing-recognizable, and so is its complement. WTS that $L$ is Turing-decidable.

True or False: The class of Turing-decidable languages is closed under complementation?

Definition: A language $L$ over an alphabet $\Sigma$ is called co-recognizable if its complement, defined as $\Sigma^* \setminus L = \{ x \in \Sigma^* \mid x \notin L \}$, is Turing-recognizable.

Notation: The complement of a set $X$ is denoted with a superscript $c$, $X^c$, or an overline, $\overline{X}$.

Week8 wednesday

Motivation: Proving that $A_{TM}$ is undecidable was hard. How can we leverage that work? Can we relate the decidability / undecidability of one problem to another?

“Problem $X$ is no harder than problem $Y$” means “Can answer questions about membership in $X$ by converting them to questions about membership in $Y$”.

Definition: $A$ is mapping reducible to $B$ means there is a computable function $f : \Sigma^* \to \Sigma^*$ such that for all strings $x$ in $\Sigma^*$, \[x \in A \qquad \qquad \text{if and only if} \qquad \qquad f(x) \in B.\] Notation: when $A$ is mapping reducible to $B$, we write $A \leq_m B$.

Intuition: $A \leq_m B$ means $A$ is no harder than $B$, i.e. that the level of difficulty of $A$ is less than or equal the level of difficulty of $B$.

Definition: A function $f: \Sigma^* \to \Sigma^*$ is a computable function means there is some Turing machine such that, for each $x$, on input $x$ the Turing machine halts with exactly $f(x)$ followed by all blanks on the tape

The function that maps a string to a string which is one character longer and whose value, when interpreted as a fixed-width binary representation of a nonnegative integer is twice the value of the input string (when interpreted as a fixed-width binary representation of a non-negative integer) \[f_1: \Sigma^* \to \Sigma^* \qquad f_1(x) = x0\]

To prove $f_1$ is computable function, we define a Turing machine computing it.

Formal definition $(\{q0, qacc, qrej\}, \{0,1\}, \{0,1,\textvisiblespace\},\delta, q0, qacc, qrej)$ where $\delta$ is specified by the state diagram:

The function that maps a string to the result of repeating the string twice. \[f_2: \Sigma^* \to \Sigma^* \qquad f_2( x ) = xx\]

The function that maps strings that are not the codes of Turing machines to the empty string and that maps strings that code Turing machines to the code of the related Turing machine that acts like the Turing machine coded by the input, except that if this Turing machine coded by the input tries to reject, the new machine will go into a loop. \[f_3: \Sigma^* \to \Sigma^* \qquad f_3( x ) = \begin{cases} \varepsilon \qquad&\text{if $x$ is not the code of a TM} \\ \langle (Q \cup \{q_{trap} \}, \Sigma, \Gamma, \delta', q_0, q_{acc}, q_{rej} ) \rangle \qquad&\text{if $x = \langle (Q, \Sigma, \Gamma, \delta, q_0, q_{acc}, q_{rej} )\rangle$}\end{cases}\] where $q_{trap} \notin Q$ and \[\delta'( (q,x) ) = \begin{cases} (r,y,d) &\text{if $q \in Q$, $x \in \Gamma$, $\delta ((q,x)) = (r,y,d)$, and $r \neq q_{rej}$} \\ (q_{trap}, \textvisiblespace, R) & \text{otherwise} \end{cases}\]

The function that maps strings that are not the codes of CFGs to the empty string and that maps strings that code CFGs to the code of a PDA that recognizes the language generated by the CFG.

Week8 friday

Intuition: $A \leq_m B$ means $A$ is no harder than $B$, i.e. that the level of difficulty of $A$ is less than or equal the level of difficulty of $B$.

Theorem (Sipser 5.22): If $A \leq_m B$ and $B$ is decidable, then $A$ is decidable.

Theorem (Sipser 5.23): If $A \leq_m B$ and $A$ is undecidable, then $B$ is undecidable.

Halting problem \[HALT_{TM} = \{ \langle M, w \rangle \mid \text{$M$ is a Turing machine, $w$ is a string, and $M$ halts on $w$} \}\]

$F( \langle \includegraphics[width=1.5in]{../../resources/machines/Lect22TM1.png} , 001 \rangle)$ =

$F( \langle \includegraphics[width=2.5in]{../../resources/machines/Lect22TM2.png} , 1 \rangle)$ =

To use this function to prove that $A_{TM} \leq_m HALT_{TM}$, we need two claims:

Week10 wednesday

Recall: For $M$ a deterministic decider, its running time is the function $f: \mathbb{N} \to \mathbb{N}$ given by \[f(n) = \text{max number of steps $M$ takes before halting, over all inputs of length $n$}\] For each function $t(n)$, the time complexity class $TIME(t(n))$, is defined by \[TIME( t(n)) = \{ L \mid \text{$L$ is decidable by a Turing machine with running time in $O(t(n))$} \}\] $P$ is the class of languages that are decidable in polynomial time on a deterministic 1-tape Turing machine \[P = \bigcup_k TIME(n^k)\]

Brute-force (worst-case exponential time) approach: iterate over all possible solutions, for each one, check if it works.

\[\begin{aligned} HAMPATH &= \{\langle G,s,t \rangle \mid \textrm{$G$ is digraph with $n$ nodes, }\\ & \qquad \textrm{there is path from $s$ to $t$ that goes through every node exactly once}\} \end{aligned}\] \[VERTEX-COVER = \{ \langle G,k\rangle \mid \textrm{$G$ is an undirected graph with $n$ nodes that has a $k$-node vertex cover}\}\] \[CLIQUE = \{ \langle G,k\rangle \mid \textrm{$G$ is an undirected graph with $n$ nodes that has a $k$-clique}\}\] \[SAT =\{ \langle X \rangle \mid \textrm{$X$ is a satisfiable Boolean formula with $n$ variables}\}\]

Problems in $P$	Problems in $NP$
(Membership in any) regular language	Any problem in $P$
(Membership in any) context-free language
$A_{DFA}$	$SAT$
$E_{DFA}$	$CLIQUE$
$EQ_{DFA}$	$VERTEX-COVER$
$PATH$	$HAMPATH$
$RELPRIME$	$\ldots$
$\ldots$

One approach to trying to answer it is to look for hardest problems in $NP$ and then (1) if we can show that there are efficient algorithms for them, then we can get efficient algorithms for all problems in $NP$ so $P = NP$, or (2) these problems might be good candidates for showing that there are problems in $NP$ for which there are no efficient algorithms.

Definition (Sipser 7.29) Language $A$ is polynomial-time mapping reducible to language $B$, written $A \leq_P B$, means there is a polynomial-time computable function $f: \Sigma^* \to \Sigma^*$ such that for every $x \in \Sigma^*$ \[x \in A \qquad \text{iff} \qquad f(x) \in B.\] The function $f$ is called the polynomial time reduction of $A$ to $B$.

Definition (Sipser 7.34; based in Stephen Cook and Leonid Levin’s work in the 1970s): A language $B$ is NP-complete means (1) $B$ is in NP and (2) every language $A$ in $NP$ is polynomial time reducible to $B$.

Theorem (Sipser 7.35): If $B$ is NP-complete and $B \in P$ then $P = NP$.

3SAT: A literal is a Boolean variable (e.g. $x$) or a negated Boolean variable (e.g. $\bar{x}$). A Boolean formula is a 3cnf-formula if it is a Boolean formula in conjunctive normal form (a conjunction of disjunctive clauses of literals) and each clause has three literals. \[3SAT = \{ \langle \phi \rangle \mid \text{$\phi$ is a satisfiable 3cnf-formula} \}\]

Are there other $NP$-complete problems? To prove that $X$ is $NP$-complete

CLIQUE: A $k$-clique in an undirected graph is a maximally connected subgraph with $k$ nodes. \[CLIQUE = \{ \langle G, k \rangle \mid \text{$G$ is an undirected graph with a $k$-clique} \}\]

Given a Boolean formula in conjunctive normal form with $k$ clauses and three literals per clause, we will map it to a graph so that the graph has a clique if the original formula is satisfiable and the graph does not have a clique if the original formula is not satisfiable.

The graph has $3k$ vertices (one for each literal in each clause) and an edge between all vertices except

Example: $(x \vee \bar{y} \vee {\bar z}) \wedge (\bar{x} \vee y \vee z) \wedge (x \vee y \vee z)$

Week10 friday


Model of Computation	Class of Languages


Deterministic finite automata: formal definition, how to design for a given language, how to describe language of a machine? Nondeterministic finite automata: formal definition, how to design for a given language, how to describe language of a machine? Regular expressions: formal definition, how to design for a given language, how to describe language of expression? Also: converting between different models.	Class of regular languages: what are the closure properties of this class? which languages are not in the class? using pumping lemma to prove nonregularity.


Push-down automata: formal definition, how to design for a given language, how to describe language of a machine? Context-free grammars: formal definition, how to design for a given language, how to describe language of a grammar?	Class of context-free languages: what are the closure properties of this class? which languages are not in the class?


Turing machines that always halt in polynomial time	$P$

Nondeterministic Turing machines that always halt in polynomial time	$NP$


Deciders (Turing machines that always halt): formal definition, how to design for a given language, how to describe language of a machine?	Class of decidable languages: what are the closure properties of this class? which languages are not in the class? using diagonalization and mapping reduction to show undecidability


Turing machines formal definition, how to design for a given language, how to describe language of a machine?	Class of recognizable languages: what are the closure properties of this class? which languages are not in the class? using closure and mapping reduction to show unrecognizability

Strategy 3: construct regular expression recognizing the language and prove it works.

Example: $L = \{ w \in \{0,1\}^* \mid \textrm{$w$ has odd number of $1$s or starts with $0$}\}$

Example: Select all and only the options that result in a true statement: “To show a language $A$ is not regular, we can…”

Example: What is the language generated by the CFG with rules \[\begin{aligned} S &\to aSb \mid bY \mid Ya \\ Y &\to bY \mid Ya \mid \varepsilon \end{aligned}\]

Example: Prove that the language $T = \{ \langle M \rangle \mid \textrm{$M$ is a Turing machine and $L(M)$ is infinite}\}$ is undecidable.

Example: Prove that the class of decidable languages is closed under concatenation.

Input string	Output string
\(\langle M, w \rangle\) where \(w \in L(M)\)

\(\langle M, w \rangle\) where \(w \notin L(M)\)

\(x\) not encoding any pair of TM and string

Input string	Output string
\(\langle M, w \rangle\) where \(M\) halts on \(w\)

\(\langle M, w \rangle\) where \(M\) loops on \(w\)

\(x\) not encoding any pair of TM and string

Problems in \(P\)	Problems in \(NP\)
(Membership in any) regular language	Any problem in \(P\)
(Membership in any) context-free language
\(A_{DFA}\)	\(SAT\)
\(E_{DFA}\)	\(CLIQUE\)
\(EQ_{DFA}\)	\(VERTEX-COVER\)
\(PATH\)	\(HAMPATH\)
\(RELPRIME\)	\(\ldots\)
\(\ldots\)