Notes for CSC 173, Tues 4 Dec. - Tues 11 Dec. 2001 Read chapter 14 in AU. Course evaluations will occur on Thurs. 6 Dec. PLEASE BE HERE. ======================================================================== Propositional Logic has several limitations. One key limitation is that it applies only to atomic propositions. There is no way to talk about properties that apply to categories of objects, or about relationships between those properties. That's what predicate logic is for. Predicate logic is a mathematical model for reasoning with predicates: functions that map variables to truth values. As in propositional logic, we can create logical expressions containing predicates, manipulate those expressions according to the algebraic laws of predicate logic, and construct proofs using rules of inference to deduce new facts from axioms. In fact, proofs in predicate logic (based on unification) are a form of computation used in the programming language Prolog. Although predicate logic is more powerful than propositional logic, it too has its limits. ------------------------------------------------------------------------ Predicates A predicate is a boolean function whose value may be true or false, depending on the arguments to the predicate. * Predicates are a generalization of propositional variables. * A propositional variable is a predicate with no arguments. Example - Consider the following boolean propositions: A (Adam is tall) B (Beth is tall) C (Carl is tall) : Z (Zeke is tall) We need a different proposition for each person; each of these propositions is either true or false. We can capture the same set of truth values using a single predicate (or boolean function), Tall(x). Tall(x) is true whenever person x is tall, and is false otherwise. * Tall(Adam) is true if proposition A above is true. * Tall(Beth) is true if proposition B above is true. * Tall(Carl) is true if proposition C above is true. Predicates are atomic operands in the logical expressions of predicate logic. An argument can be either a constant or a variable. It is conventional to write variables with an initial upper-case letter. Constants can be numbers, names that begin with a lower-case letter, or quoted strings. A relation (remember the first unit this semester?) is one way to represent a predicate. Use the relation name P as the predicate name; If P has three columns, then P(a,b,c) is true if there is a tuple with fields a, b, and c. ------------------------------------------------------------------------ Quantifiers All we want to know about a given proposition A (Adam is tall) is whether A is true or false. Given a predicate such as Tall(x), we want to know whether Tall(x) is true for different values of x. In addition, we might like to know whether Tall(x) is true for every possible value of x, or whether Tall(x) is true for some value of x. Predicate logic has two additional operators not found in propositional logic (called quantifiers) to express truth values about predicates with variable arguments. * Existential quantifier E (there exists): (E x) Tall(x) is true if there exists some value for x such that Tall(x) is true. * Universal quantifier A (for all): (A x) Tall(x) is true if Tall(x) is true for all values of x. [NB: the existential quantifier is written with a backward E. The universal quantifier is written with an upside-down A. I can't type those in ascii. Please DO NOT feed me ordinary E's and A's on the final: write them the way they're supposed to be written.] ======================================================================== Logical Expressions in Predicate Logic A logical expression in predicate logic has much the same form as a logical expression in propositional logic, with the addition of atomic formulae (ie., predicates), and the universal and existential quantifiers. 1. An atomic formula is a logical expression. o A predicate with all constant arguments is a ground atomic formula. o A proposition is a predicate with no arguments, and therefore is a ground atomic formula. o A predicate with at least one variable argument is a nonground atomic formula. o A literal is either an atomic formula or its negation. 2. If L1 and L2 are logical expressions, then L1 AND L2, L1 OR L2, NOT L1, L1 -> L2, and L1 == L2 are logical expressions. 3. If L1 is a logical expressions, then (A X) L1 is a logical expression. 4. If L1 is a logical expressions, then (E X) L1 is a logical expression. Quantifiers have the highest precedence in logical expressions. ------------------------------------------------------------------------ Bound and Free Variables The quantifiers E (there exists) and A (forall) introduce variables into logical expressions. An occurrence of variable x in a logical expression is bound to the closest enclosing quantifier containing x, either (E x) or (A x). If an occurrence of x in a logical expression is not bound to any quantifier (because there is no enclosing quantifier containing x) that occurrence of x is free. Example: (A x) L1(x) OR (E x) L2(x,y) The variable x in L1 is bound to the universal quantifier. The variable x in L2 is bound to the existential quantifier. The variable y is free in this expression. In order for an expression to be *closed*, all of its variables must be bound. Cf. scope in programming languages. A common convention is to say that all variables that are unbound at the outermost nesting level are implicitly universally quantified. ------------------------------------------------------------------------ Evaluating Predicates There are two ways to evaluate the truth of a predicate P(x,y): * Assign a real-world interpretation to P (such as addition, subtraction, or equivalence) and a domain for P (ie, the possible values for the arguments) and compute the function P(x,y) under that interpretation. o If we assign the interpretation "equivalence" to P, and let the domain for P be the set of integers, then we can evaluate P(1,2) by asking is 1 equivalent to 2 (false). o If we assign the interpretation "less than" to P, and let the domain for P be the set of integers, then we can evaluate P(1,2) by asking is 1 less than 2 (true). o If we assign the interpretation "brother" to P, and let the domain for P be all students on campus, then we can evaluate P(Adam,Barney) by asking whether or not Adam and Barney are brothers. * Consult a relational database containing pairs of values for x and y and the corresponding value of P(x,y). o If the interpretation for P is "grade in CSC173" and the domain of x is all students on campus, and the domain of y is the set of possible grades, then we can evaluate P(Rosemary,"A") by looking up Rosemary's grade in the grade file for 173. o If we assign the interpretation "has a better overall record" to P, and let the domain of P be the set of professional football teams, then we can evaluate P(Buffalo, Miami) by looking up their respective records in a football database. Note that the relation approach is limited to finite domains, or at least finite sets of arguments for which the predicate is true. The real world interpretation approach can have an unbounded number of sets of arguments for which the predicate is true. This tradeoff has implications for systems such as Prolog. Numbers are special-cased in Prolog, but for everything else the system suffers from what is called the "closed world" problem. ------------------------------------------------------------------------ Tautologies A tautology in propositional logic is a statement that is true regardless of the truth assignment of propositions. A tautology in predicate logic is a statement that is true regardless of the interpretation of predicates, and regardless of the bindings chosen for any globally unbound variables. We can combine these statements to say, in general, a logical tautology is a statement that is true regardless of *model*. In propositional logic a model is a truth assignment for propositions. In predicate logic a model is an interpretation for predicates and a binding for any globally unbound variables. ------------------------------------------------------------------------ Evaluating Predicates: Example Consider the following family history to be relations (or facts) in a family tree database: male(Adam) female(Ann) male(Barney) female(Beth) male(Bob) female(Barb) male(Carl) female(Carol) male(Chet) female(Chris) parent(Adam,Barney) parent(Ann,Barney) parent(Adam,Beth) parent(Ann,Beth) parent(Adam,Bob) parent(Ann,Bob) parent(Adam,Barb) parent(Barney,Carl) parent(Carol,Carl) parent(Carol,Chet) To find the name of Barney's father, we can assert the following to be true: parent(x,Barney) AND male(x) -> father(x,Barney) This assertion states that if x is a parent of Barney, and x is male, then x is the father of Barney. Note that we haven't defined a father relation in our database; asserting the above expression to be true is the only definition of "father" we need. We know this expression evaluates to true (since we asserted it) regardless of the value of x. We can assign a constant value to x (someone's name), producing ground atomic formulae, which we can evaluate as true or false. When we substitute Adam for x, we find parent(Adam,Barney) and male(Adam) are true, so father(Adam,Barney) must be true as well (since the whole expression must be true). ------------------------------------------------------------------------ Evaluating Quantifiers To evaluate a quantifier for a predicate we must first define * the domain over which the quantifier varies (that is, the set of values for the predicate's arguments) * the interpretation of the predicate (that is, the meaning of the predicate) When we state "there exists x such that P(x)" we must be explicit about the possible values x can take (the domain of P). We evaluate P(x) for each value of x in the domain of P (according to the interpretation for P), and if P(x) is true for some x, then the expression (E x)P(x) is true. Similarly, when we assert "for all x P(x)" we must be clear about what possible values of x we consider (again, the domain of P). We evaluate P(x) for all values of x in the domain of P (again, according to the meaning of P), and if P(x) is true for all x, then the expression (A x)P(x) is true. Note that if the domain of P is infinite, we don't have an algorithm (which terminates) to compute the value of P. * In many cases, we'll have a finite domain, so we do have an algorithm. * In many cases we're concerned about whether two expressions are equivalent, and not whether a particular expression is true or not. ------------------------------------------------------------------------ Evaluating Predicate Expressions: Example Consider the following logical expression in predicate logic: P(x,y) -> (E z)(P(x,z) AND P(z,y)) We can read this as "if P(x,y) then there exists z such that P(x,z) and P(z,y)". (Assume that x and y are universally quantified.) If we make the domain of P the set of real numbers, assign the values 5.1 and 4.2 to the free variables x and y, and interpret P to mean "greater than", then we can evaluate the expression as follows: P(x,y) = P(5.1,4.2) = 5.1 > 4.2 = true (E z) (P(x,z) AND P(z,y)) = (E z)((5.1 >z) AND (z > 4.2)) = true (for z=4.8) true -> true = true If we make the domain of P the set of integers, assign the values 5 and 4 to the free variables x and y, and interpret P to mean "greater than", then we can evaluate the expression as follows: P(x,y) = P(5,4) = 5 > 4 = true (E z) (P(x,z) AND P(z,y)) = (E z)((5 > z) AND (z > 4)) = false true -> false = false ------------------------------------------------------------------------ Summary of evaluation: (1) The truth of many statements with unbound variables is indeterminate: Some statements (e.g. P(x) OR NOT P(x)) may be true regardless of bindings (and others may be false, regardless), but in general we need bindings. (2) The truth of many statements depends on the interpretation of predicates. Again, some statements may be true or false regardless, but in general we need interpretations. (3) As we will see below, some statements under some interpretations and proof systems are true but can't be proven. ======================================================================== Laws for Manipulating Quantifiers Binding free variables in a tautology If a logical expression L with free variables x1..xn is a tautology, then (A x1)(A x2)..(A xn) L is also a tautology. We can use the above rule to bind all free variables in a tautology to a universal quantifier. P(x,y) OR NOT P(x,y) == 1 == (A x)(A y) (P(x,y) OR NOT P(x,y)) An expression with no free variables is a closed expression. Moving NOT within a quantifier There is rule analogous to DeMorgan's law that allows us to move a NOT operator through an expression containing a quantifier. NOT (A x) L(x) == (E x) (NOT L(x)) NOT (E x) L(x) == (A x) (NOT L(x)) The first rule can be read as "it is not the case that for all x, L(x) is true" is equivalent to "for some x, it is not the case that L(x) is true". The second rule can be read as "it is not the case that for some x L(x) is true" is equivalent to "for all x, it is not the case that L(x) is true". Moving quantifiers through AND and OR L1 AND (A x) L2(x) == (A x) (L1 AND L2(x)) L1 AND (E x) L2(x) == (E x) (L1 AND L2(x)) L1 OR (A x) L2(x) == (A x) (L1 OR L2(x)) L1 OR (E x) L2(x) == (E x) (L1 OR L2(x)) Note that we require x is not a free variable in expression L1 in the above rules. We may have to rename variable x (consistently) in L2 to avoid any overlap with free variables in L1. ------------------------------------------------------------------------ Manipulating Quantifiers: Example Given the following expression: (A x)P(x) OR (E x)(NOT P(x)) we can rename the second occurrence of x to avoid any overlap with the first occurrence and arrive at the equivalent expression (A x)P(x) OR (E y)(NOT P(y)) Moving the universal quantifier through the OR we arrive at (A x) (P(x) OR (E y)(NOT P(y))) Moving the existential quantifier through the OR we arrive at (A x) (E y) (P(x) OR NOT P(y)) Using these rules, we can write any expression involving quantifiers Q1, Q2, .. QN, and the logical operators AND, OR, and NOT in *prenex form*: (Q1 x1)(Q2 x2)..(Qn xn)L where all the quantifiers appear outside expression L. 1. First, rewrite the expression so that all of the quantifiers refer to distinct variables not found in other quantifiers or free in the expression. 2. Use the rules above to move the quantifiers outside NOT, AND, and OR. ======================================================================== Proofs in Predicate Logic A proof in predicate logic has much the same form as a proof in propositional logic. We begin with a set of axioms (or hypotheses) A1..An, and using the rules of inference, we construct a sequence of expressions that follow from those axioms. We can use the rules of inference from propositional logic as inference rules in predicate logic, including modus ponens, DeMorgan's laws, and the substitution of equals. We require that each hypothesis and line in the proof be a closed expression (ie, there are no free variables whose scope extends beyond a line in the proof). ------------------------------------------------------------------------ Implication, entailment, and proof It's important to distinguish among three related concepts: A -> B "A implies B". This is a logical statement. It may be true or false. A =| B "A entails B". This is a "meta-statement" about truth. B is true whenever A is true; more precisely, B is true in all models in which A is true. If we know that A =| B then we know that A -> B is a tautology. A -| B "B can be proven from A". This is a weaker meta-statement. It says that under some given set of proof rules and interpretations for predicates, if we are given A as premise we can derive B. ------------------------------------------------------------------------ Substitution Rule The law of variable substitution is an inference rule for use in proofs in predicate logic. Informally, this rule states that having established that a general fact (or expression) is true, we can assert that a specific instance of that general expression is also true. In particular, if we can prove (or assert as an axiom) a logical expression L1 containing free variables, then if we substitute constants or bound variables for some of the free variables in L1 to create expression L2, then the law of substitution states that L1 -> L2 is a tautology, and we can assert L2 in the proof. Consider the following assertion about the domain of real numbers: Lt(x,y) -> (E z) (Lt(x,z) AND Lt(z,y)) If we substitute x=2 and y=5 in the original expression, then we can assert: Lt(2,5) -> (E z) (Lt(2,z) AND Lt(z,5)) In other words, if the original expression holds for all x and y, then it must hold for x=2 and y=5. Note that choosing x=5 and y=2 makes Lt(x,y) = false, and the entire expression is still true (since false->anything is true). ------------------------------------------------------------------------ Structure of a Proof in Predicate Logic The simplest proofs in predicate logic consist of: * facts, which are ground atomic formulas male(Adam) female(Ann) parent(Adam,Barney) * rules, which are the conjunction of one or more atomic formulae that imply another atomic formula parent(y,x) AND male(x) -> son(x,y) parent(y,x) AND female(x) -> daughter(x,y) The left-hand side of a rule contains hypotheses (called the body of the rule); each atomic formula is a hypothesis or subgoal. The right-hand side is the goal (or head of the rule). Rules are general principles that we can apply to facts to prove new facts. * Assert a rule that is known to be true (that is, the body of the rule implies the head of the rule) * Find facts that (via substitution) match the atomic formulae of the body of the rule * Make consistent variable substitutions in the body and the head of the rule * Assert the head (or goal) as proven ------------------------------------------------------------------------ Example Database of Facts and Rules Facts 1. male(Adam) 2. male(Barney) 3. male(Bob) 4. male(Carl) 5. female(Ann) 6. female(Beth) 7. female(Barb) 8. female(Carol) 9. parent(Adam,Barney) 10. parent(Ann,Barney) 11. parent(Adam,Beth) 12. parent(Ann,Beth) 13. parent(Adam,Bob) 14. parent(Ann,Bob) 15. parent(Adam,Barb) 16. parent(Barney,Carl) 17. parent(Carol,Carl) Rules 1. parent(y,x) AND male(x) -> son(x,y) 2. parent(y,x) AND female(x) -> daughter(x,y) 3. male(x) AND (E z)(parent(z,x) AND parent(z,y)) -> brother(x,y) 4. female(x) AND (E z)(parent(z,x) AND parent(z,y)) -> sister(x,y) 5. male(x) AND (E y)(parent(x,y) AND parent(y,z)) -> grandfather(x,z) 6. female(x) AND (E y)(parent(x,y) AND parent(y,z)) -> grandmother(x,z) ------------------------------------------------------------------------ Example Proof using Substitution Prove son(Barney,Adam) 1. male(Adam) Fact 1 2. male(Barney) Fact 2 3. parent(Adam,Barney) Fact 9 4. parent(y,x) AND male(x) -> son(x,y) Rule 1 5. parent(Adam,Barney) AND male(Barney) L2,L3 6. parent(Adam,Barney) AND male(Barney) -> son(Barney,Adam) L5, L4, sub. 7. son(Barney,Adam) L5, L6, m.p. Note that we selected the facts and the rule that would help us prove son(Barney,Adam). * There's only one rule that allows us to infer the son relationship, so we included that rule. * We need facts about the parent relationship that include reference to Adam and Barney. * We need facts about the male relationship that include (possibly) Adam and Barney. ------------------------------------------------------------------------ Example Proof with Quantifiers Prove brother(Barney,Beth): 1. male(Barney) Fact 2 2. parent(Adam,Barney) Fact 9 3. parent(Adam,Beth) Fact 11 4. male(x) AND (E z)(parent(z,x) AND parent(z,y)) -> brother(x,y) Rule 3 5. male(Barney) AND (E z)(parent(z,Barney) AND parent(z,Beth)) -> brother(Barney,Beth) L4, sub. 6. male(Barney) AND parent(Adam,Barney) AND parent(Adam,Beth) L1, L2, L3 7. brother(Barney,Beth) L5, L6, m.p. ======================================================================== Computation in Predicate Logic Prolog is a programming language based on predicate logic. * A Prolog program attempts to prove a goal, such as brother(Barney,x), from a set of facts and rules. * In the process of proving the goal to be true, using substitution and the other rules of inference, Prolog substitutes values for the variables in the goal, thereby "computing" an answer. How does Prolog know which facts and which rules to use in the proof? * Prolog uses *unification* to determine when two clauses can be made equivalent by a substitution of variables. * The unification procedure is used to instantiate the variables in a goal clause based on the facts and rules in the database. ------------------------------------------------------------------------ Horn Clauses To simplify the resolution process in Prolog, statements must be expressed in a simplified form, called Horn clauses. * Statements are constructed from terms. * Each statement (clause) has (at most) one term on the left hand side of a left-pointing implication symbol ( :- ). * Each statement has a conjunction of zero or more terms on the right hand side. Prolog has three kinds of statements, corresponding to the structure of the Horn clause used. * A fact is a clause with an empty right hand side. * A question (or goal) is a clause with an empty left hand side. * A rule is a clause with terms on both sides. ------------------------------------------------------------------------ Terms There are three kinds of terms in Prolog: * A constant is an atom or a number. An atom is a quoted character string or a string of letters, digits, and underscores that starts with a lower-case letter. A number resembles the real or integer constants used in most programming languages. * A variable is a string of letters, digits, and underscores that starts with an upper-case letter. There are no type declarations; types are discovered implicitly by the interpreter. * A structure represents an atomic proposition of predicate calculus, and has the form "atom(parameter list)". ------------------------------------------------------------------------ Facts and Rules The Prolog environment maintains a set of facts and rules in its database. * Facts are axioms; relations between terms that are assumed to be true. * Rules are theorems that allow new inferences to be made. Example facts: male(adam). female(anne). parent(adam,barney). Example rules: son(X,Y) :- parent(Y,X) , male(X) daughter(X,Y) :- parent(Y,X) , female(X) The first rule is read as follows: for all X and Y, X is the son of Y if there exists X and Y such that Y is the parent of X and X is male. The second rule is read as follows: for all X and Y, X is the daughter of Y if there exists X and Y such that Y is the parent of X and X is female. ------------------------------------------------------------------------ Observations about Prolog Rules * The implication is from right to left! * The scope of a variable is the clause in which it appears. * Variables whose first appearance is on the left hand side of the clause have implicit universal quantifiers. * Variables whose first appearance is on the right hand side of the clause have implicit existential quantifiers. ------------------------------------------------------------------------ Executing a Prolog Program To run a Prolog program the user must ask a question (goal) by stating a theorem (asserting a predicate) which the Prolog interpreter tries to prove. If the predicate contains variables, the interpreter prints the values of the variables used to make the predicate true. The interpreter uses backward chaining to prove a goal. It begins with the thing it is trying to prove, and works backwards looking for things that would imply it, until it gets to facts. ------------------------------------------------------------------------ Example: Greatest Common Divisor Using Euclid's algorithm, we can compute the GCD of two positive integers in Prolog as follows: /* Prolog program to compute GCD */ gcd(A, 0, A). gcd(A, A, A). gcd(A, B, D) :- (A>B),(B>0), R is A mod B, gcd(B,R,D). gcd(A, B, D) :- (A10) gcd(10,5,D) (10>5) (5>0) R=0 gcd(5,0,D) D=5 ------------------------------------------------------------------------ What can we specify with Horn clauses? Ignoring quantifiers for the moment, consider a statement in CNF. Since everything in the Prolog database is assumed to be true, we'll be set if we can come up with a Horn clause for each clause of the CNF expression. Consider a single clause. Some of its terms are negated; others are plain. In general, x1 | x2 | ... | xN | ~y1 | ~y2 | ... | ~yM == (x1 | x2 | ... | xN) <- (y1 & y2 & ... & yM). If there is a single plain term and no negated terms, we have a Prolog fact. As a Horn clause, this is x1 <- true. If we have a single plain term and a bunch of negated terms, we have a Prolog rule. If we have zero plain terms we have a goal. If we choose to write this as false <- y1 & y2 & ... & yM then we can add it to our database and attempt to find a proof via contradiction. If we have more than one plain term we're stuck: we can't express our CNF clause as a Horn clause. In summary: we can express a CNF clause as a Horn clause iff it has exactly one plain term. Now what about quantifiers: recall that free variables that appear in the head of a rule are universally quantified; free variables that appear only in the body are existentially quantified. If we need different quantifiers, again we're stuck. ======================================================================== Limits on Logic Given that we can model computation as a "proof", and proceed from facts (axioms) and rules to prove new facts (ie, compute), we are interested in knowing whether there are any limits to what we can prove (ie, compute). Two of the most fundamental results in mathematical logic place limits on what we can compute: * The Incompleteness Theorem [Kurt Goedel, 1931] states that for number theory (predicate logic restricted to predicates involving integers, +, *, =, and <), there are expressions that are true, but which cannot be proved to be true. * The Halting Problem [Alan Turing, 1936] states that for any computational model equivalent to modern computers (including Pascal, C, and Prolog) there is no program that takes as input an arbitrary program and its input, and determines whether or not the program halts on that input. ------------------------------------------------------------------------ The halting problem is undecidable Suppose we could decide, given a program and its input, whether that program would halt when run on that input. Then we could create the following program, D, that takes some other program P as input, and decides whether P halts when given itself (i.e. the text of P) as input. If P halts, D runs it and looks at its output. If P prints "true", D also prints "true". Otherwise (if P doesn't halt, or prints something other than "true"), D prints "false". Given D we can build a "complement" program C that also takes programs as input. Internally C runs D and then prints the opposite of what D does. That is, if D prints "true", C prints "false". If D prints "false", C prints "true". Now the question arises: what does C print when given itself as input? If C prints "false" that means D would have printed "true", meaning that D's input program (namely C!) would have printed "true" (ouch). Similarly, if C prints "true" that means D would have printed "false", meaning that D's input program (C) would either have run forever or halted and printed something other than "true" (double ouch). The contradiction implies that our original assumption (the existence of D) must be wrong. ------------------------------------------------------------------------ Number theory is undecidable Goedel proved the incompleteness theorem before Turing proved the halting problem. Given the halting problem, however, we can write a particularly nice proof of the incompleteness theorem. Consider a program running on a computer. The computer operates in a series of discrete steps, each of which can be captured by the contents of memory (including the program counter and all other registers). Some special value, appearing in some special subset of the memory, indicates the act of halting. If the program halts there must exist some maximum, over time, of the amount of memory it uses. Call that maximum m (expressed in bits). We can capture the entire step-by-step history of the program as a (huge!) binary integer. Start with a string of m ones, followed by a zero. This indicates the grouping of subsequent bits. Each subsequent group of m bits indicates the contents of memory for the next step of the computation. The rules that take us from one step to another can be expressed in terms of simple arithmetic functions. This means we can create a predicate P(e,p,i) that is true iff e is the encoding of a computation of program p, running with input i. Now suppose we that number theory is decidable. Then every true statement has a proof. But now we can solve the halting problem! Given a program p, consider the statements (Ei)[P(e,p,i)] (the program halts) and (Ai)[~P(e,p,i)] (the program does not halt). One of these statements must have a proof, which will have finite length, and which we can check in finite time. All (!) we have to do is enumerate all the proofs in the world, one at a time, (e.g. starting with the shortest and working up), until we find one for one of our two statements. This constitutes a (very slow) algorithm to solve the halting problem. Since we know the halting problem is undecidable, or original assumption (the decidability of number theory) must be wrong.