1 00:00:25,039 --> 00:00:26,550 hi everybody um glad to have you all back for our next to last installment of theory of computation today um we are going to embark on the very last big topic for the semester and that is in some ways going to be following on what we started a couple of lectures back when we looked at probabilistic turing machines and probabilistic computation and its associated class bpp now what we're going to discuss is in some sense a probabilistic version of np and that's going to be a complexity class called ip which stands for interactive proof systems um and so we're going to present that model and look at a couple of examples uh i would just like to say at the beginning that the this model is a very important one it's really uh has been the starting point for a great deal of research in complexity theory so we're just really going to be touching on it but there's a lot more that people have pursued with this model and it's also a connection in to the cryptography field which also makes use of the interactive proof system model in fact some of the genesis of that model comes out of cryptography where you're having uh multiple parties either communicating or in some ways interacting uh to achieve certain uh goals of communication or signing or passwords or or or what have you so uh this is a both a an applied area and also one that has a lot of very interesting theory uh um associated to it so with that one we we're going to jump in um and uh start out by uh making myself smaller and just do a an introduction i'm going to introduce the model or the the concept of an interactive proof with an example and that example involves the graph isomorphism problem that's the problem of testing whether two graphs are isomorphic what we mean by two graphs being isomorphic is that they're really just the same graph with one of them perhaps being re-labeled or permuted so that they may look superficially different they may appear with a different sequence of labels or the nodes are appearing in a different order but except for that it's really just the same graph so i'm kind of illustrating that here if you can see those two graphs here which look different from each other both on eight notes they are in fact the same graph as i can illustrate by a little animation which will convert this one into that one so uh the um [Music] so the two graphs um these graphs being the same um they're we call that isomorphic so these are graphs g and h and they're really the same graph so we we um call them isomorphic graphs and we are have an associated computational problem called iso which is given a pair of graphs we'd like to know are they isomorphic or not so the iso is the collection of pairs of graphs which are isomorphic and it's easy to see that this problem is an np problem because all you need to do in order to see or to give a certificate that the two graphs are isomorphic to each other is tell you it's just to say which nodes in the one graph correspond to which other nodes in the other graph um and then you all you need to check is that the edge relationships are consistent with that mapping or that isomorphism as it's called um so it's easy to see that the iso problem is an np and if you're not getting that make sure you uh understand because this is the whole first part of the lecture will be lost if you don't understand this iso problem now the question of whether you can test two graphs being isomorphic in polynomial time is not clear and in fact that's an unsolved problem to this day uh and it's a problem that has generated an enormous literature uh there are hundreds of papers on the graph isomorphism problem as it's called um to try to uh resolve um you know uh to try to see if one can find a polynomial time algorithm and in fact it was a very big result just in the last 10 years where there was a sub-exponential algorithm given so that was more than faster than the brute force search approach but didn't get it all the way down to polynomial now why is there so much attention just to this one particular np problem it's because it's not known whether the graph isomorphism problem is np-complete iso is not known to be an np-complete problem and that puts it into a very very small class of problems in np which are no not known to be either in p or np-complete it's kind of a curiosity that for np problems almost all of them have ended up being in one side or the other and in fact um it's a uh um in fact the the um i i think it's the only problem that just involves graphs that's not known to be either in p or an np um so so i got a question here what would be in between exponential and polynomial for example i don't remember what the the bound is but it's it's something in the uh in the range of n to the log n uh time complexity for the graphite some more i might be getting that wrong um i don't remember exactly what the bound is but um that's significantly better than two to the n or some some exponential amount of time but it it's more than enter any constant so it's more than any polynomial time so um another question uh of the same sort is whether the complementary problem is in np or whether whether iso is in co-np or let's let's talk about it in terms of the complement whether the complement of iso which i'll refer to as the non-iso problem whether that's known to be in np so that's also not known um in other words if i give you two graphs and i ask you to show that they're not isomorphic suppose they aren't isomorphic and you go through the effort of you know determining that by a brute force search and now you want to prove that they're not isomorphic well that's not it's not known to be an np either so there's no known short certificate certificate of two graphs not being isomorphic we don't know how to do that either but there's something that's very interesting nevertheless um and it has to do with the ability to for one party to prove to another that graphs are either isomorphic or not isomorphic so if you're just having like a prover we haven't really been necessarily formulating that this way so much in this class but this is a completely equivalent way of formulating the notion of np whether you have a polynomial time verifier an approver who can produce certificates say it's a powerful prover so um if you have a problem that's in np approver can convince a polynomial time verifier that uh strings are in the language if in fact they are so in the case of the iso problem a prover can convince a polynomial time verifier the graphs are isomorphic just by exhibiting the isomorphism now for the non-isomorphism case we don't know that that problem's in np but it's still possible for approver to convince a verifier that graphs are not isomorphic if you change the rules of the game slightly um so even though the non-iso problem is not known to be an np approver can still convince a polynomial time verifier that graphs are not isomorphic assuming they they are in fact not isomorphic um provided that um the prover and the verifier can interact with one another so the verifier can ask questions of the prover and the verifier gets to be probabilistic so that's in this in that's in sense in which i mean that this notion is a kind of a probabilistic version of np okay so um let me show you how that's done so before we jump in to the uh to the method for approver to um show a verifier that graphs are not isomorphic um let me let let's try to get a little clearer on the model so i'm going to first show it to you informally and then we'll look at it formally okay so in in interactive proofs there are two parties um and i'm going to think about them as one of them is going to be the professor okay so the professor is going to play the role of the verifier in a sense but it's like that the one who checks um and uh the professor being kind of old and tired he's been teaching too long maybe can only operate in probabilistic polynomial time so the professor if wants to tell whether two graphs are isomorphic or not probabilistic polynomial time doesn't seem to be enough to tell whether two graphs are isomorphic or not because it seems to be a more than polynomial problem however the professor has um help it has an army of graduate students and the graduate students they're not limited uh in the same way the professor is the graduate students are young they are energetic they can stay up all night they know how to code so the graduate students have unlimited computational ability so that we're going to think of the graduate students playing the role of the approver because they're not they're not limited in their capabilities we'll assume the professor on the other hand is limited so the professor wants to know if the two graphs are isomorphic let's say whatever they are um can't do it by himself so he's going to ask his students to figure out the answer and report back now there's only one problem the professor knows that students uh well in the old days they'd like to party i guess these days they like to play on uh play computer games a lot and so they're not really that eager to spend all their time figuring out whether graphs are isomorphic so he's worried that the the students will just come up with some answer and figure that he won't be able to tell the difference so the professor does not trust the students it's not enough to he for the professor to give the problem to the students and just take any answer that they're going to give the professor wants wants to be convinced okay so um now how could the students convince the professor of the answer that they've really done the work and figured out whether the graphs are isomorphic or not well if the graphs are isomorphic if it turns out that the graphs were isomorphic and the students figure that out then life is good because what are they going to do to convince the professor they're going to hand over the isomorphism and show yeah i mean they are you know those graphs really are isomorphic and here's how the correspondence works professor can check oh yeah i i now not now i'm convinced but suppose the graphs were not isomorphic what are we going to do then um the students have figured out where after night else the professor wants wants to be convinced oh no what are we going to do well in fact we're going to engage the the professor and the students are going to engage in the following protocol dialogue what's going to happen is now you have to make sure you're you're this is critical to follow to understand this little part of the story here because it's really going to set the pattern for everything in today's and tomorrow and to today's lecture and the and the next lecture okay so we're going to engage in a following interaction between the students and the professor which is going to enable the students to convince the professor that the two graphs really are not isomorphic so how is that going to work this is a beautiful little uh thing by the way so the professor is going to take the two graphs and pick one of them at random because the two graphs g and h um let's say they're not they really are not isomorphic the professor doesn't know that for sure that's what the students claim the professor really wants to not be convinced that the students are right um so the professor's gonna pick one of the two at random randomly permute that uh that choice the one that he picked and hand it over to the students say okay here is one of those two graphs randomly scrambled then i'm going to ask the students which one did i pick okay now if the graphs were really not isomorphic the students can check whether that randomly scrambled graph is isomorphic to either g or to h it's going to be isomorphic to one or the other and then they students can figure it out and they say oh you picked g or no you picked h as the case may be the students can figure that out but if the graphs were isomorphic then that scrambled version of g or h could equally well have come from either of them and the students would have no way of knowing which one the professor picked so that there's nothing they could do which would be better than guessing so if we do that a bunch of times the professor picks at random sometimes secretly of course the picks the grip picks either g or picks h and the students get it right every time either the students are really doing the work and the graphs are really not isomorphic or the students are just incredibly lucky they're managing to guess right let's say a hundred times so how the the the professor randomly and secretly picks grh uses this uses its probabilism flips a coin just a two-sided coin and says okay sometimes if we're going to do g sometimes they're going to do h just completely at random picks one or the other and then with some more randomness gets finds a random permutation of the one that he picked and then sends that over to the students and say which one did it come from um i'm not sure okay so let's pause here let's let's make sure we all understand this because this is really important um so i'm getting a question here how do we i'm not sure what your question is um okay so let me just say yeah the professor's going to play the role the verifier the graduate students play they're all approver that's coming but i really want to understand this protocol here okay so how is the professor picking the graph skin if you're okay i don't you know picking the graphs at random you have just two graphs they're in part of the input uh the both the students and the professor can see the graphs and the professors are just picking one of them at random using a coin so i'm not sure i understand the question there could p and v engage in a protocol where the secretary is on the prover side instead the question of revealing the isomorphism i there is no why so i'm not sure i understand this question either um maybe we'll make this clear you know if for for this little illustration the professor doesn't know the graphs could be isomorphic or they could be not isomorphic and so uh the professor wants to be convinced either way whatever the students whatever answer the students come up with we're going to shift this into a problem about a um deciding a language next but right now i'm just trying to give a sense of the how the model works i want to move from this informal model and now i'm going to formalize that in terms of model which will be deciding a language okay so so the interactive proof system model we have two interacting parties a verifier which is probabilistic polynomial time playing played by the professor in the previous slide and the prover which is unlimited computational power played by the students in the previous slide both of them get to see the input which in the previous case well it could be for example the pair of graphs they exchange a polynomial number of polynomial size messages so the whole exchange including the verifier's own computation is going to be polynomial the only thing that's not not not included within the computational cost is the prover's work which is unlimited um after that the verifier after the interaction the verifier will accept or reject and we're going to define the probability that the verifier together with a particular approver ends up accepting as you look over the different possible coin tosses of the verifier which could lead to different behavior on the part of the verifier and therefore different behavior on the part of the approver so over all the different possibility possibilities for the verifiers computation we're going to look at the probability that the verifier with this particular approver ends up accepting and i've written it this way this is the probability of the verifier interacting with the prover accepts the input is just simply that um and so we're going to work through an example we're going to work through the previous example more precisely in a second the class ip for interactive proofs stands for it's the class of languages such that for some verifier and approver um for strings in the language the prover makes the verifier accept with high probability and here's the interesting part for strings not in the language the prover makes it except with low probability but every there's no prover which can make it except with high probability so there's no way to cheat if you think about it in the case of the graphic non-isomorphism there's nothing you know if if the graphs were really isomorphic and the students were trying to in a devious way prove through that protocol that they're not isomorphic they would fail because there's nothing they can do if the graphs were isomorphic then um when the verifier the the professor picks one or the other at random um and scrambles it the students would have no way of telling which one the professor did so no matter what kind of scheme they try to come up with they're going to be out of luck so it's no mat for any strategy for strings that are not in the language for any s any prover calling that p with a tilde to stand for a devious or crooked prover for any uh possibly crooked prover even that with working with the verifier is still going to end up accepting with low probability so strings in the language there's going to be an honest prover who just follows the protocol in the correct way which makes the verifier accept with high probability for strings not in the language every prover is going to fail to make it accept with high probability um okay so that i mean the way i like to think about it is that p tilde is a possibly crooked proverb which is trying to make the verifier accept when it shouldn't because the string is not in the language it's like you know it's like even you can think of this in the case of um satisfiability um you know you a crooked prover might try to convince of the verifier that the formula is satisfiable when it isn't by by somehow trying to produce a satisfying assignment but that's going to be impossible there's nothing any strategy can possibly work when the formula is not satisfiable if that's what the verifier is going to check it's going to be looking for that satisfying assignment okay and by the way this is we're not going to prove this but it's really going to be proved in the same way you can make that one third error that could that occurs here something very tiny by the same kind of repetition argument okay so let's see um so why can't the prover in the first case be crooked um the prover in the first case would could be crooked but that's not going to serve the purposes um you know what what we want to show um you think about it like we think about np for strings in the language there exists a certificate there is a proof that you're in the language so if somebody is going to not produce the proof that's irrelevant the question is if you look at the best possible case the best possible prover um you know who's going to be able we're asking does there exist a way to convince the verifier that the um string is in the language so it doesn't matter that there might be some other uh silly way that doesn't work we're just looking at the best possible way so the best possible way when you're in the language is going to end up with a verifier having high probability when you're not in the language the best possible way is still going to end up with low probability when when i talk about best possible i'm trying to maximize the probability that the verifier is going to end up accepting let's continue um not sure as clear as i would like but um maybe again we're going to we're going to stick with that example because this is a very uh helpful example and to try to understand the setup and uh so we're gonna i'm gonna revisit that previous example about non-isomorphism but now in the context of this thinking about as a language so we're going to take this non-isomorphism um uh yeah we're going to take the non-isomorphism problem and show that it's an ip so there's going to be a verifier together with approver which are going to make the verifier accept with high probability for strings in the language namely graphs not ice being isomorphic and nothing there's going to be no way to make the verifier except with high probability for strings out of the language therefore that's when the graphs are isomorphic okay um so the protocol is just gonna we're gonna repeat the following thing twice you know i said in the previous case do it a hundred times just to help us think about it but actually twice is gonna be enough to get the bound we need so the verifier is going to operate like this in terms of this is the verifiers in first communicating sending messages to the approver it's going to randomly choose grh just like what the professor did last time randomly permute the result to get a new graph k which was going to be which was which is isomorphic either to grh depending upon the choice the verifier made and then send that graph k now the provers turn is going to respond by the proof is going to compare k with the two one of the both of the original graphs it's got to be isomorphic to one or the other and it's going to report back which one just going to say well you pick g no or you picked h because the prover with its unlimited capabilities can determine that um and then v accepts if the approval was right both times um and if the approval was ever not right the verify says oh something's fishy here because we know that the prover has unlimited capability so could get it right if you know um if there was if this was an honest approver um and so um if uh if it's not getting it right then the verify is going to reject so if the graphs are not isomorphic the proofer can tell which one it picked randomly so therefore if the graphs are not isomorphic the verifier with that that honest prover will accept with probability one because that honest proof is always going to get the right answer which is at least two-thirds is what the bound we need uh we don't care about the space used in answer to a question um if we were not in the language so g and i h are not isomorphic then there's nothing any crooked pervert could possibly do because it gets a graph can't tell there's no way to tell whether it came from g or came from h um so that crooked proofer would have all it could the best thing you could do is guess sort of a 50 chance advancing correctly each time and only a 25 chance for doing it twice and that's why i did it twice in order to get that error um uh to be small so there's only a 25 chance of the approver getting lucky so that would be an error case if the proof of just by chance picked the right answer twice even though the graphs were isomorphic so therefore for the isomorphic case the verifier interacting with any prover is going to accept that input with at most one quarter 25 of the time which is less than a third so those are that's just to achieve that bound okay so let's let's answer some questions first and then i'll try to um uh i'll ask you you understand this so this it's i think it's worth trying to understand this model um of this interactive proof system it's a little little slippery i i realize but um if you just hold your hold on to your intuition of the prover trying to convince you know of uh you know a powerful prover trying to convince a limited verifier um of some string being in a language uh you want the proven to be able to succeed when the string is in the language but fail when the string is not in the language yes we are going to somebody's asking if the prover is identifying grh by brute force yes the prover is going to use its unlimited capabilities to determine given k whether it came from g or h the um the computational cost of the approver is irrelevant for this it's just like when we think about a certificate um you know for satisfiability we don't talk about the cost of finding that certificate uh for np for ip again we don't talk about the cost of the prover running so somebody's asking does the crooked per prover answer just randomly or does uh can the cr quick approver has have a strategy the crooked perimeter can have a strategy now we're not we're assuming the crooked approver is devious but it's still going to fail okay um let's do the check-in suppose we change the model so that the prover can watch the verifier picking its random choices so uh the verifier cannot act in secret anymore but the the prover can watch the verifier now let's suppose we had the same protocol that i just described what language do we end up with is it the same language different language and what is that language okay so i want to hopefully um let's let's it'll give me some sense of how well you're following me by how well this uh this goes yeah someone's asking about how this connects up for example with np so we're going to look at that also in a second okay so this is reassuring that most of you i think are on the right track at least for this check-in uh do we assume p uses this access to guess right what access p is not really guessing the p is tr is actually i don't think a p is non-deterministic or anything like that p is actually trying to get the right answer and using its computational ability to do that if it's possible may not be possible then there's nothing you can do okay so let's uh end this are you all in two seconds left please vote vote now or never okay ending um yeah so c is the correct answer here if the prover can watch what the verify is doing the prover can see what graph the verifier picked right from the beginning and so the prover without having to do any work can say you know it proves looks over the verifier's shoulder and says oh you picked g and now you're randomly permitting it but i don't care about that i just i know you pick g uh so uh the proof is going to respond back uh g even if the graphs were isomorphic the proof is going to be able to get the right answer kind of interestingly um uh you can make a you can change the protocol somewhat um to make it uh that even if the prover has access to the verifier's randomness you can still achieve this but not with the same protocol um so that's uh that's a separate question um okay so let's move on here i want to get too bogged down okay here's another check-in um uh okay so you have to tell me which of the following statements are true as far as you know now you have to think about a little bit how this these uh relate to um how np and ip or bpp and ip relate to one another okay how are we doing on this okay so we're gonna have to close this pretty soon too do the best you can interesting okay closing up shop last last vote okay one two three there's one more person out there who hasn't voted who voted last time oh well uh all right in fact they're all true um let's see why is np contained with ip contained in ip uh well many of you have seen this already so let's just quickly go through it um [Music] if we just had a deterministic v um you know the uh you can uh maybe it's just that that can be enough of deterministic v i think it's just going to be equivalent but actually just to be doubly sure the deterministic v and the approver just sends a message to the verifier and then checks it that's the way we normally think about a certificate for np uh i don't think it's going to change anything but should double check that if the verifier can still ask questions but i think as long as the verifier is deterministic you're going to get exactly np here um and um now how about bpp well there you don't even need approver because the verifier is already probabilistic so verifier can ignore the approver and this one is a little tricky uh i p containing p space because we haven't covered that so there's no way for you to know that unless you happen to read ahead in the book but it's in fact true um in some ways it's a little bit like the proof that uh np is contained in p space ip is sort of an enhanced version of np and you and there's just a basically a uh a piece based brute force algorithm that goes through the entire tree of possibilities um of the verifier exchange verifier with exchanges with approver and can uh determine that um the verifier is either going to accept for some approver or is going to end up projecting for every approver um so we're not going to prove this statement but something good for you to know anyway they're just a fact uh but we're going to do what the surprising thing in reference to part c is that the containment also goes the other way this is the amazing um uh was is an amazing result that everything in p space you can do with in ip so this is ip is actually turns out to be incredibly powerful gives you everything up in p space you get i p equals p space so that says that any problem that you can solve in p space like any of the a game for example um if you can imagine you know formulating you know checkers or chess as a piece based problem which you know depending upon some details of the rules you can do because you know you have to generalize it to an end by end board but okay let's not quibble um uh then uh um [Music] we don't know which side has a forced win in chess um and even if somebody goes through the effort of going through the game tree um and determines that let's say white has a forced win uh there's no way for them to there's no short certificate we don't know that that problem is not an np but by going through an interactive proof an all-powerful prover could still convince somebody that white had a force you know convince somebody in polynomial time that a white has a forced wind let's say in um in chess again little uh stretching things because this is you know you really need to talk about this as an n by n not an eight by eight but i think in this the spirit is is uh fair so um okay uh so let's continue so wha when we're not going to quite prove that that p base is contained in ip we're going to prove a somewhat weaker statement but very similar um is that uh and historically came first that co-np is contained in ip so not only is np contained in ip but we're going to prove that co np is contained in ip and this actually has most of the i most of the idea for the p space being contained in ip and itself it's just an amazing proof a little easier um okay and th this was done if i'm remembering somebody's asking me when how old is this it's something in the 19 i think late 90s but i'm not i don't remember but maybe early 90s i think it's late 90s when this was shown so it's been a while now um okay so um yeah so in terms of the relationship with cryptography there were two parallel threads um that both independently came up with the notion of an interactive purge system uh i was a little bit personally involved with this in in a way as well but but mainly that the the there was one group in cryptography um working on this and there was another group who was actually coming out of the graph isomorphism world working on it and they they came up with two separate models one involving the private randomness and one involving the public randomness um and it was turned out that was that they're actually equivalent um and uh it's an interesting story but unfortunately we don't have time for it uh so why don't we move on and i'm gonna start showing you how the proof that co-np is contained in ip goes and what we're going to do is work with a problem that's almost like co p complete but um going to be uh well it's going to be this number sat problem we'll see the connection with co-np in a second uh so cohen p so it's supposed to be exactly k satisfying assignments fee comma k is the set of pairs where the formula fee has exactly k satisfying assignments so really that's a problem of counting how many satisfying assignments you have in a formula um so you know for np you have at least one um but i'm i want to know exactly how many uh so the numbers that problem is the pairs formula and the count so um and uh so if we define the count number fee is the number of satisfying assignments of a fee then another way of writing this uh number stat problem is uh the pairs phi k where k is the number of satisfying assignments of fee so we're going to be using this notation number fee a lot so just make sure you get you got that notation this is the number of satisfying assignments of that formula okay and here's a definition i probably should have given you earlier in the term but better late than never um uh so the notion that a language is np hard it's like np complete except without being necessarily being in np so this is just the reduction part a language is np-hard or co-np-hard or p-space hard or any of those other classes that we looked at if every problem in the class is reducible to that language but you don't know whether this that that language is in the class so we just call it np hard instead of mp complete so you could say the language is np complete if it's hard and it's in np okay and so we're going to show that this number set problem is co np hard so everything in cohen p is polynomial time reducible to number set that's easy because what we're going to do is take a co np complete problem which is the unsatisfiability problem the complement of satisfiability and sure that reduces to the numbers that problem and that's easy because a formula is unsatisfiable exactly when it has zero satisfying assignments so if you can tell how many satisfying assignments something has exactly or you can answer the question you know does a formula have exactly a thousand satisfying assignments if you can do that in general then you can solve co and p uh you can solve the unsatisfiability problem by asking with zero satisfying assignments and that allows you to solve anything in cohen p okay so we're gonna just work with this one problem the number set problem and show that that problem's in ip okay let's take a quick break um okay feel free to send me let me see if i can catch up with some of the questions that have been cropping up here so if the approver knows the random choices of the verifier can it can flip the answer to make the verifier reject not sure what that you mean in the context just of the graph isomorphism problem or um something in general i'm not sure you'll have to explain sorry i'll respond with a question mark what else can i answer for you guys so i got a question might if i p equals p space does that mean that iso or the or non-iso might be um in might be p space complete but no that's not known so we're about out of time okay let's continue here [Music] okay so this this is where we're kind of getting going to start to get into the meat of things um [Music] and if you didn't quite understand everything up till now maybe just try to keep your intuition about how do i you know how does a powerful party convince a um a polynomial probabilistic polynomial time party of the number of satisfying assignments exact number not not at least but you won't know exactly the number um of satisfying assignments um so it could be zero for example how do you convince that how do you convince someone that there were zero assignments and and and you know you can have an interaction which does that and that's not obvious at all how you're gonna do that um uh all right so um okay so we're gonna have to introduce some notation which i don't hope this doesn't cause heartburn here uh so let's say again here is the the computational the language we're working with number set and we have a fee where that has m variables x1 to xm now here's our notation i'm going to if i write phi with this free phi of 0 that just means the formula that i get by plugging in 0 for x one and leaving all the rest of the variables uh alone okay so i substitute zero for x one where zero means false and one means true as usual and but that's it's going it's still going to be some other formula but just with that substitution if i write fee 0 1 that means i preset the first two variables to 0 and 1. um if i write fee with a bunch of preset values i'm just setting the first i variables x1 to xi to some values and leaving the other variables as unset so i'm calling the ones that i'm nailing in there as i'm already saying these are the presets okay so this is just converting some formulas into other formulas that have somewhat fewer variables all right now let's recall that number notation the number sign notation number fee is the number of satisfying assignments now if i say number fee of 0 that's the number of satisfying assignments when i've preset x1 to 0. similarly if i preset the first i variables to some values and then i take i want to think how many how many satisfying assignments subject to those prefix presets i write it this way so i'm going to use this notation a lot you have to understand this notation ask if you don't none if you don't get it so another way of writing it i don't know if this is helpful but another way of writing number fee of a1 to a i remember we have m variables all together that means i take the variables which i have not yet preset um and i allow them to range over all possible zeros and ones and i add up the the formulas values for all of those so there's a one every time i satisfy and a zero every time i don't satisfy so i'm adding up all the satisfying assignments subject to these i presets okay so here are two critical facts about this number sign notation first of all if i preset the first i values to something now i can in addition set the next variable either to zero or to one and i get this relationship which is just simply a generalization of the fact that the total number of satisfying assignments of the formula is equal to the number of satisfying assignments when x1 is 0 plus the number of satisfying assignments when x1 is 1. right they together have to add up to the total number because x1 is going to be the either zero or one so that's fact number one fact number two is that if i preset everything all of the variables so there are no variables left then the number of satisfying assignments subject to that preset of everything is just whether or not i've satisfied the formula which is the value of the formula on that those presets okay both two simple facts but it's going to be critical in the protocol i'm about to describe questions on this i think actually i do have a question for you so let's just see what do you think just to check your understanding okay got about 80 getting this um i'm not sure that's good but uh all right almost done closing okay okay so uh yes a is the correct answer you know if if there are nine satisfying assignments all together and there are six satisfying assignments where the first variable is set to zero then there's only three satisfying assignments with the first variable set to one because nine has got to be equal to six plus three that's actually this fact number one it's not going to be 15 this is not true either so it's just a good okay okay so let's try to with with that knowledge let's try to see how we can put number sat in ip so this is not going to quite work but it's really going to set us up to do this to finish this next time um so you might immediately see where this is going wrong but you'll you'll have to put up with it because um the setup is what's important um okay so understand now here's the here's the the setup we have um the input is a formula and a number where that number is supposed to be the number of satisfying assignments you know it could be wrong and in which case we're not in the language um but if it's right you're in the language so the approver is supposed to convince the verifier that it's correct if it is correct and it's not gonna it's gonna fail no matter what it tries to do if it's uh not correct so this is the approver is going to send first of all um so so the proof is going to send a claim about the number of satisfying assignments going to send when i say this value here this is what the prover if it's honest is going to send the right value of course the verifier does not know if the approver is honest but i'm describing how the honest approver is going to operate and we'll have to understand what happens if the prover tries to cheat so the proof is going to send the honest proof is going to send the number of satisfying assignments all together and the prover verifier just makes sure that that matches up with the input if it doesn't match up with the input uh the verifier is just gonna you know you know the the verifier is gonna not be convinced that the input is um in the language so it's gonna just uh reject at that point um okay uh then now the verifier says okay that was very good that you sent me this how do i know that's right so what the verif prover is going to do to try to convince the verifier that this value was correct is uh unravel that by one level by say well you know there were nine satisfying assignments all together uh six of them were when the when x one is zero and three of them were when x one is one to verify what does the verify have to check that these add up correctly when i preset x1 to zero and to one it had better add up to the total number of satisfying assignments if that works out the verifier is happy is still being it's still consistent with being convinced that this k was the right value um so um the next step is well the verifier says well how do i know those two values are correct the prover says okay well i'm going to prove send un unravel them one level further then here's a number of satisfying assignments when the next variable is set to both possibilities for each of the possibilities of the first variable now if you're understanding me about what the prover is sending you should start to be getting a little nervous because something is i mean this is going to be correct but it's going to start it looks like it's starting to blow up in terms of the number of amount of work that's involved and that's actually a problem but let's bear with that for the moment let's just worry about correctness not about complexity for the moment so the proof is going to now send the number of satisfying assignments for each of those four possible ways of presetting the first two variables and the verifier is going to check that that was consistent with the information the prover sent in the previous round right by again checking this identity here so then the proof is going to continue on doing that until it's got it's done that through m rounds where m is the number of variables so at this point the proof is going to send all possible ways of presetting all of the variables so that now there's two to the m possibilities here again this is hopelessly not allowed but okay ignoring that the proofer's got to use this at the nth round to check what happens at the at the previous round so that's when they were m minus one values sent because each one has one more uh uh you're extending the presets by one so which using this to check that the previous round was the values were correct so it's looking for um you know the m minus one presets have to add up correctly um you know in terms of the m the presets of m uh values uh for each of those ways of uh doing those uh m minus one presets and so now the provers send all of those two to the m counts which are by the way ones and zeros because at this point we have preset all of the values of the variables and so there's only one possible assignment at most that there can be and now the verifier the approver is done the verifier is going to check by itself that these values make sense that these values are correct so it's going to do that by looking back at the formula so far up at this point the verifier has not been looking at the formula it's just been checking the internal consistency of the provers messages with each other but now at the end the verifier is going to take the messages these these values that the approver sent for each of the two to the m presets and see if it matches up with what the formula would do remember that was the other sort of the base case of the the end the fact number two um from the uh slider to ago make sure that these agree okay and now the verifier says well okay if everything has checked out and all of these are are in agreement then the verifier is going to be convinced that um that uh fee had k satisfying assignments but if anywhere along the way one of these checks fails the approver is not the verifier is not going to be convinced and is going to reject okay so in a sense this is kind of dopey you know you know we've just i'm just kind of giving you a complicated way of just counting up one by one each of the satisfying assignments of the formula and seeing if that matches k but nevertheless this way of looking at it is gonna be uh help us to understand um the way to fix this so so bear with me for another minute on this one so another way of looking at this which i think is is particularly useful is to think of what happens well okay we'll get there in a second i want to look at what happens if k was wrong but before i do that let's look at the i'm going to give a kind of a graphic a graphical view of um the information that the prover sends and the and the verifiers actions in this protocol so the the values that the prover sending are going to be in yellow so the and the information that the verifier has or checks is going to be in white so the the verifier has the k the the input value which is supposed to be the number of satisfying assignments and the prover sends some value and the verifier checks that this value which is supposed to be the number of satisfying assignments corresponds with k so that's one of the checks it does then the approver is going to send kind of take to justify this value it um sends the number of satisfying assignments when you have x1 set to zero or set to one the verify adds those up to give you and it's supposed to equal the total number of satisfying assignments and so this is if you understood this protocol this is just i'm writing it out in a sort of a simplified way perhaps okay and so um keeps checking that these things add up correctly until you get down to setting all m values in all two to the impossible ways and now the verifier is going to then check to make sure that that equals the what the formula would say um okay okay so now let's what happens if k was the wrong value it did not agree with the number of satisfying assignments um and what does what happens now um could the prover what happens if the approver tries to make the verifier accept anyway so um so the only thing the prover can do at the very first step would be to lie um about you know if the approver sends the if k is wrong and the proof approver sends the correct value for the the the total count the verifier is going to reject so i'm trying to see is it could the approver try to make the verifier accept what what happens so the prover has to lie here and i'm going to indicate that by saying the approver is sending in the wrong value for um uh the the total count well if the proof is going to lie here um then just like you know if you you know you have a child who um tells a lie and then you start you know as the parent you start asking questions to try to see if the story is consistent one lie is going to lead to another lie um and that's what happens here if the uh in order to justify this lie um the proof is going to have to a lie in one or perhaps both but at least one of these two values because you can't have the two correct values adding up to the incorrect value because you have to think about what's going on here so if this is a lie that's going to force a lie at one of one side of the other one level down which is then going to force a lie to propagate down and so there's a lie at every stage is going to force a lie at least in one one place or another to propagate all the way down to the bottom and then at the bottom the verifier will see that the check doesn't work as when it turns when it tries to connect it up with the formula itself and the form verifier will reject okay so just a way of looking at this um if the form if the value if the input was not in the language um so uh but the problem is that as i said that this is exponential so how are we going to fix that so just looking ahead to what we're going to do on tuesday okay let's see if there's any questions here first of all uh okay yes i got a question should this be uh should this be a minus i i purposely made this bracket only go not include the very last zero yeah there's a total of m zeroes here all together but the i left out the last zero that's why i said n minus one maybe it would have been better to say m okay so and you got another interesting question here why can't we reject right away if k is wrong um uh well the verifier is probabilistic polynomial time how does the verifier know if k is wrong um so it means or or or right so what we're trying to do is something like you know like np where we have a certificate but now we have this kind of interactive certificate in the form of disprover maybe that's another way to look at it um where if you're in the language there should be some way for the prover to convince to get make you accept but if you're not not in the language there should be no way for the proverb to make you accept um uh so the verifier just can't reject right away because there's no way to tell how does the verifier know it's going to start rejecting things when it shouldn't if it's just going to be rejecting willy-nilly here um okay how does the verifier need to determine if the prover is internally consistent instead of just asking so why does the verifier need to determine if the approver is internally consistent instead of just asking the questions in step n plus one yeah so maybe that's because it looks like all of the work is happening at the very end um but i'm really presenting this to you as a preparation for what we're going to do on tuesday um so it's important to think about the connection from each step to the next each step is going to be justified by what happens at the next step until we get to the very end so you have to just understand it for what it is don't try to make it more efficient yeah i realize this is kind of dumb um good point we're not using the probabilism here um and moreover we're not really even using the interaction here the prover is doing all the sending the verifier is just accepting at the end yeah this is we're not using the power and we're getting a weaker result so let's move on before we run out of time here so how do we gonna fix this so the problem is is blowing up each to justify each stage where each value we're um needing to present two values which add up to it um and that's uh good leading to a blow up now it would be nice if we can do something where each value was supported by just a single value at the next level so we know here's an idea well you know in order to understand to see that uh that that this total count is correct why don't we just pick at random either zero or one and only follow that one down well the problem with doing that is because the the the sequence of lies is could be just a single path through this uh tree and the chances you're going to find that path down to a contradiction at the bottom is very low if you're just doing it at random um so just randomly picking zeros and ones as as the as the one you're going to justify used to justify the previous value is not going to be good enough so what but this is what we're going to do however the values that we're going to pick for um for these random um inputs are not going to be boolean values we're going to pick non-boolean assignments to the variables which again just as with the branching program case didn't make any sense on the surface of it we're going to have to make it make sense and we'll have to see how to do that on tues in tuesday's lecture okay so that's kind of the setup okay um yeah so in a similar question why is this any different from just non-deterministically guessing the assignments it's because of this we're really setting the stage okay so what we did today was we introduced them the the model and defined the complexity class we can we did show this one in its full glory we showed that non-iso is an ip really worth understanding this uh protocol here making sure you you're comfortable with that and and also the the model itself and so for tuesday's lecture we're going to finish this up uh well we started showing that number set as an ip which is what we need to to do to prove co-np is an ip and we'll finish that next time which will be our last time okay so that's it for today i'll stick around for questions so a good question here why can't um v just reject if some of the checks are incorrect yes v could as soon as there's a check that fails we can just reject at that stage i'm just trying to argue that at some point along the way if the input is not in the language there's going to be a check that fails so i mean i said reject at the end but yeah i mean you could have rejected at any point along the way um okay um someone's asking for what what role did i play uh so i did my my own personal role in this i were to which was twofold first of all i i came up with the idea uh of well not the idea i came up with the name interactive proof uh i remember when sylvia mccauley was explaining this to me in my apartment many many years ago he had a kind of a little bit complicated and i don't even remember what the protocol was for it was not for something simple was something involving uh prime numbers and i said oh that's a kind of an interactive proof and it's and it it stuck from that point on so um that was one thing but the other thing in terms of more mathematically i my role was so uh shafi goldwater and i approved the equivalence of the two models the public coin and the private coin version um so that that was uh my uh my role in this back and when this was all first coming out approved it approved it on an airplane on the way to a conference somewhere so i think we're going to notice any other questions i think we'll head out take care everybody see one see you on tuesday bye bye you 2 00:00:26,550 --> 00:00:26,560 hi everybody um glad to have you all back for our next to last installment of theory of computation today um we are going to embark on the very last big topic for the semester and that is in some ways going to be following on what we started a couple of lectures back when we looked at probabilistic turing machines and probabilistic computation and its associated class bpp now what we're going to discuss is in some sense a probabilistic version of np and that's going to be a complexity class called ip which stands for interactive proof systems um and so we're going to present that model and look at a couple of examples uh i would just like to say at the beginning that the this model is a very important one it's really uh has been the starting point for a great deal of research in complexity theory so we're just really going to be touching on it but there's a lot more that people have pursued with this model and it's also a connection in to the cryptography field which also makes use of the interactive proof system model in fact some of the genesis of that model comes out of cryptography where you're having uh multiple parties either communicating or in some ways interacting uh to achieve certain uh goals of communication or signing or passwords or or or what have you so uh this is a both a an applied area and also one that has a lot of very interesting theory uh um associated to it so with that one we we're going to jump in um and uh start out by uh making myself smaller and just do a an introduction i'm going to introduce the model or the the concept of an interactive proof with an example and that example involves the graph isomorphism problem that's the problem of testing whether two graphs are isomorphic what we mean by two graphs being isomorphic is that they're really just the same graph with one of them perhaps being re-labeled or permuted so that they may look superficially different they may appear with a different sequence of labels or the nodes are appearing in a different order but except for that it's really just the same graph so i'm kind of illustrating that here if you can see those two graphs here which look different from each other both on eight notes they are in fact the same graph as i can illustrate by a little animation which will convert this one into that one so uh the um [Music] so the two graphs um these graphs being the same um they're we call that isomorphic so these are graphs g and h and they're really the same graph so we we um call them isomorphic graphs and we are have an associated computational problem called iso which is given a pair of graphs we'd like to know are they isomorphic or not so the iso is the collection of pairs of graphs which are isomorphic and it's easy to see that this problem is an np problem because all you need to do in order to see or to give a certificate that the two graphs are isomorphic to each other is tell you it's just to say which nodes in the one graph correspond to which other nodes in the other graph um and then you all you need to check is that the edge relationships are consistent with that mapping or that isomorphism as it's called um so it's easy to see that the iso problem is an np and if you're not getting that make sure you uh understand because this is the whole first part of the lecture will be lost if you don't understand this iso problem now the question of whether you can test two graphs being isomorphic in polynomial time is not clear and in fact that's an unsolved problem to this day uh and it's a problem that has generated an enormous literature uh there are hundreds of papers on the graph isomorphism problem as it's called um to try to uh resolve um you know uh to try to see if one can find a polynomial time algorithm and in fact it was a very big result just in the last 10 years where there was a sub-exponential algorithm given so that was more than faster than the brute force search approach but didn't get it all the way down to polynomial now why is there so much attention just to this one particular np problem it's because it's not known whether the graph isomorphism problem is np-complete iso is not known to be an np-complete problem and that puts it into a very very small class of problems in np which are no not known to be either in p or np-complete it's kind of a curiosity that for np problems almost all of them have ended up being in one side or the other and in fact um it's a uh um in fact the the um i i think it's the only problem that just involves graphs that's not known to be either in p or an np um so so i got a question here what would be in between exponential and polynomial for example i don't remember what the the bound is but it's it's something in the uh in the range of n to the log n uh time complexity for the graphite some more i might be getting that wrong um i don't remember exactly what the bound is but um that's significantly better than two to the n or some some exponential amount of time but it it's more than enter any constant so it's more than any polynomial time so um another question uh of the same sort is whether the complementary problem is in np or whether whether iso is in co-np or let's let's talk about it in terms of the complement whether the complement of iso which i'll refer to as the non-iso problem whether that's known to be in np so that's also not known um in other words if i give you two graphs and i ask you to show that they're not isomorphic suppose they aren't isomorphic and you go through the effort of you know determining that by a brute force search and now you want to prove that they're not isomorphic well that's not it's not known to be an np either so there's no known short certificate certificate of two graphs not being isomorphic we don't know how to do that either but there's something that's very interesting nevertheless um and it has to do with the ability to for one party to prove to another that graphs are either isomorphic or not isomorphic so if you're just having like a prover we haven't really been necessarily formulating that this way so much in this class but this is a completely equivalent way of formulating the notion of np whether you have a polynomial time verifier an approver who can produce certificates say it's a powerful prover so um if you have a problem that's in np approver can convince a polynomial time verifier that uh strings are in the language if in fact they are so in the case of the iso problem a prover can convince a polynomial time verifier the graphs are isomorphic just by exhibiting the isomorphism now for the non-isomorphism case we don't know that that problem's in np but it's still possible for approver to convince a verifier that graphs are not isomorphic if you change the rules of the game slightly um so even though the non-iso problem is not known to be an np approver can still convince a polynomial time verifier that graphs are not isomorphic assuming they they are in fact not isomorphic um provided that um the prover and the verifier can interact with one another so the verifier can ask questions of the prover and the verifier gets to be probabilistic so that's in this in that's in sense in which i mean that this notion is a kind of a probabilistic version of np okay so um let me show you how that's done so before we jump in to the uh to the method for approver to um show a verifier that graphs are not isomorphic um let me let let's try to get a little clearer on the model so i'm going to first show it to you informally and then we'll look at it formally okay so in in interactive proofs there are two parties um and i'm going to think about them as one of them is going to be the professor okay so the professor is going to play the role of the verifier in a sense but it's like that the one who checks um and uh the professor being kind of old and tired he's been teaching too long maybe can only operate in probabilistic polynomial time so the professor if wants to tell whether two graphs are isomorphic or not probabilistic polynomial time doesn't seem to be enough to tell whether two graphs are isomorphic or not because it seems to be a more than polynomial problem however the professor has um help it has an army of graduate students and the graduate students they're not limited uh in the same way the professor is the graduate students are young they are energetic they can stay up all night they know how to code so the graduate students have unlimited computational ability so that we're going to think of the graduate students playing the role of the approver because they're not they're not limited in their capabilities we'll assume the professor on the other hand is limited so the professor wants to know if the two graphs are isomorphic let's say whatever they are um can't do it by himself so he's going to ask his students to figure out the answer and report back now there's only one problem the professor knows that students uh well in the old days they'd like to party i guess these days they like to play on uh play computer games a lot and so they're not really that eager to spend all their time figuring out whether graphs are isomorphic so he's worried that the the students will just come up with some answer and figure that he won't be able to tell the difference so the professor does not trust the students it's not enough to he for the professor to give the problem to the students and just take any answer that they're going to give the professor wants wants to be convinced okay so um now how could the students convince the professor of the answer that they've really done the work and figured out whether the graphs are isomorphic or not well if the graphs are isomorphic if it turns out that the graphs were isomorphic and the students figure that out then life is good because what are they going to do to convince the professor they're going to hand over the isomorphism and show yeah i mean they are you know those graphs really are isomorphic and here's how the correspondence works professor can check oh yeah i i now not now i'm convinced but suppose the graphs were not isomorphic what are we going to do then um the students have figured out where after night else the professor wants wants to be convinced oh no what are we going to do well in fact we're going to engage the the professor and the students are going to engage in the following protocol dialogue what's going to happen is now you have to make sure you're you're this is critical to follow to understand this little part of the story here because it's really going to set the pattern for everything in today's and tomorrow and to today's lecture and the and the next lecture okay so we're going to engage in a following interaction between the students and the professor which is going to enable the students to convince the professor that the two graphs really are not isomorphic so how is that going to work this is a beautiful little uh thing by the way so the professor is going to take the two graphs and pick one of them at random because the two graphs g and h um let's say they're not they really are not isomorphic the professor doesn't know that for sure that's what the students claim the professor really wants to not be convinced that the students are right um so the professor's gonna pick one of the two at random randomly permute that uh that choice the one that he picked and hand it over to the students say okay here is one of those two graphs randomly scrambled then i'm going to ask the students which one did i pick okay now if the graphs were really not isomorphic the students can check whether that randomly scrambled graph is isomorphic to either g or to h it's going to be isomorphic to one or the other and then they students can figure it out and they say oh you picked g or no you picked h as the case may be the students can figure that out but if the graphs were isomorphic then that scrambled version of g or h could equally well have come from either of them and the students would have no way of knowing which one the professor picked so that there's nothing they could do which would be better than guessing so if we do that a bunch of times the professor picks at random sometimes secretly of course the picks the grip picks either g or picks h and the students get it right every time either the students are really doing the work and the graphs are really not isomorphic or the students are just incredibly lucky they're managing to guess right let's say a hundred times so how the the the professor randomly and secretly picks grh uses this uses its probabilism flips a coin just a two-sided coin and says okay sometimes if we're going to do g sometimes they're going to do h just completely at random picks one or the other and then with some more randomness gets finds a random permutation of the one that he picked and then sends that over to the students and say which one did it come from um i'm not sure okay so let's pause here let's let's make sure we all understand this because this is really important um so i'm getting a question here how do we i'm not sure what your question is um okay so let me just say yeah the professor's going to play the role the verifier the graduate students play they're all approver that's coming but i really want to understand this protocol here okay so how is the professor picking the graph skin if you're okay i don't you know picking the graphs at random you have just two graphs they're in part of the input uh the both the students and the professor can see the graphs and the professors are just picking one of them at random using a coin so i'm not sure i understand the question there could p and v engage in a protocol where the secretary is on the prover side instead the question of revealing the isomorphism i there is no why so i'm not sure i understand this question either um maybe we'll make this clear you know if for for this little illustration the professor doesn't know the graphs could be isomorphic or they could be not isomorphic and so uh the professor wants to be convinced either way whatever the students whatever answer the students come up with we're going to shift this into a problem about a um deciding a language next but right now i'm just trying to give a sense of the how the model works i want to move from this informal model and now i'm going to formalize that in terms of model which will be deciding a language okay so so the interactive proof system model we have two interacting parties a verifier which is probabilistic polynomial time playing played by the professor in the previous slide and the prover which is unlimited computational power played by the students in the previous slide both of them get to see the input which in the previous case well it could be for example the pair of graphs they exchange a polynomial number of polynomial size messages so the whole exchange including the verifier's own computation is going to be polynomial the only thing that's not not not included within the computational cost is the prover's work which is unlimited um after that the verifier after the interaction the verifier will accept or reject and we're going to define the probability that the verifier together with a particular approver ends up accepting as you look over the different possible coin tosses of the verifier which could lead to different behavior on the part of the verifier and therefore different behavior on the part of the approver so over all the different possibility possibilities for the verifiers computation we're going to look at the probability that the verifier with this particular approver ends up accepting and i've written it this way this is the probability of the verifier interacting with the prover accepts the input is just simply that um and so we're going to work through an example we're going to work through the previous example more precisely in a second the class ip for interactive proofs stands for it's the class of languages such that for some verifier and approver um for strings in the language the prover makes the verifier accept with high probability and here's the interesting part for strings not in the language the prover makes it except with low probability but every there's no prover which can make it except with high probability so there's no way to cheat if you think about it in the case of the graphic non-isomorphism there's nothing you know if if the graphs were really isomorphic and the students were trying to in a devious way prove through that protocol that they're not isomorphic they would fail because there's nothing they can do if the graphs were isomorphic then um when the verifier the the professor picks one or the other at random um and scrambles it the students would have no way of telling which one the professor did so no matter what kind of scheme they try to come up with they're going to be out of luck so it's no mat for any strategy for strings that are not in the language for any s any prover calling that p with a tilde to stand for a devious or crooked prover for any uh possibly crooked prover even that with working with the verifier is still going to end up accepting with low probability so strings in the language there's going to be an honest prover who just follows the protocol in the correct way which makes the verifier accept with high probability for strings not in the language every prover is going to fail to make it accept with high probability um okay so that i mean the way i like to think about it is that p tilde is a possibly crooked proverb which is trying to make the verifier accept when it shouldn't because the string is not in the language it's like you know it's like even you can think of this in the case of um satisfiability um you know you a crooked prover might try to convince of the verifier that the formula is satisfiable when it isn't by by somehow trying to produce a satisfying assignment but that's going to be impossible there's nothing any strategy can possibly work when the formula is not satisfiable if that's what the verifier is going to check it's going to be looking for that satisfying assignment okay and by the way this is we're not going to prove this but it's really going to be proved in the same way you can make that one third error that could that occurs here something very tiny by the same kind of repetition argument okay so let's see um so why can't the prover in the first case be crooked um the prover in the first case would could be crooked but that's not going to serve the purposes um you know what what we want to show um you think about it like we think about np for strings in the language there exists a certificate there is a proof that you're in the language so if somebody is going to not produce the proof that's irrelevant the question is if you look at the best possible case the best possible prover um you know who's going to be able we're asking does there exist a way to convince the verifier that the um string is in the language so it doesn't matter that there might be some other uh silly way that doesn't work we're just looking at the best possible way so the best possible way when you're in the language is going to end up with a verifier having high probability when you're not in the language the best possible way is still going to end up with low probability when when i talk about best possible i'm trying to maximize the probability that the verifier is going to end up accepting let's continue um not sure as clear as i would like but um maybe again we're going to we're going to stick with that example because this is a very uh helpful example and to try to understand the setup and uh so we're gonna i'm gonna revisit that previous example about non-isomorphism but now in the context of this thinking about as a language so we're going to take this non-isomorphism um uh yeah we're going to take the non-isomorphism problem and show that it's an ip so there's going to be a verifier together with approver which are going to make the verifier accept with high probability for strings in the language namely graphs not ice being isomorphic and nothing there's going to be no way to make the verifier except with high probability for strings out of the language therefore that's when the graphs are isomorphic okay um so the protocol is just gonna we're gonna repeat the following thing twice you know i said in the previous case do it a hundred times just to help us think about it but actually twice is gonna be enough to get the bound we need so the verifier is going to operate like this in terms of this is the verifiers in first communicating sending messages to the approver it's going to randomly choose grh just like what the professor did last time randomly permute the result to get a new graph k which was going to be which was which is isomorphic either to grh depending upon the choice the verifier made and then send that graph k now the provers turn is going to respond by the proof is going to compare k with the two one of the both of the original graphs it's got to be isomorphic to one or the other and it's going to report back which one just going to say well you pick g no or you picked h because the prover with its unlimited capabilities can determine that um and then v accepts if the approval was right both times um and if the approval was ever not right the verify says oh something's fishy here because we know that the prover has unlimited capability so could get it right if you know um if there was if this was an honest approver um and so um if uh if it's not getting it right then the verify is going to reject so if the graphs are not isomorphic the proofer can tell which one it picked randomly so therefore if the graphs are not isomorphic the verifier with that that honest prover will accept with probability one because that honest proof is always going to get the right answer which is at least two-thirds is what the bound we need uh we don't care about the space used in answer to a question um if we were not in the language so g and i h are not isomorphic then there's nothing any crooked pervert could possibly do because it gets a graph can't tell there's no way to tell whether it came from g or came from h um so that crooked proofer would have all it could the best thing you could do is guess sort of a 50 chance advancing correctly each time and only a 25 chance for doing it twice and that's why i did it twice in order to get that error um uh to be small so there's only a 25 chance of the approver getting lucky so that would be an error case if the proof of just by chance picked the right answer twice even though the graphs were isomorphic so therefore for the isomorphic case the verifier interacting with any prover is going to accept that input with at most one quarter 25 of the time which is less than a third so those are that's just to achieve that bound okay so let's let's answer some questions first and then i'll try to um uh i'll ask you you understand this so this it's i think it's worth trying to understand this model um of this interactive proof system it's a little little slippery i i realize but um if you just hold your hold on to your intuition of the prover trying to convince you know of uh you know a powerful prover trying to convince a limited verifier um of some string being in a language uh you want the proven to be able to succeed when the string is in the language but fail when the string is not in the language yes we are going to somebody's asking if the prover is identifying grh by brute force yes the prover is going to use its unlimited capabilities to determine given k whether it came from g or h the um the computational cost of the approver is irrelevant for this it's just like when we think about a certificate um you know for satisfiability we don't talk about the cost of finding that certificate uh for np for ip again we don't talk about the cost of the prover running so somebody's asking does the crooked per prover answer just randomly or does uh can the cr quick approver has have a strategy the crooked perimeter can have a strategy now we're not we're assuming the crooked approver is devious but it's still going to fail okay um let's do the check-in suppose we change the model so that the prover can watch the verifier picking its random choices so uh the verifier cannot act in secret anymore but the the prover can watch the verifier now let's suppose we had the same protocol that i just described what language do we end up with is it the same language different language and what is that language okay so i want to hopefully um let's let's it'll give me some sense of how well you're following me by how well this uh this goes yeah someone's asking about how this connects up for example with np so we're going to look at that also in a second okay so this is reassuring that most of you i think are on the right track at least for this check-in uh do we assume p uses this access to guess right what access p is not really guessing the p is tr is actually i don't think a p is non-deterministic or anything like that p is actually trying to get the right answer and using its computational ability to do that if it's possible may not be possible then there's nothing you can do okay so let's uh end this are you all in two seconds left please vote vote now or never okay ending um yeah so c is the correct answer here if the prover can watch what the verify is doing the prover can see what graph the verifier picked right from the beginning and so the prover without having to do any work can say you know it proves looks over the verifier's shoulder and says oh you picked g and now you're randomly permitting it but i don't care about that i just i know you pick g uh so uh the proof is going to respond back uh g even if the graphs were isomorphic the proof is going to be able to get the right answer kind of interestingly um uh you can make a you can change the protocol somewhat um to make it uh that even if the prover has access to the verifier's randomness you can still achieve this but not with the same protocol um so that's uh that's a separate question um okay so let's move on here i want to get too bogged down okay here's another check-in um uh okay so you have to tell me which of the following statements are true as far as you know now you have to think about a little bit how this these uh relate to um how np and ip or bpp and ip relate to one another okay how are we doing on this okay so we're gonna have to close this pretty soon too do the best you can interesting okay closing up shop last last vote okay one two three there's one more person out there who hasn't voted who voted last time oh well uh all right in fact they're all true um let's see why is np contained with ip contained in ip uh well many of you have seen this already so let's just quickly go through it um [Music] if we just had a deterministic v um you know the uh you can uh maybe it's just that that can be enough of deterministic v i think it's just going to be equivalent but actually just to be doubly sure the deterministic v and the approver just sends a message to the verifier and then checks it that's the way we normally think about a certificate for np uh i don't think it's going to change anything but should double check that if the verifier can still ask questions but i think as long as the verifier is deterministic you're going to get exactly np here um and um now how about bpp well there you don't even need approver because the verifier is already probabilistic so verifier can ignore the approver and this one is a little tricky uh i p containing p space because we haven't covered that so there's no way for you to know that unless you happen to read ahead in the book but it's in fact true um in some ways it's a little bit like the proof that uh np is contained in p space ip is sort of an enhanced version of np and you and there's just a basically a uh a piece based brute force algorithm that goes through the entire tree of possibilities um of the verifier exchange verifier with exchanges with approver and can uh determine that um the verifier is either going to accept for some approver or is going to end up projecting for every approver um so we're not going to prove this statement but something good for you to know anyway they're just a fact uh but we're going to do what the surprising thing in reference to part c is that the containment also goes the other way this is the amazing um uh was is an amazing result that everything in p space you can do with in ip so this is ip is actually turns out to be incredibly powerful gives you everything up in p space you get i p equals p space so that says that any problem that you can solve in p space like any of the a game for example um if you can imagine you know formulating you know checkers or chess as a piece based problem which you know depending upon some details of the rules you can do because you know you have to generalize it to an end by end board but okay let's not quibble um uh then uh um [Music] we don't know which side has a forced win in chess um and even if somebody goes through the effort of going through the game tree um and determines that let's say white has a forced win uh there's no way for them to there's no short certificate we don't know that that problem is not an np but by going through an interactive proof an all-powerful prover could still convince somebody that white had a force you know convince somebody in polynomial time that a white has a forced wind let's say in um in chess again little uh stretching things because this is you know you really need to talk about this as an n by n not an eight by eight but i think in this the spirit is is uh fair so um okay uh so let's continue so wha when we're not going to quite prove that that p base is contained in ip we're going to prove a somewhat weaker statement but very similar um is that uh and historically came first that co-np is contained in ip so not only is np contained in ip but we're going to prove that co np is contained in ip and this actually has most of the i most of the idea for the p space being contained in ip and itself it's just an amazing proof a little easier um okay and th this was done if i'm remembering somebody's asking me when how old is this it's something in the 19 i think late 90s but i'm not i don't remember but maybe early 90s i think it's late 90s when this was shown so it's been a while now um okay so um yeah so in terms of the relationship with cryptography there were two parallel threads um that both independently came up with the notion of an interactive purge system uh i was a little bit personally involved with this in in a way as well but but mainly that the the there was one group in cryptography um working on this and there was another group who was actually coming out of the graph isomorphism world working on it and they they came up with two separate models one involving the private randomness and one involving the public randomness um and it was turned out that was that they're actually equivalent um and uh it's an interesting story but unfortunately we don't have time for it uh so why don't we move on and i'm gonna start showing you how the proof that co-np is contained in ip goes and what we're going to do is work with a problem that's almost like co p complete but um going to be uh well it's going to be this number sat problem we'll see the connection with co-np in a second uh so cohen p so it's supposed to be exactly k satisfying assignments fee comma k is the set of pairs where the formula fee has exactly k satisfying assignments so really that's a problem of counting how many satisfying assignments you have in a formula um so you know for np you have at least one um but i'm i want to know exactly how many uh so the numbers that problem is the pairs formula and the count so um and uh so if we define the count number fee is the number of satisfying assignments of a fee then another way of writing this uh number stat problem is uh the pairs phi k where k is the number of satisfying assignments of fee so we're going to be using this notation number fee a lot so just make sure you get you got that notation this is the number of satisfying assignments of that formula okay and here's a definition i probably should have given you earlier in the term but better late than never um uh so the notion that a language is np hard it's like np complete except without being necessarily being in np so this is just the reduction part a language is np-hard or co-np-hard or p-space hard or any of those other classes that we looked at if every problem in the class is reducible to that language but you don't know whether this that that language is in the class so we just call it np hard instead of mp complete so you could say the language is np complete if it's hard and it's in np okay and so we're going to show that this number set problem is co np hard so everything in cohen p is polynomial time reducible to number set that's easy because what we're going to do is take a co np complete problem which is the unsatisfiability problem the complement of satisfiability and sure that reduces to the numbers that problem and that's easy because a formula is unsatisfiable exactly when it has zero satisfying assignments so if you can tell how many satisfying assignments something has exactly or you can answer the question you know does a formula have exactly a thousand satisfying assignments if you can do that in general then you can solve co and p uh you can solve the unsatisfiability problem by asking with zero satisfying assignments and that allows you to solve anything in cohen p okay so we're gonna just work with this one problem the number set problem and show that that problem's in ip okay let's take a quick break um okay feel free to send me let me see if i can catch up with some of the questions that have been cropping up here so if the approver knows the random choices of the verifier can it can flip the answer to make the verifier reject not sure what that you mean in the context just of the graph isomorphism problem or um something in general i'm not sure you'll have to explain sorry i'll respond with a question mark what else can i answer for you guys so i got a question might if i p equals p space does that mean that iso or the or non-iso might be um in might be p space complete but no that's not known so we're about out of time okay let's continue here [Music] okay so this this is where we're kind of getting going to start to get into the meat of things um [Music] and if you didn't quite understand everything up till now maybe just try to keep your intuition about how do i you know how does a powerful party convince a um a polynomial probabilistic polynomial time party of the number of satisfying assignments exact number not not at least but you won't know exactly the number um of satisfying assignments um so it could be zero for example how do you convince that how do you convince someone that there were zero assignments and and and you know you can have an interaction which does that and that's not obvious at all how you're gonna do that um uh all right so um okay so we're gonna have to introduce some notation which i don't hope this doesn't cause heartburn here uh so let's say again here is the the computational the language we're working with number set and we have a fee where that has m variables x1 to xm now here's our notation i'm going to if i write phi with this free phi of 0 that just means the formula that i get by plugging in 0 for x one and leaving all the rest of the variables uh alone okay so i substitute zero for x one where zero means false and one means true as usual and but that's it's going it's still going to be some other formula but just with that substitution if i write fee 0 1 that means i preset the first two variables to 0 and 1. um if i write fee with a bunch of preset values i'm just setting the first i variables x1 to xi to some values and leaving the other variables as unset so i'm calling the ones that i'm nailing in there as i'm already saying these are the presets okay so this is just converting some formulas into other formulas that have somewhat fewer variables all right now let's recall that number notation the number sign notation number fee is the number of satisfying assignments now if i say number fee of 0 that's the number of satisfying assignments when i've preset x1 to 0. similarly if i preset the first i variables to some values and then i take i want to think how many how many satisfying assignments subject to those prefix presets i write it this way so i'm going to use this notation a lot you have to understand this notation ask if you don't none if you don't get it so another way of writing it i don't know if this is helpful but another way of writing number fee of a1 to a i remember we have m variables all together that means i take the variables which i have not yet preset um and i allow them to range over all possible zeros and ones and i add up the the formulas values for all of those so there's a one every time i satisfy and a zero every time i don't satisfy so i'm adding up all the satisfying assignments subject to these i presets okay so here are two critical facts about this number sign notation first of all if i preset the first i values to something now i can in addition set the next variable either to zero or to one and i get this relationship which is just simply a generalization of the fact that the total number of satisfying assignments of the formula is equal to the number of satisfying assignments when x1 is 0 plus the number of satisfying assignments when x1 is 1. right they together have to add up to the total number because x1 is going to be the either zero or one so that's fact number one fact number two is that if i preset everything all of the variables so there are no variables left then the number of satisfying assignments subject to that preset of everything is just whether or not i've satisfied the formula which is the value of the formula on that those presets okay both two simple facts but it's going to be critical in the protocol i'm about to describe questions on this i think actually i do have a question for you so let's just see what do you think just to check your understanding okay got about 80 getting this um i'm not sure that's good but uh all right almost done closing okay okay so uh yes a is the correct answer you know if if there are nine satisfying assignments all together and there are six satisfying assignments where the first variable is set to zero then there's only three satisfying assignments with the first variable set to one because nine has got to be equal to six plus three that's actually this fact number one it's not going to be 15 this is not true either so it's just a good okay okay so let's try to with with that knowledge let's try to see how we can put number sat in ip so this is not going to quite work but it's really going to set us up to do this to finish this next time um so you might immediately see where this is going wrong but you'll you'll have to put up with it because um the setup is what's important um okay so understand now here's the here's the the setup we have um the input is a formula and a number where that number is supposed to be the number of satisfying assignments you know it could be wrong and in which case we're not in the language um but if it's right you're in the language so the approver is supposed to convince the verifier that it's correct if it is correct and it's not gonna it's gonna fail no matter what it tries to do if it's uh not correct so this is the approver is going to send first of all um so so the proof is going to send a claim about the number of satisfying assignments going to send when i say this value here this is what the prover if it's honest is going to send the right value of course the verifier does not know if the approver is honest but i'm describing how the honest approver is going to operate and we'll have to understand what happens if the prover tries to cheat so the proof is going to send the honest proof is going to send the number of satisfying assignments all together and the prover verifier just makes sure that that matches up with the input if it doesn't match up with the input uh the verifier is just gonna you know you know the the verifier is gonna not be convinced that the input is um in the language so it's gonna just uh reject at that point um okay uh then now the verifier says okay that was very good that you sent me this how do i know that's right so what the verif prover is going to do to try to convince the verifier that this value was correct is uh unravel that by one level by say well you know there were nine satisfying assignments all together uh six of them were when the when x one is zero and three of them were when x one is one to verify what does the verify have to check that these add up correctly when i preset x1 to zero and to one it had better add up to the total number of satisfying assignments if that works out the verifier is happy is still being it's still consistent with being convinced that this k was the right value um so um the next step is well the verifier says well how do i know those two values are correct the prover says okay well i'm going to prove send un unravel them one level further then here's a number of satisfying assignments when the next variable is set to both possibilities for each of the possibilities of the first variable now if you're understanding me about what the prover is sending you should start to be getting a little nervous because something is i mean this is going to be correct but it's going to start it looks like it's starting to blow up in terms of the number of amount of work that's involved and that's actually a problem but let's bear with that for the moment let's just worry about correctness not about complexity for the moment so the proof is going to now send the number of satisfying assignments for each of those four possible ways of presetting the first two variables and the verifier is going to check that that was consistent with the information the prover sent in the previous round right by again checking this identity here so then the proof is going to continue on doing that until it's got it's done that through m rounds where m is the number of variables so at this point the proof is going to send all possible ways of presetting all of the variables so that now there's two to the m possibilities here again this is hopelessly not allowed but okay ignoring that the proofer's got to use this at the nth round to check what happens at the at the previous round so that's when they were m minus one values sent because each one has one more uh uh you're extending the presets by one so which using this to check that the previous round was the values were correct so it's looking for um you know the m minus one presets have to add up correctly um you know in terms of the m the presets of m uh values uh for each of those ways of uh doing those uh m minus one presets and so now the provers send all of those two to the m counts which are by the way ones and zeros because at this point we have preset all of the values of the variables and so there's only one possible assignment at most that there can be and now the verifier the approver is done the verifier is going to check by itself that these values make sense that these values are correct so it's going to do that by looking back at the formula so far up at this point the verifier has not been looking at the formula it's just been checking the internal consistency of the provers messages with each other but now at the end the verifier is going to take the messages these these values that the approver sent for each of the two to the m presets and see if it matches up with what the formula would do remember that was the other sort of the base case of the the end the fact number two um from the uh slider to ago make sure that these agree okay and now the verifier says well okay if everything has checked out and all of these are are in agreement then the verifier is going to be convinced that um that uh fee had k satisfying assignments but if anywhere along the way one of these checks fails the approver is not the verifier is not going to be convinced and is going to reject okay so in a sense this is kind of dopey you know you know we've just i'm just kind of giving you a complicated way of just counting up one by one each of the satisfying assignments of the formula and seeing if that matches k but nevertheless this way of looking at it is gonna be uh help us to understand um the way to fix this so so bear with me for another minute on this one so another way of looking at this which i think is is particularly useful is to think of what happens well okay we'll get there in a second i want to look at what happens if k was wrong but before i do that let's look at the i'm going to give a kind of a graphic a graphical view of um the information that the prover sends and the and the verifiers actions in this protocol so the the values that the prover sending are going to be in yellow so the and the information that the verifier has or checks is going to be in white so the the verifier has the k the the input value which is supposed to be the number of satisfying assignments and the prover sends some value and the verifier checks that this value which is supposed to be the number of satisfying assignments corresponds with k so that's one of the checks it does then the approver is going to send kind of take to justify this value it um sends the number of satisfying assignments when you have x1 set to zero or set to one the verify adds those up to give you and it's supposed to equal the total number of satisfying assignments and so this is if you understood this protocol this is just i'm writing it out in a sort of a simplified way perhaps okay and so um keeps checking that these things add up correctly until you get down to setting all m values in all two to the impossible ways and now the verifier is going to then check to make sure that that equals the what the formula would say um okay okay so now let's what happens if k was the wrong value it did not agree with the number of satisfying assignments um and what does what happens now um could the prover what happens if the approver tries to make the verifier accept anyway so um so the only thing the prover can do at the very first step would be to lie um about you know if the approver sends the if k is wrong and the proof approver sends the correct value for the the the total count the verifier is going to reject so i'm trying to see is it could the approver try to make the verifier accept what what happens so the prover has to lie here and i'm going to indicate that by saying the approver is sending in the wrong value for um uh the the total count well if the proof is going to lie here um then just like you know if you you know you have a child who um tells a lie and then you start you know as the parent you start asking questions to try to see if the story is consistent one lie is going to lead to another lie um and that's what happens here if the uh in order to justify this lie um the proof is going to have to a lie in one or perhaps both but at least one of these two values because you can't have the two correct values adding up to the incorrect value because you have to think about what's going on here so if this is a lie that's going to force a lie at one of one side of the other one level down which is then going to force a lie to propagate down and so there's a lie at every stage is going to force a lie at least in one one place or another to propagate all the way down to the bottom and then at the bottom the verifier will see that the check doesn't work as when it turns when it tries to connect it up with the formula itself and the form verifier will reject okay so just a way of looking at this um if the form if the value if the input was not in the language um so uh but the problem is that as i said that this is exponential so how are we going to fix that so just looking ahead to what we're going to do on tuesday okay let's see if there's any questions here first of all uh okay yes i got a question should this be uh should this be a minus i i purposely made this bracket only go not include the very last zero yeah there's a total of m zeroes here all together but the i left out the last zero that's why i said n minus one maybe it would have been better to say m okay so and you got another interesting question here why can't we reject right away if k is wrong um uh well the verifier is probabilistic polynomial time how does the verifier know if k is wrong um so it means or or or right so what we're trying to do is something like you know like np where we have a certificate but now we have this kind of interactive certificate in the form of disprover maybe that's another way to look at it um where if you're in the language there should be some way for the prover to convince to get make you accept but if you're not not in the language there should be no way for the proverb to make you accept um uh so the verifier just can't reject right away because there's no way to tell how does the verifier know it's going to start rejecting things when it shouldn't if it's just going to be rejecting willy-nilly here um okay how does the verifier need to determine if the prover is internally consistent instead of just asking so why does the verifier need to determine if the approver is internally consistent instead of just asking the questions in step n plus one yeah so maybe that's because it looks like all of the work is happening at the very end um but i'm really presenting this to you as a preparation for what we're going to do on tuesday um so it's important to think about the connection from each step to the next each step is going to be justified by what happens at the next step until we get to the very end so you have to just understand it for what it is don't try to make it more efficient yeah i realize this is kind of dumb um good point we're not using the probabilism here um and moreover we're not really even using the interaction here the prover is doing all the sending the verifier is just accepting at the end yeah this is we're not using the power and we're getting a weaker result so let's move on before we run out of time here so how do we gonna fix this so the problem is is blowing up each to justify each stage where each value we're um needing to present two values which add up to it um and that's uh good leading to a blow up now it would be nice if we can do something where each value was supported by just a single value at the next level so we know here's an idea well you know in order to understand to see that uh that that this total count is correct why don't we just pick at random either zero or one and only follow that one down well the problem with doing that is because the the the sequence of lies is could be just a single path through this uh tree and the chances you're going to find that path down to a contradiction at the bottom is very low if you're just doing it at random um so just randomly picking zeros and ones as as the as the one you're going to justify used to justify the previous value is not going to be good enough so what but this is what we're going to do however the values that we're going to pick for um for these random um inputs are not going to be boolean values we're going to pick non-boolean assignments to the variables which again just as with the branching program case didn't make any sense on the surface of it we're going to have to make it make sense and we'll have to see how to do that on tues in tuesday's lecture okay so that's kind of the setup okay um yeah so in a similar question why is this any different from just non-deterministically guessing the assignments it's because of this we're really setting the stage okay so what we did today was we introduced them the the model and defined the complexity class we can we did show this one in its full glory we showed that non-iso is an ip really worth understanding this uh protocol here making sure you you're comfortable with that and and also the the model itself and so for tuesday's lecture we're going to finish this up uh well we started showing that number set as an ip which is what we need to to do to prove co-np is an ip and we'll finish that next time which will be our last time okay so that's it for today i'll stick around for questions so a good question here why can't um v just reject if some of the checks are incorrect yes v could as soon as there's a check that fails we can just reject at that stage i'm just trying to argue that at some point along the way if the input is not in the language there's going to be a check that fails so i mean i said reject at the end but yeah i mean you could have rejected at any point along the way um okay um someone's asking for what what role did i play uh so i did my my own personal role in this i were to which was twofold first of all i i came up with the idea uh of well not the idea i came up with the name interactive proof uh i remember when sylvia mccauley was explaining this to me in my apartment many many years ago he had a kind of a little bit complicated and i don't even remember what the protocol was for it was not for something simple was something involving uh prime numbers and i said oh that's a kind of an interactive proof and it's and it it stuck from that point on so um that was one thing but the other thing in terms of more mathematically i my role was so uh shafi goldwater and i approved the equivalence of the two models the public coin and the private coin version um so that that was uh my uh my role in this back and when this was all first coming out approved it approved it on an airplane on the way to a conference somewhere so i think we're going to notice any other questions i think we'll head out take care everybody see one see you on tuesday bye bye you 3 00:00:26,560 --> 00:00:27,750 hi everybody um glad to have you all back for our next to last installment of theory of computation today um we are going to embark on the very last big topic for the semester and that is in some ways going to be following on what we started a couple of lectures back when we looked at probabilistic turing machines and probabilistic computation and its associated class bpp now what we're going to discuss is in some sense a probabilistic version of np and that's going to be a complexity class called ip which stands for interactive proof systems um and so we're going to present that model and look at a couple of examples uh i would just like to say at the beginning that the this model is a very important one it's really uh has been the starting point for a great deal of research in complexity theory so we're just really going to be touching on it but there's a lot more that people have pursued with this model and it's also a connection in to the cryptography field which also makes use of the interactive proof system model in fact some of the genesis of that model comes out of cryptography where you're having uh multiple parties either communicating or in some ways interacting uh to achieve certain uh goals of communication or signing or passwords or or or what have you so uh this is a both a an applied area and also one that has a lot of very interesting theory uh um associated to it so with that one we we're going to jump in um and uh start out by uh making myself smaller and just do a an introduction i'm going to introduce the model or the the concept of an interactive proof with an example and that example involves the graph isomorphism problem that's the problem of testing whether two graphs are isomorphic what we mean by two graphs being isomorphic is that they're really just the same graph with one of them perhaps being re-labeled or permuted so that they may look superficially different they may appear with a different sequence of labels or the nodes are appearing in a different order but except for that it's really just the same graph so i'm kind of illustrating that here if you can see those two graphs here which look different from each other both on eight notes they are in fact the same graph as i can illustrate by a little animation which will convert this one into that one so uh the um [Music] so the two graphs um these graphs being the same um they're we call that isomorphic so these are graphs g and h and they're really the same graph so we we um call them isomorphic graphs and we are have an associated computational problem called iso which is given a pair of graphs we'd like to know are they isomorphic or not so the iso is the collection of pairs of graphs which are isomorphic and it's easy to see that this problem is an np problem because all you need to do in order to see or to give a certificate that the two graphs are isomorphic to each other is tell you it's just to say which nodes in the one graph correspond to which other nodes in the other graph um and then you all you need to check is that the edge relationships are consistent with that mapping or that isomorphism as it's called um so it's easy to see that the iso problem is an np and if you're not getting that make sure you uh understand because this is the whole first part of the lecture will be lost if you don't understand this iso problem now the question of whether you can test two graphs being isomorphic in polynomial time is not clear and in fact that's an unsolved problem to this day uh and it's a problem that has generated an enormous literature uh there are hundreds of papers on the graph isomorphism problem as it's called um to try to uh resolve um you know uh to try to see if one can find a polynomial time algorithm and in fact it was a very big result just in the last 10 years where there was a sub-exponential algorithm given so that was more than faster than the brute force search approach but didn't get it all the way down to polynomial now why is there so much attention just to this one particular np problem it's because it's not known whether the graph isomorphism problem is np-complete iso is not known to be an np-complete problem and that puts it into a very very small class of problems in np which are no not known to be either in p or np-complete it's kind of a curiosity that for np problems almost all of them have ended up being in one side or the other and in fact um it's a uh um in fact the the um i i think it's the only problem that just involves graphs that's not known to be either in p or an np um so so i got a question here what would be in between exponential and polynomial for example i don't remember what the the bound is but it's it's something in the uh in the range of n to the log n uh time complexity for the graphite some more i might be getting that wrong um i don't remember exactly what the bound is but um that's significantly better than two to the n or some some exponential amount of time but it it's more than enter any constant so it's more than any polynomial time so um another question uh of the same sort is whether the complementary problem is in np or whether whether iso is in co-np or let's let's talk about it in terms of the complement whether the complement of iso which i'll refer to as the non-iso problem whether that's known to be in np so that's also not known um in other words if i give you two graphs and i ask you to show that they're not isomorphic suppose they aren't isomorphic and you go through the effort of you know determining that by a brute force search and now you want to prove that they're not isomorphic well that's not it's not known to be an np either so there's no known short certificate certificate of two graphs not being isomorphic we don't know how to do that either but there's something that's very interesting nevertheless um and it has to do with the ability to for one party to prove to another that graphs are either isomorphic or not isomorphic so if you're just having like a prover we haven't really been necessarily formulating that this way so much in this class but this is a completely equivalent way of formulating the notion of np whether you have a polynomial time verifier an approver who can produce certificates say it's a powerful prover so um if you have a problem that's in np approver can convince a polynomial time verifier that uh strings are in the language if in fact they are so in the case of the iso problem a prover can convince a polynomial time verifier the graphs are isomorphic just by exhibiting the isomorphism now for the non-isomorphism case we don't know that that problem's in np but it's still possible for approver to convince a verifier that graphs are not isomorphic if you change the rules of the game slightly um so even though the non-iso problem is not known to be an np approver can still convince a polynomial time verifier that graphs are not isomorphic assuming they they are in fact not isomorphic um provided that um the prover and the verifier can interact with one another so the verifier can ask questions of the prover and the verifier gets to be probabilistic so that's in this in that's in sense in which i mean that this notion is a kind of a probabilistic version of np okay so um let me show you how that's done so before we jump in to the uh to the method for approver to um show a verifier that graphs are not isomorphic um let me let let's try to get a little clearer on the model so i'm going to first show it to you informally and then we'll look at it formally okay so in in interactive proofs there are two parties um and i'm going to think about them as one of them is going to be the professor okay so the professor is going to play the role of the verifier in a sense but it's like that the one who checks um and uh the professor being kind of old and tired he's been teaching too long maybe can only operate in probabilistic polynomial time so the professor if wants to tell whether two graphs are isomorphic or not probabilistic polynomial time doesn't seem to be enough to tell whether two graphs are isomorphic or not because it seems to be a more than polynomial problem however the professor has um help it has an army of graduate students and the graduate students they're not limited uh in the same way the professor is the graduate students are young they are energetic they can stay up all night they know how to code so the graduate students have unlimited computational ability so that we're going to think of the graduate students playing the role of the approver because they're not they're not limited in their capabilities we'll assume the professor on the other hand is limited so the professor wants to know if the two graphs are isomorphic let's say whatever they are um can't do it by himself so he's going to ask his students to figure out the answer and report back now there's only one problem the professor knows that students uh well in the old days they'd like to party i guess these days they like to play on uh play computer games a lot and so they're not really that eager to spend all their time figuring out whether graphs are isomorphic so he's worried that the the students will just come up with some answer and figure that he won't be able to tell the difference so the professor does not trust the students it's not enough to he for the professor to give the problem to the students and just take any answer that they're going to give the professor wants wants to be convinced okay so um now how could the students convince the professor of the answer that they've really done the work and figured out whether the graphs are isomorphic or not well if the graphs are isomorphic if it turns out that the graphs were isomorphic and the students figure that out then life is good because what are they going to do to convince the professor they're going to hand over the isomorphism and show yeah i mean they are you know those graphs really are isomorphic and here's how the correspondence works professor can check oh yeah i i now not now i'm convinced but suppose the graphs were not isomorphic what are we going to do then um the students have figured out where after night else the professor wants wants to be convinced oh no what are we going to do well in fact we're going to engage the the professor and the students are going to engage in the following protocol dialogue what's going to happen is now you have to make sure you're you're this is critical to follow to understand this little part of the story here because it's really going to set the pattern for everything in today's and tomorrow and to today's lecture and the and the next lecture okay so we're going to engage in a following interaction between the students and the professor which is going to enable the students to convince the professor that the two graphs really are not isomorphic so how is that going to work this is a beautiful little uh thing by the way so the professor is going to take the two graphs and pick one of them at random because the two graphs g and h um let's say they're not they really are not isomorphic the professor doesn't know that for sure that's what the students claim the professor really wants to not be convinced that the students are right um so the professor's gonna pick one of the two at random randomly permute that uh that choice the one that he picked and hand it over to the students say okay here is one of those two graphs randomly scrambled then i'm going to ask the students which one did i pick okay now if the graphs were really not isomorphic the students can check whether that randomly scrambled graph is isomorphic to either g or to h it's going to be isomorphic to one or the other and then they students can figure it out and they say oh you picked g or no you picked h as the case may be the students can figure that out but if the graphs were isomorphic then that scrambled version of g or h could equally well have come from either of them and the students would have no way of knowing which one the professor picked so that there's nothing they could do which would be better than guessing so if we do that a bunch of times the professor picks at random sometimes secretly of course the picks the grip picks either g or picks h and the students get it right every time either the students are really doing the work and the graphs are really not isomorphic or the students are just incredibly lucky they're managing to guess right let's say a hundred times so how the the the professor randomly and secretly picks grh uses this uses its probabilism flips a coin just a two-sided coin and says okay sometimes if we're going to do g sometimes they're going to do h just completely at random picks one or the other and then with some more randomness gets finds a random permutation of the one that he picked and then sends that over to the students and say which one did it come from um i'm not sure okay so let's pause here let's let's make sure we all understand this because this is really important um so i'm getting a question here how do we i'm not sure what your question is um okay so let me just say yeah the professor's going to play the role the verifier the graduate students play they're all approver that's coming but i really want to understand this protocol here okay so how is the professor picking the graph skin if you're okay i don't you know picking the graphs at random you have just two graphs they're in part of the input uh the both the students and the professor can see the graphs and the professors are just picking one of them at random using a coin so i'm not sure i understand the question there could p and v engage in a protocol where the secretary is on the prover side instead the question of revealing the isomorphism i there is no why so i'm not sure i understand this question either um maybe we'll make this clear you know if for for this little illustration the professor doesn't know the graphs could be isomorphic or they could be not isomorphic and so uh the professor wants to be convinced either way whatever the students whatever answer the students come up with we're going to shift this into a problem about a um deciding a language next but right now i'm just trying to give a sense of the how the model works i want to move from this informal model and now i'm going to formalize that in terms of model which will be deciding a language okay so so the interactive proof system model we have two interacting parties a verifier which is probabilistic polynomial time playing played by the professor in the previous slide and the prover which is unlimited computational power played by the students in the previous slide both of them get to see the input which in the previous case well it could be for example the pair of graphs they exchange a polynomial number of polynomial size messages so the whole exchange including the verifier's own computation is going to be polynomial the only thing that's not not not included within the computational cost is the prover's work which is unlimited um after that the verifier after the interaction the verifier will accept or reject and we're going to define the probability that the verifier together with a particular approver ends up accepting as you look over the different possible coin tosses of the verifier which could lead to different behavior on the part of the verifier and therefore different behavior on the part of the approver so over all the different possibility possibilities for the verifiers computation we're going to look at the probability that the verifier with this particular approver ends up accepting and i've written it this way this is the probability of the verifier interacting with the prover accepts the input is just simply that um and so we're going to work through an example we're going to work through the previous example more precisely in a second the class ip for interactive proofs stands for it's the class of languages such that for some verifier and approver um for strings in the language the prover makes the verifier accept with high probability and here's the interesting part for strings not in the language the prover makes it except with low probability but every there's no prover which can make it except with high probability so there's no way to cheat if you think about it in the case of the graphic non-isomorphism there's nothing you know if if the graphs were really isomorphic and the students were trying to in a devious way prove through that protocol that they're not isomorphic they would fail because there's nothing they can do if the graphs were isomorphic then um when the verifier the the professor picks one or the other at random um and scrambles it the students would have no way of telling which one the professor did so no matter what kind of scheme they try to come up with they're going to be out of luck so it's no mat for any strategy for strings that are not in the language for any s any prover calling that p with a tilde to stand for a devious or crooked prover for any uh possibly crooked prover even that with working with the verifier is still going to end up accepting with low probability so strings in the language there's going to be an honest prover who just follows the protocol in the correct way which makes the verifier accept with high probability for strings not in the language every prover is going to fail to make it accept with high probability um okay so that i mean the way i like to think about it is that p tilde is a possibly crooked proverb which is trying to make the verifier accept when it shouldn't because the string is not in the language it's like you know it's like even you can think of this in the case of um satisfiability um you know you a crooked prover might try to convince of the verifier that the formula is satisfiable when it isn't by by somehow trying to produce a satisfying assignment but that's going to be impossible there's nothing any strategy can possibly work when the formula is not satisfiable if that's what the verifier is going to check it's going to be looking for that satisfying assignment okay and by the way this is we're not going to prove this but it's really going to be proved in the same way you can make that one third error that could that occurs here something very tiny by the same kind of repetition argument okay so let's see um so why can't the prover in the first case be crooked um the prover in the first case would could be crooked but that's not going to serve the purposes um you know what what we want to show um you think about it like we think about np for strings in the language there exists a certificate there is a proof that you're in the language so if somebody is going to not produce the proof that's irrelevant the question is if you look at the best possible case the best possible prover um you know who's going to be able we're asking does there exist a way to convince the verifier that the um string is in the language so it doesn't matter that there might be some other uh silly way that doesn't work we're just looking at the best possible way so the best possible way when you're in the language is going to end up with a verifier having high probability when you're not in the language the best possible way is still going to end up with low probability when when i talk about best possible i'm trying to maximize the probability that the verifier is going to end up accepting let's continue um not sure as clear as i would like but um maybe again we're going to we're going to stick with that example because this is a very uh helpful example and to try to understand the setup and uh so we're gonna i'm gonna revisit that previous example about non-isomorphism but now in the context of this thinking about as a language so we're going to take this non-isomorphism um uh yeah we're going to take the non-isomorphism problem and show that it's an ip so there's going to be a verifier together with approver which are going to make the verifier accept with high probability for strings in the language namely graphs not ice being isomorphic and nothing there's going to be no way to make the verifier except with high probability for strings out of the language therefore that's when the graphs are isomorphic okay um so the protocol is just gonna we're gonna repeat the following thing twice you know i said in the previous case do it a hundred times just to help us think about it but actually twice is gonna be enough to get the bound we need so the verifier is going to operate like this in terms of this is the verifiers in first communicating sending messages to the approver it's going to randomly choose grh just like what the professor did last time randomly permute the result to get a new graph k which was going to be which was which is isomorphic either to grh depending upon the choice the verifier made and then send that graph k now the provers turn is going to respond by the proof is going to compare k with the two one of the both of the original graphs it's got to be isomorphic to one or the other and it's going to report back which one just going to say well you pick g no or you picked h because the prover with its unlimited capabilities can determine that um and then v accepts if the approval was right both times um and if the approval was ever not right the verify says oh something's fishy here because we know that the prover has unlimited capability so could get it right if you know um if there was if this was an honest approver um and so um if uh if it's not getting it right then the verify is going to reject so if the graphs are not isomorphic the proofer can tell which one it picked randomly so therefore if the graphs are not isomorphic the verifier with that that honest prover will accept with probability one because that honest proof is always going to get the right answer which is at least two-thirds is what the bound we need uh we don't care about the space used in answer to a question um if we were not in the language so g and i h are not isomorphic then there's nothing any crooked pervert could possibly do because it gets a graph can't tell there's no way to tell whether it came from g or came from h um so that crooked proofer would have all it could the best thing you could do is guess sort of a 50 chance advancing correctly each time and only a 25 chance for doing it twice and that's why i did it twice in order to get that error um uh to be small so there's only a 25 chance of the approver getting lucky so that would be an error case if the proof of just by chance picked the right answer twice even though the graphs were isomorphic so therefore for the isomorphic case the verifier interacting with any prover is going to accept that input with at most one quarter 25 of the time which is less than a third so those are that's just to achieve that bound okay so let's let's answer some questions first and then i'll try to um uh i'll ask you you understand this so this it's i think it's worth trying to understand this model um of this interactive proof system it's a little little slippery i i realize but um if you just hold your hold on to your intuition of the prover trying to convince you know of uh you know a powerful prover trying to convince a limited verifier um of some string being in a language uh you want the proven to be able to succeed when the string is in the language but fail when the string is not in the language yes we are going to somebody's asking if the prover is identifying grh by brute force yes the prover is going to use its unlimited capabilities to determine given k whether it came from g or h the um the computational cost of the approver is irrelevant for this it's just like when we think about a certificate um you know for satisfiability we don't talk about the cost of finding that certificate uh for np for ip again we don't talk about the cost of the prover running so somebody's asking does the crooked per prover answer just randomly or does uh can the cr quick approver has have a strategy the crooked perimeter can have a strategy now we're not we're assuming the crooked approver is devious but it's still going to fail okay um let's do the check-in suppose we change the model so that the prover can watch the verifier picking its random choices so uh the verifier cannot act in secret anymore but the the prover can watch the verifier now let's suppose we had the same protocol that i just described what language do we end up with is it the same language different language and what is that language okay so i want to hopefully um let's let's it'll give me some sense of how well you're following me by how well this uh this goes yeah someone's asking about how this connects up for example with np so we're going to look at that also in a second okay so this is reassuring that most of you i think are on the right track at least for this check-in uh do we assume p uses this access to guess right what access p is not really guessing the p is tr is actually i don't think a p is non-deterministic or anything like that p is actually trying to get the right answer and using its computational ability to do that if it's possible may not be possible then there's nothing you can do okay so let's uh end this are you all in two seconds left please vote vote now or never okay ending um yeah so c is the correct answer here if the prover can watch what the verify is doing the prover can see what graph the verifier picked right from the beginning and so the prover without having to do any work can say you know it proves looks over the verifier's shoulder and says oh you picked g and now you're randomly permitting it but i don't care about that i just i know you pick g uh so uh the proof is going to respond back uh g even if the graphs were isomorphic the proof is going to be able to get the right answer kind of interestingly um uh you can make a you can change the protocol somewhat um to make it uh that even if the prover has access to the verifier's randomness you can still achieve this but not with the same protocol um so that's uh that's a separate question um okay so let's move on here i want to get too bogged down okay here's another check-in um uh okay so you have to tell me which of the following statements are true as far as you know now you have to think about a little bit how this these uh relate to um how np and ip or bpp and ip relate to one another okay how are we doing on this okay so we're gonna have to close this pretty soon too do the best you can interesting okay closing up shop last last vote okay one two three there's one more person out there who hasn't voted who voted last time oh well uh all right in fact they're all true um let's see why is np contained with ip contained in ip uh well many of you have seen this already so let's just quickly go through it um [Music] if we just had a deterministic v um you know the uh you can uh maybe it's just that that can be enough of deterministic v i think it's just going to be equivalent but actually just to be doubly sure the deterministic v and the approver just sends a message to the verifier and then checks it that's the way we normally think about a certificate for np uh i don't think it's going to change anything but should double check that if the verifier can still ask questions but i think as long as the verifier is deterministic you're going to get exactly np here um and um now how about bpp well there you don't even need approver because the verifier is already probabilistic so verifier can ignore the approver and this one is a little tricky uh i p containing p space because we haven't covered that so there's no way for you to know that unless you happen to read ahead in the book but it's in fact true um in some ways it's a little bit like the proof that uh np is contained in p space ip is sort of an enhanced version of np and you and there's just a basically a uh a piece based brute force algorithm that goes through the entire tree of possibilities um of the verifier exchange verifier with exchanges with approver and can uh determine that um the verifier is either going to accept for some approver or is going to end up projecting for every approver um so we're not going to prove this statement but something good for you to know anyway they're just a fact uh but we're going to do what the surprising thing in reference to part c is that the containment also goes the other way this is the amazing um uh was is an amazing result that everything in p space you can do with in ip so this is ip is actually turns out to be incredibly powerful gives you everything up in p space you get i p equals p space so that says that any problem that you can solve in p space like any of the a game for example um if you can imagine you know formulating you know checkers or chess as a piece based problem which you know depending upon some details of the rules you can do because you know you have to generalize it to an end by end board but okay let's not quibble um uh then uh um [Music] we don't know which side has a forced win in chess um and even if somebody goes through the effort of going through the game tree um and determines that let's say white has a forced win uh there's no way for them to there's no short certificate we don't know that that problem is not an np but by going through an interactive proof an all-powerful prover could still convince somebody that white had a force you know convince somebody in polynomial time that a white has a forced wind let's say in um in chess again little uh stretching things because this is you know you really need to talk about this as an n by n not an eight by eight but i think in this the spirit is is uh fair so um okay uh so let's continue so wha when we're not going to quite prove that that p base is contained in ip we're going to prove a somewhat weaker statement but very similar um is that uh and historically came first that co-np is contained in ip so not only is np contained in ip but we're going to prove that co np is contained in ip and this actually has most of the i most of the idea for the p space being contained in ip and itself it's just an amazing proof a little easier um okay and th this was done if i'm remembering somebody's asking me when how old is this it's something in the 19 i think late 90s but i'm not i don't remember but maybe early 90s i think it's late 90s when this was shown so it's been a while now um okay so um yeah so in terms of the relationship with cryptography there were two parallel threads um that both independently came up with the notion of an interactive purge system uh i was a little bit personally involved with this in in a way as well but but mainly that the the there was one group in cryptography um working on this and there was another group who was actually coming out of the graph isomorphism world working on it and they they came up with two separate models one involving the private randomness and one involving the public randomness um and it was turned out that was that they're actually equivalent um and uh it's an interesting story but unfortunately we don't have time for it uh so why don't we move on and i'm gonna start showing you how the proof that co-np is contained in ip goes and what we're going to do is work with a problem that's almost like co p complete but um going to be uh well it's going to be this number sat problem we'll see the connection with co-np in a second uh so cohen p so it's supposed to be exactly k satisfying assignments fee comma k is the set of pairs where the formula fee has exactly k satisfying assignments so really that's a problem of counting how many satisfying assignments you have in a formula um so you know for np you have at least one um but i'm i want to know exactly how many uh so the numbers that problem is the pairs formula and the count so um and uh so if we define the count number fee is the number of satisfying assignments of a fee then another way of writing this uh number stat problem is uh the pairs phi k where k is the number of satisfying assignments of fee so we're going to be using this notation number fee a lot so just make sure you get you got that notation this is the number of satisfying assignments of that formula okay and here's a definition i probably should have given you earlier in the term but better late than never um uh so the notion that a language is np hard it's like np complete except without being necessarily being in np so this is just the reduction part a language is np-hard or co-np-hard or p-space hard or any of those other classes that we looked at if every problem in the class is reducible to that language but you don't know whether this that that language is in the class so we just call it np hard instead of mp complete so you could say the language is np complete if it's hard and it's in np okay and so we're going to show that this number set problem is co np hard so everything in cohen p is polynomial time reducible to number set that's easy because what we're going to do is take a co np complete problem which is the unsatisfiability problem the complement of satisfiability and sure that reduces to the numbers that problem and that's easy because a formula is unsatisfiable exactly when it has zero satisfying assignments so if you can tell how many satisfying assignments something has exactly or you can answer the question you know does a formula have exactly a thousand satisfying assignments if you can do that in general then you can solve co and p uh you can solve the unsatisfiability problem by asking with zero satisfying assignments and that allows you to solve anything in cohen p okay so we're gonna just work with this one problem the number set problem and show that that problem's in ip okay let's take a quick break um okay feel free to send me let me see if i can catch up with some of the questions that have been cropping up here so if the approver knows the random choices of the verifier can it can flip the answer to make the verifier reject not sure what that you mean in the context just of the graph isomorphism problem or um something in general i'm not sure you'll have to explain sorry i'll respond with a question mark what else can i answer for you guys so i got a question might if i p equals p space does that mean that iso or the or non-iso might be um in might be p space complete but no that's not known so we're about out of time okay let's continue here [Music] okay so this this is where we're kind of getting going to start to get into the meat of things um [Music] and if you didn't quite understand everything up till now maybe just try to keep your intuition about how do i you know how does a powerful party convince a um a polynomial probabilistic polynomial time party of the number of satisfying assignments exact number not not at least but you won't know exactly the number um of satisfying assignments um so it could be zero for example how do you convince that how do you convince someone that there were zero assignments and and and you know you can have an interaction which does that and that's not obvious at all how you're gonna do that um uh all right so um okay so we're gonna have to introduce some notation which i don't hope this doesn't cause heartburn here uh so let's say again here is the the computational the language we're working with number set and we have a fee where that has m variables x1 to xm now here's our notation i'm going to if i write phi with this free phi of 0 that just means the formula that i get by plugging in 0 for x one and leaving all the rest of the variables uh alone okay so i substitute zero for x one where zero means false and one means true as usual and but that's it's going it's still going to be some other formula but just with that substitution if i write fee 0 1 that means i preset the first two variables to 0 and 1. um if i write fee with a bunch of preset values i'm just setting the first i variables x1 to xi to some values and leaving the other variables as unset so i'm calling the ones that i'm nailing in there as i'm already saying these are the presets okay so this is just converting some formulas into other formulas that have somewhat fewer variables all right now let's recall that number notation the number sign notation number fee is the number of satisfying assignments now if i say number fee of 0 that's the number of satisfying assignments when i've preset x1 to 0. similarly if i preset the first i variables to some values and then i take i want to think how many how many satisfying assignments subject to those prefix presets i write it this way so i'm going to use this notation a lot you have to understand this notation ask if you don't none if you don't get it so another way of writing it i don't know if this is helpful but another way of writing number fee of a1 to a i remember we have m variables all together that means i take the variables which i have not yet preset um and i allow them to range over all possible zeros and ones and i add up the the formulas values for all of those so there's a one every time i satisfy and a zero every time i don't satisfy so i'm adding up all the satisfying assignments subject to these i presets okay so here are two critical facts about this number sign notation first of all if i preset the first i values to something now i can in addition set the next variable either to zero or to one and i get this relationship which is just simply a generalization of the fact that the total number of satisfying assignments of the formula is equal to the number of satisfying assignments when x1 is 0 plus the number of satisfying assignments when x1 is 1. right they together have to add up to the total number because x1 is going to be the either zero or one so that's fact number one fact number two is that if i preset everything all of the variables so there are no variables left then the number of satisfying assignments subject to that preset of everything is just whether or not i've satisfied the formula which is the value of the formula on that those presets okay both two simple facts but it's going to be critical in the protocol i'm about to describe questions on this i think actually i do have a question for you so let's just see what do you think just to check your understanding okay got about 80 getting this um i'm not sure that's good but uh all right almost done closing okay okay so uh yes a is the correct answer you know if if there are nine satisfying assignments all together and there are six satisfying assignments where the first variable is set to zero then there's only three satisfying assignments with the first variable set to one because nine has got to be equal to six plus three that's actually this fact number one it's not going to be 15 this is not true either so it's just a good okay okay so let's try to with with that knowledge let's try to see how we can put number sat in ip so this is not going to quite work but it's really going to set us up to do this to finish this next time um so you might immediately see where this is going wrong but you'll you'll have to put up with it because um the setup is what's important um okay so understand now here's the here's the the setup we have um the input is a formula and a number where that number is supposed to be the number of satisfying assignments you know it could be wrong and in which case we're not in the language um but if it's right you're in the language so the approver is supposed to convince the verifier that it's correct if it is correct and it's not gonna it's gonna fail no matter what it tries to do if it's uh not correct so this is the approver is going to send first of all um so so the proof is going to send a claim about the number of satisfying assignments going to send when i say this value here this is what the prover if it's honest is going to send the right value of course the verifier does not know if the approver is honest but i'm describing how the honest approver is going to operate and we'll have to understand what happens if the prover tries to cheat so the proof is going to send the honest proof is going to send the number of satisfying assignments all together and the prover verifier just makes sure that that matches up with the input if it doesn't match up with the input uh the verifier is just gonna you know you know the the verifier is gonna not be convinced that the input is um in the language so it's gonna just uh reject at that point um okay uh then now the verifier says okay that was very good that you sent me this how do i know that's right so what the verif prover is going to do to try to convince the verifier that this value was correct is uh unravel that by one level by say well you know there were nine satisfying assignments all together uh six of them were when the when x one is zero and three of them were when x one is one to verify what does the verify have to check that these add up correctly when i preset x1 to zero and to one it had better add up to the total number of satisfying assignments if that works out the verifier is happy is still being it's still consistent with being convinced that this k was the right value um so um the next step is well the verifier says well how do i know those two values are correct the prover says okay well i'm going to prove send un unravel them one level further then here's a number of satisfying assignments when the next variable is set to both possibilities for each of the possibilities of the first variable now if you're understanding me about what the prover is sending you should start to be getting a little nervous because something is i mean this is going to be correct but it's going to start it looks like it's starting to blow up in terms of the number of amount of work that's involved and that's actually a problem but let's bear with that for the moment let's just worry about correctness not about complexity for the moment so the proof is going to now send the number of satisfying assignments for each of those four possible ways of presetting the first two variables and the verifier is going to check that that was consistent with the information the prover sent in the previous round right by again checking this identity here so then the proof is going to continue on doing that until it's got it's done that through m rounds where m is the number of variables so at this point the proof is going to send all possible ways of presetting all of the variables so that now there's two to the m possibilities here again this is hopelessly not allowed but okay ignoring that the proofer's got to use this at the nth round to check what happens at the at the previous round so that's when they were m minus one values sent because each one has one more uh uh you're extending the presets by one so which using this to check that the previous round was the values were correct so it's looking for um you know the m minus one presets have to add up correctly um you know in terms of the m the presets of m uh values uh for each of those ways of uh doing those uh m minus one presets and so now the provers send all of those two to the m counts which are by the way ones and zeros because at this point we have preset all of the values of the variables and so there's only one possible assignment at most that there can be and now the verifier the approver is done the verifier is going to check by itself that these values make sense that these values are correct so it's going to do that by looking back at the formula so far up at this point the verifier has not been looking at the formula it's just been checking the internal consistency of the provers messages with each other but now at the end the verifier is going to take the messages these these values that the approver sent for each of the two to the m presets and see if it matches up with what the formula would do remember that was the other sort of the base case of the the end the fact number two um from the uh slider to ago make sure that these agree okay and now the verifier says well okay if everything has checked out and all of these are are in agreement then the verifier is going to be convinced that um that uh fee had k satisfying assignments but if anywhere along the way one of these checks fails the approver is not the verifier is not going to be convinced and is going to reject okay so in a sense this is kind of dopey you know you know we've just i'm just kind of giving you a complicated way of just counting up one by one each of the satisfying assignments of the formula and seeing if that matches k but nevertheless this way of looking at it is gonna be uh help us to understand um the way to fix this so so bear with me for another minute on this one so another way of looking at this which i think is is particularly useful is to think of what happens well okay we'll get there in a second i want to look at what happens if k was wrong but before i do that let's look at the i'm going to give a kind of a graphic a graphical view of um the information that the prover sends and the and the verifiers actions in this protocol so the the values that the prover sending are going to be in yellow so the and the information that the verifier has or checks is going to be in white so the the verifier has the k the the input value which is supposed to be the number of satisfying assignments and the prover sends some value and the verifier checks that this value which is supposed to be the number of satisfying assignments corresponds with k so that's one of the checks it does then the approver is going to send kind of take to justify this value it um sends the number of satisfying assignments when you have x1 set to zero or set to one the verify adds those up to give you and it's supposed to equal the total number of satisfying assignments and so this is if you understood this protocol this is just i'm writing it out in a sort of a simplified way perhaps okay and so um keeps checking that these things add up correctly until you get down to setting all m values in all two to the impossible ways and now the verifier is going to then check to make sure that that equals the what the formula would say um okay okay so now let's what happens if k was the wrong value it did not agree with the number of satisfying assignments um and what does what happens now um could the prover what happens if the approver tries to make the verifier accept anyway so um so the only thing the prover can do at the very first step would be to lie um about you know if the approver sends the if k is wrong and the proof approver sends the correct value for the the the total count the verifier is going to reject so i'm trying to see is it could the approver try to make the verifier accept what what happens so the prover has to lie here and i'm going to indicate that by saying the approver is sending in the wrong value for um uh the the total count well if the proof is going to lie here um then just like you know if you you know you have a child who um tells a lie and then you start you know as the parent you start asking questions to try to see if the story is consistent one lie is going to lead to another lie um and that's what happens here if the uh in order to justify this lie um the proof is going to have to a lie in one or perhaps both but at least one of these two values because you can't have the two correct values adding up to the incorrect value because you have to think about what's going on here so if this is a lie that's going to force a lie at one of one side of the other one level down which is then going to force a lie to propagate down and so there's a lie at every stage is going to force a lie at least in one one place or another to propagate all the way down to the bottom and then at the bottom the verifier will see that the check doesn't work as when it turns when it tries to connect it up with the formula itself and the form verifier will reject okay so just a way of looking at this um if the form if the value if the input was not in the language um so uh but the problem is that as i said that this is exponential so how are we going to fix that so just looking ahead to what we're going to do on tuesday okay let's see if there's any questions here first of all uh okay yes i got a question should this be uh should this be a minus i i purposely made this bracket only go not include the very last zero yeah there's a total of m zeroes here all together but the i left out the last zero that's why i said n minus one maybe it would have been better to say m okay so and you got another interesting question here why can't we reject right away if k is wrong um uh well the verifier is probabilistic polynomial time how does the verifier know if k is wrong um so it means or or or right so what we're trying to do is something like you know like np where we have a certificate but now we have this kind of interactive certificate in the form of disprover maybe that's another way to look at it um where if you're in the language there should be some way for the prover to convince to get make you accept but if you're not not in the language there should be no way for the proverb to make you accept um uh so the verifier just can't reject right away because there's no way to tell how does the verifier know it's going to start rejecting things when it shouldn't if it's just going to be rejecting willy-nilly here um okay how does the verifier need to determine if the prover is internally consistent instead of just asking so why does the verifier need to determine if the approver is internally consistent instead of just asking the questions in step n plus one yeah so maybe that's because it looks like all of the work is happening at the very end um but i'm really presenting this to you as a preparation for what we're going to do on tuesday um so it's important to think about the connection from each step to the next each step is going to be justified by what happens at the next step until we get to the very end so you have to just understand it for what it is don't try to make it more efficient yeah i realize this is kind of dumb um good point we're not using the probabilism here um and moreover we're not really even using the interaction here the prover is doing all the sending the verifier is just accepting at the end yeah this is we're not using the power and we're getting a weaker result so let's move on before we run out of time here so how do we gonna fix this so the problem is is blowing up each to justify each stage where each value we're um needing to present two values which add up to it um and that's uh good leading to a blow up now it would be nice if we can do something where each value was supported by just a single value at the next level so we know here's an idea well you know in order to understand to see that uh that that this total count is correct why don't we just pick at random either zero or one and only follow that one down well the problem with doing that is because the the the sequence of lies is could be just a single path through this uh tree and the chances you're going to find that path down to a contradiction at the bottom is very low if you're just doing it at random um so just randomly picking zeros and ones as as the as the one you're going to justify used to justify the previous value is not going to be good enough so what but this is what we're going to do however the values that we're going to pick for um for these random um inputs are not going to be boolean values we're going to pick non-boolean assignments to the variables which again just as with the branching program case didn't make any sense on the surface of it we're going to have to make it make sense and we'll have to see how to do that on tues in tuesday's lecture okay so that's kind of the setup okay um yeah so in a similar question why is this any different from just non-deterministically guessing the assignments it's because of this we're really setting the stage okay so what we did today was we introduced them the the model and defined the complexity class we can we did show this one in its full glory we showed that non-iso is an ip really worth understanding this uh protocol here making sure you you're comfortable with that and and also the the model itself and so for tuesday's lecture we're going to finish this up uh well we started showing that number set as an ip which is what we need to to do to prove co-np is an ip and we'll finish that next time which will be our last time okay so that's it for today i'll stick around for questions so a good question here why can't um v just reject if some of the checks are incorrect yes v could as soon as there's a check that fails we can just reject at that stage i'm just trying to argue that at some point along the way if the input is not in the language there's going to be a check that fails so i mean i said reject at the end but yeah i mean you could have rejected at any point along the way um okay um someone's asking for what what role did i play uh so i did my my own personal role in this i were to which was twofold first of all i i came up with the idea uh of well not the idea i came up with the name interactive proof uh i remember when sylvia mccauley was explaining this to me in my apartment many many years ago he had a kind of a little bit complicated and i don't even remember what the protocol was for it was not for something simple was something involving uh prime numbers and i said oh that's a kind of an interactive proof and it's and it it stuck from that point on so um that was one thing but the other thing in terms of more mathematically i my role was so uh shafi goldwater and i approved the equivalence of the two models the public coin and the private coin version um so that that was uh my uh my role in this back and when this was all first coming out approved it approved it on an airplane on the way to a conference somewhere so i think we're going to notice any other questions i think we'll head out take care everybody see one see you on tuesday bye bye you 4 00:00:27,750 --> 00:00:30,310 5 00:00:30,310 --> 00:00:34,630 6 00:00:34,630 --> 00:00:37,590 7 00:00:37,590 --> 00:00:42,069 8 00:00:42,069 --> 00:00:44,630 9 00:00:44,630 --> 00:00:48,950 10 00:00:48,950 --> 00:00:51,350 11 00:00:51,350 --> 00:00:54,069 12 00:00:54,069 --> 00:00:54,079 13 00:00:54,079 --> 00:00:56,150 14 00:00:56,150 --> 00:00:58,470 15 00:00:58,470 --> 00:00:59,990 16 00:00:59,990 --> 00:01:01,750 17 00:01:01,750 --> 00:01:04,229 18 00:01:04,229 --> 00:01:07,750 19 00:01:07,750 --> 00:01:09,670 20 00:01:09,670 --> 00:01:09,680 21 00:01:09,680 --> 00:01:10,870 22 00:01:10,870 --> 00:01:12,550 23 00:01:12,550 --> 00:01:17,190 24 00:01:17,190 --> 00:01:18,710 25 00:01:18,710 --> 00:01:20,550 26 00:01:20,550 --> 00:01:22,310 27 00:01:22,310 --> 00:01:22,320 28 00:01:22,320 --> 00:01:25,190 29 00:01:25,190 --> 00:01:27,190 30 00:01:27,190 --> 00:01:27,200 31 00:01:27,200 --> 00:01:28,390 32 00:01:28,390 --> 00:01:31,749 33 00:01:31,749 --> 00:01:31,759 34 00:01:31,759 --> 00:01:32,710 35 00:01:32,710 --> 00:01:33,590 36 00:01:33,590 --> 00:01:35,030 37 00:01:35,030 --> 00:01:35,040 38 00:01:35,040 --> 00:01:35,830 39 00:01:35,830 --> 00:01:38,710 40 00:01:38,710 --> 00:01:41,590 41 00:01:41,590 --> 00:01:43,429 42 00:01:43,429 --> 00:01:43,439 43 00:01:43,439 --> 00:01:45,429 44 00:01:45,429 --> 00:01:47,510 45 00:01:47,510 --> 00:01:50,069 46 00:01:50,069 --> 00:01:53,190 47 00:01:53,190 --> 00:01:55,749 48 00:01:55,749 --> 00:01:59,670 49 00:01:59,670 --> 00:02:02,950 50 00:02:02,950 --> 00:02:05,350 51 00:02:05,350 --> 00:02:07,670 52 00:02:07,670 --> 00:02:09,749 53 00:02:09,749 --> 00:02:09,759 54 00:02:09,759 --> 00:02:11,830 55 00:02:11,830 --> 00:02:13,589 56 00:02:13,589 --> 00:02:17,670 57 00:02:17,670 --> 00:02:19,110 58 00:02:19,110 --> 00:02:19,120 59 00:02:19,120 --> 00:02:20,470 60 00:02:20,470 --> 00:02:25,510 61 00:02:25,510 --> 00:02:27,430 62 00:02:27,430 --> 00:02:30,630 63 00:02:30,630 --> 00:02:34,150 64 00:02:34,150 --> 00:02:36,710 65 00:02:36,710 --> 00:02:39,670 66 00:02:39,670 --> 00:02:39,680 67 00:02:39,680 --> 00:02:40,390 68 00:02:40,390 --> 00:02:42,150 69 00:02:42,150 --> 00:02:45,589 70 00:02:45,589 --> 00:02:47,670 71 00:02:47,670 --> 00:02:50,070 72 00:02:50,070 --> 00:02:50,080 73 00:02:50,080 --> 00:02:51,589 74 00:02:51,589 --> 00:02:54,150 75 00:02:54,150 --> 00:02:54,160 76 00:02:54,160 --> 00:02:55,350 77 00:02:55,350 --> 00:02:58,149 78 00:02:58,149 --> 00:03:00,229 79 00:03:00,229 --> 00:03:01,990 80 00:03:01,990 --> 00:03:04,470 81 00:03:04,470 --> 00:03:07,430 82 00:03:07,430 --> 00:03:09,430 83 00:03:09,430 --> 00:03:11,910 84 00:03:11,910 --> 00:03:13,509 85 00:03:13,509 --> 00:03:16,070 86 00:03:16,070 --> 00:03:18,790 87 00:03:18,790 --> 00:03:21,190 88 00:03:21,190 --> 00:03:24,149 89 00:03:24,149 --> 00:03:26,630 90 00:03:26,630 --> 00:03:28,789 91 00:03:28,789 --> 00:03:30,470 92 00:03:30,470 --> 00:03:32,710 93 00:03:32,710 --> 00:03:34,630 94 00:03:34,630 --> 00:03:34,640 95 00:03:34,640 --> 00:03:37,509 96 00:03:37,509 --> 00:03:39,350 97 00:03:39,350 --> 00:03:41,350 98 00:03:41,350 --> 00:03:43,350 99 00:03:43,350 --> 00:03:44,949 100 00:03:44,949 --> 00:03:47,990 101 00:03:47,990 --> 00:03:49,750 102 00:03:49,750 --> 00:03:51,509 103 00:03:51,509 --> 00:03:53,830 104 00:03:53,830 --> 00:03:59,350 105 00:03:59,350 --> 00:04:01,190 106 00:04:01,190 --> 00:04:01,200 107 00:04:01,200 --> 00:04:02,149 108 00:04:02,149 --> 00:04:02,159 109 00:04:02,159 --> 00:04:02,680 110 00:04:02,680 --> 00:04:02,690 111 00:04:02,690 --> 00:04:04,470 112 00:04:04,470 --> 00:04:07,270 113 00:04:07,270 --> 00:04:09,110 114 00:04:09,110 --> 00:04:12,309 115 00:04:12,309 --> 00:04:14,470 116 00:04:14,470 --> 00:04:16,789 117 00:04:16,789 --> 00:04:19,830 118 00:04:19,830 --> 00:04:21,509 119 00:04:21,509 --> 00:04:22,629 120 00:04:22,629 --> 00:04:22,639 121 00:04:22,639 --> 00:04:23,430 122 00:04:23,430 --> 00:04:25,749 123 00:04:25,749 --> 00:04:27,430 124 00:04:27,430 --> 00:04:28,629 125 00:04:28,629 --> 00:04:30,950 126 00:04:30,950 --> 00:04:32,870 127 00:04:32,870 --> 00:04:34,469 128 00:04:34,469 --> 00:04:36,469 129 00:04:36,469 --> 00:04:36,479 130 00:04:36,479 --> 00:04:38,790 131 00:04:38,790 --> 00:04:40,310 132 00:04:40,310 --> 00:04:43,430 133 00:04:43,430 --> 00:04:43,440 134 00:04:43,440 --> 00:04:44,469 135 00:04:44,469 --> 00:04:44,479 136 00:04:44,479 --> 00:04:46,070 137 00:04:46,070 --> 00:04:48,790 138 00:04:48,790 --> 00:04:50,790 139 00:04:50,790 --> 00:04:53,670 140 00:04:53,670 --> 00:04:56,230 141 00:04:56,230 --> 00:04:58,150 142 00:04:58,150 --> 00:04:59,670 143 00:04:59,670 --> 00:05:01,990 144 00:05:01,990 --> 00:05:04,469 145 00:05:04,469 --> 00:05:06,390 146 00:05:06,390 --> 00:05:08,230 147 00:05:08,230 --> 00:05:08,240 148 00:05:08,240 --> 00:05:09,110 149 00:05:09,110 --> 00:05:09,120 150 00:05:09,120 --> 00:05:11,110 151 00:05:11,110 --> 00:05:13,749 152 00:05:13,749 --> 00:05:15,510 153 00:05:15,510 --> 00:05:16,790 154 00:05:16,790 --> 00:05:20,230 155 00:05:20,230 --> 00:05:21,749 156 00:05:21,749 --> 00:05:23,590 157 00:05:23,590 --> 00:05:23,600 158 00:05:23,600 --> 00:05:25,430 159 00:05:25,430 --> 00:05:28,629 160 00:05:28,629 --> 00:05:31,110 161 00:05:31,110 --> 00:05:36,469 162 00:05:36,469 --> 00:05:38,390 163 00:05:38,390 --> 00:05:40,870 164 00:05:40,870 --> 00:05:40,880 165 00:05:40,880 --> 00:05:42,390 166 00:05:42,390 --> 00:05:44,230 167 00:05:44,230 --> 00:05:48,070 168 00:05:48,070 --> 00:05:50,070 169 00:05:50,070 --> 00:05:52,390 170 00:05:52,390 --> 00:05:56,230 171 00:05:56,230 --> 00:05:59,270 172 00:05:59,270 --> 00:06:01,110 173 00:06:01,110 --> 00:06:03,590 174 00:06:03,590 --> 00:06:05,670 175 00:06:05,670 --> 00:06:07,990 176 00:06:07,990 --> 00:06:09,590 177 00:06:09,590 --> 00:06:12,150 178 00:06:12,150 --> 00:06:14,150 179 00:06:14,150 --> 00:06:16,629 180 00:06:16,629 --> 00:06:18,070 181 00:06:18,070 --> 00:06:21,189 182 00:06:21,189 --> 00:06:23,029 183 00:06:23,029 --> 00:06:25,590 184 00:06:25,590 --> 00:06:27,189 185 00:06:27,189 --> 00:06:29,590 186 00:06:29,590 --> 00:06:32,550 187 00:06:32,550 --> 00:06:35,670 188 00:06:35,670 --> 00:06:37,430 189 00:06:37,430 --> 00:06:39,990 190 00:06:39,990 --> 00:06:40,000 191 00:06:40,000 --> 00:06:41,029 192 00:06:41,029 --> 00:06:42,950 193 00:06:42,950 --> 00:06:45,430 194 00:06:45,430 --> 00:06:46,830 195 00:06:46,830 --> 00:06:50,230 196 00:06:50,230 --> 00:06:50,240 197 00:06:50,240 --> 00:06:51,830 198 00:06:51,830 --> 00:06:51,840 199 00:06:51,840 --> 00:06:53,909 200 00:06:53,909 --> 00:06:56,390 201 00:06:56,390 --> 00:06:58,550 202 00:06:58,550 --> 00:07:00,710 203 00:07:00,710 --> 00:07:02,790 204 00:07:02,790 --> 00:07:04,950 205 00:07:04,950 --> 00:07:04,960 206 00:07:04,960 --> 00:07:06,150 207 00:07:06,150 --> 00:07:06,160 208 00:07:06,160 --> 00:07:06,950 209 00:07:06,950 --> 00:07:09,830 210 00:07:09,830 --> 00:07:11,430 211 00:07:11,430 --> 00:07:13,909 212 00:07:13,909 --> 00:07:15,830 213 00:07:15,830 --> 00:07:17,189 214 00:07:17,189 --> 00:07:20,710 215 00:07:20,710 --> 00:07:20,720 216 00:07:20,720 --> 00:07:22,710 217 00:07:22,710 --> 00:07:24,469 218 00:07:24,469 --> 00:07:26,950 219 00:07:26,950 --> 00:07:29,029 220 00:07:29,029 --> 00:07:31,510 221 00:07:31,510 --> 00:07:33,270 222 00:07:33,270 --> 00:07:35,270 223 00:07:35,270 --> 00:07:37,830 224 00:07:37,830 --> 00:07:40,150 225 00:07:40,150 --> 00:07:44,950 226 00:07:44,950 --> 00:07:47,270 227 00:07:47,270 --> 00:07:50,150 228 00:07:50,150 --> 00:07:52,070 229 00:07:52,070 --> 00:07:55,670 230 00:07:55,670 --> 00:07:57,110 231 00:07:57,110 --> 00:07:59,830 232 00:07:59,830 --> 00:08:01,990 233 00:08:01,990 --> 00:08:04,950 234 00:08:04,950 --> 00:08:10,550 235 00:08:10,550 --> 00:08:13,430 236 00:08:13,430 --> 00:08:15,589 237 00:08:15,589 --> 00:08:17,350 238 00:08:17,350 --> 00:08:18,869 239 00:08:18,869 --> 00:08:20,869 240 00:08:20,869 --> 00:08:22,950 241 00:08:22,950 --> 00:08:22,960 242 00:08:22,960 --> 00:08:23,909 243 00:08:23,909 --> 00:08:26,710 244 00:08:26,710 --> 00:08:26,720 245 00:08:26,720 --> 00:08:27,430 246 00:08:27,430 --> 00:08:29,189 247 00:08:29,189 --> 00:08:30,710 248 00:08:30,710 --> 00:08:34,070 249 00:08:34,070 --> 00:08:36,389 250 00:08:36,389 --> 00:08:38,230 251 00:08:38,230 --> 00:08:39,430 252 00:08:39,430 --> 00:08:41,430 253 00:08:41,430 --> 00:08:41,440 254 00:08:41,440 --> 00:08:42,550 255 00:08:42,550 --> 00:08:42,560 256 00:08:42,560 --> 00:08:44,070 257 00:08:44,070 --> 00:08:47,110 258 00:08:47,110 --> 00:08:49,670 259 00:08:49,670 --> 00:08:51,590 260 00:08:51,590 --> 00:08:51,600 261 00:08:51,600 --> 00:08:52,790 262 00:08:52,790 --> 00:08:55,190 263 00:08:55,190 --> 00:08:57,110 264 00:08:57,110 --> 00:08:59,110 265 00:08:59,110 --> 00:09:00,870 266 00:09:00,870 --> 00:09:02,710 267 00:09:02,710 --> 00:09:05,269 268 00:09:05,269 --> 00:09:07,670 269 00:09:07,670 --> 00:09:10,870 270 00:09:10,870 --> 00:09:14,790 271 00:09:14,790 --> 00:09:17,110 272 00:09:17,110 --> 00:09:17,120 273 00:09:17,120 --> 00:09:18,310 274 00:09:18,310 --> 00:09:20,389 275 00:09:20,389 --> 00:09:23,670 276 00:09:23,670 --> 00:09:26,070 277 00:09:26,070 --> 00:09:27,990 278 00:09:27,990 --> 00:09:29,829 279 00:09:29,829 --> 00:09:29,839 280 00:09:29,839 --> 00:09:32,790 281 00:09:32,790 --> 00:09:32,800 282 00:09:32,800 --> 00:09:34,710 283 00:09:34,710 --> 00:09:37,990 284 00:09:37,990 --> 00:09:40,150 285 00:09:40,150 --> 00:09:42,470 286 00:09:42,470 --> 00:09:45,509 287 00:09:45,509 --> 00:09:49,750 288 00:09:49,750 --> 00:09:51,269 289 00:09:51,269 --> 00:09:51,279 290 00:09:51,279 --> 00:09:53,110 291 00:09:53,110 --> 00:09:53,120 292 00:09:53,120 --> 00:09:53,990 293 00:09:53,990 --> 00:09:57,269 294 00:09:57,269 --> 00:09:59,269 295 00:09:59,269 --> 00:10:01,350 296 00:10:01,350 --> 00:10:03,269 297 00:10:03,269 --> 00:10:05,190 298 00:10:05,190 --> 00:10:09,350 299 00:10:09,350 --> 00:10:09,360 300 00:10:09,360 --> 00:10:10,389 301 00:10:10,389 --> 00:10:10,399 302 00:10:10,399 --> 00:10:11,350 303 00:10:11,350 --> 00:10:13,910 304 00:10:13,910 --> 00:10:16,470 305 00:10:16,470 --> 00:10:18,230 306 00:10:18,230 --> 00:10:19,509 307 00:10:19,509 --> 00:10:19,519 308 00:10:19,519 --> 00:10:21,829 309 00:10:21,829 --> 00:10:23,910 310 00:10:23,910 --> 00:10:26,389 311 00:10:26,389 --> 00:10:28,630 312 00:10:28,630 --> 00:10:33,910 313 00:10:33,910 --> 00:10:37,509 314 00:10:37,509 --> 00:10:38,550 315 00:10:38,550 --> 00:10:40,710 316 00:10:40,710 --> 00:10:42,949 317 00:10:42,949 --> 00:10:45,030 318 00:10:45,030 --> 00:10:48,389 319 00:10:48,389 --> 00:10:50,790 320 00:10:50,790 --> 00:10:53,030 321 00:10:53,030 --> 00:10:54,630 322 00:10:54,630 --> 00:10:56,550 323 00:10:56,550 --> 00:10:59,990 324 00:10:59,990 --> 00:11:01,030 325 00:11:01,030 --> 00:11:03,750 326 00:11:03,750 --> 00:11:03,760 327 00:11:03,760 --> 00:11:05,110 328 00:11:05,110 --> 00:11:05,120 329 00:11:05,120 --> 00:11:06,310 330 00:11:06,310 --> 00:11:08,870 331 00:11:08,870 --> 00:11:12,069 332 00:11:12,069 --> 00:11:14,949 333 00:11:14,949 --> 00:11:17,670 334 00:11:17,670 --> 00:11:20,069 335 00:11:20,069 --> 00:11:20,079 336 00:11:20,079 --> 00:11:21,269 337 00:11:21,269 --> 00:11:24,630 338 00:11:24,630 --> 00:11:26,949 339 00:11:26,949 --> 00:11:26,959 340 00:11:26,959 --> 00:11:28,630 341 00:11:28,630 --> 00:11:31,670 342 00:11:31,670 --> 00:11:34,710 343 00:11:34,710 --> 00:11:36,870 344 00:11:36,870 --> 00:11:39,670 345 00:11:39,670 --> 00:11:41,269 346 00:11:41,269 --> 00:11:42,870 347 00:11:42,870 --> 00:11:44,790 348 00:11:44,790 --> 00:11:47,269 349 00:11:47,269 --> 00:11:50,310 350 00:11:50,310 --> 00:11:50,320 351 00:11:50,320 --> 00:11:51,590 352 00:11:51,590 --> 00:11:53,269 353 00:11:53,269 --> 00:11:53,279 354 00:11:53,279 --> 00:11:54,829 355 00:11:54,829 --> 00:11:54,839 356 00:11:54,839 --> 00:11:56,710 357 00:11:56,710 --> 00:11:56,720 358 00:11:56,720 --> 00:11:58,310 359 00:11:58,310 --> 00:11:59,350 360 00:11:59,350 --> 00:12:01,430 361 00:12:01,430 --> 00:12:01,440 362 00:12:01,440 --> 00:12:02,389 363 00:12:02,389 --> 00:12:05,190 364 00:12:05,190 --> 00:12:07,190 365 00:12:07,190 --> 00:12:09,269 366 00:12:09,269 --> 00:12:11,269 367 00:12:11,269 --> 00:12:12,550 368 00:12:12,550 --> 00:12:12,560 369 00:12:12,560 --> 00:12:14,790 370 00:12:14,790 --> 00:12:17,350 371 00:12:17,350 --> 00:12:19,350 372 00:12:19,350 --> 00:12:19,360 373 00:12:19,360 --> 00:12:20,069 374 00:12:20,069 --> 00:12:21,590 375 00:12:21,590 --> 00:12:26,389 376 00:12:26,389 --> 00:12:27,350 377 00:12:27,350 --> 00:12:28,550 378 00:12:28,550 --> 00:12:31,269 379 00:12:31,269 --> 00:12:32,230 380 00:12:32,230 --> 00:12:33,990 381 00:12:33,990 --> 00:12:35,829 382 00:12:35,829 --> 00:12:38,470 383 00:12:38,470 --> 00:12:42,150 384 00:12:42,150 --> 00:12:44,389 385 00:12:44,389 --> 00:12:44,399 386 00:12:44,399 --> 00:12:45,590 387 00:12:45,590 --> 00:12:48,389 388 00:12:48,389 --> 00:12:50,069 389 00:12:50,069 --> 00:12:50,079 390 00:12:50,079 --> 00:12:52,710 391 00:12:52,710 --> 00:12:54,470 392 00:12:54,470 --> 00:12:56,310 393 00:12:56,310 --> 00:12:59,030 394 00:12:59,030 --> 00:12:59,040 395 00:12:59,040 --> 00:13:00,150 396 00:13:00,150 --> 00:13:01,350 397 00:13:01,350 --> 00:13:03,110 398 00:13:03,110 --> 00:13:04,870 399 00:13:04,870 --> 00:13:06,870 400 00:13:06,870 --> 00:13:08,470 401 00:13:08,470 --> 00:13:09,990 402 00:13:09,990 --> 00:13:11,269 403 00:13:11,269 --> 00:13:11,279 404 00:13:11,279 --> 00:13:12,790 405 00:13:12,790 --> 00:13:14,790 406 00:13:14,790 --> 00:13:16,230 407 00:13:16,230 --> 00:13:18,150 408 00:13:18,150 --> 00:13:21,430 409 00:13:21,430 --> 00:13:24,389 410 00:13:24,389 --> 00:13:27,110 411 00:13:27,110 --> 00:13:28,629 412 00:13:28,629 --> 00:13:30,470 413 00:13:30,470 --> 00:13:31,990 414 00:13:31,990 --> 00:13:32,000 415 00:13:32,000 --> 00:13:33,269 416 00:13:33,269 --> 00:13:37,430 417 00:13:37,430 --> 00:13:39,350 418 00:13:39,350 --> 00:13:39,360 419 00:13:39,360 --> 00:13:42,550 420 00:13:42,550 --> 00:13:42,560 421 00:13:42,560 --> 00:13:45,350 422 00:13:45,350 --> 00:13:46,870 423 00:13:46,870 --> 00:13:48,550 424 00:13:48,550 --> 00:13:49,910 425 00:13:49,910 --> 00:13:52,230 426 00:13:52,230 --> 00:13:54,790 427 00:13:54,790 --> 00:13:55,910 428 00:13:55,910 --> 00:13:57,430 429 00:13:57,430 --> 00:13:57,440 430 00:13:57,440 --> 00:13:59,030 431 00:13:59,030 --> 00:14:00,829 432 00:14:00,829 --> 00:14:03,269 433 00:14:03,269 --> 00:14:05,509 434 00:14:05,509 --> 00:14:06,470 435 00:14:06,470 --> 00:14:09,030 436 00:14:09,030 --> 00:14:09,040 437 00:14:09,040 --> 00:14:11,430 438 00:14:11,430 --> 00:14:12,870 439 00:14:12,870 --> 00:14:14,150 440 00:14:14,150 --> 00:14:16,550 441 00:14:16,550 --> 00:14:20,550 442 00:14:20,550 --> 00:14:22,389 443 00:14:22,389 --> 00:14:22,399 444 00:14:22,399 --> 00:14:24,870 445 00:14:24,870 --> 00:14:27,670 446 00:14:27,670 --> 00:14:30,550 447 00:14:30,550 --> 00:14:32,710 448 00:14:32,710 --> 00:14:34,550 449 00:14:34,550 --> 00:14:35,509 450 00:14:35,509 --> 00:14:38,150 451 00:14:38,150 --> 00:14:39,670 452 00:14:39,670 --> 00:14:43,030 453 00:14:43,030 --> 00:14:43,040 454 00:14:43,040 --> 00:14:45,269 455 00:14:45,269 --> 00:14:46,870 456 00:14:46,870 --> 00:14:48,949 457 00:14:48,949 --> 00:14:50,949 458 00:14:50,949 --> 00:14:52,310 459 00:14:52,310 --> 00:14:53,750 460 00:14:53,750 --> 00:14:55,350 461 00:14:55,350 --> 00:14:57,110 462 00:14:57,110 --> 00:14:58,949 463 00:14:58,949 --> 00:14:58,959 464 00:14:58,959 --> 00:15:00,230 465 00:15:00,230 --> 00:15:02,470 466 00:15:02,470 --> 00:15:04,949 467 00:15:04,949 --> 00:15:07,590 468 00:15:07,590 --> 00:15:10,230 469 00:15:10,230 --> 00:15:13,430 470 00:15:13,430 --> 00:15:17,910 471 00:15:17,910 --> 00:15:20,949 472 00:15:20,949 --> 00:15:23,430 473 00:15:23,430 --> 00:15:24,710 474 00:15:24,710 --> 00:15:27,030 475 00:15:27,030 --> 00:15:28,310 476 00:15:28,310 --> 00:15:30,069 477 00:15:30,069 --> 00:15:31,670 478 00:15:31,670 --> 00:15:34,949 479 00:15:34,949 --> 00:15:37,590 480 00:15:37,590 --> 00:15:39,110 481 00:15:39,110 --> 00:15:40,310 482 00:15:40,310 --> 00:15:42,069 483 00:15:42,069 --> 00:15:44,069 484 00:15:44,069 --> 00:15:45,430 485 00:15:45,430 --> 00:15:47,110 486 00:15:47,110 --> 00:15:48,629 487 00:15:48,629 --> 00:15:50,870 488 00:15:50,870 --> 00:15:54,310 489 00:15:54,310 --> 00:15:56,629 490 00:15:56,629 --> 00:15:58,150 491 00:15:58,150 --> 00:16:00,470 492 00:16:00,470 --> 00:16:00,480 493 00:16:00,480 --> 00:16:03,269 494 00:16:03,269 --> 00:16:04,949 495 00:16:04,949 --> 00:16:12,870 496 00:16:12,870 --> 00:16:14,069 497 00:16:14,069 --> 00:16:18,710 498 00:16:18,710 --> 00:16:21,110 499 00:16:21,110 --> 00:16:24,310 500 00:16:24,310 --> 00:16:26,710 501 00:16:26,710 --> 00:16:28,310 502 00:16:28,310 --> 00:16:28,320 503 00:16:28,320 --> 00:16:29,910 504 00:16:29,910 --> 00:16:31,430 505 00:16:31,430 --> 00:16:33,910 506 00:16:33,910 --> 00:16:35,110 507 00:16:35,110 --> 00:16:36,949 508 00:16:36,949 --> 00:16:38,550 509 00:16:38,550 --> 00:16:42,629 510 00:16:42,629 --> 00:16:43,670 511 00:16:43,670 --> 00:16:46,870 512 00:16:46,870 --> 00:16:48,870 513 00:16:48,870 --> 00:16:48,880 514 00:16:48,880 --> 00:16:51,910 515 00:16:51,910 --> 00:16:53,269 516 00:16:53,269 --> 00:16:53,279 517 00:16:53,279 --> 00:16:56,389 518 00:16:56,389 --> 00:16:58,949 519 00:16:58,949 --> 00:17:01,269 520 00:17:01,269 --> 00:17:04,789 521 00:17:04,789 --> 00:17:06,470 522 00:17:06,470 --> 00:17:08,949 523 00:17:08,949 --> 00:17:12,390 524 00:17:12,390 --> 00:17:17,350 525 00:17:17,350 --> 00:17:21,270 526 00:17:21,270 --> 00:17:23,110 527 00:17:23,110 --> 00:17:23,120 528 00:17:23,120 --> 00:17:23,990 529 00:17:23,990 --> 00:17:26,390 530 00:17:26,390 --> 00:17:26,400 531 00:17:26,400 --> 00:17:27,429 532 00:17:27,429 --> 00:17:30,310 533 00:17:30,310 --> 00:17:32,310 534 00:17:32,310 --> 00:17:38,150 535 00:17:38,150 --> 00:17:39,750 536 00:17:39,750 --> 00:17:42,310 537 00:17:42,310 --> 00:17:46,150 538 00:17:46,150 --> 00:17:47,750 539 00:17:47,750 --> 00:17:50,070 540 00:17:50,070 --> 00:17:51,510 541 00:17:51,510 --> 00:17:53,270 542 00:17:53,270 --> 00:17:55,029 543 00:17:55,029 --> 00:17:56,710 544 00:17:56,710 --> 00:17:59,590 545 00:17:59,590 --> 00:18:01,909 546 00:18:01,909 --> 00:18:04,390 547 00:18:04,390 --> 00:18:06,549 548 00:18:06,549 --> 00:18:11,990 549 00:18:11,990 --> 00:18:12,000 550 00:18:12,000 --> 00:18:15,669 551 00:18:15,669 --> 00:18:18,390 552 00:18:18,390 --> 00:18:20,310 553 00:18:20,310 --> 00:18:23,350 554 00:18:23,350 --> 00:18:25,350 555 00:18:25,350 --> 00:18:27,909 556 00:18:27,909 --> 00:18:29,750 557 00:18:29,750 --> 00:18:31,110 558 00:18:31,110 --> 00:18:32,630 559 00:18:32,630 --> 00:18:34,070 560 00:18:34,070 --> 00:18:35,590 561 00:18:35,590 --> 00:18:38,070 562 00:18:38,070 --> 00:18:40,470 563 00:18:40,470 --> 00:18:42,070 564 00:18:42,070 --> 00:18:43,669 565 00:18:43,669 --> 00:18:46,230 566 00:18:46,230 --> 00:18:48,390 567 00:18:48,390 --> 00:18:49,669 568 00:18:49,669 --> 00:18:50,950 569 00:18:50,950 --> 00:18:52,710 570 00:18:52,710 --> 00:18:54,230 571 00:18:54,230 --> 00:18:56,070 572 00:18:56,070 --> 00:18:57,590 573 00:18:57,590 --> 00:18:57,600 574 00:18:57,600 --> 00:18:58,789 575 00:18:58,789 --> 00:18:59,990 576 00:18:59,990 --> 00:19:02,070 577 00:19:02,070 --> 00:19:03,909 578 00:19:03,909 --> 00:19:06,630 579 00:19:06,630 --> 00:19:07,750 580 00:19:07,750 --> 00:19:07,760 581 00:19:07,760 --> 00:19:08,870 582 00:19:08,870 --> 00:19:11,110 583 00:19:11,110 --> 00:19:13,190 584 00:19:13,190 --> 00:19:14,630 585 00:19:14,630 --> 00:19:16,070 586 00:19:16,070 --> 00:19:16,080 587 00:19:16,080 --> 00:19:18,789 588 00:19:18,789 --> 00:19:20,950 589 00:19:20,950 --> 00:19:22,470 590 00:19:22,470 --> 00:19:24,150 591 00:19:24,150 --> 00:19:26,630 592 00:19:26,630 --> 00:19:29,990 593 00:19:29,990 --> 00:19:30,000 594 00:19:30,000 --> 00:19:31,029 595 00:19:31,029 --> 00:19:32,549 596 00:19:32,549 --> 00:19:35,029 597 00:19:35,029 --> 00:19:36,950 598 00:19:36,950 --> 00:19:38,470 599 00:19:38,470 --> 00:19:40,150 600 00:19:40,150 --> 00:19:43,750 601 00:19:43,750 --> 00:19:45,990 602 00:19:45,990 --> 00:19:47,830 603 00:19:47,830 --> 00:19:47,840 604 00:19:47,840 --> 00:19:49,430 605 00:19:49,430 --> 00:19:49,440 606 00:19:49,440 --> 00:19:50,470 607 00:19:50,470 --> 00:19:52,950 608 00:19:52,950 --> 00:19:54,630 609 00:19:54,630 --> 00:19:56,630 610 00:19:56,630 --> 00:19:58,710 611 00:19:58,710 --> 00:20:00,710 612 00:20:00,710 --> 00:20:02,789 613 00:20:02,789 --> 00:20:08,149 614 00:20:08,149 --> 00:20:10,549 615 00:20:10,549 --> 00:20:12,390 616 00:20:12,390 --> 00:20:12,400 617 00:20:12,400 --> 00:20:13,110 618 00:20:13,110 --> 00:20:15,110 619 00:20:15,110 --> 00:20:15,120 620 00:20:15,120 --> 00:20:17,270 621 00:20:17,270 --> 00:20:19,110 622 00:20:19,110 --> 00:20:21,830 623 00:20:21,830 --> 00:20:21,840 624 00:20:21,840 --> 00:20:24,070 625 00:20:24,070 --> 00:20:27,110 626 00:20:27,110 --> 00:20:29,190 627 00:20:29,190 --> 00:20:32,070 628 00:20:32,070 --> 00:20:34,630 629 00:20:34,630 --> 00:20:37,990 630 00:20:37,990 --> 00:20:38,000 631 00:20:38,000 --> 00:20:39,990 632 00:20:39,990 --> 00:20:41,750 633 00:20:41,750 --> 00:20:43,590 634 00:20:43,590 --> 00:20:43,600 635 00:20:43,600 --> 00:20:45,190 636 00:20:45,190 --> 00:20:46,830 637 00:20:46,830 --> 00:20:49,430 638 00:20:49,430 --> 00:20:51,430 639 00:20:51,430 --> 00:20:51,440 640 00:20:51,440 --> 00:20:52,950 641 00:20:52,950 --> 00:20:52,960 642 00:20:52,960 --> 00:20:54,390 643 00:20:54,390 --> 00:20:54,400 644 00:20:54,400 --> 00:20:55,590 645 00:20:55,590 --> 00:20:57,750 646 00:20:57,750 --> 00:20:57,760 647 00:20:57,760 --> 00:20:59,270 648 00:20:59,270 --> 00:21:01,350 649 00:21:01,350 --> 00:21:04,549 650 00:21:04,549 --> 00:21:06,310 651 00:21:06,310 --> 00:21:07,909 652 00:21:07,909 --> 00:21:10,390 653 00:21:10,390 --> 00:21:12,230 654 00:21:12,230 --> 00:21:14,149 655 00:21:14,149 --> 00:21:14,159 656 00:21:14,159 --> 00:21:15,430 657 00:21:15,430 --> 00:21:17,029 658 00:21:17,029 --> 00:21:18,710 659 00:21:18,710 --> 00:21:20,870 660 00:21:20,870 --> 00:21:22,149 661 00:21:22,149 --> 00:21:24,710 662 00:21:24,710 --> 00:21:27,590 663 00:21:27,590 --> 00:21:31,590 664 00:21:31,590 --> 00:21:33,830 665 00:21:33,830 --> 00:21:33,840 666 00:21:33,840 --> 00:21:34,870 667 00:21:34,870 --> 00:21:35,909 668 00:21:35,909 --> 00:21:38,230 669 00:21:38,230 --> 00:21:38,240 670 00:21:38,240 --> 00:21:39,029 671 00:21:39,029 --> 00:21:42,950 672 00:21:42,950 --> 00:21:45,830 673 00:21:45,830 --> 00:21:49,190 674 00:21:49,190 --> 00:21:53,590 675 00:21:53,590 --> 00:21:53,600 676 00:21:53,600 --> 00:21:54,390 677 00:21:54,390 --> 00:21:57,110 678 00:21:57,110 --> 00:21:59,270 679 00:21:59,270 --> 00:22:01,669 680 00:22:01,669 --> 00:22:03,590 681 00:22:03,590 --> 00:22:05,990 682 00:22:05,990 --> 00:22:07,750 683 00:22:07,750 --> 00:22:10,149 684 00:22:10,149 --> 00:22:11,350 685 00:22:11,350 --> 00:22:11,360 686 00:22:11,360 --> 00:22:12,390 687 00:22:12,390 --> 00:22:15,990 688 00:22:15,990 --> 00:22:17,350 689 00:22:17,350 --> 00:22:20,149 690 00:22:20,149 --> 00:22:21,909 691 00:22:21,909 --> 00:22:21,919 692 00:22:21,919 --> 00:22:23,510 693 00:22:23,510 --> 00:22:26,950 694 00:22:26,950 --> 00:22:28,710 695 00:22:28,710 --> 00:22:30,789 696 00:22:30,789 --> 00:22:30,799 697 00:22:30,799 --> 00:22:32,230 698 00:22:32,230 --> 00:22:34,310 699 00:22:34,310 --> 00:22:36,230 700 00:22:36,230 --> 00:22:38,470 701 00:22:38,470 --> 00:22:40,470 702 00:22:40,470 --> 00:22:42,310 703 00:22:42,310 --> 00:22:43,270 704 00:22:43,270 --> 00:22:45,909 705 00:22:45,909 --> 00:22:48,149 706 00:22:48,149 --> 00:22:50,470 707 00:22:50,470 --> 00:22:51,909 708 00:22:51,909 --> 00:22:54,710 709 00:22:54,710 --> 00:22:57,029 710 00:22:57,029 --> 00:22:59,029 711 00:22:59,029 --> 00:22:59,039 712 00:22:59,039 --> 00:22:59,990 713 00:22:59,990 --> 00:23:03,029 714 00:23:03,029 --> 00:23:04,870 715 00:23:04,870 --> 00:23:08,390 716 00:23:08,390 --> 00:23:11,110 717 00:23:11,110 --> 00:23:11,120 718 00:23:11,120 --> 00:23:12,149 719 00:23:12,149 --> 00:23:12,159 720 00:23:12,159 --> 00:23:13,350 721 00:23:13,350 --> 00:23:16,310 722 00:23:16,310 --> 00:23:18,549 723 00:23:18,549 --> 00:23:20,390 724 00:23:20,390 --> 00:23:23,510 725 00:23:23,510 --> 00:23:25,430 726 00:23:25,430 --> 00:23:27,270 727 00:23:27,270 --> 00:23:28,549 728 00:23:28,549 --> 00:23:29,990 729 00:23:29,990 --> 00:23:32,070 730 00:23:32,070 --> 00:23:34,310 731 00:23:34,310 --> 00:23:34,320 732 00:23:34,320 --> 00:23:35,430 733 00:23:35,430 --> 00:23:36,950 734 00:23:36,950 --> 00:23:39,190 735 00:23:39,190 --> 00:23:39,200 736 00:23:39,200 --> 00:23:42,230 737 00:23:42,230 --> 00:23:43,750 738 00:23:43,750 --> 00:23:46,710 739 00:23:46,710 --> 00:23:49,269 740 00:23:49,269 --> 00:23:50,870 741 00:23:50,870 --> 00:23:52,549 742 00:23:52,549 --> 00:23:54,310 743 00:23:54,310 --> 00:23:55,590 744 00:23:55,590 --> 00:23:57,590 745 00:23:57,590 --> 00:24:00,630 746 00:24:00,630 --> 00:24:00,640 747 00:24:00,640 --> 00:24:02,070 748 00:24:02,070 --> 00:24:03,669 749 00:24:03,669 --> 00:24:03,679 750 00:24:03,679 --> 00:24:04,549 751 00:24:04,549 --> 00:24:06,549 752 00:24:06,549 --> 00:24:08,789 753 00:24:08,789 --> 00:24:11,590 754 00:24:11,590 --> 00:24:12,870 755 00:24:12,870 --> 00:24:13,990 756 00:24:13,990 --> 00:24:16,310 757 00:24:16,310 --> 00:24:18,549 758 00:24:18,549 --> 00:24:20,390 759 00:24:20,390 --> 00:24:21,830 760 00:24:21,830 --> 00:24:23,190 761 00:24:23,190 --> 00:24:23,200 762 00:24:23,200 --> 00:24:24,470 763 00:24:24,470 --> 00:24:24,480 764 00:24:24,480 --> 00:24:25,430 765 00:24:25,430 --> 00:24:27,190 766 00:24:27,190 --> 00:24:28,470 767 00:24:28,470 --> 00:24:30,789 768 00:24:30,789 --> 00:24:32,789 769 00:24:32,789 --> 00:24:33,990 770 00:24:33,990 --> 00:24:35,669 771 00:24:35,669 --> 00:24:35,679 772 00:24:35,679 --> 00:24:36,470 773 00:24:36,470 --> 00:24:39,990 774 00:24:39,990 --> 00:24:40,000 775 00:24:40,000 --> 00:24:40,870 776 00:24:40,870 --> 00:24:44,870 777 00:24:44,870 --> 00:24:46,470 778 00:24:46,470 --> 00:24:49,190 779 00:24:49,190 --> 00:24:50,950 780 00:24:50,950 --> 00:24:53,269 781 00:24:53,269 --> 00:24:55,909 782 00:24:55,909 --> 00:24:58,710 783 00:24:58,710 --> 00:24:58,720 784 00:24:58,720 --> 00:24:59,590 785 00:24:59,590 --> 00:25:02,070 786 00:25:02,070 --> 00:25:02,080 787 00:25:02,080 --> 00:25:03,510 788 00:25:03,510 --> 00:25:05,350 789 00:25:05,350 --> 00:25:07,510 790 00:25:07,510 --> 00:25:09,430 791 00:25:09,430 --> 00:25:12,470 792 00:25:12,470 --> 00:25:17,029 793 00:25:17,029 --> 00:25:18,470 794 00:25:18,470 --> 00:25:20,710 795 00:25:20,710 --> 00:25:23,430 796 00:25:23,430 --> 00:25:23,440 797 00:25:23,440 --> 00:25:24,470 798 00:25:24,470 --> 00:25:26,549 799 00:25:26,549 --> 00:25:28,789 800 00:25:28,789 --> 00:25:34,070 801 00:25:34,070 --> 00:25:35,830 802 00:25:35,830 --> 00:25:37,269 803 00:25:37,269 --> 00:25:39,110 804 00:25:39,110 --> 00:25:39,120 805 00:25:39,120 --> 00:25:39,990 806 00:25:39,990 --> 00:25:41,590 807 00:25:41,590 --> 00:25:43,669 808 00:25:43,669 --> 00:25:45,110 809 00:25:45,110 --> 00:25:46,549 810 00:25:46,549 --> 00:25:49,029 811 00:25:49,029 --> 00:25:50,390 812 00:25:50,390 --> 00:25:51,909 813 00:25:51,909 --> 00:25:53,510 814 00:25:53,510 --> 00:25:55,590 815 00:25:55,590 --> 00:25:57,750 816 00:25:57,750 --> 00:25:59,029 817 00:25:59,029 --> 00:26:02,070 818 00:26:02,070 --> 00:26:03,909 819 00:26:03,909 --> 00:26:03,919 820 00:26:03,919 --> 00:26:05,669 821 00:26:05,669 --> 00:26:07,190 822 00:26:07,190 --> 00:26:08,870 823 00:26:08,870 --> 00:26:12,149 824 00:26:12,149 --> 00:26:14,789 825 00:26:14,789 --> 00:26:17,830 826 00:26:17,830 --> 00:26:21,029 827 00:26:21,029 --> 00:26:22,870 828 00:26:22,870 --> 00:26:25,430 829 00:26:25,430 --> 00:26:26,470 830 00:26:26,470 --> 00:26:28,549 831 00:26:28,549 --> 00:26:30,310 832 00:26:30,310 --> 00:26:30,320 833 00:26:30,320 --> 00:26:32,630 834 00:26:32,630 --> 00:26:32,640 835 00:26:32,640 --> 00:26:37,110 836 00:26:37,110 --> 00:26:38,710 837 00:26:38,710 --> 00:26:40,470 838 00:26:40,470 --> 00:26:40,480 839 00:26:40,480 --> 00:26:41,669 840 00:26:41,669 --> 00:26:43,510 841 00:26:43,510 --> 00:26:45,350 842 00:26:45,350 --> 00:26:47,029 843 00:26:47,029 --> 00:26:49,909 844 00:26:49,909 --> 00:26:51,750 845 00:26:51,750 --> 00:26:53,990 846 00:26:53,990 --> 00:26:54,000 847 00:26:54,000 --> 00:26:55,190 848 00:26:55,190 --> 00:26:56,630 849 00:26:56,630 --> 00:26:58,310 850 00:26:58,310 --> 00:27:00,149 851 00:27:00,149 --> 00:27:01,750 852 00:27:01,750 --> 00:27:03,990 853 00:27:03,990 --> 00:27:07,190 854 00:27:07,190 --> 00:27:08,470 855 00:27:08,470 --> 00:27:08,480 856 00:27:08,480 --> 00:27:09,350 857 00:27:09,350 --> 00:27:11,350 858 00:27:11,350 --> 00:27:13,350 859 00:27:13,350 --> 00:27:14,870 860 00:27:14,870 --> 00:27:16,230 861 00:27:16,230 --> 00:27:17,669 862 00:27:17,669 --> 00:27:19,830 863 00:27:19,830 --> 00:27:22,470 864 00:27:22,470 --> 00:27:23,750 865 00:27:23,750 --> 00:27:23,760 866 00:27:23,760 --> 00:27:25,510 867 00:27:25,510 --> 00:27:27,269 868 00:27:27,269 --> 00:27:30,470 869 00:27:30,470 --> 00:27:31,990 870 00:27:31,990 --> 00:27:32,000 871 00:27:32,000 --> 00:27:33,269 872 00:27:33,269 --> 00:27:35,590 873 00:27:35,590 --> 00:27:37,110 874 00:27:37,110 --> 00:27:39,029 875 00:27:39,029 --> 00:27:41,430 876 00:27:41,430 --> 00:27:43,029 877 00:27:43,029 --> 00:27:43,039 878 00:27:43,039 --> 00:27:44,230 879 00:27:44,230 --> 00:27:48,149 880 00:27:48,149 --> 00:27:51,190 881 00:27:51,190 --> 00:27:54,230 882 00:27:54,230 --> 00:27:56,230 883 00:27:56,230 --> 00:27:57,909 884 00:27:57,909 --> 00:27:59,350 885 00:27:59,350 --> 00:28:02,789 886 00:28:02,789 --> 00:28:05,029 887 00:28:05,029 --> 00:28:06,630 888 00:28:06,630 --> 00:28:08,149 889 00:28:08,149 --> 00:28:11,510 890 00:28:11,510 --> 00:28:11,520 891 00:28:11,520 --> 00:28:13,269 892 00:28:13,269 --> 00:28:15,269 893 00:28:15,269 --> 00:28:17,590 894 00:28:17,590 --> 00:28:17,600 895 00:28:17,600 --> 00:28:19,269 896 00:28:19,269 --> 00:28:21,669 897 00:28:21,669 --> 00:28:23,430 898 00:28:23,430 --> 00:28:26,149 899 00:28:26,149 --> 00:28:28,389 900 00:28:28,389 --> 00:28:31,430 901 00:28:31,430 --> 00:28:35,110 902 00:28:35,110 --> 00:28:36,789 903 00:28:36,789 --> 00:28:38,950 904 00:28:38,950 --> 00:28:43,190 905 00:28:43,190 --> 00:28:45,430 906 00:28:45,430 --> 00:28:47,269 907 00:28:47,269 --> 00:28:50,630 908 00:28:50,630 --> 00:28:52,630 909 00:28:52,630 --> 00:28:55,110 910 00:28:55,110 --> 00:28:58,710 911 00:28:58,710 --> 00:29:00,389 912 00:29:00,389 --> 00:29:02,230 913 00:29:02,230 --> 00:29:03,669 914 00:29:03,669 --> 00:29:06,950 915 00:29:06,950 --> 00:29:09,110 916 00:29:09,110 --> 00:29:10,830 917 00:29:10,830 --> 00:29:13,909 918 00:29:13,909 --> 00:29:16,470 919 00:29:16,470 --> 00:29:18,149 920 00:29:18,149 --> 00:29:20,149 921 00:29:20,149 --> 00:29:21,590 922 00:29:21,590 --> 00:29:23,750 923 00:29:23,750 --> 00:29:25,190 924 00:29:25,190 --> 00:29:27,909 925 00:29:27,909 --> 00:29:27,919 926 00:29:27,919 --> 00:29:29,830 927 00:29:29,830 --> 00:29:32,070 928 00:29:32,070 --> 00:29:33,590 929 00:29:33,590 --> 00:29:36,149 930 00:29:36,149 --> 00:29:39,190 931 00:29:39,190 --> 00:29:41,750 932 00:29:41,750 --> 00:29:43,669 933 00:29:43,669 --> 00:29:46,549 934 00:29:46,549 --> 00:29:46,559 935 00:29:46,559 --> 00:29:47,669 936 00:29:47,669 --> 00:29:50,789 937 00:29:50,789 --> 00:29:52,310 938 00:29:52,310 --> 00:29:53,269 939 00:29:53,269 --> 00:29:54,470 940 00:29:54,470 --> 00:29:56,710 941 00:29:56,710 --> 00:29:59,110 942 00:29:59,110 --> 00:30:02,389 943 00:30:02,389 --> 00:30:04,549 944 00:30:04,549 --> 00:30:06,549 945 00:30:06,549 --> 00:30:09,269 946 00:30:09,269 --> 00:30:12,310 947 00:30:12,310 --> 00:30:14,310 948 00:30:14,310 --> 00:30:16,710 949 00:30:16,710 --> 00:30:16,720 950 00:30:16,720 --> 00:30:17,669 951 00:30:17,669 --> 00:30:17,679 952 00:30:17,679 --> 00:30:18,630 953 00:30:18,630 --> 00:30:20,870 954 00:30:20,870 --> 00:30:24,070 955 00:30:24,070 --> 00:30:24,080 956 00:30:24,080 --> 00:30:27,909 957 00:30:27,909 --> 00:30:30,710 958 00:30:30,710 --> 00:30:33,029 959 00:30:33,029 --> 00:30:34,789 960 00:30:34,789 --> 00:30:37,029 961 00:30:37,029 --> 00:30:37,039 962 00:30:37,039 --> 00:30:39,590 963 00:30:39,590 --> 00:30:41,269 964 00:30:41,269 --> 00:30:43,510 965 00:30:43,510 --> 00:30:45,110 966 00:30:45,110 --> 00:30:47,350 967 00:30:47,350 --> 00:30:47,360 968 00:30:47,360 --> 00:30:48,470 969 00:30:48,470 --> 00:30:50,389 970 00:30:50,389 --> 00:30:51,909 971 00:30:51,909 --> 00:30:55,510 972 00:30:55,510 --> 00:30:55,520 973 00:30:55,520 --> 00:30:56,710 974 00:30:56,710 --> 00:30:56,720 975 00:30:56,720 --> 00:30:57,909 976 00:30:57,909 --> 00:30:57,919 977 00:30:57,919 --> 00:30:58,870 978 00:30:58,870 --> 00:31:01,509 979 00:31:01,509 --> 00:31:01,519 980 00:31:01,519 --> 00:31:03,190 981 00:31:03,190 --> 00:31:04,549 982 00:31:04,549 --> 00:31:06,389 983 00:31:06,389 --> 00:31:08,230 984 00:31:08,230 --> 00:31:11,590 985 00:31:11,590 --> 00:31:13,990 986 00:31:13,990 --> 00:31:16,389 987 00:31:16,389 --> 00:31:18,789 988 00:31:18,789 --> 00:31:21,669 989 00:31:21,669 --> 00:31:22,789 990 00:31:22,789 --> 00:31:28,389 991 00:31:28,389 --> 00:31:30,230 992 00:31:30,230 --> 00:31:31,990 993 00:31:31,990 --> 00:31:33,669 994 00:31:33,669 --> 00:31:35,590 995 00:31:35,590 --> 00:31:36,870 996 00:31:36,870 --> 00:31:39,269 997 00:31:39,269 --> 00:31:40,710 998 00:31:40,710 --> 00:31:40,720 999 00:31:40,720 --> 00:31:41,830 1000 00:31:41,830 --> 00:31:43,269 1001 00:31:43,269 --> 00:31:45,509 1002 00:31:45,509 --> 00:31:47,750 1003 00:31:47,750 --> 00:31:49,430 1004 00:31:49,430 --> 00:31:51,909 1005 00:31:51,909 --> 00:31:53,750 1006 00:31:53,750 --> 00:31:55,590 1007 00:31:55,590 --> 00:31:57,430 1008 00:31:57,430 --> 00:31:58,950 1009 00:31:58,950 --> 00:32:00,950 1010 00:32:00,950 --> 00:32:02,870 1011 00:32:02,870 --> 00:32:07,269 1012 00:32:07,269 --> 00:32:09,190 1013 00:32:09,190 --> 00:32:10,630 1014 00:32:10,630 --> 00:32:14,470 1015 00:32:14,470 --> 00:32:17,590 1016 00:32:17,590 --> 00:32:20,470 1017 00:32:20,470 --> 00:32:20,480 1018 00:32:20,480 --> 00:32:22,389 1019 00:32:22,389 --> 00:32:24,070 1020 00:32:24,070 --> 00:32:26,310 1021 00:32:26,310 --> 00:32:28,470 1022 00:32:28,470 --> 00:32:30,230 1023 00:32:30,230 --> 00:32:32,549 1024 00:32:32,549 --> 00:32:34,070 1025 00:32:34,070 --> 00:32:35,909 1026 00:32:35,909 --> 00:32:35,919 1027 00:32:35,919 --> 00:32:37,190 1028 00:32:37,190 --> 00:32:39,029 1029 00:32:39,029 --> 00:32:39,039 1030 00:32:39,039 --> 00:32:39,830 1031 00:32:39,830 --> 00:32:41,269 1032 00:32:41,269 --> 00:32:43,590 1033 00:32:43,590 --> 00:32:43,600 1034 00:32:43,600 --> 00:32:44,470 1035 00:32:44,470 --> 00:32:46,149 1036 00:32:46,149 --> 00:32:48,470 1037 00:32:48,470 --> 00:32:49,750 1038 00:32:49,750 --> 00:32:57,350 1039 00:32:57,350 --> 00:32:59,190 1040 00:32:59,190 --> 00:33:00,470 1041 00:33:00,470 --> 00:33:02,870 1042 00:33:02,870 --> 00:33:05,750 1043 00:33:05,750 --> 00:33:08,149 1044 00:33:08,149 --> 00:33:10,710 1045 00:33:10,710 --> 00:33:12,230 1046 00:33:12,230 --> 00:33:13,669 1047 00:33:13,669 --> 00:33:13,679 1048 00:33:13,679 --> 00:33:14,630 1049 00:33:14,630 --> 00:33:16,870 1050 00:33:16,870 --> 00:33:19,269 1051 00:33:19,269 --> 00:33:19,279 1052 00:33:19,279 --> 00:33:20,470 1053 00:33:20,470 --> 00:33:22,070 1054 00:33:22,070 --> 00:33:23,590 1055 00:33:23,590 --> 00:33:28,149 1056 00:33:28,149 --> 00:33:30,149 1057 00:33:30,149 --> 00:33:32,070 1058 00:33:32,070 --> 00:33:33,750 1059 00:33:33,750 --> 00:33:35,669 1060 00:33:35,669 --> 00:33:35,679 1061 00:33:35,679 --> 00:33:36,870 1062 00:33:36,870 --> 00:33:41,509 1063 00:33:41,509 --> 00:33:43,509 1064 00:33:43,509 --> 00:33:46,070 1065 00:33:46,070 --> 00:33:48,149 1066 00:33:48,149 --> 00:33:49,750 1067 00:33:49,750 --> 00:33:51,350 1068 00:33:51,350 --> 00:33:52,549 1069 00:33:52,549 --> 00:33:54,549 1070 00:33:54,549 --> 00:33:56,710 1071 00:33:56,710 --> 00:33:57,990 1072 00:33:57,990 --> 00:33:59,590 1073 00:33:59,590 --> 00:34:01,110 1074 00:34:01,110 --> 00:34:02,549 1075 00:34:02,549 --> 00:34:05,509 1076 00:34:05,509 --> 00:34:08,629 1077 00:34:08,629 --> 00:34:10,550 1078 00:34:10,550 --> 00:34:11,589 1079 00:34:11,589 --> 00:34:11,599 1080 00:34:11,599 --> 00:34:12,869 1081 00:34:12,869 --> 00:34:15,510 1082 00:34:15,510 --> 00:34:15,520 1083 00:34:15,520 --> 00:34:17,510 1084 00:34:17,510 --> 00:34:19,829 1085 00:34:19,829 --> 00:34:23,669 1086 00:34:23,669 --> 00:34:24,710 1087 00:34:24,710 --> 00:34:24,720 1088 00:34:24,720 --> 00:34:25,669 1089 00:34:25,669 --> 00:34:27,990 1090 00:34:27,990 --> 00:34:29,510 1091 00:34:29,510 --> 00:34:31,270 1092 00:34:31,270 --> 00:34:31,280 1093 00:34:31,280 --> 00:34:32,470 1094 00:34:32,470 --> 00:34:35,829 1095 00:34:35,829 --> 00:34:38,710 1096 00:34:38,710 --> 00:34:40,790 1097 00:34:40,790 --> 00:34:43,669 1098 00:34:43,669 --> 00:34:43,679 1099 00:34:43,679 --> 00:34:46,470 1100 00:34:46,470 --> 00:34:51,109 1101 00:34:51,109 --> 00:34:52,470 1102 00:34:52,470 --> 00:34:52,480 1103 00:34:52,480 --> 00:34:53,510 1104 00:34:53,510 --> 00:34:56,869 1105 00:34:56,869 --> 00:35:01,349 1106 00:35:01,349 --> 00:35:03,270 1107 00:35:03,270 --> 00:35:04,390 1108 00:35:04,390 --> 00:35:05,670 1109 00:35:05,670 --> 00:35:05,680 1110 00:35:05,680 --> 00:35:06,790 1111 00:35:06,790 --> 00:35:11,190 1112 00:35:11,190 --> 00:35:13,190 1113 00:35:13,190 --> 00:35:13,200 1114 00:35:13,200 --> 00:35:14,550 1115 00:35:14,550 --> 00:35:24,069 1116 00:35:24,069 --> 00:35:25,670 1117 00:35:25,670 --> 00:35:28,069 1118 00:35:28,069 --> 00:35:29,430 1119 00:35:29,430 --> 00:35:31,190 1120 00:35:31,190 --> 00:35:34,390 1121 00:35:34,390 --> 00:35:37,589 1122 00:35:37,589 --> 00:35:39,910 1123 00:35:39,910 --> 00:35:41,109 1124 00:35:41,109 --> 00:35:43,990 1125 00:35:43,990 --> 00:35:44,000 1126 00:35:44,000 --> 00:35:46,950 1127 00:35:46,950 --> 00:35:48,550 1128 00:35:48,550 --> 00:35:50,790 1129 00:35:50,790 --> 00:35:50,800 1130 00:35:50,800 --> 00:35:53,430 1131 00:35:53,430 --> 00:35:55,990 1132 00:35:55,990 --> 00:35:58,870 1133 00:35:58,870 --> 00:36:00,550 1134 00:36:00,550 --> 00:36:01,990 1135 00:36:01,990 --> 00:36:02,000 1136 00:36:02,000 --> 00:36:04,790 1137 00:36:04,790 --> 00:36:07,750 1138 00:36:07,750 --> 00:36:07,760 1139 00:36:07,760 --> 00:36:09,829 1140 00:36:09,829 --> 00:36:12,710 1141 00:36:12,710 --> 00:36:14,790 1142 00:36:14,790 --> 00:36:14,800 1143 00:36:14,800 --> 00:36:15,750 1144 00:36:15,750 --> 00:36:16,870 1145 00:36:16,870 --> 00:36:18,950 1146 00:36:18,950 --> 00:36:18,960 1147 00:36:18,960 --> 00:36:20,069 1148 00:36:20,069 --> 00:36:21,589 1149 00:36:21,589 --> 00:36:25,030 1150 00:36:25,030 --> 00:36:26,630 1151 00:36:26,630 --> 00:36:28,550 1152 00:36:28,550 --> 00:36:29,589 1153 00:36:29,589 --> 00:36:31,190 1154 00:36:31,190 --> 00:36:33,109 1155 00:36:33,109 --> 00:36:34,470 1156 00:36:34,470 --> 00:36:35,910 1157 00:36:35,910 --> 00:36:38,150 1158 00:36:38,150 --> 00:36:39,589 1159 00:36:39,589 --> 00:36:41,109 1160 00:36:41,109 --> 00:36:42,790 1161 00:36:42,790 --> 00:36:42,800 1162 00:36:42,800 --> 00:36:43,990 1163 00:36:43,990 --> 00:36:44,000 1164 00:36:44,000 --> 00:36:46,550 1165 00:36:46,550 --> 00:36:48,310 1166 00:36:48,310 --> 00:36:50,390 1167 00:36:50,390 --> 00:36:50,400 1168 00:36:50,400 --> 00:36:51,109 1169 00:36:51,109 --> 00:36:52,790 1170 00:36:52,790 --> 00:36:53,990 1171 00:36:53,990 --> 00:36:54,000 1172 00:36:54,000 --> 00:36:55,430 1173 00:36:55,430 --> 00:36:55,440 1174 00:36:55,440 --> 00:36:56,310 1175 00:36:56,310 --> 00:36:59,349 1176 00:36:59,349 --> 00:37:02,470 1177 00:37:02,470 --> 00:37:03,829 1178 00:37:03,829 --> 00:37:05,510 1179 00:37:05,510 --> 00:37:06,790 1180 00:37:06,790 --> 00:37:09,910 1181 00:37:09,910 --> 00:37:13,349 1182 00:37:13,349 --> 00:37:15,589 1183 00:37:15,589 --> 00:37:15,599 1184 00:37:15,599 --> 00:37:17,109 1185 00:37:17,109 --> 00:37:17,119 1186 00:37:17,119 --> 00:37:18,630 1187 00:37:18,630 --> 00:37:21,109 1188 00:37:21,109 --> 00:37:23,109 1189 00:37:23,109 --> 00:37:25,510 1190 00:37:25,510 --> 00:37:28,069 1191 00:37:28,069 --> 00:37:29,589 1192 00:37:29,589 --> 00:37:32,390 1193 00:37:32,390 --> 00:37:32,400 1194 00:37:32,400 --> 00:37:33,750 1195 00:37:33,750 --> 00:37:36,470 1196 00:37:36,470 --> 00:37:40,470 1197 00:37:40,470 --> 00:37:42,470 1198 00:37:42,470 --> 00:37:43,990 1199 00:37:43,990 --> 00:37:46,710 1200 00:37:46,710 --> 00:37:47,750 1201 00:37:47,750 --> 00:37:49,430 1202 00:37:49,430 --> 00:37:52,470 1203 00:37:52,470 --> 00:37:54,310 1204 00:37:54,310 --> 00:37:56,230 1205 00:37:56,230 --> 00:37:58,470 1206 00:37:58,470 --> 00:38:00,630 1207 00:38:00,630 --> 00:38:02,829 1208 00:38:02,829 --> 00:38:06,710 1209 00:38:06,710 --> 00:38:09,589 1210 00:38:09,589 --> 00:38:09,599 1211 00:38:09,599 --> 00:38:10,390 1212 00:38:10,390 --> 00:38:13,349 1213 00:38:13,349 --> 00:38:15,750 1214 00:38:15,750 --> 00:38:18,069 1215 00:38:18,069 --> 00:38:19,670 1216 00:38:19,670 --> 00:38:22,069 1217 00:38:22,069 --> 00:38:23,829 1218 00:38:23,829 --> 00:38:25,430 1219 00:38:25,430 --> 00:38:27,589 1220 00:38:27,589 --> 00:38:29,589 1221 00:38:29,589 --> 00:38:31,589 1222 00:38:31,589 --> 00:38:34,390 1223 00:38:34,390 --> 00:38:34,400 1224 00:38:34,400 --> 00:38:35,270 1225 00:38:35,270 --> 00:38:37,270 1226 00:38:37,270 --> 00:38:39,430 1227 00:38:39,430 --> 00:38:41,430 1228 00:38:41,430 --> 00:38:42,950 1229 00:38:42,950 --> 00:38:42,960 1230 00:38:42,960 --> 00:38:45,670 1231 00:38:45,670 --> 00:38:48,069 1232 00:38:48,069 --> 00:38:50,550 1233 00:38:50,550 --> 00:38:50,560 1234 00:38:50,560 --> 00:38:52,230 1235 00:38:52,230 --> 00:38:52,240 1236 00:38:52,240 --> 00:38:52,560 1237 00:38:52,560 --> 00:38:52,570 1238 00:38:52,570 --> 00:38:55,589 1239 00:38:55,589 --> 00:38:57,349 1240 00:38:57,349 --> 00:38:58,950 1241 00:38:58,950 --> 00:39:01,349 1242 00:39:01,349 --> 00:39:04,150 1243 00:39:04,150 --> 00:39:06,150 1244 00:39:06,150 --> 00:39:07,670 1245 00:39:07,670 --> 00:39:10,630 1246 00:39:10,630 --> 00:39:12,230 1247 00:39:12,230 --> 00:39:14,390 1248 00:39:14,390 --> 00:39:14,400 1249 00:39:14,400 --> 00:39:15,349 1250 00:39:15,349 --> 00:39:15,359 1251 00:39:15,359 --> 00:39:16,550 1252 00:39:16,550 --> 00:39:18,710 1253 00:39:18,710 --> 00:39:20,710 1254 00:39:20,710 --> 00:39:22,870 1255 00:39:22,870 --> 00:39:24,150 1256 00:39:24,150 --> 00:39:25,990 1257 00:39:25,990 --> 00:39:27,750 1258 00:39:27,750 --> 00:39:31,510 1259 00:39:31,510 --> 00:39:32,870 1260 00:39:32,870 --> 00:39:34,230 1261 00:39:34,230 --> 00:39:37,430 1262 00:39:37,430 --> 00:39:38,710 1263 00:39:38,710 --> 00:39:40,950 1264 00:39:40,950 --> 00:39:43,829 1265 00:39:43,829 --> 00:39:43,839 1266 00:39:43,839 --> 00:39:45,990 1267 00:39:45,990 --> 00:39:48,710 1268 00:39:48,710 --> 00:39:51,750 1269 00:39:51,750 --> 00:39:52,790 1270 00:39:52,790 --> 00:39:54,550 1271 00:39:54,550 --> 00:39:55,910 1272 00:39:55,910 --> 00:39:57,510 1273 00:39:57,510 --> 00:39:58,950 1274 00:39:58,950 --> 00:39:58,960 1275 00:39:58,960 --> 00:39:59,990 1276 00:39:59,990 --> 00:40:00,000 1277 00:40:00,000 --> 00:40:01,030 1278 00:40:01,030 --> 00:40:04,230 1279 00:40:04,230 --> 00:40:07,349 1280 00:40:07,349 --> 00:40:09,670 1281 00:40:09,670 --> 00:40:11,190 1282 00:40:11,190 --> 00:40:13,910 1283 00:40:13,910 --> 00:40:16,390 1284 00:40:16,390 --> 00:40:18,870 1285 00:40:18,870 --> 00:40:20,630 1286 00:40:20,630 --> 00:40:22,710 1287 00:40:22,710 --> 00:40:22,720 1288 00:40:22,720 --> 00:40:27,270 1289 00:40:27,270 --> 00:40:27,280 1290 00:40:27,280 --> 00:40:28,150 1291 00:40:28,150 --> 00:40:30,230 1292 00:40:30,230 --> 00:40:31,990 1293 00:40:31,990 --> 00:40:34,230 1294 00:40:34,230 --> 00:40:36,870 1295 00:40:36,870 --> 00:40:39,030 1296 00:40:39,030 --> 00:40:40,790 1297 00:40:40,790 --> 00:40:42,550 1298 00:40:42,550 --> 00:40:44,710 1299 00:40:44,710 --> 00:40:44,720 1300 00:40:44,720 --> 00:40:46,630 1301 00:40:46,630 --> 00:40:46,640 1302 00:40:46,640 --> 00:40:47,829 1303 00:40:47,829 --> 00:40:47,839 1304 00:40:47,839 --> 00:40:48,710 1305 00:40:48,710 --> 00:40:48,720 1306 00:40:48,720 --> 00:40:51,990 1307 00:40:51,990 --> 00:40:53,670 1308 00:40:53,670 --> 00:40:56,150 1309 00:40:56,150 --> 00:40:58,470 1310 00:40:58,470 --> 00:41:00,950 1311 00:41:00,950 --> 00:41:02,309 1312 00:41:02,309 --> 00:41:02,319 1313 00:41:02,319 --> 00:41:04,230 1314 00:41:04,230 --> 00:41:06,710 1315 00:41:06,710 --> 00:41:08,309 1316 00:41:08,309 --> 00:41:11,270 1317 00:41:11,270 --> 00:41:13,190 1318 00:41:13,190 --> 00:41:13,200 1319 00:41:13,200 --> 00:41:14,630 1320 00:41:14,630 --> 00:41:16,870 1321 00:41:16,870 --> 00:41:18,790 1322 00:41:18,790 --> 00:41:21,190 1323 00:41:21,190 --> 00:41:25,430 1324 00:41:25,430 --> 00:41:27,990 1325 00:41:27,990 --> 00:41:29,670 1326 00:41:29,670 --> 00:41:32,230 1327 00:41:32,230 --> 00:41:33,270 1328 00:41:33,270 --> 00:41:35,349 1329 00:41:35,349 --> 00:41:35,359 1330 00:41:35,359 --> 00:41:36,630 1331 00:41:36,630 --> 00:41:38,390 1332 00:41:38,390 --> 00:41:41,030 1333 00:41:41,030 --> 00:41:43,109 1334 00:41:43,109 --> 00:41:44,150 1335 00:41:44,150 --> 00:41:46,950 1336 00:41:46,950 --> 00:41:50,870 1337 00:41:50,870 --> 00:41:50,880 1338 00:41:50,880 --> 00:41:52,550 1339 00:41:52,550 --> 00:41:54,950 1340 00:41:54,950 --> 00:41:57,190 1341 00:41:57,190 --> 00:41:59,270 1342 00:41:59,270 --> 00:42:02,870 1343 00:42:02,870 --> 00:42:04,470 1344 00:42:04,470 --> 00:42:06,309 1345 00:42:06,309 --> 00:42:09,109 1346 00:42:09,109 --> 00:42:09,119 1347 00:42:09,119 --> 00:42:10,950 1348 00:42:10,950 --> 00:42:13,190 1349 00:42:13,190 --> 00:42:15,190 1350 00:42:15,190 --> 00:42:17,030 1351 00:42:17,030 --> 00:42:18,870 1352 00:42:18,870 --> 00:42:20,150 1353 00:42:20,150 --> 00:42:23,109 1354 00:42:23,109 --> 00:42:24,630 1355 00:42:24,630 --> 00:42:25,990 1356 00:42:25,990 --> 00:42:27,349 1357 00:42:27,349 --> 00:42:27,359 1358 00:42:27,359 --> 00:42:29,109 1359 00:42:29,109 --> 00:42:29,119 1360 00:42:29,119 --> 00:42:30,710 1361 00:42:30,710 --> 00:42:32,069 1362 00:42:32,069 --> 00:42:34,309 1363 00:42:34,309 --> 00:42:36,870 1364 00:42:36,870 --> 00:42:36,880 1365 00:42:36,880 --> 00:42:38,790 1366 00:42:38,790 --> 00:42:40,470 1367 00:42:40,470 --> 00:42:45,109 1368 00:42:45,109 --> 00:42:45,119 1369 00:42:45,119 --> 00:42:45,910 1370 00:42:45,910 --> 00:42:45,920 1371 00:42:45,920 --> 00:42:48,230 1372 00:42:48,230 --> 00:42:52,230 1373 00:42:52,230 --> 00:42:54,790 1374 00:42:54,790 --> 00:42:57,589 1375 00:42:57,589 --> 00:43:00,069 1376 00:43:00,069 --> 00:43:01,750 1377 00:43:01,750 --> 00:43:01,760 1378 00:43:01,760 --> 00:43:02,790 1379 00:43:02,790 --> 00:43:02,800 1380 00:43:02,800 --> 00:43:04,790 1381 00:43:04,790 --> 00:43:07,190 1382 00:43:07,190 --> 00:43:09,349 1383 00:43:09,349 --> 00:43:10,950 1384 00:43:10,950 --> 00:43:13,589 1385 00:43:13,589 --> 00:43:15,510 1386 00:43:15,510 --> 00:43:18,150 1387 00:43:18,150 --> 00:43:18,160 1388 00:43:18,160 --> 00:43:21,990 1389 00:43:21,990 --> 00:43:22,000 1390 00:43:22,000 --> 00:43:23,829 1391 00:43:23,829 --> 00:43:25,270 1392 00:43:25,270 --> 00:43:26,470 1393 00:43:26,470 --> 00:43:29,190 1394 00:43:29,190 --> 00:43:32,230 1395 00:43:32,230 --> 00:43:32,240 1396 00:43:32,240 --> 00:43:34,630 1397 00:43:34,630 --> 00:43:35,589 1398 00:43:35,589 --> 00:43:37,109 1399 00:43:37,109 --> 00:43:39,510 1400 00:43:39,510 --> 00:43:42,710 1401 00:43:42,710 --> 00:43:44,550 1402 00:43:44,550 --> 00:43:47,349 1403 00:43:47,349 --> 00:43:49,349 1404 00:43:49,349 --> 00:43:51,750 1405 00:43:51,750 --> 00:43:53,510 1406 00:43:53,510 --> 00:43:56,470 1407 00:43:56,470 --> 00:43:58,230 1408 00:43:58,230 --> 00:44:00,630 1409 00:44:00,630 --> 00:44:02,630 1410 00:44:02,630 --> 00:44:02,640 1411 00:44:02,640 --> 00:44:05,190 1412 00:44:05,190 --> 00:44:06,710 1413 00:44:06,710 --> 00:44:08,870 1414 00:44:08,870 --> 00:44:12,870 1415 00:44:12,870 --> 00:44:15,109 1416 00:44:15,109 --> 00:44:19,030 1417 00:44:19,030 --> 00:44:21,670 1418 00:44:21,670 --> 00:44:26,309 1419 00:44:26,309 --> 00:44:27,670 1420 00:44:27,670 --> 00:44:31,670 1421 00:44:31,670 --> 00:44:33,550 1422 00:44:33,550 --> 00:44:36,069 1423 00:44:36,069 --> 00:44:38,710 1424 00:44:38,710 --> 00:44:40,630 1425 00:44:40,630 --> 00:44:42,630 1426 00:44:42,630 --> 00:44:45,030 1427 00:44:45,030 --> 00:44:49,190 1428 00:44:49,190 --> 00:44:50,710 1429 00:44:50,710 --> 00:44:52,150 1430 00:44:52,150 --> 00:44:55,109 1431 00:44:55,109 --> 00:44:57,030 1432 00:44:57,030 --> 00:45:00,550 1433 00:45:00,550 --> 00:45:01,750 1434 00:45:01,750 --> 00:45:03,670 1435 00:45:03,670 --> 00:45:06,550 1436 00:45:06,550 --> 00:45:08,309 1437 00:45:08,309 --> 00:45:10,069 1438 00:45:10,069 --> 00:45:12,309 1439 00:45:12,309 --> 00:45:13,750 1440 00:45:13,750 --> 00:45:15,990 1441 00:45:15,990 --> 00:45:16,000 1442 00:45:16,000 --> 00:45:17,030 1443 00:45:17,030 --> 00:45:18,710 1444 00:45:18,710 --> 00:45:20,870 1445 00:45:20,870 --> 00:45:23,829 1446 00:45:23,829 --> 00:45:23,839 1447 00:45:23,839 --> 00:45:25,270 1448 00:45:25,270 --> 00:45:27,910 1449 00:45:27,910 --> 00:45:27,920 1450 00:45:27,920 --> 00:45:32,950 1451 00:45:32,950 --> 00:45:35,109 1452 00:45:35,109 --> 00:45:36,230 1453 00:45:36,230 --> 00:45:37,589 1454 00:45:37,589 --> 00:45:39,990 1455 00:45:39,990 --> 00:45:42,150 1456 00:45:42,150 --> 00:45:43,990 1457 00:45:43,990 --> 00:45:47,270 1458 00:45:47,270 --> 00:45:48,390 1459 00:45:48,390 --> 00:45:50,309 1460 00:45:50,309 --> 00:45:51,589 1461 00:45:51,589 --> 00:45:57,030 1462 00:45:57,030 --> 00:45:58,550 1463 00:45:58,550 --> 00:46:00,710 1464 00:46:00,710 --> 00:46:02,710 1465 00:46:02,710 --> 00:46:02,720 1466 00:46:02,720 --> 00:46:04,069 1467 00:46:04,069 --> 00:46:04,079 1468 00:46:04,079 --> 00:46:05,109 1469 00:46:05,109 --> 00:46:07,030 1470 00:46:07,030 --> 00:46:11,510 1471 00:46:11,510 --> 00:46:15,670 1472 00:46:15,670 --> 00:46:17,589 1473 00:46:17,589 --> 00:46:33,190 1474 00:46:33,190 --> 00:46:33,200 1475 00:46:33,200 --> 00:46:34,630 1476 00:46:34,630 --> 00:46:37,020 1477 00:46:37,020 --> 00:46:37,030 1478 00:46:37,030 --> 00:46:40,870 1479 00:46:40,870 --> 00:46:43,190 1480 00:46:43,190 --> 00:46:43,200 1481 00:46:43,200 --> 00:46:44,230 1482 00:46:44,230 --> 00:46:45,670 1483 00:46:45,670 --> 00:46:47,020 1484 00:46:47,020 --> 00:46:47,030 1485 00:46:47,030 --> 00:46:49,670 1486 00:46:49,670 --> 00:46:50,870 1487 00:46:50,870 --> 00:46:52,950 1488 00:46:52,950 --> 00:46:54,470 1489 00:46:54,470 --> 00:46:56,950 1490 00:46:56,950 --> 00:47:00,309 1491 00:47:00,309 --> 00:47:02,069 1492 00:47:02,069 --> 00:47:04,550 1493 00:47:04,550 --> 00:47:06,309 1494 00:47:06,309 --> 00:47:09,750 1495 00:47:09,750 --> 00:47:12,069 1496 00:47:12,069 --> 00:47:14,230 1497 00:47:14,230 --> 00:47:17,349 1498 00:47:17,349 --> 00:47:19,829 1499 00:47:19,829 --> 00:47:23,430 1500 00:47:23,430 --> 00:47:25,829 1501 00:47:25,829 --> 00:47:27,829 1502 00:47:27,829 --> 00:47:29,430 1503 00:47:29,430 --> 00:47:31,670 1504 00:47:31,670 --> 00:47:33,430 1505 00:47:33,430 --> 00:47:33,440 1506 00:47:33,440 --> 00:47:36,309 1507 00:47:36,309 --> 00:47:38,549 1508 00:47:38,549 --> 00:47:42,390 1509 00:47:42,390 --> 00:47:43,589 1510 00:47:43,589 --> 00:47:45,589 1511 00:47:45,589 --> 00:47:48,230 1512 00:47:48,230 --> 00:47:49,990 1513 00:47:49,990 --> 00:47:50,000 1514 00:47:50,000 --> 00:47:50,829 1515 00:47:50,829 --> 00:47:50,839 1516 00:47:50,839 --> 00:47:52,470 1517 00:47:52,470 --> 00:47:53,910 1518 00:47:53,910 --> 00:47:55,829 1519 00:47:55,829 --> 00:47:57,670 1520 00:47:57,670 --> 00:47:59,430 1521 00:47:59,430 --> 00:48:01,190 1522 00:48:01,190 --> 00:48:04,710 1523 00:48:04,710 --> 00:48:06,950 1524 00:48:06,950 --> 00:48:06,960 1525 00:48:06,960 --> 00:48:07,829 1526 00:48:07,829 --> 00:48:09,829 1527 00:48:09,829 --> 00:48:11,910 1528 00:48:11,910 --> 00:48:14,390 1529 00:48:14,390 --> 00:48:17,190 1530 00:48:17,190 --> 00:48:17,200 1531 00:48:17,200 --> 00:48:18,069 1532 00:48:18,069 --> 00:48:20,870 1533 00:48:20,870 --> 00:48:22,390 1534 00:48:22,390 --> 00:48:26,150 1535 00:48:26,150 --> 00:48:28,630 1536 00:48:28,630 --> 00:48:30,150 1537 00:48:30,150 --> 00:48:32,790 1538 00:48:32,790 --> 00:48:34,150 1539 00:48:34,150 --> 00:48:35,589 1540 00:48:35,589 --> 00:48:38,230 1541 00:48:38,230 --> 00:48:42,309 1542 00:48:42,309 --> 00:48:45,750 1543 00:48:45,750 --> 00:48:45,760 1544 00:48:45,760 --> 00:48:46,870 1545 00:48:46,870 --> 00:48:49,589 1546 00:48:49,589 --> 00:48:50,710 1547 00:48:50,710 --> 00:48:53,109 1548 00:48:53,109 --> 00:48:59,270 1549 00:48:59,270 --> 00:49:01,349 1550 00:49:01,349 --> 00:49:03,190 1551 00:49:03,190 --> 00:49:05,109 1552 00:49:05,109 --> 00:49:06,790 1553 00:49:06,790 --> 00:49:09,589 1554 00:49:09,589 --> 00:49:12,150 1555 00:49:12,150 --> 00:49:14,150 1556 00:49:14,150 --> 00:49:16,950 1557 00:49:16,950 --> 00:49:18,630 1558 00:49:18,630 --> 00:49:21,670 1559 00:49:21,670 --> 00:49:24,230 1560 00:49:24,230 --> 00:49:26,390 1561 00:49:26,390 --> 00:49:29,190 1562 00:49:29,190 --> 00:49:30,470 1563 00:49:30,470 --> 00:49:30,480 1564 00:49:30,480 --> 00:49:31,910 1565 00:49:31,910 --> 00:49:37,270 1566 00:49:37,270 --> 00:49:39,349 1567 00:49:39,349 --> 00:49:39,359 1568 00:49:39,359 --> 00:49:40,470 1569 00:49:40,470 --> 00:49:41,510 1570 00:49:41,510 --> 00:49:41,520 1571 00:49:41,520 --> 00:49:42,950 1572 00:49:42,950 --> 00:49:46,069 1573 00:49:46,069 --> 00:49:48,309 1574 00:49:48,309 --> 00:49:50,710 1575 00:49:50,710 --> 00:49:50,720 1576 00:49:50,720 --> 00:49:51,589 1577 00:49:51,589 --> 00:49:53,910 1578 00:49:53,910 --> 00:49:55,670 1579 00:49:55,670 --> 00:49:58,150 1580 00:49:58,150 --> 00:49:59,510 1581 00:49:59,510 --> 00:50:00,950 1582 00:50:00,950 --> 00:50:04,790 1583 00:50:04,790 --> 00:50:06,230 1584 00:50:06,230 --> 00:50:06,240 1585 00:50:06,240 --> 00:50:08,710 1586 00:50:08,710 --> 00:50:11,510 1587 00:50:11,510 --> 00:50:14,870 1588 00:50:14,870 --> 00:50:14,880 1589 00:50:14,880 --> 00:50:16,950 1590 00:50:16,950 --> 00:50:18,549 1591 00:50:18,549 --> 00:50:22,150 1592 00:50:22,150 --> 00:50:22,160 1593 00:50:22,160 --> 00:50:22,870 1594 00:50:22,870 --> 00:50:27,109 1595 00:50:27,109 --> 00:50:29,430 1596 00:50:29,430 --> 00:50:31,750 1597 00:50:31,750 --> 00:50:34,069 1598 00:50:34,069 --> 00:50:39,510 1599 00:50:39,510 --> 00:50:39,520 1600 00:50:39,520 --> 00:50:40,710 1601 00:50:40,710 --> 00:50:43,270 1602 00:50:43,270 --> 00:50:43,280 1603 00:50:43,280 --> 00:50:44,870 1604 00:50:44,870 --> 00:50:47,589 1605 00:50:47,589 --> 00:50:49,109 1606 00:50:49,109 --> 00:50:51,349 1607 00:50:51,349 --> 00:50:51,359 1608 00:50:51,359 --> 00:50:53,510 1609 00:50:53,510 --> 00:50:54,870 1610 00:50:54,870 --> 00:50:56,309 1611 00:50:56,309 --> 00:50:59,270 1612 00:50:59,270 --> 00:51:01,109 1613 00:51:01,109 --> 00:51:02,630 1614 00:51:02,630 --> 00:51:02,640 1615 00:51:02,640 --> 00:51:04,549 1616 00:51:04,549 --> 00:51:06,630 1617 00:51:06,630 --> 00:51:08,549 1618 00:51:08,549 --> 00:51:09,990 1619 00:51:09,990 --> 00:51:11,910 1620 00:51:11,910 --> 00:51:13,430 1621 00:51:13,430 --> 00:51:16,950 1622 00:51:16,950 --> 00:51:20,950 1623 00:51:20,950 --> 00:51:22,390 1624 00:51:22,390 --> 00:51:24,150 1625 00:51:24,150 --> 00:51:26,470 1626 00:51:26,470 --> 00:51:28,470 1627 00:51:28,470 --> 00:51:31,349 1628 00:51:31,349 --> 00:51:31,359 1629 00:51:31,359 --> 00:51:33,109 1630 00:51:33,109 --> 00:51:34,470 1631 00:51:34,470 --> 00:51:36,950 1632 00:51:36,950 --> 00:51:39,109 1633 00:51:39,109 --> 00:51:41,349 1634 00:51:41,349 --> 00:51:43,589 1635 00:51:43,589 --> 00:51:45,510 1636 00:51:45,510 --> 00:51:47,270 1637 00:51:47,270 --> 00:51:47,280 1638 00:51:47,280 --> 00:51:49,109 1639 00:51:49,109 --> 00:51:49,119 1640 00:51:49,119 --> 00:51:50,390 1641 00:51:50,390 --> 00:51:52,390 1642 00:51:52,390 --> 00:51:54,470 1643 00:51:54,470 --> 00:51:54,480 1644 00:51:54,480 --> 00:51:58,309 1645 00:51:58,309 --> 00:52:01,109 1646 00:52:01,109 --> 00:52:02,470 1647 00:52:02,470 --> 00:52:05,190 1648 00:52:05,190 --> 00:52:09,270 1649 00:52:09,270 --> 00:52:12,309 1650 00:52:12,309 --> 00:52:15,270 1651 00:52:15,270 --> 00:52:15,280 1652 00:52:15,280 --> 00:52:17,190 1653 00:52:17,190 --> 00:52:19,829 1654 00:52:19,829 --> 00:52:21,270 1655 00:52:21,270 --> 00:52:21,280 1656 00:52:21,280 --> 00:52:22,630 1657 00:52:22,630 --> 00:52:28,230 1658 00:52:28,230 --> 00:52:31,670 1659 00:52:31,670 --> 00:52:31,680 1660 00:52:31,680 --> 00:52:35,270 1661 00:52:35,270 --> 00:52:35,280 1662 00:52:35,280 --> 00:52:38,230 1663 00:52:38,230 --> 00:52:41,910 1664 00:52:41,910 --> 00:52:44,150 1665 00:52:44,150 --> 00:52:46,390 1666 00:52:46,390 --> 00:52:48,470 1667 00:52:48,470 --> 00:52:51,589 1668 00:52:51,589 --> 00:52:53,030 1669 00:52:53,030 --> 00:52:54,549 1670 00:52:54,549 --> 00:52:55,910 1671 00:52:55,910 --> 00:52:57,430 1672 00:52:57,430 --> 00:52:57,440 1673 00:52:57,440 --> 00:52:58,390 1674 00:52:58,390 --> 00:53:02,150 1675 00:53:02,150 --> 00:53:04,630 1676 00:53:04,630 --> 00:53:06,230 1677 00:53:06,230 --> 00:53:06,240 1678 00:53:06,240 --> 00:53:07,109 1679 00:53:07,109 --> 00:53:10,710 1680 00:53:10,710 --> 00:53:10,720 1681 00:53:10,720 --> 00:53:11,510 1682 00:53:11,510 --> 00:53:11,520 1683 00:53:11,520 --> 00:53:13,030 1684 00:53:13,030 --> 00:53:14,950 1685 00:53:14,950 --> 00:53:17,109 1686 00:53:17,109 --> 00:53:18,710 1687 00:53:18,710 --> 00:53:20,470 1688 00:53:20,470 --> 00:53:22,870 1689 00:53:22,870 --> 00:53:25,270 1690 00:53:25,270 --> 00:53:28,230 1691 00:53:28,230 --> 00:53:28,240 1692 00:53:28,240 --> 00:53:29,109 1693 00:53:29,109 --> 00:53:29,119 1694 00:53:29,119 --> 00:53:31,190 1695 00:53:31,190 --> 00:53:32,870 1696 00:53:32,870 --> 00:53:36,150 1697 00:53:36,150 --> 00:53:38,390 1698 00:53:38,390 --> 00:53:39,829 1699 00:53:39,829 --> 00:53:42,390 1700 00:53:42,390 --> 00:53:42,400 1701 00:53:42,400 --> 00:53:43,670 1702 00:53:43,670 --> 00:53:43,680 1703 00:53:43,680 --> 00:53:44,790 1704 00:53:44,790 --> 00:53:46,549 1705 00:53:46,549 --> 00:53:48,870 1706 00:53:48,870 --> 00:53:48,880 1707 00:53:48,880 --> 00:53:50,150 1708 00:53:50,150 --> 00:53:52,950 1709 00:53:52,950 --> 00:53:52,960 1710 00:53:52,960 --> 00:53:53,829 1711 00:53:53,829 --> 00:53:55,510 1712 00:53:55,510 --> 00:53:57,349 1713 00:53:57,349 --> 00:54:00,710 1714 00:54:00,710 --> 00:54:02,710 1715 00:54:02,710 --> 00:54:05,670 1716 00:54:05,670 --> 00:54:07,190 1717 00:54:07,190 --> 00:54:07,200 1718 00:54:07,200 --> 00:54:08,829 1719 00:54:08,829 --> 00:54:11,430 1720 00:54:11,430 --> 00:54:12,870 1721 00:54:12,870 --> 00:54:15,990 1722 00:54:15,990 --> 00:54:17,589 1723 00:54:17,589 --> 00:54:19,430 1724 00:54:19,430 --> 00:54:19,440 1725 00:54:19,440 --> 00:54:20,230 1726 00:54:20,230 --> 00:54:22,829 1727 00:54:22,829 --> 00:54:25,109 1728 00:54:25,109 --> 00:54:27,910 1729 00:54:27,910 --> 00:54:27,920 1730 00:54:27,920 --> 00:54:28,950 1731 00:54:28,950 --> 00:54:28,960 1732 00:54:28,960 --> 00:54:30,069 1733 00:54:30,069 --> 00:54:33,349 1734 00:54:33,349 --> 00:54:35,109 1735 00:54:35,109 --> 00:54:35,119 1736 00:54:35,119 --> 00:54:37,109 1737 00:54:37,109 --> 00:54:40,549 1738 00:54:40,549 --> 00:54:43,109 1739 00:54:43,109 --> 00:54:44,549 1740 00:54:44,549 --> 00:54:46,630 1741 00:54:46,630 --> 00:54:50,390 1742 00:54:50,390 --> 00:54:51,829 1743 00:54:51,829 --> 00:54:53,430 1744 00:54:53,430 --> 00:54:54,789 1745 00:54:54,789 --> 00:54:58,230 1746 00:54:58,230 --> 00:54:59,670 1747 00:54:59,670 --> 00:55:01,109 1748 00:55:01,109 --> 00:55:02,870 1749 00:55:02,870 --> 00:55:04,870 1750 00:55:04,870 --> 00:55:07,430 1751 00:55:07,430 --> 00:55:08,789 1752 00:55:08,789 --> 00:55:10,309 1753 00:55:10,309 --> 00:55:12,549 1754 00:55:12,549 --> 00:55:15,030 1755 00:55:15,030 --> 00:55:16,870 1756 00:55:16,870 --> 00:55:19,030 1757 00:55:19,030 --> 00:55:19,040 1758 00:55:19,040 --> 00:55:21,910 1759 00:55:21,910 --> 00:55:21,920 1760 00:55:21,920 --> 00:55:23,670 1761 00:55:23,670 --> 00:55:25,589 1762 00:55:25,589 --> 00:55:27,430 1763 00:55:27,430 --> 00:55:29,430 1764 00:55:29,430 --> 00:55:31,430 1765 00:55:31,430 --> 00:55:33,589 1766 00:55:33,589 --> 00:55:35,190 1767 00:55:35,190 --> 00:55:38,789 1768 00:55:38,789 --> 00:55:38,799 1769 00:55:38,799 --> 00:55:39,589 1770 00:55:39,589 --> 00:55:39,599 1771 00:55:39,599 --> 00:55:40,870 1772 00:55:40,870 --> 00:55:43,670 1773 00:55:43,670 --> 00:55:45,430 1774 00:55:45,430 --> 00:55:48,870 1775 00:55:48,870 --> 00:55:51,829 1776 00:55:51,829 --> 00:55:55,910 1777 00:55:55,910 --> 00:55:57,750 1778 00:55:57,750 --> 00:55:57,760 1779 00:55:57,760 --> 00:55:58,630 1780 00:55:58,630 --> 00:55:58,640 1781 00:55:58,640 --> 00:55:59,829 1782 00:55:59,829 --> 00:56:03,829 1783 00:56:03,829 --> 00:56:05,510 1784 00:56:05,510 --> 00:56:07,910 1785 00:56:07,910 --> 00:56:10,150 1786 00:56:10,150 --> 00:56:11,990 1787 00:56:11,990 --> 00:56:14,789 1788 00:56:14,789 --> 00:56:17,430 1789 00:56:17,430 --> 00:56:19,510 1790 00:56:19,510 --> 00:56:19,520 1791 00:56:19,520 --> 00:56:22,549 1792 00:56:22,549 --> 00:56:23,990 1793 00:56:23,990 --> 00:56:26,069 1794 00:56:26,069 --> 00:56:27,430 1795 00:56:27,430 --> 00:56:30,230 1796 00:56:30,230 --> 00:56:32,549 1797 00:56:32,549 --> 00:56:32,559 1798 00:56:32,559 --> 00:56:33,750 1799 00:56:33,750 --> 00:56:37,589 1800 00:56:37,589 --> 00:56:39,030 1801 00:56:39,030 --> 00:56:41,109 1802 00:56:41,109 --> 00:56:43,349 1803 00:56:43,349 --> 00:56:47,190 1804 00:56:47,190 --> 00:56:49,109 1805 00:56:49,109 --> 00:56:50,549 1806 00:56:50,549 --> 00:56:51,990 1807 00:56:51,990 --> 00:56:53,349 1808 00:56:53,349 --> 00:56:53,359 1809 00:56:53,359 --> 00:56:55,510 1810 00:56:55,510 --> 00:56:55,520 1811 00:56:55,520 --> 00:56:57,030 1812 00:56:57,030 --> 00:56:59,030 1813 00:56:59,030 --> 00:56:59,040 1814 00:56:59,040 --> 00:57:00,470 1815 00:57:00,470 --> 00:57:01,990 1816 00:57:01,990 --> 00:57:03,910 1817 00:57:03,910 --> 00:57:05,109 1818 00:57:05,109 --> 00:57:06,789 1819 00:57:06,789 --> 00:57:08,630 1820 00:57:08,630 --> 00:57:10,549 1821 00:57:10,549 --> 00:57:12,630 1822 00:57:12,630 --> 00:57:17,190 1823 00:57:17,190 --> 00:57:17,200 1824 00:57:17,200 --> 00:57:18,309 1825 00:57:18,309 --> 00:57:19,990 1826 00:57:19,990 --> 00:57:21,990 1827 00:57:21,990 --> 00:57:23,829 1828 00:57:23,829 --> 00:57:26,470 1829 00:57:26,470 --> 00:57:28,069 1830 00:57:28,069 --> 00:57:30,069 1831 00:57:30,069 --> 00:57:34,789 1832 00:57:34,789 --> 00:57:36,630 1833 00:57:36,630 --> 00:57:37,910 1834 00:57:37,910 --> 00:57:40,950 1835 00:57:40,950 --> 00:57:42,710 1836 00:57:42,710 --> 00:57:44,630 1837 00:57:44,630 --> 00:57:46,230 1838 00:57:46,230 --> 00:57:47,910 1839 00:57:47,910 --> 00:57:51,990 1840 00:57:51,990 --> 00:57:54,789 1841 00:57:54,789 --> 00:57:57,109 1842 00:57:57,109 --> 00:57:59,990 1843 00:57:59,990 --> 00:58:02,230 1844 00:58:02,230 --> 00:58:06,069 1845 00:58:06,069 --> 00:58:07,990 1846 00:58:07,990 --> 00:58:09,750 1847 00:58:09,750 --> 00:58:12,230 1848 00:58:12,230 --> 00:58:16,069 1849 00:58:16,069 --> 00:58:18,309 1850 00:58:18,309 --> 00:58:18,319 1851 00:58:18,319 --> 00:58:19,270 1852 00:58:19,270 --> 00:58:21,670 1853 00:58:21,670 --> 00:58:24,390 1854 00:58:24,390 --> 00:58:26,390 1855 00:58:26,390 --> 00:58:29,030 1856 00:58:29,030 --> 00:58:29,040 1857 00:58:29,040 --> 00:58:30,150 1858 00:58:30,150 --> 00:58:33,030 1859 00:58:33,030 --> 00:58:33,040 1860 00:58:33,040 --> 00:58:34,230 1861 00:58:34,230 --> 00:58:37,670 1862 00:58:37,670 --> 00:58:39,990 1863 00:58:39,990 --> 00:58:40,000 1864 00:58:40,000 --> 00:58:41,270 1865 00:58:41,270 --> 00:58:43,750 1866 00:58:43,750 --> 00:58:46,150 1867 00:58:46,150 --> 00:58:48,789 1868 00:58:48,789 --> 00:58:51,670 1869 00:58:51,670 --> 00:58:53,910 1870 00:58:53,910 --> 00:58:56,549 1871 00:58:56,549 --> 00:58:58,870 1872 00:58:58,870 --> 00:59:00,950 1873 00:59:00,950 --> 00:59:04,069 1874 00:59:04,069 --> 00:59:07,750 1875 00:59:07,750 --> 00:59:09,109 1876 00:59:09,109 --> 00:59:10,630 1877 00:59:10,630 --> 00:59:12,309 1878 00:59:12,309 --> 00:59:13,829 1879 00:59:13,829 --> 00:59:15,750 1880 00:59:15,750 --> 00:59:18,630 1881 00:59:18,630 --> 00:59:20,950 1882 00:59:20,950 --> 00:59:23,430 1883 00:59:23,430 --> 00:59:24,789 1884 00:59:24,789 --> 00:59:26,870 1885 00:59:26,870 --> 00:59:28,870 1886 00:59:28,870 --> 00:59:30,309 1887 00:59:30,309 --> 00:59:33,030 1888 00:59:33,030 --> 00:59:33,990 1889 00:59:33,990 --> 00:59:36,309 1890 00:59:36,309 --> 00:59:38,870 1891 00:59:38,870 --> 00:59:40,630 1892 00:59:40,630 --> 00:59:42,789 1893 00:59:42,789 --> 00:59:44,630 1894 00:59:44,630 --> 00:59:44,640 1895 00:59:44,640 --> 00:59:45,750 1896 00:59:45,750 --> 00:59:47,910 1897 00:59:47,910 --> 00:59:49,270 1898 00:59:49,270 --> 00:59:52,150 1899 00:59:52,150 --> 00:59:55,750 1900 00:59:55,750 --> 00:59:59,670 1901 00:59:59,670 --> 01:00:02,870 1902 01:00:02,870 --> 01:00:05,109 1903 01:00:05,109 --> 01:00:06,309 1904 01:00:06,309 --> 01:00:07,829 1905 01:00:07,829 --> 01:00:10,230 1906 01:00:10,230 --> 01:00:12,309 1907 01:00:12,309 --> 01:00:14,789 1908 01:00:14,789 --> 01:00:19,589 1909 01:00:19,589 --> 01:00:21,670 1910 01:00:21,670 --> 01:00:23,190 1911 01:00:23,190 --> 01:00:24,789 1912 01:00:24,789 --> 01:00:26,309 1913 01:00:26,309 --> 01:00:26,319 1914 01:00:26,319 --> 01:00:31,270 1915 01:00:31,270 --> 01:00:33,430 1916 01:00:33,430 --> 01:00:35,030 1917 01:00:35,030 --> 01:00:36,710 1918 01:00:36,710 --> 01:00:39,829 1919 01:00:39,829 --> 01:00:41,990 1920 01:00:41,990 --> 01:00:44,630 1921 01:00:44,630 --> 01:00:51,670 1922 01:00:51,670 --> 01:00:53,270 1923 01:00:53,270 --> 01:00:57,109 1924 01:00:57,109 --> 01:00:57,119 1925 01:00:57,119 --> 01:00:58,150 1926 01:00:58,150 --> 01:00:59,990 1927 01:00:59,990 --> 01:01:03,510 1928 01:01:03,510 --> 01:01:05,430 1929 01:01:05,430 --> 01:01:06,470 1930 01:01:06,470 --> 01:01:06,480 1931 01:01:06,480 --> 01:01:07,430 1932 01:01:07,430 --> 01:01:09,430 1933 01:01:09,430 --> 01:01:12,150 1934 01:01:12,150 --> 01:01:15,190 1935 01:01:15,190 --> 01:01:18,390 1936 01:01:18,390 --> 01:01:19,589 1937 01:01:19,589 --> 01:01:21,430 1938 01:01:21,430 --> 01:01:23,670 1939 01:01:23,670 --> 01:01:25,750 1940 01:01:25,750 --> 01:01:28,390 1941 01:01:28,390 --> 01:01:28,400 1942 01:01:28,400 --> 01:01:29,510 1943 01:01:29,510 --> 01:01:29,520 1944 01:01:29,520 --> 01:01:30,549 1945 01:01:30,549 --> 01:01:32,150 1946 01:01:32,150 --> 01:01:34,470 1947 01:01:34,470 --> 01:01:36,710 1948 01:01:36,710 --> 01:01:39,750 1949 01:01:39,750 --> 01:01:42,950 1950 01:01:42,950 --> 01:01:44,230 1951 01:01:44,230 --> 01:01:45,990 1952 01:01:45,990 --> 01:01:48,710 1953 01:01:48,710 --> 01:01:53,030 1954 01:01:53,030 --> 01:01:55,270 1955 01:01:55,270 --> 01:01:57,430 1956 01:01:57,430 --> 01:01:58,950 1957 01:01:58,950 --> 01:02:03,510 1958 01:02:03,510 --> 01:02:05,349 1959 01:02:05,349 --> 01:02:07,029 1960 01:02:07,029 --> 01:02:08,710 1961 01:02:08,710 --> 01:02:10,789 1962 01:02:10,789 --> 01:02:12,710 1963 01:02:12,710 --> 01:02:15,270 1964 01:02:15,270 --> 01:02:15,280 1965 01:02:15,280 --> 01:02:15,990 1966 01:02:15,990 --> 01:02:17,750 1967 01:02:17,750 --> 01:02:20,150 1968 01:02:20,150 --> 01:02:21,270 1969 01:02:21,270 --> 01:02:23,029 1970 01:02:23,029 --> 01:02:24,710 1971 01:02:24,710 --> 01:02:28,069 1972 01:02:28,069 --> 01:02:29,829 1973 01:02:29,829 --> 01:02:31,990 1974 01:02:31,990 --> 01:02:33,750 1975 01:02:33,750 --> 01:02:36,390 1976 01:02:36,390 --> 01:02:39,990 1977 01:02:39,990 --> 01:02:40,000 1978 01:02:40,000 --> 01:02:41,109 1979 01:02:41,109 --> 01:02:43,190 1980 01:02:43,190 --> 01:02:44,870 1981 01:02:44,870 --> 01:02:47,990 1982 01:02:47,990 --> 01:02:50,950 1983 01:02:50,950 --> 01:02:53,430 1984 01:02:53,430 --> 01:02:55,670 1985 01:02:55,670 --> 01:02:57,589 1986 01:02:57,589 --> 01:03:00,870 1987 01:03:00,870 --> 01:03:00,880 1988 01:03:00,880 --> 01:03:02,390 1989 01:03:02,390 --> 01:03:02,400 1990 01:03:02,400 --> 01:03:03,589 1991 01:03:03,589 --> 01:03:05,430 1992 01:03:05,430 --> 01:03:08,789 1993 01:03:08,789 --> 01:03:08,799 1994 01:03:08,799 --> 01:03:09,670 1995 01:03:09,670 --> 01:03:13,990 1996 01:03:13,990 --> 01:03:15,829 1997 01:03:15,829 --> 01:03:19,190 1998 01:03:19,190 --> 01:03:19,200 1999 01:03:19,200 --> 01:03:21,910 2000 01:03:21,910 --> 01:03:21,920 2001 01:03:21,920 --> 01:03:23,190 2002 01:03:23,190 --> 01:03:30,789 2003 01:03:30,789 --> 01:03:32,470 2004 01:03:32,470 --> 01:03:34,150 2005 01:03:34,150 --> 01:03:37,430 2006 01:03:37,430 --> 01:03:37,440 2007 01:03:37,440 --> 01:03:38,630 2008 01:03:38,630 --> 01:03:38,640 2009 01:03:38,640 --> 01:03:40,630 2010 01:03:40,630 --> 01:03:42,390 2011 01:03:42,390 --> 01:03:46,309 2012 01:03:46,309 --> 01:03:48,230 2013 01:03:48,230 --> 01:03:50,390 2014 01:03:50,390 --> 01:03:51,510 2015 01:03:51,510 --> 01:03:53,349 2016 01:03:53,349 --> 01:03:53,359 2017 01:03:53,359 --> 01:03:54,390 2018 01:03:54,390 --> 01:03:54,400 2019 01:03:54,400 --> 01:03:55,270 2020 01:03:55,270 --> 01:03:57,910 2021 01:03:57,910 --> 01:04:01,190 2022 01:04:01,190 --> 01:04:03,510 2023 01:04:03,510 --> 01:04:06,549 2024 01:04:06,549 --> 01:04:08,870 2025 01:04:08,870 --> 01:04:11,029 2026 01:04:11,029 --> 01:04:12,870 2027 01:04:12,870 --> 01:04:15,589 2028 01:04:15,589 --> 01:04:18,470 2029 01:04:18,470 --> 01:04:21,109 2030 01:04:21,109 --> 01:04:24,390 2031 01:04:24,390 --> 01:04:24,400 2032 01:04:24,400 --> 01:04:25,589 2033 01:04:25,589 --> 01:04:25,599 2034 01:04:25,599 --> 01:04:27,349 2035 01:04:27,349 --> 01:04:29,589 2036 01:04:29,589 --> 01:04:31,430 2037 01:04:31,430 --> 01:04:33,430 2038 01:04:33,430 --> 01:04:34,710 2039 01:04:34,710 --> 01:04:35,829 2040 01:04:35,829 --> 01:04:37,670 2041 01:04:37,670 --> 01:04:37,680 2042 01:04:37,680 --> 01:04:38,470 2043 01:04:38,470 --> 01:04:40,710 2044 01:04:40,710 --> 01:04:43,910 2045 01:04:43,910 --> 01:04:46,069 2046 01:04:46,069 --> 01:04:48,789 2047 01:04:48,789 --> 01:04:51,109 2048 01:04:51,109 --> 01:04:53,670 2049 01:04:53,670 --> 01:04:56,150 2050 01:04:56,150 --> 01:04:58,390 2051 01:04:58,390 --> 01:05:00,710 2052 01:05:00,710 --> 01:05:04,789 2053 01:05:04,789 --> 01:05:05,829 2054 01:05:05,829 --> 01:05:07,029 2055 01:05:07,029 --> 01:05:09,109 2056 01:05:09,109 --> 01:05:10,309 2057 01:05:10,309 --> 01:05:12,549 2058 01:05:12,549 --> 01:05:14,470 2059 01:05:14,470 --> 01:05:15,349 2060 01:05:15,349 --> 01:05:18,230 2061 01:05:18,230 --> 01:05:20,069 2062 01:05:20,069 --> 01:05:21,430 2063 01:05:21,430 --> 01:05:23,750 2064 01:05:23,750 --> 01:05:26,150 2065 01:05:26,150 --> 01:05:30,309 2066 01:05:30,309 --> 01:05:30,319 2067 01:05:30,319 --> 01:05:31,190 2068 01:05:31,190 --> 01:05:33,750 2069 01:05:33,750 --> 01:05:35,190 2070 01:05:35,190 --> 01:05:37,109 2071 01:05:37,109 --> 01:05:41,589 2072 01:05:41,589 --> 01:05:45,109 2073 01:05:45,109 --> 01:05:47,990 2074 01:05:47,990 --> 01:05:52,069 2075 01:05:52,069 --> 01:05:54,870 2076 01:05:54,870 --> 01:05:56,470 2077 01:05:56,470 --> 01:05:58,630 2078 01:05:58,630 --> 01:06:00,630 2079 01:06:00,630 --> 01:06:02,069 2080 01:06:02,069 --> 01:06:05,430 2081 01:06:05,430 --> 01:06:07,589 2082 01:06:07,589 --> 01:06:13,589 2083 01:06:13,589 --> 01:06:13,599 2084 01:06:13,599 --> 01:06:15,190 2085 01:06:15,190 --> 01:06:19,190 2086 01:06:19,190 --> 01:06:19,200 2087 01:06:19,200 --> 01:06:24,950 2088 01:06:24,950 --> 01:06:27,190 2089 01:06:27,190 --> 01:06:29,829 2090 01:06:29,829 --> 01:06:31,510 2091 01:06:31,510 --> 01:06:33,270 2092 01:06:33,270 --> 01:06:34,710 2093 01:06:34,710 --> 01:06:36,309 2094 01:06:36,309 --> 01:06:37,990 2095 01:06:37,990 --> 01:06:46,630 2096 01:06:46,630 --> 01:06:48,309 2097 01:06:48,309 --> 01:06:50,309 2098 01:06:50,309 --> 01:06:52,230 2099 01:06:52,230 --> 01:06:52,240 2100 01:06:52,240 --> 01:06:53,990 2101 01:06:53,990 --> 01:06:54,000 2102 01:06:54,000 --> 01:06:55,910 2103 01:06:55,910 --> 01:06:55,920 2104 01:06:55,920 --> 01:06:56,950 2105 01:06:56,950 --> 01:06:58,870 2106 01:06:58,870 --> 01:07:01,349 2107 01:07:01,349 --> 01:07:01,359 2108 01:07:01,359 --> 01:07:05,190 2109 01:07:05,190 --> 01:07:05,200 2110 01:07:05,200 --> 01:07:06,390 2111 01:07:06,390 --> 01:07:09,190 2112 01:07:09,190 --> 01:07:09,200 2113 01:07:09,200 --> 01:07:10,069 2114 01:07:10,069 --> 01:07:11,670 2115 01:07:11,670 --> 01:07:13,990 2116 01:07:13,990 --> 01:07:15,910 2117 01:07:15,910 --> 01:07:17,510 2118 01:07:17,510 --> 01:07:19,109 2119 01:07:19,109 --> 01:07:20,309 2120 01:07:20,309 --> 01:07:20,319 2121 01:07:20,319 --> 01:07:22,630 2122 01:07:22,630 --> 01:07:24,230 2123 01:07:24,230 --> 01:07:25,750 2124 01:07:25,750 --> 01:07:28,069 2125 01:07:28,069 --> 01:07:29,190 2126 01:07:29,190 --> 01:07:30,549 2127 01:07:30,549 --> 01:07:32,789 2128 01:07:32,789 --> 01:07:32,799 2129 01:07:32,799 --> 01:07:34,630 2130 01:07:34,630 --> 01:07:34,640 2131 01:07:34,640 --> 01:07:36,549 2132 01:07:36,549 --> 01:07:38,309 2133 01:07:38,309 --> 01:07:39,670 2134 01:07:39,670 --> 01:07:41,270 2135 01:07:41,270 --> 01:07:42,549 2136 01:07:42,549 --> 01:07:43,829 2137 01:07:43,829 --> 01:07:45,670 2138 01:07:45,670 --> 01:07:45,680 2139 01:07:45,680 --> 01:07:47,109 2140 01:07:47,109 --> 01:07:48,789 2141 01:07:48,789 --> 01:07:50,870 2142 01:07:50,870 --> 01:07:52,630 2143 01:07:52,630 --> 01:07:52,640 2144 01:07:52,640 --> 01:07:57,510 2145 01:07:57,510 --> 01:07:59,109 2146 01:07:59,109 --> 01:08:00,789 2147 01:08:00,789 --> 01:08:02,789 2148 01:08:02,789 --> 01:08:07,109 2149 01:08:07,109 --> 01:08:08,950 2150 01:08:08,950 --> 01:08:11,349 2151 01:08:11,349 --> 01:08:13,190 2152 01:08:13,190 --> 01:08:13,200 2153 01:08:13,200 --> 01:08:14,390 2154 01:08:14,390 --> 01:08:17,349 2155 01:08:17,349 --> 01:08:20,309 2156 01:08:20,309 --> 01:08:22,870 2157 01:08:22,870 --> 01:08:25,590 2158 01:08:25,590 --> 01:08:28,229 2159 01:08:28,229 --> 01:08:30,709 2160 01:08:30,709 --> 01:08:32,550 2161 01:08:32,550 --> 01:08:35,030 2162 01:08:35,030 --> 01:08:36,550 2163 01:08:36,550 --> 01:08:38,950 2164 01:08:38,950 --> 01:08:40,550 2165 01:08:40,550 --> 01:08:40,560 2166 01:08:40,560 --> 01:08:42,390 2167 01:08:42,390 --> 01:08:42,400 2168 01:08:42,400 --> 01:08:43,430 2169 01:08:43,430 --> 01:08:44,870 2170 01:08:44,870 --> 01:08:46,709 2171 01:08:46,709 --> 01:08:46,719 2172 01:08:46,719 --> 01:08:47,669 2173 01:08:47,669 --> 01:08:47,679 2174 01:08:47,679 --> 01:08:48,789 2175 01:08:48,789 --> 01:08:50,309 2176 01:08:50,309 --> 01:08:52,229 2177 01:08:52,229 --> 01:08:54,309 2178 01:08:54,309 --> 01:08:56,070 2179 01:08:56,070 --> 01:08:56,080 2180 01:08:56,080 --> 01:08:56,870 2181 01:08:56,870 --> 01:08:58,789 2182 01:08:58,789 --> 01:09:00,070 2183 01:09:00,070 --> 01:09:02,630 2184 01:09:02,630 --> 01:09:05,430 2185 01:09:05,430 --> 01:09:07,749 2186 01:09:07,749 --> 01:09:11,430 2187 01:09:11,430 --> 01:09:12,789 2188 01:09:12,789 --> 01:09:12,799 2189 01:09:12,799 --> 01:09:13,590 2190 01:09:13,590 --> 01:09:13,600 2191 01:09:13,600 --> 01:09:15,510 2192 01:09:15,510 --> 01:09:17,829 2193 01:09:17,829 --> 01:09:20,390 2194 01:09:20,390 --> 01:09:20,400 2195 01:09:20,400 --> 01:09:21,349 2196 01:09:21,349 --> 01:09:25,669 2197 01:09:25,669 --> 01:09:27,269 2198 01:09:27,269 --> 01:09:30,390 2199 01:09:30,390 --> 01:09:33,189 2200 01:09:33,189 --> 01:09:36,950 2201 01:09:36,950 --> 01:09:39,189 2202 01:09:39,189 --> 01:09:41,990 2203 01:09:41,990 --> 01:09:44,630 2204 01:09:44,630 --> 01:09:46,390 2205 01:09:46,390 --> 01:09:48,470 2206 01:09:48,470 --> 01:09:48,480 2207 01:09:48,480 --> 01:09:50,309 2208 01:09:50,309 --> 01:09:52,309 2209 01:09:52,309 --> 01:09:54,229 2210 01:09:54,229 --> 01:09:55,669 2211 01:09:55,669 --> 01:09:58,070 2212 01:09:58,070 --> 01:10:01,350 2213 01:10:01,350 --> 01:10:02,550 2214 01:10:02,550 --> 01:10:03,830 2215 01:10:03,830 --> 01:10:05,910 2216 01:10:05,910 --> 01:10:07,430 2217 01:10:07,430 --> 01:10:07,440 2218 01:10:07,440 --> 01:10:11,030 2219 01:10:11,030 --> 01:10:13,590 2220 01:10:13,590 --> 01:10:16,390 2221 01:10:16,390 --> 01:10:18,790 2222 01:10:18,790 --> 01:10:21,350 2223 01:10:21,350 --> 01:10:22,310 2224 01:10:22,310 --> 01:10:24,149 2225 01:10:24,149 --> 01:10:26,790 2226 01:10:26,790 --> 01:10:26,800 2227 01:10:26,800 --> 01:10:27,669 2228 01:10:27,669 --> 01:10:29,110 2229 01:10:29,110 --> 01:10:30,550 2230 01:10:30,550 --> 01:10:30,560 2231 01:10:30,560 --> 01:10:31,510 2232 01:10:31,510 --> 01:10:31,520 2233 01:10:31,520 --> 01:10:32,550 2234 01:10:32,550 --> 01:10:35,189 2235 01:10:35,189 --> 01:10:36,470 2236 01:10:36,470 --> 01:10:40,550 2237 01:10:40,550 --> 01:10:42,470 2238 01:10:42,470 --> 01:10:45,910 2239 01:10:45,910 --> 01:10:47,750 2240 01:10:47,750 --> 01:10:50,550 2241 01:10:50,550 --> 01:10:50,560 2242 01:10:50,560 --> 01:10:51,350 2243 01:10:51,350 --> 01:10:53,110 2244 01:10:53,110 --> 01:10:57,189 2245 01:10:57,189 --> 01:11:01,830 2246 01:11:01,830 --> 01:11:01,840 2247 01:11:01,840 --> 01:11:03,830 2248 01:11:03,830 --> 01:11:03,840 2249 01:11:03,840 --> 01:11:08,870 2250 01:11:08,870 --> 01:11:10,550 2251 01:11:10,550 --> 01:11:13,350 2252 01:11:13,350 --> 01:11:14,709 2253 01:11:14,709 --> 01:11:16,709 2254 01:11:16,709 --> 01:11:19,830 2255 01:11:19,830 --> 01:11:21,830 2256 01:11:21,830 --> 01:11:24,070 2257 01:11:24,070 --> 01:11:26,390 2258 01:11:26,390 --> 01:11:29,669 2259 01:11:29,669 --> 01:11:32,070 2260 01:11:32,070 --> 01:11:32,080 2261 01:11:32,080 --> 01:11:33,430 2262 01:11:33,430 --> 01:11:35,669 2263 01:11:35,669 --> 01:11:38,709 2264 01:11:38,709 --> 01:11:41,189 2265 01:11:41,189 --> 01:11:44,229 2266 01:11:44,229 --> 01:11:46,390 2267 01:11:46,390 --> 01:11:48,630 2268 01:11:48,630 --> 01:11:50,310 2269 01:11:50,310 --> 01:11:53,430 2270 01:11:53,430 --> 01:11:54,790 2271 01:11:54,790 --> 01:11:56,950 2272 01:11:56,950 --> 01:11:57,910 2273 01:11:57,910 --> 01:12:00,790 2274 01:12:00,790 --> 01:12:03,189 2275 01:12:03,189 --> 01:12:06,550 2276 01:12:06,550 --> 01:12:06,560 2277 01:12:06,560 --> 01:12:07,270 2278 01:12:07,270 --> 01:12:10,070 2279 01:12:10,070 --> 01:12:12,229 2280 01:12:12,229 --> 01:12:14,149 2281 01:12:14,149 --> 01:12:16,790 2282 01:12:16,790 --> 01:12:18,709 2283 01:12:18,709 --> 01:12:20,870 2284 01:12:20,870 --> 01:12:23,030 2285 01:12:23,030 --> 01:12:25,430 2286 01:12:25,430 --> 01:12:27,110 2287 01:12:27,110 --> 01:12:29,750 2288 01:12:29,750 --> 01:12:31,590 2289 01:12:31,590 --> 01:12:33,510 2290 01:12:33,510 --> 01:12:35,990 2291 01:12:35,990 --> 01:12:38,870 2292 01:12:38,870 --> 01:12:38,880 2293 01:12:38,880 --> 01:12:42,870 2294 01:12:42,870 --> 01:12:42,880 2295 01:12:42,880 --> 01:12:48,830 2296 01:12:48,830 --> 01:12:51,590 2297 01:12:51,590 --> 01:12:54,470 2298 01:12:54,470 --> 01:12:56,870 2299 01:12:56,870 --> 01:12:59,430 2300 01:12:59,430 --> 01:13:01,110 2301 01:13:01,110 --> 01:13:02,390 2302 01:13:02,390 --> 01:13:03,750 2303 01:13:03,750 --> 01:13:03,760 2304 01:13:03,760 --> 01:13:04,390 2305 01:13:04,390 --> 01:13:06,709 2306 01:13:06,709 --> 01:13:08,630 2307 01:13:08,630 --> 01:13:10,870 2308 01:13:10,870 --> 01:13:16,630 2309 01:13:16,630 --> 01:13:18,709 2310 01:13:18,709 --> 01:13:19,990 2311 01:13:19,990 --> 01:13:21,350 2312 01:13:21,350 --> 01:13:23,830 2313 01:13:23,830 --> 01:13:25,110 2314 01:13:25,110 --> 01:13:27,270 2315 01:13:27,270 --> 01:13:29,350 2316 01:13:29,350 --> 01:13:30,830 2317 01:13:30,830 --> 01:13:33,110 2318 01:13:33,110 --> 01:13:35,669 2319 01:13:35,669 --> 01:13:38,470 2320 01:13:38,470 --> 01:13:40,390 2321 01:13:40,390 --> 01:13:42,870 2322 01:13:42,870 --> 01:13:45,430 2323 01:13:45,430 --> 01:13:48,950 2324 01:13:48,950 --> 01:13:50,310 2325 01:13:50,310 --> 01:13:52,390 2326 01:13:52,390 --> 01:13:54,870 2327 01:13:54,870 --> 01:13:56,709 2328 01:13:56,709 --> 01:13:59,830 2329 01:13:59,830 --> 01:14:01,590 2330 01:14:01,590 --> 01:14:03,510 2331 01:14:03,510 --> 01:14:05,270 2332 01:14:05,270 --> 01:14:05,280 2333 01:14:05,280 --> 01:14:06,310 2334 01:14:06,310 --> 01:14:07,510 2335 01:14:07,510 --> 01:14:09,590 2336 01:14:09,590 --> 01:14:29,590 2337 01:14:29,590 --> 01:14:29,600 2338 01:14:29,600 --> 01:14:31,679