1
00:00:18,540 --> 00:00:22,480
all right so change of gears today I'm
2
00:00:22,480 --> 00:00:24,070
gonna start talking about consistency in
3
00:00:24,070 --> 00:00:26,230
a replication and since we only have one
4
00:00:26,230 --> 00:00:28,330
hour I want to first give a very broad
5
00:00:28,330 --> 00:00:30,880
overview of where is this going what's
6
00:00:30,880 --> 00:00:33,989
the kind of stuff we want to do in there
7
00:00:33,989 --> 00:00:36,850
and also to paint some sort of a more
8
00:00:36,850 --> 00:00:39,250
realistic picture of what you should
9
00:00:39,250 --> 00:00:41,470
expect all right so now one of the
10
00:00:41,470 --> 00:00:42,610
things I've been saying for all the
11
00:00:42,610 --> 00:00:44,710
class is you're mostly talking about
12
00:00:44,710 --> 00:00:46,210
compromises when it comes to solutions
13
00:00:46,210 --> 00:00:47,650
and there is no such thing as a perfect
14
00:00:47,650 --> 00:00:49,300
solution it turns out that for
15
00:00:49,300 --> 00:00:51,070
consistency and replication that's taken
16
00:00:51,070 --> 00:00:55,030
to almost an extreme you might have some
17
00:00:55,030 --> 00:00:56,980
sort of a shred of a class of reasonably
18
00:00:56,980 --> 00:00:58,390
good solution in the other ones but for
19
00:00:58,390 --> 00:01:00,820
this one you gain on some and
20
00:01:00,820 --> 00:01:02,620
potentially lose massively on another
21
00:01:02,620 --> 00:01:07,270
part so when it comes to consistency and
22
00:01:07,270 --> 00:01:08,830
replication replication is not gonna be
23
00:01:08,830 --> 00:01:11,200
so bad in terms of what does it do but
24
00:01:11,200 --> 00:01:13,630
consistency is gonna be something that's
25
00:01:13,630 --> 00:01:18,130
truly awful in a fundamental way okay on
26
00:01:18,130 --> 00:01:19,810
portions we can kind of resolve the
27
00:01:19,810 --> 00:01:21,369
thing but there is almost no good
28
00:01:21,369 --> 00:01:24,940
solution if you want to have a solution
29
00:01:24,940 --> 00:01:26,950
that gets close to some kind of very
30
00:01:26,950 --> 00:01:28,210
nice semantics in terms of what it
31
00:01:28,210 --> 00:01:30,430
guarantees is disastrously inefficient
32
00:01:30,430 --> 00:01:31,600
right
33
00:01:31,600 --> 00:01:33,880
so if things get pushed to really to an
34
00:01:33,880 --> 00:01:35,140
extreme when it comes to consist
35
00:01:35,140 --> 00:01:37,450
especially consistency now replication
36
00:01:37,450 --> 00:01:39,340
depends depending on how tight it is
37
00:01:39,340 --> 00:01:41,890
connected to consistency can be anywhere
38
00:01:41,890 --> 00:01:44,830
for from yeah no big deal let's just do
39
00:01:44,830 --> 00:01:47,110
it - oops it's about as hard as
40
00:01:47,110 --> 00:01:51,460
consistency okay so I will talk first
41
00:01:51,460 --> 00:01:52,960
about replication because it's easier to
42
00:01:52,960 --> 00:01:54,369
talk about the replication but keep on
43
00:01:54,369 --> 00:01:55,900
pointing out that if you want this
44
00:01:55,900 --> 00:01:57,430
property to hold you're in the land of
45
00:01:57,430 --> 00:02:00,850
consistency okay let me say a few words
46
00:02:00,850 --> 00:02:02,890
about consistency first okay so the
47
00:02:02,890 --> 00:02:05,290
trouble with consistency when the first
48
00:02:05,290 --> 00:02:08,169
problem with consistency is in fact the
49
00:02:08,169 --> 00:02:11,769
definition okay it's the fact that it's
50
00:02:11,769 --> 00:02:13,680
very hard to come up with an actual
51
00:02:13,680 --> 00:02:16,630
definition that we are really really
52
00:02:16,630 --> 00:02:18,130
really sure that that's the definition
53
00:02:18,130 --> 00:02:19,510
of consistency and I want you to
54
00:02:19,510 --> 00:02:21,190
understand this as I explain a lot more
55
00:02:21,190 --> 00:02:22,900
in a database class about this thing
56
00:02:22,900 --> 00:02:25,450
right and I want you to understand from
57
00:02:25,450 --> 00:02:27,400
the beginning you can do write a million
58
00:02:27,400 --> 00:02:29,650
papers about consistency but ultimately
59
00:02:29,650 --> 00:02:30,940
all of them are built on top of
60
00:02:30,940 --> 00:02:32,260
something and that something might be
61
00:02:32,260 --> 00:02:35,200
very shaky to define okay so I'll go
62
00:02:35,200 --> 00:02:36,430
into some details to explain what that
63
00:02:36,430 --> 00:02:37,120
means
64
00:02:37,120 --> 00:02:44,830
okay so consistency is related to the
65
00:02:44,830 --> 00:02:47,409
notion of correctness because if you
66
00:02:47,409 --> 00:02:48,670
don't have a notion of correctness it
67
00:02:48,670 --> 00:02:50,409
makes no sense to say I maintain some
68
00:02:50,409 --> 00:02:51,700
correctness consistency it's about
69
00:02:51,700 --> 00:02:53,920
maintaining some sort of correctness
70
00:02:53,920 --> 00:02:55,540
when you go and you do distributed
71
00:02:55,540 --> 00:02:57,940
systems versus something else but the
72
00:02:57,940 --> 00:02:59,319
trouble in the first place is the notion
73
00:02:59,319 --> 00:03:06,069
of correctness all right so I can define
74
00:03:06,069 --> 00:03:08,049
consistency with respect to correctness
75
00:03:08,049 --> 00:03:10,480
but then how do i define correctness so
76
00:03:10,480 --> 00:03:12,400
let's take an example for example I mean
77
00:03:12,400 --> 00:03:14,440
just to start with you write the program
78
00:03:14,440 --> 00:03:16,030
I ask you to do a task you write a
79
00:03:16,030 --> 00:03:17,470
program how do I know your program is
80
00:03:17,470 --> 00:03:17,859
correct
81
00:03:17,859 --> 00:03:24,220
I mean who says your program is correct
82
00:03:24,220 --> 00:03:26,440
by the way judging by the number of
83
00:03:26,440 --> 00:03:29,200
defects bugs whatever in software in
84
00:03:29,200 --> 00:03:31,900
general even when everybody's convinced
85
00:03:31,900 --> 00:03:33,129
something is correct you still discover
86
00:03:33,129 --> 00:03:34,720
that something is not correct because of
87
00:03:34,720 --> 00:03:37,750
bizarre things right so correctness of
88
00:03:37,750 --> 00:03:39,160
programs I want you to understand
89
00:03:39,160 --> 00:03:41,319
correctness of programs is really into
90
00:03:41,319 --> 00:03:42,849
the fabric of the notion of consistency
91
00:03:42,849 --> 00:03:44,919
and that in itself turns out to be
92
00:03:44,919 --> 00:03:46,810
extremely tricky I mean really really
93
00:03:46,810 --> 00:03:49,540
tricky by the way we do have an idea how
94
00:03:49,540 --> 00:03:51,730
to tackle correctness of programs it's
95
00:03:51,730 --> 00:03:54,340
only devastatingly bad for the
96
00:03:54,340 --> 00:03:57,879
programmer right so you can actually
97
00:03:57,879 --> 00:03:59,650
write programs that are provably correct
98
00:03:59,650 --> 00:04:03,730
but to do so I mean let's see this is
99
00:04:03,730 --> 00:04:08,040
kind of a complete shoot in the dark but
100
00:04:08,040 --> 00:04:10,569
right how many of you heard ever about
101
00:04:10,569 --> 00:04:14,260
proof carrying code proof carrying code
102
00:04:14,260 --> 00:04:16,510
is a piece of software that you wrote
103
00:04:16,510 --> 00:04:19,029
that contains annotations that prove
104
00:04:19,029 --> 00:04:21,728
that the program is correct right to
105
00:04:21,728 --> 00:04:23,020
prove that the program is a program is
106
00:04:23,020 --> 00:04:24,520
correct you have to prove that it
107
00:04:24,520 --> 00:04:25,930
produces the right output given the
108
00:04:25,930 --> 00:04:28,270
right input and you cannot run amok you
109
00:04:28,270 --> 00:04:30,550
get into subtle things like you never
110
00:04:30,550 --> 00:04:31,930
access memory you're not supposed to
111
00:04:31,930 --> 00:04:33,490
access how would you prove that how
112
00:04:33,490 --> 00:04:34,900
would you do this kind of things in
113
00:04:34,900 --> 00:04:37,300
languages right people spend a large
114
00:04:37,300 --> 00:04:39,190
amount of effort in trying to nail this
115
00:04:39,190 --> 00:04:40,120
down okay
116
00:04:40,120 --> 00:04:41,380
and they did come up with some
117
00:04:41,380 --> 00:04:44,139
mechanisms to do so but again
118
00:04:44,139 --> 00:04:46,780
you can write programs that I'm 10 lines
119
00:04:46,780 --> 00:04:48,220
long for which you can prove they are
120
00:04:48,220 --> 00:04:49,840
correct when I was a PhD student at
121
00:04:49,840 --> 00:04:51,280
Cornell for example we got one of those
122
00:04:51,280 --> 00:04:52,569
right so those guys were crazy about
123
00:04:52,569 --> 00:04:54,759
proof carrying code and no notion of
124
00:04:54,759 --> 00:04:56,590
correctness so in the qualifying exams
125
00:04:56,590 --> 00:04:58,840
for the PhD they threw a problem in
126
00:04:58,840 --> 00:05:00,520
right and that said write this program
127
00:05:00,520 --> 00:05:03,550
and prove it it's correct I spend a good
128
00:05:03,550 --> 00:05:06,250
part of half an hour to write five lines
129
00:05:06,250 --> 00:05:07,780
of code and then to prove the five lines
130
00:05:07,780 --> 00:05:09,370
of code were correct good luck in doing
131
00:05:09,370 --> 00:05:10,750
that on industrial scale when you're
132
00:05:10,750 --> 00:05:13,900
doing code design right so keep this in
133
00:05:13,900 --> 00:05:16,330
mind and this is kind of the ultimate
134
00:05:16,330 --> 00:05:18,699
way to say yeah forget about whatever
135
00:05:18,699 --> 00:05:20,440
super-strong type of consistency you
136
00:05:20,440 --> 00:05:21,610
don't even know if your program is
137
00:05:21,610 --> 00:05:24,550
correct so the traditional way to go
138
00:05:24,550 --> 00:05:26,500
around this problem in the first place
139
00:05:26,500 --> 00:05:28,389
is to hang on to some notion of
140
00:05:28,389 --> 00:05:29,889
correctness that's more likely to be
141
00:05:29,889 --> 00:05:31,300
correct another notion of correctness
142
00:05:31,300 --> 00:05:34,030
namely you say let's assume that the
143
00:05:34,030 --> 00:05:35,529
sequential version of the program is
144
00:05:35,529 --> 00:05:38,940
correct right let's now define
145
00:05:38,940 --> 00:05:41,580
correctness for distributed versions
146
00:05:41,580 --> 00:05:43,509
multi-threaded or multi computer or
147
00:05:43,509 --> 00:05:45,159
whatever with respect to the correctness
148
00:05:45,159 --> 00:05:47,110
of the sequential one now by the way
149
00:05:47,110 --> 00:05:49,960
this is the attitude and the approach
150
00:05:49,960 --> 00:05:52,349
taken in the database literature and
151
00:05:52,349 --> 00:05:54,370
database people are the first ones that
152
00:05:54,370 --> 00:05:55,779
actually looked at this notion of
153
00:05:55,779 --> 00:05:57,190
consistency it doesn't matter what the
154
00:05:57,190 --> 00:05:58,839
distributed systems people are talking
155
00:05:58,839 --> 00:06:02,560
about all right why because banks were
156
00:06:02,560 --> 00:06:03,849
extremely interested in relational
157
00:06:03,849 --> 00:06:05,919
databases and they are extremely
158
00:06:05,919 --> 00:06:08,080
concerned about some sort of guarantees
159
00:06:08,080 --> 00:06:10,150
that the distributive version of the
160
00:06:10,150 --> 00:06:11,800
program the multi-threaded multi
161
00:06:11,800 --> 00:06:14,979
anything you want doesn't run amok but
162
00:06:14,979 --> 00:06:16,569
again it's with respect to that
163
00:06:16,569 --> 00:06:19,930
sequential correctness right so one way
164
00:06:19,930 --> 00:06:21,940
to define consistency and I'm gonna come
165
00:06:21,940 --> 00:06:23,949
back to this later is to say whatever
166
00:06:23,949 --> 00:06:26,699
you do make sure in some sense is
167
00:06:26,699 --> 00:06:28,779
equivalent to the sequential program
168
00:06:28,779 --> 00:06:31,479
okay but of course you might have
169
00:06:31,479 --> 00:06:33,310
multiple sub sequential programs running
170
00:06:33,310 --> 00:06:36,430
right and in principle you have to
171
00:06:36,430 --> 00:06:37,990
overlap the execution of such sequential
172
00:06:37,990 --> 00:06:39,610
programs so when you define consistence
173
00:06:39,610 --> 00:06:40,599
you have to say not only that's
174
00:06:40,599 --> 00:06:41,740
equivalent to the sequential execution
175
00:06:41,740 --> 00:06:44,500
but it's also equivalent to a sequential
176
00:06:44,500 --> 00:06:47,620
execution of the sequential programs so
177
00:06:47,620 --> 00:06:49,259
you take your sequential programs you
178
00:06:49,259 --> 00:06:51,399
shuffle them in some order and you run
179
00:06:51,399 --> 00:06:53,380
them one by one now nobody says you
180
00:06:53,380 --> 00:06:55,419
should run it like this but the one way
181
00:06:55,419 --> 00:06:55,820
to do
182
00:06:55,820 --> 00:06:58,610
consistency would be make sure that
183
00:06:58,610 --> 00:07:00,680
you're consistent with at least one of
184
00:07:00,680 --> 00:07:03,110
this possible orders okay so things what
185
00:07:03,110 --> 00:07:05,480
I'm trying to say is when consistency
186
00:07:05,480 --> 00:07:08,990
its involved you have extremely complex
187
00:07:08,990 --> 00:07:11,390
issues popping up and there cannot be
188
00:07:11,390 --> 00:07:14,980
resolved completely okay now throughout
189
00:07:14,980 --> 00:07:17,540
these lectures it's virtually on Tuesday
190
00:07:17,540 --> 00:07:18,770
I'm going to talk about all kinds of
191
00:07:18,770 --> 00:07:20,720
notions of consistency that were
192
00:07:20,720 --> 00:07:21,770
introduced especially in the
193
00:07:21,770 --> 00:07:24,020
distribution systems literature all of
194
00:07:24,020 --> 00:07:26,600
them are iffy okay all of them are
195
00:07:26,600 --> 00:07:28,940
highly problematic there's no such thing
196
00:07:28,940 --> 00:07:30,560
as completely clean ones
197
00:07:30,560 --> 00:07:32,180
database people nailed it down a little
198
00:07:32,180 --> 00:07:33,590
bit better because they have a
199
00:07:33,590 --> 00:07:36,260
connection with this kind of a notion of
200
00:07:36,260 --> 00:07:37,690
sequential consistency where of course
201
00:07:37,690 --> 00:07:39,470
distributed systems people said hey
202
00:07:39,470 --> 00:07:40,970
that's a solved problem in databases
203
00:07:40,970 --> 00:07:44,900
which it is let's invent other kind of
204
00:07:44,900 --> 00:07:46,430
problems and let's try to solve those as
205
00:07:46,430 --> 00:07:48,320
well okay so I'll plow through all of
206
00:07:48,320 --> 00:07:49,580
these things at least at the high level
207
00:07:49,580 --> 00:07:51,920
let's go back and focus on replication
208
00:07:51,920 --> 00:07:54,050
because this is a lot easier to
209
00:07:54,050 --> 00:07:55,970
understand what what does he do and what
210
00:07:55,970 --> 00:07:57,380
is it supposed to do and so on and
211
00:07:57,380 --> 00:08:01,270
sometimes replication it's easy okay
212
00:08:01,270 --> 00:08:04,160
okay so when it comes to I mean
213
00:08:04,160 --> 00:08:05,720
replication is basically what the name
214
00:08:05,720 --> 00:08:09,050
suggests let's take the data and put it
215
00:08:09,050 --> 00:08:10,610
in multiple places now of course the
216
00:08:10,610 --> 00:08:13,070
question is why I'm sure there is a
217
00:08:13,070 --> 00:08:14,360
slide here but let's forget about this
218
00:08:14,360 --> 00:08:17,900
or let's just brainstorm about this why
219
00:08:17,900 --> 00:08:22,460
would i replicate the data that's for
220
00:08:22,460 --> 00:08:26,750
reasons out in there right one of them
221
00:08:26,750 --> 00:08:29,270
is have multiple copies in case you lose
222
00:08:29,270 --> 00:08:31,100
some okay so this is some sort of a
223
00:08:31,100 --> 00:08:33,200
backup right so one reason to do
224
00:08:33,200 --> 00:08:34,640
replication is to have a backup copy
225
00:08:34,640 --> 00:08:38,990
that's very legitimate reason now what's
226
00:08:38,990 --> 00:08:41,090
the difficulty to doing this kind of
227
00:08:41,090 --> 00:08:45,100
backups what could be the difficult I
228
00:08:45,100 --> 00:08:47,480
mean so one is headaches remembering
229
00:08:47,480 --> 00:08:49,100
where your backup copies are let's say
230
00:08:49,100 --> 00:08:54,440
okay that's one one issue I'm sorry well
231
00:08:54,440 --> 00:08:56,180
inconsistent data first of all what does
232
00:08:56,180 --> 00:08:57,890
inconsistent mean and this is very very
233
00:08:57,890 --> 00:09:01,490
tricky right but if I do it for the
234
00:09:01,490 --> 00:09:02,810
purpose of backup am I not care that
235
00:09:02,810 --> 00:09:06,050
they are in sync or was a question is
236
00:09:06,050 --> 00:09:07,910
when do I fire up the replication
237
00:09:07,910 --> 00:09:09,800
process right so when do I
238
00:09:09,800 --> 00:09:11,570
create a copy so if I want a replication
239
00:09:11,570 --> 00:09:13,430
for the purpose of backup the question
240
00:09:13,430 --> 00:09:17,990
is when do i propagate changes right so
241
00:09:17,990 --> 00:09:19,700
replication without changes is truly
242
00:09:19,700 --> 00:09:23,360
easy if you have a very slow changing
243
00:09:23,360 --> 00:09:26,329
resource of any kind replication it's a
244
00:09:26,329 --> 00:09:28,370
walk in the park you just whatever do it
245
00:09:28,370 --> 00:09:31,459
okay if you have an extremely fast
246
00:09:31,459 --> 00:09:33,800
changing resource replication becomes a
247
00:09:33,800 --> 00:09:36,829
nightmare for example imagine a database
248
00:09:36,829 --> 00:09:39,920
that processes hundreds of thousands of
249
00:09:39,920 --> 00:09:41,600
transactions per second and we have
250
00:09:41,600 --> 00:09:43,970
those databases by the way for example
251
00:09:43,970 --> 00:09:46,820
visa has no clearly all the students
252
00:09:46,820 --> 00:09:47,899
actually have to remember what they were
253
00:09:47,899 --> 00:09:49,339
how they did and how they change the
254
00:09:49,339 --> 00:09:51,829
state of the system so replication there
255
00:09:51,829 --> 00:09:54,589
is much much much harder than are you
256
00:09:54,589 --> 00:09:57,230
backing up I don't know your term papers
257
00:09:57,230 --> 00:09:57,880
or whatnot
258
00:09:57,880 --> 00:10:01,880
for example right again it depends on
259
00:10:01,880 --> 00:10:06,290
how fast the let's say the primary copy
260
00:10:06,290 --> 00:10:07,579
is changing okay
261
00:10:07,579 --> 00:10:10,760
so again this is what I'm saying that
262
00:10:10,760 --> 00:10:12,140
replication could be easy or could be
263
00:10:12,140 --> 00:10:13,579
extremely hard for example how do you
264
00:10:13,579 --> 00:10:14,750
think they do replication in that
265
00:10:14,750 --> 00:10:16,490
database they literally have to probably
266
00:10:16,490 --> 00:10:18,110
remember the sequence of actions to be
267
00:10:18,110 --> 00:10:19,940
able to apply them again all right and
268
00:10:19,940 --> 00:10:21,589
it's very tricky but the database
269
00:10:21,589 --> 00:10:23,060
community did deal with that right
270
00:10:23,060 --> 00:10:26,000
techniques like for example logs in
271
00:10:26,000 --> 00:10:27,709
which you simply write what is it that
272
00:10:27,709 --> 00:10:29,420
you were doing so you don't have a copy
273
00:10:29,420 --> 00:10:31,370
per se you have a sequence of actions
274
00:10:31,370 --> 00:10:32,899
that will produce a copy so do you need
275
00:10:32,899 --> 00:10:35,600
one so what a copy means and what you
276
00:10:35,600 --> 00:10:37,820
replicate it's kind of in the eye of the
277
00:10:37,820 --> 00:10:40,910
designer not necessarily beholder so you
278
00:10:40,910 --> 00:10:42,709
don't necessarily need a copy you don't
279
00:10:42,709 --> 00:10:45,770
need to say stop and copy you can have
280
00:10:45,770 --> 00:10:48,050
anything that could lead with lyrtle
281
00:10:48,050 --> 00:10:51,020
processing to a copy now a fundamental
282
00:10:51,020 --> 00:10:53,480
question about the replication is how
283
00:10:53,480 --> 00:10:56,029
often will you go and use the replicas
284
00:10:56,029 --> 00:10:58,070
to do something ok so when it comes to
285
00:10:58,070 --> 00:11:01,070
backups the answer is almost never right
286
00:11:01,070 --> 00:11:04,250
most of the time you don't care that the
287
00:11:04,250 --> 00:11:06,800
copy even exists only when by mistake
288
00:11:06,800 --> 00:11:07,940
you delete your files and you do
289
00:11:07,940 --> 00:11:09,620
something you go and recover from there
290
00:11:09,620 --> 00:11:10,820
from the backups for example I don't
291
00:11:10,820 --> 00:11:13,399
remember every ever going back to a
292
00:11:13,399 --> 00:11:14,990
backup and doing something bad hey you
293
00:11:14,990 --> 00:11:16,730
still need to do the backups because
294
00:11:16,730 --> 00:11:18,079
you're gonna run in trouble sooner or
295
00:11:18,079 --> 00:11:18,350
later
296
00:11:18,350 --> 00:11:20,990
supposedly okay all right so that's one
297
00:11:20,990 --> 00:11:22,810
reason to do replication not to move in
298
00:11:22,810 --> 00:11:25,000
formation and then the main question is
299
00:11:25,000 --> 00:11:26,860
how often do you replicate because
300
00:11:26,860 --> 00:11:28,420
whatever you did not replicate you're
301
00:11:28,420 --> 00:11:29,770
gonna lose in case of some sort of
302
00:11:29,770 --> 00:11:31,090
failure right so when it comes to
303
00:11:31,090 --> 00:11:33,340
backups common practice is to replicate
304
00:11:33,340 --> 00:11:35,710
every day for example of course if it
305
00:11:35,710 --> 00:11:36,970
changed if you didn't change again
306
00:11:36,970 --> 00:11:38,529
you're out of the woods you're fine
307
00:11:38,529 --> 00:11:40,120
what's what's another reason to
308
00:11:40,120 --> 00:11:42,839
replicate yes
309
00:11:48,170 --> 00:11:50,660
right so performance so another reason
310
00:11:50,660 --> 00:11:53,390
to replicate is performance and this is
311
00:11:53,390 --> 00:11:55,400
exactly what Akamai is doing right so
312
00:11:55,400 --> 00:11:56,870
you pay a lot of money as a company to
313
00:11:56,870 --> 00:11:59,320
Akamai because they have really clever
314
00:11:59,320 --> 00:12:01,790
replication algorithms and they are in
315
00:12:01,790 --> 00:12:04,820
some sense geographical replication
316
00:12:04,820 --> 00:12:07,100
algorithms under the assumption that of
317
00:12:07,100 --> 00:12:08,480
course I'm gonna access our closer
318
00:12:08,480 --> 00:12:10,910
resource faster than another far away
319
00:12:10,910 --> 00:12:13,280
resource and keep in mind that one of
320
00:12:13,280 --> 00:12:15,080
the big big problems and being enemies
321
00:12:15,080 --> 00:12:17,330
in distributed systems is latency you
322
00:12:17,330 --> 00:12:20,630
see we can deal with a lot of other
323
00:12:20,630 --> 00:12:22,220
problems right for example if you don't
324
00:12:22,220 --> 00:12:23,930
have enough bandwidth we know how to
325
00:12:23,930 --> 00:12:25,550
increase the bandwidth we put more money
326
00:12:25,550 --> 00:12:26,870
into it and we increase the bandwidth
327
00:12:26,870 --> 00:12:29,870
okay this is why the Internet gets more
328
00:12:29,870 --> 00:12:31,250
and more and more and more bandwidth but
329
00:12:31,250 --> 00:12:32,660
the internet doesn't get lower and lower
330
00:12:32,660 --> 00:12:36,380
and lower latency right so latency it's
331
00:12:36,380 --> 00:12:38,270
a physical limit we don't know how to go
332
00:12:38,270 --> 00:12:42,380
beyond I mean one way so let's say we
333
00:12:42,380 --> 00:12:44,030
maxed out how much information we can
334
00:12:44,030 --> 00:12:47,090
send through one optic fiber which we
335
00:12:47,090 --> 00:12:48,380
haven't yet by the way
336
00:12:48,380 --> 00:12:49,820
and some people compute the limit to be
337
00:12:49,820 --> 00:12:52,070
something like 40,000 times larger than
338
00:12:52,070 --> 00:12:54,050
it's actually accomplished now so let's
339
00:12:54,050 --> 00:12:55,760
say you even max that out hey you put
340
00:12:55,760 --> 00:12:57,350
more cables through the ocean and you
341
00:12:57,350 --> 00:12:59,360
get more bandwidth but that's not the
342
00:12:59,360 --> 00:13:06,590
case for for latency right so PBS yes
343
00:13:06,590 --> 00:13:08,600
they had a documentary about speed and
344
00:13:08,600 --> 00:13:10,070
part of the speed was internet speed
345
00:13:10,070 --> 00:13:12,200
right I gave you the example with they
346
00:13:12,200 --> 00:13:14,330
want to put between London and New York
347
00:13:14,330 --> 00:13:16,330
that cable for the ocean that straighter
348
00:13:16,330 --> 00:13:18,680
okay so I've heard another crazy idea
349
00:13:18,680 --> 00:13:20,390
yesterday in the documentary and this is
350
00:13:20,390 --> 00:13:22,160
all about shading a little bit of
351
00:13:22,160 --> 00:13:24,710
latency right so apparently a company
352
00:13:24,710 --> 00:13:27,050
and it kind of failed wanted to put a
353
00:13:27,050 --> 00:13:30,020
straighter fibre between Chicago and New
354
00:13:30,020 --> 00:13:31,640
York why because you have companies
355
00:13:31,640 --> 00:13:33,080
trading on both the Chicago Stock
356
00:13:33,080 --> 00:13:35,170
Exchange and the New York Stock Exchange
357
00:13:35,170 --> 00:13:37,010
whenever you have the same thing
358
00:13:37,010 --> 00:13:39,170
happening in two places whoever gets the
359
00:13:39,170 --> 00:13:40,520
information across that something
360
00:13:40,520 --> 00:13:42,080
changed will win will make money
361
00:13:42,080 --> 00:13:44,120
these guys make monies on fractions of a
362
00:13:44,120 --> 00:13:45,980
penny just because they do a tremendous
363
00:13:45,980 --> 00:13:47,420
number of transactions okay
364
00:13:47,420 --> 00:13:49,790
so it's a game of fractions of pennies
365
00:13:49,790 --> 00:13:51,770
but just doing many many many of those
366
00:13:51,770 --> 00:13:53,570
right so they were interviewing these
367
00:13:53,570 --> 00:13:57,200
guys that are basically responsible for
368
00:13:57,200 --> 00:14:01,130
1% of all the trades on New York Stock
369
00:14:01,130 --> 00:14:01,580
Exchange
370
00:14:01,580 --> 00:14:03,140
something like that right and they are
371
00:14:03,140 --> 00:14:04,730
all kind of just babysitting those
372
00:14:04,730 --> 00:14:07,370
millisecond trading machines and in that
373
00:14:07,370 --> 00:14:09,080
and they were talking about the fact
374
00:14:09,080 --> 00:14:10,490
that one of the companies were trying to
375
00:14:10,490 --> 00:14:11,960
put a straighter fiber but they had
376
00:14:11,960 --> 00:14:13,370
significant trouble because they had to
377
00:14:13,370 --> 00:14:16,100
go through people's properties then they
378
00:14:16,100 --> 00:14:18,500
try to make an almost straight fiber and
379
00:14:18,500 --> 00:14:20,480
they were shaving off a one millisecond
380
00:14:20,480 --> 00:14:22,520
at 1.4 millisecond we are talking about
381
00:14:22,520 --> 00:14:24,830
saving 1.4 milliseconds in latency
382
00:14:24,830 --> 00:14:26,900
between Chicago and New York and then
383
00:14:26,900 --> 00:14:28,700
this guy said hey we have a secret
384
00:14:28,700 --> 00:14:29,120
weapon
385
00:14:29,120 --> 00:14:30,320
we figure out something that's more
386
00:14:30,320 --> 00:14:33,950
interesting and it turns out according
387
00:14:33,950 --> 00:14:36,040
to them that the secret is to use
388
00:14:36,040 --> 00:14:38,780
microwave technology to send the signals
389
00:14:38,780 --> 00:14:40,940
through air rather the seventh route
390
00:14:40,940 --> 00:14:43,880
optic fiber because this is why it's
391
00:14:43,880 --> 00:14:45,590
important in all physics the speed of
392
00:14:45,590 --> 00:14:48,050
light is higher through air than it is
393
00:14:48,050 --> 00:14:50,930
for glass and they can shave off more
394
00:14:50,930 --> 00:14:52,700
than that one millisecond by sending
395
00:14:52,700 --> 00:14:56,360
things from microwave towers okay at
396
00:14:56,360 --> 00:14:58,730
least they hope now of course they have
397
00:14:58,730 --> 00:15:00,230
to put their service a night next to the
398
00:15:00,230 --> 00:15:01,850
microwave towers to make sure that there
399
00:15:01,850 --> 00:15:03,560
is absolutely no delay to get to the
400
00:15:03,560 --> 00:15:05,450
microwave tower which is next in the
401
00:15:05,450 --> 00:15:07,400
other place right so it can actually get
402
00:15:07,400 --> 00:15:11,900
quite crazy okay so keep this in mind I
403
00:15:11,900 --> 00:15:13,370
mean this is gonna be a fight that's
404
00:15:13,370 --> 00:15:15,080
gonna go on forever until somebody
405
00:15:15,080 --> 00:15:17,390
figures out how to do some sort of
406
00:15:17,390 --> 00:15:20,690
instantaneous communication okay latency
407
00:15:20,690 --> 00:15:22,790
cannot be reduced so when it comes to
408
00:15:22,790 --> 00:15:25,700
replicas that becomes very either
409
00:15:25,700 --> 00:15:28,520
advantageous you could do replication to
410
00:15:28,520 --> 00:15:30,650
mask latency and it's exactly what was
411
00:15:30,650 --> 00:15:31,790
mentioned before and this is precisely
412
00:15:31,790 --> 00:15:34,940
what tech - internet it's a big world it
413
00:15:34,940 --> 00:15:40,070
takes to cross the internet to go us to
414
00:15:40,070 --> 00:15:41,540
Australia you're already talking about
415
00:15:41,540 --> 00:15:43,580
hundreds of milliseconds okay no matter
416
00:15:43,580 --> 00:15:46,040
what even if you're in the financial
417
00:15:46,040 --> 00:15:48,200
markets and so on right I think the
418
00:15:48,200 --> 00:15:49,940
latency through the fiber even
419
00:15:49,940 --> 00:15:52,520
frustrated Fiber New York to Tokyo it's
420
00:15:52,520 --> 00:15:55,280
about 170 milliseconds okay that's it
421
00:15:55,280 --> 00:15:56,780
you can't go below that I mean you can
422
00:15:56,780 --> 00:15:58,400
do computations if you want and what you
423
00:15:58,400 --> 00:16:00,410
put the speed of light it's about I
424
00:16:00,410 --> 00:16:01,940
think about 60% of the speed of light
425
00:16:01,940 --> 00:16:03,740
through the for the glass now of course
426
00:16:03,740 --> 00:16:05,330
a problem we're sending it through a air
427
00:16:05,330 --> 00:16:08,780
is what you must have a straight line it
428
00:16:08,780 --> 00:16:10,820
just happens that the earth is curved if
429
00:16:10,820 --> 00:16:13,460
you actually you can do this and you can
430
00:16:13,460 --> 00:16:15,139
do this with radio waves you can bounce
431
00:16:15,139 --> 00:16:17,779
sit off the ionosphere right or some
432
00:16:17,779 --> 00:16:19,850
other parts of the atmosphere but that
433
00:16:19,850 --> 00:16:21,350
makes the path even longer so you're
434
00:16:21,350 --> 00:16:24,259
worse than the fiber it's weird it's a
435
00:16:24,259 --> 00:16:27,199
physicists world suddenly right okay
436
00:16:27,199 --> 00:16:28,639
nevertheless so you have a large
437
00:16:28,639 --> 00:16:31,579
Internet and to cross it it can take 200
438
00:16:31,579 --> 00:16:34,129
milliseconds easily but if you do
439
00:16:34,129 --> 00:16:37,579
replication a binary do you know how
440
00:16:37,579 --> 00:16:38,989
fast you can access for example of
441
00:16:38,989 --> 00:16:41,569
computing the department you can tell
442
00:16:41,569 --> 00:16:43,790
right in the pink program that's exactly
443
00:16:43,790 --> 00:16:47,299
that so what does ping do sends a
444
00:16:47,299 --> 00:16:50,389
special packet and the network cards
445
00:16:50,389 --> 00:16:51,559
that receives the packet immediately
446
00:16:51,559 --> 00:16:53,269
replies the pink was introduced
447
00:16:53,269 --> 00:16:55,189
precisely in order to estimate latencies
448
00:16:55,189 --> 00:16:59,059
right and this is how you know it's good
449
00:16:59,059 --> 00:17:00,589
when you get latencies well below a
450
00:17:00,589 --> 00:17:02,239
millisecond right so if you pink a
451
00:17:02,239 --> 00:17:03,139
machine in the department you have a
452
00:17:03,139 --> 00:17:04,398
decent connection you're gonna get zero
453
00:17:04,398 --> 00:17:06,619
point five milliseconds you ping
454
00:17:06,619 --> 00:17:08,089
yahoo.com you're gonna get 80
455
00:17:08,089 --> 00:17:09,949
milliseconds I don't even know where who
456
00:17:09,949 --> 00:17:13,250
is right you pink something in Australia
457
00:17:13,250 --> 00:17:15,619
or something in in Japan it's gonna be
458
00:17:15,619 --> 00:17:16,970
200 something milliseconds you can even
459
00:17:16,970 --> 00:17:19,609
try it right if you have a Linux machine
460
00:17:19,609 --> 00:17:21,349
now it's possible to do it from Windows
461
00:17:21,349 --> 00:17:23,659
but it's more elaborate apparently right
462
00:17:23,659 --> 00:17:27,679
so by placing replicas close by you can
463
00:17:27,679 --> 00:17:30,490
get within that one to two milliseconds
464
00:17:30,490 --> 00:17:33,200
rather than 200 milliseconds 200
465
00:17:33,200 --> 00:17:34,490
milliseconds can be devastating for
466
00:17:34,490 --> 00:17:39,070
example for user experience right so
467
00:17:39,070 --> 00:17:42,169
also that's an indication of to what
468
00:17:42,169 --> 00:17:43,700
level you should be paranoid so I want
469
00:17:43,700 --> 00:17:44,929
you to remember these numbers because
470
00:17:44,929 --> 00:17:46,250
they turn out to be very important right
471
00:17:46,250 --> 00:17:50,179
so arguably humans cannot perceive
472
00:17:50,179 --> 00:17:52,539
anything that's below 30 milliseconds
473
00:17:52,539 --> 00:17:55,309
this is equivalent to roughly 30 frames
474
00:17:55,309 --> 00:17:57,710
per second 30 35 frames per second now
475
00:17:57,710 --> 00:17:59,389
some people claim that they feel better
476
00:17:59,389 --> 00:18:01,669
when they play video games at 60 frames
477
00:18:01,669 --> 00:18:03,830
per second some people brag about their
478
00:18:03,830 --> 00:18:06,889
TVs that can do 120 frames per second
479
00:18:06,889 --> 00:18:09,110
none of that matters when it comes to
480
00:18:09,110 --> 00:18:11,179
the refresh rate for the for the brain I
481
00:18:11,179 --> 00:18:13,159
mean simply the visual system of humans
482
00:18:13,159 --> 00:18:15,409
cannot run faster than something like 30
483
00:18:15,409 --> 00:18:18,320
milliseconds whatever by the way that's
484
00:18:18,320 --> 00:18:20,779
one of the key ingredients in any magic
485
00:18:20,779 --> 00:18:24,559
show the hand can move faster than 30
486
00:18:24,559 --> 00:18:26,029
milliseconds you can move your hand if
487
00:18:26,029 --> 00:18:26,960
you're good you can move your hand from
488
00:18:26,960 --> 00:18:28,590
here to here and back
489
00:18:28,590 --> 00:18:34,169
30 milliseconds right or definitely from
490
00:18:34,169 --> 00:18:35,580
here to here faster than 30 milliseconds
491
00:18:35,580 --> 00:18:37,769
that means the visual system cannot get
492
00:18:37,769 --> 00:18:41,399
the movement right so magicians are very
493
00:18:41,399 --> 00:18:43,350
fast movers in the end and they also
494
00:18:43,350 --> 00:18:45,240
know how to distract you right
495
00:18:45,240 --> 00:18:47,279
so beyond a certain point is not worth
496
00:18:47,279 --> 00:18:49,169
sweating it below for example 30
497
00:18:49,169 --> 00:18:50,519
milliseconds no matter what people are
498
00:18:50,519 --> 00:18:52,559
telling you right but 200 milliseconds
499
00:18:52,559 --> 00:18:54,059
is devastating because you can actually
500
00:18:54,059 --> 00:18:56,519
see and you suffer through why do I have
501
00:18:56,519 --> 00:18:57,929
that delay people really hate delays
502
00:18:57,929 --> 00:19:00,299
right so even if you're tackling a
503
00:19:00,299 --> 00:19:02,460
simple problem which shows up almost all
504
00:19:02,460 --> 00:19:05,730
the time right and trust me here this
505
00:19:05,730 --> 00:19:07,529
becomes very very important right do you
506
00:19:07,529 --> 00:19:11,190
have a nice interface and for example a
507
00:19:11,190 --> 00:19:13,259
nice visual system to look at stocks so
508
00:19:13,259 --> 00:19:15,059
let's say and people really want to look
509
00:19:15,059 --> 00:19:16,230
at all kinds of pretty pictures with
510
00:19:16,230 --> 00:19:20,220
stocks graphs and whatnot okay if you
511
00:19:20,220 --> 00:19:21,240
have to communicate with a back-end
512
00:19:21,240 --> 00:19:23,340
server and come back unless you have a
513
00:19:23,340 --> 00:19:25,350
server that's plays very very close to
514
00:19:25,350 --> 00:19:25,940
the user
515
00:19:25,940 --> 00:19:28,980
this large latency will hit you and the
516
00:19:28,980 --> 00:19:30,029
large latency is going to be very
517
00:19:30,029 --> 00:19:31,769
annoying in the user interface one
518
00:19:31,769 --> 00:19:36,659
technique is to move the data that the
519
00:19:36,659 --> 00:19:38,369
user pulls off in order to display
520
00:19:38,369 --> 00:19:39,419
whatever it is on the screen or whatever
521
00:19:39,419 --> 00:19:41,309
the user pulls off closer and closer and
522
00:19:41,309 --> 00:19:44,190
closer to the user okay so this is as I
523
00:19:44,190 --> 00:19:46,019
mentioned way before that this is what
524
00:19:46,019 --> 00:19:47,940
hecka might does so for example CNN I
525
00:19:47,940 --> 00:19:50,580
know for sure it's a customer of of
526
00:19:50,580 --> 00:19:54,990
Akamai CNN basically pushes whatever
527
00:19:54,990 --> 00:19:57,269
changes to the content they have through
528
00:19:57,269 --> 00:19:59,460
the Akamai system and I can I will very
529
00:19:59,460 --> 00:20:01,110
very fast and efficiently through the
530
00:20:01,110 --> 00:20:03,210
network distributed much closer to you
531
00:20:03,210 --> 00:20:05,580
and there is some hijacking that goes on
532
00:20:05,580 --> 00:20:07,610
at your ISP in so when you say
533
00:20:07,610 --> 00:20:10,259
www.cnn.com your are will be going to
534
00:20:10,259 --> 00:20:12,480
whatever CNN is your go to your local
535
00:20:12,480 --> 00:20:15,869
replica so replication does need some
536
00:20:15,869 --> 00:20:18,029
collaboration or support either from the
537
00:20:18,029 --> 00:20:20,009
client or from the infrastructure it's
538
00:20:20,009 --> 00:20:22,440
not possible to do replication without
539
00:20:22,440 --> 00:20:24,419
somehow diverting things to different
540
00:20:24,419 --> 00:20:27,029
replicas okay so one core issue with
541
00:20:27,029 --> 00:20:28,440
replication is how are you going to
542
00:20:28,440 --> 00:20:30,779
divert things in case we fail that can
543
00:20:30,779 --> 00:20:32,730
be done as easily as for example this is
544
00:20:32,730 --> 00:20:36,179
a core idea behind mirrors you first
545
00:20:36,179 --> 00:20:37,799
tried the primary in replicas and if it
546
00:20:37,799 --> 00:20:39,179
fails you try the second one if you
547
00:20:39,179 --> 00:20:40,470
fails you try the third one of course
548
00:20:40,470 --> 00:20:41,759
it's very annoying because it can take a
549
00:20:41,759 --> 00:20:42,000
lot
550
00:20:42,000 --> 00:20:44,370
time to go through the replicas wait for
551
00:20:44,370 --> 00:20:46,860
them to fail it more not with replicas
552
00:20:46,860 --> 00:20:48,090
you could do load balancing
553
00:20:48,090 --> 00:20:49,830
so if ignore the fact that there might
554
00:20:49,830 --> 00:20:51,810
be close to you or not if you send
555
00:20:51,810 --> 00:20:55,040
people to one of the many replicas
556
00:20:55,040 --> 00:20:58,830
essentially you're reducing the load on
557
00:20:58,830 --> 00:21:00,570
a single replica and you're spreading
558
00:21:00,570 --> 00:21:01,890
the load and this is the notion of load
559
00:21:01,890 --> 00:21:03,300
balancing right so replicas are
560
00:21:03,300 --> 00:21:08,340
crucially related to replica load
561
00:21:08,340 --> 00:21:20,460
balancing reduce latency so what other
562
00:21:20,460 --> 00:21:23,720
things can we do with replicas yes
563
00:21:25,570 --> 00:21:27,430
what's the difference between a primary
564
00:21:27,430 --> 00:21:34,770
server and replica Oh mirroring well so
565
00:21:34,770 --> 00:21:39,040
anyway let me mention something people
566
00:21:39,040 --> 00:21:42,280
in general like to invent new terms even
567
00:21:42,280 --> 00:21:43,360
if they are very close to all terms
568
00:21:43,360 --> 00:21:45,010
because it looks better
569
00:21:45,010 --> 00:21:47,860
cooler newer for example cloud computing
570
00:21:47,860 --> 00:21:51,160
is one of those right it's only 50 years
571
00:21:51,160 --> 00:21:54,490
old now right the first ideas behind
572
00:21:54,490 --> 00:21:56,200
cloud computing were introduced easily
573
00:21:56,200 --> 00:21:58,180
about 50 years ago the balls they had
574
00:21:58,180 --> 00:21:59,530
two computers they wanted to create some
575
00:21:59,530 --> 00:22:00,850
sort of a connection between them okay
576
00:22:00,850 --> 00:22:03,160
and to do something except that now it's
577
00:22:03,160 --> 00:22:05,320
super cool and we win most of the
578
00:22:05,320 --> 00:22:06,970
technology behind cloud computing was
579
00:22:06,970 --> 00:22:09,760
inventing long time back right so this
580
00:22:09,760 --> 00:22:11,410
is my this is the rule of thumb that I
581
00:22:11,410 --> 00:22:15,190
apply every large area in computer
582
00:22:15,190 --> 00:22:17,590
science or technology every decade is
583
00:22:17,590 --> 00:22:20,200
gonna pick another name for something
584
00:22:20,200 --> 00:22:21,430
they've been doing for a long while to
585
00:22:21,430 --> 00:22:26,230
make it look fresher okay yeah Condor
586
00:22:26,230 --> 00:22:27,550
system for example was developed in
587
00:22:27,550 --> 00:22:30,070
1980s and it did almost anything that
588
00:22:30,070 --> 00:22:31,840
modern cloud computing stuff does it's
589
00:22:31,840 --> 00:22:33,790
just more excitement it's the same thing
590
00:22:33,790 --> 00:22:35,170
here what's the difference not much of a
591
00:22:35,170 --> 00:22:38,650
difference very I mean the problem in
592
00:22:38,650 --> 00:22:40,090
general and this is what I want you to
593
00:22:40,090 --> 00:22:42,040
remember the problem in general in
594
00:22:42,040 --> 00:22:43,990
distribute systems but a lot of other
595
00:22:43,990 --> 00:22:46,360
things is people don't really take the
596
00:22:46,360 --> 00:22:49,380
time to really produce a definition
597
00:22:49,380 --> 00:22:52,180
let's say a replication sounds cool you
598
00:22:52,180 --> 00:22:54,070
say everybody who's supposed to know
599
00:22:54,070 --> 00:22:56,140
about replication should know what a
600
00:22:56,140 --> 00:22:57,580
replication is way-hey it's English
601
00:22:57,580 --> 00:22:59,500
right what's a replication let's look in
602
00:22:59,500 --> 00:23:00,730
the dictionary well the diction is not
603
00:23:00,730 --> 00:23:03,490
gonna say yeah if it's look in the
604
00:23:03,490 --> 00:23:04,360
dictionary and see the difference
605
00:23:04,360 --> 00:23:06,790
between mirroring and replication right
606
00:23:06,790 --> 00:23:08,500
so what I'm trying to say is people
607
00:23:08,500 --> 00:23:10,030
don't take the trouble to define things
608
00:23:10,030 --> 00:23:13,960
then words morph in meaning what do they
609
00:23:13,960 --> 00:23:15,430
mean and they don't mean and ultimately
610
00:23:15,430 --> 00:23:16,990
a lot of things are very close to each
611
00:23:16,990 --> 00:23:18,700
other so to some extent you can think
612
00:23:18,700 --> 00:23:20,440
about this kind of replication as some
613
00:23:20,440 --> 00:23:22,480
sort of caching and for example can ask
614
00:23:22,480 --> 00:23:23,890
the question how is this different how
615
00:23:23,890 --> 00:23:25,450
is replication different than caching
616
00:23:25,450 --> 00:23:26,590
than different than mirroring than
617
00:23:26,590 --> 00:23:28,450
different than something else what in
618
00:23:28,450 --> 00:23:30,040
substance they are not different they
619
00:23:30,040 --> 00:23:32,890
are the same kind of thing you might
620
00:23:32,890 --> 00:23:35,260
produce some refinements which probably
621
00:23:35,260 --> 00:23:36,790
are going to be breaking broken by the
622
00:23:36,790 --> 00:23:37,960
next guy that comes and does some
623
00:23:37,960 --> 00:23:39,320
replication service just because
624
00:23:39,320 --> 00:23:40,790
he thought the name is cooler right
625
00:23:40,790 --> 00:23:42,680
let's call this replication though it's
626
00:23:42,680 --> 00:23:43,820
not quite replication or things like
627
00:23:43,820 --> 00:23:46,810
that so don't put too much in the name I
628
00:23:46,810 --> 00:23:49,520
would say in essence nothing between
629
00:23:49,520 --> 00:23:52,490
mirroring and replication now specific
630
00:23:52,490 --> 00:23:54,770
guarantee is that a specific service
631
00:23:54,770 --> 00:23:56,870
offers that's a different story
632
00:23:56,870 --> 00:23:58,790
so traditionally mirroring service
633
00:23:58,790 --> 00:24:01,250
services offer very few guarantees for
634
00:24:01,250 --> 00:24:02,840
example they never said that the copy is
635
00:24:02,840 --> 00:24:05,150
fresh and for example some of the the
636
00:24:05,150 --> 00:24:07,460
mirrors of the important software like
637
00:24:07,460 --> 00:24:09,200
the Linux kernel and notorious for being
638
00:24:09,200 --> 00:24:13,160
back a couple of months right so if you
639
00:24:13,160 --> 00:24:15,110
use that mirror you're gonna get a
640
00:24:15,110 --> 00:24:16,880
version that's couple of months old
641
00:24:16,880 --> 00:24:18,830
that's not too good Akamai its famous
642
00:24:18,830 --> 00:24:21,380
for having replica propagation in
643
00:24:21,380 --> 00:24:24,590
seconds at most right so it's not so
644
00:24:24,590 --> 00:24:25,970
much what's the difference between the
645
00:24:25,970 --> 00:24:27,320
terms is what's the difference between
646
00:24:27,320 --> 00:24:29,090
specific services offered by specific
647
00:24:29,090 --> 00:24:34,040
tools companies or whatnot okay so to a
648
00:24:34,040 --> 00:24:36,500
large extent the replicas are related to
649
00:24:36,500 --> 00:24:37,970
caching and remember caching gets a
650
00:24:37,970 --> 00:24:40,130
fundamental technique in general and
651
00:24:40,130 --> 00:24:43,520
what does caching do in general right it
652
00:24:43,520 --> 00:24:45,800
allows me a faster access to that
653
00:24:45,800 --> 00:24:47,900
resource at the expense of having it not
654
00:24:47,900 --> 00:24:49,610
quite in sync with the rest but that's
655
00:24:49,610 --> 00:24:51,410
essentially what replication potentially
656
00:24:51,410 --> 00:24:53,930
is right now one thing that we are going
657
00:24:53,930 --> 00:24:55,700
to see when we talk about various kinds
658
00:24:55,700 --> 00:24:57,140
of consistency is you could insist that
659
00:24:57,140 --> 00:24:58,490
the replicas are completely up to date
660
00:24:58,490 --> 00:25:02,330
but that's gonna be really burdensome on
661
00:25:02,330 --> 00:25:07,100
the framework right so again you can't
662
00:25:07,100 --> 00:25:09,740
really say I mean you can basically for
663
00:25:09,740 --> 00:25:12,260
example this is one one kind of a very
664
00:25:12,260 --> 00:25:13,790
harsh consistency you can impose on
665
00:25:13,790 --> 00:25:16,820
replicas if you activate the replicas
666
00:25:16,820 --> 00:25:19,250
only when you're sure that the changes
667
00:25:19,250 --> 00:25:22,070
were propagated to all other replicas so
668
00:25:22,070 --> 00:25:24,500
that to any user no matter where they
669
00:25:24,500 --> 00:25:28,550
connect right it looks like we all
670
00:25:28,550 --> 00:25:30,110
switch to the new version at the same
671
00:25:30,110 --> 00:25:33,080
time so you have changes but you don't
672
00:25:33,080 --> 00:25:34,520
make for example the changes available
673
00:25:34,520 --> 00:25:36,500
until everybody got the changes and this
674
00:25:36,500 --> 00:25:37,850
is equivalent for example and this is
675
00:25:37,850 --> 00:25:40,370
done sometimes in databases when you
676
00:25:40,370 --> 00:25:41,900
have this distributed transactions which
677
00:25:41,900 --> 00:25:43,760
is we still have to talk about so you
678
00:25:43,760 --> 00:25:45,320
want to make sure for example that you
679
00:25:45,320 --> 00:25:47,060
say commit to a particular transaction
680
00:25:47,060 --> 00:25:48,920
to the user you tell the user yes your
681
00:25:48,920 --> 00:25:50,300
transaction actually finished correctly
682
00:25:50,300 --> 00:25:53,260
only when all the parties involved
683
00:25:53,260 --> 00:25:56,330
agreed that we commit to the transaction
684
00:25:56,330 --> 00:25:58,280
and this is extremely harsh the moment
685
00:25:58,280 --> 00:26:00,350
you use such a mechanism be prepared to
686
00:26:00,350 --> 00:26:03,559
pay a very high price large number of
687
00:26:03,559 --> 00:26:05,540
message exchanges pay a large amount of
688
00:26:05,540 --> 00:26:08,000
time because of the latency forget about
689
00:26:08,000 --> 00:26:11,720
any other issue okay so we do
690
00:26:11,720 --> 00:26:13,670
replication not to lose data we do
691
00:26:13,670 --> 00:26:17,270
replication to reduce the latency what
692
00:26:17,270 --> 00:26:18,620
are the reasons so for example how would
693
00:26:18,620 --> 00:26:20,120
you do this right I'm talking about this
694
00:26:20,120 --> 00:26:21,500
40 milliseconds here I want to update
695
00:26:21,500 --> 00:26:24,280
graphs I want to do this kind of thing
696
00:26:24,280 --> 00:26:25,700
all right
697
00:26:25,700 --> 00:26:28,309
so echo what echo might does is it's
698
00:26:28,309 --> 00:26:30,740
nice and good you potentially could do
699
00:26:30,740 --> 00:26:32,270
better if you can compress the data a
700
00:26:32,270 --> 00:26:33,860
lot and the data doesn't change so much
701
00:26:33,860 --> 00:26:36,440
or at least can be considered stationary
702
00:26:36,440 --> 00:26:38,420
for for a few minutes but you can
703
00:26:38,420 --> 00:26:41,630
actually instead of having the webpage
704
00:26:41,630 --> 00:26:43,460
generated in a server you can generate
705
00:26:43,460 --> 00:26:44,630
it in a client and this is something
706
00:26:44,630 --> 00:26:45,980
you're not really told I mean this is
707
00:26:45,980 --> 00:26:48,170
the new brave new world in which you
708
00:26:48,170 --> 00:26:51,380
just ship the data to the browser and
709
00:26:51,380 --> 00:26:53,450
the browser will create all that all
710
00:26:53,450 --> 00:26:56,420
those pages now this is JavaScript magic
711
00:26:56,420 --> 00:26:59,090
if you do so you can do some amazing
712
00:26:59,090 --> 00:27:01,340
stunts that are not possible otherwise
713
00:27:01,340 --> 00:27:02,750
right so for example a particular
714
00:27:02,750 --> 00:27:05,059
problem that a number of data
715
00:27:05,059 --> 00:27:06,950
visualization companies have is they
716
00:27:06,950 --> 00:27:08,570
rely on back-end servers which means
717
00:27:08,570 --> 00:27:11,360
they they need this to is not 200
718
00:27:11,360 --> 00:27:12,740
milliseconds by the way it's much closer
719
00:27:12,740 --> 00:27:14,570
to one or two seconds the moment you
720
00:27:14,570 --> 00:27:16,100
involve database servers and all kinds
721
00:27:16,100 --> 00:27:17,600
of other processing if you say give me
722
00:27:17,600 --> 00:27:20,120
the graph that shows me the stock for
723
00:27:20,120 --> 00:27:21,670
this company over the last five years
724
00:27:21,670 --> 00:27:25,429
right your browser talks to the web
725
00:27:25,429 --> 00:27:27,260
server who talks to the backend the
726
00:27:27,260 --> 00:27:29,480
database which computes the new graph
727
00:27:29,480 --> 00:27:31,400
you push it through some sort of a ok
728
00:27:31,400 --> 00:27:33,170
compute some sort of a feature or
729
00:27:33,170 --> 00:27:36,260
whatever ship it to the front-end if you
730
00:27:36,260 --> 00:27:37,520
don't believe me you can use this tool
731
00:27:37,520 --> 00:27:41,330
it feels like a long second a second
732
00:27:41,330 --> 00:27:43,640
wait is devastating for the human brain
733
00:27:43,640 --> 00:27:46,610
it really makes you feel bad right it it
734
00:27:46,610 --> 00:27:48,020
bogs you down you cannot think
735
00:27:48,020 --> 00:27:50,000
especially if you expect to go through
736
00:27:50,000 --> 00:27:52,040
hundreds thousands of these things to
737
00:27:52,040 --> 00:27:54,800
see anything right faster than 30
738
00:27:54,800 --> 00:27:56,110
milliseconds doesn't matter either
739
00:27:56,110 --> 00:27:59,720
that's the trick right so staying close
740
00:27:59,720 --> 00:28:01,190
to 30 milliseconds makes it makes it
741
00:28:01,190 --> 00:28:02,780
look like a movie right so think about
742
00:28:02,780 --> 00:28:04,550
it you would like to find an interesting
743
00:28:04,550 --> 00:28:05,770
stock and you
744
00:28:05,770 --> 00:28:07,210
by the way your brain can easily run at
745
00:28:07,210 --> 00:28:09,280
30 milliseconds of rain and you can
746
00:28:09,280 --> 00:28:11,530
actually see patterns at 30 frames per
747
00:28:11,530 --> 00:28:14,410
second not only that but the human brain
748
00:28:14,410 --> 00:28:17,320
is really designed to see patterns in
749
00:28:17,320 --> 00:28:19,000
moving things not in aesthetic settings
750
00:28:19,000 --> 00:28:20,170
if I give you a static picture you're
751
00:28:20,170 --> 00:28:21,460
gonna have a hard time figuring out what
752
00:28:21,460 --> 00:28:22,630
changed or what didn't change if I give
753
00:28:22,630 --> 00:28:23,800
you a move you instantly know what's
754
00:28:23,800 --> 00:28:25,420
important and what's not important right
755
00:28:25,420 --> 00:28:27,880
so from that point of view you're better
756
00:28:27,880 --> 00:28:29,740
off is essentially producing some sort
757
00:28:29,740 --> 00:28:31,270
of replication of the data in the
758
00:28:31,270 --> 00:28:31,780
browser
759
00:28:31,780 --> 00:28:33,580
so suddenly replica has a different
760
00:28:33,580 --> 00:28:37,240
meaning right so now for example is that
761
00:28:37,240 --> 00:28:40,870
a good investment or not to push the
762
00:28:40,870 --> 00:28:44,350
replica in the web browser what one of
763
00:28:44,350 --> 00:28:48,040
the tricks that almost always is a good
764
00:28:48,040 --> 00:28:49,930
trade-off in modern disability systems
765
00:28:49,930 --> 00:28:53,920
is to trade-off throughput for for
766
00:28:53,920 --> 00:28:56,260
latency so how it can you do that right
767
00:28:56,260 --> 00:28:58,480
so that means do a lot more replication
768
00:28:58,480 --> 00:29:01,120
that otherwise you would think when the
769
00:29:01,120 --> 00:29:02,710
user for example starts to look at
770
00:29:02,710 --> 00:29:05,920
stocks right you have to pay for example
771
00:29:05,920 --> 00:29:07,450
200 milliseconds or you have to pay for
772
00:29:07,450 --> 00:29:10,120
example a second right to access a
773
00:29:10,120 --> 00:29:12,130
single picture one second you can ask
774
00:29:12,130 --> 00:29:14,830
yourself the question if I'm if the user
775
00:29:14,830 --> 00:29:16,810
is willing to pay one second or maybe
776
00:29:16,810 --> 00:29:18,670
even let's say ten times more but not to
777
00:29:18,670 --> 00:29:21,040
pay any price up afterwards right namely
778
00:29:21,040 --> 00:29:23,200
stay within 30 milliseconds how much
779
00:29:23,200 --> 00:29:26,020
could I ship in two seconds
780
00:29:26,020 --> 00:29:27,730
so ask yourself the question with modern
781
00:29:27,730 --> 00:29:29,590
networks how much information could I
782
00:29:29,590 --> 00:29:31,150
ship to the user in let's say two
783
00:29:31,150 --> 00:29:32,590
seconds because the trick I can play is
784
00:29:32,590 --> 00:29:33,220
the following
785
00:29:33,220 --> 00:29:35,290
instead of doing everything piece by
786
00:29:35,290 --> 00:29:37,480
piece if I have somebody in what space
787
00:29:37,480 --> 00:29:39,790
the user is gonna stay and the first
788
00:29:39,790 --> 00:29:42,160
request instead of shipping only the
789
00:29:42,160 --> 00:29:43,420
little thing that the user wants I'm
790
00:29:43,420 --> 00:29:45,640
gonna ship two seconds worth of
791
00:29:45,640 --> 00:29:47,320
information at the bandwidth that the
792
00:29:47,320 --> 00:29:50,830
user has all right now if you have a
793
00:29:50,830 --> 00:29:52,210
decent internet connection that
794
00:29:52,210 --> 00:29:55,600
bandwidth right you can buy this for
795
00:29:55,600 --> 00:29:56,950
example in certain parts of games but
796
00:29:56,950 --> 00:29:58,630
now that bandwidth could be as high as
797
00:29:58,630 --> 00:30:02,080
one megabyte per second okay so two
798
00:30:02,080 --> 00:30:05,080
seconds is two megabytes in two
799
00:30:05,080 --> 00:30:08,950
megabytes you can easily put quite a lot
800
00:30:08,950 --> 00:30:11,110
of information for all the 10000 tickers
801
00:30:11,110 --> 00:30:13,990
on the stock market so what I'm trying
802
00:30:13,990 --> 00:30:15,640
to say there is when you design
803
00:30:15,640 --> 00:30:17,290
distributed systems think about
804
00:30:17,290 --> 00:30:18,600
replication at some
805
00:30:18,600 --> 00:30:20,039
it's really your friend and bring it
806
00:30:20,039 --> 00:30:21,539
possibly who much much closer to the
807
00:30:21,539 --> 00:30:23,760
user so one way to do it would be to
808
00:30:23,760 --> 00:30:27,090
relay ship almost all the information to
809
00:30:27,090 --> 00:30:29,190
the user because then the latency is
810
00:30:29,190 --> 00:30:31,230
below the radar of what the user can
811
00:30:31,230 --> 00:30:33,900
actually proceed you have no latency to
812
00:30:33,900 --> 00:30:35,400
go to the back now sometimes this
813
00:30:35,400 --> 00:30:38,549
strategy works very very nicely because
814
00:30:38,549 --> 00:30:40,020
the data simply doesn't change that much
815
00:30:40,020 --> 00:30:42,000
right so for example if the stock date
816
00:30:42,000 --> 00:30:43,530
so if the user wants to look at
817
00:30:43,530 --> 00:30:44,970
historical data for example what
818
00:30:44,970 --> 00:30:47,490
happened at the granularity of one day
819
00:30:47,490 --> 00:30:51,360
on the stock market spending 2 3 4 or
820
00:30:51,360 --> 00:30:52,980
even 10 seconds to ship the whole
821
00:30:52,980 --> 00:30:56,400
history of all the stocks that within
822
00:30:56,400 --> 00:30:59,130
some kind of a range for the for the
823
00:30:59,130 --> 00:31:00,270
user for example you only have that many
824
00:31:00,270 --> 00:31:01,650
stocks actually about 10,000 of them
825
00:31:01,650 --> 00:31:04,679
right it's not something that you should
826
00:31:04,679 --> 00:31:07,500
dismiss once you've done so if you have
827
00:31:07,500 --> 00:31:09,570
the horsepower in JavaScript to really
828
00:31:09,570 --> 00:31:10,919
generate all those things that normally
829
00:31:10,919 --> 00:31:12,299
you would get from from the backend you
830
00:31:12,299 --> 00:31:15,720
have completely seamless execution of
831
00:31:15,720 --> 00:31:17,610
this right so an example of this for
832
00:31:17,610 --> 00:31:19,559
example is what what we try to do with
833
00:31:19,559 --> 00:31:21,600
this Medicare data it takes about two or
834
00:31:21,600 --> 00:31:24,120
three seconds to ship one hundred and
835
00:31:24,120 --> 00:31:26,100
sixty three thousand rows to the
836
00:31:26,100 --> 00:31:28,890
JavaScript after that it's all below 30
837
00:31:28,890 --> 00:31:30,539
milliseconds and you feel like there is
838
00:31:30,539 --> 00:31:32,549
no effort whatsoever right so that's
839
00:31:32,549 --> 00:31:33,929
really a form of replication I've
840
00:31:33,929 --> 00:31:36,270
replicated the backend data right I took
841
00:31:36,270 --> 00:31:38,070
care to compress it a little bit and
842
00:31:38,070 --> 00:31:39,900
make it smaller and so on so all kinds
843
00:31:39,900 --> 00:31:40,890
of tricks to make it as small as
844
00:31:40,890 --> 00:31:42,570
possible but once I ship it to the front
845
00:31:42,570 --> 00:31:46,049
you have a replica in your browser in
846
00:31:46,049 --> 00:31:47,610
the case of the Medicare data for
847
00:31:47,610 --> 00:31:49,320
example we know for sure that there is
848
00:31:49,320 --> 00:31:50,730
no updates to it the feds have no
849
00:31:50,730 --> 00:31:53,520
intention to give more of that data
850
00:31:53,520 --> 00:31:56,520
right it did not change a single bit in
851
00:31:56,520 --> 00:31:58,830
four months and it's not clear at all
852
00:31:58,830 --> 00:32:00,690
when it will change the same thing with
853
00:32:00,690 --> 00:32:02,070
the stock market won't change and the
854
00:32:02,070 --> 00:32:04,350
resolution of more than once a day right
855
00:32:04,350 --> 00:32:06,210
because I'm only providing you the low
856
00:32:06,210 --> 00:32:07,980
level services which is the average
857
00:32:07,980 --> 00:32:12,299
price per day right so in those kind of
858
00:32:12,299 --> 00:32:13,740
circumstances really the replication
859
00:32:13,740 --> 00:32:15,780
happens very very close to the client
860
00:32:15,780 --> 00:32:17,460
you can think of it as some sort of
861
00:32:17,460 --> 00:32:18,840
caching but it's in fact some sort of
862
00:32:18,840 --> 00:32:20,730
replication okay I just kind of blurred
863
00:32:20,730 --> 00:32:24,120
anyway now when it comes to this fast
864
00:32:24,120 --> 00:32:26,130
transactions fast changes things become
865
00:32:26,130 --> 00:32:28,140
much much much harder right so for
866
00:32:28,140 --> 00:32:29,610
example if the user will really be
867
00:32:29,610 --> 00:32:31,290
interested in very up-to-date
868
00:32:31,290 --> 00:32:32,280
information
869
00:32:32,280 --> 00:32:33,690
how do you provide that how you provide
870
00:32:33,690 --> 00:32:36,300
that replica so one thing and this is
871
00:32:36,300 --> 00:32:37,710
related to some deep questions about
872
00:32:37,710 --> 00:32:39,180
operating systems right I might as well
873
00:32:39,180 --> 00:32:40,980
connect them so a lot of these problems
874
00:32:40,980 --> 00:32:42,870
are really problems even if you don't
875
00:32:42,870 --> 00:32:44,340
have distributed systems at all and I'm
876
00:32:44,340 --> 00:32:46,260
gonna explain a second right so imagine
877
00:32:46,260 --> 00:32:48,810
now you're tapping into there all the
878
00:32:48,810 --> 00:32:51,000
activity that goes on any market in the
879
00:32:51,000 --> 00:32:54,030
world ok it turns out you can buy such a
880
00:32:54,030 --> 00:32:57,450
service and there are for example
881
00:32:57,450 --> 00:32:58,970
there's a company called Nanak's and
882
00:32:58,970 --> 00:33:02,390
this company provides an extremely fast
883
00:33:02,390 --> 00:33:05,370
data stream that gives you access to all
884
00:33:05,370 --> 00:33:06,870
the transactions that happen on all the
885
00:33:06,870 --> 00:33:09,450
financial markets ok now you pay an arm
886
00:33:09,450 --> 00:33:11,070
and a leg for the data stream don't get
887
00:33:11,070 --> 00:33:13,020
me wrong but to some people it's worth
888
00:33:13,020 --> 00:33:15,110
it because that's how you find out about
889
00:33:15,110 --> 00:33:18,060
activity on the on all the markets and
890
00:33:18,060 --> 00:33:21,750
they date that data stream itself is 40
891
00:33:21,750 --> 00:33:27,030
million transactions per second right by
892
00:33:27,030 --> 00:33:29,070
the way to even ship that many per
893
00:33:29,070 --> 00:33:31,830
second they have to have a very fixed
894
00:33:31,830 --> 00:33:33,540
format so they have something like a 64
895
00:33:33,540 --> 00:33:38,730
byte so they have 64 byte packets so we
896
00:33:38,730 --> 00:33:40,140
are talking about things that suddenly
897
00:33:40,140 --> 00:33:42,060
is something very very high level in the
898
00:33:42,060 --> 00:33:43,410
application and being some kind of a
899
00:33:43,410 --> 00:33:45,390
fluffy thing they are get pushed all the
900
00:33:45,390 --> 00:33:48,480
way to the bottom and they are really
901
00:33:48,480 --> 00:33:53,100
tight specified in C format 64 bytes to
902
00:33:53,100 --> 00:33:56,190
specify each of the one transactions 40
903
00:33:56,190 --> 00:33:58,200
million transactions per per second so
904
00:33:58,200 --> 00:33:59,400
they say hey if you have a fast enough
905
00:33:59,400 --> 00:34:00,930
internet connection no problem and you
906
00:34:00,930 --> 00:34:02,340
pay the money for the service will
907
00:34:02,340 --> 00:34:05,010
stream to you the data in fact what they
908
00:34:05,010 --> 00:34:07,410
are doing is they allow you to replicate
909
00:34:07,410 --> 00:34:08,969
everything that happens on that on the
910
00:34:08,969 --> 00:34:11,580
financial markets that's similar to the
911
00:34:11,580 --> 00:34:13,830
kind of stream that Akamai might send to
912
00:34:13,830 --> 00:34:15,120
their servers to propagate information
913
00:34:15,120 --> 00:34:18,179
so that that's an implicit specification
914
00:34:18,179 --> 00:34:20,550
of the replicas not an explicit one but
915
00:34:20,550 --> 00:34:22,199
you almost always have to do this so
916
00:34:22,199 --> 00:34:24,000
let's think a little bit about some form
917
00:34:24,000 --> 00:34:25,260
of replication that you might not even
918
00:34:25,260 --> 00:34:26,610
think they are replication but they're
919
00:34:26,610 --> 00:34:28,230
really replication and which are core
920
00:34:28,230 --> 00:34:30,210
operating system issues have nothing to
921
00:34:30,210 --> 00:34:32,489
do with distributed systems okay so here
922
00:34:32,489 --> 00:34:33,810
is a similar problem even though usually
923
00:34:33,810 --> 00:34:35,040
people don't paint it as a similar
924
00:34:35,040 --> 00:34:37,710
problem you have a program and it runs
925
00:34:37,710 --> 00:34:41,100
for a while a while being let's say a
926
00:34:41,100 --> 00:34:43,500
couple of weeks we have those running in
927
00:34:43,500 --> 00:34:44,500
the department by the way
928
00:34:44,500 --> 00:34:47,530
a large simulation or whatnot okay now a
929
00:34:47,530 --> 00:34:49,270
big problem with those programs is if
930
00:34:49,270 --> 00:34:50,918
the program for any reason crashes in
931
00:34:50,918 --> 00:34:51,399
the middle
932
00:34:51,399 --> 00:34:52,780
for example power outage or any other
933
00:34:52,780 --> 00:34:56,440
reason programming error or the wind
934
00:34:56,440 --> 00:34:58,720
blows too hard or whatever right you
935
00:34:58,720 --> 00:35:00,700
lost many weeks
936
00:35:00,700 --> 00:35:02,770
I know physicists that were running
937
00:35:02,770 --> 00:35:06,880
their code for months right good so one
938
00:35:06,880 --> 00:35:09,369
interesting idea there would be what why
939
00:35:09,369 --> 00:35:11,470
don't I replicate the state of the
940
00:35:11,470 --> 00:35:13,089
program at certain points
941
00:35:13,089 --> 00:35:15,819
it's called checkpointing so that if the
942
00:35:15,819 --> 00:35:17,410
system fails I can only go back to the
943
00:35:17,410 --> 00:35:19,270
checkpoint and continue from there say
944
00:35:19,270 --> 00:35:20,829
things that I mean I might lose for
945
00:35:20,829 --> 00:35:23,470
example a minute let's say if I
946
00:35:23,470 --> 00:35:25,089
checkpoint every minute which means if I
947
00:35:25,089 --> 00:35:27,010
produce a replica in another form on a
948
00:35:27,010 --> 00:35:28,960
persistent storage then I can go back
949
00:35:28,960 --> 00:35:30,369
and lose a minute and pick up the
950
00:35:30,369 --> 00:35:33,160
computation no big deal ok so then the
951
00:35:33,160 --> 00:35:34,869
fundamental question is and this is a
952
00:35:34,869 --> 00:35:36,819
problem that looks like has nothing to
953
00:35:36,819 --> 00:35:39,250
do with the rest of them is ok you have
954
00:35:39,250 --> 00:35:41,859
your own favorite sequential program no
955
00:35:41,859 --> 00:35:43,270
strings attached with distributed
956
00:35:43,270 --> 00:35:45,099
systems you need to provide checkpoint
957
00:35:45,099 --> 00:35:47,230
for your program just because it runs
958
00:35:47,230 --> 00:35:49,150
too long now supposedly your program
959
00:35:49,150 --> 00:35:51,039
doesn't just execute infinite loops it
960
00:35:51,039 --> 00:35:53,200
does some changes to the state how do
961
00:35:53,200 --> 00:35:55,270
you replicate how do you replicate the
962
00:35:55,270 --> 00:35:57,160
in-memory state to some sort of a disk
963
00:35:57,160 --> 00:35:59,319
state because that's persistence so how
964
00:35:59,319 --> 00:36:01,150
do you do it this is in fact the core
965
00:36:01,150 --> 00:36:04,630
replica issue it's you almost have the
966
00:36:04,630 --> 00:36:07,809
full-fledged replication issues right
967
00:36:07,809 --> 00:36:09,190
there in that problem now this is
968
00:36:09,190 --> 00:36:10,450
something people looked at for a very
969
00:36:10,450 --> 00:36:11,829
long time and they don't necessarily
970
00:36:11,829 --> 00:36:14,589
have a very good solution what's the
971
00:36:14,589 --> 00:36:16,450
core problem there which is very similar
972
00:36:16,450 --> 00:36:17,770
to the core problem in in distributed
973
00:36:17,770 --> 00:36:21,730
systems the core problem is you cannot
974
00:36:21,730 --> 00:36:23,260
really use the easy way out so what's
975
00:36:23,260 --> 00:36:24,750
the easy way out in that circumstance
976
00:36:24,750 --> 00:36:27,819
freeze the program take a full dump of
977
00:36:27,819 --> 00:36:31,619
the memory and put it on the disc right
978
00:36:31,619 --> 00:36:37,480
now this is probably a good solution the
979
00:36:37,480 --> 00:36:39,760
full memory so if I dump the full memory
980
00:36:39,760 --> 00:36:42,029
state in the state of the registers I
981
00:36:42,029 --> 00:36:45,359
captured the full state of the processor
982
00:36:45,359 --> 00:36:48,099
well Intel guarantees that this is the
983
00:36:48,099 --> 00:36:50,619
case so that if I fail I can just
984
00:36:50,619 --> 00:36:52,059
recover the full set of the memory
985
00:36:52,059 --> 00:36:54,970
recover the full state of the of the
986
00:36:54,970 --> 00:36:57,460
registers and say yeah continue from
987
00:36:57,460 --> 00:36:58,720
this point
988
00:36:58,720 --> 00:37:01,330
right now this solution has a major
989
00:37:01,330 --> 00:37:03,520
advantage and the major advantage is it
990
00:37:03,520 --> 00:37:05,770
does not depend on any semantics related
991
00:37:05,770 --> 00:37:09,700
to what the code was doing okay it's
992
00:37:09,700 --> 00:37:11,770
truly a universal solution I don't care
993
00:37:11,770 --> 00:37:15,550
what you do this is in fact a provably
994
00:37:15,550 --> 00:37:19,060
good and usable replica you're only
995
00:37:19,060 --> 00:37:20,800
gonna lose the amount of time you lost
996
00:37:20,800 --> 00:37:22,540
since the last replica because that's
997
00:37:22,540 --> 00:37:25,030
guaranteed by Intel who says our
998
00:37:25,030 --> 00:37:27,400
computers behave deterministically by
999
00:37:27,400 --> 00:37:28,869
the way something similar is used for
1000
00:37:28,869 --> 00:37:31,660
example when you use this virtualization
1001
00:37:31,660 --> 00:37:34,060
technology right and you say safe state
1002
00:37:34,060 --> 00:37:35,589
of the virtual machine that's by the way
1003
00:37:35,589 --> 00:37:37,150
one of the tremendous benefits of the
1004
00:37:37,150 --> 00:37:39,000
virtual machines you can freeze them
1005
00:37:39,000 --> 00:37:41,710
right and later come back to them or
1006
00:37:41,710 --> 00:37:42,910
move them on a completely different
1007
00:37:42,910 --> 00:37:44,170
machine and come back to them there you
1008
00:37:44,170 --> 00:37:46,030
really save the state and you can you
1009
00:37:46,030 --> 00:37:47,920
can continue running the state so for
1010
00:37:47,920 --> 00:37:50,280
virtual machines this kind of
1011
00:37:50,280 --> 00:37:54,430
checkpointing it's in fact trivial if
1012
00:37:54,430 --> 00:37:56,890
you're not particularly concerned about
1013
00:37:56,890 --> 00:38:00,790
the performance of this right just save
1014
00:38:00,790 --> 00:38:01,930
the state and we have to stop the
1015
00:38:01,930 --> 00:38:02,920
program why do you have to stop the
1016
00:38:02,920 --> 00:38:06,640
program that's the trouble right so if
1017
00:38:06,640 --> 00:38:08,710
you have a if you have a regime in which
1018
00:38:08,710 --> 00:38:10,720
the state is changing and you're trying
1019
00:38:10,720 --> 00:38:12,190
to do this check pointing the problem
1020
00:38:12,190 --> 00:38:13,690
suddenly becomes much much much harder
1021
00:38:13,690 --> 00:38:16,510
potentially and it's similar to the
1022
00:38:16,510 --> 00:38:17,830
conflicts you get if you don't use
1023
00:38:17,830 --> 00:38:19,570
mutual exclusion regions you don't quite
1024
00:38:19,570 --> 00:38:21,270
know what you get when you saved it
1025
00:38:21,270 --> 00:38:23,440
Bracey the problem is not saving
1026
00:38:23,440 --> 00:38:25,510
something the problem is how the saved
1027
00:38:25,510 --> 00:38:28,359
thing or how the propagated thing to
1028
00:38:28,359 --> 00:38:30,460
another replica can be used to recover
1029
00:38:30,460 --> 00:38:32,650
something that some sort of a consistent
1030
00:38:32,650 --> 00:38:34,089
state so you see how consistency comes
1031
00:38:34,089 --> 00:38:38,020
into play right if I freeze the program
1032
00:38:38,020 --> 00:38:40,480
remember all the registers and dump all
1033
00:38:40,480 --> 00:38:43,230
the memory I have a reasonably good
1034
00:38:43,230 --> 00:38:47,440
ground to say I get a consistent copy
1035
00:38:47,440 --> 00:38:49,810
for that moment in time but almost
1036
00:38:49,810 --> 00:38:52,270
anything off that it's it's plagued with
1037
00:38:52,270 --> 00:38:54,220
problems how do you know that you didn't
1038
00:38:54,220 --> 00:38:56,369
get something that's a NASA right so
1039
00:38:56,369 --> 00:38:59,500
almost anything except this very if you
1040
00:38:59,500 --> 00:39:01,900
want barbaric solution right which
1041
00:39:01,900 --> 00:39:04,000
require very very fine-grained
1042
00:39:04,000 --> 00:39:05,890
considerations with respect to what is
1043
00:39:05,890 --> 00:39:07,359
it that the application was doing and
1044
00:39:07,359 --> 00:39:09,580
proves that a particular replication
1045
00:39:09,580 --> 00:39:10,930
strategy a particular checkpointing
1046
00:39:10,930 --> 00:39:11,670
strategies
1047
00:39:11,670 --> 00:39:13,799
of the same in the circumstance that's
1048
00:39:13,799 --> 00:39:16,049
not mess things up produces a certain
1049
00:39:16,049 --> 00:39:19,470
level of consistency right now I just
1050
00:39:19,470 --> 00:39:21,540
want to point out that in the context
1051
00:39:21,540 --> 00:39:23,430
very restrictive context of relational
1052
00:39:23,430 --> 00:39:24,990
databases the database community spend
1053
00:39:24,990 --> 00:39:26,579
an enormous amount of time figuring out
1054
00:39:26,579 --> 00:39:28,200
exactly what this would mean
1055
00:39:28,200 --> 00:39:29,970
right and they came up with this kind of
1056
00:39:29,970 --> 00:39:32,190
algorithms a large number of them
1057
00:39:32,190 --> 00:39:33,990
providing a certain kind of consistency
1058
00:39:33,990 --> 00:39:35,790
right so for example an interesting idea
1059
00:39:35,790 --> 00:39:38,040
would be can you checkpoint as other
1060
00:39:38,040 --> 00:39:40,290
things are going and it's possible to do
1061
00:39:40,290 --> 00:39:41,369
any restricting domains like the
1062
00:39:41,369 --> 00:39:43,710
databases but they needed 20 30 years of
1063
00:39:43,710 --> 00:39:46,170
research to nail down the solutions in
1064
00:39:46,170 --> 00:39:48,799
general there is no universal solution
1065
00:39:48,799 --> 00:39:55,020
especially if you expect very fast kind
1066
00:39:55,020 --> 00:39:58,170
of replication process right another way
1067
00:39:58,170 --> 00:40:00,240
you could try to capture this and this
1068
00:40:00,240 --> 00:40:02,549
is potentially something interesting
1069
00:40:02,549 --> 00:40:06,210
right is you say hey the state is
1070
00:40:06,210 --> 00:40:08,880
enormous by the way size of the state is
1071
00:40:08,880 --> 00:40:11,460
a really big problem for example you can
1072
00:40:11,460 --> 00:40:12,930
buy now a hundred twenty gigs of memory
1073
00:40:12,930 --> 00:40:16,200
with about a thousand dollars it takes a
1074
00:40:16,200 --> 00:40:18,660
tremendous amount of time to save 128
1075
00:40:18,660 --> 00:40:21,030
gigs of memory anywhere push it anywhere
1076
00:40:21,030 --> 00:40:23,309
right I mean even if you talk about
1077
00:40:23,309 --> 00:40:25,380
extremely fast disk it's really fast
1078
00:40:25,380 --> 00:40:27,089
disk arrays can achieve two gigabytes
1079
00:40:27,089 --> 00:40:29,760
per second let's say let's say let's say
1080
00:40:29,760 --> 00:40:32,069
for let's be generous okay four
1081
00:40:32,069 --> 00:40:35,549
gigabytes per second 128 gigabytes right
1082
00:40:35,549 --> 00:40:37,349
but if I have four gigabytes per second
1083
00:40:37,349 --> 00:40:40,230
I need tens of hard drives right then
1084
00:40:40,230 --> 00:40:42,150
let's let's bring it up to something
1085
00:40:42,150 --> 00:40:44,160
like 512
1086
00:40:44,160 --> 00:40:49,020
let's 512 gigabytes and this is by the
1087
00:40:49,020 --> 00:40:52,369
way doable now on a single machine
1088
00:40:52,369 --> 00:40:55,500
divided by 4 gigabytes and we are
1089
00:40:55,500 --> 00:40:59,250
talking about 2028 seconds right it's a
1090
00:40:59,250 --> 00:41:01,980
disaster if I really want to capture the
1091
00:41:01,980 --> 00:41:03,390
full state and replicate the state of
1092
00:41:03,390 --> 00:41:05,190
the memory in a large computation when I
1093
00:41:05,190 --> 00:41:07,109
use half a terabyte of memory which can
1094
00:41:07,109 --> 00:41:09,720
be done with a system that's twelve
1095
00:41:09,720 --> 00:41:11,490
thousand dollars right now I need a
1096
00:41:11,490 --> 00:41:13,680
hundred and twenty eight seconds that's
1097
00:41:13,680 --> 00:41:14,609
about two minutes
1098
00:41:14,609 --> 00:41:17,940
I wanted to replicate every one minute
1099
00:41:17,940 --> 00:41:20,010
and I need two minutes to replicate if
1100
00:41:20,010 --> 00:41:22,109
you replicate often enough you don't do
1101
00:41:22,109 --> 00:41:23,940
any computation you just replicate to
1102
00:41:23,940 --> 00:41:24,930
see how these Astros
1103
00:41:24,930 --> 00:41:27,030
this potentially is you can't afford two
1104
00:41:27,030 --> 00:41:30,030
minutes of replication right in general
1105
00:41:30,030 --> 00:41:31,349
even if you do it every once an hour
1106
00:41:31,349 --> 00:41:32,550
it's gonna hurt it's gonna hurt your
1107
00:41:32,550 --> 00:41:34,170
bottom line it's gonna be problematic
1108
00:41:34,170 --> 00:41:36,119
not to mention that because the
1109
00:41:36,119 --> 00:41:39,270
replication process is so long any issue
1110
00:41:39,270 --> 00:41:42,210
happening while you replicate has to be
1111
00:41:42,210 --> 00:41:44,220
taken into consideration and could be
1112
00:41:44,220 --> 00:41:46,890
problematic in itself right so in
1113
00:41:46,890 --> 00:41:48,390
databases for example people talk about
1114
00:41:48,390 --> 00:41:51,809
failure during recovery or failure
1115
00:41:51,809 --> 00:41:55,079
during checkpointing right who says you
1116
00:41:55,079 --> 00:41:56,760
can I mean in two minutes a lot of
1117
00:41:56,760 --> 00:41:59,069
things can happen right what happens if
1118
00:41:59,069 --> 00:42:00,480
you die in the two minutes in which you
1119
00:42:00,480 --> 00:42:03,540
were dumping the state well at least you
1120
00:42:03,540 --> 00:42:04,800
have to throw away everything you were
1121
00:42:04,800 --> 00:42:06,089
doing because you're not sure where you
1122
00:42:06,089 --> 00:42:08,849
died right it's very hard to say where
1123
00:42:08,849 --> 00:42:10,680
were you you see even the information
1124
00:42:10,680 --> 00:42:13,470
where were you it's very hard to capture
1125
00:42:13,470 --> 00:42:17,480
and propagate fast enough right so
1126
00:42:17,480 --> 00:42:21,230
because these processors move so fast
1127
00:42:21,230 --> 00:42:24,119
capturing very fine-grained information
1128
00:42:24,119 --> 00:42:25,829
about what is it they are doing at any
1129
00:42:25,829 --> 00:42:27,390
given moment of time its itself a
1130
00:42:27,390 --> 00:42:29,970
tremendously hard problem right that is
1131
00:42:29,970 --> 00:42:32,400
reflected to some extent into into this
1132
00:42:32,400 --> 00:42:35,220
thing so it's a matter of relative speed
1133
00:42:35,220 --> 00:42:37,559
of various pieces right if things move
1134
00:42:37,559 --> 00:42:39,180
too fast you have no chance to maintain
1135
00:42:39,180 --> 00:42:41,010
any kind of very very tight replication
1136
00:42:41,010 --> 00:42:43,260
so in general it's going to be easy to
1137
00:42:43,260 --> 00:42:46,619
define a very nice convenient kind of
1138
00:42:46,619 --> 00:42:49,079
replication but for the most part you
1139
00:42:49,079 --> 00:42:50,640
might not get anything close to that
1140
00:42:50,640 --> 00:42:53,099
with consistency consistency is even
1141
00:42:53,099 --> 00:42:55,980
worse than that okay so for example some
1142
00:42:55,980 --> 00:42:57,720
ideas you might actually apply in here
1143
00:42:57,720 --> 00:42:59,579
might be you know what I know the state
1144
00:42:59,579 --> 00:43:00,990
is very large and so on but maybe if I
1145
00:43:00,990 --> 00:43:03,930
keep track of sequences of actions a
1146
00:43:03,930 --> 00:43:05,520
very very high level actions that led to
1147
00:43:05,520 --> 00:43:09,869
the changes in the in the state if I can
1148
00:43:09,869 --> 00:43:12,119
play back those actions so I might dump
1149
00:43:12,119 --> 00:43:14,099
all the state but very rarely maybe once
1150
00:43:14,099 --> 00:43:16,200
a day and then try to play back all the
1151
00:43:16,200 --> 00:43:17,130
things that were doing in the meantime
1152
00:43:17,130 --> 00:43:19,440
in order to recover now that's going to
1153
00:43:19,440 --> 00:43:21,839
have the good effect and a bad effect
1154
00:43:21,839 --> 00:43:23,309
the good effect is going to be is going
1155
00:43:23,309 --> 00:43:24,480
to significantly reduce the amount of
1156
00:43:24,480 --> 00:43:25,859
information I need to capture in the
1157
00:43:25,859 --> 00:43:28,109
replicas the bad effect is I might take
1158
00:43:28,109 --> 00:43:31,170
a long time to replicate I mean to
1159
00:43:31,170 --> 00:43:33,420
recover from the replicas okay this
1160
00:43:33,420 --> 00:43:35,339
problem for example it's also similar to
1161
00:43:35,339 --> 00:43:38,400
a relative related issue within spec
1162
00:43:38,400 --> 00:43:40,980
redundancy in hardware for storage how
1163
00:43:40,980 --> 00:43:43,710
many people know about raid to some
1164
00:43:43,710 --> 00:43:45,390
extent raid it's about replicas and
1165
00:43:45,390 --> 00:43:47,250
about recovery there's a replication
1166
00:43:47,250 --> 00:43:49,200
it's related to recovery at least it's
1167
00:43:49,200 --> 00:43:53,760
some aspects of it so if everything goes
1168
00:43:53,760 --> 00:43:57,060
fine right in a raid system things get
1169
00:43:57,060 --> 00:43:58,830
replicated in a funny way I mean depends
1170
00:43:58,830 --> 00:44:00,270
on which raid you use I mean they might
1171
00:44:00,270 --> 00:44:01,680
be mirrored the night actually use
1172
00:44:01,680 --> 00:44:03,840
checksums and whatnot right and it runs
1173
00:44:03,840 --> 00:44:06,000
fast as long as nothing wrong happen
1174
00:44:06,000 --> 00:44:08,490
when something wrong happens it might
1175
00:44:08,490 --> 00:44:09,810
actually take you an enormous amount of
1176
00:44:09,810 --> 00:44:11,220
time to recover that from the something
1177
00:44:11,220 --> 00:44:12,540
from for example if a hard drive fails
1178
00:44:12,540 --> 00:44:14,400
you can plug in another hard drive then
1179
00:44:14,400 --> 00:44:16,500
recover the raid array it might easily
1180
00:44:16,500 --> 00:44:19,530
take a couple of hours to do so right so
1181
00:44:19,530 --> 00:44:21,180
a big consideration with respect to
1182
00:44:21,180 --> 00:44:24,540
replication is look and the reason
1183
00:44:24,540 --> 00:44:27,240
you're replicating and figure out how
1184
00:44:27,240 --> 00:44:28,650
much do you care about the speed at
1185
00:44:28,650 --> 00:44:30,630
which you can recover from the replicas
1186
00:44:30,630 --> 00:44:33,090
so if you drap replicate for the purpose
1187
00:44:33,090 --> 00:44:35,460
of conserving the information one way or
1188
00:44:35,460 --> 00:44:36,750
another you have to ask yourself the
1189
00:44:36,750 --> 00:44:38,790
question what's the price for example a
1190
00:44:38,790 --> 00:44:41,280
very nice way to to do for example
1191
00:44:41,280 --> 00:44:44,250
backups right so usually the backups the
1192
00:44:44,250 --> 00:44:45,960
daily backups go on some reasonably fast
1193
00:44:45,960 --> 00:44:48,000
storage then the monthly backups go on
1194
00:44:48,000 --> 00:44:50,130
some further away storage and we also
1195
00:44:50,130 --> 00:44:53,010
have yearly backups and decade backups
1196
00:44:53,010 --> 00:44:55,520
on some sort of tapes that are somewhere
1197
00:44:55,520 --> 00:44:58,340
easily can take you months to recover
1198
00:44:58,340 --> 00:45:01,860
from certain backups right so the fact
1199
00:45:01,860 --> 00:45:03,150
that you have a replica doesn't mean
1200
00:45:03,150 --> 00:45:05,220
you're gonna get it anytime soon okay
1201
00:45:05,220 --> 00:45:07,440
and that in itself it's a problem and
1202
00:45:07,440 --> 00:45:09,690
you see we say months and that sounds
1203
00:45:09,690 --> 00:45:13,500
horrible but for the financial guys ten
1204
00:45:13,500 --> 00:45:16,080
milliseconds is horrible so if you do
1205
00:45:16,080 --> 00:45:17,340
some replication and it takes you ten
1206
00:45:17,340 --> 00:45:18,810
milliseconds to get to the replication
1207
00:45:18,810 --> 00:45:20,850
you lost the game because the other guys
1208
00:45:20,850 --> 00:45:22,020
aren't in front of you and they make
1209
00:45:22,020 --> 00:45:25,380
money and you don't write on the wall
1210
00:45:25,380 --> 00:45:27,660
street now with the fast trading because
1211
00:45:27,660 --> 00:45:28,860
all of them use the same kind of
1212
00:45:28,860 --> 00:45:30,120
algorithms and the same kind of things
1213
00:45:30,120 --> 00:45:32,250
right it's kind of going around if
1214
00:45:32,250 --> 00:45:33,600
you're late to the game you lose money
1215
00:45:33,600 --> 00:45:34,740
it's not so much that you don't make
1216
00:45:34,740 --> 00:45:37,020
money you literally lose money and in
1217
00:45:37,020 --> 00:45:38,640
the end this is a zero-sum game right if
1218
00:45:38,640 --> 00:45:39,960
somebody makes money somebody has to
1219
00:45:39,960 --> 00:45:42,120
lose money I mean kind of the trend is
1220
00:45:42,120 --> 00:45:44,130
things I'm going upwards but you have
1221
00:45:44,130 --> 00:45:45,210
enough of this fluctuation
1222
00:45:45,210 --> 00:45:47,730
right so
1223
00:45:47,730 --> 00:45:51,720
always speed at which things can be
1224
00:45:51,720 --> 00:45:54,390
accessed or replicated or recover from a
1225
00:45:54,390 --> 00:45:56,370
replica have to be put in the context of
1226
00:45:56,370 --> 00:45:59,040
the specific application right months to
1227
00:45:59,040 --> 00:46:02,940
recover might be nothing if things don't
1228
00:46:02,940 --> 00:46:05,430
happen faster than at the level of a
1229
00:46:05,430 --> 00:46:07,770
year for example right it's all relative
1230
00:46:07,770 --> 00:46:09,810
and all has to be considered so the
1231
00:46:09,810 --> 00:46:11,820
overall picture I'm trying to paint is
1232
00:46:11,820 --> 00:46:13,170
the following when it comes to
1233
00:46:13,170 --> 00:46:15,960
replication and consistency it's worth
1234
00:46:15,960 --> 00:46:17,970
taking the time to really find what you
1235
00:46:17,970 --> 00:46:19,350
care about and what you don't care about
1236
00:46:19,350 --> 00:46:21,750
much more so than the other issues if
1237
00:46:21,750 --> 00:46:23,820
you do so you could pick a better
1238
00:46:23,820 --> 00:46:25,950
compromise with respect to what's the
1239
00:46:25,950 --> 00:46:27,930
specific replication technique you're
1240
00:46:27,930 --> 00:46:29,340
gonna use for your specific application
1241
00:46:29,340 --> 00:46:33,480
and I'm giving this example over and
1242
00:46:33,480 --> 00:46:35,190
over and over again but I think to a
1243
00:46:35,190 --> 00:46:37,350
large extent Amazon is the master of
1244
00:46:37,350 --> 00:46:39,180
picking the right compromise for the
1245
00:46:39,180 --> 00:46:40,590
replication I mentioned this before
1246
00:46:40,590 --> 00:46:43,500
right they don't even bother that much
1247
00:46:43,500 --> 00:46:47,550
with keeping track of for them right if
1248
00:46:47,550 --> 00:46:52,710
somebody buys an item and let's say on
1249
00:46:52,710 --> 00:46:54,120
the East Coast it goes to a different
1250
00:46:54,120 --> 00:46:56,190
server than somebody buying the item on
1251
00:46:56,190 --> 00:46:57,330
the west coast even though it looks like
1252
00:46:57,330 --> 00:47:00,000
amazon.com to all of them instead of
1253
00:47:00,000 --> 00:47:01,830
freaking out and saying you can't
1254
00:47:01,830 --> 00:47:03,870
possibly sell the same item to to both
1255
00:47:03,870 --> 00:47:06,810
guys that has a significant constraint
1256
00:47:06,810 --> 00:47:08,250
on the replication that means if
1257
00:47:08,250 --> 00:47:10,530
somebody buys it here the database
1258
00:47:10,530 --> 00:47:12,210
server whatever server keeps track of
1259
00:47:12,210 --> 00:47:13,350
the transaction they have their own
1260
00:47:13,350 --> 00:47:15,480
database like technology right would
1261
00:47:15,480 --> 00:47:16,830
have immediately to contact all the
1262
00:47:16,830 --> 00:47:18,420
other servers and tell them this person
1263
00:47:18,420 --> 00:47:19,830
bought the item instead of doing that
1264
00:47:19,830 --> 00:47:22,950
they say yeah whatever why can't they do
1265
00:47:22,950 --> 00:47:24,960
that right they've done this judgment
1266
00:47:24,960 --> 00:47:27,090
and say it's not that big of a problem
1267
00:47:27,090 --> 00:47:29,130
to sell items you don't have it's much
1268
00:47:29,130 --> 00:47:32,160
bigger problem to make sure so for
1269
00:47:32,160 --> 00:47:33,930
example if you would buy an item and it
1270
00:47:33,930 --> 00:47:35,940
will take about ten seconds for the
1271
00:47:35,940 --> 00:47:37,650
system to come back and say yes indeed
1272
00:47:37,650 --> 00:47:40,950
the item is yours most people would
1273
00:47:40,950 --> 00:47:43,410
abandon the service right the
1274
00:47:43,410 --> 00:47:46,260
uncertainty the uncertainty whether your
1275
00:47:46,260 --> 00:47:47,730
purchase went through or didn't go
1276
00:47:47,730 --> 00:47:50,220
through it's really alienated for people
1277
00:47:50,220 --> 00:47:51,870
the worst of the worst was the Facebook
1278
00:47:51,870 --> 00:47:53,940
incident on the New York on the Nasdaq
1279
00:47:53,940 --> 00:47:55,890
oh by the way a twitter decided not to
1280
00:47:55,890 --> 00:47:57,720
go in Nasdaq probably for the same
1281
00:47:57,720 --> 00:47:59,970
reason right I don't know if you've
1282
00:47:59,970 --> 00:48:01,319
heard the news they decided to
1283
00:48:01,319 --> 00:48:04,410
on your socket sheet they hope they have
1284
00:48:04,410 --> 00:48:07,890
better servers maybe okay so the
1285
00:48:07,890 --> 00:48:09,239
uncertainty that something happened or
1286
00:48:09,239 --> 00:48:11,279
didn't happen is really devastating for
1287
00:48:11,279 --> 00:48:12,660
for the humans now here's an interesting
1288
00:48:12,660 --> 00:48:13,229
question
1289
00:48:13,229 --> 00:48:14,519
look computer science is not about
1290
00:48:14,519 --> 00:48:15,989
humans why is this guy talking about
1291
00:48:15,989 --> 00:48:18,420
humans because ultimately that's what
1292
00:48:18,420 --> 00:48:20,369
determines the failure or the massive
1293
00:48:20,369 --> 00:48:23,400
success of anything you do right a sewer
1294
00:48:23,400 --> 00:48:25,019
way to have a disaster on your hands is
1295
00:48:25,019 --> 00:48:28,650
to ignore the user and to say hey what's
1296
00:48:28,650 --> 00:48:30,569
a second in a plane a person's life
1297
00:48:30,569 --> 00:48:32,789
everybody can wait for a second no no no
1298
00:48:32,789 --> 00:48:37,769
okay if if users got used to faster
1299
00:48:37,769 --> 00:48:39,359
things and right it's the difference
1300
00:48:39,359 --> 00:48:41,699
between smooth people say oh I like
1301
00:48:41,699 --> 00:48:44,699
Amazon better because it's smooth what's
1302
00:48:44,699 --> 00:48:47,160
smooth well that 30 milliseconds right
1303
00:48:47,160 --> 00:48:50,640
or oh I really hate this service because
1304
00:48:50,640 --> 00:48:53,039
keeps on being choppy and a lot of
1305
00:48:53,039 --> 00:48:55,109
services died I mean a lot of ecommerce
1306
00:48:55,109 --> 00:48:57,390
sites died because the really insistent
1307
00:48:57,390 --> 00:48:59,190
on the wrong kind of consistency right
1308
00:48:59,190 --> 00:49:01,440
so this these decisions involving
1309
00:49:01,440 --> 00:49:03,059
looking yeah whatever in just a decision
1310
00:49:03,059 --> 00:49:04,229
they know it's not I mean it's the
1311
00:49:04,229 --> 00:49:07,140
difference between big fortune and
1312
00:49:07,140 --> 00:49:12,299
complete financial ruin okay and it's
1313
00:49:12,299 --> 00:49:14,039
hard as a programmer to tell what will
1314
00:49:14,039 --> 00:49:16,799
irritate the users I'm okay let me let
1315
00:49:16,799 --> 00:49:17,729
me give an example it's not quite
1316
00:49:17,729 --> 00:49:19,349
related to this bug it's an idea how
1317
00:49:19,349 --> 00:49:20,759
much money you could lose for little
1318
00:49:20,759 --> 00:49:23,160
things right in about one minute
1319
00:49:23,160 --> 00:49:26,789
because okay it's it's too short for a
1320
00:49:26,789 --> 00:49:29,609
replication story so a big success story
1321
00:49:29,609 --> 00:49:33,059
obsess the big data mining slash
1322
00:49:33,059 --> 00:49:35,519
statistical software that they have on
1323
00:49:35,519 --> 00:49:39,650
their website is that they held Expedia
1324
00:49:39,650 --> 00:49:41,910
make about thirty million dollars more a
1325
00:49:41,910 --> 00:49:44,190
year by just noticing something so what
1326
00:49:44,190 --> 00:49:45,509
they notice is the following so question
1327
00:49:45,509 --> 00:49:46,859
is how do you find this well that's a
1328
00:49:46,859 --> 00:49:47,430
different story
1329
00:49:47,430 --> 00:49:51,269
they notice that Expedia loses something
1330
00:49:51,269 --> 00:49:53,549
like a large percentage about ten
1331
00:49:53,549 --> 00:49:55,009
fifteen twenty percent of the people
1332
00:49:55,009 --> 00:49:57,420
through the checkout process when all
1333
00:49:57,420 --> 00:49:59,219
these people find a plane they want to
1334
00:49:59,219 --> 00:50:01,589
put twenty key they want to purchase a
1335
00:50:01,589 --> 00:50:02,969
girl they go they go and at some point
1336
00:50:02,969 --> 00:50:05,910
they abandon things and the question was
1337
00:50:05,910 --> 00:50:07,769
why do they abandon things I mean nobody
1338
00:50:07,769 --> 00:50:09,209
notice they abandon things nobody cared
1339
00:50:09,209 --> 00:50:11,670
is like hey they only looked at how many
1340
00:50:11,670 --> 00:50:13,259
plane tickets they sell yeah everything
1341
00:50:13,259 --> 00:50:14,640
is fine SAS comes
1342
00:50:14,640 --> 00:50:17,190
that's a massive data analysis hey guys
1343
00:50:17,190 --> 00:50:19,559
20% of the people abandon the checkout
1344
00:50:19,559 --> 00:50:21,539
process so it turns out that people who
1345
00:50:21,539 --> 00:50:22,559
are abandoning the checkout process
1346
00:50:22,559 --> 00:50:26,249
because of a single line in in the page
1347
00:50:26,249 --> 00:50:30,059
right so when asked to plug in the
1348
00:50:30,059 --> 00:50:32,700
credit card information the line there
1349
00:50:32,700 --> 00:50:38,670
said address credit card address and a
1350
00:50:38,670 --> 00:50:40,170
lot of people were confused what credit
1351
00:50:40,170 --> 00:50:42,150
card address means is if you address of
1352
00:50:42,150 --> 00:50:43,739
the bank that issued the credit card or
1353
00:50:43,739 --> 00:50:45,660
is my address and they abandon this and
1354
00:50:45,660 --> 00:50:48,749
I went to Orbitz they fix this little I
1355
00:50:48,749 --> 00:50:50,009
mean literally all you have to do is say
1356
00:50:50,009 --> 00:50:54,470
the credit card of the cardholder
1357
00:50:54,470 --> 00:50:57,420
instead of credit card address so the
1358
00:50:57,420 --> 00:50:59,880
lesson here is when you're doing this
1359
00:50:59,880 --> 00:51:01,349
kind of interface especially when real
1360
00:51:01,349 --> 00:51:03,930
humans use your stuff try to consult
1361
00:51:03,930 --> 00:51:05,700
with somebody who understands humans and
1362
00:51:05,700 --> 00:51:09,029
say does this make sense because you
1363
00:51:09,029 --> 00:51:13,289
could ruin your company right or makes
1364
00:51:13,289 --> 00:51:15,480
us rich but they discover that they need
1365
00:51:15,480 --> 00:51:17,039
to improve the English in one of the
1366
00:51:17,039 --> 00:51:18,749
sentences right so everything matters
1367
00:51:18,749 --> 00:51:20,489
because remember that okay I'll see you
1368
00:51:20,489 --> 00:51:22,140
next time when we talk in earnest about
1369
00:51:22,140 --> 00:00:00,000
consistency and replication