1
00:00:18,540 --> 00:00:22,480
all right so change of gears today I'm

2
00:00:22,480 --> 00:00:24,070
gonna start talking about consistency in

3
00:00:24,070 --> 00:00:26,230
a replication and since we only have one

4
00:00:26,230 --> 00:00:28,330
hour I want to first give a very broad

5
00:00:28,330 --> 00:00:30,880
overview of where is this going what's

6
00:00:30,880 --> 00:00:33,989
the kind of stuff we want to do in there

7
00:00:33,989 --> 00:00:36,850
and also to paint some sort of a more

8
00:00:36,850 --> 00:00:39,250
realistic picture of what you should

9
00:00:39,250 --> 00:00:41,470
expect all right so now one of the

10
00:00:41,470 --> 00:00:42,610
things I've been saying for all the

11
00:00:42,610 --> 00:00:44,710
class is you're mostly talking about

12
00:00:44,710 --> 00:00:46,210
compromises when it comes to solutions

13
00:00:46,210 --> 00:00:47,650
and there is no such thing as a perfect

14
00:00:47,650 --> 00:00:49,300
solution it turns out that for

15
00:00:49,300 --> 00:00:51,070
consistency and replication that's taken

16
00:00:51,070 --> 00:00:55,030
to almost an extreme you might have some

17
00:00:55,030 --> 00:00:56,980
sort of a shred of a class of reasonably

18
00:00:56,980 --> 00:00:58,390
good solution in the other ones but for

19
00:00:58,390 --> 00:01:00,820
this one you gain on some and

20
00:01:00,820 --> 00:01:02,620
potentially lose massively on another

21
00:01:02,620 --> 00:01:07,270
part so when it comes to consistency and

22
00:01:07,270 --> 00:01:08,830
replication replication is not gonna be

23
00:01:08,830 --> 00:01:11,200
so bad in terms of what does it do but

24
00:01:11,200 --> 00:01:13,630
consistency is gonna be something that's

25
00:01:13,630 --> 00:01:18,130
truly awful in a fundamental way okay on

26
00:01:18,130 --> 00:01:19,810
portions we can kind of resolve the

27
00:01:19,810 --> 00:01:21,369
thing but there is almost no good

28
00:01:21,369 --> 00:01:24,940
solution if you want to have a solution

29
00:01:24,940 --> 00:01:26,950
that gets close to some kind of very

30
00:01:26,950 --> 00:01:28,210
nice semantics in terms of what it

31
00:01:28,210 --> 00:01:30,430
guarantees is disastrously inefficient

32
00:01:30,430 --> 00:01:31,600
right

33
00:01:31,600 --> 00:01:33,880
so if things get pushed to really to an

34
00:01:33,880 --> 00:01:35,140
extreme when it comes to consist

35
00:01:35,140 --> 00:01:37,450
especially consistency now replication

36
00:01:37,450 --> 00:01:39,340
depends depending on how tight it is

37
00:01:39,340 --> 00:01:41,890
connected to consistency can be anywhere

38
00:01:41,890 --> 00:01:44,830
for from yeah no big deal let's just do

39
00:01:44,830 --> 00:01:47,110
it - oops it's about as hard as

40
00:01:47,110 --> 00:01:51,460
consistency okay so I will talk first

41
00:01:51,460 --> 00:01:52,960
about replication because it's easier to

42
00:01:52,960 --> 00:01:54,369
talk about the replication but keep on

43
00:01:54,369 --> 00:01:55,900
pointing out that if you want this

44
00:01:55,900 --> 00:01:57,430
property to hold you're in the land of

45
00:01:57,430 --> 00:02:00,850
consistency okay let me say a few words

46
00:02:00,850 --> 00:02:02,890
about consistency first okay so the

47
00:02:02,890 --> 00:02:05,290
trouble with consistency when the first

48
00:02:05,290 --> 00:02:08,169
problem with consistency is in fact the

49
00:02:08,169 --> 00:02:11,769
definition okay it's the fact that it's

50
00:02:11,769 --> 00:02:13,680
very hard to come up with an actual

51
00:02:13,680 --> 00:02:16,630
definition that we are really really

52
00:02:16,630 --> 00:02:18,130
really sure that that's the definition

53
00:02:18,130 --> 00:02:19,510
of consistency and I want you to

54
00:02:19,510 --> 00:02:21,190
understand this as I explain a lot more

55
00:02:21,190 --> 00:02:22,900
in a database class about this thing

56
00:02:22,900 --> 00:02:25,450
right and I want you to understand from

57
00:02:25,450 --> 00:02:27,400
the beginning you can do write a million

58
00:02:27,400 --> 00:02:29,650
papers about consistency but ultimately

59
00:02:29,650 --> 00:02:30,940
all of them are built on top of

60
00:02:30,940 --> 00:02:32,260
something and that something might be

61
00:02:32,260 --> 00:02:35,200
very shaky to define okay so I'll go

62
00:02:35,200 --> 00:02:36,430
into some details to explain what that

63
00:02:36,430 --> 00:02:37,120
means

64
00:02:37,120 --> 00:02:44,830
okay so consistency is related to the

65
00:02:44,830 --> 00:02:47,409
notion of correctness because if you

66
00:02:47,409 --> 00:02:48,670
don't have a notion of correctness it

67
00:02:48,670 --> 00:02:50,409
makes no sense to say I maintain some

68
00:02:50,409 --> 00:02:51,700
correctness consistency it's about

69
00:02:51,700 --> 00:02:53,920
maintaining some sort of correctness

70
00:02:53,920 --> 00:02:55,540
when you go and you do distributed

71
00:02:55,540 --> 00:02:57,940
systems versus something else but the

72
00:02:57,940 --> 00:02:59,319
trouble in the first place is the notion

73
00:02:59,319 --> 00:03:06,069
of correctness all right so I can define

74
00:03:06,069 --> 00:03:08,049
consistency with respect to correctness

75
00:03:08,049 --> 00:03:10,480
but then how do i define correctness so

76
00:03:10,480 --> 00:03:12,400
let's take an example for example I mean

77
00:03:12,400 --> 00:03:14,440
just to start with you write the program

78
00:03:14,440 --> 00:03:16,030
I ask you to do a task you write a

79
00:03:16,030 --> 00:03:17,470
program how do I know your program is

80
00:03:17,470 --> 00:03:17,859
correct

81
00:03:17,859 --> 00:03:24,220
I mean who says your program is correct

82
00:03:24,220 --> 00:03:26,440
by the way judging by the number of

83
00:03:26,440 --> 00:03:29,200
defects bugs whatever in software in

84
00:03:29,200 --> 00:03:31,900
general even when everybody's convinced

85
00:03:31,900 --> 00:03:33,129
something is correct you still discover

86
00:03:33,129 --> 00:03:34,720
that something is not correct because of

87
00:03:34,720 --> 00:03:37,750
bizarre things right so correctness of

88
00:03:37,750 --> 00:03:39,160
programs I want you to understand

89
00:03:39,160 --> 00:03:41,319
correctness of programs is really into

90
00:03:41,319 --> 00:03:42,849
the fabric of the notion of consistency

91
00:03:42,849 --> 00:03:44,919
and that in itself turns out to be

92
00:03:44,919 --> 00:03:46,810
extremely tricky I mean really really

93
00:03:46,810 --> 00:03:49,540
tricky by the way we do have an idea how

94
00:03:49,540 --> 00:03:51,730
to tackle correctness of programs it's

95
00:03:51,730 --> 00:03:54,340
only devastatingly bad for the

96
00:03:54,340 --> 00:03:57,879
programmer right so you can actually

97
00:03:57,879 --> 00:03:59,650
write programs that are provably correct

98
00:03:59,650 --> 00:04:03,730
but to do so I mean let's see this is

99
00:04:03,730 --> 00:04:08,040
kind of a complete shoot in the dark but

100
00:04:08,040 --> 00:04:10,569
right how many of you heard ever about

101
00:04:10,569 --> 00:04:14,260
proof carrying code proof carrying code

102
00:04:14,260 --> 00:04:16,510
is a piece of software that you wrote

103
00:04:16,510 --> 00:04:19,029
that contains annotations that prove

104
00:04:19,029 --> 00:04:21,728
that the program is correct right to

105
00:04:21,728 --> 00:04:23,020
prove that the program is a program is

106
00:04:23,020 --> 00:04:24,520
correct you have to prove that it

107
00:04:24,520 --> 00:04:25,930
produces the right output given the

108
00:04:25,930 --> 00:04:28,270
right input and you cannot run amok you

109
00:04:28,270 --> 00:04:30,550
get into subtle things like you never

110
00:04:30,550 --> 00:04:31,930
access memory you're not supposed to

111
00:04:31,930 --> 00:04:33,490
access how would you prove that how

112
00:04:33,490 --> 00:04:34,900
would you do this kind of things in

113
00:04:34,900 --> 00:04:37,300
languages right people spend a large

114
00:04:37,300 --> 00:04:39,190
amount of effort in trying to nail this

115
00:04:39,190 --> 00:04:40,120
down okay

116
00:04:40,120 --> 00:04:41,380
and they did come up with some

117
00:04:41,380 --> 00:04:44,139
mechanisms to do so but again

118
00:04:44,139 --> 00:04:46,780
you can write programs that I'm 10 lines

119
00:04:46,780 --> 00:04:48,220
long for which you can prove they are

120
00:04:48,220 --> 00:04:49,840
correct when I was a PhD student at

121
00:04:49,840 --> 00:04:51,280
Cornell for example we got one of those

122
00:04:51,280 --> 00:04:52,569
right so those guys were crazy about

123
00:04:52,569 --> 00:04:54,759
proof carrying code and no notion of

124
00:04:54,759 --> 00:04:56,590
correctness so in the qualifying exams

125
00:04:56,590 --> 00:04:58,840
for the PhD they threw a problem in

126
00:04:58,840 --> 00:05:00,520
right and that said write this program

127
00:05:00,520 --> 00:05:03,550
and prove it it's correct I spend a good

128
00:05:03,550 --> 00:05:06,250
part of half an hour to write five lines

129
00:05:06,250 --> 00:05:07,780
of code and then to prove the five lines

130
00:05:07,780 --> 00:05:09,370
of code were correct good luck in doing

131
00:05:09,370 --> 00:05:10,750
that on industrial scale when you're

132
00:05:10,750 --> 00:05:13,900
doing code design right so keep this in

133
00:05:13,900 --> 00:05:16,330
mind and this is kind of the ultimate

134
00:05:16,330 --> 00:05:18,699
way to say yeah forget about whatever

135
00:05:18,699 --> 00:05:20,440
super-strong type of consistency you

136
00:05:20,440 --> 00:05:21,610
don't even know if your program is

137
00:05:21,610 --> 00:05:24,550
correct so the traditional way to go

138
00:05:24,550 --> 00:05:26,500
around this problem in the first place

139
00:05:26,500 --> 00:05:28,389
is to hang on to some notion of

140
00:05:28,389 --> 00:05:29,889
correctness that's more likely to be

141
00:05:29,889 --> 00:05:31,300
correct another notion of correctness

142
00:05:31,300 --> 00:05:34,030
namely you say let's assume that the

143
00:05:34,030 --> 00:05:35,529
sequential version of the program is

144
00:05:35,529 --> 00:05:38,940
correct right let's now define

145
00:05:38,940 --> 00:05:41,580
correctness for distributed versions

146
00:05:41,580 --> 00:05:43,509
multi-threaded or multi computer or

147
00:05:43,509 --> 00:05:45,159
whatever with respect to the correctness

148
00:05:45,159 --> 00:05:47,110
of the sequential one now by the way

149
00:05:47,110 --> 00:05:49,960
this is the attitude and the approach

150
00:05:49,960 --> 00:05:52,349
taken in the database literature and

151
00:05:52,349 --> 00:05:54,370
database people are the first ones that

152
00:05:54,370 --> 00:05:55,779
actually looked at this notion of

153
00:05:55,779 --> 00:05:57,190
consistency it doesn't matter what the

154
00:05:57,190 --> 00:05:58,839
distributed systems people are talking

155
00:05:58,839 --> 00:06:02,560
about all right why because banks were

156
00:06:02,560 --> 00:06:03,849
extremely interested in relational

157
00:06:03,849 --> 00:06:05,919
databases and they are extremely

158
00:06:05,919 --> 00:06:08,080
concerned about some sort of guarantees

159
00:06:08,080 --> 00:06:10,150
that the distributive version of the

160
00:06:10,150 --> 00:06:11,800
program the multi-threaded multi

161
00:06:11,800 --> 00:06:14,979
anything you want doesn't run amok but

162
00:06:14,979 --> 00:06:16,569
again it's with respect to that

163
00:06:16,569 --> 00:06:19,930
sequential correctness right so one way

164
00:06:19,930 --> 00:06:21,940
to define consistency and I'm gonna come

165
00:06:21,940 --> 00:06:23,949
back to this later is to say whatever

166
00:06:23,949 --> 00:06:26,699
you do make sure in some sense is

167
00:06:26,699 --> 00:06:28,779
equivalent to the sequential program

168
00:06:28,779 --> 00:06:31,479
okay but of course you might have

169
00:06:31,479 --> 00:06:33,310
multiple sub sequential programs running

170
00:06:33,310 --> 00:06:36,430
right and in principle you have to

171
00:06:36,430 --> 00:06:37,990
overlap the execution of such sequential

172
00:06:37,990 --> 00:06:39,610
programs so when you define consistence

173
00:06:39,610 --> 00:06:40,599
you have to say not only that's

174
00:06:40,599 --> 00:06:41,740
equivalent to the sequential execution

175
00:06:41,740 --> 00:06:44,500
but it's also equivalent to a sequential

176
00:06:44,500 --> 00:06:47,620
execution of the sequential programs so

177
00:06:47,620 --> 00:06:49,259
you take your sequential programs you

178
00:06:49,259 --> 00:06:51,399
shuffle them in some order and you run

179
00:06:51,399 --> 00:06:53,380
them one by one now nobody says you

180
00:06:53,380 --> 00:06:55,419
should run it like this but the one way

181
00:06:55,419 --> 00:06:55,820
to do

182
00:06:55,820 --> 00:06:58,610
consistency would be make sure that

183
00:06:58,610 --> 00:07:00,680
you're consistent with at least one of

184
00:07:00,680 --> 00:07:03,110
this possible orders okay so things what

185
00:07:03,110 --> 00:07:05,480
I'm trying to say is when consistency

186
00:07:05,480 --> 00:07:08,990
its involved you have extremely complex

187
00:07:08,990 --> 00:07:11,390
issues popping up and there cannot be

188
00:07:11,390 --> 00:07:14,980
resolved completely okay now throughout

189
00:07:14,980 --> 00:07:17,540
these lectures it's virtually on Tuesday

190
00:07:17,540 --> 00:07:18,770
I'm going to talk about all kinds of

191
00:07:18,770 --> 00:07:20,720
notions of consistency that were

192
00:07:20,720 --> 00:07:21,770
introduced especially in the

193
00:07:21,770 --> 00:07:24,020
distribution systems literature all of

194
00:07:24,020 --> 00:07:26,600
them are iffy okay all of them are

195
00:07:26,600 --> 00:07:28,940
highly problematic there's no such thing

196
00:07:28,940 --> 00:07:30,560
as completely clean ones

197
00:07:30,560 --> 00:07:32,180
database people nailed it down a little

198
00:07:32,180 --> 00:07:33,590
bit better because they have a

199
00:07:33,590 --> 00:07:36,260
connection with this kind of a notion of

200
00:07:36,260 --> 00:07:37,690
sequential consistency where of course

201
00:07:37,690 --> 00:07:39,470
distributed systems people said hey

202
00:07:39,470 --> 00:07:40,970
that's a solved problem in databases

203
00:07:40,970 --> 00:07:44,900
which it is let's invent other kind of

204
00:07:44,900 --> 00:07:46,430
problems and let's try to solve those as

205
00:07:46,430 --> 00:07:48,320
well okay so I'll plow through all of

206
00:07:48,320 --> 00:07:49,580
these things at least at the high level

207
00:07:49,580 --> 00:07:51,920
let's go back and focus on replication

208
00:07:51,920 --> 00:07:54,050
because this is a lot easier to

209
00:07:54,050 --> 00:07:55,970
understand what what does he do and what

210
00:07:55,970 --> 00:07:57,380
is it supposed to do and so on and

211
00:07:57,380 --> 00:08:01,270
sometimes replication it's easy okay

212
00:08:01,270 --> 00:08:04,160
okay so when it comes to I mean

213
00:08:04,160 --> 00:08:05,720
replication is basically what the name

214
00:08:05,720 --> 00:08:09,050
suggests let's take the data and put it

215
00:08:09,050 --> 00:08:10,610
in multiple places now of course the

216
00:08:10,610 --> 00:08:13,070
question is why I'm sure there is a

217
00:08:13,070 --> 00:08:14,360
slide here but let's forget about this

218
00:08:14,360 --> 00:08:17,900
or let's just brainstorm about this why

219
00:08:17,900 --> 00:08:22,460
would i replicate the data that's for

220
00:08:22,460 --> 00:08:26,750
reasons out in there right one of them

221
00:08:26,750 --> 00:08:29,270
is have multiple copies in case you lose

222
00:08:29,270 --> 00:08:31,100
some okay so this is some sort of a

223
00:08:31,100 --> 00:08:33,200
backup right so one reason to do

224
00:08:33,200 --> 00:08:34,640
replication is to have a backup copy

225
00:08:34,640 --> 00:08:38,990
that's very legitimate reason now what's

226
00:08:38,990 --> 00:08:41,090
the difficulty to doing this kind of

227
00:08:41,090 --> 00:08:45,100
backups what could be the difficult I

228
00:08:45,100 --> 00:08:47,480
mean so one is headaches remembering

229
00:08:47,480 --> 00:08:49,100
where your backup copies are let's say

230
00:08:49,100 --> 00:08:54,440
okay that's one one issue I'm sorry well

231
00:08:54,440 --> 00:08:56,180
inconsistent data first of all what does

232
00:08:56,180 --> 00:08:57,890
inconsistent mean and this is very very

233
00:08:57,890 --> 00:09:01,490
tricky right but if I do it for the

234
00:09:01,490 --> 00:09:02,810
purpose of backup am I not care that

235
00:09:02,810 --> 00:09:06,050
they are in sync or was a question is

236
00:09:06,050 --> 00:09:07,910
when do I fire up the replication

237
00:09:07,910 --> 00:09:09,800
process right so when do I

238
00:09:09,800 --> 00:09:11,570
create a copy so if I want a replication

239
00:09:11,570 --> 00:09:13,430
for the purpose of backup the question

240
00:09:13,430 --> 00:09:17,990
is when do i propagate changes right so

241
00:09:17,990 --> 00:09:19,700
replication without changes is truly

242
00:09:19,700 --> 00:09:23,360
easy if you have a very slow changing

243
00:09:23,360 --> 00:09:26,329
resource of any kind replication it's a

244
00:09:26,329 --> 00:09:28,370
walk in the park you just whatever do it

245
00:09:28,370 --> 00:09:31,459
okay if you have an extremely fast

246
00:09:31,459 --> 00:09:33,800
changing resource replication becomes a

247
00:09:33,800 --> 00:09:36,829
nightmare for example imagine a database

248
00:09:36,829 --> 00:09:39,920
that processes hundreds of thousands of

249
00:09:39,920 --> 00:09:41,600
transactions per second and we have

250
00:09:41,600 --> 00:09:43,970
those databases by the way for example

251
00:09:43,970 --> 00:09:46,820
visa has no clearly all the students

252
00:09:46,820 --> 00:09:47,899
actually have to remember what they were

253
00:09:47,899 --> 00:09:49,339
how they did and how they change the

254
00:09:49,339 --> 00:09:51,829
state of the system so replication there

255
00:09:51,829 --> 00:09:54,589
is much much much harder than are you

256
00:09:54,589 --> 00:09:57,230
backing up I don't know your term papers

257
00:09:57,230 --> 00:09:57,880
or whatnot

258
00:09:57,880 --> 00:10:01,880
for example right again it depends on

259
00:10:01,880 --> 00:10:06,290
how fast the let's say the primary copy

260
00:10:06,290 --> 00:10:07,579
is changing okay

261
00:10:07,579 --> 00:10:10,760
so again this is what I'm saying that

262
00:10:10,760 --> 00:10:12,140
replication could be easy or could be

263
00:10:12,140 --> 00:10:13,579
extremely hard for example how do you

264
00:10:13,579 --> 00:10:14,750
think they do replication in that

265
00:10:14,750 --> 00:10:16,490
database they literally have to probably

266
00:10:16,490 --> 00:10:18,110
remember the sequence of actions to be

267
00:10:18,110 --> 00:10:19,940
able to apply them again all right and

268
00:10:19,940 --> 00:10:21,589
it's very tricky but the database

269
00:10:21,589 --> 00:10:23,060
community did deal with that right

270
00:10:23,060 --> 00:10:26,000
techniques like for example logs in

271
00:10:26,000 --> 00:10:27,709
which you simply write what is it that

272
00:10:27,709 --> 00:10:29,420
you were doing so you don't have a copy

273
00:10:29,420 --> 00:10:31,370
per se you have a sequence of actions

274
00:10:31,370 --> 00:10:32,899
that will produce a copy so do you need

275
00:10:32,899 --> 00:10:35,600
one so what a copy means and what you

276
00:10:35,600 --> 00:10:37,820
replicate it's kind of in the eye of the

277
00:10:37,820 --> 00:10:40,910
designer not necessarily beholder so you

278
00:10:40,910 --> 00:10:42,709
don't necessarily need a copy you don't

279
00:10:42,709 --> 00:10:45,770
need to say stop and copy you can have

280
00:10:45,770 --> 00:10:48,050
anything that could lead with lyrtle

281
00:10:48,050 --> 00:10:51,020
processing to a copy now a fundamental

282
00:10:51,020 --> 00:10:53,480
question about the replication is how

283
00:10:53,480 --> 00:10:56,029
often will you go and use the replicas

284
00:10:56,029 --> 00:10:58,070
to do something ok so when it comes to

285
00:10:58,070 --> 00:11:01,070
backups the answer is almost never right

286
00:11:01,070 --> 00:11:04,250
most of the time you don't care that the

287
00:11:04,250 --> 00:11:06,800
copy even exists only when by mistake

288
00:11:06,800 --> 00:11:07,940
you delete your files and you do

289
00:11:07,940 --> 00:11:09,620
something you go and recover from there

290
00:11:09,620 --> 00:11:10,820
from the backups for example I don't

291
00:11:10,820 --> 00:11:13,399
remember every ever going back to a

292
00:11:13,399 --> 00:11:14,990
backup and doing something bad hey you

293
00:11:14,990 --> 00:11:16,730
still need to do the backups because

294
00:11:16,730 --> 00:11:18,079
you're gonna run in trouble sooner or

295
00:11:18,079 --> 00:11:18,350
later

296
00:11:18,350 --> 00:11:20,990
supposedly okay all right so that's one

297
00:11:20,990 --> 00:11:22,810
reason to do replication not to move in

298
00:11:22,810 --> 00:11:25,000
formation and then the main question is

299
00:11:25,000 --> 00:11:26,860
how often do you replicate because

300
00:11:26,860 --> 00:11:28,420
whatever you did not replicate you're

301
00:11:28,420 --> 00:11:29,770
gonna lose in case of some sort of

302
00:11:29,770 --> 00:11:31,090
failure right so when it comes to

303
00:11:31,090 --> 00:11:33,340
backups common practice is to replicate

304
00:11:33,340 --> 00:11:35,710
every day for example of course if it

305
00:11:35,710 --> 00:11:36,970
changed if you didn't change again

306
00:11:36,970 --> 00:11:38,529
you're out of the woods you're fine

307
00:11:38,529 --> 00:11:40,120
what's what's another reason to

308
00:11:40,120 --> 00:11:42,839
replicate yes

309
00:11:48,170 --> 00:11:50,660
right so performance so another reason

310
00:11:50,660 --> 00:11:53,390
to replicate is performance and this is

311
00:11:53,390 --> 00:11:55,400
exactly what Akamai is doing right so

312
00:11:55,400 --> 00:11:56,870
you pay a lot of money as a company to

313
00:11:56,870 --> 00:11:59,320
Akamai because they have really clever

314
00:11:59,320 --> 00:12:01,790
replication algorithms and they are in

315
00:12:01,790 --> 00:12:04,820
some sense geographical replication

316
00:12:04,820 --> 00:12:07,100
algorithms under the assumption that of

317
00:12:07,100 --> 00:12:08,480
course I'm gonna access our closer

318
00:12:08,480 --> 00:12:10,910
resource faster than another far away

319
00:12:10,910 --> 00:12:13,280
resource and keep in mind that one of

320
00:12:13,280 --> 00:12:15,080
the big big problems and being enemies

321
00:12:15,080 --> 00:12:17,330
in distributed systems is latency you

322
00:12:17,330 --> 00:12:20,630
see we can deal with a lot of other

323
00:12:20,630 --> 00:12:22,220
problems right for example if you don't

324
00:12:22,220 --> 00:12:23,930
have enough bandwidth we know how to

325
00:12:23,930 --> 00:12:25,550
increase the bandwidth we put more money

326
00:12:25,550 --> 00:12:26,870
into it and we increase the bandwidth

327
00:12:26,870 --> 00:12:29,870
okay this is why the Internet gets more

328
00:12:29,870 --> 00:12:31,250
and more and more and more bandwidth but

329
00:12:31,250 --> 00:12:32,660
the internet doesn't get lower and lower

330
00:12:32,660 --> 00:12:36,380
and lower latency right so latency it's

331
00:12:36,380 --> 00:12:38,270
a physical limit we don't know how to go

332
00:12:38,270 --> 00:12:42,380
beyond I mean one way so let's say we

333
00:12:42,380 --> 00:12:44,030
maxed out how much information we can

334
00:12:44,030 --> 00:12:47,090
send through one optic fiber which we

335
00:12:47,090 --> 00:12:48,380
haven't yet by the way

336
00:12:48,380 --> 00:12:49,820
and some people compute the limit to be

337
00:12:49,820 --> 00:12:52,070
something like 40,000 times larger than

338
00:12:52,070 --> 00:12:54,050
it's actually accomplished now so let's

339
00:12:54,050 --> 00:12:55,760
say you even max that out hey you put

340
00:12:55,760 --> 00:12:57,350
more cables through the ocean and you

341
00:12:57,350 --> 00:12:59,360
get more bandwidth but that's not the

342
00:12:59,360 --> 00:13:06,590
case for for latency right so PBS yes

343
00:13:06,590 --> 00:13:08,600
they had a documentary about speed and

344
00:13:08,600 --> 00:13:10,070
part of the speed was internet speed

345
00:13:10,070 --> 00:13:12,200
right I gave you the example with they

346
00:13:12,200 --> 00:13:14,330
want to put between London and New York

347
00:13:14,330 --> 00:13:16,330
that cable for the ocean that straighter

348
00:13:16,330 --> 00:13:18,680
okay so I've heard another crazy idea

349
00:13:18,680 --> 00:13:20,390
yesterday in the documentary and this is

350
00:13:20,390 --> 00:13:22,160
all about shading a little bit of

351
00:13:22,160 --> 00:13:24,710
latency right so apparently a company

352
00:13:24,710 --> 00:13:27,050
and it kind of failed wanted to put a

353
00:13:27,050 --> 00:13:30,020
straighter fibre between Chicago and New

354
00:13:30,020 --> 00:13:31,640
York why because you have companies

355
00:13:31,640 --> 00:13:33,080
trading on both the Chicago Stock

356
00:13:33,080 --> 00:13:35,170
Exchange and the New York Stock Exchange

357
00:13:35,170 --> 00:13:37,010
whenever you have the same thing

358
00:13:37,010 --> 00:13:39,170
happening in two places whoever gets the

359
00:13:39,170 --> 00:13:40,520
information across that something

360
00:13:40,520 --> 00:13:42,080
changed will win will make money

361
00:13:42,080 --> 00:13:44,120
these guys make monies on fractions of a

362
00:13:44,120 --> 00:13:45,980
penny just because they do a tremendous

363
00:13:45,980 --> 00:13:47,420
number of transactions okay

364
00:13:47,420 --> 00:13:49,790
so it's a game of fractions of pennies

365
00:13:49,790 --> 00:13:51,770
but just doing many many many of those

366
00:13:51,770 --> 00:13:53,570
right so they were interviewing these

367
00:13:53,570 --> 00:13:57,200
guys that are basically responsible for

368
00:13:57,200 --> 00:14:01,130
1% of all the trades on New York Stock

369
00:14:01,130 --> 00:14:01,580
Exchange

370
00:14:01,580 --> 00:14:03,140
something like that right and they are

371
00:14:03,140 --> 00:14:04,730
all kind of just babysitting those

372
00:14:04,730 --> 00:14:07,370
millisecond trading machines and in that

373
00:14:07,370 --> 00:14:09,080
and they were talking about the fact

374
00:14:09,080 --> 00:14:10,490
that one of the companies were trying to

375
00:14:10,490 --> 00:14:11,960
put a straighter fiber but they had

376
00:14:11,960 --> 00:14:13,370
significant trouble because they had to

377
00:14:13,370 --> 00:14:16,100
go through people's properties then they

378
00:14:16,100 --> 00:14:18,500
try to make an almost straight fiber and

379
00:14:18,500 --> 00:14:20,480
they were shaving off a one millisecond

380
00:14:20,480 --> 00:14:22,520
at 1.4 millisecond we are talking about

381
00:14:22,520 --> 00:14:24,830
saving 1.4 milliseconds in latency

382
00:14:24,830 --> 00:14:26,900
between Chicago and New York and then

383
00:14:26,900 --> 00:14:28,700
this guy said hey we have a secret

384
00:14:28,700 --> 00:14:29,120
weapon

385
00:14:29,120 --> 00:14:30,320
we figure out something that's more

386
00:14:30,320 --> 00:14:33,950
interesting and it turns out according

387
00:14:33,950 --> 00:14:36,040
to them that the secret is to use

388
00:14:36,040 --> 00:14:38,780
microwave technology to send the signals

389
00:14:38,780 --> 00:14:40,940
through air rather the seventh route

390
00:14:40,940 --> 00:14:43,880
optic fiber because this is why it's

391
00:14:43,880 --> 00:14:45,590
important in all physics the speed of

392
00:14:45,590 --> 00:14:48,050
light is higher through air than it is

393
00:14:48,050 --> 00:14:50,930
for glass and they can shave off more

394
00:14:50,930 --> 00:14:52,700
than that one millisecond by sending

395
00:14:52,700 --> 00:14:56,360
things from microwave towers okay at

396
00:14:56,360 --> 00:14:58,730
least they hope now of course they have

397
00:14:58,730 --> 00:15:00,230
to put their service a night next to the

398
00:15:00,230 --> 00:15:01,850
microwave towers to make sure that there

399
00:15:01,850 --> 00:15:03,560
is absolutely no delay to get to the

400
00:15:03,560 --> 00:15:05,450
microwave tower which is next in the

401
00:15:05,450 --> 00:15:07,400
other place right so it can actually get

402
00:15:07,400 --> 00:15:11,900
quite crazy okay so keep this in mind I

403
00:15:11,900 --> 00:15:13,370
mean this is gonna be a fight that's

404
00:15:13,370 --> 00:15:15,080
gonna go on forever until somebody

405
00:15:15,080 --> 00:15:17,390
figures out how to do some sort of

406
00:15:17,390 --> 00:15:20,690
instantaneous communication okay latency

407
00:15:20,690 --> 00:15:22,790
cannot be reduced so when it comes to

408
00:15:22,790 --> 00:15:25,700
replicas that becomes very either

409
00:15:25,700 --> 00:15:28,520
advantageous you could do replication to

410
00:15:28,520 --> 00:15:30,650
mask latency and it's exactly what was

411
00:15:30,650 --> 00:15:31,790
mentioned before and this is precisely

412
00:15:31,790 --> 00:15:34,940
what tech - internet it's a big world it

413
00:15:34,940 --> 00:15:40,070
takes to cross the internet to go us to

414
00:15:40,070 --> 00:15:41,540
Australia you're already talking about

415
00:15:41,540 --> 00:15:43,580
hundreds of milliseconds okay no matter

416
00:15:43,580 --> 00:15:46,040
what even if you're in the financial

417
00:15:46,040 --> 00:15:48,200
markets and so on right I think the

418
00:15:48,200 --> 00:15:49,940
latency through the fiber even

419
00:15:49,940 --> 00:15:52,520
frustrated Fiber New York to Tokyo it's

420
00:15:52,520 --> 00:15:55,280
about 170 milliseconds okay that's it

421
00:15:55,280 --> 00:15:56,780
you can't go below that I mean you can

422
00:15:56,780 --> 00:15:58,400
do computations if you want and what you

423
00:15:58,400 --> 00:16:00,410
put the speed of light it's about I

424
00:16:00,410 --> 00:16:01,940
think about 60% of the speed of light

425
00:16:01,940 --> 00:16:03,740
through the for the glass now of course

426
00:16:03,740 --> 00:16:05,330
a problem we're sending it through a air

427
00:16:05,330 --> 00:16:08,780
is what you must have a straight line it

428
00:16:08,780 --> 00:16:10,820
just happens that the earth is curved if

429
00:16:10,820 --> 00:16:13,460
you actually you can do this and you can

430
00:16:13,460 --> 00:16:15,139
do this with radio waves you can bounce

431
00:16:15,139 --> 00:16:17,779
sit off the ionosphere right or some

432
00:16:17,779 --> 00:16:19,850
other parts of the atmosphere but that

433
00:16:19,850 --> 00:16:21,350
makes the path even longer so you're

434
00:16:21,350 --> 00:16:24,259
worse than the fiber it's weird it's a

435
00:16:24,259 --> 00:16:27,199
physicists world suddenly right okay

436
00:16:27,199 --> 00:16:28,639
nevertheless so you have a large

437
00:16:28,639 --> 00:16:31,579
Internet and to cross it it can take 200

438
00:16:31,579 --> 00:16:34,129
milliseconds easily but if you do

439
00:16:34,129 --> 00:16:37,579
replication a binary do you know how

440
00:16:37,579 --> 00:16:38,989
fast you can access for example of

441
00:16:38,989 --> 00:16:41,569
computing the department you can tell

442
00:16:41,569 --> 00:16:43,790
right in the pink program that's exactly

443
00:16:43,790 --> 00:16:47,299
that so what does ping do sends a

444
00:16:47,299 --> 00:16:50,389
special packet and the network cards

445
00:16:50,389 --> 00:16:51,559
that receives the packet immediately

446
00:16:51,559 --> 00:16:53,269
replies the pink was introduced

447
00:16:53,269 --> 00:16:55,189
precisely in order to estimate latencies

448
00:16:55,189 --> 00:16:59,059
right and this is how you know it's good

449
00:16:59,059 --> 00:17:00,589
when you get latencies well below a

450
00:17:00,589 --> 00:17:02,239
millisecond right so if you pink a

451
00:17:02,239 --> 00:17:03,139
machine in the department you have a

452
00:17:03,139 --> 00:17:04,398
decent connection you're gonna get zero

453
00:17:04,398 --> 00:17:06,619
point five milliseconds you ping

454
00:17:06,619 --> 00:17:08,089
yahoo.com you're gonna get 80

455
00:17:08,089 --> 00:17:09,949
milliseconds I don't even know where who

456
00:17:09,949 --> 00:17:13,250
is right you pink something in Australia

457
00:17:13,250 --> 00:17:15,619
or something in in Japan it's gonna be

458
00:17:15,619 --> 00:17:16,970
200 something milliseconds you can even

459
00:17:16,970 --> 00:17:19,609
try it right if you have a Linux machine

460
00:17:19,609 --> 00:17:21,349
now it's possible to do it from Windows

461
00:17:21,349 --> 00:17:23,659
but it's more elaborate apparently right

462
00:17:23,659 --> 00:17:27,679
so by placing replicas close by you can

463
00:17:27,679 --> 00:17:30,490
get within that one to two milliseconds

464
00:17:30,490 --> 00:17:33,200
rather than 200 milliseconds 200

465
00:17:33,200 --> 00:17:34,490
milliseconds can be devastating for

466
00:17:34,490 --> 00:17:39,070
example for user experience right so

467
00:17:39,070 --> 00:17:42,169
also that's an indication of to what

468
00:17:42,169 --> 00:17:43,700
level you should be paranoid so I want

469
00:17:43,700 --> 00:17:44,929
you to remember these numbers because

470
00:17:44,929 --> 00:17:46,250
they turn out to be very important right

471
00:17:46,250 --> 00:17:50,179
so arguably humans cannot perceive

472
00:17:50,179 --> 00:17:52,539
anything that's below 30 milliseconds

473
00:17:52,539 --> 00:17:55,309
this is equivalent to roughly 30 frames

474
00:17:55,309 --> 00:17:57,710
per second 30 35 frames per second now

475
00:17:57,710 --> 00:17:59,389
some people claim that they feel better

476
00:17:59,389 --> 00:18:01,669
when they play video games at 60 frames

477
00:18:01,669 --> 00:18:03,830
per second some people brag about their

478
00:18:03,830 --> 00:18:06,889
TVs that can do 120 frames per second

479
00:18:06,889 --> 00:18:09,110
none of that matters when it comes to

480
00:18:09,110 --> 00:18:11,179
the refresh rate for the for the brain I

481
00:18:11,179 --> 00:18:13,159
mean simply the visual system of humans

482
00:18:13,159 --> 00:18:15,409
cannot run faster than something like 30

483
00:18:15,409 --> 00:18:18,320
milliseconds whatever by the way that's

484
00:18:18,320 --> 00:18:20,779
one of the key ingredients in any magic

485
00:18:20,779 --> 00:18:24,559
show the hand can move faster than 30

486
00:18:24,559 --> 00:18:26,029
milliseconds you can move your hand if

487
00:18:26,029 --> 00:18:26,960
you're good you can move your hand from

488
00:18:26,960 --> 00:18:28,590
here to here and back

489
00:18:28,590 --> 00:18:34,169
30 milliseconds right or definitely from

490
00:18:34,169 --> 00:18:35,580
here to here faster than 30 milliseconds

491
00:18:35,580 --> 00:18:37,769
that means the visual system cannot get

492
00:18:37,769 --> 00:18:41,399
the movement right so magicians are very

493
00:18:41,399 --> 00:18:43,350
fast movers in the end and they also

494
00:18:43,350 --> 00:18:45,240
know how to distract you right

495
00:18:45,240 --> 00:18:47,279
so beyond a certain point is not worth

496
00:18:47,279 --> 00:18:49,169
sweating it below for example 30

497
00:18:49,169 --> 00:18:50,519
milliseconds no matter what people are

498
00:18:50,519 --> 00:18:52,559
telling you right but 200 milliseconds

499
00:18:52,559 --> 00:18:54,059
is devastating because you can actually

500
00:18:54,059 --> 00:18:56,519
see and you suffer through why do I have

501
00:18:56,519 --> 00:18:57,929
that delay people really hate delays

502
00:18:57,929 --> 00:19:00,299
right so even if you're tackling a

503
00:19:00,299 --> 00:19:02,460
simple problem which shows up almost all

504
00:19:02,460 --> 00:19:05,730
the time right and trust me here this

505
00:19:05,730 --> 00:19:07,529
becomes very very important right do you

506
00:19:07,529 --> 00:19:11,190
have a nice interface and for example a

507
00:19:11,190 --> 00:19:13,259
nice visual system to look at stocks so

508
00:19:13,259 --> 00:19:15,059
let's say and people really want to look

509
00:19:15,059 --> 00:19:16,230
at all kinds of pretty pictures with

510
00:19:16,230 --> 00:19:20,220
stocks graphs and whatnot okay if you

511
00:19:20,220 --> 00:19:21,240
have to communicate with a back-end

512
00:19:21,240 --> 00:19:23,340
server and come back unless you have a

513
00:19:23,340 --> 00:19:25,350
server that's plays very very close to

514
00:19:25,350 --> 00:19:25,940
the user

515
00:19:25,940 --> 00:19:28,980
this large latency will hit you and the

516
00:19:28,980 --> 00:19:30,029
large latency is going to be very

517
00:19:30,029 --> 00:19:31,769
annoying in the user interface one

518
00:19:31,769 --> 00:19:36,659
technique is to move the data that the

519
00:19:36,659 --> 00:19:38,369
user pulls off in order to display

520
00:19:38,369 --> 00:19:39,419
whatever it is on the screen or whatever

521
00:19:39,419 --> 00:19:41,309
the user pulls off closer and closer and

522
00:19:41,309 --> 00:19:44,190
closer to the user okay so this is as I

523
00:19:44,190 --> 00:19:46,019
mentioned way before that this is what

524
00:19:46,019 --> 00:19:47,940
hecka might does so for example CNN I

525
00:19:47,940 --> 00:19:50,580
know for sure it's a customer of of

526
00:19:50,580 --> 00:19:54,990
Akamai CNN basically pushes whatever

527
00:19:54,990 --> 00:19:57,269
changes to the content they have through

528
00:19:57,269 --> 00:19:59,460
the Akamai system and I can I will very

529
00:19:59,460 --> 00:20:01,110
very fast and efficiently through the

530
00:20:01,110 --> 00:20:03,210
network distributed much closer to you

531
00:20:03,210 --> 00:20:05,580
and there is some hijacking that goes on

532
00:20:05,580 --> 00:20:07,610
at your ISP in so when you say

533
00:20:07,610 --> 00:20:10,259
www.cnn.com your are will be going to

534
00:20:10,259 --> 00:20:12,480
whatever CNN is your go to your local

535
00:20:12,480 --> 00:20:15,869
replica so replication does need some

536
00:20:15,869 --> 00:20:18,029
collaboration or support either from the

537
00:20:18,029 --> 00:20:20,009
client or from the infrastructure it's

538
00:20:20,009 --> 00:20:22,440
not possible to do replication without

539
00:20:22,440 --> 00:20:24,419
somehow diverting things to different

540
00:20:24,419 --> 00:20:27,029
replicas okay so one core issue with

541
00:20:27,029 --> 00:20:28,440
replication is how are you going to

542
00:20:28,440 --> 00:20:30,779
divert things in case we fail that can

543
00:20:30,779 --> 00:20:32,730
be done as easily as for example this is

544
00:20:32,730 --> 00:20:36,179
a core idea behind mirrors you first

545
00:20:36,179 --> 00:20:37,799
tried the primary in replicas and if it

546
00:20:37,799 --> 00:20:39,179
fails you try the second one if you

547
00:20:39,179 --> 00:20:40,470
fails you try the third one of course

548
00:20:40,470 --> 00:20:41,759
it's very annoying because it can take a

549
00:20:41,759 --> 00:20:42,000
lot

550
00:20:42,000 --> 00:20:44,370
time to go through the replicas wait for

551
00:20:44,370 --> 00:20:46,860
them to fail it more not with replicas

552
00:20:46,860 --> 00:20:48,090
you could do load balancing

553
00:20:48,090 --> 00:20:49,830
so if ignore the fact that there might

554
00:20:49,830 --> 00:20:51,810
be close to you or not if you send

555
00:20:51,810 --> 00:20:55,040
people to one of the many replicas

556
00:20:55,040 --> 00:20:58,830
essentially you're reducing the load on

557
00:20:58,830 --> 00:21:00,570
a single replica and you're spreading

558
00:21:00,570 --> 00:21:01,890
the load and this is the notion of load

559
00:21:01,890 --> 00:21:03,300
balancing right so replicas are

560
00:21:03,300 --> 00:21:08,340
crucially related to replica load

561
00:21:08,340 --> 00:21:20,460
balancing reduce latency so what other

562
00:21:20,460 --> 00:21:23,720
things can we do with replicas yes

563
00:21:25,570 --> 00:21:27,430
what's the difference between a primary

564
00:21:27,430 --> 00:21:34,770
server and replica Oh mirroring well so

565
00:21:34,770 --> 00:21:39,040
anyway let me mention something people

566
00:21:39,040 --> 00:21:42,280
in general like to invent new terms even

567
00:21:42,280 --> 00:21:43,360
if they are very close to all terms

568
00:21:43,360 --> 00:21:45,010
because it looks better

569
00:21:45,010 --> 00:21:47,860
cooler newer for example cloud computing

570
00:21:47,860 --> 00:21:51,160
is one of those right it's only 50 years

571
00:21:51,160 --> 00:21:54,490
old now right the first ideas behind

572
00:21:54,490 --> 00:21:56,200
cloud computing were introduced easily

573
00:21:56,200 --> 00:21:58,180
about 50 years ago the balls they had

574
00:21:58,180 --> 00:21:59,530
two computers they wanted to create some

575
00:21:59,530 --> 00:22:00,850
sort of a connection between them okay

576
00:22:00,850 --> 00:22:03,160
and to do something except that now it's

577
00:22:03,160 --> 00:22:05,320
super cool and we win most of the

578
00:22:05,320 --> 00:22:06,970
technology behind cloud computing was

579
00:22:06,970 --> 00:22:09,760
inventing long time back right so this

580
00:22:09,760 --> 00:22:11,410
is my this is the rule of thumb that I

581
00:22:11,410 --> 00:22:15,190
apply every large area in computer

582
00:22:15,190 --> 00:22:17,590
science or technology every decade is

583
00:22:17,590 --> 00:22:20,200
gonna pick another name for something

584
00:22:20,200 --> 00:22:21,430
they've been doing for a long while to

585
00:22:21,430 --> 00:22:26,230
make it look fresher okay yeah Condor

586
00:22:26,230 --> 00:22:27,550
system for example was developed in

587
00:22:27,550 --> 00:22:30,070
1980s and it did almost anything that

588
00:22:30,070 --> 00:22:31,840
modern cloud computing stuff does it's

589
00:22:31,840 --> 00:22:33,790
just more excitement it's the same thing

590
00:22:33,790 --> 00:22:35,170
here what's the difference not much of a

591
00:22:35,170 --> 00:22:38,650
difference very I mean the problem in

592
00:22:38,650 --> 00:22:40,090
general and this is what I want you to

593
00:22:40,090 --> 00:22:42,040
remember the problem in general in

594
00:22:42,040 --> 00:22:43,990
distribute systems but a lot of other

595
00:22:43,990 --> 00:22:46,360
things is people don't really take the

596
00:22:46,360 --> 00:22:49,380
time to really produce a definition

597
00:22:49,380 --> 00:22:52,180
let's say a replication sounds cool you

598
00:22:52,180 --> 00:22:54,070
say everybody who's supposed to know

599
00:22:54,070 --> 00:22:56,140
about replication should know what a

600
00:22:56,140 --> 00:22:57,580
replication is way-hey it's English

601
00:22:57,580 --> 00:22:59,500
right what's a replication let's look in

602
00:22:59,500 --> 00:23:00,730
the dictionary well the diction is not

603
00:23:00,730 --> 00:23:03,490
gonna say yeah if it's look in the

604
00:23:03,490 --> 00:23:04,360
dictionary and see the difference

605
00:23:04,360 --> 00:23:06,790
between mirroring and replication right

606
00:23:06,790 --> 00:23:08,500
so what I'm trying to say is people

607
00:23:08,500 --> 00:23:10,030
don't take the trouble to define things

608
00:23:10,030 --> 00:23:13,960
then words morph in meaning what do they

609
00:23:13,960 --> 00:23:15,430
mean and they don't mean and ultimately

610
00:23:15,430 --> 00:23:16,990
a lot of things are very close to each

611
00:23:16,990 --> 00:23:18,700
other so to some extent you can think

612
00:23:18,700 --> 00:23:20,440
about this kind of replication as some

613
00:23:20,440 --> 00:23:22,480
sort of caching and for example can ask

614
00:23:22,480 --> 00:23:23,890
the question how is this different how

615
00:23:23,890 --> 00:23:25,450
is replication different than caching

616
00:23:25,450 --> 00:23:26,590
than different than mirroring than

617
00:23:26,590 --> 00:23:28,450
different than something else what in

618
00:23:28,450 --> 00:23:30,040
substance they are not different they

619
00:23:30,040 --> 00:23:32,890
are the same kind of thing you might

620
00:23:32,890 --> 00:23:35,260
produce some refinements which probably

621
00:23:35,260 --> 00:23:36,790
are going to be breaking broken by the

622
00:23:36,790 --> 00:23:37,960
next guy that comes and does some

623
00:23:37,960 --> 00:23:39,320
replication service just because

624
00:23:39,320 --> 00:23:40,790
he thought the name is cooler right

625
00:23:40,790 --> 00:23:42,680
let's call this replication though it's

626
00:23:42,680 --> 00:23:43,820
not quite replication or things like

627
00:23:43,820 --> 00:23:46,810
that so don't put too much in the name I

628
00:23:46,810 --> 00:23:49,520
would say in essence nothing between

629
00:23:49,520 --> 00:23:52,490
mirroring and replication now specific

630
00:23:52,490 --> 00:23:54,770
guarantee is that a specific service

631
00:23:54,770 --> 00:23:56,870
offers that's a different story

632
00:23:56,870 --> 00:23:58,790
so traditionally mirroring service

633
00:23:58,790 --> 00:24:01,250
services offer very few guarantees for

634
00:24:01,250 --> 00:24:02,840
example they never said that the copy is

635
00:24:02,840 --> 00:24:05,150
fresh and for example some of the the

636
00:24:05,150 --> 00:24:07,460
mirrors of the important software like

637
00:24:07,460 --> 00:24:09,200
the Linux kernel and notorious for being

638
00:24:09,200 --> 00:24:13,160
back a couple of months right so if you

639
00:24:13,160 --> 00:24:15,110
use that mirror you're gonna get a

640
00:24:15,110 --> 00:24:16,880
version that's couple of months old

641
00:24:16,880 --> 00:24:18,830
that's not too good Akamai its famous

642
00:24:18,830 --> 00:24:21,380
for having replica propagation in

643
00:24:21,380 --> 00:24:24,590
seconds at most right so it's not so

644
00:24:24,590 --> 00:24:25,970
much what's the difference between the

645
00:24:25,970 --> 00:24:27,320
terms is what's the difference between

646
00:24:27,320 --> 00:24:29,090
specific services offered by specific

647
00:24:29,090 --> 00:24:34,040
tools companies or whatnot okay so to a

648
00:24:34,040 --> 00:24:36,500
large extent the replicas are related to

649
00:24:36,500 --> 00:24:37,970
caching and remember caching gets a

650
00:24:37,970 --> 00:24:40,130
fundamental technique in general and

651
00:24:40,130 --> 00:24:43,520
what does caching do in general right it

652
00:24:43,520 --> 00:24:45,800
allows me a faster access to that

653
00:24:45,800 --> 00:24:47,900
resource at the expense of having it not

654
00:24:47,900 --> 00:24:49,610
quite in sync with the rest but that's

655
00:24:49,610 --> 00:24:51,410
essentially what replication potentially

656
00:24:51,410 --> 00:24:53,930
is right now one thing that we are going

657
00:24:53,930 --> 00:24:55,700
to see when we talk about various kinds

658
00:24:55,700 --> 00:24:57,140
of consistency is you could insist that

659
00:24:57,140 --> 00:24:58,490
the replicas are completely up to date

660
00:24:58,490 --> 00:25:02,330
but that's gonna be really burdensome on

661
00:25:02,330 --> 00:25:07,100
the framework right so again you can't

662
00:25:07,100 --> 00:25:09,740
really say I mean you can basically for

663
00:25:09,740 --> 00:25:12,260
example this is one one kind of a very

664
00:25:12,260 --> 00:25:13,790
harsh consistency you can impose on

665
00:25:13,790 --> 00:25:16,820
replicas if you activate the replicas

666
00:25:16,820 --> 00:25:19,250
only when you're sure that the changes

667
00:25:19,250 --> 00:25:22,070
were propagated to all other replicas so

668
00:25:22,070 --> 00:25:24,500
that to any user no matter where they

669
00:25:24,500 --> 00:25:28,550
connect right it looks like we all

670
00:25:28,550 --> 00:25:30,110
switch to the new version at the same

671
00:25:30,110 --> 00:25:33,080
time so you have changes but you don't

672
00:25:33,080 --> 00:25:34,520
make for example the changes available

673
00:25:34,520 --> 00:25:36,500
until everybody got the changes and this

674
00:25:36,500 --> 00:25:37,850
is equivalent for example and this is

675
00:25:37,850 --> 00:25:40,370
done sometimes in databases when you

676
00:25:40,370 --> 00:25:41,900
have this distributed transactions which

677
00:25:41,900 --> 00:25:43,760
is we still have to talk about so you

678
00:25:43,760 --> 00:25:45,320
want to make sure for example that you

679
00:25:45,320 --> 00:25:47,060
say commit to a particular transaction

680
00:25:47,060 --> 00:25:48,920
to the user you tell the user yes your

681
00:25:48,920 --> 00:25:50,300
transaction actually finished correctly

682
00:25:50,300 --> 00:25:53,260
only when all the parties involved

683
00:25:53,260 --> 00:25:56,330
agreed that we commit to the transaction

684
00:25:56,330 --> 00:25:58,280
and this is extremely harsh the moment

685
00:25:58,280 --> 00:26:00,350
you use such a mechanism be prepared to

686
00:26:00,350 --> 00:26:03,559
pay a very high price large number of

687
00:26:03,559 --> 00:26:05,540
message exchanges pay a large amount of

688
00:26:05,540 --> 00:26:08,000
time because of the latency forget about

689
00:26:08,000 --> 00:26:11,720
any other issue okay so we do

690
00:26:11,720 --> 00:26:13,670
replication not to lose data we do

691
00:26:13,670 --> 00:26:17,270
replication to reduce the latency what

692
00:26:17,270 --> 00:26:18,620
are the reasons so for example how would

693
00:26:18,620 --> 00:26:20,120
you do this right I'm talking about this

694
00:26:20,120 --> 00:26:21,500
40 milliseconds here I want to update

695
00:26:21,500 --> 00:26:24,280
graphs I want to do this kind of thing

696
00:26:24,280 --> 00:26:25,700
all right

697
00:26:25,700 --> 00:26:28,309
so echo what echo might does is it's

698
00:26:28,309 --> 00:26:30,740
nice and good you potentially could do

699
00:26:30,740 --> 00:26:32,270
better if you can compress the data a

700
00:26:32,270 --> 00:26:33,860
lot and the data doesn't change so much

701
00:26:33,860 --> 00:26:36,440
or at least can be considered stationary

702
00:26:36,440 --> 00:26:38,420
for for a few minutes but you can

703
00:26:38,420 --> 00:26:41,630
actually instead of having the webpage

704
00:26:41,630 --> 00:26:43,460
generated in a server you can generate

705
00:26:43,460 --> 00:26:44,630
it in a client and this is something

706
00:26:44,630 --> 00:26:45,980
you're not really told I mean this is

707
00:26:45,980 --> 00:26:48,170
the new brave new world in which you

708
00:26:48,170 --> 00:26:51,380
just ship the data to the browser and

709
00:26:51,380 --> 00:26:53,450
the browser will create all that all

710
00:26:53,450 --> 00:26:56,420
those pages now this is JavaScript magic

711
00:26:56,420 --> 00:26:59,090
if you do so you can do some amazing

712
00:26:59,090 --> 00:27:01,340
stunts that are not possible otherwise

713
00:27:01,340 --> 00:27:02,750
right so for example a particular

714
00:27:02,750 --> 00:27:05,059
problem that a number of data

715
00:27:05,059 --> 00:27:06,950
visualization companies have is they

716
00:27:06,950 --> 00:27:08,570
rely on back-end servers which means

717
00:27:08,570 --> 00:27:11,360
they they need this to is not 200

718
00:27:11,360 --> 00:27:12,740
milliseconds by the way it's much closer

719
00:27:12,740 --> 00:27:14,570
to one or two seconds the moment you

720
00:27:14,570 --> 00:27:16,100
involve database servers and all kinds

721
00:27:16,100 --> 00:27:17,600
of other processing if you say give me

722
00:27:17,600 --> 00:27:20,120
the graph that shows me the stock for

723
00:27:20,120 --> 00:27:21,670
this company over the last five years

724
00:27:21,670 --> 00:27:25,429
right your browser talks to the web

725
00:27:25,429 --> 00:27:27,260
server who talks to the backend the

726
00:27:27,260 --> 00:27:29,480
database which computes the new graph

727
00:27:29,480 --> 00:27:31,400
you push it through some sort of a ok

728
00:27:31,400 --> 00:27:33,170
compute some sort of a feature or

729
00:27:33,170 --> 00:27:36,260
whatever ship it to the front-end if you

730
00:27:36,260 --> 00:27:37,520
don't believe me you can use this tool

731
00:27:37,520 --> 00:27:41,330
it feels like a long second a second

732
00:27:41,330 --> 00:27:43,640
wait is devastating for the human brain

733
00:27:43,640 --> 00:27:46,610
it really makes you feel bad right it it

734
00:27:46,610 --> 00:27:48,020
bogs you down you cannot think

735
00:27:48,020 --> 00:27:50,000
especially if you expect to go through

736
00:27:50,000 --> 00:27:52,040
hundreds thousands of these things to

737
00:27:52,040 --> 00:27:54,800
see anything right faster than 30

738
00:27:54,800 --> 00:27:56,110
milliseconds doesn't matter either

739
00:27:56,110 --> 00:27:59,720
that's the trick right so staying close

740
00:27:59,720 --> 00:28:01,190
to 30 milliseconds makes it makes it

741
00:28:01,190 --> 00:28:02,780
look like a movie right so think about

742
00:28:02,780 --> 00:28:04,550
it you would like to find an interesting

743
00:28:04,550 --> 00:28:05,770
stock and you

744
00:28:05,770 --> 00:28:07,210
by the way your brain can easily run at

745
00:28:07,210 --> 00:28:09,280
30 milliseconds of rain and you can

746
00:28:09,280 --> 00:28:11,530
actually see patterns at 30 frames per

747
00:28:11,530 --> 00:28:14,410
second not only that but the human brain

748
00:28:14,410 --> 00:28:17,320
is really designed to see patterns in

749
00:28:17,320 --> 00:28:19,000
moving things not in aesthetic settings

750
00:28:19,000 --> 00:28:20,170
if I give you a static picture you're

751
00:28:20,170 --> 00:28:21,460
gonna have a hard time figuring out what

752
00:28:21,460 --> 00:28:22,630
changed or what didn't change if I give

753
00:28:22,630 --> 00:28:23,800
you a move you instantly know what's

754
00:28:23,800 --> 00:28:25,420
important and what's not important right

755
00:28:25,420 --> 00:28:27,880
so from that point of view you're better

756
00:28:27,880 --> 00:28:29,740
off is essentially producing some sort

757
00:28:29,740 --> 00:28:31,270
of replication of the data in the

758
00:28:31,270 --> 00:28:31,780
browser

759
00:28:31,780 --> 00:28:33,580
so suddenly replica has a different

760
00:28:33,580 --> 00:28:37,240
meaning right so now for example is that

761
00:28:37,240 --> 00:28:40,870
a good investment or not to push the

762
00:28:40,870 --> 00:28:44,350
replica in the web browser what one of

763
00:28:44,350 --> 00:28:48,040
the tricks that almost always is a good

764
00:28:48,040 --> 00:28:49,930
trade-off in modern disability systems

765
00:28:49,930 --> 00:28:53,920
is to trade-off throughput for for

766
00:28:53,920 --> 00:28:56,260
latency so how it can you do that right

767
00:28:56,260 --> 00:28:58,480
so that means do a lot more replication

768
00:28:58,480 --> 00:29:01,120
that otherwise you would think when the

769
00:29:01,120 --> 00:29:02,710
user for example starts to look at

770
00:29:02,710 --> 00:29:05,920
stocks right you have to pay for example

771
00:29:05,920 --> 00:29:07,450
200 milliseconds or you have to pay for

772
00:29:07,450 --> 00:29:10,120
example a second right to access a

773
00:29:10,120 --> 00:29:12,130
single picture one second you can ask

774
00:29:12,130 --> 00:29:14,830
yourself the question if I'm if the user

775
00:29:14,830 --> 00:29:16,810
is willing to pay one second or maybe

776
00:29:16,810 --> 00:29:18,670
even let's say ten times more but not to

777
00:29:18,670 --> 00:29:21,040
pay any price up afterwards right namely

778
00:29:21,040 --> 00:29:23,200
stay within 30 milliseconds how much

779
00:29:23,200 --> 00:29:26,020
could I ship in two seconds

780
00:29:26,020 --> 00:29:27,730
so ask yourself the question with modern

781
00:29:27,730 --> 00:29:29,590
networks how much information could I

782
00:29:29,590 --> 00:29:31,150
ship to the user in let's say two

783
00:29:31,150 --> 00:29:32,590
seconds because the trick I can play is

784
00:29:32,590 --> 00:29:33,220
the following

785
00:29:33,220 --> 00:29:35,290
instead of doing everything piece by

786
00:29:35,290 --> 00:29:37,480
piece if I have somebody in what space

787
00:29:37,480 --> 00:29:39,790
the user is gonna stay and the first

788
00:29:39,790 --> 00:29:42,160
request instead of shipping only the

789
00:29:42,160 --> 00:29:43,420
little thing that the user wants I'm

790
00:29:43,420 --> 00:29:45,640
gonna ship two seconds worth of

791
00:29:45,640 --> 00:29:47,320
information at the bandwidth that the

792
00:29:47,320 --> 00:29:50,830
user has all right now if you have a

793
00:29:50,830 --> 00:29:52,210
decent internet connection that

794
00:29:52,210 --> 00:29:55,600
bandwidth right you can buy this for

795
00:29:55,600 --> 00:29:56,950
example in certain parts of games but

796
00:29:56,950 --> 00:29:58,630
now that bandwidth could be as high as

797
00:29:58,630 --> 00:30:02,080
one megabyte per second okay so two

798
00:30:02,080 --> 00:30:05,080
seconds is two megabytes in two

799
00:30:05,080 --> 00:30:08,950
megabytes you can easily put quite a lot

800
00:30:08,950 --> 00:30:11,110
of information for all the 10000 tickers

801
00:30:11,110 --> 00:30:13,990
on the stock market so what I'm trying

802
00:30:13,990 --> 00:30:15,640
to say there is when you design

803
00:30:15,640 --> 00:30:17,290
distributed systems think about

804
00:30:17,290 --> 00:30:18,600
replication at some

805
00:30:18,600 --> 00:30:20,039
it's really your friend and bring it

806
00:30:20,039 --> 00:30:21,539
possibly who much much closer to the

807
00:30:21,539 --> 00:30:23,760
user so one way to do it would be to

808
00:30:23,760 --> 00:30:27,090
relay ship almost all the information to

809
00:30:27,090 --> 00:30:29,190
the user because then the latency is

810
00:30:29,190 --> 00:30:31,230
below the radar of what the user can

811
00:30:31,230 --> 00:30:33,900
actually proceed you have no latency to

812
00:30:33,900 --> 00:30:35,400
go to the back now sometimes this

813
00:30:35,400 --> 00:30:38,549
strategy works very very nicely because

814
00:30:38,549 --> 00:30:40,020
the data simply doesn't change that much

815
00:30:40,020 --> 00:30:42,000
right so for example if the stock date

816
00:30:42,000 --> 00:30:43,530
so if the user wants to look at

817
00:30:43,530 --> 00:30:44,970
historical data for example what

818
00:30:44,970 --> 00:30:47,490
happened at the granularity of one day

819
00:30:47,490 --> 00:30:51,360
on the stock market spending 2 3 4 or

820
00:30:51,360 --> 00:30:52,980
even 10 seconds to ship the whole

821
00:30:52,980 --> 00:30:56,400
history of all the stocks that within

822
00:30:56,400 --> 00:30:59,130
some kind of a range for the for the

823
00:30:59,130 --> 00:31:00,270
user for example you only have that many

824
00:31:00,270 --> 00:31:01,650
stocks actually about 10,000 of them

825
00:31:01,650 --> 00:31:04,679
right it's not something that you should

826
00:31:04,679 --> 00:31:07,500
dismiss once you've done so if you have

827
00:31:07,500 --> 00:31:09,570
the horsepower in JavaScript to really

828
00:31:09,570 --> 00:31:10,919
generate all those things that normally

829
00:31:10,919 --> 00:31:12,299
you would get from from the backend you

830
00:31:12,299 --> 00:31:15,720
have completely seamless execution of

831
00:31:15,720 --> 00:31:17,610
this right so an example of this for

832
00:31:17,610 --> 00:31:19,559
example is what what we try to do with

833
00:31:19,559 --> 00:31:21,600
this Medicare data it takes about two or

834
00:31:21,600 --> 00:31:24,120
three seconds to ship one hundred and

835
00:31:24,120 --> 00:31:26,100
sixty three thousand rows to the

836
00:31:26,100 --> 00:31:28,890
JavaScript after that it's all below 30

837
00:31:28,890 --> 00:31:30,539
milliseconds and you feel like there is

838
00:31:30,539 --> 00:31:32,549
no effort whatsoever right so that's

839
00:31:32,549 --> 00:31:33,929
really a form of replication I've

840
00:31:33,929 --> 00:31:36,270
replicated the backend data right I took

841
00:31:36,270 --> 00:31:38,070
care to compress it a little bit and

842
00:31:38,070 --> 00:31:39,900
make it smaller and so on so all kinds

843
00:31:39,900 --> 00:31:40,890
of tricks to make it as small as

844
00:31:40,890 --> 00:31:42,570
possible but once I ship it to the front

845
00:31:42,570 --> 00:31:46,049
you have a replica in your browser in

846
00:31:46,049 --> 00:31:47,610
the case of the Medicare data for

847
00:31:47,610 --> 00:31:49,320
example we know for sure that there is

848
00:31:49,320 --> 00:31:50,730
no updates to it the feds have no

849
00:31:50,730 --> 00:31:53,520
intention to give more of that data

850
00:31:53,520 --> 00:31:56,520
right it did not change a single bit in

851
00:31:56,520 --> 00:31:58,830
four months and it's not clear at all

852
00:31:58,830 --> 00:32:00,690
when it will change the same thing with

853
00:32:00,690 --> 00:32:02,070
the stock market won't change and the

854
00:32:02,070 --> 00:32:04,350
resolution of more than once a day right

855
00:32:04,350 --> 00:32:06,210
because I'm only providing you the low

856
00:32:06,210 --> 00:32:07,980
level services which is the average

857
00:32:07,980 --> 00:32:12,299
price per day right so in those kind of

858
00:32:12,299 --> 00:32:13,740
circumstances really the replication

859
00:32:13,740 --> 00:32:15,780
happens very very close to the client

860
00:32:15,780 --> 00:32:17,460
you can think of it as some sort of

861
00:32:17,460 --> 00:32:18,840
caching but it's in fact some sort of

862
00:32:18,840 --> 00:32:20,730
replication okay I just kind of blurred

863
00:32:20,730 --> 00:32:24,120
anyway now when it comes to this fast

864
00:32:24,120 --> 00:32:26,130
transactions fast changes things become

865
00:32:26,130 --> 00:32:28,140
much much much harder right so for

866
00:32:28,140 --> 00:32:29,610
example if the user will really be

867
00:32:29,610 --> 00:32:31,290
interested in very up-to-date

868
00:32:31,290 --> 00:32:32,280
information

869
00:32:32,280 --> 00:32:33,690
how do you provide that how you provide

870
00:32:33,690 --> 00:32:36,300
that replica so one thing and this is

871
00:32:36,300 --> 00:32:37,710
related to some deep questions about

872
00:32:37,710 --> 00:32:39,180
operating systems right I might as well

873
00:32:39,180 --> 00:32:40,980
connect them so a lot of these problems

874
00:32:40,980 --> 00:32:42,870
are really problems even if you don't

875
00:32:42,870 --> 00:32:44,340
have distributed systems at all and I'm

876
00:32:44,340 --> 00:32:46,260
gonna explain a second right so imagine

877
00:32:46,260 --> 00:32:48,810
now you're tapping into there all the

878
00:32:48,810 --> 00:32:51,000
activity that goes on any market in the

879
00:32:51,000 --> 00:32:54,030
world ok it turns out you can buy such a

880
00:32:54,030 --> 00:32:57,450
service and there are for example

881
00:32:57,450 --> 00:32:58,970
there's a company called Nanak's and

882
00:32:58,970 --> 00:33:02,390
this company provides an extremely fast

883
00:33:02,390 --> 00:33:05,370
data stream that gives you access to all

884
00:33:05,370 --> 00:33:06,870
the transactions that happen on all the

885
00:33:06,870 --> 00:33:09,450
financial markets ok now you pay an arm

886
00:33:09,450 --> 00:33:11,070
and a leg for the data stream don't get

887
00:33:11,070 --> 00:33:13,020
me wrong but to some people it's worth

888
00:33:13,020 --> 00:33:15,110
it because that's how you find out about

889
00:33:15,110 --> 00:33:18,060
activity on the on all the markets and

890
00:33:18,060 --> 00:33:21,750
they date that data stream itself is 40

891
00:33:21,750 --> 00:33:27,030
million transactions per second right by

892
00:33:27,030 --> 00:33:29,070
the way to even ship that many per

893
00:33:29,070 --> 00:33:31,830
second they have to have a very fixed

894
00:33:31,830 --> 00:33:33,540
format so they have something like a 64

895
00:33:33,540 --> 00:33:38,730
byte so they have 64 byte packets so we

896
00:33:38,730 --> 00:33:40,140
are talking about things that suddenly

897
00:33:40,140 --> 00:33:42,060
is something very very high level in the

898
00:33:42,060 --> 00:33:43,410
application and being some kind of a

899
00:33:43,410 --> 00:33:45,390
fluffy thing they are get pushed all the

900
00:33:45,390 --> 00:33:48,480
way to the bottom and they are really

901
00:33:48,480 --> 00:33:53,100
tight specified in C format 64 bytes to

902
00:33:53,100 --> 00:33:56,190
specify each of the one transactions 40

903
00:33:56,190 --> 00:33:58,200
million transactions per per second so

904
00:33:58,200 --> 00:33:59,400
they say hey if you have a fast enough

905
00:33:59,400 --> 00:34:00,930
internet connection no problem and you

906
00:34:00,930 --> 00:34:02,340
pay the money for the service will

907
00:34:02,340 --> 00:34:05,010
stream to you the data in fact what they

908
00:34:05,010 --> 00:34:07,410
are doing is they allow you to replicate

909
00:34:07,410 --> 00:34:08,969
everything that happens on that on the

910
00:34:08,969 --> 00:34:11,580
financial markets that's similar to the

911
00:34:11,580 --> 00:34:13,830
kind of stream that Akamai might send to

912
00:34:13,830 --> 00:34:15,120
their servers to propagate information

913
00:34:15,120 --> 00:34:18,179
so that that's an implicit specification

914
00:34:18,179 --> 00:34:20,550
of the replicas not an explicit one but

915
00:34:20,550 --> 00:34:22,199
you almost always have to do this so

916
00:34:22,199 --> 00:34:24,000
let's think a little bit about some form

917
00:34:24,000 --> 00:34:25,260
of replication that you might not even

918
00:34:25,260 --> 00:34:26,610
think they are replication but they're

919
00:34:26,610 --> 00:34:28,230
really replication and which are core

920
00:34:28,230 --> 00:34:30,210
operating system issues have nothing to

921
00:34:30,210 --> 00:34:32,489
do with distributed systems okay so here

922
00:34:32,489 --> 00:34:33,810
is a similar problem even though usually

923
00:34:33,810 --> 00:34:35,040
people don't paint it as a similar

924
00:34:35,040 --> 00:34:37,710
problem you have a program and it runs

925
00:34:37,710 --> 00:34:41,100
for a while a while being let's say a

926
00:34:41,100 --> 00:34:43,500
couple of weeks we have those running in

927
00:34:43,500 --> 00:34:44,500
the department by the way

928
00:34:44,500 --> 00:34:47,530
a large simulation or whatnot okay now a

929
00:34:47,530 --> 00:34:49,270
big problem with those programs is if

930
00:34:49,270 --> 00:34:50,918
the program for any reason crashes in

931
00:34:50,918 --> 00:34:51,399
the middle

932
00:34:51,399 --> 00:34:52,780
for example power outage or any other

933
00:34:52,780 --> 00:34:56,440
reason programming error or the wind

934
00:34:56,440 --> 00:34:58,720
blows too hard or whatever right you

935
00:34:58,720 --> 00:35:00,700
lost many weeks

936
00:35:00,700 --> 00:35:02,770
I know physicists that were running

937
00:35:02,770 --> 00:35:06,880
their code for months right good so one

938
00:35:06,880 --> 00:35:09,369
interesting idea there would be what why

939
00:35:09,369 --> 00:35:11,470
don't I replicate the state of the

940
00:35:11,470 --> 00:35:13,089
program at certain points

941
00:35:13,089 --> 00:35:15,819
it's called checkpointing so that if the

942
00:35:15,819 --> 00:35:17,410
system fails I can only go back to the

943
00:35:17,410 --> 00:35:19,270
checkpoint and continue from there say

944
00:35:19,270 --> 00:35:20,829
things that I mean I might lose for

945
00:35:20,829 --> 00:35:23,470
example a minute let's say if I

946
00:35:23,470 --> 00:35:25,089
checkpoint every minute which means if I

947
00:35:25,089 --> 00:35:27,010
produce a replica in another form on a

948
00:35:27,010 --> 00:35:28,960
persistent storage then I can go back

949
00:35:28,960 --> 00:35:30,369
and lose a minute and pick up the

950
00:35:30,369 --> 00:35:33,160
computation no big deal ok so then the

951
00:35:33,160 --> 00:35:34,869
fundamental question is and this is a

952
00:35:34,869 --> 00:35:36,819
problem that looks like has nothing to

953
00:35:36,819 --> 00:35:39,250
do with the rest of them is ok you have

954
00:35:39,250 --> 00:35:41,859
your own favorite sequential program no

955
00:35:41,859 --> 00:35:43,270
strings attached with distributed

956
00:35:43,270 --> 00:35:45,099
systems you need to provide checkpoint

957
00:35:45,099 --> 00:35:47,230
for your program just because it runs

958
00:35:47,230 --> 00:35:49,150
too long now supposedly your program

959
00:35:49,150 --> 00:35:51,039
doesn't just execute infinite loops it

960
00:35:51,039 --> 00:35:53,200
does some changes to the state how do

961
00:35:53,200 --> 00:35:55,270
you replicate how do you replicate the

962
00:35:55,270 --> 00:35:57,160
in-memory state to some sort of a disk

963
00:35:57,160 --> 00:35:59,319
state because that's persistence so how

964
00:35:59,319 --> 00:36:01,150
do you do it this is in fact the core

965
00:36:01,150 --> 00:36:04,630
replica issue it's you almost have the

966
00:36:04,630 --> 00:36:07,809
full-fledged replication issues right

967
00:36:07,809 --> 00:36:09,190
there in that problem now this is

968
00:36:09,190 --> 00:36:10,450
something people looked at for a very

969
00:36:10,450 --> 00:36:11,829
long time and they don't necessarily

970
00:36:11,829 --> 00:36:14,589
have a very good solution what's the

971
00:36:14,589 --> 00:36:16,450
core problem there which is very similar

972
00:36:16,450 --> 00:36:17,770
to the core problem in in distributed

973
00:36:17,770 --> 00:36:21,730
systems the core problem is you cannot

974
00:36:21,730 --> 00:36:23,260
really use the easy way out so what's

975
00:36:23,260 --> 00:36:24,750
the easy way out in that circumstance

976
00:36:24,750 --> 00:36:27,819
freeze the program take a full dump of

977
00:36:27,819 --> 00:36:31,619
the memory and put it on the disc right

978
00:36:31,619 --> 00:36:37,480
now this is probably a good solution the

979
00:36:37,480 --> 00:36:39,760
full memory so if I dump the full memory

980
00:36:39,760 --> 00:36:42,029
state in the state of the registers I

981
00:36:42,029 --> 00:36:45,359
captured the full state of the processor

982
00:36:45,359 --> 00:36:48,099
well Intel guarantees that this is the

983
00:36:48,099 --> 00:36:50,619
case so that if I fail I can just

984
00:36:50,619 --> 00:36:52,059
recover the full set of the memory

985
00:36:52,059 --> 00:36:54,970
recover the full state of the of the

986
00:36:54,970 --> 00:36:57,460
registers and say yeah continue from

987
00:36:57,460 --> 00:36:58,720
this point

988
00:36:58,720 --> 00:37:01,330
right now this solution has a major

989
00:37:01,330 --> 00:37:03,520
advantage and the major advantage is it

990
00:37:03,520 --> 00:37:05,770
does not depend on any semantics related

991
00:37:05,770 --> 00:37:09,700
to what the code was doing okay it's

992
00:37:09,700 --> 00:37:11,770
truly a universal solution I don't care

993
00:37:11,770 --> 00:37:15,550
what you do this is in fact a provably

994
00:37:15,550 --> 00:37:19,060
good and usable replica you're only

995
00:37:19,060 --> 00:37:20,800
gonna lose the amount of time you lost

996
00:37:20,800 --> 00:37:22,540
since the last replica because that's

997
00:37:22,540 --> 00:37:25,030
guaranteed by Intel who says our

998
00:37:25,030 --> 00:37:27,400
computers behave deterministically by

999
00:37:27,400 --> 00:37:28,869
the way something similar is used for

1000
00:37:28,869 --> 00:37:31,660
example when you use this virtualization

1001
00:37:31,660 --> 00:37:34,060
technology right and you say safe state

1002
00:37:34,060 --> 00:37:35,589
of the virtual machine that's by the way

1003
00:37:35,589 --> 00:37:37,150
one of the tremendous benefits of the

1004
00:37:37,150 --> 00:37:39,000
virtual machines you can freeze them

1005
00:37:39,000 --> 00:37:41,710
right and later come back to them or

1006
00:37:41,710 --> 00:37:42,910
move them on a completely different

1007
00:37:42,910 --> 00:37:44,170
machine and come back to them there you

1008
00:37:44,170 --> 00:37:46,030
really save the state and you can you

1009
00:37:46,030 --> 00:37:47,920
can continue running the state so for

1010
00:37:47,920 --> 00:37:50,280
virtual machines this kind of

1011
00:37:50,280 --> 00:37:54,430
checkpointing it's in fact trivial if

1012
00:37:54,430 --> 00:37:56,890
you're not particularly concerned about

1013
00:37:56,890 --> 00:38:00,790
the performance of this right just save

1014
00:38:00,790 --> 00:38:01,930
the state and we have to stop the

1015
00:38:01,930 --> 00:38:02,920
program why do you have to stop the

1016
00:38:02,920 --> 00:38:06,640
program that's the trouble right so if

1017
00:38:06,640 --> 00:38:08,710
you have a if you have a regime in which

1018
00:38:08,710 --> 00:38:10,720
the state is changing and you're trying

1019
00:38:10,720 --> 00:38:12,190
to do this check pointing the problem

1020
00:38:12,190 --> 00:38:13,690
suddenly becomes much much much harder

1021
00:38:13,690 --> 00:38:16,510
potentially and it's similar to the

1022
00:38:16,510 --> 00:38:17,830
conflicts you get if you don't use

1023
00:38:17,830 --> 00:38:19,570
mutual exclusion regions you don't quite

1024
00:38:19,570 --> 00:38:21,270
know what you get when you saved it

1025
00:38:21,270 --> 00:38:23,440
Bracey the problem is not saving

1026
00:38:23,440 --> 00:38:25,510
something the problem is how the saved

1027
00:38:25,510 --> 00:38:28,359
thing or how the propagated thing to

1028
00:38:28,359 --> 00:38:30,460
another replica can be used to recover

1029
00:38:30,460 --> 00:38:32,650
something that some sort of a consistent

1030
00:38:32,650 --> 00:38:34,089
state so you see how consistency comes

1031
00:38:34,089 --> 00:38:38,020
into play right if I freeze the program

1032
00:38:38,020 --> 00:38:40,480
remember all the registers and dump all

1033
00:38:40,480 --> 00:38:43,230
the memory I have a reasonably good

1034
00:38:43,230 --> 00:38:47,440
ground to say I get a consistent copy

1035
00:38:47,440 --> 00:38:49,810
for that moment in time but almost

1036
00:38:49,810 --> 00:38:52,270
anything off that it's it's plagued with

1037
00:38:52,270 --> 00:38:54,220
problems how do you know that you didn't

1038
00:38:54,220 --> 00:38:56,369
get something that's a NASA right so

1039
00:38:56,369 --> 00:38:59,500
almost anything except this very if you

1040
00:38:59,500 --> 00:39:01,900
want barbaric solution right which

1041
00:39:01,900 --> 00:39:04,000
require very very fine-grained

1042
00:39:04,000 --> 00:39:05,890
considerations with respect to what is

1043
00:39:05,890 --> 00:39:07,359
it that the application was doing and

1044
00:39:07,359 --> 00:39:09,580
proves that a particular replication

1045
00:39:09,580 --> 00:39:10,930
strategy a particular checkpointing

1046
00:39:10,930 --> 00:39:11,670
strategies

1047
00:39:11,670 --> 00:39:13,799
of the same in the circumstance that's

1048
00:39:13,799 --> 00:39:16,049
not mess things up produces a certain

1049
00:39:16,049 --> 00:39:19,470
level of consistency right now I just

1050
00:39:19,470 --> 00:39:21,540
want to point out that in the context

1051
00:39:21,540 --> 00:39:23,430
very restrictive context of relational

1052
00:39:23,430 --> 00:39:24,990
databases the database community spend

1053
00:39:24,990 --> 00:39:26,579
an enormous amount of time figuring out

1054
00:39:26,579 --> 00:39:28,200
exactly what this would mean

1055
00:39:28,200 --> 00:39:29,970
right and they came up with this kind of

1056
00:39:29,970 --> 00:39:32,190
algorithms a large number of them

1057
00:39:32,190 --> 00:39:33,990
providing a certain kind of consistency

1058
00:39:33,990 --> 00:39:35,790
right so for example an interesting idea

1059
00:39:35,790 --> 00:39:38,040
would be can you checkpoint as other

1060
00:39:38,040 --> 00:39:40,290
things are going and it's possible to do

1061
00:39:40,290 --> 00:39:41,369
any restricting domains like the

1062
00:39:41,369 --> 00:39:43,710
databases but they needed 20 30 years of

1063
00:39:43,710 --> 00:39:46,170
research to nail down the solutions in

1064
00:39:46,170 --> 00:39:48,799
general there is no universal solution

1065
00:39:48,799 --> 00:39:55,020
especially if you expect very fast kind

1066
00:39:55,020 --> 00:39:58,170
of replication process right another way

1067
00:39:58,170 --> 00:40:00,240
you could try to capture this and this

1068
00:40:00,240 --> 00:40:02,549
is potentially something interesting

1069
00:40:02,549 --> 00:40:06,210
right is you say hey the state is

1070
00:40:06,210 --> 00:40:08,880
enormous by the way size of the state is

1071
00:40:08,880 --> 00:40:11,460
a really big problem for example you can

1072
00:40:11,460 --> 00:40:12,930
buy now a hundred twenty gigs of memory

1073
00:40:12,930 --> 00:40:16,200
with about a thousand dollars it takes a

1074
00:40:16,200 --> 00:40:18,660
tremendous amount of time to save 128

1075
00:40:18,660 --> 00:40:21,030
gigs of memory anywhere push it anywhere

1076
00:40:21,030 --> 00:40:23,309
right I mean even if you talk about

1077
00:40:23,309 --> 00:40:25,380
extremely fast disk it's really fast

1078
00:40:25,380 --> 00:40:27,089
disk arrays can achieve two gigabytes

1079
00:40:27,089 --> 00:40:29,760
per second let's say let's say let's say

1080
00:40:29,760 --> 00:40:32,069
for let's be generous okay four

1081
00:40:32,069 --> 00:40:35,549
gigabytes per second 128 gigabytes right

1082
00:40:35,549 --> 00:40:37,349
but if I have four gigabytes per second

1083
00:40:37,349 --> 00:40:40,230
I need tens of hard drives right then

1084
00:40:40,230 --> 00:40:42,150
let's let's bring it up to something

1085
00:40:42,150 --> 00:40:44,160
like 512

1086
00:40:44,160 --> 00:40:49,020
let's 512 gigabytes and this is by the

1087
00:40:49,020 --> 00:40:52,369
way doable now on a single machine

1088
00:40:52,369 --> 00:40:55,500
divided by 4 gigabytes and we are

1089
00:40:55,500 --> 00:40:59,250
talking about 2028 seconds right it's a

1090
00:40:59,250 --> 00:41:01,980
disaster if I really want to capture the

1091
00:41:01,980 --> 00:41:03,390
full state and replicate the state of

1092
00:41:03,390 --> 00:41:05,190
the memory in a large computation when I

1093
00:41:05,190 --> 00:41:07,109
use half a terabyte of memory which can

1094
00:41:07,109 --> 00:41:09,720
be done with a system that's twelve

1095
00:41:09,720 --> 00:41:11,490
thousand dollars right now I need a

1096
00:41:11,490 --> 00:41:13,680
hundred and twenty eight seconds that's

1097
00:41:13,680 --> 00:41:14,609
about two minutes

1098
00:41:14,609 --> 00:41:17,940
I wanted to replicate every one minute

1099
00:41:17,940 --> 00:41:20,010
and I need two minutes to replicate if

1100
00:41:20,010 --> 00:41:22,109
you replicate often enough you don't do

1101
00:41:22,109 --> 00:41:23,940
any computation you just replicate to

1102
00:41:23,940 --> 00:41:24,930
see how these Astros

1103
00:41:24,930 --> 00:41:27,030
this potentially is you can't afford two

1104
00:41:27,030 --> 00:41:30,030
minutes of replication right in general

1105
00:41:30,030 --> 00:41:31,349
even if you do it every once an hour

1106
00:41:31,349 --> 00:41:32,550
it's gonna hurt it's gonna hurt your

1107
00:41:32,550 --> 00:41:34,170
bottom line it's gonna be problematic

1108
00:41:34,170 --> 00:41:36,119
not to mention that because the

1109
00:41:36,119 --> 00:41:39,270
replication process is so long any issue

1110
00:41:39,270 --> 00:41:42,210
happening while you replicate has to be

1111
00:41:42,210 --> 00:41:44,220
taken into consideration and could be

1112
00:41:44,220 --> 00:41:46,890
problematic in itself right so in

1113
00:41:46,890 --> 00:41:48,390
databases for example people talk about

1114
00:41:48,390 --> 00:41:51,809
failure during recovery or failure

1115
00:41:51,809 --> 00:41:55,079
during checkpointing right who says you

1116
00:41:55,079 --> 00:41:56,760
can I mean in two minutes a lot of

1117
00:41:56,760 --> 00:41:59,069
things can happen right what happens if

1118
00:41:59,069 --> 00:42:00,480
you die in the two minutes in which you

1119
00:42:00,480 --> 00:42:03,540
were dumping the state well at least you

1120
00:42:03,540 --> 00:42:04,800
have to throw away everything you were

1121
00:42:04,800 --> 00:42:06,089
doing because you're not sure where you

1122
00:42:06,089 --> 00:42:08,849
died right it's very hard to say where

1123
00:42:08,849 --> 00:42:10,680
were you you see even the information

1124
00:42:10,680 --> 00:42:13,470
where were you it's very hard to capture

1125
00:42:13,470 --> 00:42:17,480
and propagate fast enough right so

1126
00:42:17,480 --> 00:42:21,230
because these processors move so fast

1127
00:42:21,230 --> 00:42:24,119
capturing very fine-grained information

1128
00:42:24,119 --> 00:42:25,829
about what is it they are doing at any

1129
00:42:25,829 --> 00:42:27,390
given moment of time its itself a

1130
00:42:27,390 --> 00:42:29,970
tremendously hard problem right that is

1131
00:42:29,970 --> 00:42:32,400
reflected to some extent into into this

1132
00:42:32,400 --> 00:42:35,220
thing so it's a matter of relative speed

1133
00:42:35,220 --> 00:42:37,559
of various pieces right if things move

1134
00:42:37,559 --> 00:42:39,180
too fast you have no chance to maintain

1135
00:42:39,180 --> 00:42:41,010
any kind of very very tight replication

1136
00:42:41,010 --> 00:42:43,260
so in general it's going to be easy to

1137
00:42:43,260 --> 00:42:46,619
define a very nice convenient kind of

1138
00:42:46,619 --> 00:42:49,079
replication but for the most part you

1139
00:42:49,079 --> 00:42:50,640
might not get anything close to that

1140
00:42:50,640 --> 00:42:53,099
with consistency consistency is even

1141
00:42:53,099 --> 00:42:55,980
worse than that okay so for example some

1142
00:42:55,980 --> 00:42:57,720
ideas you might actually apply in here

1143
00:42:57,720 --> 00:42:59,579
might be you know what I know the state

1144
00:42:59,579 --> 00:43:00,990
is very large and so on but maybe if I

1145
00:43:00,990 --> 00:43:03,930
keep track of sequences of actions a

1146
00:43:03,930 --> 00:43:05,520
very very high level actions that led to

1147
00:43:05,520 --> 00:43:09,869
the changes in the in the state if I can

1148
00:43:09,869 --> 00:43:12,119
play back those actions so I might dump

1149
00:43:12,119 --> 00:43:14,099
all the state but very rarely maybe once

1150
00:43:14,099 --> 00:43:16,200
a day and then try to play back all the

1151
00:43:16,200 --> 00:43:17,130
things that were doing in the meantime

1152
00:43:17,130 --> 00:43:19,440
in order to recover now that's going to

1153
00:43:19,440 --> 00:43:21,839
have the good effect and a bad effect

1154
00:43:21,839 --> 00:43:23,309
the good effect is going to be is going

1155
00:43:23,309 --> 00:43:24,480
to significantly reduce the amount of

1156
00:43:24,480 --> 00:43:25,859
information I need to capture in the

1157
00:43:25,859 --> 00:43:28,109
replicas the bad effect is I might take

1158
00:43:28,109 --> 00:43:31,170
a long time to replicate I mean to

1159
00:43:31,170 --> 00:43:33,420
recover from the replicas okay this

1160
00:43:33,420 --> 00:43:35,339
problem for example it's also similar to

1161
00:43:35,339 --> 00:43:38,400
a relative related issue within spec

1162
00:43:38,400 --> 00:43:40,980
redundancy in hardware for storage how

1163
00:43:40,980 --> 00:43:43,710
many people know about raid to some

1164
00:43:43,710 --> 00:43:45,390
extent raid it's about replicas and

1165
00:43:45,390 --> 00:43:47,250
about recovery there's a replication

1166
00:43:47,250 --> 00:43:49,200
it's related to recovery at least it's

1167
00:43:49,200 --> 00:43:53,760
some aspects of it so if everything goes

1168
00:43:53,760 --> 00:43:57,060
fine right in a raid system things get

1169
00:43:57,060 --> 00:43:58,830
replicated in a funny way I mean depends

1170
00:43:58,830 --> 00:44:00,270
on which raid you use I mean they might

1171
00:44:00,270 --> 00:44:01,680
be mirrored the night actually use

1172
00:44:01,680 --> 00:44:03,840
checksums and whatnot right and it runs

1173
00:44:03,840 --> 00:44:06,000
fast as long as nothing wrong happen

1174
00:44:06,000 --> 00:44:08,490
when something wrong happens it might

1175
00:44:08,490 --> 00:44:09,810
actually take you an enormous amount of

1176
00:44:09,810 --> 00:44:11,220
time to recover that from the something

1177
00:44:11,220 --> 00:44:12,540
from for example if a hard drive fails

1178
00:44:12,540 --> 00:44:14,400
you can plug in another hard drive then

1179
00:44:14,400 --> 00:44:16,500
recover the raid array it might easily

1180
00:44:16,500 --> 00:44:19,530
take a couple of hours to do so right so

1181
00:44:19,530 --> 00:44:21,180
a big consideration with respect to

1182
00:44:21,180 --> 00:44:24,540
replication is look and the reason

1183
00:44:24,540 --> 00:44:27,240
you're replicating and figure out how

1184
00:44:27,240 --> 00:44:28,650
much do you care about the speed at

1185
00:44:28,650 --> 00:44:30,630
which you can recover from the replicas

1186
00:44:30,630 --> 00:44:33,090
so if you drap replicate for the purpose

1187
00:44:33,090 --> 00:44:35,460
of conserving the information one way or

1188
00:44:35,460 --> 00:44:36,750
another you have to ask yourself the

1189
00:44:36,750 --> 00:44:38,790
question what's the price for example a

1190
00:44:38,790 --> 00:44:41,280
very nice way to to do for example

1191
00:44:41,280 --> 00:44:44,250
backups right so usually the backups the

1192
00:44:44,250 --> 00:44:45,960
daily backups go on some reasonably fast

1193
00:44:45,960 --> 00:44:48,000
storage then the monthly backups go on

1194
00:44:48,000 --> 00:44:50,130
some further away storage and we also

1195
00:44:50,130 --> 00:44:53,010
have yearly backups and decade backups

1196
00:44:53,010 --> 00:44:55,520
on some sort of tapes that are somewhere

1197
00:44:55,520 --> 00:44:58,340
easily can take you months to recover

1198
00:44:58,340 --> 00:45:01,860
from certain backups right so the fact

1199
00:45:01,860 --> 00:45:03,150
that you have a replica doesn't mean

1200
00:45:03,150 --> 00:45:05,220
you're gonna get it anytime soon okay

1201
00:45:05,220 --> 00:45:07,440
and that in itself it's a problem and

1202
00:45:07,440 --> 00:45:09,690
you see we say months and that sounds

1203
00:45:09,690 --> 00:45:13,500
horrible but for the financial guys ten

1204
00:45:13,500 --> 00:45:16,080
milliseconds is horrible so if you do

1205
00:45:16,080 --> 00:45:17,340
some replication and it takes you ten

1206
00:45:17,340 --> 00:45:18,810
milliseconds to get to the replication

1207
00:45:18,810 --> 00:45:20,850
you lost the game because the other guys

1208
00:45:20,850 --> 00:45:22,020
aren't in front of you and they make

1209
00:45:22,020 --> 00:45:25,380
money and you don't write on the wall

1210
00:45:25,380 --> 00:45:27,660
street now with the fast trading because

1211
00:45:27,660 --> 00:45:28,860
all of them use the same kind of

1212
00:45:28,860 --> 00:45:30,120
algorithms and the same kind of things

1213
00:45:30,120 --> 00:45:32,250
right it's kind of going around if

1214
00:45:32,250 --> 00:45:33,600
you're late to the game you lose money

1215
00:45:33,600 --> 00:45:34,740
it's not so much that you don't make

1216
00:45:34,740 --> 00:45:37,020
money you literally lose money and in

1217
00:45:37,020 --> 00:45:38,640
the end this is a zero-sum game right if

1218
00:45:38,640 --> 00:45:39,960
somebody makes money somebody has to

1219
00:45:39,960 --> 00:45:42,120
lose money I mean kind of the trend is

1220
00:45:42,120 --> 00:45:44,130
things I'm going upwards but you have

1221
00:45:44,130 --> 00:45:45,210
enough of this fluctuation

1222
00:45:45,210 --> 00:45:47,730
right so

1223
00:45:47,730 --> 00:45:51,720
always speed at which things can be

1224
00:45:51,720 --> 00:45:54,390
accessed or replicated or recover from a

1225
00:45:54,390 --> 00:45:56,370
replica have to be put in the context of

1226
00:45:56,370 --> 00:45:59,040
the specific application right months to

1227
00:45:59,040 --> 00:46:02,940
recover might be nothing if things don't

1228
00:46:02,940 --> 00:46:05,430
happen faster than at the level of a

1229
00:46:05,430 --> 00:46:07,770
year for example right it's all relative

1230
00:46:07,770 --> 00:46:09,810
and all has to be considered so the

1231
00:46:09,810 --> 00:46:11,820
overall picture I'm trying to paint is

1232
00:46:11,820 --> 00:46:13,170
the following when it comes to

1233
00:46:13,170 --> 00:46:15,960
replication and consistency it's worth

1234
00:46:15,960 --> 00:46:17,970
taking the time to really find what you

1235
00:46:17,970 --> 00:46:19,350
care about and what you don't care about

1236
00:46:19,350 --> 00:46:21,750
much more so than the other issues if

1237
00:46:21,750 --> 00:46:23,820
you do so you could pick a better

1238
00:46:23,820 --> 00:46:25,950
compromise with respect to what's the

1239
00:46:25,950 --> 00:46:27,930
specific replication technique you're

1240
00:46:27,930 --> 00:46:29,340
gonna use for your specific application

1241
00:46:29,340 --> 00:46:33,480
and I'm giving this example over and

1242
00:46:33,480 --> 00:46:35,190
over and over again but I think to a

1243
00:46:35,190 --> 00:46:37,350
large extent Amazon is the master of

1244
00:46:37,350 --> 00:46:39,180
picking the right compromise for the

1245
00:46:39,180 --> 00:46:40,590
replication I mentioned this before

1246
00:46:40,590 --> 00:46:43,500
right they don't even bother that much

1247
00:46:43,500 --> 00:46:47,550
with keeping track of for them right if

1248
00:46:47,550 --> 00:46:52,710
somebody buys an item and let's say on

1249
00:46:52,710 --> 00:46:54,120
the East Coast it goes to a different

1250
00:46:54,120 --> 00:46:56,190
server than somebody buying the item on

1251
00:46:56,190 --> 00:46:57,330
the west coast even though it looks like

1252
00:46:57,330 --> 00:47:00,000
amazon.com to all of them instead of

1253
00:47:00,000 --> 00:47:01,830
freaking out and saying you can't

1254
00:47:01,830 --> 00:47:03,870
possibly sell the same item to to both

1255
00:47:03,870 --> 00:47:06,810
guys that has a significant constraint

1256
00:47:06,810 --> 00:47:08,250
on the replication that means if

1257
00:47:08,250 --> 00:47:10,530
somebody buys it here the database

1258
00:47:10,530 --> 00:47:12,210
server whatever server keeps track of

1259
00:47:12,210 --> 00:47:13,350
the transaction they have their own

1260
00:47:13,350 --> 00:47:15,480
database like technology right would

1261
00:47:15,480 --> 00:47:16,830
have immediately to contact all the

1262
00:47:16,830 --> 00:47:18,420
other servers and tell them this person

1263
00:47:18,420 --> 00:47:19,830
bought the item instead of doing that

1264
00:47:19,830 --> 00:47:22,950
they say yeah whatever why can't they do

1265
00:47:22,950 --> 00:47:24,960
that right they've done this judgment

1266
00:47:24,960 --> 00:47:27,090
and say it's not that big of a problem

1267
00:47:27,090 --> 00:47:29,130
to sell items you don't have it's much

1268
00:47:29,130 --> 00:47:32,160
bigger problem to make sure so for

1269
00:47:32,160 --> 00:47:33,930
example if you would buy an item and it

1270
00:47:33,930 --> 00:47:35,940
will take about ten seconds for the

1271
00:47:35,940 --> 00:47:37,650
system to come back and say yes indeed

1272
00:47:37,650 --> 00:47:40,950
the item is yours most people would

1273
00:47:40,950 --> 00:47:43,410
abandon the service right the

1274
00:47:43,410 --> 00:47:46,260
uncertainty the uncertainty whether your

1275
00:47:46,260 --> 00:47:47,730
purchase went through or didn't go

1276
00:47:47,730 --> 00:47:50,220
through it's really alienated for people

1277
00:47:50,220 --> 00:47:51,870
the worst of the worst was the Facebook

1278
00:47:51,870 --> 00:47:53,940
incident on the New York on the Nasdaq

1279
00:47:53,940 --> 00:47:55,890
oh by the way a twitter decided not to

1280
00:47:55,890 --> 00:47:57,720
go in Nasdaq probably for the same

1281
00:47:57,720 --> 00:47:59,970
reason right I don't know if you've

1282
00:47:59,970 --> 00:48:01,319
heard the news they decided to

1283
00:48:01,319 --> 00:48:04,410
on your socket sheet they hope they have

1284
00:48:04,410 --> 00:48:07,890
better servers maybe okay so the

1285
00:48:07,890 --> 00:48:09,239
uncertainty that something happened or

1286
00:48:09,239 --> 00:48:11,279
didn't happen is really devastating for

1287
00:48:11,279 --> 00:48:12,660
for the humans now here's an interesting

1288
00:48:12,660 --> 00:48:13,229
question

1289
00:48:13,229 --> 00:48:14,519
look computer science is not about

1290
00:48:14,519 --> 00:48:15,989
humans why is this guy talking about

1291
00:48:15,989 --> 00:48:18,420
humans because ultimately that's what

1292
00:48:18,420 --> 00:48:20,369
determines the failure or the massive

1293
00:48:20,369 --> 00:48:23,400
success of anything you do right a sewer

1294
00:48:23,400 --> 00:48:25,019
way to have a disaster on your hands is

1295
00:48:25,019 --> 00:48:28,650
to ignore the user and to say hey what's

1296
00:48:28,650 --> 00:48:30,569
a second in a plane a person's life

1297
00:48:30,569 --> 00:48:32,789
everybody can wait for a second no no no

1298
00:48:32,789 --> 00:48:37,769
okay if if users got used to faster

1299
00:48:37,769 --> 00:48:39,359
things and right it's the difference

1300
00:48:39,359 --> 00:48:41,699
between smooth people say oh I like

1301
00:48:41,699 --> 00:48:44,699
Amazon better because it's smooth what's

1302
00:48:44,699 --> 00:48:47,160
smooth well that 30 milliseconds right

1303
00:48:47,160 --> 00:48:50,640
or oh I really hate this service because

1304
00:48:50,640 --> 00:48:53,039
keeps on being choppy and a lot of

1305
00:48:53,039 --> 00:48:55,109
services died I mean a lot of ecommerce

1306
00:48:55,109 --> 00:48:57,390
sites died because the really insistent

1307
00:48:57,390 --> 00:48:59,190
on the wrong kind of consistency right

1308
00:48:59,190 --> 00:49:01,440
so this these decisions involving

1309
00:49:01,440 --> 00:49:03,059
looking yeah whatever in just a decision

1310
00:49:03,059 --> 00:49:04,229
they know it's not I mean it's the

1311
00:49:04,229 --> 00:49:07,140
difference between big fortune and

1312
00:49:07,140 --> 00:49:12,299
complete financial ruin okay and it's

1313
00:49:12,299 --> 00:49:14,039
hard as a programmer to tell what will

1314
00:49:14,039 --> 00:49:16,799
irritate the users I'm okay let me let

1315
00:49:16,799 --> 00:49:17,729
me give an example it's not quite

1316
00:49:17,729 --> 00:49:19,349
related to this bug it's an idea how

1317
00:49:19,349 --> 00:49:20,759
much money you could lose for little

1318
00:49:20,759 --> 00:49:23,160
things right in about one minute

1319
00:49:23,160 --> 00:49:26,789
because okay it's it's too short for a

1320
00:49:26,789 --> 00:49:29,609
replication story so a big success story

1321
00:49:29,609 --> 00:49:33,059
obsess the big data mining slash

1322
00:49:33,059 --> 00:49:35,519
statistical software that they have on

1323
00:49:35,519 --> 00:49:39,650
their website is that they held Expedia

1324
00:49:39,650 --> 00:49:41,910
make about thirty million dollars more a

1325
00:49:41,910 --> 00:49:44,190
year by just noticing something so what

1326
00:49:44,190 --> 00:49:45,509
they notice is the following so question

1327
00:49:45,509 --> 00:49:46,859
is how do you find this well that's a

1328
00:49:46,859 --> 00:49:47,430
different story

1329
00:49:47,430 --> 00:49:51,269
they notice that Expedia loses something

1330
00:49:51,269 --> 00:49:53,549
like a large percentage about ten

1331
00:49:53,549 --> 00:49:55,009
fifteen twenty percent of the people

1332
00:49:55,009 --> 00:49:57,420
through the checkout process when all

1333
00:49:57,420 --> 00:49:59,219
these people find a plane they want to

1334
00:49:59,219 --> 00:50:01,589
put twenty key they want to purchase a

1335
00:50:01,589 --> 00:50:02,969
girl they go they go and at some point

1336
00:50:02,969 --> 00:50:05,910
they abandon things and the question was

1337
00:50:05,910 --> 00:50:07,769
why do they abandon things I mean nobody

1338
00:50:07,769 --> 00:50:09,209
notice they abandon things nobody cared

1339
00:50:09,209 --> 00:50:11,670
is like hey they only looked at how many

1340
00:50:11,670 --> 00:50:13,259
plane tickets they sell yeah everything

1341
00:50:13,259 --> 00:50:14,640
is fine SAS comes

1342
00:50:14,640 --> 00:50:17,190
that's a massive data analysis hey guys

1343
00:50:17,190 --> 00:50:19,559
20% of the people abandon the checkout

1344
00:50:19,559 --> 00:50:21,539
process so it turns out that people who

1345
00:50:21,539 --> 00:50:22,559
are abandoning the checkout process

1346
00:50:22,559 --> 00:50:26,249
because of a single line in in the page

1347
00:50:26,249 --> 00:50:30,059
right so when asked to plug in the

1348
00:50:30,059 --> 00:50:32,700
credit card information the line there

1349
00:50:32,700 --> 00:50:38,670
said address credit card address and a

1350
00:50:38,670 --> 00:50:40,170
lot of people were confused what credit

1351
00:50:40,170 --> 00:50:42,150
card address means is if you address of

1352
00:50:42,150 --> 00:50:43,739
the bank that issued the credit card or

1353
00:50:43,739 --> 00:50:45,660
is my address and they abandon this and

1354
00:50:45,660 --> 00:50:48,749
I went to Orbitz they fix this little I

1355
00:50:48,749 --> 00:50:50,009
mean literally all you have to do is say

1356
00:50:50,009 --> 00:50:54,470
the credit card of the cardholder

1357
00:50:54,470 --> 00:50:57,420
instead of credit card address so the

1358
00:50:57,420 --> 00:50:59,880
lesson here is when you're doing this

1359
00:50:59,880 --> 00:51:01,349
kind of interface especially when real

1360
00:51:01,349 --> 00:51:03,930
humans use your stuff try to consult

1361
00:51:03,930 --> 00:51:05,700
with somebody who understands humans and

1362
00:51:05,700 --> 00:51:09,029
say does this make sense because you

1363
00:51:09,029 --> 00:51:13,289
could ruin your company right or makes

1364
00:51:13,289 --> 00:51:15,480
us rich but they discover that they need

1365
00:51:15,480 --> 00:51:17,039
to improve the English in one of the

1366
00:51:17,039 --> 00:51:18,749
sentences right so everything matters

1367
00:51:18,749 --> 00:51:20,489
because remember that okay I'll see you

1368
00:51:20,489 --> 00:51:22,140
next time when we talk in earnest about

1369
00:51:22,140 --> 00:00:00,000
consistency and replication