Transcript


00:00
the following content is provided under
00:01
a Creative Commons license your support
00:04
will help MIT OpenCourseWare continue to
00:06
offer high quality educational resources
00:08
for free
00:09
to make a donation or view additional
00:12
materials from hundreds of MIT courses
00:14
visit MIT opencourseware at ocw.mit.edu
00:28
so my goal for today is twofold the
00:30
first is we've looked at the history of
00:33
the internet in the 60s and 70s and
00:35
today my goal is to tell you about what
00:37
happened in the 80s and the 90s and as
00:40
well as the last decade and we'll do
00:43
that maybe you know 3035 minutes but you
00:46
know I want to tell you about two
00:47
interesting problems that God solved
00:49
both related to topics we've studied the
00:51
first is on how the internet dealt with
00:54
a serious problem called congestion
00:56
collapse which happened in the mid-1980s
00:58
where TCP the transmission protocol at
01:02
the time was still the dominant protocol
01:04
in the world didn't have many of the new
01:08
ideas that it now has in fact the
01:10
implementation of TCP in the 1980s was
01:14
almost exactly the implementation of
01:16
your sliding window protocol from the
01:19
second lab from the second task of the
01:22
last lab and in fact some of you are I
01:25
think there was I think Tyler posted on
01:28
Piazza you know there was this
01:29
particular way in which he had dealt
01:30
with the windowing scheme that happened
01:32
to do better than the lab solutions and
01:33
I remarked there that the reason the lab
01:36
solutions is the way they are a little
01:37
bit more general than the particular
01:39
topology but the topology that I picked
01:41
was specifically picked in the lab so
01:43
you didn't have to worry about
01:44
congestion happening but today I'll tell
01:47
you what happens when congestion happens
01:48
and the solutions that were adopted
01:51
it'll be a little bit of a preview to
01:52
six or three three you'll study this
01:53
topic at some length in six or three
01:55
three the other problem I want to tell
01:57
you about today when through these
01:59
remarks is about how easy it is to
02:02
hijack routes on the Internet and I'll
02:04
go through some examples of this
02:06
happening in reality I mean it is
02:08
literally something a bunch of you know
02:09
you can do in your dorm you just have to
02:11
convince some sort of and you know
02:13
pretend you're an ISP or set yourself up
02:15
as a small Internet service provider and
02:17
you can actually recover a bit of damage
02:19
on to the rest of the world
02:21
if you're so inclined and then pretend
02:23
that it was just a bar you know editor
02:25
so those are the two technical ideas I
02:28
mean none of the stuff is really on the
02:30
you know on the quiz though it's
02:32
probably helpful to think about these
02:35
things about route hijacking because you
02:37
know those apply to some of the concepts
02:39
we've studied but you know a lot of this
02:41
is just for out of your own interest and
02:42
I'm hoping to pique your curiosity so
02:44
later courses okay so in the 1980s the
02:50
internet the people designing the
02:52
protocols of the internet started to get
02:54
organized and as you recall Vint Cerf
02:56
and Bob Kahn were the leaders of the
02:58
effort was a big community of people
03:00
contributing to what became the Internet
03:02
and back in those days it was still the
03:04
ARPANET ARPA had funded this entire
03:06
project and it was starting to be
03:08
successful in the 1980s rapid growth
03:10
started people say how the Internet is
03:12
booming and how it's exploding at 80 or
03:14
90 percent a year it's been growing at
03:17
about 80 to 90 percent a year since
03:18
about 1983 or 1984 this explosive growth
03:21
has been happening for many many for
03:24
several decades now Dave Clark who's the
03:27
senior research scientist at MIT and on
03:29
the faculty in our department was
03:32
designated as the Internet's chief
03:34
architect and one of the things he did
03:36
was to get the community organized and
03:38
formalize the creation of internet
03:41
standards you know you have all these
03:42
different companies and organizations
03:43
and universities coming together how do
03:46
you standardized protocols and the
03:48
argument that the way they came up with
03:50
the the approach they came up with was
03:54
called the internet Engineering Task
03:55
Force or IETF and they would write these
03:58
proposals they had been there had been
04:01
the strain of writing these proposals
04:03
for people to comment on called RFC's or
04:05
requests for comment
04:07
and now there are several thousand
04:08
requests for common nowadays request for
04:10
comment or after the comments have been
04:12
made usually go through an internet
04:13
draft stage and by the time there's it's
04:15
a request for comment it's pretty much
04:17
means the comments have already been
04:18
done but historically they were requests
04:20
for comments so they're called requests
04:22
for comments so if you are if you ever
04:24
go to a company where you're asked to
04:25
you know lots of things that are on
04:27
internet requests for comments and often
04:29
people are asked to implement pieces of
04:31
various standards and you typically look
04:33
at this document and you try to see if
04:34
there's someone written code around it
04:35
and then you adapt it or write it from
04:37
scratch to the specification so it's and
04:41
the interesting part of the standard was
04:44
very much in keeping with the general
04:46
approach of the if the internal ethos
04:47
which was to try to make everything be
04:49
open there are many standards bodies in
04:51
the world ie a Triple E he has things
04:55
various committees many many standards
05:01
and often they're based on voting and
05:02
voting means that people horse trade you
05:04
know you have a favorite feature I have
05:06
a favorite feature they both pretty
05:09
crappy features but you know I'll vote
05:10
for your feature if you vote for mine
05:12
and people end up horse trading and on
05:14
in the ITF this the good old days
05:17
this kind of stuff didn't happen now
05:18
it's changed but in the good old days it
05:21
you know there was this quote which says
05:23
we reject Kings presidents and voting we
05:25
believe in rough consensus and running
05:27
code the idea was that you show me what
05:29
it does a lot of people to experiment
05:31
with it and only after we see some
05:33
prototypes and some actual experiments
05:35
in the lab will we even consider making
05:37
it into a standard and this was the good
05:39
old days of the internet in which things
05:41
were on rough consensus and running code
05:43
so it wasn't around you know what let's
05:45
just vote and we'll get all our lousy
05:48
features in just because we care enough
05:49
about it personally
05:52
a big break happened in 1982 when the US
05:54
Department of Defense decided that
05:56
proprietary technology was not the way
05:57
to go with networking and standardized
06:00
on tcp/ip as the standard and back in
06:03
those days the Defense Department was a
06:04
huge consumer of IT it still is but back
06:08
then it was just usually dominant and
06:10
they could 1983 MIT created Athena the
06:16
large first really big large-scale
06:19
computing project we still have Athena
06:21
machines and it showed how to build
06:24
campus area networks and campus area
06:26
technologies they built distributed file
06:28
systems they built systems like Kerberos
06:29
and in fact they were one of the first
06:32
groups the first networks to experience
06:34
network congestion problems because
06:36
everybody started using the network to
06:38
do their work rather than have their own
06:41
computers at their desk or use the
06:43
terminal to log in to some remote
06:45
mainframe which is the way in which
06:46
things used to be done I mentioned last
06:51
time that in 1984 we created the domain
06:53
name system this is the system that goes
06:54
between domain names like mit.edu to a
06:58
an IP address and before that it was an
07:00
FTP dot how many people have heard of
07:02
FTP File Transfer Protocol nobody really
07:06
uses that these days but you used to
07:08
download this every night it computers
07:10
with computers were downloaded from one
07:12
machine located I believe in California
07:13
and didn't really scale and and people
07:19
were decided that you needed to get
07:21
organized and build a domain name system
07:22
in 1985 our powers the Defense
07:26
Department is starting to get out a
07:27
little bit of maintaining the entire
07:30
communication network and the National
07:32
Science Foundation started taking over
07:34
the running of the backbone connecting
07:36
all the non-military networks in the
07:38
United States and this led to the NSFNET
07:40
I'll have more to say about NSFNET a
07:43
little bit now one of the there are two
07:47
big growth problems that were expired
07:49
the first was congestion and the second
07:51
had to do with how addresses started
07:53
running out and we had to deal with
07:56
problems in the routing system so I want
07:58
to talk about congestion in 1986 the
08:01
internet experienced the first of a
08:03
series of congestion collapses a
08:06
congestion collapse is a situation where
08:08
you end up with a picture that looks
08:10
like this where if you were to draw the
08:15
offered load on a network or any system
08:17
for that matter
08:20
which is just you know how many people
08:23
are clamoring to get access to the
08:24
network and send their data on the
08:25
network if you were to plot the
08:27
throughput that you get you know or the
08:33
utilization that you get from the
08:35
network you typically would see a curve
08:37
that looks like this as the often in old
08:39
drawers you might see a somewhat linear
08:41
increase in the throughput
08:42
but slope one because the network isn't
08:44
congested no packets are being dropped
08:45
you push some data and you get the data
08:47
out the offered load that throughput
08:50
tracks the offered load and at some
08:51
point you reach the capacity of the of
08:53
the link or of the network in general
08:55
and you might end up with a flat curve
08:57
like that and that's fine what you would
09:00
really like to do is as the offered load
09:02
keeps increasing and the throughput
09:04
saturates at the capacity of course it
09:07
means that what does it mean either the
09:09
delays are growing to infinity or
09:11
packets are being dropped right and what
09:14
you really like intuitively is for the
09:16
sources to realize that packets are
09:18
being dropped or they're not being
09:20
delivered in time so it may be slowed
09:21
down something has to happen but
09:23
congestive collapse is a worse
09:24
phenomenon what happens is beyond a
09:25
point the throughput sachet drops
09:28
sometimes precipitously and it might go
09:30
down to zero so people were running
09:33
Network applications
09:35
you know FTP and remote logins and so
09:38
email and so forth and what they were
09:40
finding was that you could be running
09:43
the ratio between what the capacity of
09:46
the network was and what you were
09:48
getting when you were running was a
09:50
factor of 100
09:53
so this is a real collapse even you
09:56
wouldn't be able to get anything through
09:57
so people were talking about going from
09:58
many kill in those days tens of kilobits
10:01
per second to bits per second in fact
10:04
there was a joke that there was a path
10:05
between in Berkeley between the
10:08
University of California and Lawrence
10:10
Berkeley labs which is probably four
10:11
hundred meters from each other and the
10:14
network rate was such that you could
10:16
actually run up the hill with a tape
10:18
drive and you'd get a hundred times
10:20
higher throughput than the network and
10:24
you know this is not even running up
10:25
very fast so this was a serious problem
10:27
this problem was dealt with by multiple
10:30
people who were working on it and it led
10:33
to a set of algorithms where the idea
10:36
was that you would like you know because
10:40
all these switches and routers were
10:41
already deployed people were interested
10:43
in end to end solutions to the problem
10:44
by end to end I mean solutions you could
10:47
deploy at the senders and the receivers
10:48
in the network without worrying about
10:50
what the network was doing in between
10:52
the actual algorithm that we all run
10:56
today had its roots in an algorithm
10:58
developed by van jacobson but and
11:03
there's a lot of work that has been done
11:04
since then by many other people in the
11:06
community as well and in parallel there
11:08
were people inside of Digital Equipment
11:10
Corporation over here in Massachusetts
11:12
working on similar problems and they
11:14
came up with ideas both these ideas were
11:17
basically the same idea
11:18
more or less except in the detail and
11:20
they also resemble what we've seen with
11:23
Mac protocols the idea is that if the
11:26
network is working reasonably well we're
11:28
going to try to be greedy and start to
11:31
send faster and faster
11:32
at some point we're going to find that
11:35
the network doesn't work so well and we
11:38
can determine that in one of a few
11:39
different ways one way to determine that
11:41
is that packets start getting lost and
11:44
we will assume that packets are lost
11:47
because queues overflow and congestion
11:49
happens we might also alternatively
11:52
assume that as queues in the network
11:55
grow packets get delayed more and more
11:57
when the network starts to get congested
11:59
and if we find that the round-trip times
12:02
are starting to increase
12:03
maybe we determined that congestion has
12:06
happened now there's been 30 years of
12:10
literature the problem has not yet been
12:11
completely solved and in fact we have
12:14
now an active research project going on
12:16
in my group about how to deal with these
12:17
problems in the context of video and
12:20
video conferencing in cellular in
12:22
wireless networks that you might that
12:23
you run with your phone on your phone so
12:26
it's still an open problem lots of
12:27
interesting research but the basic idea
12:29
is that
12:34
you have to adapt to what the network is
12:36
doing and the way you adapt is by
12:38
watching things in the network you watch
12:40
where the packets are being lost you
12:42
watch weather delays are drawing you
12:44
might watch what the receivers getting
12:47
you know how fast is the receiver
12:49
getting data from the sender and the
12:51
network and the intuition is the
12:53
following let's suppose that we were to
12:56
pick the correct window size you know
12:59
this goes to the sliding window and the
13:01
window size that you use we've said that
13:03
the right value of the window size is
13:05
roughly the bandwidth delay product well
13:07
the bandwidth delay product is the
13:09
product of the bottleneck link rate
13:11
multiplied by the minimum round-trip
13:13
time but the problem is the bottleneck
13:15
link rate is not fixed it is in the lab
13:17
we studied but in reality you have many
13:19
connections sharing the network many
13:21
applications sharing the network and
13:22
people come and go so the rate keeps
13:25
changing at one moment you might be
13:27
getting 100 kilobits a second the next
13:29
moment you might be getting a megabit
13:30
per second and on wireless networks it's
13:32
even worse
13:33
quite literally if I were to start an
13:36
application on my phone now connecting
13:38
to you know Verizon or AT&T but if I
13:40
step out of this room it's quite likely
13:42
that the actual rate experienced by my
13:45
phone might change by a factor of 4 or
13:48
factor of 8 within 2 seconds and that
13:52
has to do of course with the fact that
13:54
the signal-to-noise ratio I mean we're
13:55
surrounded by thick walls and metal and
13:59
the moment I go out it's going to be so
14:01
how do you deal with this problem
14:02
there's this one basic idea that's used
14:04
that's a fundamental idea very important
14:07
idea and it's called conservation of
14:08
packets
14:10
it's the same idea that you saw when you
14:12
build your sliding window protocol it
14:13
says that when you put in a packet into
14:15
the network and the packet reaches the
14:17
other end and you get an acknowledgement
14:19
for the other end it means that one
14:23
packet has left this point if you view
14:25
this as that picture as you see that
14:27
picture up there packets are like water
14:29
entering the pipe and then they leave
14:31
the pipe and then comes back if you have
14:35
managed to somehow by some magic pick
14:38
the appropriate window size then
14:40
conservation of packets is a good
14:43
principle for you to apply because what
14:45
it says is that the only time you're
14:47
allowed to put one more packet into the
14:48
network is when you're sure that a
14:50
packet has left the network and the way
14:53
you know that a packet has left the
14:55
network is because you receive an
14:57
acknowledgement so that packet of course
15:01
thus assume that you know the right
15:02
window size conservation package has
15:05
another really nice advantage which is
15:07
that let's say that you think you have
15:09
the right window size but in fact more
15:13
traffic comes in and the bandwidth the
15:18
rate at which you can send data reduces
15:21
what's going to happen is that because
15:24
other traffic came in the round-trip
15:26
times are going to grow which means that
15:30
acknowledgments to your packets are
15:32
going to come back a little slower
15:34
which means that you have a natural slow
15:37
down because acknowledgments come slower
15:39
you naturally slow down and send packets
15:42
a little slower and then of course then
15:45
if the congestion persists packets are
15:48
going to get lost and when packets get
15:51
lost the trick is you have to reduce
15:53
your window size so let's say you're
15:56
running at a window size in your lab or
15:58
50 packets when you implement this if
16:01
you find that packets up getting lost
16:03
you should drop your window size and one
16:06
way to drop the window size that TCP
16:08
uses is to reduce it by 1/2 and then
16:13
every time you find that acknowledgments
16:16
are coming back you get a little greedy
16:18
and you try to increase the window size
16:20
and there are many ways to increase the
16:22
window size now all of the stuff leads
16:26
you know requires a way when you start
16:28
the connection when you start an
16:29
application what do you do and I think
16:32
you pointed out an idea at the last time
16:34
when I first talked about this problem
16:35
which is in the beginning you let your
16:38
window size be one packet so you start
16:41
your window size at one packet so it's
16:43
like stop and wait but the beginning of
16:46
the connection if I were to draw time
16:47
here against the window size you started
16:50
one packet
16:52
and you just ship that packet out it
16:54
takes an entire round trip to get to the
16:55
other side you get an acknowledgement
16:57
back at that point with regular stop and
17:00
wait you keep the same packed window
17:02
size of one and you said one more target
17:03
you're not being greedy enough so one
17:07
thing you can do is when you get one
17:08
acknowledgment to double the window size
17:10
or rather increase the window size by
17:12
one every time you get an acknowledgment
17:14
you increase the window size by one so
17:16
the rule is what does that do well after
17:27
one round trip if I draw the same in
17:28
multiples of the round-trip time this is
17:30
one RTT this is to our titties
17:33
this is three our titties and so forth
17:37
so at the first at the beginning at the
17:41
zeroth or TT your window size is one you
17:43
get an acknowledgment you make your
17:47
window size be 2 right 1 plus 1 so at
17:49
this point your window size is 2 what is
17:53
the window size after 2 RT keys it's 4
17:58
because why is it for we sent our two
18:01
packets
18:02
you get two acts back your winner so for
18:04
the first one you went from two to three
18:05
then you went from three to four so your
18:07
winner size goes to four and then out
18:12
here if there's no losses that you
18:14
haven't yet reached the window size
18:16
hasn't yet reached the bandwidth delay
18:18
product of the connection you're
18:20
increasing exponentially so this is
18:24
growing fairly rapidly and at some point
18:27
you're going to grow too fast your
18:29
window size is going to exceed the
18:32
bandwidth delay product plus the queue
18:35
size of the bottleneck causing a packet
18:38
to be lost and at that point you can do
18:39
a bunch of things but one thing you can
18:41
do is to drop the window size by a
18:42
factor of two so whenever that happens
18:43
if there's a packet loss you drop by a
18:46
factor of two and you could continue to
18:48
try to grow exponentially but that would
18:50
be stupid because you draw exponentially
18:52
and if the network conduct conditions
18:54
haven't changed you're going to again
18:55
drop so at this point you could do
18:57
something else and what TCP does is to
18:59
start to grow linearly rather than
19:01
exponential growth once you experience
19:03
congestion you start to grow linearly
19:04
and then maybe you experience congestion
19:07
here you drop by 1/2 and linearly if you
19:09
have the sort of sort of behavior and
19:11
for most web connections that involve
19:14
downloading a small amount of data you
19:15
end over here for video and everything
19:17
else you go all the way and this is one
19:19
strategy there are many many others and
19:21
like I said
19:22
in various kinds of networks like
19:24
wireless networks it turns out the
19:26
suppression is not really that good and
19:28
there are open questions around how you
19:30
should actually design is everyone
19:33
understand the basic idea this is an
19:35
example of adaptive congestion control
19:36
you look at this in 6 or 3 3 and in 6 8
19:39
to 9 in more detail if you took these
19:41
classes any questions ok
19:47
all right I'm now moving over to the
19:49
1990s nothing else interesting happened
19:51
ladies but in the 1990s more things
19:55
happen ARPANET essentially ended as far
19:59
as the universities and everybody else
20:00
was concerned and in fact the transition
20:02
into Mena they were separate military
20:04
networks and in 1991 Tim berners-lee who
20:06
was also now here at MIT an engineer
20:09
professor invented saw a little thing
20:12
called the World Wide Web
20:13
he called with one word World Wide Web
20:15
was the name of the program and I found
20:17
this thing so he wrote a proposal in my
20:21
case I think 1989 to CERN where he was
20:28
working to his boss at CERN and it was
20:29
called information management a proposal
20:31
I don't know all these things you know
20:33
about how the word how what you know
20:35
what became the work should work with
20:36
links and so forth and his boss at the
20:38
time on top wrote vague but interesting
20:40
as his feedback and the proposal prism
20:42
of the interesting part from the make
20:44
part and you know allowed him to
20:46
actually proceed on this project which
20:48
became the worldwide web obviously it's
20:50
been tremendously successful
20:54
now by the mid-1990s the NSFNET wheat
20:58
was the backbone connecting the US
21:00
various US organizations the government
21:03
decided to essentially get out of the
21:04
internet service provider business or
21:07
rather the government decided not to
21:09
fund that activity anymore and you know
21:10
many of you have probably heard about
21:12
this joke about a little bit of truth in
21:20
this Al Gore was very instrumental in
21:22
the government kind of getting out of
21:24
the internet business and set up was
21:27
involved in committees that set of
21:29
regulations that led to Internet service
21:30
providers commercial Internet service
21:32
providers actually forming and
21:34
commercialize fees started to take off
21:36
which was a really really big change for
21:38
the internet because no longer was it
21:41
the case that there's this one
21:42
organization and one backbone network
21:44
that connects you know MIT and Harvard
21:46
and everybody else together and all the
21:48
companies together you had many many
21:50
people who could offer Internet service
21:52
and in fact compete with each other the
21:55
idea was this you know the Internet
21:58
service providers are different network
21:59
operators have to cooperate with each
22:01
other because we are interested in
22:03
connecting everybody on the internet
22:05
together but they also compete with each
22:07
other and the reason the computers they
22:09
compete for customers they farm customer
22:11
Verizon on our customer comcast for
22:14
example and yet Verizon and Comcast and
22:16
other ISPs have to actually cooperate to
22:19
get package through so how do you do
22:20
this
22:21
and it turns out that this is a tougher
22:23
problem than you might think and the
22:27
world.they people invented this protocol
22:29
called BGP or the border gateway
22:30
protocol which uses an idea we've seen
22:33
it uses tack vector to solve this
22:36
problem I'll talk a little bit about
22:37
that as well
22:38
the other thing that happened in the
22:41
internet was that IP address has started
22:44
to get depleted and we saw why the last
22:47
time everybody wanted those truss the
22:49
addresses and now in fact quite
22:52
literally there are no more IP version 4
22:55
IP addresses and so there were there was
22:58
a lot of work done on moving to other
23:00
versions of IP but the part that is
23:03
interesting is this idea of class
23:05
trustless addressing so the idea was
23:08
rather than have organizations that
23:11
either had to have 2 to 24 addresses or
23:15
2 to the 16 addresses so through the 8
23:17
addresses let's allow organizations to
23:18
have any number of addresses so I want
23:20
to tell you a little bit about what an
23:21
IP address means because everyone has
23:23
seen an IP address I want to explain how
23:25
what it means and how it really actually
23:27
was
23:29
so if you look at an IP address 18:30
23:31
1.0 282 which is which is one of my
23:34
machines that dotted decimal notation
23:37
with even a human readable numbers used
23:40
to make sense in the old days it used to
23:42
be that this is a Class A address from
23:43
MIT it's 18 dot whatever and MIT owned
23:47
all of that stuff now that also happens
23:49
to be true but as far as the network
23:52
infrastructure the switches are
23:53
concerned this thing is nothing more
23:54
than a 32-bit number that looks like
23:56
that now when this one a packet with
24:02
that number shows up at the switch with
24:04
a destination address with that number
24:07
really what happens is that the switch
24:09
the router does not have an entry for
24:12
every one of those destinations in the
24:14
world if it did it just would be too
24:15
much information so what it has is
24:17
information corresponding to a certain
24:19
thickness now that prefix could be of
24:22
arbitrary length it could have an entry
24:25
in it with just the first eight bits
24:28
which would signify that all packets
24:32
that show up with that first prefix of
24:35
eight bits would have to be forwarded
24:37
according to a rule according to the
24:39
link that was set for those first eight
24:41
principles or it could have an entry in
24:43
the table for sixteen bits or it could
24:45
have an entry in the routing table for
24:46
19 minutes or whatever and that depends
24:49
on how the routing system you know how
24:50
we advertise the routes and what it
24:53
contains so there's an important lesson
24:54
here when a switch advertisers are out
24:56
for a destination on the Internet the
24:58
destination is not the destination of
25:00
the is not the IP address of an end
25:02
point but what that destination is is
25:06
a prefix that represents a range of IP
25:09
addresses all of which are forwarded the
25:12
same way by the switch so one way of
25:17
writing this in notation that we can
25:19
understand as human beings more
25:20
conveniently is this idea of writing it
25:23
as 18/8 what that means is this notation
25:28
stands for all IP addresses which have
25:32
the first 8 bits in common which would
25:35
be 0 0 0 1 0 1 0 1 0 which stands for
25:38
the human readable number 18 and it
25:41
contains all 2 to the 24 addresses
25:44
corresponding to that reference so as
25:47
another example if I have that in my
25:49
routing table with a slash 17
25:51
it stands for 2 to the 15 consecutive IP
25:56
addresses all of which share the first
25:58
17 bits in common ok this makes sense
26:05
so this is what an IP address minion as
26:07
far as a switch is concerned an order
26:08
table entry is not the IP is not a
26:12
destination IP address but it's
26:14
something in this form it contains a
26:15
prefix and it contains a length so in
26:22
human notation 18 that 31 / 17 the 17 2
26:27
to the 15 bits which share the first 17
26:29
bits in home so what this means is that
26:33
with this notation inside the forwarding
26:36
table you can have an entry for one IP
26:39
address or two or four or eight or
26:42
sixteen all the way up to you know
26:45
whatever the maximum is right so it
26:48
allows us to build networks of different
26:50
sizes and let that networks identifier
26:53
be known to the rest of the internet now
26:55
in principle you could put every host in
26:57
the network and that would mean that you
26:59
have an entry that each of which looks
27:01
like a slash 32 if I did a slash 32 it
27:04
meant that it's an individual IP address
27:06
but that wouldn't scale and so we want
27:09
to allow people the flexibility of
27:10
having very different ranges sitting
27:12
inside the routing system and there's
27:15
one more rule and this rule is important
27:17
because I want to tell you this rule
27:18
because I'm going to tell you how
27:19
YouTube was hijacked by an isp in
27:21
pakistan
27:22
and it relies on your understanding this
27:24
particular forwarding rule and then I'll
27:27
tell you about how an ice beam china
27:29
hijacked 15% of the internet for a
27:31
couple of hour for a few hours and that
27:34
didn't require this rule but it's two
27:36
examples I want to tell you about but
27:37
let me explain the rule the forwarding
27:40
at a switch uses an idea called a
27:41
longest prefix match so what that means
27:44
is that if you have entries in your
27:46
forwarding table let's take those two
27:49
example let's say I have an entry in the
27:50
forwarding table this is at a particular
27:52
switch I have a forwarding table or a
27:54
routing table and the first entry says
27:57
18/8 so what this means is it's 2 to the
28:01
24 addresses that share the first 8 bits
28:06
which is whatever corresponds to 1800
28:09
whatever it is right and then
28:26
and then let's say I have another entry
28:28
which is some eighteen or thirty 1/17
28:30
and what that would be of course is two
28:33
to the 15 addresses and the prefix would
28:37
be as shown in that picture there
28:38
whatever the first seventeen bits are so
28:40
some sanity in this now if the switch
28:48
received a packet with an IP address and
28:50
that IP address matched multiple entries
28:54
you know you might have other entries
28:55
sitting here you know let's say that I
28:58
have when a packet arrives it may in
29:07
general match multiple entries here
29:09
right because the packet has a certain
29:11
bit string in its destination address
29:14
and that destination address now matches
29:16
multiple of these it could match one or
29:19
more of these entries here what an IP
29:22
router does when it gets such a packet
29:24
is to find the entry which matches in
29:26
the longest prefix in other words the
29:30
routing table entry that corresponds to
29:33
the longest prefix match between the
29:37
destination address of the packet and
29:39
between the entry in the forwarding
29:40
table is what you use to send the packet
29:43
on so if you got a packet that was you
29:46
know 18 dot 31.6 dot five and it
29:49
happened to match this entry it would go
29:50
on one leg let's say this is link 1 and
29:53
if you got this other thing that didn't
29:55
match this
29:58
so the output link that you use depends
30:01
on the longest prefix match and MIT many
30:04
organizations use this extremely well so
30:06
I was remarking to solve you the law at
30:08
the end of class last time that you know
30:10
it's funny mit has multiple internet
30:12
service providers and it turns out that
30:14
if I use this network here and I
30:17
download here I go to Linux guitar which
30:19
is a place you could download lengths
30:20
code if I go from here it so happens
30:23
that MIT uses level 3 which is I think
30:25
the world's biggest or USS biggest
30:27
internet service provider to get those
30:29
packets the same thing if I just go up
30:31
to Stata
30:32
and I connect to the standard Wireless
30:34
and go to Linux not all packets from the
30:36
next guitar come back to me through a
30:38
different ISP Cochin so MIT has decided
30:41
that it wants to load balanced its
30:42
traffic so it advertises the prefix
30:45
corresponding to the network in this
30:48
room whatever the Wi-Fi network in his
30:49
room or through one of the ISPs and it
30:52
advertises the other prefix in Stata to
30:55
through the other ISP and it does it
30:57
presumably to load balanced traffic and
30:59
it also has this idea that if one of
31:01
those links were to fail it would switch
31:03
the traffic to the other leg
31:05
organizations do this because they would
31:06
like to provide good service to the
31:08
people inside their cookie inside their
31:09
organization so the longest prefix match
31:12
is very crucial to how this all the how
31:14
the stuff really kind of works so keep
31:18
that in mind I'm going to come back to
31:19
this on to the next slide
31:25
in the rest of the 1990s futuristic
31:28
things happen one of them was that work
31:31
started on this new proposal for IP
31:33
called ipv6 which said let's not use
31:36
32-bit addresses let's go to 128 bit
31:38
bigger addresses and try to solve the
31:40
address depletion problem ipv6 has taken
31:42
a really really long time to get
31:43
deployed for reasons I won't go into
31:45
here but it seems to be happening now
31:47
but people keep saying that it seems to
31:49
be happening now I said that three years
31:50
ago when I did the wrap-up in this class
31:52
so sometimes at some point in the future
31:54
that statement will actually be true now
32:00
you know everybody knows about Google
32:02
reinventing how search is done and
32:04
starts to dominate distribution network
32:08
started getting created and these are
32:10
networks that you deploy as only
32:12
networks atop the internet to serve
32:14
content better and more reliable way now
32:18
in the 2000 the Internet mature and I
32:20
have the top five things that I think
32:21
happened in networking in the networking
32:23
industry and the networking world in
32:25
2000 so this you know dot-com bust
32:28
happened and then 9/11 happened
32:31
the first thing that happened was the
32:33
rise of peer-to-peer networks I'm sure
32:34
many of you have used this Nutella and
32:37
BitTorrent is the latest one there was a
32:40
lot of research done including here at
32:41
MIT you on how you build these
32:42
peer-to-peer networks the idea being you
32:45
don't have a central point of failure
32:46
and you can use this to distribute files
32:48
extremely efficiently I mean BitTorrent
32:50
is still highly dominant and the stupid
32:53
hash tables like card which was
32:56
developed here and other schemes are
32:57
used now by systems like Skype they're
32:59
used inside data centers like Amazon if
33:01
you go to Amazon and buy stock it uses
33:03
dynamo which is a key value store that
33:06
basically builds a district has tables
33:08
so it's not a lot of impact both in data
33:11
centers and in systems like Skype the
33:16
second thing that happened was that in
33:17
the early to mid 2006 was became a huge
33:20
deal people started attacking the
33:22
internet which was which came somewhat
33:23
of a surprise to people who grew up in
33:25
the good old days of the internet where
33:27
you know as I mentioned last time
33:29
computers with root passwords that were
33:31
empty because you know everybody could
33:32
be trusted and then they found that as
33:34
people started making money on the
33:35
internet people started trying to attack
33:38
the Internet as well denial of service
33:41
attacks started where people would
33:42
launch attacks on websites and they
33:45
would often use it to extort money this
33:47
would be like if you don't pay me I'm
33:50
gonna continue to pummel your website so
33:52
that you can't sell flowers or whatever
33:53
it is people found vulnerabilities in
33:58
software and there were many worms that
34:00
spread often pretty quickly sequel
34:02
slammer is a particularly interesting
34:03
one of these we studied this stuff in 8
34:05
to 9 and in 6.33 in some detail but this
34:08
was remarkable because in 30 minutes
34:10
it clawed the world's networks here's a
34:13
picture of a screen shot this was at
34:15
5:30 or 5:30 in the morning I guess UTC
34:20
banished time and you know nothing's
34:24
going on and then half an hour later the
34:28
blue splotches show the networks that
34:30
were clogged and almost every computer
34:32
that was vulnerable to this level that
34:34
many computers relevant to the world's
34:35
computers that were vulnerable to this
34:36
but all these networks got hammered and
34:39
in fact traffic came to a halt and this
34:41
room showed that the power of spreading
34:45
you know if machines trust each other or
34:47
are you know either implicitly or
34:49
explicitly it's very easy to actually
34:52
find a vulnerability in one and then
34:53
spread very very quickly so a lot of
34:55
work was done on how to handle worm
34:57
attacks a lot of this has to do with
35:00
putting things inside of networks which
35:02
is running at high speeds to identify
35:04
patterns that of in the payload in the
35:07
data that's being sent in
35:09
packets to quickly identify that this
35:11
corresponds to a worm and then throw
35:13
those packets away before they hit the
35:15
actual end points right now we don't see
35:17
too many wolves reading the one that's
35:18
spread now our slow spreading worms
35:20
they're often spread by human come home
35:23
you know human contact
35:24
it's like people clicking on one links
35:25
they shouldn't click on it runs the
35:27
program finds a vulnerability on their
35:29
machine and then it resides on the
35:31
machine and often these are then used to
35:34
create these big botnets that are used
35:36
to launch denial of service attacks or
35:38
often used to send spam and do other
35:40
things like that
35:41
so they're still going on but you don't
35:43
hear about them in the newspaper a spam
35:47
became a huge problem and it continues
35:50
to be somewhat of a problem though these
35:52
days you know the distinction between
35:54
spam and marketing the gaps closing but
36:00
by and large spam now is why I wouldn't
36:05
say the salt problem is generally
36:06
combatted by you know big organizations
36:09
that have enough data enough email
36:11
coming in that they can identify spam
36:12
and then filter those away not hijacking
36:16
is the other problem that remains a huge
36:17
vulnerability so one can you've got two
36:19
examples of route hijacking and I don't
36:22
think there's easy the technical side of
36:24
this problem we understand how to solve
36:26
but how to deploy good solutions is
36:28
still unclear so the first example of is
36:32
from this problem isn't going on every
36:34
three years you see a big route
36:35
hijacking problem the first one was from
36:37
2008 2008 where YouTube was unavailable
36:41
to people for a few hours everywhere in
36:43
the world
36:44
now here you could argue whether
36:47
watching cats dance or whatever is that
36:49
important but the fact is that YouTube
36:51
has a lot of money Google has a lot of
36:52
money and even they were vulnerable to
36:54
this problem
36:55
the second example was a little
36:57
different China Telecom managed to get
37:02
about 15% roughly speaking of the
37:04
internet traffic to go through that now
37:07
the interesting thing about that attack
37:09
is that it wasn't clear it was an attack
37:11
I should just said of that error or
37:17
failure was that people didn't even
37:18
notice because unlike YouTube where you
37:21
know you go you couldn't get your data
37:22
and then you know people noticed it and
37:24
then they were clam are trying to
37:25
traveling to solve the problem with the
37:27
Chinese attack what another part of the
37:28
Chinese vulnerability what happened was
37:30
the China Telecom managed to divert the
37:32
traffic that wasn't supposed to go
37:34
through them to them and then they
37:36
forwarded the traffic on to the rest of
37:37
their destinations the actual
37:38
destination so you would find things
37:41
like instead of my latency being 100
37:43
milliseconds it might be 500
37:44
milliseconds which is yeah you know you
37:47
may not even notice or sometimes you
37:50
notice it and you go wow there's got
37:51
some paint in it being the ingenuity
37:52
that sometimes that happens but the fact
37:54
is that they were able to you know an
37:56
ISP was able to essentially divert a
37:57
large fraction of the world's traffic
37:59
and
38:01
both these attacks fundamentally stand
38:03
from the following problem which is that
38:04
at the end of the day despite all of
38:07
this investment into the Internet and
38:08
the importance of Internet Envy it is
38:10
the amount of money in a Internet
38:13
routing works because of essentially an
38:15
honor code it's like ice Pease at some
38:18
level Internet service providers and
38:20
organizations trust each other and
38:22
there's this transitive trust which is I
38:24
might trust you and you might trust her
38:26
and implicitly the way routing works is
38:29
that ends up in me trusting her because
38:31
I trust everything you tell me and you
38:34
happen to trust everything she tells you
38:35
which means that if she were to make a
38:38
mistake and you were to believe it then
38:40
in essence everybody else in the world
38:42
is vulnerable to this problem so let me
38:46
explain what happened within the case of
38:48
YouTube because it's reflective of how
38:52
things really work so here's this little
38:54
high-speed called Pakistan
38:59
okay tiny tiny high-speed you know
39:02
hardly anyone uses it obviously I mean
39:03
everyone in Pakistan probably uses it
39:05
but in the grand scale of things it's
39:07
it's completely tiny so they end up
39:10
connecting to a bunch of people outside
39:12
in different parts of the world and one
39:14
of the people they ended up connecting
39:15
to was another I spy out in Hong Kong or
39:18
pccw and these guys connect to the rest
39:24
of the Internet and presumably Pakistan
39:27
telecom connects to all the people I I
39:28
don't know now here's what happened
39:32
so where does YouTube fit into all this
39:33
you know YouTube is sitting somewhere
39:35
around here and presumably it's not
39:42
directly these guys have nothing to do
39:44
with each other
39:45
this these guys are connected to some
39:47
other big ISPs and small eyes these
39:49
women actually there is a set of
39:50
Internet service providers that somehow
39:52
you know
39:55
don't steal the internet and each of
39:58
these is independent each of these is
39:59
known as an autonomous system or AAS and
40:03
each of these autonomous systems has a
40:04
number a 16-bit number MIT is an
40:08
autonomous system and because MIT was
40:09
very early in internet MIT is autonomous
40:11
system number is 3 but right now there
40:15
are many many ISPs you know right now
40:17
you know you can have eyes there are
40:20
tens of thousands of autonomous systems
40:21
I don't know 35 40 45 thousand something
40:23
like that so anyway who's number one BBN
40:31
VPN and I don't know who a BBN and of
40:34
course is number one but actually that
40:36
number is owned by somebody who acquired
40:38
BBN who apart now here's an interesting
40:42
thing that's happening use these
40:44
autonomous system numbers I remember I
40:45
was a very young graduate student when
40:47
people were talking about these
40:49
autonomous system numbers and I remember
40:51
these mailing list discussions I wasn't
40:52
working on this problem then I worked on
40:54
it much later but people were saying you
40:56
know 16 bits is plenty enough for an
40:57
autonomous system identifier because
40:59
remember NSFNET was one there was one
41:02
internet service provider and the early
41:04
90s mid-90s people who are talking about
41:06
ISPs and they said 16 bits is plenty and
41:09
I remember there were some people
41:10
actually graduate students so we're
41:11
saying maybe we should make it 32 bits
41:14
because we're people remember that
41:16
Internet star remember those all things
41:18
of the internet where you know people
41:20
said eight network servers eight bits
41:22
for a network identifier or plenty
41:24
enough and of course they got screwed so
41:27
anyway what's happening now of course is
41:28
the older guys well
41:30
with the spread enough because of you
41:31
know we don't have too much overhead on
41:32
packets and they put it you know 16 bits
41:36
and now at 45,000 to 50,000 and guess
41:39
what there's a proposal on our how do
41:40
you how the heck can you extend this to
41:42
you know more than 200 to to the 16
41:45
autonomous systems because now the 18
41:47
that is growing so you know whenever if
41:50
every you're given an opportunity to
41:51
design the number of bits for something
41:53
and you will always have to do something
41:55
just pick something much much bigger
41:57
than you imagined and then double it
41:59
because it's always think you'll never
42:01
get it right so anyway each of these
42:04
autonomous systems has an identifier in
42:06
it okay and each of these guys when they
42:10
make an announcement in the routing
42:12
system the way it works is that you
42:13
create your your identifier and then you
42:16
tell people all of the IP prefixes that
42:18
you own okay so when I create my
42:22
distance vector or in this case a path
42:24
vector my advertisement if I am
42:26
autonomous system free I have a set of
42:29
IP addresses so MIT might have 18
42:31
whatever MIT has you know and what they
42:41
don't say is I'm autonomous system 3 and
42:43
I know how to get to these guys because
42:45
I own these guys this is the origin
42:48
announcement and then other people you
42:50
know MIT might but this is there's three
42:52
MIT sends it to its highest fees and
42:53
they send it to their osp's and every
42:56
time an autonomous system receives
42:58
multiple advertisements along different
43:01
paths for the same prefix they pick one
43:04
they select among them they have some
43:07
rules to select among them and these
43:08
rules have
43:10
length of the path between autonomous
43:12
system they have to do with how much you
43:14
pay so for example MIT might be getting
43:16
a better deal from cogent Dan from level
43:18
three and so it would want more of its
43:20
traffic to come from cogent and
43:22
therefore it would decide that it would
43:25
only advertise certain of its addresses
43:28
on 30 passes lots of policy has very
43:30
very complicated but yet you know some
43:32
miracle metal the whole thing works so
43:34
anyway these things go through from
43:36
autonomous system to Thomas so what's
43:38
this path vector right remember I talked
43:40
about the path the path vector is a
43:42
sequence of paths so it could be 3 17 26
43:45
and so forth so I have an animation of
43:47
this thing so I want to show that to you
43:49
because it's totally interesting so what
43:52
there's this website called BGP play I
43:55
can find it so there's a way to get mi
43:58
t--'s router advertisements over the
44:01
past month and you can kind of see how
44:04
these knots change so even what happened
44:06
to you - what happened because the
44:09
government of pakistan decided that what
44:11
they would tell pakistan telecom was to
44:14
not allow their users to go to youtube
44:16
so what they did was rather than simply
44:19
drop those requests they wanted to get
44:21
pakistan telecom when somebody clicked
44:23
on a youtube link to go to a website
44:25
that pakistan telecom would run that
44:29
basically said you're not allowed to use
44:30
youtube but would you like to see
44:32
something else ok so the way they did
44:34
that was they decided youtube has some
44:36
address let me call youtubes and just
44:38
why
44:39
take something let me just call it why
44:41
right why is this is a certified the
44:42
addresses that correspond to YouTube
44:44
it's not one they have many machines so
44:46
what these guys did was they did
44:48
something they thought was very clever
44:49
they have a whole network of users there
44:51
they decided they would advertise a
44:53
route for destination why inside their
44:58
network but rather than use the actual
45:03
route advertisement for why they would
45:05
actually send it to their own machine so
45:08
remember they change the routing now
45:09
they're telling the users they're
45:11
hijacking the route they're telling
45:13
their users that to go to youtube you
45:15
should not use the actual link that I'm
45:17
telling you to use but instead go to
45:18
this other place inside my network where
45:20
I can show you this other website
45:22
you know maybe pretend it's YouTube or
45:23
whatever now everything is so far so
45:26
good right I mean people do this all the
45:28
time you know when you go to a hotel or
45:29
any internet kiosk internet place you
45:31
take your laptop and you go to be
45:33
keynote cnn.com and your next thing you
45:35
say is would you like to sign in how
45:37
does that work well that works because
45:39
they essentially hijack the route they
45:41
make it look like you're going to CNN
45:43
but in fact you're going somewhere else
45:44
right after all your computer wrote to
45:47
go to CNN
45:49
the solution was presumably CNN but then
45:52
they made a mistake somehow here
45:53
probably he was too tired set up a
45:56
configuration so he advertised his new
45:59
route to why that he created out of a
46:01
CCW now
46:04
PCCW was to some degree at fault because
46:07
pccw should have known to some degree
46:10
what actual IP addresses pakistan
46:14
telecom owns and only owner Rob requests
46:18
route advertisements coming from those
46:20
things right PCCW needs to know how to
46:23
go how to send packets to nodes inside
46:26
of Pakistan telecom but clearly this guy
46:30
has nothing useful to say about how to
46:31
send packets to YouTube but yet this guy
46:35
honored this message so there were two
46:36
mistakes he liked to do many mistakes
46:38
but the two big ones were there was a
46:40
mistake made here sending something out
46:41
there was a mistake made here honoring
46:43
this request by this time you not
46:45
transitive trust because PCCW would find
46:48
and what they also did was they made
46:50
this a more specific prefix so YouTube
46:54
was advertising a slash I believe a
46:56
slash 21 which meant it was many many
46:59
bits in the prefix 11 bits in common and
47:02
twenty-one bits in common a slash 21 but
47:05
these guys advertise to guarantee that
47:07
all traffic coming they advertise 2/4 so
47:12
it's more specific so what happened was
47:14
pccw believe that and they react ties
47:17
back to their ISPs went to their eyes
47:19
please and so forth now the reason why
47:22
this really got all of the Internet's
47:24
traffic were sent towards the spoor guy
47:26
in Pakistan was because everybody is
47:29
doing this longest prefix match and it's
47:32
true that there's a legitimate route to
47:34
YouTube in those routers but they're
47:35
ignoring it because they find a more
47:37
specific route that somebody had
47:38
advertised so if you want to get traffic
47:40
sent to Google you cannot a way to
47:42
convince some bigger isp to take your
47:44
route to a more specific prefix that you
47:47
know is owned by Google and just
47:49
advertise it out ok you're probably
47:51
gonna get a lot of traffic now you know
47:53
you may not want all that traffic but
47:55
you get it so you see how the stuff
47:58
spread right how do you solve this
48:00
problem alright this is going on and
48:02
then people are not able to see their
48:03
cats and you know they're scrambling I
48:06
mean this was literally the whole
48:07
internet wasn't able to get to YouTube
48:09
which you know admittedly is not the
48:11
biggest problem in the world but still
48:12
if you are YouTube this is a big problem
48:14
so how do you solve this problem what
48:18
are you actually doing the whole world
48:19
has
48:20
that's bad entry in the routing table
48:22
now there's this long term solution
48:24
which is of course let's figure out a
48:25
way to authenticate the advertisements
48:27
and use some public key and this and
48:28
that you'll study this in six eight to
48:31
nine and other courses but today we
48:34
don't have that and this problem exists
48:36
so how do we ever come out of it yeah
48:40
actually you know it's interesting you
48:42
and I every new certainly thought about
48:43
this for 30 seconds but YouTube actually
48:45
did it took them a while to figure out
48:47
what was really going on because one of
48:48
the problems is you don't want to do
48:50
something like that without knowing for
48:51
sure but eventually they did exactly
48:53
this they figured out the prefixes that
48:55
were being advertised and that took a
48:57
while because you don't know somebody
48:59
else's what's in your routing table I
49:00
don't know but whatever there's a
49:07
there's a way to figure it's not right
49:09
following email or something and then
49:12
they figured out what it was and then
49:14
they inserted slash 25 and for more
49:17
specific and then once a problem kind of
49:19
resolved itself they got out of it so
49:22
let me jump on a move on give me two
49:24
more minutes and I'll finish finish up
49:27
then a similar problem happened with
49:29
China Telecom
49:30
they didn't advertise a more specific
49:31
route so they only got about 15 percent
49:33
of the Internet and they were nice
49:34
enough to forward it around to the
49:35
rescue to the actual destinations but
49:38
this was a problem I'm gonna skip
49:41
through the decade ahead because
49:47
all right what I do want to do is to
49:48
summarize six or two in one slide this
49:52
course was about how to design digital
49:53
communication networks I'm sure you all
49:56
kind of know that we did this with three
49:58
layers of abstraction very simple story
50:00
its signals and packets and I think as
50:04
far as these kinds of courses go across
50:06
the world it's a pretty unique storyline
50:09
there aren't very many courses that we
50:11
know which had this vertical study
50:13
across all the layers and there are some
50:16
schools now starting to adopt that we
50:18
know of that are starting to adopt this
50:19
idea and we feel like this is a pretty
50:21
unique way in which to teach this
50:23
because it demystifies all of the layers
50:25
so we didn't cover anything but we
50:27
didn't tell you 15 minutes to solve any
50:28
given problem but we told you one good
50:30
way to solve each of the problems that
50:32
you will see so we went from point to
50:35
point links to multi-hop communication
50:38
networks and across these different
50:40
topics and you can see that you know we
50:42
studied these different topics the two
50:44
big themes that I want to leave you with
50:46
our reliability and sharing because
50:49
that's really what makes our
50:50
communication systems work how do you
50:52
make it reliable
50:53
we don't have a perfectly reliable
50:56
communication medium at any layer so how
50:58
you make things reliable is important we
51:00
study this over and over again and how
51:02
do you share those are the two big