The idea of executing user code (a handler) in kernel mode is not new. In Chapter 3 we will discuss previous work in this area as well as other approaches to avoid costly context switches. In this section we discuss four software approaches for user level handlers in the kernel.
All four methods deal with the problem of protecting the integrity of the system from user code that is running inside the kernel with supervisor mode privileges. Specifically, a handler running inside the kernel must not be able to access memory or memory mapped devices that it cannot access when run in user mode. Furthermore, privileged instructions that cannot be executed in user mode must not be executed by kernel embedded handlers. The running time of a handler also needs to be restricted to limit processor resource use on behalf of a user process. We call these restrictions privilege restrictions. The methods described in the following sections either enforce these privilege restrictions, or ensure that a handler voluntarily follows them.
Common to all methods are the following steps. At code insertion time a user level handler is inserted into the kernel and associated with an event. When the event occurs, the handler is executed inside the kernel without a context switch to user mode. During code insertion the kernel may perform a method-specific inspection and link step. The code is then stored inside the kernel where it cannot be altered by the user. For each method, we describe the steps performed during code insertion and how the privilege restrictions are enforced or guaranteed during execution.
The first method, software based fault isolation, comes in two
flavors: sandboxing and segment matching. Sandboxing forces all
memory accesses to specific memory segments called the
sandbox
. Segment matching checks each memory access and reports
an error if an attempt to access memory outside the sandbox is
detected.
The second approach is to trust a compiler to produce code that does not misuse any of the privileges gained by running in supervisor mode. Code produced by the trusted compiler is assumed to not violate the privilege restrictions. Software based fault isolation and the trusted compiler approach have attracted much attention recently with the advent of extensible operating systems such as the MIT Exo kernel [28] and SPIN [7, ].
A third approach, interpretation, is currently out of fashion based on the expectation that interpretation is too slow [86]. On the other hand, interpretation is enjoying a comeback in the form of Java, a byte code interpreted language. Java is used to create applets that are sent across the World Wide Web (WWW) [63, ]. Although not executed in kernel mode, applets cannot be trusted since origin and content are unknown at the time of execution. Applets can access files on the client, connect to other hosts, and send mail with the same privileges as the user who down-loaded the applet. This ability poses a safety hazard. Java bytecode is interpreted in software to protect against such unwanted activities of an applet.
The fourth possible approach to embed a handler inside a kernel is code inspection. If the kernel can formally verify that a given function does not violate the restrictions imposed on inserted code, then the kernel can execute that function safely.