Number of Processes in a System

A first objective in analyzing a system is to determine how many processes are present at any given time, since the number of processes (roughly) determines the load on the system and the waiting time for each process. Clearly, the number of processes will fluctuate continually, sometime going down to zero, sometime growing to very large values. We cannot predict the fluctuations and thus are only interested in steady-state (long-term, ``average'') values. If the number of processes is the only figure of interest, we can characterize the state of the system simply by the number of processes present, with the result that a system can be in one of states , , , , , where the system's being in state denotes the presence of n processes. A process leaves the system on completion; new processes enter the system; thus transitions take place between the various states. If we observe the system at small enough intervals of time, say every units of time, then at most one event can take place between any two observations: a single process completion or a single process arrival. Hence a transition from state can only lead to one of states (due to a completion) or (due to an arrival). The resulting state diagram is pictured below.

The next problem is to place some rates (probabilities) on these state transitions. We can observe the system for a while and estimate the process arrival rate, at some number of processes per unit time. (Viewed differently, is the expected time between consecutive arrivals.) Similarly, we can deduce from observation the average number of processes completed per unit time, , in the current system--and scale the value as needed for a system of higher or lower performance. Note that the quantity now denotes the expected number of processes completed within an interval of time units; if we let that interval become the infinitesimally small dt of calculus, then is the probability of a completion within dt time units.

These values normally fluctuate quite a bit; we shall make two assumptions to allow us to treat them as constants:

• the rates are independent of state (with real users, we could expect lower arrival rates in a very busy system, so that the arrival rate in state would be somewhat smaller than the arrival rate in state ; similarly, a larger number of processes in the system may lead to a somewhat higher completion rate, up to a certain limit beyond which the overhead associated with a large number of processes may lead to steadily decreasing completion rates);
• the rates are independent of past events--in particular, the arrival rate does not depend on how long ago the last arrival took place.

The latter property is called the memoryless property and is a fundamental approximation in most of queueing analysis--it leads to so-called Markov models, which are mostly amenable to analysis, whereas systems with ``memory'' lead to intractable models. The former property, as we shall shortly see, is only a minor convenience--we can easily dispense with it and use state-dependent rates, at the expense of messier formulae.

Now consider again our state diagram. We can associate a probability with each state: will denote the probability of the system's being in state , i.e., the probability of there being i processes in the system. Clearly, the system must be in one of the states , , , so that we can write

Since we want to analyze our system in its steady-state, there cannot be any ``drift'' in the system; put more formally, the probability of moving out of a state must equal the probability of moving into that state. For state (i>0), this can be written as

In this equation the left-hand side is the probability of leaving state (through either a completion or an arrival), while the right-hand side is the probability of entering state , either by being in state and experiencing an arrival or by being in state and experiencing a completion. State has a simpler equation associated with it, since there is no state :

A bit of algebra shows that all of these equations can be put in the form

or simply

where is the utilization ratio of the system. (Notice that, if we had state-dependent transition rates, the only change would be that each equation would have its own value of ; i.e., we would have a system of equations of the type and thus .)

Now we have a system of equations given by

with the ``binding'' equation

Substituting the former in the latter, we get

or

We know from discrete mathematics that the infinite sum in the denominator converges for :

and diverges (grows to infinity) otherwise. We can thus conclude that, for , we have , while, for , we have . (With state-dependent rates, we cannot write such clean formulae, but the derivations remain the same.)

The probability distribution is a geometric distribution; we can look at as the probability of turning up i heads before turning up a tail in i+1 coin flips (so-called Bernoulli trials), where the coin is slightly biassed (instead of having a 50-50 chance of turning up a head, it has a probability of doing so). Strictly speaking, we have defined a Bernoulli arrival process, where the interarrival time obeys a geometric distribution. These are the discrete versions of the continuous arrival process known as a Poisson process and the continuous distribution known as the exponential distribution. Our setting is discrete, since the number of processes in the system at any given time must be an integer; however, you will often see ``Poisson process'' used in a discrete setting in lieu of the more correct ``Bernoulli process.'' While we are on the unimportant topic of terminology, the queueing system we are considering is generally symbolized as M/M/1, indicating that the arrival process is of type M (Markov, or memoryless), the departure process is also of type M, and there is 1 server. It can be proved that the memoryless property implies an exponential or geometric distribution.

Note that, somewhat surprisingly, the system is not stable when arrivals and departures are perfectly matched ( ). What happens is best understood by considering a system where we would also have states , , ; in that system, perfectly matched and give rise to good behavior (as you are invited to verify), because the system is allowed to move into ``negative states'' and thus accumulate a reserve. In the real system, once the number of processes drops to zero, every unit of time until the next arrival is wasted time: the system is idle and cannot accumulate any reserves, yet we know that, since , the system needs to be active every unit of time just to keep up. Hence, somewhat paradoxically, every time the system returns to idle, it gets farther behind! \ You can run a simulation of this behavior by yourself by using a coin: mark a line (a sidewalk will do, if you don't mind displaying strange behavior in public!) and move one step forward for each toss that turns up a head and one step backwards for each toss that turns up a tail. On average, you will never wander very far from your point of departure. Now, change the rules by disallowing backward steps whenever you have returned to your starting point; the new rules now model our computer system and you will find that you wander farther and farther away from your departure point.

From our solution for the probabilities, we can compute the average number of jobs in the system; it is given simply by

Again, the infinite sum converges for ; we can easily show that we have

so that the average number of jobs in the system is

Notice that, as expected, this number grows infinitely large as approaches 1.

In most systems, however, our assumption of an unbounded queue of processes is unrealistic. Most systems have bounded queues and buffers and will simply reject any new process when their queue or buffer is full. Assume that we have a queue of size k and that any arrival when in state is simply discarded. How do our previous derivations change? Well, state , like state , has a simplified equation; but otherwise, all of our equations hold, with the sum taken from from i=0 to i=k rather than to . Elementary discrete math tells us that

(It no longer matters that may exceed 1, since extra processes, instead of enlarging the queue, just get discarded.) Plugging that value into our equation for , we get

Notice that, as expected, this formula reduces to our earlier one if we assume and take the limit for . Likewise, we can compute the average number of process in the system, call it :

Discrete mathematics tells us that the finite sum can be reduced to

from which we get our expected number of processes in the system

Notice again that this formula reduces to that for if we assume and take the limit for .

Knowing both and , we can find out the fraction of processes that are discarded, simply by computing (for )

Notice that, as expected, the percentage of discarded processes decreases as k increases; in particular, in the limit as k grows to infinity, no process is discarded ( ).

Another useful value we want to know is the average time spent in the system by a job, call it W. One reasonable guesstimate would be the product of the average number of jobs in the system, , by the time required for completion of a job, (because the system completes jobs per unit time); thus we would write . Actually, while reasonable, this is not the correct derivation. The formal derivation, unfortunately, is very complex, unlike what we have done so far; on the other hand, the result is just as simple: the correct formula, known as Little's law, is

Thus, it is the arrival rate, not the departure rate, that we need. If you look at the formula, it does make intuitive sense: the number of jobs in the system should indeed be the ratio of the time spent in the system by a job to the time between new arrivals.