Agent Dynamics

The term ``dynamics'' refers to the behavior of a system in response to inputs or control. In this case, the ``system'' is the environment and the AGENT's STATE. The ``control'' is the ACTION that the AGENT chooses at each step. Therefore, dynamics, for JRoboExplorer, is how the AGENT's STATE changes in response to its ACTIONs.

In principle, dynamics are a function of both AGENT and environment. For example, a legged robot will have very different dynamics in mud versus gravel, while the legged robot and a wheeled robot will have very different dynamics in mud. This project only requires one AGENT, however, so it is sufficient to think of dynamics as being purely a function of the environment. The designer MAY provide additional AGENTs and specialize dynamics by AGENT as well.

Figure: Example of an AGENT in an outdoor environment. The terrain types of the pictured cells are, from upper left to lower right, Rock, Bush, Grass, Mud, Bush, Bush, Grass, Grass, Rock. The AGENT's current ORIENTATION is NorthEast. (Note that this figure is in color and may be easier to interpret on the screen than on paper.)

[width=0.3]pics/outdoor_example

No AGENT is perfect - small uncertainties in the environment, measurement errors, noise in the control lines, imperfections in motors and wheels - all of these contribute to making the results of an AGENT's ACTIONs unpredictable or ``noisy''. Mathematically, this noise is modeled with probabilities. For example, in Figure , the AGENT is standing in Bushand facing NorthEast toward a Grasscell. If the AGENT attempts to move FORWARD, it might end up in the Grasscell straight in front of it with probability 0.4, the Bushcell to its North with probability 0.1, the cell to its East with probability 0.1, and stay where it is with probability 0.4 ( $=1.0-(0.4+0.1+0.1)$). (These probabilities are for example only and do not correspond to the real dynamics parameters given below.)

The job of the WORLD SIMULATOR is to decide what the probabilities of various outcomes are and pick one with the appropriate probability. For example, in the scenario above, the WORLD SIMULATOR might assess the four possible outcome cells and generate the cumulative probability distribution of Table . It then picks a double uniformly at random from the range $[0,1]$. Suppose it picks the value $r=0.5731824$. The resulting LOCATION that the WORLD SIMULATOR will generate for the AGENT's next STATE is then North, because $P[\mathtt{East}]<r\leq P[\mathtt{North}]$.

NOTE: This exampled was phrased in terms of generating and testing the CDF explicitly, because it is simpler to discuss and understand that way. That is not necessarily the best way to implement the simulator.

Table: Cumulative probability distribution for scenario of Figure .

Outcome	Probability (PDF)	Cum. Prob (CDF)
NorthEast	0.4	0.4
East	0.1	0.5
North	0.1	0.6
Unchanged	0.4	1.0