net.sourceforge.jabm.learning
Class RothErevLearner

java.lang.Object
  net.sourceforge.jabm.learning.AbstractLearner
      net.sourceforge.jabm.learning.RothErevLearner

All Implemented Interfaces:

java.io.Serializable, DiscreteLearner, Learner, StimuliResponseLearner

Direct Known Subclasses:

NPTRothErevLearner

public class RothErevLearner
extends AbstractLearner
implements StimuliResponseLearner, java.io.Serializable

A class implementing the Roth-Erev learning algorithm. This learning algorithm is designed to mimic human-like behaviour in extensive form games. See:

A.E.Roth and I. Erev "Learning in extensive form games: experimental data and simple dynamic models in the intermediate term" Games and Economic Behiour, Volume 8

See Also:

Serialized Form

Field Summary

protected double	deltaP The total amount of update to the probability vector on the last iteration.
protected double	e The experimentation parameter.
protected int	iteration The current iteration.
protected int	k The number of choices available to make at each iteration.
protected int	lastAction The last action chosen.
protected DiscreteProbabilityDistribution	probabilities Probabilities for each possible action.
protected double[]	q Propensity for each possible action.
protected double	r The recency parameter.
protected double	s1 The scaling parameter.

Fields inherited from class net.sourceforge.jabm.learning.AbstractLearner

monitor

Constructor Summary

RothErevLearner(int k, double r, double e, double s1, cern.jet.random.engine.RandomEngine prng)
Construct a new learner.

RothErevLearner(int k, cern.jet.random.engine.RandomEngine prng)

RothErevLearner(int k, cern.jet.random.engine.RandomEngine prng, double[] propensities)

RothErevLearner(cern.jet.random.engine.RandomEngine prng)

Method Summary

int	act() Generate the next action for this learner.
int	bestAction()
int	choose() Choose a random number according to the probability distribution defined by the probabilities.
void	dumpState(DataWriter out) Write out our state data to the specified data writer.
double	experience(int i, int action, double reward) The experience function
double	getE()
int	getIteration()
int	getK() Get the total number of actions.
int	getLastAction()
double	getLearningDelta() Return a value indicative of the amount of learning that occured during the last iteration.
int	getNumberOfActions() Get the total number of actions
DiscreteProbabilityDistribution	getProbabilities() Get the probability distribution corresponding to the current propensities.
double	getProbability(int i) Get the probability of the ith action.
double	getR()
double	getS1()
java.lang.Object	protoClone()
void	resetPropensities()
void	reward(double reward) Reward the last action taken by the learner according to some payoff.
void	setExperimentation(double e)
void	setPropensities(double[] q) Replace the current propensities with the supplied propensity array.
void	setRecency(double r)
void	setScaling(double s1)
java.lang.String	toString()
protected void	updateProbabilities() Update the probabilities from the propensities.
protected void	updatePropensities(int action, double reward) Update the propensities for each possible action.
protected void	validateParams()
int	worstAction()

Methods inherited from class net.sourceforge.jabm.learning.AbstractLearner

monitor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface net.sourceforge.jabm.learning.Learner

monitor

Field Detail

k

protected int k

The number of choices available to make at each iteration.

r

protected double r

The recency parameter.

e

protected double e

The experimentation parameter.

s1

protected double s1

The scaling parameter.

q

protected double[] q

Propensity for each possible action.

probabilities

protected DiscreteProbabilityDistribution probabilities

Probabilities for each possible action.

iteration

protected int iteration

The current iteration.

lastAction

protected int lastAction

The last action chosen.

deltaP

protected double deltaP

The total amount of update to the probability vector on the last iteration.

Constructor Detail

RothErevLearner

public RothErevLearner(int k,
                       double r,
                       double e,
                       double s1,
                       cern.jet.random.engine.RandomEngine prng)

Construct a new learner.

Parameters:

k - The no. of possible actions.

r - The recency parameter.

e - The experimentation parameter.

RothErevLearner

public RothErevLearner(cern.jet.random.engine.RandomEngine prng)

RothErevLearner

public RothErevLearner(int k,
                       cern.jet.random.engine.RandomEngine prng)

RothErevLearner

public RothErevLearner(int k,
                       cern.jet.random.engine.RandomEngine prng,
                       double[] propensities)

Method Detail

protoClone

public java.lang.Object protoClone()

validateParams

protected void validateParams()

act

public int act()

Generate the next action for this learner.

Specified by:

act in interface DiscreteLearner

Returns:

An int in the range [0..k) representing the choice made by the learner.

reward

public void reward(double reward)

Reward the last action taken by the learner according to some payoff.

Specified by:

reward in interface StimuliResponseLearner

Parameters:

reward - The payoff for the last action taken by the learner.

choose

public int choose()

Choose a random number according to the probability distribution defined by the probabilities.

Returns:

one of [0..k) according to the probabilities [0..k-1].

updatePropensities

protected void updatePropensities(int action,
                                  double reward)

Update the propensities for each possible action.

Parameters:

action - The last action chosen by the learner

updateProbabilities

protected void updateProbabilities()

Update the probabilities from the propensities.

experience

public double experience(int i,
                         int action,
                         double reward)

The experience function

Parameters:

i - The action under consideration

action - The last action chosen

setPropensities

public void setPropensities(double[] q)

Replace the current propensities with the supplied propensity array.

Parameters:

q - The new propensity array to use.

resetPropensities

public void resetPropensities()

setRecency

public void setRecency(double r)

setExperimentation

public void setExperimentation(double e)

setScaling

public void setScaling(double s1)

dumpState

public void dumpState(DataWriter out)

Description copied from interface: Learner

Write out our state data to the specified data writer.

Specified by:

dumpState in interface Learner

Specified by:

dumpState in class AbstractLearner

getK

public int getK()

Get the total number of actions.

getNumberOfActions

public int getNumberOfActions()

Get the total number of actions

Specified by:

getNumberOfActions in interface DiscreteLearner

Returns:

An integer value representing the number of actions available.

getLearningDelta

public double getLearningDelta()

Description copied from interface: Learner

Return a value indicative of the amount of learning that occured during the last iteration. Values close to 0.0 indicate that the learner has converged to an equilibrium state.

Specified by:

getLearningDelta in interface Learner

Specified by:

getLearningDelta in class AbstractLearner

Returns:

A double representing the amount of learning that occured.

getProbability

public double getProbability(int i)

Get the probability of the ith action.

getProbabilities

public DiscreteProbabilityDistribution getProbabilities()

Get the probability distribution corresponding to the current propensities.

getE

public double getE()

getIteration

public int getIteration()

getLastAction

public int getLastAction()

getR

public double getR()

getS1

public double getS1()

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

bestAction

public int bestAction()

Specified by:

bestAction in interface StimuliResponseLearner

worstAction

public int worstAction()

Specified by:

worstAction in interface StimuliResponseLearner