net.sourceforge.jabm.learning
Class StatelessQLearner

java.lang.Object
  net.sourceforge.jabm.learning.AbstractLearner
      net.sourceforge.jabm.learning.StatelessQLearner

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, DiscreteLearner, Learner, StimuliResponseLearner, Prototypeable, Resetable

public class StatelessQLearner
extends AbstractLearner
implements StimuliResponseLearner, Resetable, java.io.Serializable, Prototypeable

A memory-less version of the Q-Learning algorithm. This class implements StimuliResponseLearner instead of MDPLearner, and so can be used in place of, e.g. a RothErevLearner. We use the standard MDP QLearner class, but fool it with this wrapper into thinking that there is only one state.

See Also:

Serialized Form

Field Summary

Fields inherited from class net.sourceforge.jabm.learning.AbstractLearner

monitor

Constructor Summary

StatelessQLearner(int numActions, double learningRate, double discountRate, cern.jet.random.engine.RandomEngine prng)

StatelessQLearner(int numActions, cern.jet.random.engine.RandomEngine prng)

StatelessQLearner(cern.jet.random.engine.RandomEngine prng)

Method Summary

int	act() Request that the learner perform an action.
int	bestAction()
void	dumpState(DataWriter out) Write out our state data to the specified data writer.
ActionSelector	getActionSelector()
double	getDiscountRate()
double	getInitialQValue()
int	getLastActionChosen()
double	getLearningDelta() Return a value indicative of the amount of learning that occured during the last iteration.
double	getLearningRate()
int	getNumActions()
int	getNumberOfActions() Get the number of different possible actions this learner can choose from when it performs an action.
int	getPreviousState()
cern.jet.random.engine.RandomEngine	getPrng()
QLearner	getqLearner()
int	getState()
void	initialise()
double	maxQ(int newState)
java.lang.Object	protoClone()
void	reset() Reinitialise our state to the original settings.
void	reward(double reward) Reward the learning algorithm according to the last action it chose.
void	setActionSelector(ActionSelector actionSelector)
void	setDiscountRate(double discountRate)
void	setInitialQValue(double initialQ)
void	setLearningRate(double learningRate)
void	setNumberOfActions(int n)
void	setqLearner(QLearner qLearner)
java.lang.String	toString()
int	worstAction()

Methods inherited from class net.sourceforge.jabm.learning.AbstractLearner

monitor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface net.sourceforge.jabm.learning.Learner

monitor

Constructor Detail

StatelessQLearner

public StatelessQLearner(cern.jet.random.engine.RandomEngine prng)

StatelessQLearner

public StatelessQLearner(int numActions,
                         double learningRate,
                         double discountRate,
                         cern.jet.random.engine.RandomEngine prng)

StatelessQLearner

public StatelessQLearner(int numActions,
                         cern.jet.random.engine.RandomEngine prng)

Method Detail

act

public int act()

Description copied from interface: DiscreteLearner

Request that the learner perform an action. Users of the learning algorithm should invoke this method on the learner when they wish to find out which action the learner is currently recommending.

Specified by:

act in interface DiscreteLearner

Returns:

An integer representing the action to be taken.

bestAction

public int bestAction()

Specified by:

bestAction in interface StimuliResponseLearner

worstAction

public int worstAction()

Specified by:

worstAction in interface StimuliResponseLearner

getDiscountRate

public double getDiscountRate()

getLastActionChosen

public int getLastActionChosen()

getLearningRate

public double getLearningRate()

getNumActions

public int getNumActions()

getPreviousState

public int getPreviousState()

getPrng

public cern.jet.random.engine.RandomEngine getPrng()

getState

public int getState()

initialise

public void initialise()

maxQ

public double maxQ(int newState)

setDiscountRate

public void setDiscountRate(double discountRate)

setLearningRate

public void setLearningRate(double learningRate)

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

reward

public void reward(double reward)

Description copied from interface: StimuliResponseLearner

Reward the learning algorithm according to the last action it chose.

Specified by:

reward in interface StimuliResponseLearner

reset

public void reset()

Description copied from interface: Resetable

Reinitialise our state to the original settings.

Specified by:

reset in interface Resetable

getLearningDelta

public double getLearningDelta()

Description copied from interface: Learner

Return a value indicative of the amount of learning that occured during the last iteration. Values close to 0.0 indicate that the learner has converged to an equilibrium state.

Specified by:

getLearningDelta in interface Learner

Specified by:

getLearningDelta in class AbstractLearner

Returns:

A double representing the amount of learning that occured.

getNumberOfActions

public int getNumberOfActions()

Description copied from interface: DiscreteLearner

Get the number of different possible actions this learner can choose from when it performs an action.

Specified by:

getNumberOfActions in interface DiscreteLearner

Returns:

An integer value representing the number of actions available.

setNumberOfActions

public void setNumberOfActions(int n)

getqLearner

public QLearner getqLearner()

setqLearner

public void setqLearner(QLearner qLearner)

getActionSelector

public ActionSelector getActionSelector()

setActionSelector

public void setActionSelector(ActionSelector actionSelector)

dumpState

public void dumpState(DataWriter out)

Description copied from interface: Learner

Write out our state data to the specified data writer.

Specified by:

dumpState in interface Learner

Specified by:

dumpState in class AbstractLearner

protoClone

public java.lang.Object protoClone()

Specified by:

protoClone in interface Prototypeable

setInitialQValue

public void setInitialQValue(double initialQ)

getInitialQValue

public double getInitialQValue()