The "basic" back-propagation network using a logistic sigmoid, as described by Rumelhart, Hinton and Williams (and many others). This class is used by output blending and h-as-input networks. More...

#include <bpnet.hpp>

Inheritance diagram for BPNet:

[legend]

Collaboration diagram for BPNet:

[legend]

Public Member Functions
	BPNet (int nlayers, const int *layerCounts)
	Constructor - does not initialise the weights to random values so that we can reinitialise networks. More...

virtual void	setH (double h)
	Set the modulator level for subsequent runs and training of this network. More...

virtual double	getH () const
	get the modulator level More...

virtual	~BPNet ()
	destructor More...

virtual void	setInputs (double *d)
	Set the inputs to the network before running or training. More...

void	setInput (int n, double d)
	Used to set inputs manually, typically in HInputNet. More...

virtual double *	getOutputs () const
	Get the outputs after running. More...

virtual int	getLayerSize (int n) const
	Get the number of nodes in a given layer. More...

virtual int	getLayerCount () const
	Get the number of layers. More...

virtual int	getDataSize () const
	Get the length of the serialised data block for this network. More...

virtual void	save (double *buf) const
	Serialize the data (not including any network type magic number or layer/node counts) to the given memory (which must be of sufficient size). More...

virtual void	load (double *buf)
	Given that the pointer points to a data block of the correct size for the current network, copy the parameters from that data block into the current network overwriting the current parameters. More...

Public Member Functions inherited from Net
virtual	~Net ()
	virtual destructor which does nothing More...

void	setSeed (long seed)
	Set this network's random number generator, which is used for weight initialisation done at the start of training. More...

int	getInputCount () const
	get the number of inputs More...

int	getOutputCount () const
	get the number of outputs More...

double *	run (double *in)
	Run the network on some data. More...

double	test (ExampleSet &examples, int start=0, int num=-1)
	Test a network. Runs the network over a set of examples and returns the mean MSE for all outputs $\frac{1}{N\cdot N_{outs}}\sum^N_{e \in Examples} \sum_{i=0}^{N_{outs}} (e_o(i) - e_y(i))^2$ where is the number of examples, $N_{outs}$ is the number of outputs, is network's output for example , and is the desired output for the same example. More...

double	trainSGD (ExampleSet &examples, SGDParams &params)
	Train using stochastic gradient descent. Note that cross-validation parameters are slightly different from those given in the thesis. Here we give the number of slices and number of examples per slice; in the thesis we give the total number of examples to be held out and the number of slices. More...

Protected Member Functions
	BPNet ()
	Special constructor for subclasses which need to manipulate layer count before initialisation (e.g. HInputNet). More...

void	init (int nlayers, const int *layerCounts)
	Initialiser for use by the main constructor and the ctors of those subclasses mentioned in BPNet() More...

virtual void	initWeights (double initr)
	initialise weights to random values More...

double &	getw (int tolayer, int toneuron, int fromneuron) const
	get the value of a weight. More...

double &	getb (int layer, int neuron) const
	get the value of a bias More...

double &	getavggradw (int tolayer, int toneuron, int fromneuron) const
	get the value of the gradient for a given weight More...

double	getavggradb (int l, int n) const
	get the value of a bias gradient More...

void	calcError (double in, double out)
	run a single example and calculate the errors; used in training. More...

virtual void	update ()
	Run a single update of the network. More...

virtual double	trainBatch (ExampleSet &ex, int start, int num, double eta)
	Train a network for batch (or mini-batch) (or single example). More...

Protected Member Functions inherited from Net
	Net (NetType tp)
	Constructor - protected because others inherit it and it's not used directly. More...

double	drand (double mn, double mx)
	get a random number using this net's PRNG data More...

Protected Attributes
int	numLayers
	number of layers, including input and output More...

int *	layerSizes
	array of layer sizes More...

int	largestLayerSize
	number of nodes in largest layer More...

double **	weights
	Array of weights as [tolayer][tonode+largestLayerSize*fromnode]. More...

double **	biases
	array of biases, stored as a rectangular array of [layer][node] More...

double **	outputs
	outputs of each layer: one array of doubles for each More...

double **	errors
	the error for each node, calculated by calcError() More...

double **	gradAvgsWeights
	average gradient for each weight (built during training) More...

double **	gradAvgsBiases
	average gradient for each bias (built during training) More...

Additional Inherited Members
Public Attributes inherited from Net
NetType	type
	type of the network, used for load/save More...

drand48_data	rd
	PRNG data (thread safe) More...

Detailed Description

The "basic" back-propagation network using a logistic sigmoid, as described by Rumelhart, Hinton and Williams (and many others). This class is used by output blending and h-as-input networks.

Definition at line 18 of file bpnet.hpp.

Constructor & Destructor Documentation

BPNet::BPNet ( )

inlineprotected

Special constructor for subclasses which need to manipulate layer count before initialisation (e.g. HInputNet).

Definition at line 24 of file bpnet.hpp.

BPNet::BPNet	(	int	nlayers,
		const int *	layerCounts
	)

inline

Constructor - does not initialise the weights to random values so that we can reinitialise networks.

Parameters

nlayers	number of layers
layerCounts	array of layer counts

Definition at line 69 of file bpnet.hpp.

virtual BPNet::~BPNet ( )

inlinevirtual

destructor

Definition at line 86 of file bpnet.hpp.

Member Function Documentation

void BPNet::calcError	(	double *	in,
		double *	out
	)

inlineprotected

run a single example and calculate the errors; used in training.

Parameters

in	inputs
out	required outputs

Postcondition: the errors will be in the errors variable

Definition at line 294 of file bpnet.hpp.

double BPNet::getavggradb	(	int	l,
		int	n
	)		const

inlineprotected

get the value of a bias gradient

Precondition: gradients must have been calculated as part of training step

Parameters

l	index of layer
n	index of neuron within layer

Definition at line 283 of file bpnet.hpp.

double& BPNet::getavggradw	(	int	tolayer,
		int	toneuron,
		int	fromneuron
	)		const

inlineprotected

get the value of the gradient for a given weight

Precondition: gradients must have been calculated as part of training step

Parameters

tolayer	the layer of the destination node (from is assumed to be previous layer)
toneuron	the index of the destination node in that layer
fromneuron	the index of the source node

Definition at line 272 of file bpnet.hpp.

double& BPNet::getb	(	int	layer,
		int	neuron
	)		const

inlineprotected

get the value of a bias

Parameters

layer	index of layer
neuron	index of neuron within layer

Definition at line 259 of file bpnet.hpp.

virtual int BPNet::getDataSize ( ) const

inlinevirtual

Get the length of the serialised data block for this network.

Returns: the size in doubles

Implements Net.

Definition at line 134 of file bpnet.hpp.

virtual double BPNet::getH ( ) const

inlinevirtual

get the modulator level

Implements Net.

Reimplemented in HInputNet, and UESNet.

Definition at line 77 of file bpnet.hpp.

virtual int BPNet::getLayerCount ( ) const

inlinevirtual

Get the number of layers.

Implements Net.

Definition at line 128 of file bpnet.hpp.

virtual int BPNet::getLayerSize ( int n ) const

inlinevirtual

Get the number of nodes in a given layer.

Parameters

n	layer number

Implements Net.

Reimplemented in HInputNet.

Definition at line 124 of file bpnet.hpp.

virtual double* BPNet::getOutputs ( ) const

inlinevirtual

Get the outputs after running.

Returns: pointer to the output layer outputs

Implements Net.

Definition at line 120 of file bpnet.hpp.

double& BPNet::getw	(	int	tolayer,
		int	toneuron,
		int	fromneuron
	)		const

inlineprotected

get the value of a weight.

Parameters

tolayer	the layer of the destination node (from is assumed to be previous layer)
toneuron	the index of the destination node in that layer
fromneuron	the index of the source node

Definition at line 249 of file bpnet.hpp.

void BPNet::init	(	int	nlayers,
		const int *	layerCounts
	)

inlineprotected

Initialiser for use by the main constructor and the ctors of those subclasses mentioned in BPNet()

Definition at line 32 of file bpnet.hpp.

virtual void BPNet::initWeights ( double initr )

inlineprotectedvirtual

initialise weights to random values

Parameters

initr range of weights [-n,n], or -1 for Bishop's rule.

Implements Net.

Definition at line 218 of file bpnet.hpp.

virtual void BPNet::load ( double * buf )

inlinevirtual

Given that the pointer points to a data block of the correct size for the current network, copy the parameters from that data block into the current network overwriting the current parameters.

Parameters

buf	the buffer to load the data from, must be at least getDataSize() doubles

Implements Net.

Definition at line 170 of file bpnet.hpp.

virtual void BPNet::save ( double * buf ) const

inlinevirtual

Serialize the data (not including any network type magic number or layer/node counts) to the given memory (which must be of sufficient size).

Parameters

buf	the buffer to save the data, must be at least getDataSize() doubles

Implements Net.

Definition at line 151 of file bpnet.hpp.

virtual void BPNet::setH ( double h )

inlinevirtual

Set the modulator level for subsequent runs and training of this network.

Implements Net.

Reimplemented in HInputNet, and UESNet.

Definition at line 73 of file bpnet.hpp.

void BPNet::setInput	(	int	n,
		double	d
	)

inline

Used to set inputs manually, typically in HInputNet.

Definition at line 115 of file bpnet.hpp.

virtual void BPNet::setInputs ( double * d )

inlinevirtual

Set the inputs to the network before running or training.

Parameters

d	array of doubles, the size of the input layer

Implements Net.

Reimplemented in HInputNet.

Definition at line 104 of file bpnet.hpp.

virtual double BPNet::trainBatch	(	ExampleSet &	ex,
		int	start,
		int	num,
		double	eta
	)

inlineprotectedvirtual

Train a network for batch (or mini-batch) (or single example).

This will

zero the average gradient variables for all weights and biases
zero the total error
for each example
- calculate the error with calcError() which itself calls update()
- add to the total mean squared error (see NOTE below)
for each weight and bias
- calculate the means across all provided examples
- apply the mean to the weight or bias
return the mean squared error (NOTE: different from original, which returned mean absolute error) for all outputs and examples:
$\frac{1}{N\cdot N_{outs}}\sum^N_{e \in Examples} \sum_{i=0}^{N_{outs}} (e_o(i) - e_y(i))^2$
where is the number of examples, $N_{outs}$ is the number of outputs, is network's output for example , and is the desired output for the same example.
Parameters

ex example set

start index of first example to use

num number of examples. For a single example, you'd just use 1.

eta learning rate

Returns
the sum of mean squared errors in the output layer (see formula in method documentation)

Implements Net.

Reimplemented in UESNet.

Definition at line 332 of file bpnet.hpp.

virtual void BPNet::update ( )

inlineprotectedvirtual

Run a single update of the network.

Precondition: input layer must be filled with values

Postcondition: output layer contains result

Implements Net.

Reimplemented in UESNet.

Definition at line 320 of file bpnet.hpp.

Member Data Documentation

double** BPNet::biases

protected

array of biases, stored as a rectangular array of [layer][node]

Definition at line 208 of file bpnet.hpp.

double** BPNet::errors

protected

the error for each node, calculated by calcError()

Definition at line 213 of file bpnet.hpp.

double** BPNet::gradAvgsBiases

protected

average gradient for each bias (built during training)

Definition at line 216 of file bpnet.hpp.

double** BPNet::gradAvgsWeights

protected

average gradient for each weight (built during training)

Definition at line 215 of file bpnet.hpp.

int BPNet::largestLayerSize

protected

number of nodes in largest layer

Definition at line 192 of file bpnet.hpp.

int* BPNet::layerSizes

protected

array of layer sizes

Definition at line 191 of file bpnet.hpp.

int BPNet::numLayers

protected

number of layers, including input and output

Definition at line 190 of file bpnet.hpp.

double** BPNet::outputs

protected

outputs of each layer: one array of doubles for each

Definition at line 212 of file bpnet.hpp.

double** BPNet::weights

protected

Array of weights as [tolayer][tonode+largestLayerSize*fromnode].

Weights are stored as a square matrix, even though less than half is used. Less than that, if not all layers are the same size, since the dimension of the matrix must be the size of the largest layer. Each array has its own matrix, so index by [layer][i+largestLayerSize*j], where

layer is the "TO" layer
layer-1 is the FROM layer
i is the TO neuron (i.e. the end of the connection)
j is the FROM neuron (the start)

Definition at line 205 of file bpnet.hpp.

The documentation for this class was generated from the following file:

/home/travis/build/jimfinnis/uesmanncpp/bpnet.hpp

ex	example set
start	index of first example to use
num	number of examples. For a single example, you'd just use 1.
eta	learning rate

Public Member Functions

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation