The abstract network type upon which all others are based. It's not pure virtual, in that it encapsulates some high level operations (such as the top-level training algorithm). More...

#include <net.hpp>

Inheritance diagram for Net:

[legend]

Classes
struct	SGDParams
	Training parameters for trainSGD(). This structure holds the parameters for the trainSGD() method, and serves as a better way of passing them than a long parameter list. All values have defaults set up by the constructor, which are given as constants. You can set parameters by hand, but there are fluent (chainable) setters for many members. More...

Public Member Functions
virtual	~Net ()
	virtual destructor which does nothing More...

void	setSeed (long seed)
	Set this network's random number generator, which is used for weight initialisation done at the start of training. More...

virtual int	getLayerSize (int n) const =0
	Get the number of nodes in a given layer. More...

virtual int	getLayerCount () const =0
	Get the number of layers. More...

int	getInputCount () const
	get the number of inputs More...

int	getOutputCount () const
	get the number of outputs More...

virtual void	setInputs (double *d)=0
	Set the inputs to the network before running or training. More...

virtual double *	getOutputs () const =0
	Get the outputs after running. More...

double *	run (double *in)
	Run the network on some data. More...

virtual void	setH (double h)=0
	Set the modulator level for subsequent runs and training of this network. More...

virtual double	getH () const =0
	get the modulator level More...

double	test (ExampleSet &examples, int start=0, int num=-1)
	Test a network. Runs the network over a set of examples and returns the mean MSE for all outputs $\frac{1}{N\cdot N_{outs}}\sum^N_{e \in Examples} \sum_{i=0}^{N_{outs}} (e_o(i) - e_y(i))^2$ where is the number of examples, $N_{outs}$ is the number of outputs, is network's output for example , and is the desired output for the same example. More...

double	trainSGD (ExampleSet &examples, SGDParams &params)
	Train using stochastic gradient descent. Note that cross-validation parameters are slightly different from those given in the thesis. Here we give the number of slices and number of examples per slice; in the thesis we give the total number of examples to be held out and the number of slices. More...

virtual int	getDataSize () const =0
	Get the length of the serialised data block for this network. More...

virtual void	save (double *buf) const =0
	Serialize the data (not including any network type magic number or layer/node counts) to the given memory (which must be of sufficient size). More...

virtual void	load (double *buf)=0
	Given that the pointer points to a data block of the correct size for the current network, copy the parameters from that data block into the current network overwriting the current parameters. More...

Public Attributes
NetType	type
	type of the network, used for load/save More...

drand48_data	rd
	PRNG data (thread safe) More...

Protected Member Functions
virtual void	update ()=0
	Run a single update of the network. More...

	Net (NetType tp)
	Constructor - protected because others inherit it and it's not used directly. More...

double	drand (double mn, double mx)
	get a random number using this net's PRNG data More...

virtual void	initWeights (double initr)=0
	initialise weights to random values More...

virtual double	trainBatch (ExampleSet &ex, int start, int num, double eta)=0
	Train a network for batch (or mini-batch) (or single example). More...

Friends
class	OutputBlendingNet

class	HInputNet

Detailed Description

The abstract network type upon which all others are based. It's not pure virtual, in that it encapsulates some high level operations (such as the top-level training algorithm).

Definition at line 39 of file net.hpp.

Constructor & Destructor Documentation

virtual Net::~Net ( )

inlinevirtual

virtual destructor which does nothing

Definition at line 47 of file net.hpp.

Net::Net ( NetType tp )

inlineprotected

Constructor - protected because others inherit it and it's not used directly.

Parameters

tp	network type enumeration

Definition at line 580 of file net.hpp.

Member Function Documentation

double Net::drand	(	double	mn,
		double	mx
	)

inlineprotected

get a random number using this net's PRNG data

Parameters

mn	minimum value (inclusive)
mx	maximum value (inclusive)

Definition at line 591 of file net.hpp.

virtual int Net::getDataSize ( ) const

pure virtual

Get the length of the serialised data block for this network.

Returns: the size in doubles

Implemented in BPNet, and OutputBlendingNet.

virtual double Net::getH ( ) const

pure virtual

get the modulator level

Implemented in BPNet, HInputNet, OutputBlendingNet, and UESNet.

int Net::getInputCount ( ) const

inline

get the number of inputs

Definition at line 75 of file net.hpp.

virtual int Net::getLayerCount ( ) const

pure virtual

Get the number of layers.

Implemented in BPNet, and OutputBlendingNet.

virtual int Net::getLayerSize ( int n ) const

pure virtual

Get the number of nodes in a given layer.

Parameters

n	layer number

Implemented in BPNet, HInputNet, and OutputBlendingNet.

int Net::getOutputCount ( ) const

inline

get the number of outputs

Definition at line 82 of file net.hpp.

virtual double* Net::getOutputs ( ) const

pure virtual

Get the outputs after running.

Returns: pointer to the output layer outputs

Implemented in BPNet, and OutputBlendingNet.

virtual void Net::initWeights ( double initr )

protectedpure virtual

initialise weights to random values

Parameters

initr range of weights [-n,n], or -1 for Bishop's rule.

Implemented in BPNet, and OutputBlendingNet.

virtual void Net::load ( double * buf )

pure virtual

Given that the pointer points to a data block of the correct size for the current network, copy the parameters from that data block into the current network overwriting the current parameters.

Parameters

buf	the buffer to load the data from, must be at least getDataSize() doubles

Implemented in BPNet, and OutputBlendingNet.

double* Net::run ( double * in )

inline

Run the network on some data.

Parameters

in	pointer to the input double array

Returns: pointer to the output double array

Definition at line 107 of file net.hpp.

virtual void Net::save ( double * buf ) const

pure virtual

Serialize the data (not including any network type magic number or layer/node counts) to the given memory (which must be of sufficient size).

Parameters

buf	the buffer to save the data, must be at least getDataSize() doubles

Implemented in BPNet, and OutputBlendingNet.

virtual void Net::setH ( double h )

pure virtual

Set the modulator level for subsequent runs and training of this network.

Implemented in BPNet, HInputNet, OutputBlendingNet, and UESNet.

virtual void Net::setInputs ( double * d )

pure virtual

Set the inputs to the network before running or training.

Parameters

d	array of doubles, the size of the input layer

Implemented in BPNet, HInputNet, and OutputBlendingNet.

void Net::setSeed ( long seed )

inline

Set this network's random number generator, which is used for weight initialisation done at the start of training.

Definition at line 57 of file net.hpp.

double Net::test	(	ExampleSet &	examples,
		int	start = `0`,
		int	num = `-1`
	)

inline

Test a network. Runs the network over a set of examples and returns the mean MSE for all outputs

$\frac{1}{N\cdot N_{outs}}\sum^N_{e \in Examples} \sum_{i=0}^{N_{outs}} (e_o(i) - e_y(i))^2$

where $N$ is the number of examples, $N_{outs}$ is the number of outputs, $e_o(i)$ is network's output for example $e$ , and $e_y(i)$ is the desired output for the same example.

Parameters

examples	Example set to test (or partially test).
start	index of example to start test at.
num	number of examples to test (or -1 for all after start point).

Definition at line 142 of file net.hpp.

virtual double Net::trainBatch	(	ExampleSet &	ex,
		int	start,
		int	num,
		double	eta
	)

protectedpure virtual

Train a network for batch (or mini-batch) (or single example).

This will

zero the average gradient variables for all weights and biases
zero the total error
for each example
- calculate the error with calcError() which itself calls update()
- add to the total mean squared error (see NOTE below)
for each weight and bias
- calculate the means across all provided examples
- apply the mean to the weight or bias
return the mean squared error (NOTE: different from original, which returned mean absolute error) for all outputs and examples:
$\frac{1}{N\cdot N_{outs}}\sum^N_{e \in Examples} \sum_{i=0}^{N_{outs}} (e_o(i) - e_y(i))^2$
where is the number of examples, $N_{outs}$ is the number of outputs, is network's output for example , and is the desired output for the same example.
Parameters

ex example set

start index of first example to use

num number of examples. For a single example, you'd just use 1.

eta learning rate

Returns
the sum of mean squared errors in the output layer (see formula in method documentation)

Implemented in BPNet, OutputBlendingNet, and UESNet.

double Net::trainSGD	(	ExampleSet &	examples,
		SGDParams &	params
	)

inline

Train using stochastic gradient descent. Note that cross-validation parameters are slightly different from those given in the thesis. Here we give the number of slices and number of examples per slice; in the thesis we give the total number of examples to be held out and the number of slices.

Precondition: Network has weights initialised to random values

Postcondition: The network will be set to the best network found if bestNetBuffer is set, otherwise the final network will be used.

Exceptions

std::out_of_range	Too many CV examples
std::logic_error	Trying to select best by CV when there's no CV done

Parameters

examples	training set (including cross-validation data)
params	a filled-in SGDParams structure giving the parameters for the training.

Returns: If storeBestNet is null, the MSE of the final network; otherwise the MSE of the best network found. This is done across the entire validation set if provided, or the entire training set if not.

Definition at line 428 of file net.hpp.

virtual void Net::update ( )

protectedpure virtual

Run a single update of the network.

Precondition: input layer must be filled with values

Postcondition: output layer contains result

Implemented in BPNet, OutputBlendingNet, and UESNet.

Friends And Related Function Documentation

friend class HInputNet

friend

Definition at line 41 of file net.hpp.

friend class OutputBlendingNet

friend

Definition at line 40 of file net.hpp.

Member Data Documentation

drand48_data Net::rd

PRNG data (thread safe)

Definition at line 50 of file net.hpp.

NetType Net::type

type of the network, used for load/save

Definition at line 49 of file net.hpp.

The documentation for this class was generated from the following file:

/home/travis/build/jimfinnis/uesmanncpp/net.hpp

ex	example set
start	index of first example to use
num	number of examples. For a single example, you'd just use 1.
eta	learning rate

Classes

Public Member Functions

Public Attributes

Protected Member Functions

Friends

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Friends And Related Function Documentation

Member Data Documentation