--OO-- Magnus Norgaard's NEURAL NETWORK BASED SYSTEM IDENTIFICATION TOOLBOX Version 1.0 Institute of Automation Building 326 2800 Lyngby, Denmark Technical University of Denmark --OO-- The toolbox contains a number of m-files for training and evaluation of multi-layer perceptron type neural networks. There are functions for working ordinary feed-forward networks as well as for identification of nonlinear dynamic systems and time-series analysis. Not all functions have been thoroughly tested yet and even if I haven't been able to find any bugs, I can't guarantee that you won't encounter any problems. Below an overview of the files contained in this directory is given along with a brief description of what they do. The on-line help facility explains how to call the different functions. You simply write help in the MATLAB command window. Along with the m- and c-files in this directory you will find a manual for the toolbox. Start by printing this out and read the release notes. A number of simple demonstration programs have been made to illustrate how most of the functions work. Run these to get an idea of what the toolbox provides. Be careful with the demos, though. Due to the existence of several local minimas, the results will often vary a great deal from run to run. Enjoy! - MN --OO-- TO COME (HOPEFULLY): - Improved stop criterion. I regret that I didn't include the normed length of the gradient as a stop criterion at an early stage. Now it seems too overwhelming to include. - Improve check for invalid function calls. - Improved validation (higher order correlation functions, for example). - Adaptive regularization. --OO-- CURRENTLY WORKING FUNCTIONS IN THIS DIRECTORY: --------------------------------------------- FUNCTIONS FOR TRAINING NETWORKS: batbp : Batch version of the back-propagation algorithm. incbp : Recursive (/incremental) version of back-propagation. marq : Levenberg-Marquardt method. marq2 : Levenberg-Marquardt method. Works for fully connected networks only. marqlm : Memory-saving implementation of the Levenberg-Marquardt method. rpe : Recursive prediction error method. FUNCTIONS FOR PRETREATING THE DATA: dscale : Scale data to zero mean and variance one. FUNCTIONS FOR TRAINING NETWORKS TO MODEL DYNAMIC SYSTEMS: lipschit : Determine the lag space. nnarmax1 : Identify a Neural Network ARMAX (or ARMA) model (Linear MA filter). nnarmax2 : Identify a Neural Network ARMAX (or ARMA) model. nnarx : Identify a Neural Network ARX (or AR) model. nniol : Identify a Neural Network model suited for I-O linearization control. nnoe : Identify a Neural Network Output Error model. nnrarmx1 : Recursive counterpart to NNARMAX1. nnrarmx2 : Recursive counterpart to NNARMAX2. nnrarx : Recursive counterpart to NNARX. nnssif : Identify a NN State Space Innovations form model. FUNCTIONS FOR PRUNING NETWORKS: netstruc : Extract weight matrices from matrix of parameter vectors. nnprune : Prune models of dynamic systems with Optimal Brain Surgeon (OBS). obdprune : Prune feed-forward networks with Optimal Brain Damage (OBD). obsprune : Prune feed-forward networks with Optimal Brain Surgeon (OBS). FUNCTIONS FOR EVALUATING TRAINED NETWORKS: fpe : FPE estimate of the generalization error for feed-forward nets. ifvalid : Validation of models generated by NNSSIF. ioleval : Validation of models generated by NNIOL. loo : Leave-One-Out estimate of generalization error for feed-forward nets. nneval : Validation of feed-forward networks (trained by marq,rpe,bp). nnfpe : FPE for I/O models of dynamic systems. nnsimul : Simulate model of dynamic system from control sequence alone. nnvalid : Validation of I/O models of dynamic systems. wrescale : Rescale weights of trained network. MISCELLANOUS FUNCTIONS: README : This file. RELEASE : Release notes. Contents : Contents file. drawnet : Draws a two layer neural network. getgrad : Derivative of network outputs w.r.t. the weights. pmntanh : Fast tanh function. DEMOS: test1 : Demonstrates different training methods on a curve fitting example. test2 : Demonstrates the NNARX function. test3 : Demonstrates the NNARMAX2 function. test4 : Demonstrates the NNSSIF function. test5 : Demonstrates the NNOE function. test6 : Demonstrates the effect of regularization by weight decay. test7 : Demonstrates pruning by OBS on the sunspot benchmark problem. OTHER FILES IN DIRECTORY pmnshow, test6mat, test7mat, and solplet.asc are used by the test programs. manual1.ps, manual2.ps contains the manual for the toolbox in Postscript format. Makefile, marq.c, nnarmax2.c, nnoe.c, nnssif.c, matrix.c, matrix2.h, nnmisc.c, nnmisc.h contains the source code necessary for generating CMEX versions of the functions marq, nnarmax2, nnoe, and nnssif.