## Evolving connectionist systems: Characterisation, simplification, formalisation, explanation and optimisation

dc.contributor.author | Watts, Michael John | en_NZ |

dc.date.available | 2011-04-07T03:17:02Z | |

dc.date.copyright | 2004-09-06 | en_NZ |

dc.identifier.citation | Watts, M. J. (2004, September 6). Evolving connectionist systems: Characterisation, simplification, formalisation, explanation and optimisation (Thesis, Doctor of Philosophy). Retrieved from http://hdl.handle.net/10523/1489 | en |

dc.identifier.uri | http://hdl.handle.net/10523/1489 | |

dc.description.abstract | There are several well-known problems with conventional artificial neural networks (ANN), such as difficulties with selecting the structure of the network, and problems with forgetting previously-learned knowledge after further training. Constructive neural network algorithms attempt to solve these problems, but in turn have problems of their own. The Evolving Connectionist System (ECoS) is a class of open architecture artificial neural networks that are similar in the way in which neurons are added to their structures, and in the way in which their connection weights are modified. The ECoS algorithm is intended to address the problems with constructive neural networks. Several problems with ECoS are identified and discussed in this thesis. These problems are: the lack of comparison of ECoS with constructive neural networks; the excessive complexity of the Evolving Fuzzy Neural Network (EFuNN), which is the seminal ECoS network: the lack of a testable formalisation of ECoS; the dependence on fuzzy logic elements embedded within the network for fuzzy rule extraction; and the lack of methods for optimising ECoS networks. The research in this thesis addresses these problems. The overall theme of the research can be summarised as the characterisation, simplification, formalisation, explanation and optimisation of ECoS. Characterisation in this thesis means the comparison of ECoS with existing constructive ANN. Simplification means reducing the network to a minimalist implementation. Formalisation means the creation of a testable predictive model of ECoS training. Explanation means explaining ECoS networks via the extraction of fuzzy rules. Finally, optimisation means creating ECoS networks that have a minimum number of neurons with maximum accuracy. Each of these themes is approached in ways that build upon, and are complementary to, the basic ECoS network and ECoS training algorithm. The basic ECoS structure and algorithm is left unchanged, and the problems are addressed by extending that structure, rather than altering it as has been done in other work on EcoS. The principal contributions of this thesis are: a qualitative comparison of ECoS to constructive neural network algorithms; a proposed simplified version of EFuNN called SECoS; an experimentally tested formalisation of ECoS: novel algorithms for explicating SECoS via the extraction of fuzzy rules; and several novel algorithms for the optimisation of ECoS networks. The formalisation of ECoS and the proposed algorithms are evaluated on data from a set of standard benchmarking problems. Further experiments are performed with a data set with real-world applications, namely the recognition of isolated New Zealand English phonemes. The analyses of the experimental results show that the proposed algorithms are effective across both the benchmark data sets and the case study data set. | en_NZ |

dc.format.mimetype | application/pdf | |

dc.subject | Artificial Neural Networks | en_NZ |

dc.subject | Evolutionary Algorithms | en_NZ |

dc.subject | Fuzzy Rules | en_NZ |

dc.subject | Rule Extraction | en_NZ |

dc.subject | Knowledge Discovery | en_NZ |

dc.subject | Contructive Networks | en_NZ |

dc.subject.lcsh | T Technology (General) | en_NZ |

dc.subject.lcsh | Q Science (General) | en_NZ |

dc.subject.lcsh | QA76 Computer software | en_NZ |

dc.title | Evolving connectionist systems: Characterisation, simplification, formalisation, explanation and optimisation | en_NZ |

dc.type | Thesis | en_NZ |

dc.description.version | Unpublished | en_NZ |

otago.bitstream.pages | 21 | en_NZ |

otago.date.accession | 2006-07-13 | en_NZ |

otago.school | Information Science | en_NZ |

thesis.degree.discipline | Information Science | en_NZ |

thesis.degree.name | Doctor of Philosophy | |

thesis.degree.grantor | University of Otago | |

thesis.degree.level | Doctoral Theses | |

otago.openaccess | Open | |

dc.identifier.eprints | 333 | en_NZ |

otago.school.eprints | Knowledge, Intelligence & Web Informatics Laboratory | en_NZ |

otago.school.eprints | Information Science | en_NZ |

dc.description.references | Abraham, A. (2002). Optimization of evolutionary neural networks using hybrid learning algorithms. In Proceedings of IJCNN 2002, pages 2797-2802. Abreu, A. and Pinto-Ferreira, L. C. (1996). Fuzzy modeling: a rule based approach. In Proceedings of the fifth IEEE International Conference on Fuzzy Systems, pages 162-168. Aguiler, J. and Colmenares, A. (1997). Recognition algorithm using evolutionary learning on the random neural network. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 2, pages 1023-1028. IEEE Press. Alander, J. T. (1993). On robot navigation using a genetic algorithm. In Albrecht, R., Reeves, C., and Steele, N., editors, Artificial Neural Nets and Genetic Algorithms, pages 471-478. Alpaydin, E. (1994). GAL: Networks that grow when they learn and shrink when they forget. International Journal of Pattern Recognition and Artificial Intelligence, 8(1):391-414. Anderson, E. (1935). The irises of the gaspe peninsula. Bulletin of the American Iris Society, 59. Anderson, S., Merrill, J., and Port, R. (1988). Dynamic speech categorization with recurrent networks. In Touretzky, D., Hinton, G., and Sejnowski, T., editors, Proceedings of the 1988 Connectionist Models Summer School, pages 398-396. Morgan Kaufmann. Andreassen, H., Bohr, H., Bohr, J., Brunak, S., Bugge, T., Cotterill, R., Jacobsen, C., Kusk, P., Lautrup, B., Petersen, S., Saermark, T., and Ulrich, K. (1990). Analysis of the secondary structure of the human immunodeficiency virus (HIV) proteins p17, gp120, and gp41 by computer modeling based on neural networks methods. Journal of Acquired Immune Deficiency Syndromes, 3:615-622. Andrews, R., Diederich, J., and Tickle, A. B. (1995). Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems, 8(6):373-389. Andrews, R. and Geva, S. (1997). Refining expert knowledge with an artificial neural network. In Kasabov, N., Kozma, R., Ko, K., S'Shea, R., Coghill, G., and Gedeon, T., editors, Progress in Connectionist-Based Information Systems, volume 2, pages 847-850. Springer. Angeline, P. K., Saunders, G. M., and Pollack, J. B. (1994). An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(1):54-65. Antonisse, J. (1989). A new interpretation of schema notation that overturns the binary encoding constraint. In Schaffer, J., editor, Proceedings of the Third International Conference on Genetic Algorithms, pages 86-91. Arena, P., Caponetto, R., Fortuna, L., and Xibilia, M. G. (1993). M.L.P. optimal topology via genetic algorithms. In Artificial Neural Nets and Genetic Algorithms, pages 670-674. Springer-Verlag Wien New York. Ash, T. (1989). Dynamic node creation in backpropagation networks. Connection Science, 1(4):365-375. Baba, N., Marume, M., and Itoh, K. (1992). Utilization of stochastic automaton and genetic algorithm for neural network design. In Proceedings of the 2nd International Conference on Fuzzy Logic and Neural Networks, volume 2, pages 837-840, Iizuka, Japan. Back, T., Hoffmeister, F., and Schwefel, H.-P. (1991). A survey of evolution strategies. In Belew, R. K. and Booker, L. B., editors, Proceedings of the Fourth International Conference on Genetic Algorithms, pages 2-9. Balakrishnan, K. and Honavar, V. (1996). Analysis of neurocontrollers designed by simulated evolution. In International Conference on Neural Networks 1996: Plenary, Panel and Special Sessions, pages 130-135. Baldi, P. and Brunak, S. (1998). Bioinformatics: The Machine Learning Approach. MIT Press. Bebis, G., Georgiopoulos, and Kasparis, T. (1996). Coupling weight elimination and genetic algorithms. In Proceedings of the 1996 IEEE International Conference on Neural Networks, pages 1115-1120. Belew, R. K., McInerney, J., and Schraudolph, N. N. (1990). Evolving networks: Using the genetic algorithm with connectionist learning. In Langton, C. G., Taylor, C., Farmer, J. D., and Rasmussen, S., editors, Artificial Life II, pages 511-547, Santa Fe, New Mexico. Addison-Wesley Publishing Company. Bengio, Y. and De Mori, R. (1988). Speaker normalization and automatic speech recognition using spectral lines and neural networks. In Touretzky, D., Hinton, G., and Sejnowski, T., editors, Proceedings of the 1988 Connectionist Models Summer School, pages 388-397. Morgan Kaufmann. Billings, S. A. and Zheng, G. L. (1995). Radial basis function network configuration using genetic algorithms. Neural Networks, 8(6):877-890. Bisant, D. and Maizel, J. (1995). Identification of ribosome binding sites in Escherichia coli using neural network models. Nucleic Acids Research, 23(9):1632-1639. Bohr, H., Bohr, J., Brunak, S., Cotterill, R. M., Lautrup, B., Norskov, L., Olsen, 0. H., and Petersen, S. B. (1988). Protein secondary structure and homology by neural networks. The a-helices in rhodopsin. FEBS Letters, 241(1,2):223-228. Bornholdt, S. and Graudenz, D. (1992). General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327-334. Bose, N. and Garga, A. K. (1993). Neural network design using voronoi diagrams. IEEE Transactions on Neural Networks, 4(5):778-787. Bourland, H. and Wellekens, C. (1987). Multiplayer perceptrons and automatic speech recognition. In IEEE First Annual Conference on Neural Networks, volume IV, pages 407-416, San Diego. Box, G. E. and Jenkins, G. M. (1970). Time Series Analysis forecasting and control. Holden-Day. Brasil, L. M., de Azevedo, F. M., and Barreto, J. M. (2000). A hybrid expert system for the diagnosis of epileptic crisis. Artificial Intelligence in Medicine, 585:1-7. Brown, A. and Card, H. (1997). Evolutionary artificial neural networks for competitive learning. In Proceedings of /CNN, pages 1558-1562. Brunak, S., Engelbrecht, J., and Knudsen, S. (1991). Prediction of human mRNA donor and acceptor sites from the DNA sequence. Journal of Molecular Biology, 220:49-65. Bruske, J. and Sommer, G. (1995a). Dynamic cell structure learns perfectly topology preserving map. Neural Computation, 7 :845-865. Bruske, J. and Sommer, G. (1995b). Dynamic cell structures. In Tesauro, G., Touretzky, D., and Leen, T., editors, Advances in Neural Information Processing Systems 7, pages 497-504. The MIT Press. Bud, A. and Nocholson, A. (1997). Scheduling trains with genetic algorithms. In Kasabov, N., Kozma, R., Ko, K., O'Shea, R., Coghill, G., and Gedeon, T., editors, Progress in Connectionist-Based Information Systems, volume 2, pages 1017-1020. Carpenter, G., Grossberg, S., Markuzon, M., Reynolds, J., and Rosen, D. (1992). Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks, 3:698-713. Casdagli, M. (1989). Nonlinear prediction of chaotic time-series. Physica D, 35:335-356. Castellano, G. and Fanelli, A. (2000). Fuzzy inference and rule extraction using a neural network. Neural Network World Journal, 3:361-371. Cechin, A. L., Epperlin, U., Rosentiel, W., and Koppenhoefer, B. (1996). The extraction of sugeno fuzzy rules from neural networks. In Andrews, R. and Diederich, J., editors, Rules and Networks, pages 16-24. Queensland University of Technology, Neurocomputing Research Centre. Chandonia, J.-M. and Karplus, M. (1995). Neural networks for secondary structure and structural class predictions. Protein Science, 4:275-285. Chauvin, Y. (1990). A back-propagation algorithm with optimal use of hidden units. In Touretzky, D., editor, Advances in Neural Information Processing Systems (Denver 1988), pages 519-526. Morgan Kaufmann, San Mateo. Chellapilla, K. and Fogel, D. B. (1999). Evolving neural networks to play checkers without relying on expert knowledge. IEEE Transactions on Neural Networks, 10(6):1382-1391. Chiu, S. (1994). Fuzzy model identification based on cluster estimation. Journal of Intelligent and Fuzzy Systems, 2. Cho, S.-B. and Shimohara, K. (1998). Cooperative behavior in evolved modular neural networks. In Methodologies for the Conception, Design and Application of Soft Computing: Proceedings of IIZUKA '98, pages 606-609. Choi, B. and Bluff, K. (1995). Genetic optimisation of control parameters of a neural network. In Kasabov, N. K. and Coghill, G., editors, Artificial Neural Networks and Expert Systems, pages 174-177. IEEE Computer Society Press. Cortes, C. and Vapnik, V. (1995). Support vector networks. Machine Learning, 20:273-297. Crick, F. (1989). The recent excitement about neural networks. Nature, 337:129-132. Crowder, R. (1990). Predicting the Mackey-Glass timeseries with cascade-correlation learning. In Touretzky, D., Hinton, G., and Sejnowski, T., editors, Proceedings of the 1990 Connectionist Models Summer School, pages 117-123, Carnegie Mellon Univ. Cybenko, G. (1989). Approximation by superpositions of sigmoidal function. Mathematics of Control, Signals, and Systems, 2:303-314. Darwin, C. (1859). The Origin of Species by means of natural selection. John Murray, London. Davis, L., editor (1996). Handbook of Genetic Algorithms. International Thomson Computer Press. Davis, S. B. and Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions of Acoustics, Speech, and Signal Processing, 28(4):357-366. de Castro, L. N., Iyoda, E. M., Von Zuben, F. J., and Gudwin, R. (1998). Feedforward neural network initialization: an evolutionary approach. In Proceedings of Vth Brazilian Conference on Neural Networks, December 9-11, 1998, pages 43-48. Deng, D. and Kasabov, N. (1999). Evolving self-organizing map and its application in generating a world macroeconomic map. In Kasabov, N. and Ko, K., editors, Emerging Knowledge Engineering and Connectionist-based Systems (Proceedings of the ICONIP/ANZIIS/ANNES'99 Workshop "Future directions for intelligent systems and information sciences", Dunedin, 22-23 November 1999), pages 7-12. University of Otago Press. Diederichs, K., Freigang, J., Umhau, S., Zeth, K., and Breed, J. (1998). Prediction by a neural network of outer membrane /3-strand protein topology. Protein Science, 7:2413-2420. Dorado, J., Rabunal, J., Rivero, D., Santos, A., and Pazos, A. (2002). Automatic recurrent ANN rule extraction with genetic programming. In Proceedings of IJCNN 2002, pages 1552-1557. East, I. R. and Rowe, J. (1997). Abstract genetic representation of dynamical neural networks using kauffman networks. Artificial Life, 3:67-80. Eldracher, M. (1992). Classification of non-linear-separable real-world-problems usin 8-rule, perceptrons and topologically distributed encoding. In Proceedings of the 1992 ACM/SIGAPP Symposium on Applied Computing, volume 2, pages 1098-1104. ACM Press. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14:179-211. Esat, I., Kothari, B., and Wrathall, P. (1999). Encoding neural networks for GA based structural construction. In ICONIP '99 6th International Conference on Neural Information Processing, pages 359-365. Fahlman, S. E. (1988). An empirical study of learning speed in back-propagation networks. Technical Report CMU-CS-88-162, Department of Computer Science, Carnegie-Mellon University. Fahlman, S. E. and Lebiere, C. (1990). The cascade-correlation learning architecture. In Touretzky, D. S., editor, Advances in Neural Information Processing Systems 2, pages 524-532. Morgan Kaufman Publishers. Faraq, W. and Tawfik, A. (2000). On fuzzy model identification and the gas furnace data. In Proceedings of the LASTED International Conference. Faraq, W. A., Quintana, V. H., and Lambert-Torres, G. (1997). Neuro-fuzzy modeling of complex systems using genetic algorithms. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 1, pages 444-449. IEEE Press. Fisher, R. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179-188. Fogel, D. B., Wasson, E. C., Boughton, E. M., and Porto, V. W. (1997). A step toward computer-assisted mammography using evolutionary programming and neural networks. Cancer Letters, 119:93-97. Fogel, L. J., Owens, A. J., and Walsh, M. J. (1965). Artificial intelligence through a simulation of evolution. In Maxfield, M., Callahan, A., and Fogel, L., editors, Biophysics and Cybernetic Systems: Proceedings of the 2nd Cybernetic Sciences Symposium, pages 131-155. Fontanari, J. and Meir, R. (1991). Evolving a learning algorithm for the binary perceptron. Network, 2:353-359. Franzini, M. A. (1988). Learning to recognize spoken words: A study in connectionist speech recognition. In Touretzky, D., Hinton, G., and Sejnowski, T., editors, Proceedings of the 1988 Connectionist Models Summer, pages 407-416. Morgan Kaufmann. Frean, M. (1990). The upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation, 2(2):198-209. Fritzke, B. (1991). Unsupervised clustering with growing cell structures. In Proceedings of the IJCNN-91 Seattle. IEEE Press. Fritzke, B. (1993a). Growing cell structures - a self organizing network for unsupervised and supervised learning. Technical Report TR-93-026, International Computer Science Institute. Fritzke, B. (1993b). Kohonen feature maps and growing cell structures - a performance comparison. In Giles, C., Hanson, S., and Cowan, J., editors, Advances in Neural Information Processing Systems 5. Morgan Kaufmann. Fritzke, B. (1994). Supervised learning with growing cell structures. In Cowan, J. D., Tesauro, G., and Alspector, J., editors, Advances in Neural Information Processing Systems 6, pages 255-262. Morgan Kaufmann. Fritzke, B. (1995). A growing neural gas network learns topologies. In Tesauro, G., Tourezky, D., and Leen, T., editors, Advances in Neural Information Processing Systems 7, pages 625-632. The MIT Press. Fu, L. (1999). An expert network for DNA sequence analysis. IEEE Intelligent Systems, 14(January / February):65-71. Fukuda, T., Komata, Y., and Arakawa, T. (1997a). Recurrent neural networks with self-adaptive GAs for biped locomotion robot. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 3, pages 1710-1715. IEEE Press. Fukuda, T., Komata, Y., and Arakawa, T. (1997b). Recurrent neural networks with self-adaptive GAs for biped locomotion robot. In 1997 International Conference on Neural Networks (ICNN '97), volume 3, pages 1710- 1715, Westin Galleria Hotel, Houston, Texas, USA. IEEE Press. Fukumi, M. and Akamatsu, N. (1996). A genetic approach to feature selection for pattern recognition systems. In Methodologies for the Conception, Design and Application on Intelligent Systems: Proceedings of IIZUKA '96, pages 907-910. Furuhashi, T., Hasegawa, T., Horikawa, S.-i., and Uchikawa, Y. (1993). An adaptive fuzzy controller using fuzzy neural networks. In Proceedings of Fifth IFSA World Congress, pages 769-772. Furuhashi, T., Matushita, S., Tsutsui, H., and Uchikawa, Y. (1997). Knowledge extraction from hierarchical fuzzy model obtained by fuzzy neural networks and genetic algorithm. In Proceedings of the 1997 International Conference on Neural Networks (ICNN'97), volume 4, pages 2374-2379. IEEE Press. Gallant, S. I. (1993). Neural Network Learning and Expert Systems. MIT Press. Gan, M., Lan, H., and Zhang, L. (1995). A genetic-based method of generating fuzzy rules and membership functions by learning from example. In Proceedings of International Conference on Neural Information Processing (ICONIP'95), volume 1, pages 335-338. Gates, G. (1972). The reduced nearest neighbor rule. IEEE Transactions on Information Theory, pages 431-433. Gaweda, A. E., Zurada, J. M., and Aronhime, P. B. (2002). Efficient data-driven modeling with fuzzy relational rule network. In Proceedings of FUZZ-IEEE 2002, pages 174-178. Ghobakhlou, A., Watts, M., and Kasabov, N. (2000). On-line expansion of output space in evolving fuzzy neural networks. In Proceedings ICONIP 2000, Taejon, Korea, November, 2000, volume 2, pages 891-896. Ghobakhlou, A. A. and Seesink, R. (2001). An interactive multi modal system for mobile robotic control. In Proceedings of the Fifth Biannual Conference on Artificial Neural Networks and Expert Systems (ANNES2001 ), pages 93-99. Glaeser, A. (1998). Modular neural networks for low-complex phoneme recognition. In Proceedings of ICSLP'98, pages 1303-1306. Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimisation and Machine Learning. Addison-Wesley. Grefenstette, J. J. (1986). Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics, 16(1):122-128. Gueriot, D. and Maillard, E. (1996). A local approach for a fuzzy error function used in multilayer perceptron training through a genetic algorithm. In Proceedings of the 1996 IEEE international conference on neural networks, pages 1050-1055. Gupta, M. M. and Ding, H. (1994). Fuzzy neuronal networks and genetic algorithms. In Proceedings of the 3rd International Conference on Fuzzy Logic, Neural Nets and Soft Computing (Iizuka, Japan), pages 187-188. Hakim, B. A. (2001). Extraction and optimization of fuzzy rules. In Zhang, L. and Gu, F., editors, Proceedings of ICONIP 2001, November 14-18, 2001, Shanghai, China, volume 1, pages 361-365. Fudan University Press. Hamker, F. H. (2001). Life-long learning cell structures-continuously learning without catastrophic interference. Neural Networks, 14:551-573. Hanebeck, D. and Schmidt, G. K. (1994). Optimization of fuzzy networks via genetic algorithms. In Proceedings of International Conference on Neural Information Processing, volume 3, pages 1583-1588. Hansen, L., Rasmussen, C., Svarer, C., and Larsen, J. (1994). Adaptive regularization. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing IV, pages 78-87, Piscataway, New Jersey. IEEE Press. Harp, S. A., Samad, T., and Guha, A. (1990). Designing application-specific neural networks using the genetic algorithm. In Toretzky, D. S., editor, Advances in Neural Information Processing Systems 5, pages 447-454. Morgan Kauffman Publishers. Hasegawa, T., Horikawa, Furuhashi, T., and Uchikawa, Y. (1992). A study'on fuzzy modeling of BOF using a fuzzy neural network. In Proceedings of the 2nd International Conference on Fuzzy Logic and Neural Networks (lizuka, Japan, July 17-22, 1992), pages 1061-1064. Hasegawa, T., Horikawa, Furuhashi, T., and Uchikawa, Y. (1993). An application of fuzzy neural networks to design of adaptive fuzzy controllers. In Proceedings of 1993 International Joint Conference on Neural Networks, pages 1761-1764. Hashiyama, T., Furuhashi, T., and Uchikawa, Y. (1993a). A fuzzy neural network for identifying changes of degrees of attention in a multi-attribute decision making process. In Proceedings of 1993 International Joint Conference on Neural Networks, pages 705-708. Hashiyama, T., Furuhashi, T., and Uchikawa, Y. (1993b). A study on a multi-attribute decision making process using a fuzzy neural network. In Proceedings of Fifth IFSA World Congress, pages 810-813. Haskey, S. and Datta, S. (1998). A comparative study of OCON and MLP architectures for phoneme recognition. In Proceedings of ICSLP 98. Hassibi, B. and Stork, D. (1993). Optimal brain surgeon. In Hanson, S., Cowan, J., and Giles, C., editors, Advances in Neural Information Processing Systems (Denver, 1992), pages 164-171. Morgan Kaufmann, San Mateo. Heinke, D. and Hamker, F. H. (1998). Comparing neural networks: A benchmark on growing neural gas, growing cell structures, and fuzzy ARTMAP. IEEE Transactions on Neural Networks, 9(6):1279-1291. Heistermann, J. (1990). The application of a genetic approach as an algorithm for neural networks. In Schwefel, H.-P. and Manner, R., editors, Parallel Problem Solving from Nature, volume 496 of Lecture Notes in Computer Science, pages 297-301. Springer-Verlag. Hingston, P., Barone, L., and L., W. (2002). Evolving crushers. In Proceedings of CEC 2002, pages 1109-1114. Hiraga, I. and Furuhashi, T. (1995). An acquisition of operator's rules for collision avoidance using fuzzy neural networks. In IEEE Transactions on Fuzzy Systems, 3(3). Hoffmeister, F. and Back, T. (1991). Genetic algorithms and evolution strategies: Similarities and differences. In H-P, S. and Manner, R., editors, Parallel Problem Solving from Nature. Springer Verlag. Holland, J. H. (1975). Adaptation in Natural and Artificial Systems. MIT Press. Homma, T., Atlas, L. E., and Marks, R. J. (1988). An artificial neural network for spatio-temporal bipolar patterns: Application to phoneme classification. In Touretzky, D., Hinton, G., and Sejnowski, T., editors, Proceedings of the 1988 Connectionist Models Summer School, pages 380-387. Morgan Kaufmann. Horikawa, S. -i., Furuhashi, T., Okuma, S., and Uchikawa, Y. (1990). Composition methods of fuzzy neural networks. In Proceedings of IEEE/IE CON '90, pages 1253-1258. Huang, S. H. and Benjamin, M. (2001). Automated knowledge acquisition for design and manufacturing: The case of micromachined atomizer. Journal of Intelligent Manufacturing, 12:377-391. Hung, S. and Adeli, H. (1994). A parallel genetic/neural network learning algorithm for MIMD shared memory machines. IEEE Transactions on Neural Networks, 5(6):900-909. Ichimura, T., Matsumoto, N., Tazaki, E., and Yoshida, K. (1997). Extraction method of rules from reflective neural network architecture. In Proceedings of the 1997 International Conference on Neural Networks (ICNN'97), volume 1, pages 510-515. IEEE Press. Ishibuchi, H., Nii, M., and Murata, T. (1997). Linguistic rule extraction from neural networks and geneticalgorithm0based rule selection. In Proceedings of the 1997 International Conference on Neural Networks (ICNN'97), volume 4, pages 2390-2395. IEEE Press. Ishikawa, M. (1996). Structural learning with forgetting. Neural Networks, pages 501-521. Ivanova, I. and Kubat, M. (1995). Initialization of neural networks by means of decision trees. Knowledge Based Systems, 8(6):333-344. IzquierdO, J. M. C., Dimitriadis, Y. A., Sanchez, E. G., and Coronado, J. L. (2001). Learning from noisy information in FasArt and FasBack neuro-fuzzy systems. Neural Networks, 14:407-425. Jacobsson, H. and Olsson, B. (2000). An evolutionary algorithm for inversion of ANNs. In Wang, P. P., editor, Proceedings of the Fifth Joint Conference on Information Sciences, volume 1, pages 1070-1073. Jagielska, I., Matthews, C., and Whitfort, T. (1996). The application of neural networks, fuzzy logic, genetic algorithms, and rough sets to automated knowledge acquisition. In Yamakawa, T. and Matsumoto, G., editors, Methodologies for the Conception, Design, and Application of Intelligent Systems: Proceedings of IIZUKA'96, volume 2, pages 565-569. World Scientific. Jang, J.-S. R. (1993). ANFIS: Adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man and Cybernetics, 23:665-684. Janikow, C. Z. and Michalewicz, Z. (1991). An experimental comparison of binary and floating point representations in genetic algorithms. In Belew, R. K. and Booker, L. B., editors, Fourth International Conference on Genetic Algorithms, pages 31-36, University of California, San Diego. Morgan Kaufmann Publishers. Jenkins, N. and Gedeon, T. (1997). Genetic algorithms applied to university exam scheduling. In Kasabov, N., Kozma, R., Ko, K., O'Shea, R., Coghill, G., and Gedeon, T., editors, Progress in Connectionist-Based Information Systems, volume 2, pages 1034-1037. Jones, R., Less, Y., Barnes, C., Flake, G., Lee, K., and Lewis, P. (1990). Function approximation and time series prediction with neural networks. In Proc. IEEE Int. Joint Conf. Neural Networks, volume 1, pages 649-665. Karnin, E. (1990). A simple procedure for pruning back-propagation trained neural networks. IEEE Transactions on Neural Networks, 1(2):239-242. Kasabov, N. (1998a). The ECOS framework and the ECO learning method for evolving connectionist systems. Journal of Advanced Computational Intelligence, 2(6):195-202. Kasabov, N. (1999). Evolving connectionist systems: A theory and a case study on adaptive speech recognition. In International Joint Conference on Neural Networks (IJCNN), July 10-16. Kasabov, N. and Fedrizzi, M. (1999). Fuzzy neural networks and evolving connectionist systems for intelligent decision making. In Proceedings of the Eighth International Fuzzy Systems Association World Congress, Taiwan, August 17-20, pages 30-35. Kasabov, N., Kim, J., Watts, M., and Gray, A. (1997a). FuNN/2 - a fuzzy neural network architecture for adaptive learning and knowledge acquisition in multi-modular distributed environments. Information Sciences - Applications. Kasabov, N., Kozma, R., Kilgour, R., Laws, M., Taylor, J., Watts, M., and Gray, A. (1997b). A methodology for speech data analysis and a framework for adaptive speech recognition using fuzzy neural networks. In Progress in Connectionist-Based Information Systems, Proceedings of the ICONIP / ANZIIS / ANNES '97, Dunedin, 24-28 November 1997. Springer Verlag. Kasabov, N., Kozma, R., Kilgour, R., Laws, M., Watts, M., Gray, A., and Taylor, J. (1999). Speech data analysis and recognition using fuzzy neural networks and self-organising maps. In Kasabov, N. and Kozma, R., editors, Neuro-Fuzzy Techniques for Intelligent Information Systems, pages 241-263. Physica-Verlag. Kasabov, N. and Song, Q. (2000). Dynamic evolving neuro-fuzzy inference system (DENFIS): On-line learning and application for time-series prediction. In Proceedings of the 6th International Conference on Soft Computing, October 1-4, 2000, Fuzzy Logic Systems Institute, Iizuka, Japan, pages 696-702. Kasabov, N. and Song, Q. (2002). DENFIS: Dynamic evolving neural-fuzzy inference systems. IEEE Transactions on Fuzzy Systems, 10(2):144-154. Kasabov, N. and Woodford, B. (1999). Rule insertion and rule extraction from evolving fuzzy neural networks: Algorithms and applications for building adaptive, intelligent expert systems. In IEEE International Fuzzy Systems Conference, pages 1406-1411. Kasabov, N. K. (1996a). Foundations of Neural Networks, Fuzzy Systems and Knowledge Engineering. MIT Press. Kasabov, N. K. (1996b). Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid systems. Fuzzy Sets and Systems, 82(2). Kasabov, N. K. (1998b). ECOS: Evolving connectionist systems and the ECO learning paradigm. In Usui, S. and Omori, T., editors, ICONIP'98 Proceedings, volume 2, pages 1232-1235. Kasabov, N. K. (1998c). Evolving fuzzy neural networks - algorithms, applications and biological motivation. In Yamakawa, T. and Matsumoto, G., editors, Methodologies for the Conception, Design and Application of Soft Computing, volume 1, pages 271-274. World Scientific. Kasabov, N. K. (2003). Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines. Springer. Kasabov, N. K. and Watts, M. J. (1997). Genetic algorithms for structural optimisation, dynamic adaptation and automated design of fuzzy neural networks. In 1997 International Conference on Neural Networks (ICNN '97), volume 4, pages 2546-2549, Westin Galleria Hotel, Houston, Texas, USA. IEEE Press. Kermani, B. G., Schiffman, S. S., and Nagle, H. T. (1999). Using neural networks and genetic algorithms to enhance performance in an electronic nose. IEEE Transactions on Biomedical Engineering, 46(4):429-439. Kilgour, R. (2003). Evolving Systems for Connectionist-Based Speech Recognition. PhD thesis, University of Otago. Kim, E., Park, M., Ji, S., and Park, M. (1997). A new approach to fuzzy modeling. IEEE Transactions on Fuzzy Systems, 5:328-337. Kim, E., Park, M., Kim, S., and Park, M. (1998). A transformed input-domain approach to fuzzy modeling. IEEE Transactions on Fuzzy Systems, 6(4):596-604. Kim, Y.-W. and Park, D.-J. (1996). Ship collision avoidance using genetic algorithm. In Methodologies for the Conception, Design, and Application of Intelligent Systems, pages 545-548. Kirchhoff, K. (1998). Combining articulatory and acoustic information for speech recognition in noisy and reverberant environments. In Proceedings of the International Conference on Spoken Language Processing, Sydney, Australia, pages 891-894. Kohonen, T. (1990). The self-organizing map. Proceedings of the IEEE, 78(9):1464-1479. Kohonen, T. (1997). Self-Organizing Maps. Springer, second edition. Koizumi, T., Mori, M., Taniguchi, S., and Maruya, M. (1996). Recurrent neural networks for phoneme recognition. In Proceedings of ICSLP'96, volume 1, pages 326-329. Kolmogorov, A. (1957). On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition. Dokl. Akad. Nauk. USSR, 114:953-956. (in Russian). Kong, S.-G. and Kosko, B. (1992). Adaptive fuzzy systems for backing up a truck-and-trailer. IEEE Transactions on Neural Networks, 3(2):211-223. Koprinska, I. and Kasabov, N. (1999). An application of evolving fuzzy neural network for compressed video parsing. In ICONIP/ANZIIS/ANNES'99 Workshop, Dunedin, New Zealand, November 22-24, pages 96-102. Kosko, B. (1992). Neural Networks and Expert Systems: A Dynamical Systems Approach to Machine Intelligence. Prentice-Hall, Englewood Cliffs, New Jersey. Kosko, B. (1993). Fuzzy Thinking. Flamingo. Koza, J. R. (1993). Genetic programming: on the programming of computers by means of natural selection. MIT Press, 3rd edition. Kwok, T.-Y. and Yeung, D.-Y. (1999). Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Transactions on Neural Networks. Lang, K. J. and Witbrock, M. J. (1988). Learning to tell two spirals apart. In Touretzky, D., Hinton, G., and Sejnowski, T., editors, Proceedings of the 1988 Connectionist Models Summer School, pages 52-57. Lapedes, A. and Farber, R. (1987). Nonlinear signal processing using neural networks: prediction and system modeling. Technical Report LA-UR-87-2662, Los Alamos Nat. Lab., Los Alamos, NM. Lawrence, S., Tsoi, A. C., and Back, A. D. (1996). The gamma MLP for speech phoneme recognition. In Touretzky, D., Mozer, M., and Hasselmo, M., editors, Advances in Neural Information Processing Systems 8, pages 785-791. MIT Press. Le Cun, Y., Denker, J., and Solla, S. (1990). Optimal brain damage. In Touretzky, D., editor, Advances in Neural Information Processing Systems, pages 598-605. Morgan-Kaufmann, San Mateo. Lee, D.-W. and Sim, K.-B. (1998). Ontogenesis of artificial neural networks based on L-System and genetic algorithms. In Yamakawa, T. and Matsumoto, G., editors, Methodologies for the Conception, Design and Application of Soft Computing: Proceedings of IIZUKA'98, volume 2, pages 817-820. World Scientific. Lee, K.-M., Kwak, D.-H., and Lee-Kwang, H. (1994). On fuzzy modeling with fuzzy neural networks. In Proceedings of International Conference on Neural Information Processing, volume 3, pages 1589-1594. Lee, K.-M., Yamakawa, T., Uchino, E., and Lee, K.-M. (1997). A genetic algorithm approach to job shop scheduling. In Kasabov, N., Kozma, R., Ko, K., O'Shea, R., Coghill, G., and Gedeon, T., editors, Progress in Connectionist-Based Information Systems, volume 2, pages 1030-1033. Lei. J., He, G., and Jiang, P. (1997). The state estimation of the CSTR system based on a recurrent neural network trained by HGAs. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 2, pages 779-782. IEEE Press. Leichter, C. S., Cichocki, A., and Kasabov, N. (2001). Independent component analysis and evolving fuzzy neural networks for the classification of single trial EEG data. In Proceedings of the Fifth Biannual Conference on Artificial Neural Networks and Expert Systems (ANNES2001), pages 100-105. Leung, H. C., Glass, J. R., Philips, M. S., and Zue, V. W. (1990). Phonetic classification and recognition using the multi-layer perceptron. In Advances in Neural Information Processing, pages 248-254. Lin, Y. and Cunningham ITT, G. (1995). A new approach to fuzzy-neural modeling. IEEE Transactions on Fuzzy Systems, 3(2):190-197. Lippmann, R. P. (1987). An introduction to computing with neural nets. IEEE ASSP Mag., pages 4-22. Lippmann, R. P. (1989). Review of neural networks for speech recognition. Neural Computation, 1:1-38. Lippmann, R. P. (1997). Speech recognition by machines and humans. Speech Communications, 22:1-15. Littmann, E. and Ritter, H. (1996). Learning and generalization in cascade network architectures. Neural Computation, 8:1521-1539. Liu, Y. and Yao, X. (1996). A population-based learning algorithm which learns both architectures and weights of neural networks. In Yao, X. and Li, X., editors, Proceedings of ICYCS'95 Workshop on Soft Computing, pages54-65. Mackey, M. C. and Glass, L. (1977). Oscillation and chaos in physiological control systems. Science, 197:287- 289. Maillard, E. P. (1997). RBF neural network, basis functions and genetic algorithm. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 4, pages 2187-2192. IEEE Press. Mamdani, E. (1976). Advances in linguistic synthesis of fuzzy controllers. International journal of Man-Machine Studies, 8(6):669-678. Mandischer, M. (1993a). Genetic optimization and representation of neural networks. In Proceedings of the Fourth Australian Conference on Neural Networks (ACNN93), pages 122-125. Mandischer, M. (1993b). Representation and evolution of neural networks. In Albrecht, R., Reeves, C., and Steele, N., editors, Artificial Neural Nets and Genetic Algorithms, pages 643-649. Springer-Verlag Wien New York. MATLAB Manual (2002). MATLAB Neural Networks Toolbox Manual. The MathWorks, Inc. Matthews, B. (1975). Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim. Biophys. Acta, 405:442-451. Matthews, C. and Jagielska, I. (1995). Fuzzy rule extraction from a trained multilayered neural network. In Proceedings of the 1995 IEEE International Conference on Neural Networks (ICNN'95), Perth, Australia. McCloskey, M. and Cohen, N. (1989). Catastrophic interference in connectionist networks: The sequential learning project. The Psychology of Learning and Motivation, 24:109-164. McCullagh, J. and Bluff, K. (1993). Genetic modification of a neural networks training data. In Kasabov, N. K., editor, Artificial Neural Networks and Expert Systems, pages 58-59. McCullagh, J., Choi, B., and Bluff, K. (1997). Genetic evolution of a neural network's input vector for meteorological estimations. In Kasabov, N., Kozma, R., Ko, K., O'Shea, R., Coghill, G., and Gedeon, T., editors, 1997 International Conference on Neural Information Processing and Intelligent Information Systems, volume 2, pages 1046-1049, Dunedin, New Zealand. Springer. McCulloch, W. and Pitts, W. (1943). A logical calculus of the ideas imminent in nervous activity. Bulletin of Mathematical Biophysics, 5:115-133. McDonnell, J. and Waagen, D. (1994). Evolving recurrent perceptrons for time-series modeling. IEEE Transactions on Neural Networks, 5(1):24-38. Mentzer, F. and Parisi, D. (1992). Recombination and unsupervised learning: effects of crossover in the genetic optimization of neural networks. Network, 3:423-442. Mezard, M. and Nadal, J.-P. (1989). Learning in feedforward layered networks: the tiling algorithm. Journal of Physics A, 22:2191-2203. Michalewicz, Z. (1992). Genetic Algorithms + Data Structures = Evolution Programs. Springer Verlag. Michalski, R. S. (1983). A theory and methodology of inductive learning. Artificial Intelligence, 20:111-161. Minsky, M. L. and Papert, S. A. (1969). Perceptrons. MIT Press. Mitchell, M. (1996). An Introduction to Genetic Algorithms. MIT Press. Mitra, S., De, R. K., and Pal, S. K. (1997). Knowledge-based fuzzy MLP for classification and rule generation. IEEE Transactions on Neural Networks, 8(6):1338-1350. Mitra, S. and Hayashi, Y. (2000). Neuro-fuzzy rule generation: Survey in soft computing framework. IEEE Transactions on Neural Networks, 11(3):748-768. Mitra, S. and Pal, S. K. (1996). Fuzzy self-organization, inferencing, and rule generations. IEEE Transactions of Systems, Man, and Cybernetics, 26(5):608-620. Mizutani, E. and Dreyfus, S. E. (2002). MLP's hidden-node saturations and insensitivity to initial weights in two classification benchmark problems: parity and two-spirals. In Proceedings of the Congress on Evolutionary Computation, pages 2831-2836. Monfroglio, A. (1996). Timetabling through constrained heuristic search and genetic algorithms. Software-practice and experience, 26(3):251-279. Moody, J. (1989). Fast learning in multi-resolution hierarchies. In Touretzky, D., editor, Advances in Neural Information Processing Systems 1, pages 29-39. Morgan Kaufman. Moreira, M. and Fiesler, E. (1995). Neural networks with adaptive learning rate and momentum terms. Technical Report 95-04, Institut Dalle D'Intelligence Artificielle Perceptive. Moriarty, D. E. and Miikkulainen, R. (1998). Forming neural networks through efficient and adaptive coevolution. Evolutionary Computation, 5(4):373-399. Mozer, M. and Smolensky, P. (1989). Skeletonization: A technique for trimming the fat from a network via relevance assessment. In Touretzky, D., editor, Advances in Neural Information Processing Systems (Denver, 1988), pages 107-115. Morgan-Kaufmann, San Mateo. Miihlenbein, H. and Kindermann, J. (1989). The dynamics of evolution and learning - towards genetic neural networks. In Pfeifer, R., Schreter, Z., and Fogelman-Soulie, editors, Connectionism in Perspective, pages 173-197. North-Holland. Mukaidono, M. and Yamaoka, M. (1992). A learning method of fuzzy inference with neural networks and its application. In Proceedings of the 2nd International Conference on Fuzzy Logic and Neural Networks, volume 1, pages 185-187. Fuzzy Logic Systems Institute. Mukherjee, S., Osuna, E., and Girosi, F. (1997). Nonlinear prediction of chaotic time series using support vector machines. In Principe, J., Giles, L., Morgan, N., and Wilson, E., editors, IEEE Workshop on Neural Networks for Signal Processing VII, page 511. IEEE Press. Munro, P. W. (1993). Genetic search for optimal representations in neural networks. In Albrecht, R., Reeves, C., and Steele, N., editors, Artificial Neural Nets and Genetic Algorithms, pages 628-634. Springer-Verlag. Nyquist, H. (1928). Certain topics in telegraph transmission theory. Trans. AIEE, 47:617-644. Okabe, A., Boots, B., and Sugihara, K. (1992). Spatial Tessellations: Concepts and Applications of Voronoi Diagrams. John Wiley and Sons, Ltd. Optiz, D. W. and Shavlik, J. W. (1997). Connectionist theory refinement: Genetically searching the space of network topologies. Journal of Artificial Intelligence Research, 6:177-290. Paredis, J. (1994). Steps towards co-evolutionary classification neural networks. In Brooks, R. A. and Maes, P., editors, Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems, pages 102-108. The MIT Press. Parekh, R., Yang, J., and Honavar, V. (2000). Constructive neural-network learning algorithms for pattern classifcation. IEEE Transactions on Neural Networks, 11(2):436-451. Pedrycz, W. (1984). An identification algorithm in fuzzy relational systems. Fuzzy Sets and Systems, 13:153-167. Philipsen, W. and Cluitmans, L. (1993). Using a genetic algorithm to tune potts neural networks. In Albrecht, R., Reeves, C., and Steele, N., editors, Artificial Neural Nets and Genetic Algorithms, pages 650-657. Springer- Verlag. Pican, N., Fohr, D., and Mari, J.-F. (1996). HIMMs and OWE neural network for continuous speech recognition. In Proceedings of ICSLP. Platt, J. (1991a). Learning by combining memorization and gradient descent. In Advances in Neural Information Processing Systems III. Platt, J. (1991b). A resource-allocating network for function interpolation. Neural Computation, 3(2):213-225. Prechelt, L. (1997). Investigation of the CasCor family of learning algorithms. Neural Networks, 10(5):885-896. Principe, J. and Kuo, J.-M. (1995). Non-linear modelling of chaotic time series with neural networks. In Advances in Neural Information Processing Systems VII. Qian, H. (1996). Prediction of a-helices in proteins based on thermodynamic parameters from solution chemistry. Journal of Molecular Biology, 256:663-666. Qian, N. and Sejnowski, T. J. (1988). Predicting the secondary structure of globular proteins using neural network models. Journal of Molecular Biology, 202:865-884. Rantala, J. and Koivisto, H. (2002). Optimised subtractive clustering for neuro-fuzzy models. In Proceedings of the 3rd International Conference on Fuzzy Sets & Fuzzy Systems (FSFS'02) Interlaken, Switzerland, February 11-15. Ray, K. S. and Ghoshal, J. (1996). Neuro genetic approach to pattern recognition. In Methodologies for the Conception, Design, and Application of Intelligent Systems: Proceedings of IIZUKA '96, pages 221-224. Reed, R. D. and Marks, R. J. (1999). Neural Smithing. MIT Press, Cambridge, Massachusetts. Renals, S. and Rohwer, R. (1989). Phoneme classification experiments using radial basis functions. In Proceedings of International Joint Conference on Neural Networks - IJCNN, Washington, D.C., volume I, pages 461-467. Ribeiro, B. (2002). Kernelized based functions with minkovsky's norm for SVM regression. In IJCNN-2002, pages 2198-2203. Ripley, B. D. (1993). Statistical aspects of neural networks. In Barndorrf-Nielsen, 0., Jensen, J., and Kendall, W., editors, Networks and Chaos - Statistical and Probabilistic Aspects, chapter 2, pages 40-123. Chapman and Hall. Robbins, P., Soper, A., and Rennolls, K. (1993). Use of genetic algorithms for optimal topology determination in back propagation neural networks. In Albrecht, R., Reeves, C., and Steele, N., editors, Artificial Neural Nets and Genetic Algorithms, pages 726-730. Springer-Verlag Wien New York. Robinson, T. and Fallside, F. (1990). Phoneme recognition from the TIMIT database using recurrent error propagation networks. Technical report, Cambridge University, Engineering Department. Romero, E. and Alquëzar, R. (2002). A new incremental method for function approximation using feed-forward neural networks. In Proceedings of the International Joint Conference on Neural Networks (IJCCN) 2002, pages 1968-1973. Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review, 65:386-408. Rost, B. (1996). PHD: Predicting one-dimensional protein structure by profile-based neural networks. Methods in Enzymology, 266:525-539. Rost, B., Fariselli, P., and Casadio, R. (1996). Topology prediction for helical trans e brane proteins at 86% accuracy. Protein Science, 5:1704-1718. Rozich, R., Ioerger, T., and Yager, R. (2002). FURL - a theory revision approach to learning fuzzy rules. In Proceedings of FUZZ-IEEE 2002, pages 791-796. Rumelhart, D., Hinton, G., and Williams, R. (1986). Learning representations by back-propagating errors. Nature, 323:533-536. Sanger, T. (1991). A tree-structured adaptive network for function approximation in high-dimensional spaces. IEEE Trans. Neural Networks, 2(2):285-293. Sarkar, M. and Yegnanarayana, B. (1997a). An evolutionary programming-based probabilisitc neural networks construction technique. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 1, pages 456-461. IEEE Press. Sarkar, M. and Yegnanarayana, B. (1997b). Feedforward neural networks configuration using evolutionary programming. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 1, pages 438-443. Schiffmann, W., Joost, M., and Werner, R. (1990). Performance evaluation of evolutionarily created neural network topoplogies. In Schwefel, H.-P. and Manner, R., editors, Parallel Problem Solving from Nature, volume 496 of Lecture Notes in Computer Science, pages 274-283. Springer-Verlag. Schiffmann, W., Joost, M., and Werner, R. (1993). Application of genetic algorithms to the construction of topologies for multilayer perceptrons. In Albrecht, R., Reeves, C. R., and Steele, N., editors, Artificial Neural Nets and Genetic Algorithms, pages 675-682. Springer-Verlag Wien New York. Schiffmann, W., Joost, M., and Werner, R. (1994). Optimization of backpropagation algorithm for training multiplayer perceptrons. Technical report, Institute of Physics, University of Koblenz. Scholz, M. (1990). A learning strategy for neural networks based on a modified evolutionary strategy. In Schwefel, H.-P. and Manner, R., editors, Parallel Problem Solving from Nature, volume 496 of Lecture Notes in Computer Science, pages 314-318. Springer-Verlag. Shibata, T., Fukuda, T., Kosuge, K., and Arai, F. (1996). Path-planning for multiple mobile robots by genetic algorithms. In Methodologies for the Conception, Design, and Application of Intelligent Systems, pages 747- 750. Siddiqi, A. and Lucas, S. (1998). A comparison of matrix rewriting versus direct encoding for evolving neural networks. In Proceedings of IEEE Conference on Evolutionary Computation 1998. Siddique, M. and Tokhi, M. (2001). Training neural networks: Backpropagation vs genetic algorithms. In Proceedings of IJCNN 2001, pages 2673-2678. Sima, M., Croitoru, V., and Burileanu, D. (1998). Performance analysis on speech recognition using neural networks. In Proceedings of the International Conference and Development and Application Systems, Suceava,Romania, pages 259-266. Sinclair, S. and Watson, C. (1995). The development of the Otago speech database. In Kasabov, N. K. and Coghill, G., editors, Proceedings of ANNES '95. IEEE Computer Society Press, Los Alamitos, CA. Sirvadam, V., McCloone, S., and Irwin, G. (2002). Separable recursive training algorithms for feedforward neural networks. In Proceedings of IJCNN 2002, pages 1212-1217. Smalz, R. and Conrad, M. (1994). Combining evolution with credit apportionment: A new learning algorithm for neural nets. Neural Networks, 7(2):341-351. Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society, 36:111-147. Sugeno, M. and Tanaka, K. (1991). Identification of a fuzzy model and its application to prediction of a complex system. Fuzzy Sets and Systems, 42:315-334. Sugeno, M. and Yasukawa, T. (1991). Linguistic modeling based on numerical data. In Proceedings of IFSA'91, Brussels. Sugeno, M. and Yasukawa, T. (1993). A fuzzy-logic based approach to qualitative modeling. IEEE Transactions on Fuzzy Systems, 1(1):7-31. Svarer, C., Hansen, L., Larsen, J., and Rasmussen, C. (1993). Designer networks for time series processing. In et al., C. K., editor, Proceedings of the 1993 IEEE Workshop on Neural Networks for Signal Processing (NNSP '93 ), pages 78-87, Baltimore. Takagi, T. and Sugeno, M. (1985). Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on Systems, Man, and Cybernetics, SMC-15:116-132. Tino, P. and Koteles, M. (1999). Extracting finite-state representations from recurrent neural networks trained on chaotic symbolic sequences. IEEE Transactions on Neural Networks, 10(2):284-302. Togneri, R., Forrokhi, D., Zhang, Y., and Attikiouzel, Y. (1992). A comparison of the LBG, MLP, SOM and GMM algorithms for vector quantisation and clustering analysis. In Proceedings Fourth Australian International Conference of Speech Science and Technology, Brisbane, Australia, pages 173-177. Tong, R. (1978). Synthesis of fuzzy models for industrial processed: Some recent results. International Journal General Systems, 4:143-162. Tong, R. (1980). The evaluation of fuzzy models derived from experimental data. Fuzzy Sets and Systems, 4:1-12. Torreele, J. (1991). Temporal processing with recurrent networks: An evolutionary approach. In Belew, R. K. and Booker, L. B., editors, Proceedings of the Fourth International Conference on Genetic Algorithms, pages 555-561. Towell, G. and Shavlik, J. (1993). The extraction of refined rules from knowledge based neural networks. Machine Learning, 131:71-101. Towell, G. G., Shavlik, J. W., and Noordwier, M. 0. (1990). Refinement of approximate domain theories by knowledge-based neural networks. In Proceedings of the Eighth National Conference on Artificial Intelligence (AAA1-90), volume 2, pages 861-866. Uchino, E. and Yamakawa, T. (1995). System modeling by a neo-fuzzy neuron with applications to acoustic and chaotic systems. International Journal on Artificial Intelligence Tools, 1(2):73-91. Umano, M., Fukunaka, S., Hatono, I., and Tamura, H. (1997). Acquisition of fuzzy rules using fuzzy neural networks with forgetting. In 1997 International Conference on Neural Networks (ICNN '97), volume 4, pages 2369-2373. IEEE Press. Valdes, J. J. (2002). Time series discovery with similarity-based neuro-fuzzy networks and evolutionary algorithms. In Proceedings of IJCNN 2002, pages 2345-2350. Vaughn, M., Ong, E., and Cavill, S. (1993). Direct rule extraction form a MLP network that performs whole life assurance risk assessment. In Usui, S. and Omori, T., editors, Proceedings of the Fifth International Conference on Neural Information Processing, volume 2, pages 909-914. IOS Press. Vesanto, J. (1997). Using the SOM and local models in time-series prediction. In Proceedings of WSOM'97, Workshop on Self-Organizing Maps, Espoo, Finland, June 4-6, pages 209-214. Helsinki University of Technology, Neural Networks Research Centre, Espoo, Finland. Villee, C. A. (1972). Biology. W.B. Saunders Company, 6th edition. Waibel, A., Hanazawa, T., Hinton, G., and Shikano, K. (1989). Phoneme recognition using time-delay neural networks. IEEE Transactions of Acoustics, Speech and Signal Processing, 37(3):328-339. Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., and Lang, J. (1988). Phoneme recognition: neural networks versus hidden markov models. In Proceedings ICASSP, New York, NY, pages 107-110. Wang, L. and Langari, R. (1996). Complex systems modeling via fuzzy logic. IEEE Transactions on Systems, Man, and Cybernetics, 26(1):100-106. Wang, X. (2001). "on-line" time series prediction system—EFuNN-T. In Proceedings of the Fifth Biannual Conference on Artificial Neural Networks and Expert Systems (ANNES2001), pages 82-86. Watrous, R. and Shastri, L. (1987). Learning phonetic features using connectionist networks: An experiment in speech recognition. In Proceedings of the 10th International Conference on Artificial Intelligence, pages 351-354. Watrous, R. L., Ladendorf, B., and Kuhn, G. (1990). Complete gradient optimization of a recurrent network applied to /b/,/d/,/g/ discrimination. Journal of the Acoustical Society of America, 87(3):1301-1309. Watts, M. (1999a). Evolving connectionist systems for biochemical applications. In Kasabov, N. and Ko, K., editors, Emerging Knowledge Engineering and Connectionist-based Systems (Proceedings of the ICONIP/ANZIIS/ANNES'99 Workshop "Future directions for intelligent systems and information sciences", Dunedin, 22-23 November 1999), pages 147-151. University of Otago Press. Watts, M. (1999b). An investigation of the properties of evolving fuzzy neural networks. In Proceedings of ICONIP'99, November 1999, Perth, Australia, pages 217-221. Watts, M. and Kasabov, N. (1998). Genetic algorithms for the design of fuzzy neural networks. In Usui, S. and Omori, T., editors, The Fifth International Conference on Neural Information Processing, volume 2, pages 793-796, Kitakyushu, Japan. IOS Press. Watts, M. and Kasabov, N. (1999). Spatial-temporal adaptation in evolving fuzzy neural networks for on-line adaptive phoneme recognition. Technical Report TR99/03, Department of Information Science, University of Otago. Watts, M. and Kasabov, N. (2000). Simple evolving connectionist systems and experiments on isolated phoneme recognition. In Proceedings of the first IEEE conference on evolutionary computation and neural networks, San Antonio, May 2000, pages 232-239. IEEE Press. Watts, M., Major, L., and Tate, W. (2002). Evolutionary optimisation of MLP for modelling protein synthesis termination signal efficiency. In Proceedings of the Congress on Evolutionary Computation (CEC) 2002, pages 606-610. Watts, M., Major, L., Tate, W., and Kasabov, N. (2001). Neural network analysis of protein synthesis termination signal efficiency. In Proceedings of International Conference on Neural Information Processing (ICONIP) 2001, Shanghai, China, pages 975-980. Watts, M. J. and Kasabov, N. K. (2002). Evolutionary optimisation of evolving connectionist systems. In Proceedings of the Congress on Evolutionary Computation (CEC) 2002, pages 606-610. Weigund, A., Rumelhart, D., and Huberman, B. (1991). Generalization by weight-elimination with application to forecasting. In Lippmann, R., Moody, J., and Touretzky, D., editors, Advances in Neural information Processing Systems (3), pages 875-882. Morgan Kaufmann, San Mateo. Weiss, S. and Kapouleas, I. (1991). An empirical comparison of pattern recognition, neural nets and machine learning classification methods. In Proceedings of the 11th International Joint Conference on Artificial Intelligence, Detroit, pages 781-787. Whitfort, T., Matthews, C., and Jagielska, I. (1995). Automated knowledge acquisition for a fuzzy classification problem. In Kasabov, N. K. and Coghill, G., editors, The Second New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, pages 227-230. IEEE Computer Society Press. Widrow, B. (1962). Self Organizing Systems, chapter Generalization and information storage in networks of adaline, pages 435-461. Sparta. Widrow, B., Rumelhart, D. E., and Lehr, M. A. (1994). Neural networks: Applications in industry, business and science. Communications of the ACM, 37(3):93-105. Wolpert, D. H. and Macready, W. G. (1995). No free lunch theorems for search. Technical Report SFI-TR-95-02- 010, Santa Fe Institute. Woodford, B. (2001). Comparative analysis of the EFuNN and the support vector machine models for the classification of horticulture data. In Proceedings of the Fifth Biannual Conference on Artificial Neural Networks and Expert Systems (ANNES2001), pages 70-75. Woodford, B. J. and Kasabov, N. K. (2001). Ensembles of EFuNNs: An architecture for a multi module classifier. In The proceedings of FUZZ-IEEE'2001. The 10th IEEE International Conference on Fuzzy Systems, December 2-5 2001, Melbourne, Australia, pages 441-445. Wright, A. H. (1991). Genetic algorithms for real parameter optimization. In Rawlins, G. J. E., editor, Foundations of Genetic Algorithms, pages 205-218. Wu, C. and McLarty, J. (2000). Neural Networks and Genome Informatics. Elsevier Health Sciences. Wu, Z. and Li, W. (1995). Optimization of floor plate structure in railway passenger train by genetic algorithm. In Proceedings of ICONIP 95, volume 1, pages 347-350. Xu, C. and Lu, Y. (1987). Fuzzy model identification and self-learning for dynamic systems. IEEE Transactions of Systems, Man, and Cybernetics, 17:683-689. Yang, J. T., Huang, H.-D., and Horng, J.-T. (2002). Devising a cost effective baseball scheduling by evolutionary algorithms. In Proceedings of CEC 2002, pages 1660-1665. Yang, X. and Furuhashi, T. (1993). A basic study on apparel CAD using a fuzzy neural network. In Proceedings of 1993 International Joint Conference on Neural Networks, pages 701-704. Yao, X. (1997). A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks, 8(3):694-713. Yao, X. (1999). Evolving artificial neural networks. Proceedings of the IEEE, pages 1423-1447. Yao, X. and Liu, Y. (1996a). Ensemble structure of evolutionary artificial neural networks. In Proceedings of the Third IEEE Conference on Evolutionary Computation (ICEC '96), pages 659-664, Nagoya, Japan. Yao, X. and Llu, Y. (1996b). Evolving artificial neural network through evolutionary programming. In Fogel, L. J., Angeline, P. J., and Back, T., editors, Evolutionary Programming V, pages 257-266. MIT Press. Yao, X. and Liu, Y. (1998). Making use of population information in evolutionary artificial neural networks. IEEE Transactions on Systems, Man and Cybernetics, 28(3):417-425. Yen, G. G. and Lu, H. (2002). Hierarchical rank density genetic algorithm for radial-basis function neural network design. In Proceedings of CEC 2002, pages 25-30. Yu, C.-C. and Liu, B.-D. (2002). A backpropagation algorithm with adaptive learning rate and momentum coefficient. In Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2002, pages 1218-1223. Zadeh, L. (1965). Fuzzy sets. Information and Control. 8:338-353. Zhao, Q. (1997). A co-evolutionary algorithm for neural network learning. In Proceedings of the 1997 IEEE International Conference on Neural Networks, volume 1, pages 432-437. IEEE Press. Zhao, Q. and Higuchi, T. (1996). Evolutionary learning of nearest-neighbour MLP. IEEE Transactions on Neural Networks, 7(3):762-767. ZISC Manual (2002). ZISC Zero Instruction Set Computer. Silicon Recognition, Inc., version 4.2 edition. http://www.silirec.com. | en_NZ |