Data Mining in Finance

by Kovalerchuk, Boris; Vityaev, Evgenii

• ISBN13: 9780792378044
• ISBN10: 0792378040
• Format: Hardcover
• Copyright: 2000-03-01
• Publisher: Kluwer Academic Pub

Summary

Data Mining in Finance presents a comprehensive overview of major algorithmic approaches to predictive data mining, including statistical, neural networks, ruled-based, decision-tree, and fuzzy-logic methods, and then examines the suitability of these approaches to financial data mining. The book focuses specifically on relational data mining (RDM), which is a learning method able to learn more expressive rules than other symbolic approaches. RDM is thus better suited for financial mining, because it is able to make greater use of underlying domain knowledge. Relational data mining also has a better ability to explain the discovered rules - an ability critical for avoiding spurious patterns which inevitably arise when the number of variables examined is very large. The earlier algorithms for relational data mining, also known as inductive logic programming (ILP), suffer from a relative computational inefficiency and have rather limited tools for processing numerical data. Data Mining in Finance introduces a new approach, combining relational data mining with the analysis of statistical significance of discovered rules. This reduces the search space and speeds up the algorithms. The book also presents interactive and fuzzy-logic tools for 'mining' the knowledge from the experts, further reducing the search space. Data Mining in Finance contains a number of practical examples of forecasting S&P 500, exchange rates, stock directions, and rating stocks for portfolio, allowing interested readers to start building their own models. This book is an excellent reference for researchers and professionals in the fields of artificial intelligence, machine learning, data mining, knowledge discovery, and applied mathematics.

Table of Contents

Foreword	xi
Gregory Piatetsky-Shapiro
Preface	xiii
Acknowledgements	xv
The Scope and Methods of the Study
Introduction	1	(2)
Problem definition	3	(1)
Data mining methodologies	4	(5)
Parameters	4	(2)
Problem ID and profile	6	(1)
Comparison of intelligent decision support methods	7	(2)
Modern methodologies in financial knowledge discovery	9	(3)
Deterministic dynamic system approach	9	(1)
Efficient market theory	10	(1)
Fundamental and technical analyses	11	(1)
Data mining and database management	12	(2)
Data mining: definitions and practice	14	(3)
Learning paradigms for data mining	17	(2)
Intellectual challenges in data mining	19	(2)
Numerical Data Mining Models with Financial Applications
Statistical, autoregression models	21	(9)
ARIMA models	22	(3)
Steps in developing ARIMA model	25	(2)
Seasonal ARIMA	27	(1)
Exponential smoothing and trading day regression	28	(1)
Comparison with other methods	28	(2)
Financial applications of autoregression models	30	(2)
Instance--based learning and financial applications	32	(4)
Neural networks	36	(4)
Introduction	36	(2)
Steps	38	(1)
Recurrent networks	39	(1)
Dynamically modifying network structure	40	(1)
Neural networks and hybrid systems in finance	40	(2)
Recurrent neural networks in finance	42	(2)
Modular networks and genetic algorithms	44	(3)
Mixture of neural networks	44	(1)
Genetic algorithms for modular neural networks	45	(2)
Testing results and the complete round robin method	47	(11)
Introduction	47	(1)
Approach and method	47	(5)
Multithreaded implementation	52	(2)
Experiments with SP500 and neural networks	54	(4)
Expert mining	58	(8)
Interactive learning of monotone Boolean functions	66	(5)
Basic definitions and results	66	(1)
Algorithm for restoring a monotone Boolean function	67	(2)
Construction of Hansel chains	69	(2)
Rule-Based and Hybrid Financial Data Mining
Decision tree and DNF learning	71	(17)
Advantages	71	(1)
Limitation: size of the tree	72	(9)
Constructing decision trees	81	(3)
Ensembles and hybrid methods for decision trees	84	(3)
Discussion	87	(1)
Decision tree and DNF learning in finance	88	(7)
Decision-tree methods in finance	88	(1)
Extracting decision tree and sets of rules for SP500	89	(4)
Sets of decision trees and DNF learning in finance	93	(2)
Extracting decision trees from neural networks	95	(2)
Approach	95	(1)
Trepan algorithm	96	(1)
Extracting decision trees from neural networks in finance	97	(5)
Predicting the Dollar-Mark exchange rate	97	(2)
Comparison of performance	99	(3)
Probabilistic rules and knowledge-based stochastic modeling	102	(10)
Probabilistic networks and probabilistic rules	103	(3)
The naive Bayes classifier	106	(1)
The mixture of experts	107	(1)
The hidden Markov model	108	(3)
Uncertainty of the structure of stochastic models	111	(1)
Knowledge-based stochastic modeling in finance	112	(3)
Markov chains in finance	112	(2)
Hidden Markov models in finance	114	(1)
Relational Data Mining (RDM)
Introduction	115	(3)
Examples	118	(5)
Relational data mining paradigm	123	(4)
Challenges and obstacles in relational data mining	127	(2)
Theory of RDM	129	(11)
Data types in relational data mining	129	(1)
Relational representation of examples	130	(5)
First-order logic and rules	135	(5)
Background knowledge	140	(6)
Arguments constraints and skipping useless hypotheses	140	(1)
Initial rules and improving search of hypotheses	141	(3)
Relational data mining and relational databases	144	(2)
Algorithms: FOIL and FOCL	146	(5)
Introduction	146	(1)
FOIL	147	(3)
FOCL	150	(1)
Algorithm MMDR	151	(15)
Approach	151	(3)
MMDR algorithm and existence theorem	154	(5)
Fisher test	159	(3)
MMDR pseudocode	162	(3)
Comparison of FOIL and MMDR	165	(1)
Numerical relational data mining	166	(3)
Data types	169	(10)
Problem of data types	169	(5)
Numerical data type	174	(1)
Representative measurement theory	174	(1)
Critical analysis of data types in ABL	175	(4)
Empirical axiomatic theories: empirical contents of data	179	(10)
Definitions	179	(2)
Representation of data types in empirical axiomatic theories	181	(5)
Discovering empirical regularities as universal formulas	186	(3)
Financial Applications of Relational Data Mining
Introduction	189	(2)
Transforming numeric data into relations	191	(2)
Hypotheses and probabilistic ``laws''	193	(3)
Markov chains as probabilistic ``law'' in finance	196	(3)
Learning	199	(3)
Method of forecasting	202	(2)
Experiment 1	204	(8)
Forecasting Performance for hypotheses H1-H4	204	(3)
Forecasting performance for a specific regularity	207	(2)
Forecasting performance for Markovian expressions	209	(3)
Experiment 2	212	(1)
Interval stock forecast for portfolio selection	213	(2)
Predicate invention for financial applications: calendar effects	215	(3)
Conclusion	218	(1)
Comparison of Performance of RDM and other methods in financial applications
Forecasting methods	219	(1)
Approach: measures of performance	220	(2)
Experiment 1: simulated trading performance	222	(3)
Experiment 1: comparison with ARIMA	225	(2)
Experiment 2: forecast and simulated gain	227	(1)
Experiment 2: analysis of performance	227	(2)
Conclusion	229	(2)
Fuzzy logic approach and its financial applications
Knowledge discovery and fuzzy logic	231	(4)
``Human logic'' and mathematical principles of uncertainty	235	(4)
Difference between fuzzy logic and probability theory	239	(1)
Basic concepts of fuzzy logic	240	(8)
Inference problems and solutions	248	(4)
Constructing coordinated contextual linguistic variables	252	(14)
Examples	252	(7)
Context space	259	(3)
Acquisition of fuzzy sets and membership function	262	(3)
Obtaining linguistic variables	265	(1)
Constructing coordinated fuzzy inference	266	(12)
Approach	266	(2)
Example	268	(2)
Advantages of ``exact complete'' context for fuzzy inference	270	(8)
Fuzzy logic in finance	278	(7)
Review of applications of fuzzy logic in finance	278	(3)
Fuzzy logic and technical analysis	281	(4)
References	285	(14)
Subject Index	299