Nội dung

Power Systems Abhisek Ukil Intelligent Systems and Signal Processing in Power Engineering Abhisek Ukil Author Intelligent Systems and Signal Processing in Power Engineering With 239 Figures and 36 Tables Author Dr. Abhisek Ukil ABB Corporate Research Segelhofstrasse 1K CH-5405 Baden-Daetwill Switzerland abhisek.ukil@ch.abb.com abhiukil@yahoo.com ISSN 1612-1287 ISBN 978-3-540-73169-6 Springer Berlin Heidelberg New York Library of Congress Control Number: 2007929722 Matlab is a registered trademark of Mathworks Inc. In short, no guaraentees, whatsoever, are given for the example computer programs provided in this book. They are intended for demonstration purpose only. The author or the publisher would not be responsible for any consequences or damages of any sort from the usage of the programs or any other relevant ideas from the book. This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com c Springer-Verlag Berlin Heidelberg 2007  The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Integra Software Services Pvt. Ltd., India Cover Design: deblik, Berlin Printed on acid-free paper SPIN: 11884910 60/3180/Integra 5 4 3 2 1 0 Dedicated to all my teachers who enabled me to write this book, and my family & friends for supporting me all along. Preface Power engineering is truly one of the main pillars of the electricity-driven modern civilization. And over the years, power engineering has also been a multidisciplinary field in terms of numerous applications of different subjects. This ranges from linear algebra, electronics, signal processing to artificial intelligence including recent trends like bio-inspired computation, lateral computing and the like. Considering the reasons behind this, in one hand, we have vast variety of application sub-domains in power engineering itself; on the other hand, problems in these sub-domains are complex and nonlinear, requiring other complementary techniques/fields to solve them. Therefore, there is always the need of bridging these different fields and the power engineering. We often encounter the problem of distributed and scattered nature of these different disciplines while working in various sub-domains of power engineering, trying to apply some other useful techniques for some specific problem. Oftentimes, these are not direct field of work for many power engineers/researchers, but we got to use them. This book is urged by that practical need. As the name suggests, the book looks into two major fields (without undermining others!) used in modern power systems. These are the intelligent systems and the signal processing. These broad fields include many topics. Some of the common and useful topics are addressed in this book. In this book, the intelligent systems section comprises of fuzzy logic, neural network and support vector machine. Fuzzy logic, driven by practical humanoid knowledge incorporation, has been a powerful technique in solving many nonlinear, complex problems, particularly in the field of control engineering. Neural network, on the other hand, is inspired by the biological neuronal assemblies that enable the animal kingdom (including us!) to perform complex tasks in everyday life. Support vector machine is a relatively newer field in machine learning (neural network also falls in this category) domain. It augments the robustness of machine learning scenario with some new concepts and techniques. Although there are many more extensions of the concept of machine learning and intelligent systems, we confine ourselves to these three topics in this book. We look at some theories on them without assuming much particular background. Following the theoretical basics, we study their applications in various problems in power engineering, like, load forecasting, phase balancing, disturbance analysis and so on. Purpose of these, so called, application studies in power engineering is to demonstrate how we can utilize vii viii Preface the theoretical concepts. Finally, some research information are included, showing utilizations of these fields in various power systems domains as a starting point for further futuristic studies/research. In the second part, we look into the signal processing which is another universal field. Whenever and oftentimes we encounter signals, we need to process them! Power engineering and its enormous sub-fields are no exceptions, providing us with ample voltage, current, active/reactive power signals, and so forth. Therefore, we look in this section about the basics of the system theory, followed by fundamentals of different signal processing transforms with examples. After that, we look into the digital signal processing basics including the sampling technique and the digital filters which are the ultimate (signal) processing tools. Similar to the intelligent systems part, here also the theoretical basics are substantiated by some of the applications in power engineering. These applications are of two types: full application studies explained like in-depth case-studies, and semi-developed application ideas with scope for further extension. This also ends up with pointers to further research information. As a whole, the book looks into the fields of intelligent systems and signal processing from theoretical background and their application examples in power systems altogether. It has been kind of hard to balance the theoretical aspects as each of these fields are vast in itself. However, efforts have been made to cover the essential topics. Specific in-depth further studies are pointed to the dedicated subject intensive resources for interested readers. Application studies are chosen with as much real implications as possible. Finally, the book is a small effort to bridge and put together three great fields as a composite resource: intelligent systems, signal processing and power engineering. I hope this book will be helpful to undergraduate/graduate students, researchers and engineers, trying to solve power engineering problems using intelligent systems and signal processing, or seeking applications of intelligent systems and signal processing in power engineering. April, 2007 Abhisek Ukil Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 About the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Prospective Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Organization of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Book Chapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Chapter Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 2 2 3 2 Fuzzy Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 History and Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Pros and Cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Fuzzy Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Linguistic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 Fuzzy Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.4 Classical Set Theory vs. Fuzzy Set Theory . . . . . . . . . . . . . . . . . . 2.2.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Fuzzy System Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Fuzzification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Fuzzy Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.3 Defuzzification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Application Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Brake Test Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Fuzzification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Fuzzy Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.4 Defuzzification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Feeder Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 Proposed Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.3 Designing Fuzzy Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 5 6 7 8 8 9 12 24 27 28 29 29 34 35 36 37 40 42 45 46 46 47 48 49 51 55 ix x Contents 2.6 Energy Efficient Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.1 Project Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.2 Designing Fuzzy Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.3 Final Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Use of Fuzzy Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Demand Side Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.1 Load Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.2 Energy Consumption Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 Power Flow Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9.1 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 Research Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.1 General Fuzzy Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.2 Fuzzy Logic and Power Engineering . . . . . . . . . . . . . . . . . . . . . . . 2.10.3 Electrical Load Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.4 Fault Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.5 Power Systems Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.6 Distance Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.7 Relay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.8 Power Flow Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.9 Power Systems Equipments & Control . . . . . . . . . . . . . . . . . . . . . 2.10.10 Frequency Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.11 Harmonic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.12 Power Systems Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.13 Power Systems Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.14 Power Systems Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.15 Power Systems Stabilizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.16 Power Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.17 Renewable Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.18 Transformers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.19 Rotating Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.20 Energy Economy, Market & Management . . . . . . . . . . . . . . . . . . 2.10.21 Unit Commitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.22 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10.23 Power Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 56 56 57 58 58 58 59 59 59 60 61 61 61 62 62 62 63 63 64 65 65 65 66 67 67 68 68 69 69 70 71 71 71 72 73 73 73 74 3 Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 History and Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Pros and Cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 75 76 76 78 Contents xi 3.2 Artificial Neural Networks (ANN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.2.1 Basic Structure of the Artificial Neural Networks . . . . . . . . . . . . 78 3.2.2 Structure of a Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.2.3 Transfer Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.2.4 Architecture of the ANN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.2.5 Steps to Construct a Neural Network . . . . . . . . . . . . . . . . . . . . . . . 85 3.3 Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.3.1 The Delta Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.3.2 Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 3.3.3 Energy Equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.3.4 The Backpropagation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.3.5 The Hebb Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 3.4 Different Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 3.4.1 Perceptron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 3.4.2 Multilayer Perceptrons (MLP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 3.4.3 Backpropagation (BP) Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 3.4.4 Radial Basis Function (RBF) Network . . . . . . . . . . . . . . . . . . . . . 96 3.4.5 Hopfield Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 3.4.6 Adaline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.4.7 Kohonen Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.4.8 Special Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.4.9 Special Issues in NN Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 3.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 3.5.1 Linear Network: Boolean Logic Operation . . . . . . . . . . . . . . . . . . 115 3.5.2 Pattern Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 3.5.3 Incomplete Pattern Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 3.6 Load Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 3.6.1 Data set for the Application Study . . . . . . . . . . . . . . . . . . . . . . . . . 128 3.6.2 Use of Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 3.6.3 Linear Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 3.6.4 Backpropagation Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 3.6.5 Radial Basis Function Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 3.7 Feeder Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 3.7.1 Phase Balancing Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 3.7.2 Feeder Reconfiguration Technique . . . . . . . . . . . . . . . . . . . . . . . . . 139 3.7.3 Neural Network-based Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 3.7.4 Network Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 3.7.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 3.8 Fault Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 3.8.1 Simple Ground Fault Classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 3.8.2 Advanced Fault Classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 xii Contents 3.9 Advanced Load Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 3.10 Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 3.11 Research Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 3.11.1 General Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 3.11.2 Neural Network and Power Engineering . . . . . . . . . . . . . . . . . . . . 148 3.11.3 Electrical Load Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 3.11.4 Fault Locator & Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 3.11.5 Power Systems Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 3.11.6 Harmonic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 3.11.7 Transient Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 3.11.8 Power Flow Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 3.11.9 Power Systems Equipments & Control . . . . . . . . . . . . . . . . . . . . . 153 3.11.10 Power Systems Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 3.11.11 Power Systems Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 3.11.12 Power Systems Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 3.11.13 Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 3.11.14 Renewable Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 3.11.15 Transformers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 3.11.16 Rotating Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 3.11.17 Power Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 3.11.18 State Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 3.11.19 Energy Market . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 3.11.20 Power Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 4 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 4.1.1 History and Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 4.1.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 4.1.3 Pros and Cons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 4.2 Basics about Statistical Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 4.2.1 Machine Learning & Associated Problem . . . . . . . . . . . . . . . . . . . 164 4.2.2 Statistical Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 4.2.3 Vapnik Chervonenkis (VC) Dimension . . . . . . . . . . . . . . . . . . . . . 167 4.2.4 Structural Risk Minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 4.3 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 4.3.1 Linear Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 4.3.2 Optimal Separating Hyperplane . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 4.3.3 Support Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 4.3.4 Convex Optimization Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 4.3.5 Overlapping Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 4.3.6 Nonlinear Classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 4.3.7 Kernel Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 4.3.8 Support Vector Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193