Nội dung

Adaptive Wireless Tranceivers L. Hanzo, C.H. Wong, M.S. Yee Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-470-84689-5 (Hardback); 0-470-84776-X (Electronic) Neural Network Based Equalization In this chapter, we will give an overview of neural network based equalization. Channel equalization can be viewed as a classification problem. The optimal solution to this classification problem is inherentlynonlinear. Hence we will discuss, how the nonlinear structureof the artificial neural network can enhancethe performance of conventional channel equalizers and examinevarious neural network designs amenableto channel equalization, suchas the socalled multilayer perceptron network [236-2401, polynomial perceptron network 1241-2441 and radial basis function network 185,245-2471. We will examine a neural network structure referred to as the Radial Basis Function (RBF) network in detail in the context of equalization. As further reading, the contribution by Mulgrew [248] provides an insightful briefing on applying RBF network for both channel equalization and interference rejection problems. Originally RBF networks were developed for the generic problem of data interpolation in a multi-dimensional space 1249,2501. We will describe the RBF network in general and motivate its application. Before we proceed, our forthcoming section will describe the discrete time channel model inflicting intersymbol interference thatwill be used throughout this thesis. 8.1 Discrete Time Model for Channels Exhibiting Intersymbol Interference A band-limited channel that results in intersymbol interference (ISI) can be represented by a discrete-time transversal filter having a transfer function of n=O + where f n is the nth impulse response tap of the channel and L 1 is the length of the channel impulse response (CIR). In this context, the channel represents the convolution of 299 300 CHAPTER 8. NEURAL NETWORK BASED EOUALIZATION t Figure 8.1: Equivalent discrete-time model of a channel exhibiting intersymbol interference and expe- riencing additive white Gaussian noise. the impulse responsesof the transmitter filter, the transmission medium and the receiver filter. In our discrete-time model discrete symbols I , are transmitted to the receiver at a rate of $ symbols per second and the output ‘uk at the receiveris also sampled ata rate of per second. Consequently, as depicted in Figure 8.1, the passage of the input sequence { I k } through the channel results in the channel output sequence{vk} that can be expressed as n=o where { q k } is a white Gaussian noise sequence with zero mean and variance 0:. The number IS1 is L. In general, the sequences {vk}, { I k } , of interfering symbols contributing to the (7,) and { f n } are complex-valued. Again, Figure 8.1 illustrates the model of the equivalent discrete-time system corruptedby Additive White Gaussian Noise (AWGN). 8.2 Equalization as a Classification Problem In this section we will show that the characteristics of the transmitted sequence can be exploited by capitalising on the finite state nature of the channel and by considering the equalwas first expounded ization problem as a geometric classification problem. This approach by Gibson, Siu and Cowan [237], who investigated utilizing nonlinear structures offered by Neural Networks (NN)as channel equalisers. of logical ones We assume that the transmitted sequence is binary with equal probability and zeros in order to simplify the analysis. Referring to Equation 8.2 and using the notation 8.2. EQUALIZATION A AS CLASSIFICATION PROBLEM 301 Vk I 1 l Equaliser Decision Function Figure 8.2: Linear m-tap equalizer schematic. of Section 8.1, the symbol-spaced channel output is defined by L where { q k } is the additive Gaussian noise sequence, { f i L } , n = 0, l!.. . , L is the CIR, { I I ; } is the channel input sequence and {Vk} is the noise-free channel output. The mth order equaliser, as -illustrated in Figure 8.2, has m taps as well as a delay of 7 , and it produces an estimate Ik-T of the transmitted signal IkPT. The delay T is due to the precursor section of the CIR, sinceit is necessary to facilitate the causal operation of the equalizer by supplying the past and future received samples, when generating the delayed detected symbol IkP7. Hence the required lengthof the decision delay is typically the length of the CIR's precursor section, since outside this interval the CIR is zero and therefore the equaliser does not have to take into account any other received symbols. The channel output observed by the linear mth order equaliser can be writtenin vectorial form as [ vk vk Vk-1 ... VkPm+l ,'l (8.4) and hence we can say that the equalizer has an m-dimensional channel output observation space. For a CIR of length L 1, there are hence n, = 2L+m possible combinations of the binary channel input sequence + II,= [ II, that produce 71, = 2L+7n different Vk = [ Ik-1 ... Ik-m-L+1 IT (8.5) possible noise-free channel output vectors Vk Vk-1 ... Vk-m+l ]T . (8.6) The possible noise-free channel output vectors Vk or particular points in the observation space will be referred to as the desired channel states. Expounding further, we denote each of the n, = 2L+m possible combinationsof the channel input sequenceIk of length L f m symbols 302 CHAPTER S. NEURAL NETWORK BASED EQUALIZATION as si, 1 5 i 5 R, = 2L+Tn, where the channel input statesi determines the desired channel output state ri, i = 1,2, . . . , n,$= 2L+m. This is formulated as: v k = r, if I k i = 1 , 2 , . . . , n,. = S,, The desired channel output states can be partitioned into two classes according to the binary value of the transmitted symbolI k P r , as seen below: and We can denote the desired channel output states according to these two classes as follows: rt where the quantities nf and 71.; represent the number of channel states and r; in the set K:,7 and V&, respectively. The relationship between the transmitted symbol I , and the channel output U k can also be written in a compactform as: (8.10) where vkis an m-component vector that represents the AWGN sequence, is the noise-free channel output vector andF is an m x ( m + L ) CIR-related matrix in the form of with f 3 , j = 0 , . . . , L being the CIR taps. Below we demonstrate the concept of finite channel states in a two-dimensional output observation space ( m = 2) using a simple two-coefficient channel ( L = l), assumming the CIR of: F ( z ) = 1 + 0.52-l. Thus, F = [ 1, Vk = [ ijk ijk-1 ]T (8.12) and 11, = [ I,+ 1,-l 1k-2 ]T . All the possible combinations of the transmitted binary symbol I k and the noiseless channel outputs cl;, i j k - 1 , are listed in Table8.1. 8.2. EQUALIZATION CLASSIFICATION PROBLEM AS A 303 2 - 1 - - l -3 ' -3 Figure 8.3: The noiseless BPSK-related channel states V k = ri and the noisy channel outputs Vk of a Gaussian channel having a CIR of F ( z ) = 1 + 0 . 5 in~ a two-dimensional ~ ~ observation space. The noise variance a: = 0.05, the number of noisy received Vk samples output by the channel and input to the equalizer is 2000 and the decision delay is T = 0. The linear decision boundary separates the noisy received vk clusters that correspond to I k P r = +l from those that correspond toIk--r = -1. CHAPTER 8. NEURAL EOUALIZATION BASED NETWORK 304 II, -1 -1 -1 -1 +l +l +l +l Ik,-l -1 -1 +l +l -1 -1 +l +l Ik-2 ‘(?k -1 +l -1 +l -1 +l -1 +l -1.5 -1.5 -1.5 -0.5 -0.5 +0.5 +0.5 +1.5 +1.5 -0.5 +0.5 +1.5 -1.5 -0.5 +OS +IS Table 8.1: Transmitted signal and noiseless channel states for the CIR of F ( z ) = 1 equalizer order of m = 2. + 0 . 5 ~ and ~ ’ an Figure 8.3 shows the 8 possible noiseless channel states VI, for a BPSK modem and the noisy channel output vk in the presence of zero mean AWGN with variance 0; = 0.05. It is seen that the observation vector VI,forms clusters and the centroids of these clusters are the noiseless channel states rz.The equalization problem hence involves identifying the regions within the observation space spanned by the noisy channel output v k that correspond to the transmitted symbol of either II,= +l or 1, = -1. A linear equalizer performs the classification in conjunction with a decision device, which is often a simple sign function. The decision boundary, as seen in Figure 8.3, is constituted as it is by the locus of all values of vk, where the output of the linear equalizer is zero demonstrated below. For example, for a two tap linear equalizer having tap coefficients ( - 1 and Q,at the decision boundary wehave: and (8.14) gives a straight line decision boundary as shown in Figure 8.3, which divides the observation space into two regions corresponding to II, = +l and 1, = -1. In general, the linear equalizer can only implement a hyperplane decision boundary, which in our two-dimensional example was constituted by a line. This is clearly a non-optimum classification strategy, as our forthcoming geometric visualization will highlight. For example, we can see in Figure 8.3 ] associated with the II, = +l decision is closer to the dethat the point V = [ 0.5 -0.5 cision boundary than the point V = [ -1.5 -0.5 ] associated with the II, = -1 decision. Therefore, in the presenceof noise, there is a higher probabilityof the channel output centred ] tobewronglydetectedas I k = -1, thanthat of thechannel at point V = [ 0.5 -0.5 output centred around V = [ - 1.5 -0.5 ] being incorrectly detected as I , = +l. Gibson et ul. [237] have shown examples of linearly non-separable channels, when the decision delay is zero and the channel is of non-minimum phase nature. The linear separability of the channel depends on the equalizer order,m , on the delayr and in situations where the channel characteristics are time varying, it may not be possible to specify values of m and r , which will guarantee linear separability. 8.3. INTRODUCTION TO NEURAL NETWORKS 305 According to Chen,Gibson and Cowan [241], the above shortcomings of the linear equalizer are circumvented by a Bayesian approach [25 l ] to obtaining an optimal equalizationsolution. In this spirit, for an observed channel output vector v k , if the probability that it was caused by I k P T = + l exceeds the probability that it was caused by I k P T = -1, then we should decide in favour of +l and vice versa. Thus, the optimal Bayesian equalizer solution is defined as [241]: (8.15) where the optimal Bayesian decision function f s a y e s (based . ) , on the difference of the associated conditional density functions is given by [85]: where p+ and p i is the a priori probability of appearance of each desired state r t E Vz,T and r i E V;,T, respectively and p ( . ) denotes the associated probability density function. The quantities nf and n; represent the number of desired channel states in VA,, and V;,T, respectively, which are defined implicitly in Figure 8.3. If the noise distribution is Gaussian, Equation 8.16 can be rewritten as: j=1 Again, the optima1 decision boundary is the locus of all values of Vk, where the probability Ik-T = +l given a value v k is equal to the probabilityI k P T = -1 for the same v k . In general, the optimal Bayesian decision boundary is a hyper-surface, rather than just a hyper-plane in the m-dimensional observation space and the realization of this nonlinear boundary requires a nonlinear decision capability. Neural networks provide this capability and the following section will discuss the various neural network structures that have been investigated in the context of channel equalization, while also highlighting the learning algorithms used. 8.3 Introduction to NeuralNetworks 8.3.1 Biological and Artificial Neurons The human brain consists of a dense interconnection of simple computational elements referred to as neurons. Figure 8.4(a) shows a network of biological neurons. As seen in the CHAPTER 8. RASED NETWORK NEURAL 306 W ( > Apical dendrlte -&on-# (Initla1segment) EQUALIZATION \ Basal dendrlte Inputs (a) Anatomy of a typical biological neuron, from Kandel [252] l *\ \ 4- \ \ Activation function -.2 ’ / - (b) An artificial neuron (jth-neuron) Figure 8.4: Comparison between biological and artificial neurons. figure, the neuron consists of a cell body - which provides the information-processing functions - and of the so-called axon with its terminal fibres. The dendrites seenin the figure are the neuron’s ‘inputs’, receiving signals from other neurons. These input signals may cause the neuron tofire, i.e. to producea rapid, short-term change in the potential difference across the cell’s membrane. Input signals to the cell may be excitatory, increasing the chances of neuron firing, or inhibitory, decreasing these chances. The axon is the neuron’s transmission line that conducts the potential difference away from the cell body towards the terminal fisynapses, which form either excitatoryor inhibitory bres. This process produces the so-called connections to the dendrites of other neurons, thereby forming a neural network. Synapses mediate the interactions between neurons and enable the nervous system to adapt and react to its surrounding environment. In Artificial Neural Networks (ANN), which mimic the operation of biological neural networks, the processing elements are artificial neurons and their signal processing properties are loosely based on those of biological neurons. Refemng to Figure 8.4(b), the jth-neuron has a set of I synapses or connection links. Each link is characterized by a synaptic weight wiJ, i = l, 2, . . . , I . The weight wij is positive, if the associated synapse is excitatory and it is negative, if the synapse is inhibitory. Thus, signal xi at the input of synapse i, connected to neuron j, is multiplied by the synaptic weight wij. These synaptic weights that store ‘knowledge’ and provide connectivity, are adapted during the learning process. The weighted input signals of the neuron are summed up by an adder. If this summation 8.3. INTRODUCTION TO NEURAL NETWORKS 307 e,, exceeds a so-calledfiring threshold then the neuron fires and issues an output. Otherwise it remains inactive. In Figure 8.4(b) the effect of the firing threshold 0, is represented by a bias, arising from an input which is always ‘on’, corresponding to x0 = 1, and weighted by W O , ~= -Bj = b J . The importance of this is that the bias can be treated as just another weight. Hence, if we have a training algorithm for finding an appropriate set of weights for a network of neurons, designed to perform a certain function,we do not need to consider the biases separately. 2 I .5 1 a v 9 0.5 0 -0.5 -1 m -2 -1 0 1 1.5 1 0.5 a v 0 9 -0.5 -1 -1.5 -2 2 -1 1 2 21 U (a) Threshold activation function 0 (b) Piecewise-linear activation function 1.5 1 0.5 h S v 9 0 -0.5 -1 -1.5 -10 -5 0 5 10 71 (c) Sigmoid activation function Figure 8.5: Various neural activation functions f ( u ) . The activation function f(.)of Figure 8.5 limits the amplitude of the neuron’s output to some permissible range and provides nonlinearities. Haykin[ 2 5 3 ] identifies three basic types of activation functions: 308 CHAPTER 8. NETWORK NEURAL BASED EQUALIZATION 1. Threshold Function. For the threshold function shown in Figure 8.5(a), we have 1 ifv 2 0 0 if21 < O (8.18) ' Neurons using this activation function are referred in the to literatureas theMcCullochPirrs model [253].In this model, the output of the neuron gives the value of 1 if the total internal activity levelof that neuron is nonnegative and 0 otherwise. 2. Piecewise-Linear Function.This neural activation function, portrayedin Figure 8.5(b), is represented mathematically by: i 1, f(v) = v > l -1 > W 21 < - l v, -1, >1 , (8.19) where the amplification factor inside the linear region is assumed to be unity . This activation function approximates a nonlinear amplifier. 3. Sigmoid Function. A commonly used neural activation function in the constructionof artificial neural networks is the sigmoid activation function. It is defined as a strictly increasing function that exhibits smoothness and asymptotic properties, as seen in Figure 8.5(c). An example of the sigmoid function is the hyperbolic tangent function, which is shown in Figure 8.5(c) andit is defined by [253]: (8.20) This activation function is differentiable, which is an important feature in work theory [253]. neural net- The model of the j t h artificial neuron, shown in Figure 8.4(b) can be described in mathematical terms by the following pair of equations: where: I vj = >, W,lX,. (8.22) i=O Having introduced the basic elements of neural networks, we will focus next on the associated network structures or architectures. The different neural network structures yield different functionalities and capabilities. The basic structures will be described in the following section. 8.3.2 NeuralNetworkArchitectures The network's architecture defines the neurons' arrangement in the network. Various neural network architectures have been investigated for different applications, including for example