1. Introduction

As a crucial part,engine performance has great influence on the vehicle. The engine fault rate always ranks first among the vehicle components because of its complex structure and the running conditions. Accordingly, how to detect engine problems is of importance for vehicle inspection and maintenance in automotive workshops. So the development of an expert system for engine diagnosis for the automotive workshop is currently an active research topic. Traditionally, the description of the engine faulty symptom in the automotive workshop is just existence or not. However, this description cannot lead to a high diagnosis performance because the symptom always appearsin different degrees instead of existence or not. Moreover, the engine fault is sometimes a multiple fault problem, so the occurrence of the engine fault should also be represented as probability instead of binary or fuzzy values. In addition, the relationship between faults and symptoms is a complex nonlinearity. In view of the natures of the above problems, an advanced expert system for engine diagnosis in automotive workshops should consider fuzzy logic and probabilistic fault classifier to quantify the degreesof symptoms anddetermine the possibilities of multiplefaults respectively. By fuzzy logic technique, the symptomsare fuzzified into fuzzy value and then based on the values, the diagnosis is carried out. By going through multi-fault classification, the output of the diagnostic systemis then defuzzified into fault labels.

Recently, many modeling/classification methods combined with fuzzy logic have been developed to model the nonlinear relationship between symptoms and engine faults. In 2003, Fuzzy Neural Network (FNN) was proposed to detect diesel engine faults [1]. Vonget.al, [2,3] applied multi-class support vector machine (SVM) and probabilistic SVM for engine ignition system diagnosis based on signal patterns, however the signal-based method is not considered in this study because it is difficult to apply to automotive workshops. In reference [4], Fuzzy Support Vector Machines (FSVM) was proposed and put forward to classify complex patterns; it is believed that the FSVM technique can also be applied to fault diagnosis problems.

Both FNN and FSVM have their own limitations. For FNN, firstly, the construction of FNN is so complex (involving number of hidden neurons and layers, andtrigger functions, etc) that the choice of themis difficult. Improper selection will result in a poor performance. Secondly, the network model depends on the training data, thus, if the data is not large enough, the model will be inaccurate, but if it is excessive, which causes over fitting problem, then the FNN model will be inaccurate either. Asfor FSVM, it suffers from solving the hyperparameters. There are two hyperparameters (σ, c) for user adjustment.These parameters constitute a very large combination of values and the user has to spend a lot of effort to determine the parameters.

Recently, an improved statistical method based on extreme learning machine, namely, sparse Bayesian extreme learning machine (SBELM) was developed to deal with the aboveproblemsin classification [5]. SBELM is a probabilistic classifier. SBELM inherits the fast training time from extreme learning machine and the sparsity of weights, which prunes the number of corresponding hidden neurons to minimum, from the sparse Bayesian learning approach. It is believed that the fast training time and the property of sparsity can enable SBELM to effectively deal with big data point problems. Besides, SBELM can let the user easily define its architecture because the classification accuracy of SBELM is insensitive to its hyperparameter, number of hidden nodes (L), as long as L is over 49 [5], whereas FPNN and FPSVM do not have this attractive feature. As a result, SBELM is selected as a training algorithm for building the probabilistic classifier in this study. Moreover, there is no research applying fuzzy logic to SBELMforany diagnosis problems yet. So a promising avenue of research is to apply fuzzy logic to SBELMforcar engine multiple-fault diagnosis.

In this paper, a new framework of fuzzy sparse Bayesian extreme learning machine (FSBELM) is proposed for fault diagnosis of car engines. Firstly, fuzzy logic gives the memberships of the symptoms depending on their degrees. Then, SBELM is employed to construct some probabilistic diagnostic modelsor classifiers based on the memberships. Finally, a decision threshold is employed to defuzzify the output probabilitiesof the diagnostic models to be decision values.

Because of multiple fault problems, standard evaluation criterion, exact match error, is not the most suitable performance measure as it does not count partial matches. Hence F-measure isconsidered in this paper to evaluate the diagnostic performance because it is a partial matching scheme.

2. System Design

Depending on domain analysis, the typical symptoms and car engine faults are listed in Tables1 and 2, respectively. Table3 shows the relationship betweenthe symptoms and the engine faults. If one engine expressesthe ith symptom, then xiis set as 1, otherwise it is set as 0. In a similar manner, if one engine is diagnosed with the jth fault, then yjis set as 1, otherwise it will be set as 0. Hereby, the symptoms of one engine could be expressed as a vector x=[x₁, x₂,…, x₁₁]. Similarly, the faults of an engine are also expressed as a binary vector y=[y₁, y₂,…, y₁₁].

Table 1: Typical car engine symptoms.

Table 2: Typical car engine faults.

Table 3: Relationship ofsymptoms and possible car engine faults.

3. Fuzzy Sparse Bayesian Extreme Learning Machine

Fuzzy sparse Bayesian extreme learning machine is defined as SBELM with fuzzified input. As the fuzzification of the input is presented in Section 2, this section introduces SBELM only.

Different from extreme learning machine thatcalculates the inverse of matrix hidden layer output H [6,7], SBELM employs the Bayesian mechanism to learn the output weights w. Given a training dataset (s_i, t_i) of N cases for a d-class problem for i= 1 to N wheresi is the fuzzified input vector and ti is the corresponding label of si. Then, the input for SBELM is the hidden layer outputs H, in which H=[h₁(s₁),… ,h₁(s_N)] ^TЄ R^N×(L+1) and hi(si)=[1, g₁ (θ1∙s_i+b₁),… ,g_L(θ_L∙s_i+b_L)],where g(.) is activation function of hidden layer, θ is weight vector connecting the hidden and input nodes, b is the threshold of the hidden node. For two-class classification, every training sample can be considered as an independent Bernoulli event P(t\s). The likelihood is expressed as:

P( t|w,h )= ∏ i=1 N σ [ y( h i ;w ) ] t i { 1−σ[ y( h i ;w ) ] } 1− t i (13)

where σ(.) is sigmoid function σ[ y( h;w ) ]= 1 1+ e −y( h;w ) , y(h;w) =hw, t=(t₁...t_N), t_i={0,1} and w=(w₀,w₁,...,w_L)^T. A zero-mean Gaussian distribution over each parameter wi conditions on an automatic relevance determination (ARD) of hyperparameter a_i [8,9] is expressed by

P( w i | a i )=N( w i |0, a i −1 ) a= [ a 0 , a 1 ,…, a L ] T (14)

P( w|a )= ∏ K=0 L α i 2π exp(− α i w i 2 2 ) (15)

There always exists an independent a_i associated with each w_i; some values of w_i is to be zero when a_i tends to infinity. The value of hyperparameter a are calculated by maximizing the marginal likelihood by integrating the weight parameters w.

P( t|a,H )= ∫ ​ P( t|a,H )P( w|a )dw (16)

However, Eq. (16) cannot be directly integrated out. To solve this problem, ARD approximates a Guass for it with Laplace approximation approach, such that P( t|a,H )P( w|a )∝N( w MP ,Σ ) . Where w_MP and Σ are the center and covariance matrix of Gaussian distribution respectively. Generally, Newton-Raphson method - iterativereweighted least-squares algorithm (IRLS) is effectively applied to find w_MP.

w MP = w MP − Φ −1 ∇E (17)

Where

∇E= ∇ w log{ P( t|w,H )P( w|a ) }= H T ( t−y )−Aw (18)

Φ= ∇ w ∇ w log{ P( t|w,H )P( w|a ) }| w MP =−( H T BH+A ) (19)

where y=[y₁,y₂,…,y_N]^T, A=diag(a), B=diag(β₁,β₂,…,β_N) is a diagonal matrix with β_i=y_i(1-y_i). The center w_MP and covariance matrix Σ of Gauss distribution over w by Laplace approximation are:

Σ= ( H T BH+A) −1 (20)

w MP =Σ H T B t ^ (21)

Where t ^ =Hw+ B −1 ( t−y ) . After gaining Gaussian approximation for w, the integral of product of the two prior probability functions of Eq. (16) becomestractable. Setting the differential of L( α )=Log P( t|a,H ) with respect to α to zero, it yields

∂L( α ) ∂ α i = 1 2 α i − 1 2 Σ ii − 1 2 w MP i 2 =0⇒ α i new = 1− α i Σ ii w MP i 2 (22)

After the maximum number of iterations through Eq. (22), most elements a of atend to infinity. According to the mechanism of ARD, ARD prior prunes the corresponding hidden neurons when the elements of w associated with a tend to zero. The final probability distribution P(t_new|S_new,w_MP ) is predicted by using sparse weight based on y(h; w ^ )=h( w ^ ) and σ[Y(h; w ^ )]= (1+ e (−Y(h; w ^ ) ) (−1) .

The above formulation is designed only for binary classification. For multi-classification and producing probabilistic output, oneversus- all strategy is usually employed to deal with multi-classification problems. One-versus-all strategy constructs a group of classifiers l_class = [C₁,C₂,…,C_d] in a d-label classification problem. The one-versusall strategy is simple and easy to implement. However, it generally gives a poor result [10,11] since one-versus-all does not consider the pairwise correlation and hence induces a much larger indecisive region than pairwise coupling strategy (using one-versus-one). In pairwise coupling strategy, it also constructs a group of classifiers l_class= [C₁,C₂,…,C_d] in a d-label classification problem, but each C_i = [C_i1,…C_ij,…,C_id] is composed of a set of d-1 different pairwise classifiers C_ij, i≠j. Since C_ij and C_ji are complementary, there are totally d(d-1)/2 classifiers in lclass as shown in Figure 1. To solve the multiclassification as well as produce probabilistic output, pairwise coupling strategy is adopted forSBELM. The strategy combines all the output of every pair of classes to re-estimate the overall probability for a new instance. In this research, the following simple pairwise coupling strategy for multiple-fault diagnosis is proposed. The probability of every ρ_i is calculated as

ρ i = C i (s)= ∑ j=1:i≠j d n ij C ij (s) ∑ j=1:i≠j d n ij = ∑ j=1:i≠j d n ij ρ ij ∑ j=1:i≠j d n ij , for i=1,2,..d (23)

Figure 1: Pairwise coupling strategy for SBELM [12].

where n_ij is the number of training vectorswith eitherith orjth labels, ands is an unseen case. Hence, the probability can be more accurately estimated from ρ_ij=C_ij (s) because the pairwise correlation betweenthe labels is taken into account.

4. Experiments

4.1 Design of experiments

The FSBELMwas implemented byMatLab.As the output of each FSBELM classifier is a probability vector. Some well-known probabilistic diagnostic methods, such as fuzzy probabilistic neural network (FPNN) [13] and fuzzy probabilistic support vector machine (FPSVM) were also implemented with MatLab in order to compare their performances with FSBELM fairly. For the structure of the FPSVM, the kernel wasradial basis function. In terms of the hyperparameters in FPSVM, the hyperparameterscand σ were allset tobe 1 according to usual practice.Regarding the network architecture of the FPNN, there are 11 input neurons, 15 neurons with Gaussian basis function in the hidden layer and11output neurons with sigmoid activation function in the output layer.

In total, 308 symptom vectors were prepared by collecting the knowledge from ten experienced mechanics. The whole data was then divided into 2 groups: 77 as test dataset and 231 as training dataset. All engine symptoms were fuzzified using the fuzzy memberships of Eqs. (2)~(12) and produced the fuzzified training dataset TRAIN and the fuzzified test dataset TEST. For training FSBELM and FPSVM, each algorithm constructed 11 fuzzy classifiers f_i, i Є {1,2,..,d, & d=11}, based on TRAIN. The training procedures of FSBELM and FPSVM are shown in Figure 2, whereas the procedure for FPNN is not presented in Figure 2, because it is a network structure instead of individual classifier.

Figure 2: Workflow of training of FSBELM and FPSVM.

4.2 Multiple fault identification

The outputs of FPNN, FPSVM and FSBELM are probabilities, so a simple threshold probability can be adopted todistinguishtheexistence of multiple faults. According to reference [13], the threshold probability was set to be 0.8. The whole fault identification procedure is shown below.

1. Input x = [x₁, x₂,…, x₁₁] into every classifier fiand FPNN. Each fiand the output neurons of FPNN could return a probability vector ρ= [ρ₁, ρ₂, …, ρ₁₁]. ρ_i is the probability of the ith fault label. Where x is a test instance and ρis the predicted vector of engine faults.
2. The final classification vector y = [y₁, y₂,…, y₁₁]is obtained based onEq. (24).

y i ={ 1 if  ρ i ≥0.8 0 otherwise   , for i= 1 to 11 (24)

The above steps are equivalent to a defuzzificationprocedure. The entire fault diagnostic procedures of FSBELM and FPSVM are depicted in Figures 3 whereas the procedure of FPNN is not shown in Figure 3, because it uses an entire network to predict the outputs, but the fault identification procedure usingthe threshold is the same.

Figure 3: Workflow of fault diagosis based on FSBELM and FPSVM.

4.4 Experiment results and evulation

The overall F-measure of predicted faults over TEST is shown in Table 4. All the results were run using a PC with Intel Core i5 @3.2 GHz and 4GB RAM onboard. The FSBELM has the best diagnostic performance and its F-measure is as high as 0.964. The F-measure indicates that FSBELM outperforms FPSVM and FPNN. The F-measure for each fault is shown in Table 5 where the F-measure for each fault of FSBELMishigher than that of FPNN and FPSVM. The reason of why FPNN gives poor performance is that the training data in this research is not large enough (231 only). The relatively low performance of FPSVM is due to the fact that its parameters (σ, c) may not be optimal. In fact, it is very difficult to determine the optimal parameters. On the other hands,FSBELM only needs to set the number of hidden node L to be 50. Table4also shows that FSBELM runs much faster than FPNN and FSVMunder the same TRAIN and TEST. So, FSBELM is a very promising approach for this application.

Table 4: Overall F-measure and computational time comparison for the three classifiers in diagnostic performance.

Table 5: F-measure comparison for each fault of the three classifiers in diagnostic performance.

5. Conclusion

In this paper, FSBELM has been successfully applied to multiplefault diagnosis of the car engine. Moreover, FPNN, FPSVM and FSBELM have been compared to detect the car engine faults based on various combinations and degrees of symptoms. This research is the first attempt atapplying fuzzy logic to SBELM for engine multiplefault diagnosis and comparingthe diagnostic performance of several fuzzy classifiers.Experimental results show that FSBELMoutperforms FPSVM and FPNN in terms of accuracy, training time and diagnostic time. So, it can be concluded that FSBELM is a very promising approach for engine multiple fault diagnosis.

Competing Interests

The author declare that there is no competing interests regarding the publication of this article.

Acknowledgments

The author would like to thank the support from Prof. Chi-man Vong, Department of Computer Science, University of Macau, and Mr. Jiahua Lou who develops the MatLab toolbox of SBELM.