3.1.1.1 Internet Article Preprocessing

As illustrated in Figure 2, the CKIP system [3,13] is first used for sentence and word segmentation and POS tagging to obtain the POS structure of these sentences. The obtained POS structures are then viewed as POS-combination features.

Figure 2: Internet Article Preprocessing.

Sentence segmentation and POS tagging: Chinese Knowledge and Information Processing (CKIP) with the functions of sentence and word segmentation and POS Tagging is adopted to analyze the POS structure of Chinese sentences, such as “這樣(Vi)的(T)美食(N)能 (ADV)錯過(Vt)嗎(T)”.

Proper noun tagging: For the results of POS tagging, some proper nouns did not appear in the word library of the CKIP system, which results in wrong POS tagging. A list of proper noun tag thus is artificially established to revise the proper noun tag of the enterprise and products. For instance, the Chinese sentence “千萬(Neu)別(D) 錯過(VJ)這(Nep)個(Nf)芋頭(Na)酥(VH)啦(T)” has wrong POS tags, and the revised POS tag is “芋頭酥(Na)”.

Antonym and confused term tagging: In the CKIP system, antonym terms and confused terms are not obviously tagged, which easily cause errors in polarity analysis. Hence, a list of antonym term and confused term tag is artificially established to revise the POS tag. For example, “不(ADV)” is revised as “不(ADV1)” and “不(ADV1)小心 (Vt)” is revised as “不小心(Vt)”.

3.1.1.2 Non-Polarity Sentence Filtering

Since this study merely clusters POS-combination sentences with polarity, the library of polarity term is used to filter non-polarity sentences and keep polarity sentences with polarity terms. Figure 3 presents the process of non-polarity sentence filtering.

Figure 3: Non-Polarity Sentence Filtering.

Polarity sentence filtering: In this step, prepossessed sentences are compared with polarity terms in the polarity term library to keep the prepossessed sentences that include the polarity terms and the POScombination features. For example, “好” in the prepossessed sentence “服務(Nv)真(ADV)好(Vi)” is the polarity term in the library of polarity term. The POS-combination “(Nv)(ADV)(Vi)” is kept.

POS-combination count: The number of generated POScombinations is counted, e.g. the POS-combination “(Nv)(ADV)(Vi)” appears seven times.

3.1.1.4 POS-combination clustering

Clustering is utilized to integrate types of POS-combination. POScombinations can be clustered through their permutations. Similar POS-combinations are regarded as the same type, and the semantic rules of such a type are determined to support the analysis of Chinese eWOM. The steps involves POS-combination clustering and POScombination structure clustering, as illustrated in Figure 4.

Figure 4: POS-Combination Clustering.

POS-combination clustering: FDOD (Function of Degree Of Disagreement) [7,20] is used and modified to cluster POScombinations. The similarity between POS-combinations is calculated using the revised formula in Equation (1).

FDOD( S 1 , S 2 )= ∑ i=1 n cis( S 1 , H n )*cis( S 2 , H n ) (1)

where FDOD(S₁,S₂) denotes the difference between POScombinations

S₁ and S₂ calculated using FODD;
S is the set of POS-combinations, expressed as S∈{ S 1 , S 2 , S 3 ,..., S x } ;
L is window size of per unit, which is normally set to 2;
H is the set of sequences from cutting two sequences with length L;
H_n is the content in the n-th unit of set H;
cis(S ,H_n) denotes the number of POS being continuously same in matching S with H_n;

In Equation (8), the similarity between longer sequences is easily enhanced while the equation is applied to calculate the similarity between shorter POS-combinations. This study designs a weighted method that divides the results of in Equation (8) with the possible combinations that are generated by pair of sentences with POS. Each pair of sentences is expected to have a close POS quantity and a different penalty value of POS quantity. The formula is written as Equation (2).

FDODPatch( S 1 , S 2 )=FDOD( S 1 , S 2 )* C 1 /(len( S 1 )*len( S 2 ))−|(len( S 1 )−len( S 2 ))|* C 2 (2)

where FDODPatch(S₁,S₂) represents the standardization of the results that are calculated by FDOD(S₁,S₂);
C₁ and C₁ are fixed constants;
len(S) denotes the length of POS quantity in POS combination S;

Subsequently, the results that are calculated using Equation (2) undergo Density-based Spatial Clustering of Applications with Noise (DBSCAN) [5,6] to achieve POS-combination clustering.

POS-combination structure clustering: Since the matching of POS-combination structure can be regarded as a problem of sequence alignment, dynamic programming [16,18] is thus applied to cluster POS-combination structures based on the results of POS-combination clustering. Sequence alignment determines the alignment positions between different POS combinations so that different POS-combinations have a uniform position of polarity term. The calculation is detailed as follows.

Assume that each POS tag in POS-combination a is expressed as (a₁,a₂,...,a_i) and each POS tag in POS-combination b is expressed as (b₁,b₂,...,b_i), the score of each sub-question is recorded as matrix A. The element A(i,j) is calculated using Equation (3).

A(0,0)=0 A(i,0)=−i A(0,j)=−j A(i,j)={ max{ A(i−1,j−1)−1 A(i−1,j)−1 A(i,j−1)−1 }​  if​ a i ≠ b j A(i−1,j−1)+2 if​ a i = b j ​ (3)

Finally, POS-combination types are clustered using the DBSCAN clustering algorithm according to the similarity score between two sequences.

3.1.3.1 Polarity term location selection

The selection of polarity term location facilitates the accuracy of analyzing Chinese eWOM. Figure 6 depicts the detailed steps in selection of polarity term location.

Figure 6: Polarity Term Location Selection.

POS-combination sentence obtainment: POS-combinations in a single polarity POS-combination type are matched with sentences in Internet articles to extract sentences with the same POS-combination.

Polarity term location tagging: The feature of the polarity term is firstly acquired from the library of polarity term, and its location in the POS-combination is then tagged for polarity POS-combination example sentences.

Polarity term location judgment: The probability of each location in polarity POS-combination example sentences that are tagged with polarity is calculated. When the probability of the location with the feature of the polarity term exceeds the threshold T_w, the location is regarded as the location of an emotional word.

3.1.3.2 Polarity POS-combination rule judgment

The judgment of the polarity POS-combination rule mainly analyzes whether the sentence meaning is changed while the polarity term appears in polarity POS-combination. For example, while the polarity term “好吃” appears in a POS-combination example sentence “不能 (N)說(Vt)是(Vt)不(ADV)好吃(Vi)”, the sentence meaning tends to be positive. Figure 7 presents the steps in judging the polarity POScombination rule.

Figure 7: Polarity POS-Combination Rule Judgment.

Sentence polarity tagging: According to the POS-combination, location of the polarity term, and sentences in the library of articles with known polarity sentences, POS-combinations are matched to extract those with the same POS-combination.

Term polarity tagging: The example sentences and the location of the polarity term that were acquired in tagging of sentence polarity are used to determine the polarity terms. The polarity terms thus obtained are then matched with those in the library of polarity term, and the polarities of those term are recorded.

Polarity POS-combination rule judgment: The meaning of the example sentence is matched with the term polarity to record whether the meaning of the example sentence and the term polarity are the same or opposite. In the process, if multi-polarity terms appear, they are combined on the basis that a double negative is a positive. The example sentences in the same polarity POS-combination type are processed using probability and statistics. When the same or the opposite probability exceeds the 70% threshold, the POS-combination rule for the POS-combination type is judged; otherwise, the POScombination type is ignored.

3.1.4.1 Topic article preprocessing

Articles in the set of topic article, regardless of whether they conform to the topic, are preprocessed to acquire meaningful term units. The process of topic article prepossessing is the same as that of Internet article preprocessing described before (Figure 2).

3.1.4.2 Topic feature term identification

The identification of the topic feature term involves the selection of more important terms related to topics based on the prepossessed terms obtained from the topic article preprocessing, as shown in Figure 8.

Figure 8: Topic Feature Term Identification.

In Figure 8, topic term selection utilizes term frequency-inverse document frequency (TF-IDF) to examine the importance of topic terms in the topic article. Equation (4) presents the formula for TF-IDF.

W i,j =t f i,j *log( N df i ) (4)

where W_i,j denotes the importance of vocabulary i appearing in topic article j;
TF denotes the frequency of vocabulary appearing in the topic article, with higher frequency indicating higher importance of the vocabulary in the topic article;
IDF=log( N df i ) represents the frequency of vocabulary appearing in all topic articles. Lower frequency indicates better representativeness of the vocabulary, and thus that the article is easy to differentiate;
i is a vocabulary;
j is the article appearing vocabulary i;
N is the total of all topic article types;
Df is the number of articles in which vocabulary i appears;

According to the importance of topic terms calculated by TF-IDF, expletives that are the same as stop words and meaningless terms are removed. The remaining terms, with importance value W higher than the threshold, are then selected as the topic feature terms.

3.1.4.3 Term expansion

Term expansion is to expand the relevant terms for the topic feature terms selected from step (2) to derive keywords that do not appear in the article but are related to the topic. The detailed steps are shown in (Figure 9).

Figure 9: Term Expansion.

Context clue expansion: Google N-gram [21] is used to search for the terms frequently appearing in front of or behind the topic term. For instance, the topic term “mobile” is given for Google 5-gram, and the search results may be “the mobile screen is too small”. Subsequently, the number of occurrences of the terms is counted, and terms that are the same as stop words are removed. Finally, a threshold is set to keep the terms whose occurrence number exceeds the threshold.

Synonym expansion: Synonyms of the terms selected by TF-IDF can be determined using the online Chinese dictionary for artificially tagged synonyms [22], as screened in (Figure 10).

Figure 10: The Online Chinese Dictionary for Artificially Tagged Synonyms.

The synonym dictionary is used to identify synonyms related to the topic term. The number of occurrence for these synonyms is counted and those that are the same as stop words are removed. Finally, a threshold is set to keep the synonyms whose occurrence number exceeds the threshold.

3.2.1.1 eWOM Preprocessing

This step preprocesses unknown Internet Chinese articles to obtain meaningful term units. The details in eWOM preprocessing are the same as Internet article preprocessing presented before (Figure 2).

3.2.1.2 Topics confirmation

Topic confirmation mainly selects sentences conforming to the products of the target enterprise to enhance the accuracy of Chinese eWOM identification. Figure 11 shows the detailed steps involved in topic confirmation.

Figure 11: Topic Confirmation Process.

Sentence segmentation: The article preprocessed in step (1) is segmented based on the sentence unit, and all segmented sentences in the article are outputted sequentially.

Topic term matching: Based on the results in step (a), each sentence is compared with the topic term library. The sentence with the topic term and the following sentence are regarded as the topic sentence. For example, the topic term “oil content” appears in the sentence “the oil content of snacks is reduced by 70%”. This sentence and the subsequent sentence “to eat healthier” thus are considered the topic sentences conforming to the products of the target enterprise.

3.2.2.1 Sentence POS-combination classification

As shown in Figure 12, the POS-combination rules are determined from the library of POS-combination rule according to the topic sentences analyzed before.

Figure 12: Sentence POS-Combination Classification.

Based on the POS-combination of topic sentences, the most similar POS-combination rules are determined from the library if POS-combination rule via dynamic programming (Equation 3) [16,18]. If POS-combination rules have high similarity and exceed the threshold, they are regarded as the POS-combination rules of the topic sentence. If their similarity is below the threshold, the topic sentence is considered to lack suitable POS-combination rules.

3.2.2.2 Sentence polarity analysis

The topic sentences and their POS-combination rules acquired in step (1) are processed with the library of polarity term to derive the polarity of the topic sentence, as shown in Figure 13.

Figure 13: Sentence Polarity Analysis.

Analysis by POS-combination: The location of the polarity term in a sentence is first judged using the POS-combination rules. The polarity term is then matched with the library of polarity terms to obtain its polarity. Finally, the polarity of the term is combined with POS-combination rules to determine the polarity of the topic sentence. For example, the Chinese topic sentence “(這樣)(的)( 美食)(能)(錯過)(嗎)” has the POS tags “Vi”, “T”, “N”, “ADV”, and “Vt”, respectively. POS-combination rules are determined using the libraryof POS-combination rule. The fifth term “錯過” in the Chinese topic sentence is a polarity term, and the sentence meaning is opposite to the polarity term. The term “錯過” is thus searched in the library of polarity term and identified as a negative term. Based on this analysis, the topic sentence tends to have a positive meaning.

Analysis by logic rule: Based on the traditional logic rules, the features of polarity term and the antonyms are used to judge the polarity of sentences. This allows all terms in the topic sentence to first be matched with the library of polarity term to obtain the polarity of one or more polarity terms. The front of the polarity term in the topic sentence is then examined to determine whether it includes an antonym. If so, its polarity is considered opposite.

3.2.2.3 Term polarity analysis

The polarity of unknown polarity terms is artificially analyzed by the knowledge constructor, and is stored in the library of polarity term for the use of Chinese eWOM analysis.