Language Processing

Use the Language Processing flag of the Translate Table to indicate whether the language specific routine defined in the Language field should be used.  Set the flag to ON to use a language specific routine when stemming.  Set the flag to OFF to do only the stemming specified in the Suffix List.

Required

Default

Valid Values

YES

OFF

OFF, ON

With the Language field set to ENGLISH language special processing and the Language Processing flag turned ON allows the designer to use a much less complex stemming algorithm since the double consonant and long vowel logic is left to the English-specific special processing function.

Example

With the Language Processing set to OFF a pass for the Suffix List used to stem the suffix "ing" from keywords would look like the following:

Threshold

Pattern

Replacement

Description

6

!%ing

.

compress double consonant

6

%cing

.ce

replace “ing” with “e”

6

%%ing

..

no substitution for “ing”

6

!%@%ing

....

no substitution for “ing”

6

%@%ing

...e

replace “ing” with “e”

6

@@%ing

...

no substitution for “ing”

With the Language Processing turned on a pass for the Suffix List used to stem the suffix "ing" from keywords would look like the following:

Threshold

Pattern

Replacement

Description

6

ing

.

no substitution for “ing”

The English language special processing algorithm takes care of the common cases for stemming.