1887
Volume 28, Issue 2
  • ISSN 0378-4169
  • E-ISSN: 1569-9927
USD
Buy:$35.00 + Taxes

Abstract

The computational treatment of multilingual language resources (as in the Prolex projet, cf (Grass et al. 2002)) should respect lexical conventions admitted by each language’s native speakers. These conventions may vary from one language to another, as in the case of alphabetical sorting algorithm. This algorithm must take a number of universal as well as language-dependent particularities into account, such as the distinction of upper- and lowercase letters, the sorting bi-directionality (from the left to the right or conversely), the role of diacritics (resulting either in variants of a letter, as é, è and ê in French, or in independent letters, as å in Danish or ą in Polish), the role of punctuation characters, the multi-character letters (as ch or ll in Spanish, or dzs in Hungarian) and the ligatures (as œ in French, or ß in German). We describe a Unicode-based sorting algorithm inspired by (LaBonté 1998) for proper names. In the particular case of the sorting of the proper names, three additional points are to be taken into account : the presence of numerical values (Arab numerals or Roman numerals), the variation of spelling of the ligatures and the permutation in the sorting of the multi-word units. Apart from the word list to be sorted, its input is a language-dependent code table which defines the language’s alphabet, the number of algorithm’s passes, the direction of each pass, and the order of letters or groups of letters in each pass. The implementation of the algorithm is done by a finite-state transducer which allows a fast assignment of sort keys to words. The algorithm proved correct for European languages such as English, French, and Polish, as well as for Thai. It outperforms other sorting algorithms, such as those implemented in Intex (Silberztein 1993) and Unitex (Paumier 2003) systems.

Loading

Article metrics loading...

/content/journals/10.1075/li.28.2.07tra
2005-01-01
2024-10-04
Loading full text...

Full text loading...

/content/journals/10.1075/li.28.2.07tra
Loading
  • Article Type: Research Article
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error