Myths about collation

By Cathy Wissink & Michael S.Kaplan - Windows Globalization, Microsoft Corporation

Misconception #1: "If I localize my product, I don't need to worry about details like collation."
This misconception shows a lack of understanding of the difference between localization and globalization. While localization clearly helps prepare a product to be used in a particular market, it is not sufficient to translate strings—the product must still be enabled for a local experience, including the ability to search and sort strings in a manner appropriate for the culture. As such, it is necessary to call (or write) functions that use culturally-correct collation.

Misconception #2: "If I use Unicode in my product, sorting is already covered".
This particular misconception is widespread, not only regarding Unicode, but also encodings in general (that is, there is a common belief that encoding order is a satisfactory collation order). Unicode clearly is the heart of a well-globalized product (for reasons too numerous to list here). That said, Unicode is a character encoding, not a means of collation. While there are some code points that are in a reasonable order within Unicode, the default order given by Unicode is in no way an acceptable collation for any culture.

Misconception #3: "One collation is good enough for Europe (North America, Asia), right?"
There are great differences in collation results depending on the language, even within the same geographical region. Therefore, it is highly unlikely that a "one sort fits all" approach would be satisfactory for all users within that particular part of the world. In addition, multiple writing systems are in use within particular geographical regions, and the approach taken for one writing system may be in no way sufficient for another writing system.

Misconception #4: "One collation is good enough for the Latin (Han, Cyrillic) script, right?"
This misconception is the same as #3, expanded to include more users. While there may be some writing systems that are used for only one language, and as such, one sort may be sufficient, this is the exception. Again, a "one sort fits all" approach would not be satisfactory to most users.