For 10 years, experts have tried to bring Tulu to smartphones. But they can’t agree on how.

Plain text data consists of a sequence of encoded characters or “code points” from a given standard such as the Unicode Standard. Some of the most common file formats for digital data used in eScience (CSV, XML, and JSON, for example) are built atop plain text standards. Plain text representations of digital data are often preferred because plain text formats are relatively stable, and they facilitate reuse and interoperability. Despite its ubiquity, plain text is not as plain as it may seem. The set of standards used in modern text encoding (principally, the Unicode Character Set and the related encoding format, UTF-8) have complex architectures when compared to historical standards like ASCII. Further, while the Unicode standard has gained in prominence, text encoding problems are not uncommon in research data curation. This primer provides conceptual foundations for modern text encoding and guidance for common curation and preservation actions related to textual data.

"800M people could use a period emoji right now. How many people are talking about broccoli?"

script to clean bad characters out of csvs, mah boy - GitHub - BAM-PFA/csvboy: script to clean bad characters out of csvs, mah boy

In 1978 Japan's Ministry of Economy, Trade and Industry established the encoding that would later be known as JIS X 0208, which still serves as an important reference for all Japanese encodings. However, after the JIS standard was released people noticed something strange - several of the added characters had no obvious sources, and nobody could tell what they meant or how they should be pronounced. Nobody was sure where they came from. These are what came to be known as the ghost characters (幽霊文字).

Phishers and other online crooks are taking advantage of Unicode domain names in their pursuit of your passwords and other sensitive information. Here's a simple way to protect yourself.

Apple's high-level networking APIs can do a lot of the heavy lifting for you.

Table used for debugging common UTF-8 character encoding problems

We can’t ignore the composition of the Unicode Consortium’s members, directors, and officers -- the people who define the everyday writing systems of all languages across the globe.

Unicode can be confusing. Here’s a “just the facts I need” presentation about how to handle it correctly.

Tools for cataloging in languages you don't know, or recognizing languages on the item in hand.

Good is the enemy of GreatLatin-1 is the enemy of UTF-8