Arabic Script Gap Analysis

Abstract

This document describes and prioritises gaps for the support of the Arabic script on the Web and in eBooks. In particular, it is concerned with text layout. It checks that needed features are supported in W3C specifications, such as HTML and CSS and those relating to digital publications. It also checks whether the features have been implemented in browsers and ereaders.

See also General page layout & progression for features such as column layout, page turning direction, etc. that are affected by text direction.

In what direction does text flow along a line and across a page? (If the basic direction is right-to-left see 2.2 Bidirectional text.) If the script uses vertically oriented text, what are the requirements? What about if you mix vertical text with scripts that are normally only horizontal? Do you need a switch to use different characters in vertical vs. horizontal text? Does the browser support short runs of horizontal text in vertical lines (tate-chu-yoko in Japanese) as expected? Is the orientation of characters and the directional ordering of characters supported as needed?

We need to clarify whether there is a particular requirement for handling arabic text specially in vertical lines, such as upright glyphs. We are also waiting on implementation of sideways values of writing-modes in order to be able to effectively use arabic text in vertical arrangements (such as book spines, table headings, etc), but that is not a problem specific to arabic.

GitHub issue #242

Languages: arb pes ug ur ks 2

This issue is applicable to most languages.

Vertical text may occur for special effects (the spine of a book, table column headings, etc). Typographically, it is simply horizontal text that is rotated. There is no way to do this effectively until browsers support the new CSS properties.

For more details, see this GitHub issue, which is being used to track this gap. Please add any discussion there, and not to this issue.

If the general inline direction is right-to-left, are there any issues when handling that? Where the inline direction of text is mixed, is this bidirectional text adequately supported? What about numbers and expressions? Do the Unicode bidi controls and HTML markup provide the support needed? Is isolation of directional runs problematic?

GitHub issue #256

Languages: arb pes ug ur ks 3

This issue is applicable to all languages with RTL orthographies.

Style sheets need to add special rules for RTL styles if they are not supported by logical properties or values.

One approach is to create a second style sheet which, when pulled into an HTML page, overrides styles in the main style sheet with settings for RTL text. This approach is not ideal because it requires maintaining the styles in two separate locations, which can therefore get out of synch, and it requires explicit addition of a call to the second style sheet in every page that will support RTL text.

The :dir() pseudo-class avoids these issues by allowing the content author to include the RTL variations in the same style sheet as the others. However, it is not yet supported by all major browser engines.

A workaround that precedes selectors with :root[dir=rtl] selector_here can help, but :dir() is a better way forward. There also appear to be Shadow DOM implications if this is not supported.

:dir is not interoperably supported by all major browser engines.

selectors-4 Describes the :dir() feature in CSS. html5 Describes the :dir() feature in HTML.

Can I Use indicates that Gecko supports :dir(), but Blink, and Webkit do not. This was confirmed while trying to get this to work recently.

The functionality appears to be available in Blink, but behind a flag.

This simple test below checks whether :dir() causes styling to be changed. The test currently passes for Gecko, but not for Blink or WebKit browsers.

This pseudo-class significantly improves the process of designing style sheets that can cater for both LTR and RTL pages. It is a valuable enabler for better internationalisation of the Web for the billion or so potential users of RTL scripts. This was therefore marked as Basic. interactive test, The browser supports :dir() selectors.
Blink • Webkit Gecko, Blink and WebKit browsers now support the :dir selector.

Can I Use now confirms that this has been fixed.

GitHub issue #244

Languages: arb pes ug ks 3

This issue is applicable to all languages using a right-to-left script.

For support of bidirectional plain text, the Unicode Standard provides a number of formatting characters, which include RLI, LRI, PDI and FSI. See an explanation of how these work.

Although markup should be used most of the time in HTML pages, there are parts of an HTML document that don't support markup, such as the title element and title, alt, and other attributes. These characters can be necessary for managing inline runs of such text.

For more details, see this GitHub issue, which is being used to track this gap.

GitHub issue #217

Languages: arb pes ug ur ks 1

This issue is common to all RTL scripts, and vertically set scripts.

Adoption of logical keywords such as -start and -end, rather than -left and -right needs to be completed.

For margins, padding, block size, border colour, width & style, logical keywords such as margin-inline-start or margin-block-end are widely supported by major browsers in their simplest forms (such as those just mentioned). However, logical properties are not well supported in shorthands such as margin-block or margin-inline or the margin property. The lack of support for shorthands is significant, since they are expected to have high use.

Support for the margin shorthand is currently held up by a discussion about the appropriate syntax. See this CSS issue.

Other logical properties that are not widely supported include border radius, caption side, and float. Firefox does support border-radius and floats, but otherwise these are not supported.

See a set of test results.

These new values also need to be widely supported in editing applications.

GitHub issue #216

Languages: arb pes ug ur ks 3

This issue is common to all RTL scripts. When strings are passed around, some applications don't receive or use information about the appropriate base direction to use for those strings when they are rendered as part of a page. This can lead to text being incorrectly aligned, and to text within a sentence or paragraph being incorrectly ordered. Some of this can be addressed by using heuristics to detect the direction first-strongly directional character in the string, but some strings can fail such heuristics. For example, imagine a service which retrieves book names on an English page. If the page retrieves from the database an Arabic book name that begins with LTR letters, it should look like this: Screenshot 2021-01-21 at 18 12 42 However, if just first-strong heuristics are used to decide the base direction for the inserted book name, it will incorrectly produce: Screenshot 2021-01-21 at 18 14 00 For these cases the databases or JSON files, etc., from which such strings are pulled need to contain metadata about the base direction which needs to be applied to correct the display. The appropriate direction can be communicated to the backend storage from a form input by using the dirname attribute. This indicates the base direction in force for a form control, whether it is derived from the surrounding text or set manually by the user while typing. More:

Reporting direction to the server

The direction of the field is passed with the form data by Chrome and WebKit, but not by Gecko. Note that Gecko also doesn't change the computed direction of the form field when the user manually sets the direction. Basic, because of its usefulness in ensuring correct directional display of RTL text that starts with a LTR character. i18n test suite HTML5, dirname Gecko bug (already raised) All 3 major browser engines, Gecko, Blink, & WebKit, now support relaying the field direction to the server using dirname. It also supports changing the computed direction of a form field when the user manually sets the direction. See Can I Use?

How are fonts grouped into recognisable writing styles? How is each writing style used? Do the standard fallback fonts used in browsers (eg. serif, sans-serif, cursive, etc.) match expectations? Or are additional generic font styles needed? Are special font or OpenType features needed for this script that are not available? What other general, font-related issues arise? The font styles described here refer to alternative types of writing style, such as naskh vs nastaliq; for oblique, italic, and weights see instead 3.3 Letterform slopes, weights, & italics.

GitHub issue #287

Languages: ks 1

This issue is applicable to Kashmiri. Palatalisation is a frequent feature of Kashmiri words. It is represented using ؠ U+0620 LETTER KASHMIRI YEH after the consonant to be palatalised. Medial forms have a small circle beneath them. These changes follow the same pattern as ی U+06CC LETTER FARSI YEH, which has 2 dots below initial and medial forms, but no dots below final and isolate forms. More:

Orthography notes

Although most nastaliq fonts add the small circles only below initial and medial forms, naskh style fonts used for Kashmiri text tend to put the circle below final and independent forms also, which is incorrect. This may be in part due to the Unicode Standard, which says

U+0620 LETTER KASHMIRI YEH is used in Kashmiri text to indicate that the preceding consonantal sound is palatalised. The letter has the form of a *yeh* with a diacritic small circle below. It has the YEH joining group, with the shapes shown in the fifth row of Table 9-10. However, when Kashmiri is written in the Nastaliq style, the final and isolated forms of *kashmiri yeh* usually appear as truncated *yeh* shapes (ؠ) without the diacritic ring.

This is not a browser issue, and will need to be fixed in fonts, and in the Unicode Standard. A key set of fonts to address are Noto fonts. Currently, the following Noto fonts show circles below each positional form: Naskh Arabic, Sans Arabic, Kufi Arabic, and the two UI fonts. For Kashmiri users the impact is mitigated to an extent by nastaliq fonts doing the right thing; this is an issue for naskh font styles, which are not commonly used for Kashmiri text. Nontheless, they are used sometimes, particularly for educational or expository content, and in those circumstances not only is the rendering incorrect, but it can lead to authors trying to add circles where they are not needed. This is therefore marked as a Basic gap. The relevant fonts need to be installed for the following to show results:

Issue, Noto fonts The Unicode Standard text was changed to the following, with bolding used here to emphasis changes:

U+0620 ARABIC LETTER KASHMIRI YEH is used in Kashmiri text to indicate that the preceding consonantal sound is palatalized. The letter has the form of a yeh with a diacritic small circle below in initial and medial forms, but its final and isolated forms appear as truncated yeh shapes (ؠ) without the diacritic ring. It has a joining group of its own, KASHMIRI YEH, with the shapes as shown in Table 9-10, as well as Table 9-7. (Prior to Version 16.0, the Unicode Standard had specified that when written in the Naskh style, the letter had different shapes than when written in Nastaliq style; that specification was erroneous.)

https://unicode.org/versions/Unicode16.0.0/core-spec/chapter-9/#G37640 (6th paragraph)

GitHub issue #279

Languages: arb pes ur 2

This issue is applicable to Persian, Urdu and Sindhi. Persian, Urdu and Sindhi use extended-arabic-indic codepoints for digits in list counter styles. However, languages that use these code points tend to prefer specific glyph shapes for certain numbers, as shown here. More:

requirements_doc

An intelligent opentype font may be able to detect the language of the text and substitute appropriate glyphs if it is used for the content, but here we are looking at the default behaviour of the browser, with no fonts explicitly assigned to the text. Blink displays Persian digit glyphs in all three cases. Webkit renders the correct glyphs for Persian and Urdu, but not for Sindhi. Gecko renders Persian glyph shapes throughout, even though it uses different fonts for Persian vs. Urdu/Sindhi. While this is clearly not right, more user feedback is needed if we are to determine that this causes readability issues. Therefore labelling this, for now, as Advanced. Interactive test, The browser uses appropriate digit glyph shapes for Persian, Urdu, and Sindhi by default.
Gecko • Blink • Webkit tbd

GitHub issue #276

Languages: arb pes ur ks 1

This issue is applicable to Urdu, Kashmiri, and other Arabic script orthographies.

Languages such as Urdu and Kashmiri are written in a nastaliq style of Arabic, and authors will typically want any font fallback to select another nastaliq font, rather than a naskh or other font.

Currently there is no way to tell the browser to fall back to a nastaliq font, rather than a naskh or other font.

Neither Gecko, Blink, nor Webkit support this. Before they can, CSS needs to provide a way for authors to indicate that a nastaliq generic font should be used.

This is a high priority for languages such as Urdu and Kashmiri, where nastaliq fonts are the norm, and incorrect substitutions may cause readability and cultural issues. It is also a useful feature for other languages, such as Persian or Kurdish, where nastaliq styles may be used for certain types of text.

Interactive test, font-family:generic(nastaliq) will apply a nastaliq font in Urdu

Discussion document: Generic font families

CSS discussion threads:

Specify what generic font family maps to nastaliq

Bug reports: Gecko • Blink • Webkit

The CSS Fonts 4 spec now defines a generic(ident) syntax which will be used for newly-introduced, and especially for script-specific, generics.

generic(nastaliq) has been added as one of the generic family names.

Browsers are not yet supporting that.

GitHub issue #249

Languages: ks 1

This issue is applicable to Kashmiri written with the Perso-arabic script.

Kashmiri is written using the nastaliq style of Arabic writing. Although the Kashmiri orthography has some resemblance to that used for Urdu, to represent Kashmiri sounds it uses a number of unique characters or combinations.

There are almost no fonts that properly support Kashmiri written in that orthography. (Noto Nastaliq Urdu was only updated in Feb 2022 to support Kashmiri.)

The result of this is that people resort to using inappropriate characters in their text so that the content looks visually more like they are expecting, and even then gaps remain. For example, to make the sukun look like an inverted v rather than a circle, users often use U+065B ARABIC VOWEL SIGN INVERTED SMALL V ABOVE, which is supposed to be used as an African vowel diacritic. There are several such problems in Kashmiri. Lists can be found here and here

Keyboards and input methods also need to be configured to insert the correct characters, but this doesn't help while there are so few fonts available that can display the characters.

This issue is not likely to be fixed by specifications or browser fixes, but does cause a significant constraint for Kashmiris wishing to use the Web.

There is an additional issue, however, related to pre-installed fonts on macOS (see below).

Clarifying and standardising the correct usage of characters to represent Kashmiri is a fundamental requirement for interoperable and unerstandable text, so this issue is given a priority of Basic.

interactive test, A given font will correctly render characters needed for Kashmiri in the perso-arabic script.
The glyph shapes when the text in the test are displayed should resemble those in the image just below. In particular: farsi yeh with small v above should join to the left; the 4 forms of kashmiri yeh should appear; hamzas should use the round form; the sukun over PA should be an inverted v.

Screenshot 2022-03-22 at 12 43 33

As of March 2022, the latest version of Noto Nastaliq Urdu supports the needed glyphs, if the language is set to 'ks', and displays correctly on Windows10. However, on macOS 12.2.1 the pre-installed version of the font cannot be overwritten and is used to display Kashmiri text in browsers, meaning that there is no support on macOS at the time of writing.

The SIL's Awami Nastaliq font succeeds in correctly rendering all but one feature: the hamza is s-shaped, as used for Urdu, rather than rounded. However, this is a Graphite font, and so only works currently on Gecko browsers.

The Gulmarg Nastaleeq font supports some features in Windows, but appears to not have glyphs for KASHMIRI YEH or for LETTER WAW WITH RING. It also doesn't work on macOS, presumably for the same reason as the Noto font.

Webkit

Version 3.002 and above of Noto Nastaliq Urdu now supports all characters needed for Kashmiri, and will also automatically provide the correct shape for things such as the sukun diacritic if the language of the text is set to Kashmiri. A Unicode submission was approved by the Unicode Technical Committee that says that a word-final half-yeh should not be written using U+06CD ARABIC LETTER YEH WITH TAIL.

If context-sensitive rendering support is needed to shape combinations of letters or position certain glyphs relative to others, is this adequately provided for? Does the script in question require additional user control features to support alterations to the position or shape of glyphs, for example adjusting the distance between the base text and diacritics, or changing the glyphs used in a systematic way? Do you need to be able to compose/decompose conjuncts or ligatures, or show characters that are otherwise hidden, etc? If text is cursive, see the separate section 3.4 Cursive text.

GitHub issue #219

Languages: arb pes 2

For advanced typographic purposes, the fixed position of diacritical marks relative to base letter or baseline and the logical stacking behavior of them might not be sufficient. Therefore, it should be possible to adjust the positioning and combining order and changes to the individual marks when combining. Fonts and other systems may implement the logic needed to result in an optimal presentation of diacritical mark clusters by default.

This covers ways of modifying the glyphs for a range of text, such as for italicisation, bolding, oblique, etc. Are italicisation, bolding, oblique, etc relevant? Do italic fonts lean in the right direction? Is synthesised italicisation or oblique problematic? Are there other problems relating to bolding or italicisation - perhaps relating to generalised assumptions of applicability? For alternative writing/font styles, see 3.1 Fonts & font styles.

GitHub issue #218

Languages: arb pes 2

There should be means available to control the direction in which 'italicised' or 'oblique' text slants, since in some schools of Arabic script typography, text in these styles need to slant to the left.

See the example just below the large letters in this newspaper masthead.

a841092a-f834-11e6-9d55-65efd06f1ea0

For more details, see this GitHub issue, which is being used to track this gap. Please add any discussion there, and not to this issue.

If this script is cursive (ie. letters are generally joined up, like in Arabic, N’Ko, Syriac, etc), are there problems or needed features related to the handling of cursive text? Do cursive links break if parts of a word are marked up or styled? Do Unicode joiner and non-joiner characters behave as expected?

GitHub issue #222

Languages: arb pes ug ur ks 2

This issue is applicable to text in all cursive scripts.

When elements surround part of a cursive run of text, and apply styling, the results often break the cursive joins. (See the results of trying to colour individual letters in the illustration below – as expected above, unsuccessful below.)

Screenshot 2021-01-22 at 15 19 49

shaping-2

Specs: After some discussion, the CSS spec requires the following (see CSS Text: Shaping Across Element Boundaries):

Markup alone around part of a joined up sequence must not disturb the joining behaviour.
Styling that doesn't affect the characters, such as text-decoration, must not break the joins.
Styling that does affect the shape of the characters should not break the joins, however the result is not well defined for complex glyph arrangements such as ligatures where the markup occurs between characters that make up the ligature.
Non-zero margins, padding, and borders, will break the join, as will isolation boundaries.

Tests & results: Interactive test, A span with a colour change for one letter in an Arabic word doesn't break the joining behaviour

Gecko: ✅ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0*
Blink: ✅ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36*
Webkit: ❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15*

I18n test suite, Cursive joining

Gecko: ✅❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0*
Blink: ✅ ❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36*
Webkit: ❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15*

Webkit breaks cursive joining as soon as markup appears around a character, and so obviously fails for any type of styling application, too.

Gecko and Blink keep joins for styling that doesn't affect the shape of the characters (eg. text-decoration), and keeps it for colour changes, however Firefox fails for changes in font-weight, font-style, and font-size, as well as for markup such as em and b tags.

(Gecko and Blink also only pass some of the tests for non-zero margin/padding/border and bdi isolation. Which expect the cursive joins to be broken.)

Browser bug reports: Gecko • Blink • Webkit

Priority: It would be useful to decide on the potential impact of the failures described here, so as to prioritise the issue. Is the inability to surround/style parts of a word a significant issue? It may be problematic when defining a term (using dfn) if the term is only part of the run of letters between spaces, eg. after the definite article.

GitHub issue #221

Languages: arb pes ug ur ks 2

This issue is likely to apply to all cursive script text.

When opacity is applied to text, Gecko and Blink produce dark patches where the cursive glyphs overlap, but Webkit and Legacy Edge don't. See the test.

text-opacity

For more details, see this GitHub issue, which is being used to track this gap.

GitHub issue #220

Languages: arb pes ug ur ks 2

This issue is likely to apply to all cursive script text, but also to other scripts that have joins between letters.

When text stroke effects are applied to cursive text, they should not interrupt the cursive flow.

Unfortunately, that's not the case for current implementations. Overlaps where the glyphs join produce double breaks in the cursive flow.

This example shows the typical result for Arabic.

text-stroke

The set of modern languages using cursive scripts includes those that use the following scripts: Arabic, Syriac, Mongolian, Mandaic, N'Ko, Adlam, Hanifi Rohingya. It also affects text in archaic scripts, including Phags Pa, Manichaean, Psalter Pahlavi, Sogdian, Old Uyghur, Chorasmian.

However, this also affects scripts such as Devanagari (Hindi, Marathi, etc), Gurmukhi (Punjabi), Bengali, Gondi, Modi, Newa, etc. This example is Hindi:

Screenshot 2022-02-08 at 17 12 09

In Gecko, Blink, and Webkit the stroke around the text interrupts the cursive flow.

The property text-stroke is not yet in CSS, but has been implemented in major browsers under the name -webkit-text-stroke.

-webkit-text-stroke is currently specified in WhatWG's Compatibility spec.

Marking as advanced because this is a decorative feature.

Interactive test, Text stroke applied to Arabic text doesn't cut joining glyphs apart
i18n test suite, Text stroke

Issue, Standardize text-stroke Open.

tbd

Does your script need special text transforms that are not supported? For example, do you need to to convert between half-width and full-width presentation forms? Does your script convert letters to uppercase, capitalised and lowercase alternatives according to your typographic needs? How about other transforms?

Most languages are now supported by Unicode, but there are still occasional issues. In particular, there may be issues related to ordering of characters, or competing encodings (as in Myanmar), or standardisation of variation selectors or the encoding model (as in Mongolian). Are there any character repertoire issues preventing use of this script on the Web? Do variation selectors need attention? Are there any other encoding-related issues?

GitHub issue #249

Languages: ks 1

This issue is applicable to Kashmiri written with the Perso-arabic script.

There are almost no fonts that properly support Kashmiri written in that orthography. (Noto Nastaliq Urdu was only updated in Feb 2022 to support Kashmiri.)

Keyboards and input methods also need to be configured to insert the correct characters, but this doesn't help while there are so few fonts available that can display the characters.

This issue is not likely to be fixed by specifications or browser fixes, but does cause a significant constraint for Kashmiris wishing to use the Web.

There is an additional issue, however, related to pre-installed fonts on macOS (see below).

Clarifying and standardising the correct usage of characters to represent Kashmiri is a fundamental requirement for interoperable and unerstandable text, so this issue is given a priority of Basic.

Screenshot 2022-03-22 at 12 43 33

Webkit

This is about how text is divided into graphemes, words, sentences, etc., and behaviour associated with that. Are there special requirements for the following operations: forwards/backwards deletion, cursor movement & selection, character counts, searching & matching, text insertion, line-breaking, justification, case conversions, sorting? Are words separated by spaces, or other characters? Are there special requirements when double-clicking or triple-clicking on the text? Are words hyphenated? (Some of the answers to these questions may be picked up in other sections, such as 6.1 Line breaking & hyphenation, or 6.6 Styling initials.)

Arabic script word boundaries, similar to Latin script, generally can be distinguished by white space and a specific subset of punctuations. There are few exceptions which are listed in Arabic Layout Requirement document.

To enforce a disjoining behavior between the letters which under circumstances of their position normally join, special Unicode character U+200C ZERO WIDTH NON-JOINER (ZWNJ) is used. This usage is detailed in Arabic Layout Requirement document.

All major desktop browsers select a whole word when you double-click on it, including when the word contains ZWNJ. Single-letter prepositions attached to the word are also selected. See some tests.

What characters are used to indicate the boundaries of phrases, sentences, and sections? What about other punctuation, such as dashes, connectors, separators, etc? Are there specific problems related to punctuation or the interaction of the text with punctuation (for example, punctuation that is separated from preceding text but must not be wrapped alone to the next line)? Are there problems related to bracketing information or demarcating things such as proper nouns, etc? Some of these topics have their own sections; see also 5.2 Quotations & citations, and 5.4 Abbreviation, ellipsis & repetition.

This is a subtopic of phrase & section boundaries that is worth handling separately. What characters are used to indicate quotations? Do quotations within quotations use different characters? What characters are used to indicate dialogue? Are the same mechanisms used to cite words, or for scare quotes, etc? What about citing book or article names? Are there any issues when dealing with quotations marks, especially when nested? Should block quotes be indented or handled specially? Do quotation marks take text direction into account appropriately?

GitHub issue #224

Languages: arb pes 2

This issue is common across all languages that use the q element.

When an Arabic or Persian page contains a quotation in another language, the quotation marks used around that quotation (and inside it for embedded quotes) should be the Arabic or Persian ones – not those of the language of the quotation.

Currently, if the language of the quotation is declared on the q tag in HTML using the lang attribute, browsers instead set the quotation marks based on the language of the quote.

For example, if English text is quoted in a Persian sentence surrounded by just <q>, the quotation marks will be correct:

یک «two ‹three›».

However, if lang="en" is added to the q tag, the result becomes:

یک “two ‘three’”.

For more details, see this GitHub issue, which is being used to track this gap. Please add any discussion there, and not to this issue.

How are emphasis and highlighting achieved? If lines or marks are drawn alongside, over or through the text, do they need to be a special distance from the text itself? Is it important to skip characters when underlining, etc? How do things change for vertically set text?

What characters or other methods are used to indicate abbreviation, ellipsis & repetition? Are there problems?

What mechanisms, if any, are used to create *inline* notes and annotations? Are the appropriate methods for inline annotations supported for this script? The ruby spec currently specifies an initial subset of requirements for fine-tuning the typography of phonetic and semantic annotations of East Asian text, including furigana, pinyin and zhuyin fuhao systems. Is is adequate for what it sets out to do? What other controls will be needed in the future? What about other types of inline annotation, such as warichu? This section deals with inline annotation approaches. For annotation methods where a marker in the text points out to another part of the document see 7.3 Footnotes, endnotes, etc..

This section is a catch-all for inline features that don't fit under the previous sections. It can also be used to describe in one place a set of general requirements related to inline features when those features appear in more than one of the sections above. It covers characters or methods (eg. text decoration) that are used to convey information about a range of text. Are all needed forms of highlighting or marking of text available, such as wavy underlining, numeric overbars, etc. If lines are drawn alongside, over or through the text, do they need to be a special distance from the text itself? Is it important to skip characters when underlining, etc? How do things change for vertically set text? Are there other punctuation marks that were not covered in preceding sections? Are lines correctly drawn relative to vertical text?

GitHub issue #223

Languages: arb pes 1

It must be possible to position underlines and overlines further away from the baseline than for Latin text, because ascenders and descenders are longer in many Arabic script fonts. It is common for underlines in Arabic text to be low enough to avoid crossing the ink of descenders in the content, however there are also examples of printed text where ink is skipped – nonetheless, the underline is still usually further from the baseline than, for example, Latin text.

The following image shows typical metrics for Arabic font glyphs compared to Latin. Note the extra vertical space required.

qhxإِودكٍّخٍ

There needs to be a way of adjusting the positioning of the underline to move it further away from the baseline than is currently possible on the Web.

For underlines, the CSS Text Decoration spec describes 2 properties that can help with this.

text-underline-position:under is described in Level 3 (CR). This value positions the underline under the element’s text content so that it usually does not cross the descenders. However, a note in the spec says that it does not guarantee that the underline will not conflict with glyphs, as some fonts have descenders or diacritics that extend below the font’s descent metrics.

text-underline-position:from-font is described in Level 4 (WD). Uses font metrics to position the underline, if available, otherwise reverts to auto.

text-underline-offset is described in Level 4 (WD) and allows you to specify the distance from the text-underline-position as auto, as a fixed distance, or as a percentage of of 1em. This can be effective in positioning the underline appropriately, but has to be paired with the font you are using (to make appropriate adjustments) .

Currently text-underline-offset is not supported by Webkit, but is supported by Gecko and Blink.

text-underline-position:under often but not always produces sufficient clearance, depending on the font, with Gecko and Webkit. The position is lower with Blink than for the auto setting, but cuts through text in all fonts.

text-underline-position:from-font fails to clear the text for Gecko, Webkit, and Blink .

See the discussion in this issue.

Various interactive tests.

Chromium bug for text-underline-offset.

tbd

This is a basic aspect of text decoration for Arabic.

Relevant here are formats related to number, currency, dates, personal names, addresses, and so forth. If the script has its own set of number digits, are there any issues in how they are used? Does the script or language use special format patterns that are problematic (eg. 12,34,000 in India)? What about date/time formats and selection - and are non-Gregorian calendars needed? Do percent signs and other symbols associated with number work correctly, and do numbers need special decorations, (like in Ethiopic or Syriac)? How about the management of personal names, addresses, etc. in web pages: are there issues?

Internationalization software libraries identify Arabic-Indic numerals as the set of numerals which should be used with text in Arabic language excluding a number of Arabic-speaking countries of Northern and Northwestern Africa which should use Arabic (ASCII) numerals. Surveys of publications, monetary and governmental documents, and manuscripts confirm these precedences. However, there is a considerable diverging trend from these recommendations observable on the web, digital products, and in user-generated content. If not considered in design and implementation of software products dealing with Arabic text, this discrepency could be potentially disadvantageous to the quality of text layout, digital typography, and locale-specific data processing.

Does the browser capture the rules about the way text in your script wraps when it hits the end of a line? Does line-breaking wrap whole 'words' at a time, or characters, or something else (such as syllables in Tibetan and Javanese)? What characters should not appear at the end or start of a line, and what should be done to prevent that? Is hyphenation used for your script, or something else? If hyphenation is used, does it work as expected? (Note, this is about line-end hyphenation when text is wrapped, rather than use of the hyphen and related characters as punctuation marks.)

There is no hyphenation in Arabic or Persian language text.

When text in a paragraph needs to have flush lines down both sides, does it follow the rules for your script? Does the script need assistance to conform to a grid pattern? Does your script allow punctuation to hang outside the text box at the start or end of a line? Where adjustments are need to make a line flush, how is that done? Do you shrink/stretch space between words and/or letters? Are word baselines stretched, as in Arabic? What about paragraph indents, or the need for logical alignment keywords, such as start/end, rather than left/right? Does the script indent the first line of a paragraph?

GitHub issue #226

Languages: arb pes 1

It is very common in Arabic script to stretch words or phrases to a particular width (eg. to match a Latin translation or transcription above or below). This is achieved by lengthening the connections between letters, and to some extent by use of wide glyph variants, etc. The rules for which part of the text to stretch and how far are complicated - this is not the even spacing that usually occurs in tracked Latin text. There are currently no mechanisms for managing this process effectively in HTML/CSS.

GitHub issue #225

Languages: arb pes 1

For justification of Arabic script text, there are various common strategies available. These could be categorized in two major groups; strategies based on adjusting inter-word or inter-character whitespace and strategies based on adjusting the letterforms.

A basic implementation must provide at least one of these strategies for adequate justification results. Advanced implementations should provide users with the necessary means to control the selection of strategies, adjustment of their attributes, and the priority with which they are being applied.

Currently, CSS specifications do not provide these advanced features, but recommend that the implementations select the justification strategy appropriate to the text.

Arabic Layout Requirement document includes a section dedicated to this topic.

This section is concerned with spacing that is adjusted around and between characters on a line in ways other than attempts to fit text to a given width (ie. justification). Some scripts create emphasis or other effects by spacing out the words, letters or syllables in a word. Are there requirements for this script/language that are unsupported? If spacing needs to be applied between letters and numbers, is that possible? What about space associated with punctuation, such as the gap before a colon in French? (For justification related spacing, see 6.2 Text alignment & justification.)

Does the browser support requirements for baseline alignment between mixed scripts and in general? Are there issues related to line height or inter-line spacing, etc.? Are the requirements for baseline or line height in vertical text covered?

Are there list or other counter styles in use? If so, what is the format used and can that be achieved? Are the correct separators available for use after list counters? Are there other aspects related to counters and lists that need to be addressed? Are list counters handled correctly in vertical text?

GitHub issue #227

Languages: arb pes ug ks 3

Customisable counters are needed for many languages.

There is no way for users to create local counter styles that are not built in to the browser, and users also often want to tweak the counter style for particular contexts (esp. the prefix/suffix).

Arabic script text uses local counter styles. While some of these are supported by some browsers, the set of symbols used and their order varies by language.

This gap is now fixed. For more details, see this GitHub issue, which is being used to track this gap. Please add any discussion there, and not to this issue.

Does the browser or ereader correctly handle special styling of the initial letter of a line or paragraph, such as for drop caps or similar? How about the size relationship between the large letter and the lines alongide? where does the large letter anchor relative to the lines alongside? is it normal to include initial quote marks in the large letter? is the large letter really a syllable? etc. Are all of these things working as expected?

Although initial letter styling is not an innate feature of the Arabic script, there have been occurences of its usage noted. However, the specifications and guidelines for composition of these decorative elements are undefined or insubstantial; for example, which of the joining forms of letters is to be used or how the joining behaviour is treated accross the boundary between the styled initial letter and rest of the paragraph.

Arabic Script Gap Analysis

Abstract

Status of This Document

1. Introduction

1.1 Contributors

1.2 About this document

1.3 Prioritization

2. Text direction

2.1 Writing mode

#242 CSS Writing-modes sideways values not implemented

2.2 Bidirectional text

#256 :dir lacks wide support – FIXED !

The Gap (2022)

Priority

Tests & results

Action taken

Outcomes - FIXED !

#244 Support for isolating formatting characters lacking in some browsers – FIXED !

#217 Logical CSS keyword support needed

#216 Need support for dirname attribute – FIXED !

The GAP

Priority

Tests & results

Action taken

Outcomes

3. Glyph shaping & positioning

3.1 Fonts & font styles

#287 Small circles should only appear below initial & medial forms of KASHMIRI YEH

The GAP

Priority

Tests

Action taken

Outcomes

#279 Wrong glyph digit shapes for extended arabic-indic digits

The GAP

Priority

Tests & results

Action taken

Outcomes

#276 Font fallback should allow selection of a Nastaliq font

The GAP

Priority

Tests & results

Action taken

Outcomes

#249 Lack of font support for Kashmiri characters makes text deviate from true semantics

The GAP

Priority

Tests & results

Action taken

Outcomes

3.2 Glyph shaping and positioning

#219 Controls needed to adjust diacritic positioning

3.3 Letterform slopes, weights, & italics

#218 Controls needed for italic slant direction

3.4 Cursive text

#222 Inline elements/styling break cursive shaping

#221 Text opacity shows glyph overlap

#220 Text stroke cuts joining glyphs apart

The GAP

Priority:

Tests

Action taken

Outcomes

3.5 Case & other character transforms

4. Typographic units

4.1 Characters & encoding

#249 Lack of font support for Kashmiri characters makes text deviate from true semantics

The GAP

Priority

Tests & results

Action taken

Outcomes

4.2 Grapheme/word segmentation & selection

5. Punctuation & inline features

5.1 Phrase & section boundaries

5.2 Quotations & citations

#224 q element produces incorrect quotation marks when language changes

5.3 Emphasis & highlighting

5.4 Abbreviation, ellipsis & repetition