Network Working Group                                   A. Phillips, Ed.
Internet-Draft                                                Yahoo! Inc
Obsoletes: 3066 (if approved)                              M. Davis, Ed.
Expires: August 10, 27, 2006                                          Google
                                                       February 6, 23, 2006

                       Matching of Language Tags
                      draft-ietf-ltru-matching-09
                      draft-ietf-ltru-matching-10

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on August 10, 27, 2006.

Copyright Notice

   Copyright (C) The Internet Society (2006).

Abstract

   This document describes different mechanisms for comparing, matching,
   and evaluating language tags.  Possible algorithms for language
   negotiation or content selection, filtering, and lookup are
   described.  This document, in combination with RFC 3066bis (replace (Ed.:
   replace "3066bis" with the RFC number assigned to
   draft-ietf-ltru-registry-14), replaces RFC 3066, which replaced RFC
   1766.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  The Language Range . . . . . . . . . . . . . . . . . . . . . .  4
     2.1.  Basic Language Range . . . . . . . . . . . . . . . . . . .  4
     2.2.  Extended Language Range  . . . . . . . . . . . . . . . . .  5
     2.3.  The Language Priority List . . . . . . . . . . . . . . . .  7  5
   3.  Types of Matching  . . . . . . . . . . . . . . . . . . . . . .  8  7
     3.1.  Choosing a Type of Matching  . . . . . . . . . . . . . . .  8  7
     3.2.  Filtering  . . . . . . . . . . . . . . . . . . . . . . . .  9  8
       3.2.1.  Filtering with  Basic Language Ranges Filtering  . . . . . . . . . . . 11
       3.2.2.  Filtering with Extended Language Ranges . . . . . . . 11
       3.2.3.  Scored Filtering .  9
       3.2.2.  Extended Filtering . . . . . . . . . . . . . . . . . . 11 10
     3.3.  Lookup . . . . . . . . . . . . . . . . . . . . . . . . . . 15 10
   4.  Other Considerations . . . . . . . . . . . . . . . . . . . . . 19 14
     4.1.  Choosing Language Ranges . . . . . . . . . . . . . . . . . 19 14
     4.2.  Meaning of Language Tags and Ranges  . . . . . . . . . . . 20 15
     4.3.  Considerations for Private Use Subtags . . . . . . . . . . 21 15
     4.4.  Length Considerations in Matching  . . . . . . . . . . . . 22 15
   5.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 24 17
   6.  Changes  . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 18
   7.  Security Considerations  . . . . . . . . . . . . . . . . . . . 26 19
   8.  Character Set Considerations . . . . . . . . . . . . . . . . . 27 20
   9.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 28 21
     9.1.  Normative References . . . . . . . . . . . . . . . . . . . 28 21
     9.2.  Informative References . . . . . . . . . . . . . . . . . . 28 21
   Appendix A.  Acknowledgements  . . . . . . . . . . . . . . . . . . 29 22
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 30 23
   Intellectual Property and Copyright Statements . . . . . . . . . . 31 24

1.  Introduction

   Human beings on our planet have, past and present, used a number of
   languages.  There are many reasons why one would want to identify the
   language used when presenting or requesting information.

   Information about a user's language preferences commonly needs to be
   identified so that appropriate processing can be applied.  For
   example, the user's language preferences in a browser can be used to
   select web pages appropriately.  Language preferences can also be
   used to select among tools (such as dictionaries) to assist in the
   processing information or understanding of content in different languages.

   Given a some
   specific set of information items or "content".

   One use for language identifiers, such as those defined in
   [RFC3066bis], various mechanisms can be envisioned for performing is to select content by matching the associated
   language negotiation and tag matching. tags to a user's language preferences.

   This document defines a syntax (called a language range (Section 2))
   for specifying a items in the user's language preferences, preferences (called a
   language priority list (Section 2.3)), as well as several schemes for
   selecting or filtering sets of content by comparing language
   ranges to the content's
   language tags [RFC3066bis] used to identify the natural
   language of that content. user's preferences.  Applications, protocols, or
   specifications will have varying needs and requirements that affect
   the choice of a suitable matching scheme.  Depending on the choice of
   scheme, there are various options left to the implementation.
   Protocols that implement a matching scheme either need to specify
   each particular choice or indicate the options that are left to the
   implementation to decide.

   This document is divided into three main sections.  One describes how
   to indicate a user's preferences using language ranges.  Then a
   section describes various schemes for matching these ranges to a set
   of language tags in order to select specific content. tags.  There is also a section that deals with various
   practical considerations that apply to implementing and using these
   schemes.

   This document, in combination with [RFC3066bis] (Ed.: replace
   "3066bis" globally in this document with the RFC number assigned to
   draft-ietf-ltru-registry-14), replaces [RFC3066], which replaced
   [RFC1766].

   The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

2.  The Language Range

   Language Tags [RFC3066bis] are used to identify the language of some
   information item or content.  Applications or protocols that use
   language tags are often faced with the problem of identifying sets of
   content that share certain language attributes.  For example,
   HTTP/1.1 [RFC2616] describes one such mechanism in its discussion of
   the Accept-Language header (Section 14.4), which is used when
   selecting content from servers based on the language of that content.

   When selecting content according to its language, it is useful to
   have a mechanism for identifying sets of language tags that share
   specific attributes.  This allows users to select or filter content
   based on specific requirements.  Such an identifier is called a
   "Language Range".

   There are different types of language range, whose specific
   attributes vary to match their application.  Language ranges are
   similar in structure and content to language tags: they consist of alphanumeric "subtags" a sequence of
   subtags separated by hyphens,
   plus hyphens.  In a special language range, each subtag consisting MUST
   either be a sequence of ASCII alphanumeric characters or the single
   character "*" '*' (%2A,
   ASTERISK), which ASTERISK).  The character '*' is used in ranges as a "wildcard", that is, a value "wildcard"
   that matches any subtag. sequence of subtags.  Restrictions on the meaning
   and use of wildcards vary according to the type of language range.

   Language tags and thus language ranges are to be treated as case-
   insensitive: there exist conventions for the capitalization of some
   of the subtags, but these MUST NOT be taken to carry meaning.
   Matching of language tags to language ranges MUST be done in a case-
   insensitive manner as well. manner.

2.1.  Basic Language Range

   A "basic language range" identifies the set of content whose language tags that all
   begin with the same sequence of subtags.  Each range consists of a
   sequence of alphanumeric subtags separated by hyphens.  The basic
   language range is defined by the following ABNF[RFC4234]: ABNF [RFC4234]:

   language-range   = language-tag (1*8ALPHA *("-" 1*8alphanum)) / "*"
   language-tag   = 1*8[alphanum] *["-" 1*8alphanum]
   alphanum         = ALPHA / DIGIT

   Basic language ranges (originally described by HTTP/1.1 [RFC2616] and
   later [RFC3066]) have the same syntax as an [RFC3066] language tag or
   are the single character "*".  They differ from the language tags
   defined in [RFC3066bis] only in that there is no requirement that
   they be "well-formed" or be validated against the IANA Language
   Subtag Registry (although such ill-formed ranges will probably not
   match anything).  (Note that the ABNF [RFC4234] in [RFC2616] is
   incorrect, since it disallows the use of digits anywhere in the
   'language-range': this is mentioned in the errata)

   Use of a basic language range seems to imply that there is a semantic
   relationship between language tags that share the same prefix.  While
   this is often the case, it is not always true and users should note
   that the set of language tags that match a specific language-range language range
   may not be represent mutually intelligible. intelligible languages.

2.2.  Extended Language Range

   A

   Basic Language Range does not always provide the most appropriate
   way language ranges allow users to specify a user's preferences.  Sometimes it is beneficial to
   use a more fine-grained matching scheme that takes advantage of the
   internal structure set of language tags.  This allows tags
   that share the user same initial subtags.  Occasionally users will wish to
   specify, for example,
   select a set of language tags based on the value presence of a specific field in
   subtags.  For example, a language tag
   or user might wish to indicate which values are of interest in filtering or selecting
   the content.

   In an extended select all language range, the identifier takes tags
   that contains the form of region subtag 'CH'.  Extended language ranges are
   useful in specifying a
   series of subtags which MUST consist particular sequence of well-formed subtags or the
   special subtag "*".  For example, that appear in
   the language range "en-*-US"
   specifies a primary language set of matching tags without having to specify all of 'en', followed by any script subtag,
   followed by the region subtag 'US'.
   intervening subtags.

   An extended language range can be represented by the following ABNF:

   extended-language-range = range ; a range
                 / privateuse       ; a private-use range (1*8ALPHA / grandfathered    ; a grandfathered registration

   range         = (language
                    ["-" script]
                    ["-" region]
                    *("-" variant) "*")
                             *("-" extension)
                    ["-" privateuse])

   language      = (2*3ALPHA [ extlang ]) ; shortest ISO 639 code
                 / 4ALPHA                 ; reserved for future use
                 / 5*8ALPHA               ; registered language subtag
                 / "*"                    ; or wildcard

   extlang       = *2("-" 3ALPHA) ("-" ( 3ALPHA (1*8alphanum / "*"))
                                          ; reserved for future use
                                          ; wildcard can only appear
                                          ;   at the end

   script        = 4ALPHA                 ; ISO 15924 code
                 / "*"                    ; or wildcard

   region        = 2ALPHA                 ; ISO 3166 code
                 / 3DIGIT                 ; UN M.49 code
                 / "*"                    ; or wildcard

   variant       = 5*8alphanum            ; registered variants
                 / (DIGIT 3alphanum)      ;
                 / "*"                    ; or wildcard

   extension     = singleton *("-" (2*8alphanum)) [ "-*" ]
                                          ; extension subtags
                                          ;

   Figure 2: Extended Language Range

   The wildcard can only appear
                                          ;   at the end

   singleton     = %x41-57 / %x59-5A / %x61-77 / %x79-7A / DIGIT
                 ; single letters (except for "x") or digits

   privateuse    = "x" 1*("-" (1*8alphanum))

   grandfathered = 1*3ALPHA 1*2("-" (2*8alphanum))
                   ; grandfathered registration
                   ; Note: I is the only singleton
                   ; that starts a grandfathered tag

   alphanum      = (ALPHA / DIGIT)       ; letters and numbers
   A field not present subtag '*' MAY occur in any position in the middle of an extended
   language range is
   treated as if the field contained a "*".  Implementations range, where it matches any sequence of subtags that
   normalize might
   occur in that position in a language tag.  However wildcards outside
   the first position in an extended language ranges range are ignored by most
   matching schemes.  Use of multiple wildcards SHOULD expand missing fields to NOT be
   "*" so taken to
   imply that the semantic meaning a certain number of subtags will appear in the matching
   set of language range is clear tags.

   Implementations that specify basic ranges MAY map extended language
   ranges to basic language ranges: if the user.  At the same time, multiple wildcards in a row are
   redundant and implementations SHOULD collapse these to first subtag is a single
   wildcard when normalizing "*" then
   the entire range (for brevity). is treated as "*" (which matches the default
   content), otherwise each wildcard subtag is removed.  For example, both if
   the language range "sl-nedis" and were "en-*-US", then the range "sl-*-*-nedis" are equivalent to
   and should would be normalized as "sl-*-nedis". mapped to
   "en-US".

2.3.  The Language Priority List

   When users specify a language preference they often need to specify a
   prioritized list of language ranges in order to best reflect their
   language preferences.  This is especially true for speakers of
   minority languages.  A speaker of Breton in France, for example, may
   specify "be" followed by "fr", meaning that if Breton is available,
   it is preferred, but otherwise French is the best alternative.  It
   can get more complex: a speaker may wish to fall back from Skolt Sami
   to Northern Sami to Finnish.

   A "Language Priority List" is a prioritized or weighted list of
   language ranges.  One well known example of such a list is the
   "Accept-Language" header defined in RFC 2616 [RFC2616] (see Section
   14.4) and RFC 3282 [RFC3282].  A simple list of ranges, i.e. one that
   contains no weighting information, is considered to be in descending
   order of priority.

   The various matching operations described in this document include
   considerations for using a language priority list.  This document
   does not define any syntax for a language priority list; defining
   such a syntax is the responsibility of the protocol, application, or
   implementation that uses it.  When given as examples in this
   document, language priority lists will be shown as a quoted sequence
   of ranges separated by semi-colons, semicolons, like this: "en; fr; zh-Hant"
   (which would be read as "English before French before Chinese as
   written in the Traditional script").

   Where a language priority list provides "quality weights" for the
   language ranges, such as the use of Q weights in the syntax of the
   "Accept-Language" header (defined in [RFC2616], Section 14.4, and
   [RFC3282]), language ranges without a weight are given values equal
   to the value of the previous language range (processing from first to
   last).  If the first language range has no weight, it is given a
   value of 1.0.  Then language ranges with zero weights are removed.
   For example, "fr, en;q=0.5, de, it" becomes "fr;q=1.0, en;q=0.5,
   de;q=0.5, it;q=0.5".  The language priority list is then sorted from
   highest priority to lowest, with language ranges that share the same
   weights remain in the same order as in the original language priority
   list.

3.  Types of Matching

   Matching language ranges to language tags can be done in a number of
   different ways.  This section describes several different matching
   schemes, as well as the considerations for choosing between them.
   Protocols and specifications SHOULD clearly indicate the particular
   mechanism used in selecting or matching language tags.

   There are two basic several types of matching scheme: scheme.  This document presents
   two types: those that produce zero or more information items (called
   "filtering") and those that produce a single information item for a
   given request (called "lookup").

   A key difference between these two types of

   Implementations or protocols MAY use different matching scheme is that schemes than
   the language ranges ones described in the language priority list represent the
   _least_ specific content one will accept this document, as a match, while for lookup
   operations the language ranges represent the _most_ specific content. long as those mechanisms are
   clearly specified.

3.1.  Choosing a Type of Matching

   Applications, protocols, and specifications are faced with the
   decision of what type of matching to use.  Sometimes, different
   styles of matching might be suited for different kinds of processing
   within a particular application or protocol.

   Language tag matching is a tool, and does not by itself specify a
   complete procedure for the use of language tags.  Such procedures are
   intimately tied to the application protocol in which they occur.
   When specifying a protocol operation using matching, the protocol
   MUST specify:

   o  Which type(s) of language tag matching it uses

   o  Whether the operation returns a single result (lookup) or a
      possibly empty set of results (filtering)

   o  For lookup, what the result is when no matching tag is found.  For
      instance, a protocol might define the result in as failure of the
      operation, an empty value, returning some protocol defined or
      implementation defined default, or returning i-default [RFC2277].

   This document describes three types of matching:

   1.  Basic Filtering (Section 3.2.1) matches a language priority list
       consisting of basic language ranges (Section 2.1) to sets of
       language tags.

   2.  Extended Filtering (Section 3.2.2) matches a language priority
       list consisting of extended language ranges (Section 2.2) to sets
       of language tags.

   3.  Lookup (Section 3.3) matches a language priority list consisting
       of basic language ranges to sets of language tags find the
       _exactly_ one language tag that best matches the range.

   Both types of filtering can be used to produce a set of results (such
   as a collection of documents). documents) by comparing the user's preferences to
   language tags associated with the set of content.  For example, if using when
   performing a search engine, search, one might use filtering to limit the results to
   documents tagged as being written in French.  It can  They might also be used
   when deciding whether to perform a language-sensitive process on some
   content.  For example, a process might cause paragraphs whose
   language tag matched the language range "nl" to be displayed in
   italics within a document.

   This document describes four types of matching (three types of
   filtering, plus the lookup scheme):

   1.  Basic Filtering (Section 3.2.1) is used to match content using
       basic language ranges (Section 2.1).

   2.  Extended Range Filtering (Section 3.2.2) is used to match content
       using extended language ranges (Section 2.2).

   3.  Scored Filtering (Section 3.2.3)

   Lookup produces an ordered set of
       content using extended language ranges.  It SHOULD be used when the quality of the match within a specific language range is
       important, as when presenting single result that best matches a list given set of documents resulting from
       a search.

   4.  Lookup (Section 3.3)
   user preferences, so it is used when each request needs to produce
       _exactly_ one piece of content. useful in cases in which only a single
   item can be returned.  For example, if a process were to insert a
   human readable error message into a protocol header, it might select
   the text based on the user's language preference. priority list.  Since it the
   process can return only one item, it must choose a single item and it
   must return some item, even if no content content's language tag matches the
   language priority list supplied by the user.

   Most

   The types of matching in this document are designed so that
   implementations are not required to validate or understand any of the
   semantics of the language tags or ranges or of the subtags supplied and, except for scored filtering,
   they do not need in them.
   None of them require access to the IANA Language Subtag Registry (see
   Section 3 in [RFC3066bis]).  This simplifies and speeds the
   performance of implementations.

   Regardless of the matching scheme chosen, protocols and
   implementations MAY canonicalize language tags and ranges by mapping
   grandfathered and obsolete tags or subtags into modern equivalents.
   If an implementation canonicalizes either ranges or tags, then the
   implementation will require the IANA Language Subtag Registry
   information for that purpose.  Implementations MAY also use semantic
   information external to the registry when matching tags.  For
   example, the primary language subtags 'nn' (Nynorsk Norwegian) and
   'nb' (Bokmal Norwegian) might both be usefully matched to the more
   general subtag 'no' (Norwegian).  Or an implementation might infer
   that content labeled "zh-CN" is more likely to match the range "zh-
   Hans" than equivalent content labeled "zh-TW".

3.2.  Filtering

   Filtering is used to select the set of content language tags that matches a
   given language priority list. list and return the associated content.  It
   is called "filtering" because this set of
   content may might contain no items at all
   or it may might return an arbitrarily large number of matching items: as
   many items as match the language priority list, thus "filtering out"
   the non-matching items.

   In filtering, the language range represents the _least_ specific
   (that is, the fewest number of subtags) language tag which is an
   acceptable match.  That is, all  All of the language tags in the matching set of
   filtered content
   tags will have an equal or greater number of subtags than the
   language range.  Every non-wildcard subtag in the language range will
   appear in every one of the matching language tags.  For example, if
   the language priority list consists of the range "de-CH", one might
   see matching content with
   the tag tags such as "de-CH-1996" but one will never see a match with the tag
   "de". such as
   "de" (because the 'CH' subtag is missing).

   If the language priority list (see Section 2.3) contains more than
   one range, the content returned is typically ordered in descending
   level of preference.

   Some examples of applications where filtering might be appropriate
   include:

   o  Applying a style to sections of a document in a particular set of
      languages.

   o  Displaying the set of documents containing a particular set of
      keywords written in a specific set of languages.

   o  Selecting all email items written in a specific set of languages.

   Filtering can produce

   The content returned MAY either an be ordered or an unordered set of
   results.  For example, applying formatting according to a document based on the
   language of specific pieces of content does not require
   the content
   to be ordered.  It is sufficient to know whether a specific piece of
   content is selected by priority in the language priority list (or not).  A search
   application, on the (and other hand, probably would want criteria),
   according to order the
   results.

   If an ordered set is desired, as described above, then the
   application or protocol needs to determine the relative "quality" of the match between different application or protocol.

3.2.1.  Basic Filtering

   When filtering using basic language tags and ranges, each basic language range
   in the language range.

   This measurement priority list is called a "distance metric".  A distance metric
   assigns a numeric value considered in turn, according to the comparison of a
   priority.  A particular language tag to matches a language range that represents the 'distance' between if it
   exactly equals the two.  A
   distance of zero means that they are identical, a small distance
   indicates that they are very similar, and tag, or if it exactly equals a large distance indicates
   that they are very different.  Using a distance metric,
   implementations can, for example, allow users to select a threshold
   distance for a match to be "successful" while filtering, or they
   might use the numeric values to order the results.

3.2.1.  Filtering with Basic Language Ranges

   When filtering using basic language ranges, each basic language range
   in the language priority list is considered in turn, according to
   priority.  A particular language tag matches a language range if it
   exactly equals the tag, or if it exactly equals a prefix of the tag
   such prefix of the tag
   such that the first character following the prefix is "-".  (That is,  For
   example, the language-range "de-de" matches the language tag "de-DE-1996", "de-DE-
   1996", but not the language tag "de-Deva".) tags "de-Deva" or "de-Latn-DE".

   The special range "*" in a language priority list matches any tag.  A
   protocol which uses language ranges MAY specify additional rules
   about the semantics of "*"; for instance, HTTP/1.1 [RFC2616]
   specifies that the range "*" matches only languages not matched by
   any other range within an "Accept-Language" header.

3.2.2.  Filtering with  Extended Language Ranges Filtering

   When filtering using extended language ranges, each extended language
   range in the language priority list is considered in turn, according
   to priority.  The subtags in each extended  A particular language range are is compared to each
   language tag using the corresponding subtags following process:

   Compare the first subtag in the extended language tag being
   examined.  The to the first
   subtag from in the language tag in a case insensitive manner.  If the
   first subtag in the range is considered to match if "*", it
   exactly matches any value.  Otherwise
   the two values must match or the corresponding overall match fails.

   Take each non-wildcard subtag in the language range and compare it to
   the next subtag in the language tag in turn until a matching subtag
   is found or the range's langauge tag is exhausted.  If the end of the
   language tag is found first, the match fails.  If a match is found,
   this step is repeated with the next non-wildcard subtag has in the value "*" (which matches all subtags, including
   language range (and beginning with the
   empty subtag). next subtag in the language
   tag) until the list of subtags in the language range is exhausted or
   the match fails.

   Subtags not specified, including those at the end of the language
   range, are thus treated as if assigned the wildcard value "*".  This makes each range
   into a prefix
   Extended filtering works, therefore, much like that used in basic language range matching. filtering.  For
   example, the extended language range "de-*-DE" matches all of the
   following tags, in part because the unspecified variant, extension,
   and private-use subtags are expanded to "*": tags:

      de-DE

      de-Latn-DE

      de-Latf-DE

      de-DE-x-goethe

      de-Latn-DE-1996

3.2.3.  Scored Filtering

   Both basic and extended language range filtering produce simple
   boolean matches between a language range and a language tag.

   Sometimes it may be useful

3.3.  Lookup

   Lookup is used to provide an array of results with
   different levels of matching, for example, sorting results based on select the overall "quality" of single language tag that best matches
   the match.  Scored (or "distance metric")
   filtering provides language priority list for a way to generate these quality values.

   As with the other forms of filtering, given request and return the process considers
   associated content.  When performing lookup, each language range in
   the language priority list is considered in order of turn, according to
   priority.

   Each extended  By contrast with filtering, each language range and language tag MUST first be
   canonicalized by mapping grandfathered and obsolete tags into modern
   equivalents.  This requires the information in represents
   the IANA Language
   Subtag Registry (see Section 3 of [RFC3066bis]).

   The language range and each language _most_ specific tag it which is to be compared to are
   then transformed into an acceptable match.  The first
   content found with a "quintuple" consisting of five "elements" in matching tag, according to the form (language, script, country, variant, extension).

   Any extended language subtags are user's priority,
   is considered part of the language
   "element". closest match and is the content returned.  For
   example, if the language element for range is "de-ch", a lookup operation might
   produce content with the tag "zh-cmn-
   Hans" would be "zh-cmn".

   Private-use subtag sequences are considered part of tags "de" or "de-CH" but never one with the language
   "element"
   tag "de-CH-1996".  Usually if in the initial position in no content matches the tag and part of request, the
   variant "element"
   "default" content is returned.

   For example, if not.  The different handling of private-use
   sequences prevents a range such as "x-twain" from matching all
   possible tags, while an application inserts some dynamic content into a range such as "en-US-x-twain" would closely
   document, returning an empty string if there is no exact match nearly all tags for English as used in the United States.

   Language subtags 'und', 'mul', and is not
   an option.  Instead, the script subtag 'Zyyy' are
   converted to "*": these subtag values represent undetermined,
   multiple, or private-use values which are consistent application "falls back" until it finds a
   matching language tag associated with the use a suitable piece of
   the wildcard.

   For language tags that have no script subtag but whose language
   subtag's record in the IANA Language Subtag Registry contains the
   field "Suppress-Script", the script element in the quintuple MUST be
   set content to
   insert.  Examples of lookup might include:

   o  Selection of a template containing the script subtag text for an automated email
      response.

   o  Selection of a item containing some text for inclusion in the Suppress-Script field.  This is
   necessary because [RFC3066bis] strongly recommends that users not use
   this subtag to form language tags and this document (see Section 4.1)
   recommends that users not use them to form ranges.  Languages which
   have a "Suppress-Script" field
      particular Web page.

   o  Selection of a string of text for inclusion in an error log.

   In the registry are predominantly
   written in that single script, making lookup scheme, the subtag redundant in forming
   a language tag or range.  Thus if range is progressively truncated
   from the script were not expanded in
   this manner, end until a matching piece of content is located.  For
   example, starting with the range such "zh-Hant-CN-x-private", the lookup
   progressively searches for content as "de-DE" would produce shown below:

   Range to match: zh-Hant-CN-x-private
   1. zh-Hant-CN-x-private
   2. zh-Hant-CN
   3. zh-Hant
   4. zh
   5. (default content)

   Figure 3: Example of a Lookup Fallback Pattern

   This scheme allows some flexibility in finding a more-distant
   score match.  For example,
   lookup provides better results for cases in which content is not
   available that happened to be labeled "de-Latn-DE" exactly matches the user request than users
   would expect that it should.

   Any remaining missing components in if the default
   language tag are set to "*";
   thus an empty language tag becomes the quintuple ("*", "*", "*", "*",
   "*").  Missing components in the language range are handled similarly
   to extended range lookup: missing internal subtags are expanded to
   "*".  Missing end subtags are expanded as the empty string.  Thus a
   pattern "en-US" becomes the quintuple ("en","*","US","","").

   Here are some examples of language tags, showing their quintuples as
   both language tags and language ranges:

   en-US
      Tag:   (en, *, US, *, *)
      Range: (en, *, US, "", "")

   sr-Latn
      Tag:   (sr, Latn, *, *, *)
      Range: (sr, Latn, "", "", "")

   zh-cmn-Hant
      Tag:   (zh-cmn, Hant, *, *, *)
      Range: (zh-cmn, Hant, "", "", "")

   x-foo
      Tag:   (x-foo, *, *, *, *)
      Range: (x-foo, "", "", "", "")

   en-x-foo
      Tag:   (en, *, *, x-foo, *)
      Range: (en, *, *, x-foo, "")

   i-default
      Tag:   (i-default, *, *, *, *)
      Range: (i-default, "", "", "", "")

   sl-Latn-IT-rozaj
      Tag:   (sl, Latn, IT, rozaj, *)
      Range: (sl, Latn, IT, rozaj, "")

   zh-r-wadegile (hypothetical)
      Tag:   (zh, *, *, *, r-wadegile)
      Range: (zh, *, *, *, r-wadegile)

   Figure 3: Examples of Distance Metric Quintuples

   Each pair of quintuples being compared is assigned a distance value,
   in which small values indicate better matches and large values
   indicate worse ones.  The distance between the pair is the sum of the
   distances for each of the corresponding elements of the quintuple.
   If the elements are identical or one is '*', then the distance value
   between them is zero.  Otherwise, it is given by the following table:
     256    language mismatch
     128    script mismatch
      32    region mismatch
       4    variant mismatch
       1    extension mismatch

   A value of 0 is a perfect match; 421 is no match at all.  Different
   threshold values might be appropriate for different applications or
   protocols.  Implementations will usually allow users to choose the
   most appropriate selection value, ranking the matched items based on
   score.

   Examples of various tag's distances from the range "en-US":

   "fr-FR"          384 (language & region mismatch)
   "fr"             256 (language mismatch, region match)
   "en-GB"           32 (region mismatch)
   "en-Latn-US"       0 (all fields match)
   "en-Brai"         32 (region mismatch)
   "en-US-x-foo"      4 (variant mismatch: range is the empty string)
   "en-US-r-wadegile" 1 (extension mismatch: range is the empty string)

   Where a language priority list follows the syntax of the "Accept-
   Language" header defined in [RFC2616] (see Section 14.4) and
   [RFC3282], language ranges without a Q value are given values equal
   to the value of the previous language range in the list (processing
   from first to last).  If the first language range has no Q value, it
   is given a value of 1.0.  Language ranges with Q values of zero are
   removed.  For example, "fr, en;q=0.5, de, it" becomes
   "fr;q=1.0,en;q=0.5,de;q=0.5,it;q=0.5".  The distance values given
   above are then divided by the Q values.  For example, if that
   language tag "fr-FR" has a distance of 384 from a language range with
   a Q value of 0.8, then the resulting distance is 480 (384 div 0.8).

   Implementations or protocols MAY use different weighting systems than
   the ones described above, as long as the weightings and weighting
   mechanisms are clearly specified.  Thus, for example, an
   implementation or protocol could give all language tags with missing
   Q values a value of 1.0, or give the distance value 1000 to a
   language mismatch.  They MAY also use more sophisticated weights that
   depend on the values of the corresponding elements.  For example, an
   implementation might give a small distance to the difference closely
   related subtags.  Some examples of closely related subtags might be:

   Language:
     no (Norwegian)
     nb (Bokmal Norwegian)
     nn (Nynorsk Norwegian)

   Script:
     Kata (katakana)
     Hira (hiragana)

   Region:
     US (United States of America)
     UM (United States Minor Outlying Islands)

   Figure 6: Examples of Closely Related Subtags

3.3.  Lookup

   Lookup is used to select the single information item that best
   matches the language priority list for a given request.  When
   performing lookup, each language range in the language priority list
   is considered in turn, according to priority.  By contrast with
   filtering, each language ranges represents the _most_ specific tag
   which is an acceptable match.  The first information item found with
   a matching tag, according the user's priority, is considered the
   closest match and is the item returned.  For example, if the language
   range is "de-CH", one might expect to receive an information item
   with the tag "de" but never one with the tag "de-CH-1996".  Usually
   if no content matches the request, a "default" item is returned.

   For example, if an application inserts some dynamic content into a
   document, returning an empty string if there is no exact match is not
   an option.  Instead, the application "falls back" until it finds a
   suitable piece of content to insert.  Other examples of lookup might
   include:

   o  Selection of a template containing the text for an automated email
      response.

   o  Selection of a item containing some text for inclusion in a
      particular Web page.

   o  Selection of a string of text for inclusion in an error log.

   In the lookup scheme, the language range is progressively truncated
   from the end until a matching piece of content is located.  For
   example, starting with the range "zh-Hant-CN-x-private", the lookup
   progressively searches for content as shown below:

   Range to match: zh-Hant-CN-x-private
   1. zh-Hant-CN-x-private
   2. zh-Hant-CN
   3. zh-Hant
   4. zh
   5. (default content or the empty tag)

   Figure 7: Example of a Lookup Fallback Pattern

   This scheme allows some flexibility in finding content.  For example,
   it provides better results for cases in which data is not available
   that exactly matches the user request than if the default language
   for the system or content were returned immediately.  Not every
   specific level of tag granularity is usually available or language
   content may be sparsely populated, so "falling back" through the
   subtag sequence provides more opportunity to find a match between
   available content and the user's request.

   The default content is implementation defined.  It might be content
   with no language tag; might have an empty value (the built-in
   attribute xml:lang in [XML10] permits the empty value); might be a
   particular language designated for that bit of content; or it might
   be content that is labeled with the tag "i-default" (see [RFC2277]).
   When performing lookup using a language priority list, the
   progressive search MUST proceed to consider each language range in
   the list before finding the default content or empty tag.

   One common way for an application or implementation to provide for
   default content is to allow a specific language range to be set as
   the default for a specific type of request.  This language range is
   then treated as if it were appended to the end of the language
   priority list as a whole, rather than after each item in the language
   priority list.

   For example, if a particular user's language priority list were
   "fr-FR; zh-Hant" and the program doing the matching had a default
   language range of "ja-JP", the program would search for content as
   follows:
   1. fr-FR
   2. fr
   3. zh-Hant // next language
   4. zh
   5. (search for the default content)
      a. ja-JP
      b. ja
      c. (implementation defined default)

   Figure 8: Lookup Using a Language Priority List
   Implementations SHOULD ignore extensions and unrecognized private-use
   subtags when performing lookup, since these subtags are usually
   orthogonal to the user's request.

   The special language range "*" matches any language tag.  In the
   lookup scheme, this range does not convey enough information by
   itself to determine which content is most appropriate, since it
   matches everything.  If the language range "*" is the only one in the
   language priority list, it matches the default content.  If the
   language range "*" is followed by other language ranges, it should be
   skipped.

   In some cases, the language priority list might contain one or more
   extended language ranges (as, for example, when the same language
   priority list is used as input for both lookup and filtering
   operations).  Wildcard values in an extended language range normally
   match any value that occurs in that position in a language tag.
   Since only one item can be returned for any given lookup request,
   wildcards in a language range have to be processed in a consistent
   manner or the same request will produce widely varying results.
   Implementations that accept extended language ranges MUST define
   which content is returned when more than one item matches the
   extended language range.

   For example, an implementation could return the matching content that
   is first in ASCII-order.  For example, if the language range were
   "*-CH" and the set of content included "de-CH", "fr-CH", and "it-CH",
   then the content labeled "de-CH" would be returned.

   Implementations MAY also map extended language ranges to basic
   language ranges: if the first subtag is a "*" then the entire range
   is treated as "*" (which matches the default content), otherwise each
   wildcard subtag is removed.  For example, if the language range were
   "en-*-US", then the range would be mapped to "en-US".

   Where a language priority list contains Q values as in the syntax of
   the "Accept-Language" header defined in [RFC2616] (see Section 14.4)
   and [RFC3282], language tags without a Q value are given values equal
   to the value of the previous language tag (processing from first to
   last).  If the first language tag has no Q value, it is given a value
   of 1.0.  Then language tags with zero Q values are removed.  For
   example, "fr, en;q=0.5, de, it" becomes "fr;q=1.0, en;q=0.5,
   de;q=0.5, it;q=0.5".  The language priority list is then sorted from
   highest priority to lowest, whereby any two language tags with the
   same Q values are remain in the same order as in the original
   language priority list.  This list is then traversed as described
   above in doing lookup.

   Implementations or protocols MAY use different lookup mechanisms
   systems than the ones described above, as long as those mechanisms
   are clearly specified.

4.  Other Considerations

   When working with language ranges and matching schemes, there are
   some additional points that may influence the choice of either.

4.1.  Choosing Language Ranges

   Users indicate their language preferences via the choice of a
   language range or the list of language ranges in a language priority
   list.  The type of matching affects what the best choice is for a
   given user.

   Most matching schemes make no attempt to process the semantic meaning
   of the subtags.  The language range (or its subtags) is usually
   compared in a case-insensitive manner to each language tag being
   matched, using basic string processing.

   Users SHOULD avoid subtags that add no distinguishing value to a
   language range.  Generally, the fewer subtags that appear in the
   language range, the more system or content the range will match.

   Most notably, script subtags SHOULD NOT be used to form a language
   range in combination with language subtags that have a matching
   Suppress-Script field in their registry entry.  Thus the language
   range "en-Latn" is probably inappropriate in most cases (because the
   vast majority of English documents are written in the Latin script
   and thus the 'en' language subtag has a Suppress-Script field for
   'Latn' in the registry).

   When working with tags and ranges, note that extensions and most
   private-use subtags are orthogonal to language tag matching, in that
   they specify additional attributes of the text not related to the
   goals of most matching schemes.  Users SHOULD avoid using these
   subtags in language ranges, since they interfere with the selection
   of available content.  When used in language tags (as opposed to
   ranges), these subtags normally do not interfere with filtering
   (Section 3), since they appear at the end were returned immediately.  Not
   every specific level of the tag and will match
   all prefixes.

   When working with language tags and granularity is usually available or
   language ranges note that:

   o  Private-use and Extension subtags are normally orthogonal content may be sparsely populated, so "falling back" through
   the subtag sequence provides more opportunity to
      language tag fallback.  Implementations or specifications that use find a lookup (Section 3.3) matching scheme often ignore unrecognized
      private-use match between
   available language tags and extension subtags the user's request.

   The default behavior when performing language no tag
      fallback.  In addition, since these subtags are always at the end
      of matches the sequence of subtags, their use in language tags normally
      doesn't interfere priority list
   is implementation defined.  An implementation might, for example,
   return content with the use of ranges that omit them in the
      filtering (Section 3.2) matching schemes described below.
      However, they do interfere no language tag; might supply content with filtering when used in an
   empty language
      ranges and SHOULD be avoided tag value (the built-in attribute xml:lang in ranges as [XML10]
   permits the empty value); might be a result.

   o  Applications, specifications, or protocols that choose not to
      interpret one or more private-use or extension subtags SHOULD NOT
      remove or modify these extensions in particular language designated
   for the bit of content that they are
      processing. being selected; or it might select the tag
   "i-default" (see [RFC2277]).  When performing lookup using a language tag instance is
   priority list, the progressive search MUST proceed to be used consider each
   language range in the list before finding the default content or
   empty tag.

   One common way for an application or implementation to provide for a
      specific, known protocol, and
   default is not being passed through to other
      protocols, allow a specific language tags MAY be filtered range to remove subtags and
      extensions that are not supported by that protocol.  Such
      filtering SHOULD be avoided, set as the
   default for a specific type of request.  This language range is then
   treated as if possible, since it removes
      information that might be relevant were appended to services on the other end of the protocol that would make use of that information.

   o  Some applications language priority
   list as a whole, rather than after each item in the language priority
   list.

   For example, if a particular user's language priority list were
   "fr-FR; zh-Hant" and the program doing the matching had a default
   language range of "ja-JP", the program would search for content as
   follows:
   1. fr-FR
   2. fr
   3. zh-Hant // next language tags might want or need to consider
   4. zh
   5. (search for the default content)
      a. ja-JP
      b. ja
      c. (implementation defined default)

   Figure 4: Lookup Using a Language Priority List

   Implementations SHOULD ignore extensions and unrecognized private-use
   subtags when matching tags.  If
      extensions and private-use performing lookup, since these subtags are included in a matching or
      filtering process that utilizes one of usually
   orthogonal to the schemes described in user's request.

   The special language range "*" matches any language tag.  In the
   lookup scheme, this document, then range does not convey enough information by
   itself to determine which content is most appropriate, since it
   matches everything.  If the implementation SHOULD canonicalize language range "*" is the only one in the
   language priority list, it matches the default content.  If the
   language range "*" is followed by other language ranges, it should be
   skipped.

   In some cases, the language tags and/or priority list might contain one or more
   extended language ranges before performing (as, for example, when the matching.  Note
      that same language tag processors that claim to be "well-formed"
      processors
   priority list is used as defined in [RFC3066bis] generally fall into this
      category.

4.2.  Meaning of Language Tags input for both lookup and Ranges

   Selecting content using language ranges requires some understanding
   by users of what they are selecting.  A filtering
   operations).  Wildcard values in an extended language tag or range
   identifies normally
   match any value that occurs in that position in a language as spoken (or written, signed or otherwise
   signaled) by human beings tag.
   Since only one item can be returned for communication of information to other
   human beings.

   If any given lookup request,
   wildcards in a language tag B contains language tag A as range have to be processed in a prefix, then B is
   typically "narrower" consistent
   manner or "more specific" than A. For example, "zh-
   Hant-TW" the same request will produce widely varying results.
   Implementations that accept extended language ranges MUST define
   which content is returned when more specific than "zh-Hant".

   This relationship is not guaranteed in all cases: specifically,
   languages that begin with one item matches the same sequence of subtags are NOT
   guaranteed to be mutually intelligible, although they might be.
   extended language range.

   For example, an implementation could return the matching tag "az" shares a prefix with both "az-Latn"
   (Azerbaijani written using that is
   first in ASCII-order.  If the Latin script) language range were "*-CH" and "az-Arab"
   (Azerbaijani written using the Arabic script).  A person fluent in
   one script might not be able to read the other, even though set
   of tags included "de-CH", "fr-CH", and "it-CH", then the text
   might tag "de-CH"
   would be otherwise identical.  Content tagged as "az" most probably
   is written in just one script and thus might not returned.  Another example would be intelligible for an implementation to a
   reader familiar with
   map the other script.

   Variant subtags in particular seem to represent specific divisions in
   mutual understanding, since they often encode dialects or other
   idiosyncratic variations within a language.  They also seem extended language ranges to
   represent relatively low divisions basic ranges.

4.  Other Considerations

   When working with a high chance of at least
   limited understanding, although this depends on the specific variant
   in question.

   The relationship between the language tag ranges and matching schemes, there are
   some additional points that may influence the information it
   relates to is defined by the standard describing choice of either.

4.1.  Choosing Language Ranges

   Users indicate their language preferences via the context in which
   it appears.  Accordingly, this section can only give possible
   examples choice of its usage:

   o  For a single information object, the associated
   language tags
      might be interpreted as range or the set list of languages that are necessary
      for language ranges in a complete comprehension language priority
   list.  The type of matching affects what the complete object.  Example:
      Plain text documents.

   o  For an aggregation best choice is for a
   given user.

   Most matching schemes make no attempt to process the semantic meaning
   of information objects, the associated subtags.  The language
      tags could be taken as the set of languages used inside components
      of that aggregation.  Examples: Document stores and libraries.

   o  For information objects whose purpose range (or its subtags) is usually
   compared in a case-insensitive manner to provide alternatives, each language tag being
   matched, using basic string processing.

   Users SHOULD avoid subtags that add no distinguishing value to a
   language range.  Generally, the fewer subtags that appear in the associated
   language tags could range, the more content the range will match.

   Most notably, script subtags SHOULD NOT be regarded as used to form a hint language
   range in combination with language subtags that have a matching
   Suppress-Script field in their registry entry.  Thus the
      content language
   range "en-Latn" is provided probably inappropriate in most cases (because the
   vast majority of English documents are written in several languages, the Latin script
   and that one has to
      inspect each of thus the alternatives in order to find its 'en' language or
      languages.  In this case, subtag has a Suppress-Script field for
   'Latn' in the presence of multiple registry).

   When working with tags might not
      mean and ranges, note that one needs to be multi-lingual to get complete
      understanding of the document.  Example: MIME multipart/
      alternative.

   o  In markup languages, such as HTML extensions and XML, language information
      can be added most
   private-use subtags are orthogonal to each part language tag matching, in that
   they specify additional attributes of the document identified by the markup
      structure (including the whole document itself).  For example, one
      could write <span lang="FR">C'est la vie.</span> inside a
      Norwegian document; the Norwegian-speaking user could then access
      a French-Norwegian dictionary text not related to find out what the marked section
      meant.  If
   goals of most matching schemes.  Users SHOULD avoid using these
   subtags in language ranges, since they interfere with the user were listening to that document through a
      speech synthesis interface, this formation could be selection
   of available content.  When used in language tags (as opposed to signal
   ranges), these subtags normally do not interfere with filtering
   (Section 3), since they appear at the synthesizer to appropriately apply French text-to-speech
      pronunciation rules to that span of text, instead end of misapplying the Norwegian rules.

4.3.  Considerations for Private Use Subtags tag and will match
   all prefixes.

   Private-use and Extension subtags require private agreement between the parties
   that intend are normally orthogonal to use or exchange language tags
   tag fallback.  Implementations or specifications that use them and great
   caution SHOULD be used in employing them in content or protocols
   intended for general use.  Private-use subtags are simply useless for
   information exchange without prior arrangement.

   The value and semantic meaning of a lookup
   (Section 3.3) matching scheme often ignore unrecognized private-use tags
   and of the extension subtags
   used within such a when performing language tag fallback.  In
   addition, since these subtags are not defined.  Matching private- always at the end of the sequence
   of subtags, their use tags using in language tags normally doesn't interfere
   with the use of ranges or extended that omit them in the filtering (Section 3.2)
   matching schemes described below.  However, they do interfere with
   filtering when used in language ranges can result
   in unpredictable content being returned.

4.4.  Length Considerations and SHOULD be avoided in Matching

   RFC 3066 [RFC3066] did
   ranges as a result.

   Applications, specifications, or protocols that choose not provide an upper limit on the size of
   language tags to
   interpret one or ranges.  RFC 3066 did define the semantics of
   particular more private-use or extension subtags SHOULD NOT
   remove or modify these extensions in such a way content that most language tags or ranges
   consisted of they are
   processing.  When a language and region subtags with tag instance is to be used in a combined total length
   of up
   specific, known protocol, and is not being passed through to six characters.  Larger other
   protocols, language tags and ranges (in terms of both MAY be filtered to remove subtags and characters) did exist, however.

   [RFC3066bis] also does
   extensions that are not impose a fixed upper limit supported by that protocol.  Such filtering
   SHOULD be avoided, if possible, since it removes information that
   might be relevant to services on the number other end of subtags in a language tag or range (and thus an upper bound on the
   size protocol that
   would make use of either).  The syntax in that document suggests that,
   depending on the specific information.

   Some applications of language tags might want or range of languages, more
   subtags (and thus characters) are sometimes necessary as a result.
   Length considerations and their impact on the selection need to consider
   extensions and
   processing of tags private-use subtags when matching tags.  If extensions
   and private-use subtags are described included in Section 2.1.1 of that document.

   An application a matching or protocol MAY choose to limit the length filtering
   process that utilizes one of the
   language tags or ranges used schemes described in matching.  Any such limitation SHOULD
   be clearly documented, and such documentation this document,
   then the implementation SHOULD include canonicalize the
   disposition of any longer language tags or and/or
   ranges (for example, whether an
   error value is generated or before performing the matching.  Note that language tag or range is truncated).
   If truncation is permitted it MUST NOT permit a subtag
   processors that claim to be divided,
   since "well-formed" processors as defined in
   [RFC3066bis] generally fall into this changes the semantics category.

4.2.  Meaning of the subtag being matched Language Tags and can
   result in false positives or negatives.

   Applications or protocols that restrict storage SHOULD consider the
   impact Ranges

   Selecting content using language ranges requires some understanding
   by users of tag or range truncation on the resulting matches.  For
   example, removing the "*" from the end what they are selecting.  The meaning of an extended the various
   subtags in a language range are identical to their meaning in a
   language tag (see Section 2.2) can greatly modify 4.2 in [RFC3066bis]), with the set of returned matches.  A
   protocol addition
   that allows tags or ranges to be truncated at an arbitrary
   limit, without giving the wildcard "*" represents any indication matching sequence of what that limit is, has the
   potential values.

4.3.  Considerations for causing harm by changing the meaning of values in
   substantial ways.

   In practice, most tags do not require additional subtags or
   substantially more characters.  Additional subtags sometimes add
   useful distinguishing information, but extraneous Private Use Subtags

   Private-use subtags interfere
   with require private agreement between the meaning, understanding, and especially matching of language
   tags.  Since language tags or ranges MAY be truncated by an
   application or protocol parties
   that limits storage, when choosing language
   tags or ranges users and applications SHOULD avoid adding subtags intend to use or exchange language tags that add no distinguishing value.  In particular, users use them and
   implementations great
   caution SHOULD follow the 'Prefix' and 'Suppress-Script'
   fields be used in the registry (defined employing them in Section 3.6 of [RFC3066bis]):
   these fields provide guidance on when specific additional content or protocols
   intended for general use.  Private-use subtags
   SHOULD (and SHOULD NOT) be used.

   Implementations MUST support a limit are simply useless for
   information exchange without prior arrangement.

   The value and semantic meaning of at least 33 characters.  This
   limit includes at least one subtag private-use tags and of each non-extension, non-private
   use type.  When choosing a buffer limit, the subtags
   used within such a length of at least 42
   characters is strongly RECOMMENDED.

   The practical limit on language tag are not defined.  Matching private-
   use tags using language ranges or extended language ranges derived solely from registered
   values is 42 characters.  Implementations MUST be able can result
   in unpredictable content being returned.

4.4.  Length Considerations in Matching

   Language ranges are very similar to handle language tags
   and ranges in terms of this length.  Support for tags content
   and ranges usage.  The same types of at least
   62 characters in restrictions on length is RECOMMENDED.  Implementations MAY support
   longer values, including matching extensive sets of private-use or
   extension subtags.

   Applications or protocols which have that apply to truncate a tag MUST do so by
   progressively removing subtags along with their preceding "-" from
   the right side of the
   language tag until the tag is short enough for
   the given buffer.  If the resulting tag ends with a single-character
   subtag, that subtag and its preceding "-" MUST tags could also be removed.  For
   example:

   Tag apply to truncate: zh-Latn-CN-variant1-a-extend1-x-wadegile-private1
   1. zh-Latn-CN-variant1-a-extend1-x-wadegile
   2. zh-Latn-CN-variant1-a-extend1
   3. zh-Latn-CN-variant1
   4. zh-Latn-CN
   5. zh-Latn
   6. zh

   Figure 9: Example of Tag Truncation language ranges.  Implementation,
   protocol, and specificiation authors SHOULD apply the considerations
   in [RFC3066bis] Section 4.3 (Length Considerations) where appropriate
   to language ranges and language priority lists.

5.  IANA Considerations

   This document presents no new or existing considerations for IANA.

6.  Changes

   This is the first version of this document.

   The following changes were put into this document since draft-07:

      Added a mention of "*" to the Character Set Considerations section
      (D.Ewell)

7.  Security Considerations

   Language ranges used in content negotiation might be used to infer
   the nationality of the sender, and thus identify potential targets
   for surveillance.  In addition, unique or highly unusual language
   ranges or combinations of language ranges might be used to track a
   specific individual's activities.

   This is a special case of the general problem that anything you send
   is visible to the receiving party.  It is useful to be aware that
   such concerns can exist in some cases.

   The evaluation of the exact magnitude of the threat, and any possible
   countermeasures, is left to each application or protocol.

8.  Character Set Considerations

   Language tags permit only the characters A-Z, a-z, 0-9, and HYPHEN-
   MINUS (%x2D).  Language ranges also use the character ASTERISK
   (%x2A).  These characters are present in most character sets, so
   presentation or exchange of language tags or ranges should not be
   constrained by character set issues.

9.  References

9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2277]  Alvestrand, H., "IETF Policy on Character Sets and
              Languages", BCP 18, RFC 2277, January 1998.

   [RFC3066bis]
              Phillips, A., Ed. and M. Davis, Ed., "Tags for the
              Identification of Languages", October 2005, <http://
              www.ietf.org/internet-drafts/
              draft-ietf-ltru-registry-14.txt>.

   [RFC4234]  Crocker, D. and P. Overell, "Augmented BNF for Syntax
              Specifications: ABNF", RFC 4234, October 2005.

9.2.  Informative References

   [RFC1766]  Alvestrand, H., "Tags for the Identification of
              Languages", RFC 1766, March 1995.

   [RFC2616]  Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
              Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
              Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.

   [RFC2616errata]
              IETF, "HTTP/1.1 Specification Errata", 10 2004,
              <http://purl.org/NET/http-errata>.

   [RFC3066]  Alvestrand, H., "Tags for the Identification of
              Languages", BCP 47, RFC 3066, January 2001.

   [RFC3282]  Alvestrand, H., "Content Language Headers", RFC 3282,
              May 2002.

   [XML10]    Bray (et al), T., "Extensible Markup Language (XML) 1.0",
              02 2004.

Appendix A.  Acknowledgements

   Any list of contributors is bound to be incomplete; please regard the
   following as only a selection from the group of people who have
   contributed to make this document what it is today.

   The contributors to [RFC3066bis], [RFC3066] and [RFC1766], each of
   which is a precursor to this document, made enormous contributions
   directly or indirectly to this document and are generally responsible
   for the success of language tags.

   The following people (in alphabetical order by family name)
   contributed to this document:

   Harald Alvestrand, Jeremy Carroll, John Cowan, Martin Duerst, Frank
   Ellermann, Doug Ewell, Marion Gunn, Kent Karlsson, Ira McDonald, M.
   Patton, Randy Presuhn, Eric van der Poel, Markus Scherer, and many,
   many others.

   Very special thanks must go to Harald Tveit Alvestrand, who
   originated RFCs 1766 and 3066, and without whom this document would
   not have been possible.

   For this particular document, John Cowan originated the scheme
   described in Section 3.2.3. scoring
   scheme.  Mark Davis originated the scheme described in the Section 3.3.

Authors' Addresses

   Addison Phillips (editor)
   Yahoo! Inc

   Email: addison at inter dash locale dot com

   Mark Davis (editor)
   Google

   Email: mark dot davis at macchiato dot com

Intellectual Property Statement

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.

Disclaimer of Validity

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Copyright Statement

   Copyright (C) The Internet Society (2006).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.

Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.