draft-ietf-ltru-matching-11.txt   draft-ietf-ltru-matching-12.txt 
Network Working Group A. Phillips, Ed. Network Working Group A. Phillips, Ed.
Internet-Draft Yahoo! Inc Internet-Draft Yahoo! Inc.
Obsoletes: 3066 (if approved) M. Davis, Ed. Obsoletes: 3066 (if approved) M. Davis, Ed.
Expires: September 5, 2006 Google Expires: October 8, 2006 Google
March 4, 2006 April 6, 2006
Matching of Language Tags Matching of Language Tags
draft-ietf-ltru-matching-11 draft-ietf-ltru-matching-12
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at page 1, line 35 skipping to change at page 1, line 35
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 5, 2006. This Internet-Draft will expire on October 8, 2006.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2006). Copyright (C) The Internet Society (2006).
Abstract Abstract
This document describes different mechanisms for comparing and This document describes different mechanisms for comparing and
matching language tags. Possible algorithms for language negotiation matching language tags. Possible algorithms for language negotiation
or content selection, filtering, and lookup are described. This or content selection, filtering, and lookup are described. This
skipping to change at page 2, line 14 skipping to change at page 2, line 14
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. The Language Range . . . . . . . . . . . . . . . . . . . . . . 4 2. The Language Range . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Basic Language Range . . . . . . . . . . . . . . . . . . . 4 2.1. Basic Language Range . . . . . . . . . . . . . . . . . . . 4
2.2. Extended Language Range . . . . . . . . . . . . . . . . . 5 2.2. Extended Language Range . . . . . . . . . . . . . . . . . 5
2.3. The Language Priority List . . . . . . . . . . . . . . . . 5 2.3. The Language Priority List . . . . . . . . . . . . . . . . 5
3. Types of Matching . . . . . . . . . . . . . . . . . . . . . . 7 3. Types of Matching . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Choosing a Type of Matching . . . . . . . . . . . . . . . 7 3.1. Choosing a Type of Matching . . . . . . . . . . . . . . . 7
3.2. Filtering . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2. Implementation Considerations . . . . . . . . . . . . . . 8
3.2.1. Basic Filtering . . . . . . . . . . . . . . . . . . . 9 3.3. Filtering . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.2. Extended Filtering . . . . . . . . . . . . . . . . . . 10 3.3.1. Basic Filtering . . . . . . . . . . . . . . . . . . . 10
3.3. Lookup . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3.2. Extended Filtering . . . . . . . . . . . . . . . . . . 10
4. Other Considerations . . . . . . . . . . . . . . . . . . . . . 15 3.4. Lookup . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.1. Choosing Language Ranges . . . . . . . . . . . . . . . . . 15 3.4.1. Default Values . . . . . . . . . . . . . . . . . . . . 14
4.2. Meaning of Language Tags and Ranges . . . . . . . . . . . 16 4. Other Considerations . . . . . . . . . . . . . . . . . . . . . 16
4.3. Considerations for Private Use Subtags . . . . . . . . . . 16 4.1. Choosing Language Ranges . . . . . . . . . . . . . . . . . 16
4.4. Length Considerations for Language Ranges . . . . . . . . 17 4.2. Meaning of Language Tags and Ranges . . . . . . . . . . . 17
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 4.3. Considerations for Private Use Subtags . . . . . . . . . . 17
6. Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.4. Length Considerations for Language Ranges . . . . . . . . 18
7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19
8. Character Set Considerations . . . . . . . . . . . . . . . . . 21 6. Security Considerations . . . . . . . . . . . . . . . . . . . 20
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 7. Character Set Considerations . . . . . . . . . . . . . . . . . 21
9.1. Normative References . . . . . . . . . . . . . . . . . . . 22 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22
9.2. Informative References . . . . . . . . . . . . . . . . . . 22 8.1. Normative References . . . . . . . . . . . . . . . . . . . 22
8.2. Informative References . . . . . . . . . . . . . . . . . . 22
Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 23 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24
Intellectual Property and Copyright Statements . . . . . . . . . . 25 Intellectual Property and Copyright Statements . . . . . . . . . . 25
1. Introduction 1. Introduction
Human beings on our planet have, past and present, used a number of Human beings on our planet have, past and present, used a number of
languages. There are many reasons why one would want to identify the languages. There are many reasons why one would want to identify the
language used when presenting or requesting information or in some language used when presenting or requesting information or in some
specific set of information items or "content". specific set of information items.
One use for language identifiers, such as those defined in Applications, protocols, or specifications that use language
[RFC3066bis], is to select content by matching the associated identifiers, such as the language tags defined in [RFC3066bis],
language tags to a user's language preferences. sometimes need to match language tags to a user's language
preferences.
This document defines a syntax (called a language range (Section 2)) This document defines a syntax (called a language range (Section 2))
for specifying items in the user's list of language preferences for specifying items in the user's list of language preferences
(called a language priority list (Section 2.3)), as well as several (called a language priority list (Section 2.3)), as well as several
schemes for selecting or filtering sets of content by comparing the schemes for selecting or filtering sets of language tags by comparing
content's language tags to the user's preferences. Applications, the language tags to the user's preferences. Applications,
protocols, or specifications will have varying needs and requirements protocols, or specifications will have varying needs and requirements
that affect the choice of a suitable matching scheme. Depending on that affect the choice of a suitable matching scheme.
the choice of scheme, there are various options left to the
implementation. Protocols that implement a matching scheme either
need to specify each particular choice or indicate the options that
are left to the implementation to decide.
This document is divided into three main sections. One describes how This document describes: how to indicate a user's preferences using
to indicate a user's preferences using language ranges. Then a language ranges; three schemes for matching these ranges to a set of
section describes various schemes for matching these ranges to a set language tags; and the various practical considerations that apply to
of language tags. There is also a section that deals with various implementing and using these schemes.
practical considerations that apply to implementing and using these
schemes.
This document, in combination with [RFC3066bis] (Ed.: replace This document, in combination with [RFC3066bis] (Ed.: replace
"3066bis" globally in this document with the RFC number assigned to "3066bis" globally in this document with the RFC number assigned to
draft-ietf-ltru-registry-14), replaces [RFC3066], which replaced draft-ietf-ltru-registry-14), replaces [RFC3066], which replaced
[RFC1766]. [RFC1766].
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
2. The Language Range 2. The Language Range
Language Tags [RFC3066bis] are used to identify the language of some Language tags [RFC3066bis] are used to help identify languages,
information item or "content". Applications or protocols that use whether spoken, written, signed, or otherwise signaled, for the
language tags are often faced with the problem of identifying sets of purpose of communication. Applications, protocols, or specifications
content that share certain language attributes. For example, that use language tags are often faced with the problem of
HTTP/1.1 [RFC2616] describes one such mechanism in its discussion of identifying sets of content that share certain language attributes.
the Accept-Language header (Section 14.4), which is used when For example, HTTP/1.1 [RFC2616] describes one such mechanism in its
selecting content from servers based on the language of that content. discussion of the Accept-Language header (Section 14.4), which is
used when selecting content from servers based on the language of
that content.
When selecting content according to its language, it is useful to It is, thus, useful to have a mechanism for identifying sets of
have a mechanism for identifying sets of language tags that share language tags that share specific attributes. This allows users to
specific attributes. This allows users to select or filter content select or filter the language tags based on specific requirements.
based on specific requirements. Such an identifier is called a Such an identifier is called a "language range".
"language range".
There are different types of language range, whose specific There are different types of language range, whose specific
attributes vary according to their application. Language ranges are attributes vary according to their application. Language ranges are
similar to language tags: they consist of a sequence of subtags similar to language tags: they consist of a sequence of subtags
separated by hyphens. In a language range, each subtag MUST either separated by hyphens. In a language range, each subtag MUST either
be a sequence of ASCII alphanumeric characters or the single be a sequence of ASCII alphanumeric characters or the single
character '*' (%2A, ASTERISK). The character '*' is a "wildcard" character '*' (%2A, ASTERISK). The character '*' is a "wildcard"
that matches any sequence of subtags. The meaning and uses of that matches any sequence of subtags. The meaning and uses of
wildcards vary according to the type of language range. wildcards vary according to the type of language range.
Language tags and thus language ranges are to be treated as case- Language tags and thus language ranges are to be treated as case-
insensitive: there exist conventions for the capitalization of some insensitive: there exist conventions for the capitalization of some
of the subtags, but these MUST NOT be taken to carry meaning. of the subtags, but these MUST NOT be taken to carry meaning.
Matching of language tags to language ranges MUST be done in a case- Matching of language tags to language ranges MUST be done in a case-
insensitive manner. insensitive manner.
2.1. Basic Language Range 2.1. Basic Language Range
A "basic language range" describes a user's language preference as a A "basic language range" consists of a sequence of alphanumeric
specific, uninterrupted, sequence of subtags. Each range consists of subtags separated by hyphens. It is defined by the following ABNF
a sequence of alphanumeric subtags separated by hyphens. The basic [RFC4234]:
language range is defined by the following ABNF [RFC4234]:
language-range = (1*8ALPHA *("-" 1*8alphanum)) / "*" language-range = (1*8ALPHA *("-" 1*8alphanum)) / "*"
alphanum = ALPHA / DIGIT alphanum = ALPHA / DIGIT
Basic language ranges (originally described by HTTP/1.1 [RFC2616] and Basic language ranges (originally described by HTTP/1.1 [RFC2616] and
later [RFC3066]) have the same syntax as an [RFC3066] language tag or later [RFC3066]) have the same syntax as an [RFC3066] language tag or
are the single character "*". They differ from the language tags are the single character "*". They differ from the language tags
defined in [RFC3066bis] only in that there is no requirement that defined in [RFC3066bis] only in that there is no requirement that
they be "well-formed" or be validated against the IANA Language they be "well-formed" or be validated against the IANA Language
Subtag Registry (although such ill-formed ranges will probably not Subtag Registry. Such ill-formed ranges will probably not match
match anything). (Note that the ABNF [RFC4234] in [RFC2616] is anything. Note that the ABNF [RFC4234] in [RFC2616] is incorrect,
incorrect, since it disallows the use of digits anywhere in the since it disallows the use of digits anywhere in the 'language-range'
'language-range': this is mentioned in the errata) (see: [RFC2616errata]).
Use of a basic language range seems to imply that there is a semantic
relationship between language tags that share the same prefix. While
this is often the case, it is not always true and users should note
that the set of language tags that match a specific language range
may not represent mutually intelligible languages.
2.2. Extended Language Range 2.2. Extended Language Range
Occasionally users will wish to select a set of language tags based Occasionally users will wish to select a set of language tags based
on the presence of specific subtags. An "extended language range" on the presence of specific subtags. An "extended language range"
describes a user's language preference as an ordered sequence of describes a user's language preference as an ordered sequence of
subtags. For example, a user might wish to select all language tags subtags. For example, a user might wish to select all language tags
that contain the region subtag 'CH' (Switzerland). Extended language that contain the region subtag 'CH' (Switzerland). Extended language
ranges are useful in specifying a particular sequence of subtags that ranges are useful in specifying a particular sequence of subtags that
appear in the set of matching tags without having to specify all of appear in the set of matching tags without having to specify all of
skipping to change at page 5, line 34 skipping to change at page 5, line 28
An extended language range can be represented by the following ABNF: An extended language range can be represented by the following ABNF:
extended-language-range = (1*8ALPHA / "*") extended-language-range = (1*8ALPHA / "*")
*("-" (1*8alphanum / "*")) *("-" (1*8alphanum / "*"))
Figure 2: Extended Language Range Figure 2: Extended Language Range
The wildcard subtag '*' can occur in any position in the extended The wildcard subtag '*' can occur in any position in the extended
language range, where it matches any sequence of subtags that might language range, where it matches any sequence of subtags that might
occur in that position in a language tag. However, wildcards outside occur in that position in a language tag. However, wildcards outside
the first position in an extended language range are ignored by most the first position are ignored by Extended Filtering (see Section
matching schemes. Use of one or more wildcards SHOULD NOT be taken 3.2.2). The use or absence of one or more wildcards cannot be taken
to imply that a certain number of subtags will appear in the matching to imply that a certain number of subtags will appear in the matching
set of language tags. set of language tags.
Implementations that specify basic ranges MAY map extended language
ranges to basic language ranges: if the first subtag is a "*" then
the entire range is treated as "*", otherwise each wildcard subtag is
removed. For example, if the language range were "en-*-US", then the
range would be mapped to "en-US".
2.3. The Language Priority List 2.3. The Language Priority List
A user's language preferences will often need to specify more than A user's language preferences will often need to specify more than
one language range and thus users often need to specify a prioritized one language range and thus users often need to specify a prioritized
list of language ranges in order to best reflect their language list of language ranges in order to best reflect their language
preferences. This is especially true for speakers of minority preferences. This is especially true for speakers of minority
languages. A speaker of Breton in France, for example, may specify languages. A speaker of Breton in France, for example, may specify
"be" followed by "fr", meaning that if Breton is available, it is "br" followed by "fr", meaning that if Breton is available, it is
preferred, but otherwise French is the best alternative. It can get preferred, but otherwise French is the best alternative. It can get
more complex: a user may wish to fall back from Skolt Sami to more complex: a user may wish to fall back from Skolt Sami to
Northern Sami to Finnish. Northern Sami to Finnish.
A "language priority list" is a prioritized or weighted list of A "language priority list" is a prioritized or weighted list of
language ranges. One well known example of such a list is the language ranges. One well known example of such a list is the
"Accept-Language" header defined in RFC 2616 [RFC2616] (see Section "Accept-Language" header defined in RFC 2616 [RFC2616] (see Section
14.4) and RFC 3282 [RFC3282]. 14.4) and RFC 3282 [RFC3282].
The various matching operations described in this document include The various matching operations described in this document include
skipping to change at page 7, line 7 skipping to change at page 7, line 7
A simple list of ranges is considered to be in descending order of A simple list of ranges is considered to be in descending order of
priority. Other language priority lists provide "quality weights" priority. Other language priority lists provide "quality weights"
for the language ranges in order to specify the relative priority of for the language ranges in order to specify the relative priority of
the user's language preferences. An example of this would be the use the user's language preferences. An example of this would be the use
of "q" values in the syntax of the "Accept-Language" header (defined of "q" values in the syntax of the "Accept-Language" header (defined
in [RFC2616], Section 14.4, and [RFC3282]). in [RFC2616], Section 14.4, and [RFC3282]).
3. Types of Matching 3. Types of Matching
Matching language ranges to language tags can be done in a number of Matching language ranges to language tags can be done in many
different ways. This section describes several different matching different ways. This section describes three such matching schemes,
schemes, as well as the considerations for choosing between them. as well as the considerations for choosing between them. Protocols
Protocols and specifications SHOULD clearly indicate the particular and specifications requiring conformance to this specification MUST
mechanism used in selecting or matching language tags. clearly indicate the particular mechanism used in selecting or
matching language tags.
There are several types of matching scheme. This document presents
two types: those that produce zero or more information items (called
"filtering") and those that produce a single information item for a
given request (called "lookup").
Implementations or protocols MAY use different matching schemes from There are two types of matching scheme in this document. A matching
the ones described in this document, as long as those mechanisms are scheme that produces zero or more matching language tags is called
clearly specified. "filtering". A matching scheme that produces exactly one match for a
given request is called "lookup".
3.1. Choosing a Type of Matching 3.1. Choosing a Type of Matching
Applications, protocols, and specifications are faced with the Applications, protocols, and specifications are faced with the
decision of what type of matching to use. Sometimes, different decision of what type of matching to use. Sometimes, different
styles of matching are suited to different kinds of processing within styles of matching are suited to different kinds of processing within
a particular application or protocol. a particular application or protocol.
Language tag matching is a tool, and does not by itself specify a
complete procedure for the use of language tags. Such procedures are
intimately tied to the application protocol in which they occur.
When specifying a protocol operation using matching, the protocol
MUST specify:
o Which type(s) of language tag matching it uses
o Whether the operation returns a single result (lookup) or a
possibly empty set of results (filtering)
o For lookup, what the result is when no matching tag is found. For
instance, a protocol might define the result as failure of the
operation, an empty value, returning some protocol defined or
implementation defined default, or returning i-default [RFC2277].
This document describes three types of matching: This document describes three types of matching:
1. Basic Filtering (Section 3.2.1) matches a language priority list 1. Basic Filtering (Section 3.3.1) matches a language priority list
consisting of basic language ranges (Section 2.1) to sets of consisting of basic language ranges (Section 2.1) to sets of
language tags. language tags.
2. Extended Filtering (Section 3.2.2) matches a language priority 2. Extended Filtering (Section 3.3.2) matches a language priority
list consisting of extended language ranges (Section 2.2) to sets list consisting of extended language ranges (Section 2.2) to sets
of language tags. of language tags.
3. Lookup (Section 3.3) matches a language priority list consisting 3. Lookup (Section 3.4) matches a language priority list consisting
of basic language ranges to sets of language tags to find the one of basic language ranges to sets of language tags to find the one
_exact_ language tag that best matches the range. _exact_ language tag that best matches the range.
Filtering can be used to produce a set of results (such as a Filtering can be used to produce a set of results (such as a
collection of documents) by comparing the user's preferences to collection of documents) by comparing the user's preferences to a set
language tags associated with the set of content. For example, when of language tags. For example, when performing a search, one might
performing a search, one might use filtering to limit the results to use filtering to limit the results to items tagged as being in the
items tagged as being in the French language. Filtering can also be French language. Filtering can also be used when deciding whether to
used when deciding whether to perform a language-sensitive process on perform a language-sensitive process on some content. For example, a
some content. For example, a process might cause paragraphs whose process might cause paragraphs whose language tag matched the
language tag matched the language range "nl" to be displayed in language range "nl" to be displayed in italics within a document.
italics within a document.
Lookup produces the single result that best matches the user's Lookup produces the single result that best matches the user's
preferences, so it is useful in cases in which only a single item can preferences from the list of available tags, so it is useful in cases
be returned. For example, if a process were to insert a human in which a single item is required (and for which only a single item
can be returned). For example, if a process were to insert a human
readable error message into a protocol header, it might select the readable error message into a protocol header, it might select the
text based on the user's language priority list. Since the process text based on the user's language priority list. Since the process
can return only one item, it must choose a single item and it must can return only one item, it must choose a single item and it must
return some item, even if none of the content's language tags match return some item, even if none of the content's language tags match
the language priority list supplied by the user. the language priority list supplied by the user.
The types of matching in this document are designed so that 3.2. Implementation Considerations
implementations are not required to validate or understand any of the
semantics of the language tags or ranges or of the subtags in them.
None of them require access to the IANA Language Subtag Registry (see
Section 3 in [RFC3066bis]). This simplifies implementation of these
schemes. An implementation MAY choose to check if either the
language ranges or language tags being matched are "well-formed" or
"valid" (see [RFC3066bis], Section 2.2.9) and MAY choose not to
process invalid ranges.
Regardless of the matching scheme chosen, protocols and Language tag matching is a tool, and does not by itself specify a
implementations MAY canonicalize language tags and ranges by mapping complete procedure for the use of language tags. Such procedures are
grandfathered and obsolete tags or subtags into modern equivalents. intimately tied to the application protocol in which they occur.
If an implementation canonicalizes either ranges or tags, then the When specifying a protocol operation using matching, the protocol
implementation will require the IANA Language Subtag Registry MUST specify:
information for that purpose. Implementations MAY also use semantic
information external to the registry when matching tags. For
example, the primary language subtags 'nn' (Nynorsk Norwegian) and
'nb' (Bokmal Norwegian) might both be usefully matched to the more
general subtag 'no' (Norwegian). Or an implementation might infer
that content labeled "zh-Hans" (Chinese as written in the Simplified
script) is more likely to match the range "zh-CN" (Chinese as used in
China, where the Simplified script is predominant) than equivalent
content labeled "zh-TW" (Chinese as used in Taiwan, where the
Traditional script is predominant).
3.2. Filtering o Which type(s) of language tag matching it uses
o Whether the operation returns a single result (lookup) or a
possibly empty set of results (filtering)
o For lookup, what the default item is (or the sequence of
operations or configuration information used to determine the
default) when no matching tag is found. For instance, a protocol
might define the result as failure of the operation, an empty
value, returning some protocol defined or implementation defined
default, or returning i-default [RFC2277].
Applications, protocols, and specifications are not required to
validate or understand any of the semantics of the language tags or
ranges or of the subtags in them, nor do they require access to the
IANA Language Subtag Registry (see Section 3 in [RFC3066bis]). This
simplifies implementation.
However, designers of applications, protocols, or specifications are
encouraged to use the information from the IANA Language Subtag
Registry to support canonicalizing language tags and ranges in order
to map grandfathered and obsolete tags or subtags into modern
equivalents.
Applications, protocols, or specifications that canonicalize ranges
MUST either perform matching operations with both the canonical and
original (unmodified) form of the range or MUST also canonicalize
each tag for the purposes of comparison.
Note that canonicalizing language ranges makes certain operations
impossible. For example, an implementation that canonicalizes the
language range "art-lojban" to use the more modern "jbo" cannot be
used to select just the items with the older tag.
Applications, protocols, or specifications that use basic ranges
might sometimes receive extended language ranges instead. An
application, protocol, or specification MUST choose to: a) map
extended language ranges to basic ranges using the algorithm below,
b) reject any extended language ranges in the language priority list
that are not valid basic language ranges, or c) treat each extended
language range as if it were a basic language range, which will have
the same result as ignoring them, since these ranges will won't match
any valid language tags.
An extended language range is mapped to a basic language range as
follows: if the first subtag is a '*' then the entire range is
treated as "*", otherwise each wildcard subtag is removed. For
example, if the language range were "en-*-US", then the range would
be mapped to "en-US".
Applications, protocols, or specifications, in addressing their
particular requirements, can offer pre-processing or configuration
options. For example, an implementation could allow a user to
associate or map a particular language range to a different value.
Such a user might wish to associate the language range subtags 'nn'
(Nynorsk Norwegian) and 'nb' (Bokmal Norwegian) with the more general
subtag 'no' (Norwegian). Or perhaps the user could associate the
range "zh-Hans" (Chinese as written in the Simplified script) with
the language tag "zh-CN" (Chinese as used in China, where the
Simplified script is predominant) because content is available with
that tag. Documentation on how the ranges or tags are altered,
prioritized, or compared in the subsequent match in such an
implementation will assist users in making the best configuration
choices.
3.3. Filtering
Filtering is used to select the set of language tags that matches a Filtering is used to select the set of language tags that matches a
given language priority list and return the associated content. It given language priority list. It is called "filtering" because this
is called "filtering" because this set might contain no items at all set might contain no items at all or it might return an arbitrarily
or it might return an arbitrarily large number of matching items: as large number of matching items: as many items as match the language
many items as match the language priority list, thus "filtering out" priority list, thus "filtering out" the non-matching items.
the non-matching items.
In filtering, each language range represents the _least_ specific In filtering, each language range represents the _least_ specific
language tag (that is, the language tag with fewest number of language tag (that is, the language tag with fewest number of
subtags) which is an acceptable match. All of the language tags in subtags) which is an acceptable match. All of the language tags in
the matching set of tags will have an equal or greater number of the matching set of tags will have an equal or greater number of
subtags than the language range. Every non-wildcard subtag in the subtags than the language range. Every non-wildcard subtag in the
language range will appear in every one of the matching language language range will appear in every one of the matching language
tags. For example, if the language priority list consists of the tags. For example, if the language priority list consists of the
range "de-CH", one might see tags such as "de-CH-1996" but one will range "de-CH", one might see tags such as "de-CH-1996" but one will
never see a tag such as "de" (because the 'CH' subtag is missing). never see a tag such as "de" (because the 'CH' subtag is missing).
skipping to change at page 9, line 44 skipping to change at page 10, line 23
o Applying a style to sections of a document in a particular set of o Applying a style to sections of a document in a particular set of
languages. languages.
o Displaying the set of documents containing a particular set of o Displaying the set of documents containing a particular set of
keywords written in a specific set of languages. keywords written in a specific set of languages.
o Selecting all email items written in a specific set of languages. o Selecting all email items written in a specific set of languages.
o Selecting audio files spoken in a particular language. o Selecting audio files spoken in a particular language.
3.2.1. Basic Filtering Filtering seems to imply that there is a semantic relationship
between language tags that share the same prefix. While this is
often the case, it is not always true and users should note that the
set of language tags that match a specific language range do not
necessarily represent mutually intelligible languages.
When filtering using basic language ranges, each basic language range 3.3.1. Basic Filtering
in the language priority list is considered in turn, according to
priority. A particular language tag matches a language range if, in Basic filtering uses basic language ranges. Each basic language
a case-insensitive comparison, it exactly equals the tag, or if it range in the language priority list is considered in turn, according
to priority. A language range matches a particular language tag if,
in a case-insensitive comparison, it exactly equals the tag, or if it
exactly equals a prefix of the tag such that the first character exactly equals a prefix of the tag such that the first character
following the prefix is "-". For example, the language-range "de-de" following the prefix is "-". For example, the language-range "de-de"
matches the language tag "de-DE-1996", but not the language tags "de- matches the language tag "de-DE-1996", but not the language tags "de-
Deva" or "de-Latn-DE". Deva" or "de-Latn-DE".
The special range "*" in a language priority list matches any tag. A The special range "*" in a language priority list matches any tag. A
protocol which uses language ranges MAY specify additional rules protocol which uses language ranges MAY specify additional rules
about the semantics of "*"; for instance, HTTP/1.1 [RFC2616] about the semantics of "*"; for instance, HTTP/1.1 [RFC2616]
specifies that the range "*" matches only languages not matched by specifies that the range "*" matches only languages not matched by
any other range within an "Accept-Language" header. any other range within an "Accept-Language" header.
Basic filtering is identical to the type of matching described in Basic filtering is identical to the type of matching described in
[RFC3066], Section 2.5 (Language-range). [RFC3066], Section 2.5 (Language-range).
3.2.2. Extended Filtering 3.3.2. Extended Filtering
When filtering using extended language ranges, each extended language Extended filtering compares extended language ranges to language
range in the language priority list is considered in turn, according tags. Each extended language range in the language priority list is
to priority. A particular language range is compared to each considered in turn, according to priority. A language range matches
language tag using the following process: a particular language tag if their list of subtags match. To
determine a match:
Compare the first subtag in the extended language tag to the first 1. Split both the extended language range and the language tag being
subtag in the language tag in a case insensitive manner. If the compared into a list of subtags by dividing on the hyphen (%2D)
first subtag in the range is "*", it matches any value. Otherwise character. Two subtags match if either they are the same when
the two values must match or the overall match fails. compared case-insensitively or the language range's subtag is the
wildcard '*'.
Take each non-wildcard subtag in the language range and compare it in 2. Begin with the first subtag in each list. If the first subtag in
a case-insensitive manner to the next subtag in the language tag. If the range does not match the first subtag in the tag, the overall
the range's subtag exactly matches the tag's subtag, proceed to the match fails. Otherwise, move to the next subtag in both the
next non-wildcard subtag in the language range (and beginning with range and the tag.
the next subtag in the language tag) until the list of subtags in the
language range is exhausted or the match fails. If the tag's subtag 3. While there are more subtags left in the language range's list:
is a "singleton" (a single letter or digit, which, in this case,
includes the private-use subtag 'x') and the range's subtag does not A. If the subtag currently being examined in the range is the
match or if the language tag's list of subtags is exhausted, the wildcard ('*'), move to the next subtag in the range and
match fails. If the language range's list of subtags is exhausted, continue with the loop.
the match succeeds.
B. Else, if there are no more subtags in the language tag's
list, the match fails.
C. Else, if the current subtag in the range's list matches the
current subtag in the language tag's list, move to the next
subtag in both lists and continue with the loop.
D. Else, if the language tag's subtag is a "singleton" (a single
letter or digit, which includes the private-use subtag 'x')
the match fails.
E. Else, move to the next subtag in the language tag's list and
continue with the loop.
4. When the language range's list has no more subtags, the match
succeeds.
Subtags not specified, including those at the end of the language Subtags not specified, including those at the end of the language
range, are thus treated as if assigned the wildcard value "*". Much range, are thus treated as if assigned the wildcard value '*'. Much
like basic filtering, extended filtering selects content with like basic filtering, extended filtering selects content with
arbitrarily long tags that share the same initial subtags as the arbitrarily long tags that share the same initial subtags as the
language range. In addition extended filtering selects content with language range. In addition, extended filtering selects language
any intermediate subtags unspecified in the language range. For tags that contain any intermediate subtags not specified in the
example, the extended language range "de-*-DE" matches all of the language range. For example, the extended language range "de-*-DE"
following tags: (or its synonym "de-DE") matches all of the following tags:
de-DE de-DE
de-Latn-DE de-Latn-DE
de-Latf-DE de-Latf-DE
de-de de-de
de-DE-x-goethe de-DE-x-goethe
de-Latn-DE-1996 de-Latn-DE-1996
skipping to change at page 11, line 14 skipping to change at page 12, line 17
de-Latn-DE de-Latn-DE
de-Latf-DE de-Latf-DE
de-de de-de
de-DE-x-goethe de-DE-x-goethe
de-Latn-DE-1996 de-Latn-DE-1996
de-Deva-DE
The same range does not match any of the following tags for the The same range does not match any of the following tags for the
reasons shown: reasons shown:
de (missing 'DE') de (missing 'DE')
de-x-DE (singleton 'x' occurs before 'DE') de-x-DE (singleton 'x' occurs before 'DE')
de-Deva ('Deva' not equal to 'DE') de-Deva ('Deva' not equal to 'DE')
Note: The structure of language tags defined by [RFC3066bis] defines Note: [RFC3066bis] defines each type of subtag (language, script,
each type of subtag (language, script, region, and so forth) region, and so forth) according to position, size, and content. This
according to position, size, and content. This means that subtags in means that subtags in a language range can only match specific types
a language range can only match specific types of subtags in a of subtags in a language tag. For example, a subtag such as 'Latn'
language tag. For example, a subtag such as 'Latn' is always a is always a script subtag (unless it follows a singleton) while a
script subtag (unless it follows a singleton) while a subtag such as subtag such as 'nedis' can only match the equivalent variant subtag.
'nedis' can only match the equivalent variant subtag. One such difference is that two-letter subtags in initial position
have a different type (language) than two-letter subtags in later
positions (region). This is the reason why a wildcard in the
extended language range is significant in the first position and
subsequently ignored.
3.3. Lookup 3.4. Lookup
Lookup is used to select the single language tag that best matches Lookup is used to select the single language tag that best matches
the language priority list for a given request and return the the language priority list for a given request. When performing
associated content. When performing lookup, each language range in lookup, each language range in the language priority list is
the language priority list is considered in turn, according to considered in turn, according to priority. By contrast with
priority. By contrast with filtering, each language range represents filtering, each language range represents the _most_ specific tag
the _most_ specific tag which is an acceptable match. The first which is an acceptable match. The first matching tag found,
content found with a matching tag, according to the user's priority, according to the user's priority, is considered the closest match and
is considered the closest match and is the content returned. For is the item returned. For example, if the language range is "de-ch",
example, if the language range is "de-ch", a lookup operation might a lookup operation can produce content with the tags "de" or "de-CH"
produce content with the tags "de" or "de-CH" but never one with the but never content with the tag "de-CH-1996". If no language tag
tag "de-CH-1996". Usually if no content matches the request, the matches the request, the "default" value is returned.
"default" content is returned.
For example, if an application inserts some dynamic content into a For example, if an application inserts some dynamic content into a
document, returning an empty string if there is no exact match is not document, returning an empty string if there is no exact match is not
an option. Instead, the application "falls back" until it finds a an option. Instead, the application "falls back" until it finds a
matching language tag associated with a suitable piece of content to matching language tag associated with a suitable piece of content to
insert. Examples of lookup might include: insert. Examples of lookup might include:
o Selection of a template containing the text for an automated email o Selection of a template containing the text for an automated email
response. response.
o Selection of a item containing some text for inclusion in a o Selection of a item containing some text for inclusion in a
particular Web page. particular Web page.
o Selection of a string of text for inclusion in an error log. o Selection of a string of text for inclusion in an error log.
o Selection of an audio file to play as a prompt in a phone system. o Selection of an audio file to play as a prompt in a phone system.
In the lookup scheme, the language range is progressively truncated In the lookup scheme, the language range is progressively truncated
from the end until a matching piece of content is located. Single from the end until a matching language tag is located. Single letter
letter or digit subtags (including both the letter 'x' which or digit subtags (including both the letter 'x' which introduces
introduces private-use sequences, and the subtags that introduce private-use sequences, and the subtags that introduce extensions) are
extensions) are removed at the same time as their closest trailing removed at the same time as their closest trailing subtag. For
subtag. For example, starting with the range "zh-Hant-CN-x-private1- example, starting with the range "zh-Hant-CN-x-private1-private2",
private2", the lookup progressively searches for content as shown the lookup progressively searches for content as shown below:
below:
Range to match: zh-Hant-CN-x-private1-private2 Range to match: zh-Hant-CN-x-private1-private2
1. zh-Hant-CN-x-private1-private2 1. zh-Hant-CN-x-private1-private2
2. zh-Hant-CN-x-private1 2. zh-Hant-CN-x-private1
3. zh-Hant-CN 3. zh-Hant-CN
4. zh-Hant 4. zh-Hant
5. zh 5. zh
6. (default content) 6. (default)
Figure 3: Example of a Lookup Fallback Pattern Figure 3: Example of a Lookup Fallback Pattern
This allows some flexibility in finding a match. For example, lookup This allows some flexibility in finding a match. For example, lookup
provides better results for cases in which content is not available provides better results for cases in which content is not available
that exactly matches the user request than if the default language that exactly matches the user request than if the default language
for the system or content were returned immediately. Not every for the system or content were returned immediately. Language
specific level of tag granularity is usually available or language material is sometimes sparsely populated, so an item might not be
content may be sparsely populated. "Falling back" through the subtag available at every level of tag granularity. "Falling back" through
sequence provides more opportunity to find a match between available the subtag sequence provides more opportunity to find a match between
language tags and the user's request. available language tags and the user's request.
The default behavior when no tag matches the language priority list Extensions and unrecognized private-use subtags might be unrelated to
is implementation defined. An implementation might, for example, a particular application of lookup. Since these subtags come at the
return content: end of the subtag sequence, they are removed first during the
fallback process and usually pose no barrier to interoperability.
However, an implementation MAY remove these from ranges prior to
performing the lookup (provided the implementation also removes them
from the tags being compared). Such modification is internal to the
implementation and applications, protocols, or specifications SHOULD
NOT remove or modify subtags in content that they return or forward,
because this removes information that might be used elsewhere.
o with no language tag The special language range "*" matches any language tag. In the
lookup scheme, this range does not convey enough information by
itself to determine which language tag is most appropriate, since it
matches everything. If the language range "*" is followed by other
language ranges, it is skipped. If the language range "*" is the
only one in the language priority list or if no other language range
follows, the default value is computed and returned.
o of a non-linguistic nature, such as an image or sound In some cases, the language priority list might contain one or more
extended language ranges (as, for example, when the same language
priority list is used as input for both lookup and filtering
operations). Wildcard values in an extended language range normally
match any value that can occur in that position in a language tag.
Since only one item can be returned for any given lookup request,
wildcards in a language range have to be processed in a consistent
manner or the same request will produce widely varying results.
Applications, protocols, or specifications that accept extended
language ranges MUST define which item is returned when more than one
item matches the extended language range.
o with an empty language tag value, in cases where the protocol For example, an implementation could return the matching tag that is
permits the empty value (see, for example, "xml:lang" in [XML10], first in ASCII-order. If the language range were "*-CH" and the set
which indicates that the element contains non-linguistic content) of tags included "de-CH", "fr-CH", and "it-CH", then the tag "de-CH"
would be returned. Another possibility would be for an
implementation to map the extended language ranges to basic ranges.
o in a particular language designated for the bit of content being 3.4.1. Default Values
selected
o labelled with the tag "i-default" (see [RFC2277]) Each application, protocol, or specification MUST define the
defaulting behavior when no tag matches the language priority list.
What this action consists of strongly depends on how lookup is being
applied. Some examples of defaulting behavior might include:
o return an item with no language tag or an item of a non-linguistic
nature, such as an image or sound
o return a null string as the language tag value, in cases where the
protocol permits the empty value (see, for example, "xml:lang" in
[XML10])
o return a particular language tag designated for the operation
o return the language tag "i-default" (see: [RFC2277])
o return an error condition or error message
o return a list of available languages for the user to select from
When performing lookup using a language priority list, the When performing lookup using a language priority list, the
progressive search MUST process each language range in the list progressive search MUST process each language range in the list
before finding the default content or empty tag. before seeking or calculating the default.
One common way for an application or implementation to provide for a The default value MAY be calculated and might include additional
default is to allow a specific language range to be set as the searching or matching. Applications, protocols, or specifications
default for a specific type of request. This language range is then can specify different ways in which users can specify or override the
defaults.
One common way to provide for a default is to allow a specific
language range to be set as the default for a specific type of
request. If this approach is chosen, this language range MUST be
treated as if it were appended to the end of the language priority treated as if it were appended to the end of the language priority
list as a whole, rather than after each item in the language priority list as a whole, rather than after each item in the language priority
list. list. The application, protocol, or specification MUST also define
the defaulting behavior if that search fails to find a matching tag
or item.
For example, if a particular user's language priority list were For example, if a particular user's language priority list were
"fr-FR, zh-Hant" and the program doing the matching had a default "fr-FR, zh-Hant" and the program doing the matching had a default
language range of "ja-JP", the program would search for content as language range of "ja-JP", the program would search as follows:
follows:
1. fr-FR 1. fr-FR
2. fr 2. fr
3. zh-Hant // next language 3. zh-Hant // next language
4. zh 4. zh
5. (search for the default content) 5. ja-JP // now searching for the default content
a. ja-JP 6. ja
b. ja 7. (implementation defined default)
c. (implementation defined default)
Figure 4: Lookup Using a Language Priority List Figure 4: Lookup Using a Language Priority List
Implementations SHOULD ignore extensions and unrecognized private-use
subtags when performing lookup, since these subtags are usually
orthogonal to the user's request.
The special language range "*" matches any language tag. In the
lookup scheme, this range does not convey enough information by
itself to determine which content is most appropriate, since it
matches everything. If the language range "*" is followed by other
language ranges, it SHOULD be skipped. If the language range "*" is
the only one in the language priority list or if no other language
range follows, the default content SHOULD be returned.
In some cases, the language priority list might contain one or more
extended language ranges (as, for example, when the same language
priority list is used as input for both lookup and filtering
operations). Wildcard values in an extended language range normally
match any value that occurs in that position in a language tag.
Since only one item can be returned for any given lookup request,
wildcards in a language range have to be processed in a consistent
manner or the same request will produce widely varying results.
Implementations that accept extended language ranges MUST define
which content is returned when more than one item matches the
extended language range.
For example, an implementation could return the matching tag that is
first in ASCII-order. If the language range were "*-CH" and the set
of tags included "de-CH", "fr-CH", and "it-CH", then the tag "de-CH"
would be returned. Another possibility would be for an
implementation to map the extended language ranges to basic ranges.
4. Other Considerations 4. Other Considerations
When working with language ranges and matching schemes, there are When working with language ranges and matching schemes, there are
some additional points that may influence the choice of either. some additional points that may influence the choice of either.
4.1. Choosing Language Ranges 4.1. Choosing Language Ranges
Users indicate their language preferences via the choice of a Users indicate their language preferences via the choice of a
language range or the list of language ranges in a language priority language range or the list of language ranges in a language priority
list. The type of matching affects what the best choice is for a list. The type of matching affects what the best choice is for a
user. user.
Most matching schemes make no attempt to process the semantic meaning Most matching schemes make no attempt to process the semantic meaning
of the subtags and the language range is compared, in a case- of the subtags. The language range is compared, in a case-
insensitive manner, to each language tag being matched, using basic insensitive manner, to each language tag being matched, using basic
string processing. Users SHOULD select language ranges that are string processing. Users SHOULD select language ranges that are
well-formed, valid language tags according to [RFC3066bis] well-formed, valid language tags according to [RFC3066bis]
(substituting wildcards as appropriate in extended language ranges). (substituting wildcards as appropriate in extended language ranges).
Users SHOULD replace tags or subtags which have been deprecated with Applications are encouraged to canonicalize language tags and ranges
the Preferred-Value from the IANA Language Subtag Registry. If the by using the Preferred-Value from the IANA Language Subtag Registry
user is working with content that might use the older form, the user for tags or subtags which have been deprecated. If the user is
might include both the new and old forms in a language priority list. working with content that might use the older form, the user might
For example, the tag "art-lojban" is deprecated. The subtag 'jbo' is want to include both the new and old forms in a language priority
supposed to be used instead, so the user might use it to form the list. For example, the tag "art-lojban" is deprecated. The subtag
language range. Or the user might include both in a language 'jbo' is supposed to be used instead, so the user might use it to
priority list: "jbo, art-lojban". form the language range. Or the user might include both in a
language priority list: "jbo, art-lojban".
Users SHOULD avoid subtags that add no distinguishing value to a Users SHOULD avoid subtags that add no distinguishing value to a
language range. When filtering, the fewer the number of subtags that language range. When filtering, the fewer the number of subtags that
appear in the language range, the more content the range will appear in the language range, the more content the range will
probably match, while in lookup unnecessary subtags might cause probably match, while in lookup unnecessary subtags might cause
"better", more-specific content to be skipped in favor of less "better", more-specific content to be skipped in favor of less
specific content. For example, the range "de-Latn-DE" would return specific content. For example, the range "de-Latn-DE" would return
content tagged "de" instead of content tagged "de-DE", even though content tagged "de" instead of content tagged "de-DE", even though
the latter is probably a better match. the latter is probably a better match.
Whether a subtag adds distinguishing value can depend on the context
of the request. For example, a user who reads both Simplified and
Traditional Chinese, but who prefers Simplified, might use the range
"zh" for filtering (matching all items that user can read) but "zh-
Hans" for lookup (making sure that user gets the preferred form if
it's available, but the fallback to "zh" will still work). On the
other hand, content in this case should be labeled as "zh-Hans" (or
"zh-Hant" if that applies) for filtering, but for lookup, if there is
either "zh-Hans" content or "zh-Hant" content, then one of them (the
one considered 'default') should also be available under a simple
"zh". Note that the user can create a language priority list "zh-
Hans, zh" that delivers the best possible results for both schemes.
If the user cannot be sure which scheme is being used (or if more
than one might be applied to a given request), the user SHOULD
specify the most specific (largest number of subtags) range first and
then supply shorter prefixes later in the list to ensure that
filtering returns a complete set of tags.
Many languages are written predominantly in a single script. This is Many languages are written predominantly in a single script. This is
usually recorded in the Suppress-Script field in that language usually recorded in the Suppress-Script field in that language
subtag's registry entry. For these languages, script subtags SHOULD subtag's registry entry. For these languages, script subtags SHOULD
NOT be used to form a language range. Thus the language range "en- NOT be used to form a language range. Thus the language range "en-
Latn" is inappropriate in most cases (because the vast majority of Latn" is inappropriate in most cases (because the vast majority of
English documents are written in the Latin script and thus the 'en' English documents are written in the Latin script and thus the 'en'
language subtag has a Suppress-Script field for 'Latn' in the language subtag has a Suppress-Script field for 'Latn' in the
registry). registry).
When working with tags and ranges, note that extensions and most When working with tags and ranges, note that extensions and most
private-use subtags are orthogonal to language tag matching, in that private-use subtags are orthogonal to language tag matching, in that
they specify additional attributes of the text not related to the they specify additional attributes of the text not related to the
goals of most matching schemes. Users SHOULD avoid using these goals of most matching schemes. Users SHOULD avoid using these
subtags in language ranges, since they interfere with the selection subtags in language ranges, since they interfere with the selection
of available content. When used in language tags (as opposed to of available content. When used in language tags (as opposed to
ranges), these subtags normally do not interfere with filtering ranges), these subtags normally do not interfere with filtering
(Section 3), since they appear at the end of the tag and will match (Section 3), since they appear at the end of the tag and will match
all prefixes. Lookup (Section 3.3) implementations often ignore all prefixes. Lookup (Section 3.4) implementations are advised to
unrecognized private-use and extension subtags when performing ignore unrecognized private-use and extension subtags when performing
language tag fallback. language tag fallback.
Applications, specifications, or protocols that choose not to
interpret one or more private-use or extension subtags SHOULD NOT
remove or modify these extensions in content that they are
processing. When a language tag instance is to be used in a
specific, known protocol, and is not being passed through to other
protocols, language tags MAY be altered to remove subtags and
extensions that are not supported by that protocol. Such alterations
SHOULD be avoided, if possible, since they remove information that
might be relevant elsewhere that would make use of that information.
Some applications of language tags might want or need to consider
extensions and private-use subtags when matching tags. If extensions
and private-use subtags are included in a matching process that
utilizes one of the schemes described in this document, then the
implementation SHOULD canonicalize the language tags and/or ranges
before performing the matching. Note that language tag processors
that claim to be "well-formed" processors as defined in [RFC3066bis]
generally fall into this category.
4.2. Meaning of Language Tags and Ranges 4.2. Meaning of Language Tags and Ranges
Selecting content using language ranges requires some understanding Selecting language tags using language ranges requires some
by users of what they are selecting. The meaning of the various understanding by users of what they are selecting. The meaning of
subtags in a language range are identical to their meaning in a the various subtags in a language range are identical to their
language tag (see Section 4.2 in [RFC3066bis]), with the addition meaning in a language tag (see Section 4.2 in [RFC3066bis]), with the
that the wildcard "*" represents any matching sequence of values. addition that the wildcard "*" represents any matching sequence of
values.
4.3. Considerations for Private Use Subtags 4.3. Considerations for Private Use Subtags
Private-use subtags require private agreement between the parties Private-use subtags require private agreement between the parties
that intend to use or exchange language tags that use them and great that intend to use or exchange language tags that use them. They
caution SHOULD be used in employing them in content or protocols SHOULD NOT be used in content or protocols intended for general use.
intended for general use. Private-use subtags are simply useless for Private-use subtags are simply useless for information exchange
information exchange without prior arrangement. without prior arrangement.
The value and semantic meaning of private-use tags and of the subtags The value and semantic meaning of private-use tags and of the subtags
used within such a language tag are not defined. Matching private- used within such a language tag are not defined. Matching private-
use tags using language ranges or extended language ranges can result use tags using language ranges or extended language ranges can result
in unpredictable content being returned. in unpredictable content being returned.
4.4. Length Considerations for Language Ranges 4.4. Length Considerations for Language Ranges
Language ranges are very similar to language tags in terms of content Language ranges are very similar to language tags in terms of content
and usage. The same types of restrictions on length that apply to and usage. The same types of restrictions on length that apply to
language tags can also apply to language ranges. See [RFC3066bis] language tags can also apply to language ranges. See [RFC3066bis]
Section 4.3 (Length Considerations). Section 4.3 (Length Considerations).
5. IANA Considerations 5. IANA Considerations
This document presents no new or existing considerations for IANA. This document presents no new or existing considerations for IANA.
6. Changes 6. Security Considerations
This is the first version of this document.
7. Security Considerations
Language ranges used in content negotiation might be used to infer Language ranges used in content negotiation might be used to infer
the nationality of the sender, and thus identify potential targets the nationality of the sender, and thus identify potential targets
for surveillance. In addition, unique or highly unusual language for surveillance. In addition, unique or highly unusual language
ranges or combinations of language ranges might be used to track a ranges or combinations of language ranges might be used to track a
specific individual's activities. specific individual's activities.
This is a special case of the general problem that anything you send This is a special case of the general problem that anything you send
is visible to the receiving party. It is useful to be aware that is visible to the receiving party. It is useful to be aware that
such concerns can exist in some cases. such concerns can exist in some cases.
The evaluation of the exact magnitude of the threat, and any possible The evaluation of the exact magnitude of the threat, and any possible
countermeasures, is left to each application or protocol. countermeasures, is left to each application or protocol.
8. Character Set Considerations 7. Character Set Considerations
Language tags permit only the characters A-Z, a-z, 0-9, and HYPHEN- Language tags permit only the characters A-Z, a-z, 0-9, and HYPHEN-
MINUS (%x2D). Language ranges also use the character ASTERISK MINUS (%x2D). Language ranges also use the character ASTERISK
(%x2A). These characters are present in most character sets, so (%x2A). These characters are present in most character sets, so
presentation or exchange of language tags or ranges should not be presentation or exchange of language tags or ranges should not be
constrained by character set issues. constrained by character set issues.
9. References 8. References
9.1. Normative References 8.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2277] Alvestrand, H., "IETF Policy on Character Sets and [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and
Languages", BCP 18, RFC 2277, January 1998. Languages", BCP 18, RFC 2277, January 1998.
[RFC3066bis] [RFC3066bis]
Phillips, A., Ed. and M. Davis, Ed., "Tags for the Phillips, A., Ed. and M. Davis, Ed., "Tags for the
Identification of Languages", October 2005, <http:// Identification of Languages", October 2005, <http://
www.ietf.org/internet-drafts/ www.ietf.org/internet-drafts/
draft-ietf-ltru-registry-14.txt>. draft-ietf-ltru-registry-14.txt>.
[RFC4234] Crocker, D. and P. Overell, "Augmented BNF for Syntax [RFC4234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", RFC 4234, October 2005. Specifications: ABNF", RFC 4234, October 2005.
9.2. Informative References 8.2. Informative References
[RFC1766] Alvestrand, H., "Tags for the Identification of [RFC1766] Alvestrand, H., "Tags for the Identification of
Languages", RFC 1766, March 1995. Languages", RFC 1766, March 1995.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC2616errata] [RFC2616errata]
IETF, "HTTP/1.1 Specification Errata", 10 2004, IETF, "HTTP/1.1 Specification Errata", October 2004,
<http://purl.org/NET/http-errata>. <http://purl.org/NET/http-errata>.
[RFC3066] Alvestrand, H., "Tags for the Identification of [RFC3066] Alvestrand, H., "Tags for the Identification of
Languages", BCP 47, RFC 3066, January 2001. Languages", BCP 47, RFC 3066, January 2001.
[RFC3282] Alvestrand, H., "Content Language Headers", RFC 3282, [RFC3282] Alvestrand, H., "Content Language Headers", RFC 3282,
May 2002. May 2002.
[XML10] Bray (et al), T., "Extensible Markup Language (XML) 1.0", [XML10] Bray (et al), T., "Extensible Markup Language (XML) 1.0",
02 2004. February 2004.
Appendix A. Acknowledgements Appendix A. Acknowledgements
Any list of contributors is bound to be incomplete; please regard the Any list of contributors is bound to be incomplete; please regard the
following as only a selection from the group of people who have following as only a selection from the group of people who have
contributed to make this document what it is today. contributed to make this document what it is today.
The contributors to [RFC3066bis], [RFC3066] and [RFC1766], each of The contributors to [RFC3066bis], [RFC3066] and [RFC1766], each of
which is a precursor to this document, made enormous contributions which is a precursor to this document, made enormous contributions
directly or indirectly to this document and are generally responsible directly or indirectly to this document and are generally responsible
for the success of language tags. for the success of language tags.
The following people (in alphabetical order by family name) The following people (in alphabetical order by family name)
contributed to this document: contributed to this document:
Harald Alvestrand, Jeremy Carroll, John Cowan, Martin Duerst, Frank Harald Alvestrand, Stephane Bortzmeyer, Jeremy Carroll, John Cowan,
Ellermann, Doug Ewell, Marion Gunn, Kent Karlsson, Ira McDonald, M. Martin Duerst, Frank Ellermann, Doug Ewell, Debbie Garside, Marion
Patton, Randy Presuhn, Eric van der Poel, Markus Scherer, and many, Gunn, Kent Karlsson, Ira McDonald, M. Patton, Randy Presuhn, Eric van
many others. der Poel, Markus Scherer, and many, many others.
Very special thanks must go to Harald Tveit Alvestrand, who Very special thanks must go to Harald Tveit Alvestrand, who
originated RFCs 1766 and 3066, and without whom this document would originated RFCs 1766 and 3066, and without whom this document would
not have been possible. not have been possible.
Authors' Addresses Authors' Addresses
Addison Phillips (editor) Addison Phillips (editor)
Yahoo! Inc Yahoo! Inc.
Email: addison at inter dash locale dot com Email: addison@inter-locale.com
Mark Davis (editor) Mark Davis (editor)
Google Google
Email: mark dot davis at macchiato dot com Email: mark.davis@macchiato.com
Intellectual Property Statement Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be on the procedures with respect to rights in RFC documents can be
 End of changes. 75 change blocks. 
316 lines changed or deleted 370 lines changed or added

This html diff was produced by rfcdiff 1.29, available from http://www.levkowetz.com/ietf/tools/rfcdiff/