- ietf-charsets - lists.iana.org

Re: [IANA #1365934] Re: Error in bibxlm2 entry for reference.ANSI.X3-4.1986.xml
by Martin J. Dürst Sept. 11, 2025

Sept. 11, 2025

Hello Amanda, Sorry for the long delay of my answer, and for the length of my answer. [Copying ietf-charsets(a)iana.org, where people interested in charsets (if there's anybody left) might be subscribed.] On 2025-07-04 09:52, Amanda Baber via RT wrote: > Hi Martin, > > It was brought to our attention some time ago that the reference to ANSI_X3.4-1968 in the note at the top of the Character Sets registry may be incorrect, and that it should probably point to "INCITS.4-1986," or "INCITS.4-1986, originally published as ANSI X3.4-1986" instead. > > https://www.iana.org/assignments/character-sets [For the record, this gets redirected to https://www.iana.org/assignments/character-sets/character-sets.xhtml, so strictly speaking, the discussion below is about the later.] > Should IANA update that note? If so, which reference should we use? TL;DR: It doesn't matter at all. Main proposal: Because it's irrelevant, eliminate the reference to ANSI_X3.4-1968 in the note at the top, so that we don't need to discuss what reference would be best. Comments are of course welcome. Details: The text in question, with a bit of context, is: ==== Note These are the official names for character sets that may be used in the Internet and may be referred to in Internet documentation. These names are expressed in ANSI_X3.4-1968 which is commonly called US-ASCII or simply ASCII. The character set most commonly use in the Internet and used especially in protocol standards is US-ASCII, this is strongly encouraged. The use of the name US-ASCII is also encouraged. ==== [The note itself continues, but this is the relevant part.] The first question is what "These names are expressed in" refers to. It may refer to the entries in the page itself. In that case, the statement is correct, because the page is served with "Content-Type: text/html; charset=UTF-8". US-ASCII is by definition also UTF-8, and all the 'names' are in the US-ASCII subset of UTF-8. "These names are expressed in" may also refer to how these names are to be used in protocols and data. In that case, the "strongly encouraged" applies, but the statement in its general form would be wrong because charset labels may easily be expressed in EBCDIC on an IBM mainframe, or in UTF-16 or UTF-32 inside programs e.g. in Java or Python. The second question is why ANSI_X3.4-1968/US-ASCII/INCITS.4-1986 is explicitly mentioned with respect to names. All the names use only characters from the invariant subset of ISO 646 (see e.g. https://en.wikipedia.org/wiki/ISO/IEC_646#Code_page_layout) I checked this by searching for each of these characters in the page. The only characters that appear in the IANA page and are redefined in national variants of ISO 646 are '[' and ']', and these only appear around references, not in any of the charset names themselves. My guess is that ANSI_X3.4-1968 was mentioned to stress the importance of US-ASCII, because when the registry was set up, there was quite a chance that people would send emails without charset information that were in any national variant of ISO 646. But this is just a guess, and if correct, it would only be a secondary reason. The next question is whether the mention of ANSI_X3.4-1968, or anything in place of it, is necessary or helpful. My answer would be that it may have had some value, but is no longer necessary. I don't think that any of the many other IANA registries registering 'names' (as opposed to numbers) says anything about how these names are (to be) encoded. Why should this page say anything more? The national variants of ISO 646 are mostly gone, US-ASCII is the base of text encoding on the Internet and the Web not only because the various standards say so, but also and much more importantly, because it's what's used in practice. So *my proposal* is to change the first paragraph of the note to: ==== These are the official names for character sets that may be used in the Internet and may be referred to in Internet documentation. The character set most commonly used in the Internet and used especially in protocol standards is US-ASCII, this is strongly encouraged. The use of the name US-ASCII is also encouraged. ==== [also fixing one instance of 'use' -> 'used'] If the above proposal is not accepted, the next question is what are the alternatives, and in particular whether "INCITS.4-1986" is an improvement over "ANSI_X3.4-1968". First, let's ignore the year. Researching the relationship between ANSI and INCITS, one immediately finds things such as "The InterNational Committee for Information Technology Standards (INCITS), is an ANSI-accredited standards development organization composed of Information technology developers. It was formerly known as the X3 and NCITS.". So INCITS.4-19XX and ANSI_X3.4-19XX would simply be equivalent. A relationship such as the above between ANSI and INCITS is very frequent in the realm of standards development and approval. Such relationships also come in various shapes and forms, from co-operation to rubber stamping, from republication to simple reference. The main reason for this is that the technical expertise is not in the same place as the official (or semi-official) authority. Just to mention an example, what's generally known as JavaScript is standardized by ECMA as ECMA-262, and is also ISO/IEC 16262 (later ISO/IEC 22275). Next, let's look at the years. It's currently "ANSI_X3.4-1968", whereas "INCITS.4-1986" is proposed. The question is what difference the year would make. I don't have actual copies of these standards. But it's easy to find the relevant parts on the Web. For the 1968 version, RFC 20, in its original paper version as scanned at https://www.rfc-editor.org/rfc/rfc20.pdf contains the relevant parts of what it calls USAS X3,4-1968. https://archive.org/details/enf-ascii-1968-1970/ seems to be a full copy, including several appendices and additional material. For the 1986 version, we can see it in the 2007 proposed review at https://www.unicode.org/L2/L2006/06388-review-incits4.pdf. While I'm sure that a careful comparison between these two versions will bring up many differences of interest to a historian, there are no essential differences in the allocation of characters to code points. The most noticeable difference is the use of a broken line glyph for the "VERTICAL LINE" character in 1968, which became a single line in 1986. But this is irrelevant for us, because none of the charset names contains a '|', and because glyph variants wouldn't be relevant anyway. So the conclusion for whether we use the 1968 version or the 1986 version is that it doesn't matter (assuming we use any version at all). Just in case we use any version, the question would be what exact labels to use. For the 1986 version, according to https://www.unicode.org/L2/L2006/06388-review-incits4.pdf, page 2, would be "ANSI INCITS 4-1986 (R2002)" or "ANSI X3.4-1986 (R1997)", and extrapolating back from the later we should end up at simply "ANSI X3.4-1986". For the 1968 version, it is entitled "USA Standard Code for Information Interchange" and labeled USAS X3.4-1968, with "Business Equipment Manufacturers Association" as a 'Sponsor' and approved by the "United States of America Standards Institute". As an aside, it's interesting to note that the initials in "USA Standard Code for Information Interchange" are the same as in US-ASCII, although my guess is that this is a somewhat indirect coincidence (first "USA Standard Code for Information Interchange" being shortened to "American Standard Code for Information Interchange" (ASCII), and later "US" being added again). In general, one has to note that the labels under discussion ("ANSI_X3.4-1968" and "INCITS.4-1986") seem both to be out of sync when it comes to correspondence between organization name and year of issue. In 1968, the name ANSI didn't exist (see e.g. the timeline and history by 10s of years at https://www.ansi.org/about/history, in particular the entries for 1960'S and 1970'S), and in 1986, the name INCITS didn't exist (see https://www.incits.org/about/history) Looking at https://en.wikipedia.org/wiki/ASCII#Revisions, if we want to refer the actual versions in 1968 or 1986, it would be "USAS X3.4-1968" or "ANSI X3.4-1986". If we want to use INCITS, we should make sure that we refer to one of the revisions/reviews of the 1986 explicitly, e.g. "ANSI INCITS 4-1986 (R2002)" (if we want ANSI to be included) or "INCITS 4-1986 (R2022)" (if we do not want ANSI to be included, and/or want to make sure we refer to the latest version). > The suggested reference format comes from a discussion with the RFC Editor, which resulted in the text of the first normative reference entry here: > > https://datatracker.ietf.org/doc/html/draft-ietf-emailcore-rfc5322bis-12#na… Checked. That one seems to prefer the newest version as explained above. Personally, I'd probably cited RFC 20 (STD 80) instead. Also, I'd probably preferred ANSI X3.4-1986 in order to show that this is something extremely stable, rather than to run the danger that some readers start to wonder what the changes between 1986 and 2022 may be. > If this is OK, we'll ask an AD to sign off. If it's okay with you, I'd like you to ask the AD to sign off on my proposal, i.e. to *remove* the mention of the standard from the Note at the start of the page. But please let's wait for a week or so to see if there are comment from others. Regards, Martin. > > thanks, > > Amanda Baber > IANA Operations Manager

2 1

Mailman Upgrade Announcement
by David Dong Aug. 15, 2024

Aug. 15, 2024

Hello, Over the coming days the infrastructure behind this mailing list will be upgraded. IANA hosts a number of operational mailing lists using ICANN's mailing list infrastructure. This infrastructure is currently undergoing a major upgrade, including to the mailman software that operates this list. Users should not observe a difference except for modernization of the mailing list administration website. Mailing list archives will move to a new archiver, but old archives will be retained to preserve references to archived messages. The archives may be delayed by several hours during the migration. We will send an announcement following the migration that will also serve as a confirmation of the operational status of the mailing list and will provide a link to the new archives. If you have problems caused by the migration, please contact the IANA team at iana(a)iana.org so we may help remedy them. Best regards, David Dong IANA Services Sr. Specialist

1 1

Fix links for International Register of Coded Charater Sets to be Used with Escape Sequences
by Martin J. Dürst June 7, 2024

June 7, 2024

Dear IANA, I'm writing to you as the designated Expert for Character Sets (https://www.iana.org/assignments/character-sets/character-sets.xhtml) I found out that the links (https://www.itscj-ipsj.jp/custom_contents/cms/linkfile/ISO-IR.pdf) for the text [ISO-IR: International Register of Escape Sequences] appearing close to 80 times on the above page are to a site that no longer exists. The reason seems to be that at some point in time, www.itscj-ipsj.jp was moved to itscj.ipsj.or.jp, and the site was reorganized. I found the relevant document at https://itscj.ipsj.or.jp/english/vbcqpr00000004qn-att/ISO-IR.pdf so this is the link you should use. I can also contact ITSCJ to tell them that we depend on their registry, and they shouldn't move it around without notifying us. But my guess is that they will agree but later forget. This happens on average in organizations around the world. Regards, Martin.

3 3

Re: [ietf-charsets] [art] Fwd: [IANA #1297322] IANA characters-sets US-ASCII entry incorrect
by Martin J. Dürst Dec. 26, 2023

Dec. 26, 2023

Hello Steffen, others, On 2023-12-21 04:54, Steffen Nurpmeso wrote: > [i take iana completely off] For the record, the ietf-charsets(a)iana.org mailing list setting has been changed so that mail from non-subscribers gets moderated, not just discarded. IANA will see how long they can do that (depends on the amount of spam they will get). > Martin J. Dürst wrote in > <6a5eaaab-0c5e-4751-b007-090ce9b35a27(a)it.aoyama.ac.jp>: > |To everybody interested in the recent discussion on the character set > |registry, in particular the (absence of the) entry "ASCII". > | > |Many thanks to IANA, and in particular Sabrina Tanamal, for digging up > |the relevant correspondence from 10 and 20 years ago. > > Is it normal that bugtracker participants are not included in > follow-up communication? I have seen nothing. Hm. Only > wondering. But i agree, of course. That mail from Sabrina was only sent to me. That's why I forwarded it. > |Please accept my apology for not remembering this correspondence and > |therewith seriously confusing the discussion. > | > |My summary based on this new information is as follows: > | > |- Ned Freed sent a request to IANA in February 2003 concerning the > | entry for "US-ASCII" and related aliases in the Character Set > | registry, requesting (among else) the removal of the alias "ASCII". > | The request for this removal was based on the fact that RFC 2046 says > | 'The character set name "ASCII" is reserved and must not be used for > | any purpose.'. > | > |- When IANA was moving their registries from .txt to .xml, this request > | was rediscovered and acted upon. Both Ned and me agreed with the > | removal of "ASCII". We decided that there was no need to inform > | the ietf-charset mailing list, which in hindsight was probably a > | mistake (not the least because it would have had the potential to > | shorten the current discussion by quite a bit). > | > |Given the fact that RFC 2046 clearly says that 'The character set name > |"ASCII" is reserved and must not be used for any purpose.', I think that > |the only choice is to leave the registry as it is. > | > |<charset reviewer hat on> > |I'm of course ready to reevaluate this and adding this label back in if > |anybody is able to come up with really strong and convincing arguments > |to do so. > |<charset reviewer hat off> > > So i would ask to please do this. To put the conclusion first, I'm sorry, but you haven't convinced me. As John Klensin puts it in a very recent mail, you would have to create an internet-draft and gain IETF consensus on it to update RFC 2046. > The term "ASCII" is in use for longer than my lifespan, and > i cannot understand why a paragraph in some MIME-related RFC > overrules a generic, omnipresent standard that was established in > 1968. [Copying your last sentence here, because this simplifies my answer.] > But again, if it would be only about MIME, i would be fine, and > end just with this rhyme. My understanding is that this is indeed only or mostly about MIME. I included 'mostly', because the registry also covers MIB, which is certainly not MIME, and there may be other IETF or external standards referring to it. This is different from other standards groups or software (OSes, libraries, languages, and up). As an example, you mention the character encoding conversion library iconv below. One version on one of my systems (iconv (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35), with `iconv -l`, produces a long list of overall 1179 labels for encodings. That includes a lot of (what is easy to guess are) aliases such as 8859_1, ISO-8859-1*, ISO8859-1, ISO88591, ISO_8859-1*, ISO-8859-1:1987*. Only the ones with a * (plus some others that I'm sure iconv also listed but didn't have the time to check) are in the IANA registry. Overall, as mentioned above, my iconv version listed 1179 labels, and the IANA registry lists 887 unique names/preferred MIME names/aliases. There is quite some overlap, but there are 626 labels listed by iconv that are not in the IANA registry, and there are 334 aliases in the IANA registry that are not listed as labels by iconv. I haven't done an analysis of how many actual encodings these numbers cover. I'd have to dive into the iconv source code, and I'd rather avoid doing this because I think it doesn't really contribute much to this discussion. > I want to point to standards like POSIX from the OpenGroup, and to > other billions of distributed packages, including manuals, which > refer to the name ASCII as a shorter synomym (and not more) to the > standard ANSI_X3.4-1968, and its further line that goes to > ANSI_X3.4-1986, ISO_646.irv:1991, ISO646-US etc. That's just fine. The IETF cannot and doesn't dictate how other standards organizations or software producers write their documents. > It is something different if the _IANA character set database_ > manages _(preferred) MIME alias names_ in conjunction with > character sets, and it is absolutely fine if some MIME RFC gives > or takes on specific aliases. But ASCII is plain forever meaning > what it means, since 1968. Apparently not. John Klensin gave a very recent example. At least in the 90ies, "ASCII" must have been used in many ways, or we wouldn't have the very specific sentence in RFC 2046. I can easily imagine people having said "this file is ASCII, not EBCDIC", and the reason such a phrase is rare today is only due to the fact that EBCDIC is rare today. > Maybe i understand the IANA character set database wrong? > I would understand this decision of yours in respect to the > "preferred MIME name". > > But Let me for example point to the really great ones from Bell > Labs, who gained the U.S. "National Medal of Technology" not for > nitpicking or overengineering, and for example, five days before > i was born, on 1972-06-20, during Research V1 development, created > the file u9.s with content as > > + bisb 200,r1 / if entry is less than 0 add 128 to ASCII > + / code for char to be output > > + bit $1,r1 / is bit 0 of ASCII char = 1 (char = lf) The IETF isn't Bell Labs. And in the 1972 timeframe in the US, character encodings for e.g. European languages were not really a concern at all. > For UNIX v3 we already have a manual page ascii(7): > > CommitDate: 1973-02-13 15:37:00 -0500 > > Research V3 development > Work on file man/man7/ascii.7 > > Co-Authored-By: Dennis Ritchie <dmr(a)research.uucp> > Synthesized-from: v3 > --- > man/man7/ascii.7 | 37 +++++++++++++++++++++++++++++++++++++ > 1 file changed, 37 insertions(+) > > diff --git a/man/man7/ascii.7 b/man/man7/ascii.7 > new file mode 100644 > index 0000000000..ee8839580d > --- /dev/null > +++ b/man/man7/ascii.7 > @@ -0,0 +1,37 @@ > +.pa 1 > +.he 'ASCII (VII)'6/12/72'ASCII (VII)' > +.ti 0 > +NAME ascii -- map of ASCII character set > +.sp > +.ti 0 > +SYNOPSIS c__ /_____________ > +.sp > +.ti 0 > +DESCRIPTION a____ > +is a map of the ASCII character set, to be printed as needed. > +It contains: > +.in -16 > +.nf > + > +|000 nul|001 soh|002 stx|003 etx|004 eot|005 enq|006 ack|007 bel| > +|010 bs |011 ht |012 nl |013 vt |014 np |015 cr |016 so |017 si | > +|020 dle|021 dc1|022 dc2|023 dc3|024 dc4|025 nak|026 syn|027 etb| > +|030 can|031 em |032 sub|033 esc|034 fs |035 gs |036 rs |037 us | > +|040 sp |041 ! |042 " |043 # |044 $ |045 % |046 & |047 ' | > +|050 ( |051 ) |052 * |053 + |054 , |055 - |056 . |057 / | > +|060 0 |061 1 |062 2 |063 3 |064 4 |065 5 |066 6 |067 7 | > +|070 8 |071 9 |072 : |073 ; |074 < |075 = |076 > |077 ? | > +|100 @ |101 A |102 B |103 C |104 D |105 E |106 F |107 G | > +|110 H |111 I |112 J |113 K |114 L |115 M |116 N |117 O | > +|120 P |121 Q |122 R |123 S |124 T |125 U |126 V |127 W | > +|130 X |131 Y |132 Z |133 [ |134 \\ |135 ] |136 ^ |137 _ | > +|140 ` |141 a |142 b |143 c |144 d |145 e |146 f |147 g | > +|150 h |151 i |152 j |153 k |154 l |155 m |156 n |157 o | > +|160 p |161 q |162 r |163 s |164 t |165 u |166 v |167 w | > +|170 x |171 y |172 z |173 { |174 | |175 } |176 ~ |177 del| > + > +.fi > +.in +16 > +.sp > +.ti 0 > +FILES found in /usr/pub > > This is after Vincent Cerf's RFC 20 from 1969-10-16. > But shows the deep penetration of the standard and the name. In 1973, yes. Later, than name penetrated even deeper, into areas where it was very strictly speaking not appropriate. And that led to the sentence in RFC 2046. > So letting aside that "US-ASCII" appears twice in the current DB > varient, as a name and an alias, which is outstanding in this DB, > and surely an editor error. There are quite a few entries where a name appears as an alias. If it doesn't help, it definitely doesn't hurt either. > The name ASCII is well-known and cannot be detoriated by some > nitpicking in a MIME related RFC that only a rather small set of > people in some dark corner of the internet know about. > > |For data labeled with charset=ASCII, the correct interpretation is to > |ignore the charset parameter because of an undefined parameter value. > > See -- in my eyes this has no relation whatsoever with application > code in the wild, as at least one all UNIX systems like Linux etc > which use the iconv(3) series of functions, as well as any > scripting language which uses that under the hood, you call > iconv_open(charset-1, charset-2), and iconv does everything for > you. This is an unfortunate side-effect of the closed-box iconv > interface, which is capable of everything under the hood, but does > not expose this to users. (Ie name normalization.) > > Anyhow iconv(3) will, in reality, treat this ASCII as US-ASCII, > unless it uses automatic synchronization with the IANA DB. > It seems the most widely used implementations do not do that, > luckily. And should not. > > |The implementation would then fall back to the default, which in case of > |email and text/plain is "US-ASCII". The overall result is the same as an > |"ASCII" alias in the registry. > > I would *instead* think that most software implements some kind of > name normalization, and silently treat ASCII as the MIME name > US-ASCII. Please note that software is free to implement stuff the way they want, as long as the results are the same. And they are the same here. > ..so, for example number one terminal mutt(1) does exactly that, > it funnily misses the notorious :1986 variant that the MIME RFC > points to so deliberately.. but it knows 1968. > Black panther fist here. > > I am stunnded to detect that number two, alpine, University of > Washington (in the past, but now "a fellow in private") does it > differently, and only maps from ASCII to US-ASCII for outcome of > the function nl_langinfo(). It seems to have been rewritten for > a very late version (the last maybe even before becoming > a private project) to be UTF-8 centric, says the changelog. > > But again, if it would be only about MIME, i would be fine, and > end just with this rhyme. Well, it's indeed (mostly) only about MIME. Happy holidays to everybody, Martin. > --steffen

2 1

Re: [ietf-charsets] [art] [IANA #1297322] IANA characters-sets US-ASCII entry incorrect
by Martin J. Dürst Dec. 26, 2023

Dec. 26, 2023

Hello Carsten, others, Just tying a lose end below. On 2023-12-20 20:50, Carsten Bormann wrote: > On 2023-12-20, at 08:00, Martin J. Dürst <duerst(a)it.aoyama.ac.jp> wrote: >> >> The request for this removal was based on the fact that RFC 2046 says >> 'The character set name "ASCII" is reserved and must not be used for >> any purpose.'. > > Makes sense to me — this reflects the state of the world from 27 years ago. > > I don’t think there is a need to change this reserved status now. > (Assuming that even if HTML5 does not have a default charset, any locale-derived charset will be ASCII-compatible, so the right thing will be done.) Well, the latest version of HTML these days is just called HTML. W3C documents point to https://html.spec.whatwg.org/multipage/. That document, at https://html.spec.whatwg.org/multipage/iana.html, says for the 'charset' parameter: "The parameter's value must be an ASCII case-insensitive match for the string "utf-8"." And of course, UTF-8 is ASCII-compatible. Regards, Martin. > Grüße, Carsten >

1 0

Fwd: [IANA #1297322] IANA characters-sets US-ASCII entry incorrect
by Martin J. Dürst Dec. 20, 2023

Dec. 20, 2023

To everybody interested in the recent discussion on the character set registry, in particular the (absence of the) entry "ASCII". Many thanks to IANA, and in particular Sabrina Tanamal, for digging up the relevant correspondence from 10 and 20 years ago. Please accept my apology for not remembering this correspondence and therewith seriously confusing the discussion. My summary based on this new information is as follows: - Ned Freed sent a request to IANA in February 2003 concerning the entry for "US-ASCII" and related aliases in the Character Set registry, requesting (among else) the removal of the alias "ASCII". The request for this removal was based on the fact that RFC 2046 says 'The character set name "ASCII" is reserved and must not be used for any purpose.'. - When IANA was moving their registries from .txt to .xml, this request was rediscovered and acted upon. Both Ned and me agreed with the removal of "ASCII". We decided that there was no need to inform the ietf-charset mailing list, which in hindsight was probably a mistake (not the least because it would have had the potential to shorten the current discussion by quite a bit). Given the fact that RFC 2046 clearly says that 'The character set name "ASCII" is reserved and must not be used for any purpose.', I think that the only choice is to leave the registry as it is. <charset reviewer hat on> I'm of course ready to reevaluate this and adding this label back in if anybody is able to come up with really strong and convincing arguments to do so. <charset reviewer hat off> For data labeled with charset=ASCII, the correct interpretation is to ignore the charset parameter because of an undefined parameter value. The implementation would then fall back to the default, which in case of email and text/plain is "US-ASCII". The overall result is the same as an "ASCII" alias in the registry. Regards, Martin. -------- Forwarded Message -------- Subject: [IANA #1297322] IANA characters-sets US-ASCII entry incorrect Date: Tue, 19 Dec 2023 01:30:10 +0000 From: Sabrina Tanamal via RT <iana-issues-comment(a)iana.org> Reply-To: iana-issues-comment(a)iana.org CC: duerst(a)it.aoyama.ac.jp Hi Martin (trimming the list), It looks like this change was completed in 2013 (reported by Ned and approved by you). Please see the thread below. Let me know if you need us to forward anything to the list or if any changes are required. Thanks, Sabrina ===== Fri Jan 04 08:10:25 2013 Martin Duerst <duerst(a)it.aoyama.ac.jp> - Correspondence added CC: ned.freed(a)mrochek.com Subject: Re: [IANA #111894] Possible update to character-sets (#2) Date: Fri, 04 Jan 2013 16:48:30 +0900 To: iana-matrix(a)iana.org From: "Martin J. Dürst" <duerst(a)it.aoyama.ac.jp> Hello Amanda, On 2013/01/04 10:37, Amanda Baber via RT wrote: Hide quoted text > Hi Martin, > > Are you OK with going ahead with this? Yes, please go ahead with this. Ned is correct on each and every point. Sorry for the delay in answering this. Regards, Martin. > thanks, > Amanda > > On Thu Dec 20 23:11:34 2012, ned.freed(a)mrochek.com wrote: >>> Hi, >> >>> I haven't seen anything about this proposal on the charset list: >> >> I don't see any need to post this, but since I'm the source of the >> change >> it should be Martin's call. I think he's OK with going ahead, but he >> should >> probably confirm. >> >> >> Ned >> >>>>>> I therefore suggest that this entry [ANSI_X3.4-1968] be changed >> to read: >>>>>> >>>>>> Name: US-ASCII (preferred MIME name) [RFC2046] >>>>>> MIBenum: 3 >>>>>> Source: ANSI X3.4-1986 >>>>>> Alias: iso-ir-6 >>>>>> Alias: ANSI_X3.4-1968 >>>>>> Alias: ANSI_X3.4-1986 >>>>>> Alias: ISO_646.irv:1991 >>>>>> Alias: ISO646-US >>>>>> Alias: us >>>>>> Alias: IBM367 >>>>>> Alias: cp367 >>>>>> Alias: csASCII >> >>> Since you both approved it, as Martin noted, should we go ahead and >> make this change? Or would one of you prefer to post it to the list >> first? We'd prefer to leave it up to you. >> >>> thanks, >>> Amanda >> >>> On Fri Sep 28 06:42:32 2012, duerst(a)it.aoyama.ac.jp wrote: >>>> Hello Amanda, >>>> >>>> I have looked at the proposed change, and I think it makes *a lot* >> of >>>> sense. I think we have two choices: >>>> >>>> a) Just go ahead and fix it. We have both reviewers agreeing with >> it. >>>> b) Just to be sure, send a mail to the charset mailing list saying >> we >>>> plan to do this, so that anybody who may have a complaint (I don't >>>> expect anybody, but anyway). Then after a few weeks, go ahead and >> do it. >>>> >>>> I'm okay with either. >>>> >>>> Regards, Martin. >>>> On 2012/09/28 7:23, Amanda Baber via RT wrote: > Martin and Ned, > > We have this one last character-sets email from several years ago that > needs to be addressed. Ned, I know you submitted this in the first > place, but can you verify that this still needs to be done/won't break > anything? > > all apologies, and thanks, > Amanda > >> -------- Original Message -------- >> Subject: Fix for one particularly serious charset registry problem >> Date: Sun, 09 Feb 2003 15:51:23 -0800 (PST) >> From: ned.freed(a)mrochek.com >> To: iana(a)iana.org >> CC: ned.freed(a)mrochek.com, paf(a)cisco.com, harald(a)alvestrand.no >> >> >> The first entry in the charset registry reads as follows: >> >> Name: ANSI_X3.4-1968 [RFC1345,KXS2] >> MIBenum: 3 >> Source: ECMA registry >> Alias: iso-ir-6 >> Alias: ANSI_X3.4-1986 >> Alias: ISO_646.irv:1991 >> Alias: ASCII >> Alias: ISO646-US >> Alias: US-ASCII (preferred MIME name) >> Alias: us >> Alias: IBM367 >> Alias: cp367 >> Alias: csASCII >> >> There are, unfortunately, many problems with this entry: >> >> (1) The primary name of the charset is US-ASCII, not ANSI_X3.4-1968. > It has >> always been this way as far back as RFC 1341. And this actually > matters, >> since the primary charset name is the one that's supposed to be used >> in encoded words, and the restrictions on encoded words don't > allow use >> of ANSI_X3.4-1968! >> >> (2) The source of the registration should be the ANSI standards document, >> not some ECMA registry. >> >> (3) The alias "ASCII" is specifically prohibited by RFC 2046 section > 4.1.2. >> >> (4) The defining document for this charset is RFC 2046, not RFC 1345. >> >> I therefore suggest that this entry be changed to read: >> >> Name: US-ASCII (preferred MIME name) [RFC2046] >> MIBenum: 3 >> Source: ANSI X3.4-1986 >> Alias: iso-ir-6 >> Alias: ANSI_X3.4-1968 >> Alias: ANSI_X3.4-1986 >> Alias: ISO_646.irv:1991 >> Alias: ISO646-US >> Alias: us >> Alias: IBM367 >> Alias: cp367 >> Alias: csASCII >> >> There is no formal procedure for fixing errors in the charset registry. >> However, I believe that identification of an actual problem along with >> signoff of the charset reviewer should be sufficient to make the change. >> >> Ned On Mon Dec 18 21:59:24 2023, steffen(a)sdaoden.eu wrote: > Martin J. Dürst wrote in > <6fc74600-cff6-4751-9efa-1200a9ec5aa5(a)it.aoyama.ac.jp>: > |Hello Stephen, > > Actually Miss Jaeger named me Steve in English, because another > one got Stephen earlier. > > |On 2023-12-16 04:06, Steffen Nurpmeso wrote: > |> To add that for backward compatibility the plain ASCII alias > |> cannot go away, > | > |I seem to remember too that ASCII was listed as an alias, and have > |confirmed this with > |https://web.archive.org/web/20051229042158/http://www.iana.org/assignmen\ > |ts/character-sets > > My local copy is from 2011. > > |Stephen, maybe you can do a bisection to find out where this alias > |disappeared. > > I have only a local copy from 2011. I have no access to an IANA > revision control system, shall such a thing exist. Sorry. > > |<charset reviewer hat on> > |I don't remember ever having dealt with a request to remove this ALIAS, > |and I strongly doubt that Ned ever did that. > |</charset reviewer hat on> > > Well i could imagine it was broken when taking over the beautiful > text-only version to that really, really machine readable XML > variant that is now used as the base. > I hope the conversion was mechanical and the bugs are little > jokes, otherwise the entire IANA data collection possibly needs > an audit :-) > > |> i do have emails with charset=ascii in my archive, > | > |It would be interesting to know how many (i.e. what percentage of > |overall mails, and what percentage in comparison to those labeled > |US-ASCII), and how old they are. > > The latter is very large, the former is very low. Two, to be > exact. But they do exist (and i only have a very sparse private > email archive). > > --steffen > | > |Der Kragenbaer, The moon bear, > |der holt sich munter he cheerfully and one by one > |einen nach dem anderen runter wa.ks himself off > |(By Robert Gernhardt) > | > | Only in December: lightful Dubai COP28 Narendra Modi quote: > | A small part of humanity has ruthlessly exploited nature. > | But the entire humanity is bearing the cost of it, > | especially the inhabitants of the Global South. > | The selfishness of a few will lead the world into darkness, > | not just for themselves but for the entire world. > | [Christians might think of Revelation 11:18 > | The nations were angry, and your wrath has come[.] > | [.]for destroying those who destroy the earth. > | But i find the above more kind, and much friendlier] >

1 0

Re: [ietf-charsets] [art] US-ASCII and its various names
by Martin J. Dürst Dec. 19, 2023

Dec. 19, 2023

Hello Carsten, others, (taking iana(a)iana.org out) On 2023-12-19 16:54, Carsten Bormann wrote: > Hi John, > > On 2023-12-18, at 19:07, John C Klensin <john-ietf(a)jck.com> wrote: >> >> Carsten, >> >> (Top post... and I'm out of time for this) >> (1) My primary objective was to try to explain how we got here, >> in part because my involvement with the relevant decisions goes >> back further than Martin's tenure as charset reviewer and Ned is >> not here to defend himself. > > Much appreciated! > I was not aware that there actually was a discussion that rejected “ASCII” as a label for ASCII. Although I haven't been part of it, I'm quite sure there was a discussion about avoiding "ASCII" as the recommended label; "US-ASCII" always was and is clearly labeled as preferred for MIME. On the other hand, I'm quite sure the discussion did NOT reject "ASCII" as a label for ASCII. Otherwise, it would not have been listed as an alias in the registry until 2013. The first instance I have found in the wayback machine is from December 1998, at http://web.archive.org/web/19981202100649/http://www.isi.edu/in-notes/iana/…. > My argument is that it is detrimental to call ASCII something else than ASCII in conversation, because there are no other ASCIIs (except for the historical ASCII 63 etc.). Imagine always saying “XML with angle brackets” when we mean XML... That sounds fine, but communication is always context-dependent. > What labels we should use for interoperability is a separate matter. Yes indeed. > I’m also intrigued by Martin’s research about how we arrived at the current state of the registry… It was Steffen who found the difference. I just expanded on his "local copy" with something more generally accessible (the Web Archive). > If there really was a process error, that would sway me a bit to actually aligning the registry with the (feeble) reality of charset=ascii being in use. I don't think there was a process error (which would to me mean that a process was not followed, or followed the wrong way). There may have been a processing error (to the extent that the conversion from the old text file in or around 2013 was done e.g. by a script) and/or a human clerical error (to the extent that the conversion was done by a human). > (But then errors that have been in place for a decade develop their own normative reality.) My current understanding of reality would be that within a margin of error, the label "ASCII" was used about as rarely during the period it was listed in the registry (i.e. ~2013) as during the period it wasn't listed (2013~2023). If there's not much reality, it's difficult to speak of normative reality :-). Regards, Martin.

1 0

Re: [ietf-charsets] [art] US-ASCII and its various names
by Martin J. Dürst Dec. 19, 2023

Dec. 19, 2023

Hello John, others, [Removing iana(a)iana.org, because this doesn't contain any requests for them.] On 2023-12-18 12:57, John C Klensin wrote: > > > --On Monday, 18 December, 2023 09:21 +0900 "Martin J. Dürst" > <duerst(a)it.aoyama.ac.jp> wrote: > >> Hello Stephen, >> >> On 2023-12-16 04:06, Steffen Nurpmeso wrote: >> >>> To add that for backward compatibility the plain ASCII alias >>> cannot go away, >> >> I seem to remember too that ASCII was listed as an alias, and >> have confirmed this with >> https://web.archive.org/web/20051229042158/http://www.iana.org >> /assignments/character-sets > > That article appears to be to be discussing MIBs, not charset > parameters for what are now called Media Types, particularly > text and its subtypes. Sorry, wrong. Yes, it *also* talks about MIBs, but that's just because the MIB stuff is integrated in the registry. As for the many contributions of historical background, that's very much appreciated. I definitely haven't been around charsets when RFC 20 was created, although my experience with charsets goes back quite a bit longer than my role as expert reviewer. My summary of the history as relevant for the problem at hand (probably a clerical miss) is as follows: - Around 1969 (RFC 20, "ASCII format for Network Interchange", Oct 16), the issue was more about ASCII vs. EBCDIC (IBM) vs. encodings from other vendors that nobody remembers these days. The ARPANET was just being started (The first ARPANET communication is dated 22:30 hours on October 29, 1969, California time). This was a purely US-only undertaking, and nobody in that undertaking was worrying too much (if at all) about encodings for languages other than English. [RFC 20 is STD 80 now, but that designation must have happened in or close to 2014, because STD 79/RFC 7296 dates from Oct. 2014.] - For quite some time before and around 2000 (RFC 2978, "IANA Charset Registration Procedures"; RFC 2278, same name, Jan 1998 (*)), the name "ASCII" was in various contexts used with somewhat different, meanings, to the extent that people involved found it very prudent to strongly insist on using the label "US-ASCII" for "the real thing", while still listing the label "ASCII" as an alias. - Somewhere around 2013, the "ASCII" alias got dropped from the registry, probably as a result of a clerical error (and not related to the fact that "ASCII" was essentially deprecated for a long time although 'deprecated' isn't used in the registry). - In this day and age, most of the 'ASCII-like' character encodings are virtually out of use. "ASCII" as a label is also used extremely rarely in contexts where the IANA charset registry is relevant (email and the Web definitely count, OSes and programming languages don't). UTF-8 is recommended and used widely, and elegantly subsumes (US-)ASCII. >> Stephen, maybe you can do a bisection to find out where this >> alias disappeared. > > Oh, at the risk of repeating parts of the note I sent some days > ago, let me tell you what I remember before I co-wrote the > document that created the charset review role: (*) Can you tell us about that document? RFC 2048 (MIME Registration Procedures, which you co-wrote) explicitly says "Registration of character sets for use in MIME is covered elsewhere and is no longer addressed by this document." I didn't find anything about registration in the RFC 2045~49 series nor in RFC 1522 or 1590, but maybe I didn't look hard enough. > While it seems odd to not have it among the collection of > synonyms, is isn't there (and was probably dropped from various > earlier specs) because of the potential for ambiguity. > > If you are going to add it to the registry, that should be done, > not as a synonym for "US-ASCII", but as a separate items with an > explanation of the ambiguity problem. It was there as a synonym, and I'm not going to ask IANA to add it separately. >> <charset reviewer hat on> >> I don't remember ever having dealt with a request to remove >> this ALIAS, and I strongly doubt that Ned ever did that. >> </charset reviewer hat on> > > I don't believe it was ever listed as a valid charset value, > regardless of other uses. As I said, not having it there was a > very explicit decision. It was there up to around 2013, and that must have been an explicit decision, the same way that designating US-ASCII as the preferred MIME label was a very explicit decision. Regards, Martin.

1 0

Re: [ietf-charsets] [art] US-ASCII and its various names
by Martin J. Dürst Dec. 19, 2023

Dec. 19, 2023

Hello Stephen, On 2023-12-16 04:06, Steffen Nurpmeso wrote: > To add that for backward compatibility the plain ASCII alias > cannot go away, I seem to remember too that ASCII was listed as an alias, and have confirmed this with https://web.archive.org/web/20051229042158/http://www.iana.org/assignments/… Stephen, maybe you can do a bisection to find out where this alias disappeared. <charset reviewer hat on> I don't remember ever having dealt with a request to remove this ALIAS, and I strongly doubt that Ned ever did that. </charset reviewer hat on> > i do have emails with charset=ascii in my archive, It would be interesting to know how many (i.e. what percentage of overall mails, and what percentage in comparison to those labeled US-ASCII), and how old they are. Regards, Martin. > and GNU "iconv -l" gives > > $ iconv -l|grep -i ascii > ASCII// > CSASCII// > US-ASCII// > > so if there are any character-sets downstream consumers, let alone > automated ones, they need to have the alias also in the future, > how deprecated it may be. > (For the MUA i maintain i fixed it manually.) > > --steffen

1 1

Re: [ietf-charsets] US-ASCII and its various names
by Martin J. Dürst Dec. 19, 2023

Dec. 19, 2023

Hello Steffen and others, On 2023-12-19 06:52, Steffen Nurpmeso wrote: > John C Klensin wrote in > <6A76695E8BBACF208CEE084B@PSB>: > > I took you off To: as i remember you bounce my emails. > > |Steffen, Martin, > | > |Starting with IANA and copying the charsets list, as Martin > > I did, but as i am not subscribed it was not there. > I then looked there, but it seems this list has only ever seen two > emails. It is true that this list hasn't seen much traffic recently, because (fortunately, as I'd say) not many new charsets get invented. The situation with the archive for ietf-charsets(a)iana.org is indeed a bit involved. It looks like the archives are split into three parts: 1993~2004 at https://lists.w3.org/Archives/Public/ietf-charsets/ 1993~2020 at https://data.iana.org/archive/ietf-charsets/maillist.html (longer period, but not indexed as well) 2022~2023 at https://mm.icann.org/pipermail/ietf-charsets/ There seem to have been no mails to this list in 2021. My very recent mail to the list made it to the last archive without problems (see https://mm.icann.org/pipermail/ietf-charsets/2023-December/000002.html) The mails from others seem not to have made it. That's partially due to some of them using iana(a)isna.org (ISNA, not IANA; there was one from John Klensin and a reply from Steffen with this address). The other part is most probably that the mailing list doesn't accept mails from non-subscribers. I'm sure I don't have to explain why to the participants in this discussion. So please before you post subscribe at https://mm.icann.org/mailman/listinfo/ietf-charsets; this is *truely* a low-traffic mailing list. Regards, Martin.

1 0