This is great feedback for a future updated scoring mechanism. When assigning scores to the tests in the self-certification guide, we had a few challenges and tradeoffs: 1. Distinguishing between components and systems which could host internationalized mailboxes and those which could only send/receive to/from internationalized addresses 2. Scoring the individual components from which a system could be assembled versus scoring complete end to end systems 3. Checklist pass/fail scoring versus weighed scoring 4. Limiting the number of badges The tradeoffs we selected resulted in scoring wherein even the very best system which can only send/receive can never get a score higher than Silver. There was a lot of debate about that! In contrast, a more-complex 6-badge system might have awarded Gold or Platinum to Gmail even as a send/receive system (what we used to call “level 1”) if we had made the tradeoffs differently. /marksv From: Mutegeki Cliff via UA-discuss <ua-discuss@icann.org> Sent: Monday, June 30, 2025 5:00 AM To: anil Jain <asdj1990@gmail.com> Cc: John Levine <john.levine@standcore.com>; ua-discuss@icann.org Subject: [EXTERNAL] [UA-discuss] Re: Final report on the EAI self-hosting project Dear John, Thanks for sharing all of this—great work! It’s impressive to see the EAI self-hosting server project wrapped up so thoroughly. I went through the report and slides, and the level of detail is excellent. The demo months ago already gave a strong impression of the system’s performance, so it’s interesting (and a bit surprising) to see the disparity between the real-world performance and the self-evaluation test results. I completely agree with your point about considering an overall user experience score. Some of the failed tests do seem marginal in terms of actual impact. It might be helpful to weight the evaluation criteria accordingly or create a separate UX effectiveness metric. I'll review the scripts and updated documentation more closely over the next few days and get back to you with any specific comments or suggestions. Also, I wanted to ask if you might be open to mentoring during a DNS Hackathon we’re organizing this August as part of the upcoming Uganda DNS Forum. Your experience with the EAI self-hosting project would be incredibly valuable to participants, especially those working on internationalization and DNS infrastructure. The time commitment would be minimal — mainly offering guidance and feedback to teams — and your insights could make a real difference. Let me know if this is something you’d be interested in. We’d truly appreciate your support! Thanks again for pulling all of this together! Mutegeki Cliff Agaba Digital Adoption Consultant Internet Society MidCareer Fellow, Member Internet Society, AFRALO, ICANN, UASG and IGF - WGWSP +256 701 800 679 +256 776 800 679 www.mutegekicliff.com<http://www.mutegekicliff.com/> [facebook icon]<http://www.facebook.com/mutegekicliffagaba> [linkedin icon] <https://www.linkedin.com/in/mutegekicliff/> [instagram icon] <http://www.instagram.com/mutegekicliff> On Sat, 28 Jun 2025 at 20:42, anil Jain via UA-discuss <ua-discuss@icann.org<mailto:ua-discuss@icann.org>> wrote: John, I must congratulate you for exemplary work undertaken by you. I hope UA community will get highly benefitted with your work. Congratulations John. Anil Sent from my iPhone
On 27 Jun 2025, at 23:19, John Levine via UA-discuss <ua-discuss@icann.org<mailto:ua-discuss@icann.org>> wrote:
I think I have finally finished the EAI self-hosting server project. The report on the resuts is here:
https://docs.google.com/document/d/1ThjISfMkAn6bP0yTXP-uBHebvMkPwr3LxuG_B4Yn...
And a slide presentation is here:
https://docs.google.com/document/d/1ThjISfMkAn6bP0yTXP-uBHebvMkPwr3LxuG_B4Yn...
One surprising result is that even though the server works quite well, as we saw in the demo I did a few months ago, it scores poorly on the self-evaluation tests. It looks to me like many of the tests it fails have little effect on the user's experience, so you might want to think about an overall experience score rather than just adding up individual tests.
The scripts are in an ICANN github repo here:
https://github.com/icann/eaiselfhost
The installation instructions, slightly updated from last time are here:
https://docs.google.com/document/d/1azNEZAdk6wyg0PNHBuXWWrKF1ZexQjv0Txyd1zaU...
And the installation tutorial is here:
https://docs.google.com/presentation/d/1dva_lS7nMJNx8mgRQExMy0jGONUGaLFMiM8Q...
Take a look, let me know if you have any comments.
R's, John _______________________________________________ UA-discuss mailing list -- ua-discuss@icann.org<mailto:ua-discuss@icann.org> To unsubscribe send an email to ua-discuss-leave@icann.org<mailto:ua-discuss-leave@icann.org> _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
UA-discuss mailing list -- ua-discuss@icann.org<mailto:ua-discuss@icann.org> To unsubscribe send an email to ua-discuss-leave@icann.org<mailto:ua-discuss-leave@icann.org> _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.