Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: enable all languages on world; fix language dropdown (#9209) #9225

Closed
wants to merge 1 commit into from
Closed

fix: enable all languages on world; fix language dropdown (#9209) #9225

wants to merge 1 commit into from

Conversation

NooaLumi
Copy link

@NooaLumi NooaLumi commented Oct 29, 2023

What

  • Added all language codes under 'world' in countries.txt
    • This will enable the selection of all languages in the world domain (which is default)
    • All unique language codes were extracted from the rest of countries.txt with a script, matched with their display text counterparts (eg. {"name": "Magyar", "code": "hu"}), and then sorted by name. In the UI, the languages appear in alphabetical order
  • Made the language dropdown act similarly to the menu dropdown
    • The active language (dropdown toggle) is now an empty anchor tag (no href), so that when the user clicks it, the page won't refresh. It will instead act like the menu icon, and open the dropdown
    • The active language is now included in the dropdown as well (not sure if this is something we want, but it was clickable before so I kept it)
    • Note: I felt this change was necessary, because the language selection in 'world' is now a very long list. It is difficult to maintain hover. Clicking to toggle it open is much easier

Screenshot

image
image

Related issue(s) and discussion

List of language codes mapped to their names, in alphabetical order (for reference):

[{"name":"Af-Soomaali","code":"so"},{"name":"Afrikaans","code":"af"},{"name":"An Ghaeilge","code":"ga"},{"name":"Avañe'ẽ","code":"gn"},{"name":"Aymar aru","code":"ay"},{"name":"Azərbaycan dili","code":"az"},{"name":"Bahasa Indonesia","code":"id"},{"name":"Bahasa Melayu","code":"ms"},{"name":"Bislama","code":"bi"},{"name":"Bokmål","code":"nb"},{"name":"Bosanski jezik","code":"bs"},{"name":"Català","code":"ca"},{"name":"ChiShona","code":"sn"},{"name":"Chilankhulo cha Chichewa","code":"ny"},{"name":"Cymraeg","code":"cy"},{"name":"Dansk","code":"da"},{"name":"Deutsch","code":"de"},{"name":"Dorerin Naoero","code":"na"},{"name":"Eesti keel","code":"et"},{"name":"English","code":"en"},{"name":"Español","code":"es"},{"name":"Euskara","code":"eu"},{"name":"Fijian language","code":"fj"},{"name":"Fino' Chamoru","code":"ch"},{"name":"Fiteny malagasy","code":"mg"},{"name":"Fiteny malagasy","code":"mg"},{"name":"Français","code":"fr"},{"name":"Gaelg","code":"gv"},{"name":"Gagana fa'a Sāmoa","code":"sm"},{"name":"Gjuha shqipe","code":"prs"},{"name":"Gàidhlig na h-Alba","code":"gd"},{"name":"Hiri Motu","code":"ho"},{"name":"Hrvatski","code":"hr"},{"name":"Ikinyarwanda","code":"rw"},{"name":"Ikirundi","code":"rn"},{"name":"IsiXhosa","code":"xh"},{"name":"IsiZulu","code":"zu"},{"name":"Italiano","code":"it"},{"name":"Język polski","code":"pl"},{"name":"Kajin M̧ajeļ","code":"mh"},{"name":"Kalaallisut","code":"kl"},{"name":"Kiswahili","code":"sw"},{"name":"Kreyòl","code":"ht"},{"name":"Latviešu valoda","code":"lv"},{"name":"Lea fakatonga","code":"to"},{"name":"Lietuvių kalba","code":"lt"},{"name":"Limba română","code":"ro"},{"name":"Lingua Latina","code":"la"},{"name":"Lingua galega","code":"gl"},{"name":"Lëtzebuergesch","code":"lb"},{"name":"Magyar","code":"hu"},{"name":"Malti","code":"mt"},{"name":"Nederlands","code":"nl"},{"name":"Northern Ndebele language","code":"nd"},{"name":"Oʻzbek tili","code":"uz"},{"name":"Português","code":"pt"},{"name":"Qhichwa simi","code":"qu"},{"name":"Reo Māori","code":"mi"},{"name":"Sesotho","code":"st"},{"name":"Setswana","code":"tn"},{"name":"SiSwati","code":"ss"},{"name":"Slovenčina","code":"sk"},{"name":"Slovenščina","code":"sl"},{"name":"Southern Ndebele","code":"nr"},{"name":"Suomi","code":"fi"},{"name":"Svenska","code":"sv"},{"name":"Sängö","code":"sg"},{"name":"Tiếng Việt","code":"vi"},{"name":"TshiVenḓa","code":"ve"},{"name":"Türkmen dili","code":"tk"},{"name":"Türkçe","code":"tr"},{"name":"Wikang Tagalog","code":"tl"},{"name":"Xitsonga","code":"ts"},{"name":"Zimanê kurdî","code":"ku"},{"name":"Íslenska","code":"is"},{"name":"Čeština","code":"cs"},{"name":"Ελληνικά","code":"el"},{"name":"Беларуская мова","code":"be"},{"name":"Български език","code":"bg"},{"name":"Забони тоҷикӣ","code":"tg"},{"name":"Кыргыз тили","code":"ky"},{"name":"Лимба молдовеняскэ","code":"mo"},{"name":"Македонски јазик","code":"mk"},{"name":"Монгол хэл","code":"mn"},{"name":"Русский язык","code":"ru"},{"name":"Српски језик","code":"sr"},{"name":"Українська мова","code":"uk"},{"name":"Қазақ тілі","code":"kk"},{"name":"Հայերեն","code":"hy"},{"name":"עברית","code":"he"},{"name":"اردو","code":"ur"},{"name":"العربية","code":"ar"},{"name":"زبان فارسی","code":"fa"},{"name":"پښتو ژبه","code":"ps"},{"name":"ދިވެހި","code":"dv"},{"name":"नेपाली भाषा","code":"ne"},{"name":"हिन्दी","code":"hi"},{"name":"বাংলা ভাষা","code":"bn"},{"name":"தமிழ்","code":"ta"},{"name":"සිංහල භාෂාව","code":"si"},{"name":"ภาษาไทย","code":"th"},{"name":"ພາສາລາວ","code":"lo"},{"name":"རྫོང་ཁ་","code":"dz"},{"name":"မြန်မာဘာသာစကား","code":"my"},{"name":"ქართული ენა","code":"ka"},{"name":"ትግርኛ ፊደል","code":"ti"},{"name":"አማርኛ","code":"am"},{"name":"ភាសាខ្មែរ","code":"km"},{"name":"日本語","code":"ja"},{"name":"汉语","code":"zh"},{"name":"한국어","code":"ko"}]

@NooaLumi NooaLumi requested a review from a team as a code owner October 29, 2023 08:07
@github-actions github-actions bot added 🧬 Taxonomies https://wiki.openfoodfacts.org/Global_taxonomies Display labels Oct 29, 2023
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@codecov-commenter
Copy link

codecov-commenter commented Oct 29, 2023

Codecov Report

Merging #9225 (b7b6ed5) into main (ac353a5) will increase coverage by 0.01%.
Report is 1 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #9225      +/-   ##
==========================================
+ Coverage   47.80%   47.82%   +0.01%     
==========================================
  Files          65       65              
  Lines       20292    20292              
  Branches     4932     4932              
==========================================
+ Hits         9701     9705       +4     
+ Misses       9336     9333       -3     
+ Partials     1255     1254       -1     
Files Coverage Δ
lib/ProductOpener/Display.pm 11.34% <100.00%> (+0.02%) ⬆️

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@alexgarel
Copy link
Member

Hi @NooaLumi thank you so much for this contribution 🎉

Before we merge let's raise two questions:

  • @raphael0202 - can you confirm we have a no-index to avoid bots re-indexing same world page in all the languages
  • @stephanegigandet the language list on world becomes very long (and not filtered). I would say it's ok though and we can improve later (we could for example have the javascript trigger the filter as in country when there is more than 10 entries)

@alexgarel
Copy link
Member

  • @raphael0202 - can you confirm we have a no-index to avoid bots re-indexing same world page in all the languages

Hum the test that fails seems to be exactly on this !

@stephanegigandet
Copy link
Contributor

It works but for some reason I get a different behavior on Firefox and Chrome. On Firefox I get the click to open behaviour, while on Chrome I have the old hover to open behaviour.

Copy link
Member

@alexgarel alexgarel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really appreciate that you try to tackle this, but let's had a hardcoded exception for world in code instead off adding all langages in taxonomy.

@@ -265,6 +265,7 @@ zh_min_nan:Sè-kài
zh_yue:世界
country_code_2:en:world
wikidata:en:Q16502
language_codes:en:so,af,ga,gn,ay,az,id,ms,bi,nb,bs,ca,sn,ny,cy,da,de,na,et,en,es,eu,fj,ch,mg,mg,fr,gv,sm,prs,gd,ho,hr,rw,rn,xh,zu,it,pl,mh,kl,sw,ht,lv,to,lt,ro,la,gl,lb,hu,mt,nl,nd,uz,pt,qu,mi,st,tn,ss,sk,sl,nr,fi,sv,sg,vi,ve,tk,tr,tl,ts,ku,is,cs,el,be,bg,tg,ky,mo,mk,mn,ru,sr,uk,kk,hy,he,ur,ar,fa,ps,dv,ne,hi,bn,ta,si,th,lo,dz,my,ka,ti,am,km,ja,zh,ko
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative would be to add an exception in the code, but why not that way, it does not move too much, so it might be ok indeed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test is failing because of this, because we use this property to add the no_index flag (and this is very important).
See Display.pm on line 773

	if ((!($lc ~~ $country_languages{$cc})) or $subdomain =~ /^(ssl-)?api/) {
		# Use robots.txt with disallow: / for all agents
		$request_ref->{deny_all_robots_txt} = 1;

		if ($request_ref->{is_crawl_bot} eq 1) {
			$request_ref->{no_index} = 1;
		}
	}

So instead of adding this property, I propose that you add an exception in the code you modified in Display.pm with an hardcoded exception for "world".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: the no_index is very important because otherwise we get hammered by indexing bots.

@raphael0202
Copy link
Contributor

@alexgarel yes indeed, we forbid indexation on all world pages that are not in English language.
This is done through no index tag + in robots.txt

@NooaLumi NooaLumi closed this by deleting the head repository May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Display 🎯 P0 🧬 Taxonomies https://wiki.openfoodfacts.org/Global_taxonomies
Projects
Development

Successfully merging this pull request may close these issues.

Enable selecting language on world
6 participants