Improve generated process for the language database files

Fix some issues and streamline regeneration process: - A few glitches (like duplicate language symbols, wrong Windows language ids etc) were fixed. - Scripts were added to allow to regenerate the list of supported locales from the list of known Windows locales. - Version info when language symbols became available was added to the documentation of the language symbol enum. - Synonyms are no longer hard-coded in the script genlang.py. See #23453. Closes #23437.
2023-04-16 23:20:57 +02:00 · 2023-04-16 23:20:57 +02:00 · a20aa4de75
commit a20aa4de75
parent b07a77cbf9
22 changed files with 2659 additions and 1699 deletions
--- a/misc/languages/README
+++ b/misc/languages/README
@ -1,12 +1,13 @@
 Run the genlang.py script from the top level wxWidgets directory to
 update include/wx/language.h (wxLanguage enum), interface/wx/language.h
 (its documentation) and src/common/languageinfo.cpp (conversion tables)
-with the data from langtabl.txt and scripttabl.txt.
+with the data from langtabl.txt, synonymtabl.txt, and scripttabl.txt.

 langtabl.txt contains a tabular list of language entries. Each entry
 contains

 - a symbolic language identifier used in enum wxLanguage,
+- a wxWidgets version when the entry was first introduced (a hyphen if not known)
 - a BCP 47-like locale identifier,
 - a Unix locale identifier,
 - a Unix locale identifier including a region id (if the default Unix
@ -24,5 +25,70 @@ to convert between script code used in BCP 47-like identifiers and
 script modifiers used in Unix locale names. The data in (2) can be used
 to update scripttabl.txt if necessary.

+synonymtabl.txt contains a list of aliases for symbolic language identifiers.
+This list is used to generate specific entries in wxLanguage enumeration.
+
+Note: None of the files langtabl.txt, synonymtabl.txt, and scripttabl.txt should be
+edited manually. Instead these files should be regenerated under Windows 11 or
+above.
+
+Windows provides an extensive list of locales. This list is used to regenerate
+the files langtabl.txt, synonymtabl.txt, and scripttabl.txt. The subdirectory
+util contains the C source of a small utility application that queries Windows
+for a list of known locales.
+
+2 additional tools are required to perform the regeneration process:
+
+1) SQLite3 shell
+   Precompiled binaries can be downloaded from https://www.sqlite.org/download.html.
+   The download link is under the heading "Precompiled Binaries for Windows" and
+   looks like "sqlite-tools-win32-x86-3xxyyzz.zip" (where xx, yy, zz denote the
+   current SQLite version). Alternatively, the SQLite shell can be compiled from
+   sources - the archive "sqlite-amalgamation-3xxyyzz.zip" contains the required
+   source files.
+
+2) Lua shell
+   Precompiled binaries are available at https://luabinaries.sourceforge.net/.
+   Download lua-x.y.z_Win32_bin.zip or lua-x.y.z_Win64_bin.zip (where x, y, z denotes
+   the lua version) from the download page. Rename the executable luaxy.exe to
+   luashell.exe or adjust the batch file mklangtablnew.bat accordingly.
+
+The regeneration process consists of the following steps:
+
+1) Regenerate the list of known Windows locales (optional)
+   This step is usually only required when a new major Windows version is published.
+   The utility showlocales should be invoked from a command prompt as follows:
+
+     showlocales > win-locale-table-win.txt
+
+   The resulting file win-locale-table-win.txt has to be placed into subdirectory
+   data/windows.
+
+2) Update the Unicode data files (optional)
+   This step is only required when the Unicode data were actually updated.
+   To perform this step execute the batch file getunicodefiles.bat, located in the
+   data subdirectory.
+
+3) Regenerate langtabl.txt, synonymtabl.txt, and scripttabl.txt
+   To perform this step execute the batch file mklangtablnew.bat, located in the
+   data subdirectory. The new versions will be placed in the data directory.
+
+4) Check resulting new files langtabl.txt, synonymtabl.txt, and scripttabl.txt
+   The messages from step 3 issued by the batch file and the resulting files should
+   be carefully checked. If no errors occurred, the batch file replacetables.bat
+   can be executed.
+
+5) Run the Python script genlang.py from the top level wxWidgets directory.
+
+6) Commit the changes.
+
+Notes:
+1) Do not perform the regeneration process for older wxWidgets versions.
+   The scripts expect the data table files in a new format that was first introduced
+   in version 3.3.0.
+2) If you need to add locales not present in the list of known Windows locales, then
+   they should be added at the end of the script win_genlocaletable.lua.
+
+Footnotes
 (1) used on Windows only, deprecated by Microsoft
 (2) http://www.unicode.org/iso15924/iso15924-codes.html
--- a/misc/languages/data/.gitignore
+++ b/misc/languages/data/.gitignore
@ -0,0 +1,12 @@
+# Ignore temporary files that are generated in course of
+# regenerating the lists of known languages, scripts, and synonyms
+
+# exclude database and data files
+*.db3
+*.txt
+
+# exclude everything in temp
+temp/*
+
+# exception to the rule
+!temp/.gitkeep
--- a/misc/languages/data/getunicodefiles.bat
+++ b/misc/languages/data/getunicodefiles.bat
@ -0,0 +1,14 @@
+rem Download files from Unicode Org
+
+rem Retrieve working directory
+set "workdir=%cd%"
+
+rem Create backup copies
+copy unicode\iso15924.txt unicode\iso15924.txt.bak
+copy unicode\territory_codes.txt unicode\territory_codes.txt.bak
+
+rem Download ISO 15924 (list of scripts)
+bitsadmin /transfer wxDownloadJob /download /priority normal https://www.unicode.org/iso15924/iso15924.txt %workdir%\unicode\iso15924.txt
+
+rem Download ISO 3166 (list of territories)
+bitsadmin /transfer wxDownloadJob /download /priority normal https://github.com/unicode-org/cldr/raw/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/territory_codes.txt %workdir%\unicode\territory_codes.txt
--- a/misc/languages/data/mkdatabase.sql
+++ b/misc/languages/data/mkdatabase.sql
@ -0,0 +1,20 @@
+.print Initialize database
+.print Import territory codes
+.read temp/uni_loadterritorycodes.sql
+
+.print Import locales supported by Windows
+.read temp/win_loadlocaletable.sql
+
+.print Import wxWidgets language list
+.read temp/wx_loadlangtabl.sql
+
+.print Import wxWidgets language synonyms
+.read temp/wx_loadsynonymtabl.sql
+
+.print Import script map
+.read temp/uni_loadscriptmap.sql
+
+.read temp/wx_loadversion.sql
+
+.print Generate new tables (langtabl, scripttabl, synonymtabl)
+.read mkwxtables.sql
--- a/misc/languages/data/mklangtablnew.bat
+++ b/misc/languages/data/mklangtablnew.bat
@ -0,0 +1,25 @@
+rem Batch script to regenerate the list of known locales
+
+rem Create a copy of the current lists
+copy ..\langtabl.txt temp\langtabl_current.txt
+copy ..\scripttabl.txt temp\scripttabl_current.txt
+copy ..\synonymtabl.txt temp\synonymtabl_current.txt
+
+rem Generate lists based on Unicode data
+luashell unicode\uni_territorycodes.lua
+luashell unicode\uni_genscriptmap.lua
+
+rem Generate list of locales based on known Windows locales
+luashell windows\win_genlocaletable.lua
+
+rem Generate lists based on current wxWidgets data
+rem   Extract current wxWidgets version
+luashell wx\wx_readversion.lua
+rem   Load current version of langtabl.txt
+luashell wx\wx_loadlangtabl.lua
+rem   Load list of current synonyms
+luashell wx\wx_loadsynonymtabl.lua
+
+rem Generate locale data database
+del localedata.db3
+sqlite3 localedata.db3 ".read mkdatabase.sql"
--- a/misc/languages/data/mkwxtables.sql
+++ b/misc/languages/data/mkwxtables.sql
@ -0,0 +1,88 @@
+-- Create unique wx language mapping
+create table if not exists wx_langmapuni (wxlangname char, wxlangid char, wxlocregion char, primary key (wxlangname));
+delete from wx_langmapuni;
+insert into wx_langmapuni select wxlangname, wxlangid, wxlocregion from wx_langmap;
+
+-- Create mapping between wx and Windows ids
+create table if not exists wx2winmapping (wxlocidnew char, wxnamenew char, wxlocidold char, wxnameold char, primary key (wxlocidnew));
+delete from wx2winmapping;
+
+insert into wx2winmapping select wt.wxlocid, wt.wxname, ifnull(wm.wxlangid,'-'), ifnull(wm.wxlangname,'-') from win_localetable wt
+  left join wx_langmapuni wm on wt.wxlocid=wm.wxlangid;
+
+update wx2winmapping set wxlocidold = ifnull((select wxlangid from wx_langmapuni where wx_langmapuni.wxlangname=wx2winmapping.wxnamenew),'-') where wx2winmapping.wxlocidold='-';
+
+-- Create mapping between language and country
+create table if not exists lang2iso2 (lang char, iso2 char, wintag char, primary key (lang));
+delete from lang2iso2;
+insert into lang2iso2 select w.locid, w.locid||'_'||c.iso2, w.locid||'-'||c.iso2 from win_localetable w, uni_territorycodes c where w.locid not like '%-%' and w.country3=c.iso3;
+
+-- Perform checks
+.output temp/wxmissing.log
+select '*** Missing via wx_langmapuni';
+select wxlangname, wxlangid from wx_langmapuni where not exists (select wxnameold from wx2winmapping where wxnameold=wxlangname);
+select '*** Missing via wx_langmap';
+select wxlangname, wxlangid from wx_langmap where not exists (select wxname from win_localetable where wxname=wxlangname);
+.output stdout
+
+select '  #missing wxlangmapuni =', count(wxlangname) from wx_langmapuni where not exists (select wxnameold from wx2winmapping where wxnameold=wxlangname);
+select '  #matched wxlangmapuni =', count(wxlangname) from wx_langmapuni where exists (select wxnameold from wx2winmapping where wxnameold=wxlangname);
+select '  #matched wx2win       =', count(distinct wxnameold) from wx2winmapping where wxnameold <> '-';
+
+select '  #missing wxlangmap    =', count(wxlangname) from wx_langmap where not exists (select wxname from win_localetable where wxname=wxlangname) and not exists (select wxnamesyn from wx_synonyms where wxnamesyn=wxlangname);
+select wxlangname from wx_langmap where not exists (select wxname from win_localetable where wxname=wxlangname) and not exists (select wxnamesyn from wx_synonyms where wxnamesyn=wxlangname);
+
+select wxname from win_localetable where exists (select wxnamesyn from wx_synonyms where wxnamesyn = win_localetable.wxname);
+
+-- Generate references to previous wx mappings
+update win_localetable set (wxlocidold, wxnameold) =
+  (select wxlocregion, wxlangname from wx_langmap m1 where m1.wxlangid=win_localetable.wxlocid);
+
+-- Replace NULL values with strings ('-')
+update win_localetable set wxlocidold = '-' where wxlocidold is null;
+update win_localetable set wxnameold = '-' where wxnameold is null;
+
+-- Adjust Windows language ids
+--.changes on
+update win_localetable as t1 set (idlang,idsublang) =
+    (t2.idlang, t2.idsublang) from win_localetable t2
+    where t1.wxlocid <> '-' and t2.wxlocid = t1.wxlocidold and (t1.idlang <> t2.idlang or t1.idsublang <> t2.idsublang);
+--.changes off
+
+-- Adjust canonical reference for language-only entries
+update win_localetable as w1 set wxlocidold =
+  (select m.iso2 from win_localetable w, lang2iso2 m
+    where w.locid=w1.locid and w.locid=m.lang
+      and exists (select 1 from win_localetable w2 where w2.locid=m.wintag))
+  where w1.locid not like '%-%' and (w1.wxlocidold='-' or w1.wxlocidold is null);
+
+update win_localetable set wxlocidold = '-' where wxlocidold is null;
+
+-- Initialize parameter holding the current wx version
+.parameter init
+.print Set wx version parameter
+.parameter set @wxversion "(select version from wx_version)"
+.parameter list
+
+-- Generate new list of languages
+.output langtabl.txt
+select printf('%-55s %-9s %-14s %-25s %-14s %-4s %-4s %s "%s","%s"',
+              w.wxname, ifnull(x.wxversion,@wxversion), w.locid, w.wxlocid, w.wxlocidold,
+              w.idlang, w.idsublang, w.layout, w.englishdisplay, w.nativedisplayhex)
+  from win_localetable w left join wx_langmap x on w.wxlocid = x.wxlangid order by w.wxname;
+.output stdout
+
+-- Generate new list of script mappings
+.output scripttabl.txt
+select printf('%-4s %s', sctag, lower(scalias)) from uni_scriptmap where scalias <> '-' order by sctag;
+.output stdout
+
+-- Generate new list of wxLANGUAGE synonyms
+insert into wx_synonyms
+  select wl.wxnameold, wl.wxname, wl.locid, ifnull(wm.wxversion,@wxversion)
+    from win_localetable wl left join wx_langmap wm on wl.wxnameold = wm.wxlangname
+    where wl.wxnameold <> '-' and wl.wxname <> wl.wxnameold;
+delete from wx_synonyms where wxnamesyn in (select wxname from win_localetable);
+.output synonymtabl.txt
+select printf('%-55s %-55s %-14s %s', wxnamesyn, wxnameprim, winlocid, wxversion) from wx_synonyms order by wxnamesyn;
+.output stdout
--- a/misc/languages/data/replacetables.bat
+++ b/misc/languages/data/replacetables.bat
@ -0,0 +1,6 @@
+rem Batch script to replace the data tables by new versions
+
+rem Replace previous versions with new versions
+copy langtabl.txt ..\langtabl_current.txt
+copy scripttabl.txt ..\scripttabl_current.txt
+copy synonymtabl.txt ..\synonymtabl_current.txt
--- a/misc/languages/data/temp/.gitkeep
+++ b/misc/languages/data/temp/.gitkeep
--- a/misc/languages/data/unicode/uni_genscriptmap.lua
+++ b/misc/languages/data/unicode/uni_genscriptmap.lua
@ -0,0 +1,58 @@
+-- Generate a script map based on ISO 15924 data
+--
+-- Data source: Unicode
+-- URL: https://www.unicode.org/iso15924/iso15924.txt
+-- License: UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE
+
+codeFileName = "unicode/iso15924.txt"
+
+function split(str, character)
+  result = {}
+
+  index = 1
+  for s in string.gmatch(str .. character, "(.-)"..character) do
+    result[index] = s
+    index = index + 1
+  end
+
+  return result
+end
+
+fo2 = io.open('temp/uni_loadscriptmap.sql','w')
+fo2:write('-- Mapping of script codes based on ISO 15924\n-- Date: ' .. os.date("%Y-%m-%d %H:%M") .. '\n\n')
+fo2:write('-- Data source: Unicode Org (https://www.unicode.org)\n')
+fo2:write('-- URL: https://www.unicode.org/iso15924/iso15924.txt\n')
+fo2:write('-- License: UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE\n\n')
+
+fo2:write("create table if not exists uni_scriptmap (sctag char, scname char, scalias char, primary key (sctag));\n")
+fo2:write('delete from uni_scriptmap;\nbegin;\n')
+
+-- Names and order of data columns
+-- Code|N°|English Name|Nom français|Alias|Age|Date
+
+count = 0
+for line in io.lines(codeFileName) do
+  rem = string.sub(line,1,1)
+  if rem ~= "#"  and rem ~= "" then
+    values = split(line, ";")
+    sctag = values[1]
+    scnum = values[2]
+    scname = values[3]
+    scname_fr = values[4]
+    scalias = values[5]
+    scage = values[6]
+    scdate = values[7]
+
+    scname = scname:gsub("'", "''")
+    if scalias == "" then
+      scalias = "-"
+    end
+
+    fo2:write("insert into uni_scriptmap values ('" .. sctag .. "', '" .. scname .. "', '" .. scalias .. "');\n")
+    count = count + 1
+  end
+end
+print("Number of script mappings=" .. count)
+
+fo2:write('commit;\n')
+fo2:close()
--- a/misc/languages/data/unicode/uni_territorycodes.lua
+++ b/misc/languages/data/unicode/uni_territorycodes.lua
@ -0,0 +1,61 @@
+-- Generate a script map based on ISO 15924 data
+--
+-- Data source: Unicode CLDR
+-- URL: https://github.com/unicode-org/cldr/raw/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/territory_codes.txt
+-- License: UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE
+
+codeFileName = "unicode/territory_codes.txt"
+
+function split(str)
+  result = {}
+
+  index = 1
+  for s in string.gmatch(str, "%S+") do
+    result[index] = s
+    index = index + 1
+  end
+
+  return result
+end
+
+fo1 = io.open('temp/uni_loadterritorycodes.sql','w')
+fo1:write('-- List of territory codes based on ISO 3166\n-- Date: ' .. os.date("%Y-%m-%d %H:%M") .. '\n\n')
+fo1:write('-- Data source: Unicode CLDR (https://cldr.unicode.org)\n')
+fo1:write('-- File: https://github.com/unicode-org/cldr/raw/main/tools/cldr-code/src/main/resources/org/unicode/cldr/util/data/territory_codes.txt\n')
+fo1:write('-- License: UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE\n\n')
+
+fo1:write("create table if not exists uni_territorycodes (iso2 char, iso3 char, primary key (iso2));\n")
+fo1:write('delete from uni_territorycodes;\nbegin;\n')
+
+count = 0
+ccprev = ""
+for line in io.lines(codeFileName) do
+  rem = string.sub(line,1,1)
+  if rem ~= "#" then
+    values = split(line)
+
+    -- 1 RFC3066
+    -- 2 UN Numeric
+    -- 3 ISO3166 Alpha-3
+    -- 4 internet
+    -- 5 FIPS-10
+
+    iso2 = values[1]
+    iso3 = values[3]
+
+    if iso3 ~= nil then
+      fo1:write("insert into uni_territorycodes values ('" .. iso2 .. "', '" .. iso3 .. "');\n")
+    else
+      print('*** Warning: no iso3 code for ' .. iso2)
+    end
+    count = count + 1
+  else
+    if line == "# Deprecated codes" then
+      break
+    end
+  end
+end
+print("Number of countries=" .. count)
+
+fo1:write('commit;\n')
+fo1:close()
--- a/misc/languages/data/windows/win_genlocaletable.lua
+++ b/misc/languages/data/windows/win_genlocaletable.lua
@ -0,0 +1,380 @@
+-- Assign script arguments and check validity
+-- Data source:
+-- File generated by utility app from Windows data
+codeFileName = "windows/win-locale-table-win.txt"
+
+function scriptmap()
+-- mapping script id to script name
+-- needs to be updated manually, if new scripts are encountered
+return {
+  ["Arab"] = "arabic",
+  ["Adlm"] = "adlam",
+  ["Beng"] = "bengali",
+  ["Cakm"] = "chakma",
+  ["Cans"] = "canadian_aboriginal",
+  ["Cher"] = "cherokee",
+  ["Cyrl"] = "cyrillic",
+  ["Deva"] = "devanagari",
+  ["Guru"] = "gurmukhi",
+  ["Java"] = "javanese",
+  ["Latn"] = "latin",
+  ["Mong"] = "mongolian",
+  ["Olck"] = "ol_chiki",
+  ["Tfng"] = "tifinagh",
+  ["Vaii"] = "vai"
+}
+end
+
+function initmap()
+-- mapping locale id to wx language identifier
+return {
+  ["ar_AE"] = "wxLANGUAGE_ARABIC_UAE",
+  ["bn_IN"] = "wxLANGUAGE_BANGLA_INDIA",
+  ["ca_ES@valencia"] = "wxLANGUAGE_VALENCIAN",
+  ["chr@cherokee"] = "wxLANGUAGE_CHEROKEE_CHEROKEE",
+  ["chr_US@cherokee"] = "wxLANGUAGE_CHEROKEE_US",
+  ["cy_GB"] = "wxLANGUAGE_WELSH_UK",
+  ["de_AT"] = "wxLANGUAGE_GERMAN_AUSTRIAN",
+  ["de_CH"] = "wxLANGUAGE_GERMAN_SWISS",
+  ["en_IE"] = "wxLANGUAGE_ENGLISH_EIRE",
+  ["en_TT"] = "wxLANGUAGE_ENGLISH_TRINIDAD",
+  ["en_GB"] = "wxLANGUAGE_ENGLISH_UK",
+  ["en_US"] = "wxLANGUAGE_ENGLISH_US",
+  ["es_MX"] = "wxLANGUAGE_SPANISH_MEXICAN",
+  ["es_US"] = "wxLANGUAGE_SPANISH_US",
+  ["ff"] = "wxLANGUAGE_FULAH",
+  ["fo"] = "wxLANGUAGE_FAEROESE",
+  ["fo_DK"] = "wxLANGUAGE_FAEROESE_DENMARK",
+  ["fo_FO"] = "wxLANGUAGE_FAEROESE_FAROE_ISLANDS",
+  ["fr_BE"] = "wxLANGUAGE_FRENCH_BELGIAN",
+  ["fr_CA"] = "wxLANGUAGE_FRENCH_CANADIAN",
+  ["fr_CH"] = "wxLANGUAGE_FRENCH_SWISS",
+  ["fy"] = "wxLANGUAGE_FRISIAN",
+  ["fy_NL"] = "wxLANGUAGE_FRISIAN_NETHERLANDS",
+  ["gd"] = "wxLANGUAGE_SCOTS_GAELIC",
+  ["gd_GB"] = "wxLANGUAGE_SCOTS_GAELIC_UK",
+  ["haw_US"] = "wxLANGUAGE_HAWAIIAN_US",
+  ["it_CH"] = "wxLANGUAGE_ITALIAN_SWISS",
+  ["jv@latin"] = "wxLANGUAGE_JAVANESE_LATIN",
+  ["jv_ID@latin"] = "wxLANGUAGE_JAVANESE_LATIN_INDONESIA",
+  ["kr"] = "wxLANGUAGE_KANURI",
+  ["ku@arabic"] = "wxLANGUAGE_CENTRAL_KURDISH_ARABIC",
+  ["ku_IQ@arabic"] = "wxLANGUAGE_CENTRAL_KURDISH_ARABIC_IRAQ",
+  ["kw_GB"] = "wxLANGUAGE_CORNISH_UK",
+  ["ky"] = "wxLANGUAGE_KIRGHIZ",
+  ["ky_KG"] = "wxLANGUAGE_KIRGHIZ_KYRGYZSTAN",
+  ["lkt_US"] = "wxLANGUAGE_LAKOTA_US",
+  ["lo"] = "wxLANGUAGE_LAOTHIAN",
+  ["lo_LA"] = "wxLANGUAGE_LAOTHIAN_LAOS",
+  ["mni"] = "wxLANGUAGE_MANIPURI",
+  ["mni@bengali"] = "wxLANGUAGE_MANIPURI_BENGALI",
+  ["mni_IN"] = "wxLANGUAGE_MANIPURI_INDIA",
+  ["mn@cyrillic"] = "wxLANGUAGE_MONGOLIAN_CYRILLIC",
+  ["mn@mongolian"] = "wxLANGUAGE_MONGOLIAN_TRADITIONAL",
+  ["mn_CN@mongolian"] = "wxLANGUAGE_MONGOLIAN_TRADITIONAL_CHINA",
+  ["mn_MN@mongolian"] = "wxLANGUAGE_MONGOLIAN_TRADITIONAL_MONGOLIA",
+  ["nl_BE"] = "wxLANGUAGE_DUTCH_BELGIAN",
+  ["pa@arabic"] = "wxLANGUAGE_PUNJABI_ARABIC",
+  ["pa@gurmukhi"] = "wxLANGUAGE_PUNJABI_GURMUKHI",
+  ["pt_BR"] = "wxLANGUAGE_PORTUGUESE_BRAZILIAN",
+  ["quc@latin"] = "wxLANGUAGE_KICHE_LATIN",
+  ["rm"] = "wxLANGUAGE_RHAETO_ROMANCE",
+  ["rm_CH"] = "wxLANGUAGE_RHAETO_ROMANCE_SWITZERLAND",
+  ["rn"] = "wxLANGUAGE_KIRUNDI",
+  ["rn_BI"] = "wxLANGUAGE_KIRUNDI_BURUNDI",
+  ["sd@arabic"] = "wxLANGUAGE_SINDHI_ARABIC",
+  ["se"] = "wxLANGUAGE_SAMI",
+  ["se_FI"] = "wxLANGUAGE_SAMI_FINLAND",
+  ["se_NO"] = "wxLANGUAGE_SAMI_NORWAY",
+  ["se_SE"] = "wxLANGUAGE_SAMI_SWEDEN",
+  ["sg"] = "wxLANGUAGE_SANGHO",
+  ["sg_CF"] = "wxLANGUAGE_SANGHO_CENTRAL_AFRICAN_REPUBLIC",
+  ["si"] = "wxLANGUAGE_SINHALESE",
+  ["si_LK"] = "wxLANGUAGE_SINHALESE_SRI_LANKA",
+  ["sw"] = "wxLANGUAGE_SWAHILI",
+  ["sw_CD"] = "wxLANGUAGE_SWAHILI_CONGO_DRC",
+  ["sw_KE"] = "wxLANGUAGE_SWAHILI_KENYA",
+  ["sw_TZ"] = "wxLANGUAGE_SWAHILI_TANZANIA",
+  ["sw_UG"] = "wxLANGUAGE_SWAHILI_UGANDA",
+  ["to"] = "wxLANGUAGE_TONGA",
+  ["to_TO"] = "wxLANGUAGE_TONGA_TONGA",
+  ["ts"] = "wxLANGUAGE_TSONGA",
+  ["ts_ZA"] = "wxLANGUAGE_TSONGA_SOUTH_AFRICA",
+  ["ug"] = "wxLANGUAGE_UIGHUR",
+  ["ug_CN"] = "wxLANGUAGE_UIGHUR_CHINA",
+  ["xh"] = "wxLANGUAGE_XHOSA",
+  ["xh_ZA"] = "wxLANGUAGE_XHOSA_SOUTH_AFRICA",
+  ["zu"] = "wxLANGUAGE_ZULU",
+  ["zu_ZA"] = "wxLANGUAGE_ZULU_SOUTH_AFRICA",
+  ["zh@Hans"] = "wxLANGUAGE_CHINESE_SIMPLIFIED_EXPLICIT",
+  ["zh@Hant"] = "wxLANGUAGE_CHINESE_TRADITIONAL_EXPLICIT",
+  ["zh_CN"] = "wxLANGUAGE_CHINESE_CHINA",
+  ["zh_HK"] = "wxLANGUAGE_CHINESE_HONGKONG",
+  ["zh_HK@Hans"] = "wxLANGUAGE_CHINESE_SIMPLIFIED_HONGKONG",
+  ["zh_MO"] = "wxLANGUAGE_CHINESE_MACAO",
+  ["zh_MO@Hans"] = "wxLANGUAGE_CHINESE_SIMPLIFIED_MACAO",
+  ["zh_SG"] = "wxLANGUAGE_CHINESE_SINGAPORE",
+  ["zh_TW"] = "wxLANGUAGE_CHINESE_TAIWAN"
+}
+end
+
+--??  ["kr@latin"] = "wxLANGUAGE_KANURI_LATIN",
+--??  ["mn_MN"] = "wxLANGUAGE_MONGOLIAN_MONGOLIA",
+
+function split(str, character)
+  result = {}
+
+  index = 1
+  for s in string.gmatch(str .. character, "(.-)"..character) do
+    result[index] = s
+    index = index + 1
+  end
+
+  return result
+end
+
+function string.tohex(str)
+  return (str:gsub('.', function (c)
+      cbyte = string.byte(c)
+      if cbyte >= 20 and cbyte < 127 then
+        return c
+      else
+        return string.format('\\%03o', string.byte(c))
+      end
+    end))
+end
+
+function string.fromhex(str)
+  return (str:gsub('..', function (cc)
+    return tonumber(cc, 16)
+  end))
+end
+
+script2alias = scriptmap();
+locid2wxid = initmap()
+
+fo2 = io.open('temp/langtabl_test.txt','w')
+
+fo1 = io.open('temp/win_loadlocaletable.sql','w')
+fo1:write('-- List of locales supported by Windows\n-- Date: ' .. os.date("%Y-%m-%d %H:%M") .. '\n\n')
+fo1:write('-- Data source: MSDN\n\n')
+
+fo1:write("create table if not exists win_localetable (locid char, wxlocid char, wxname char, wxlocidold char, wxnameold char, englishdisplay char, englishlanguage char, englishcountry char, scripts char, country3 char, script4 char, currency char, layout char, idlang char, idsublang char, nativedisplay char, nativelanguage char, nativecountry char, nativedisplayhex char, nativelanguagehex char, nativecountryhex char, primary key (locid));\n")
+fo1:write('delete from win_localetable;\nbegin;\n')
+
+count = 0
+for line in io.lines(codeFileName) do
+  rem = string.sub(line,1,1)
+  if rem ~= "#" then
+    values = split(line, "|")
+    -- Column names and descriptions
+    --  1 LOCALE_SNAME,                   // Tag identifying the locale
+    --  2 LOCALE_SENGLISHDISPLAYNAME,     // English locale name (example: "German (Germany)"
+    --  3 LOCALE_SENGLISHLANGUAGENAME,    // English language name
+    --  4 LOCALE_SENGLISHCOUNTRYNAME,     // English country name
+    --  5 LOCALE_SSCRIPTS,                // List of scripts
+    --    LOCALE_SISOCTRYNAME2
+    --  6 LOCALE_SINTLSYMBOL,             // Currency symbol ISO
+    --  7 LOCALE_IREADINGLAYOUT,          // Layout direction 0=LTR, 1=RTL, 2=TTB-RTL or LTR, 3=TTB-LTR
+    --  8 LOCALE_ILANGUAGE,               // Numeric language code
+    --  9 LOCALE_SNATIVEDISPLAYNAME,      // Native locale name (example: "Deutsch (Deutschland)"
+    -- 10 LOCALE_SNATIVELANGUAGENAME,     // Native language name
+    -- 11 LOCALE_SNATIVECOUNTRYNAME       // Native country name
+
+    locid = values[1]
+
+    -- Extract language, script, and region
+    sep1, sep2 = string.find(locid, "-")
+    if sep1 ~= nil then
+      lcid = string.sub(locid, 1, sep1-1)
+      ccid = string.sub(locid, sep2+1)
+      sep1, sep2 = string.find(ccid, "-")
+      if sep1 ~= nil then
+        script = string.sub(ccid, 1, sep1-1)
+        if #script < 4 then
+          temp = script
+          script = string.sub(ccid, sep2+1)
+          ccid = temp
+        else
+          ccid = string.sub(ccid, sep2+1)
+        end
+      else
+        if #ccid >= 4 then
+          script = ccid
+          ccid = ""
+        else
+          script = ""
+        end
+      end
+    else
+      lcid = locid
+      script = ""
+      ccid = ""
+    end
+
+    ENGLISHDISPLAYNAME = values[2]
+    ENGLISHDISPLAYNAME = ENGLISHDISPLAYNAME:gsub("&", "and")
+    ENGLISHDISPLAYNAME = ENGLISHDISPLAYNAME:gsub("'", "''")
+    ENGLISHDISPLAYNAMEHEX = string.tohex(ENGLISHDISPLAYNAME)
+    if ENGLISHDISPLAYNAME ~= ENGLISHDISPLAYNAMEHEX then
+      print('*** ' .. locid .. ' - Display name contains non-ASCII characters')
+    end
+    ENGLISHLANGUAGENAME = values[3]
+    ENGLISHLANGUAGENAME = ENGLISHLANGUAGENAME:gsub("'", "''")
+    ENGLISHCOUNTRYNAME = values[4]
+    ENGLISHCOUNTRYNAME = ENGLISHCOUNTRYNAME:gsub("'", "''")
+    ENGLISHCOUNTRYNAME = ENGLISHCOUNTRYNAME:gsub("&", "and")
+
+    SCRIPTS = values[5]
+    if SCRIPTS ~= nil and #SCRIPTS > 4 then
+      local b = string.find(SCRIPTS, ';')
+      script4 = string.sub(SCRIPTS,1,b-1)
+    else
+      scripts4 = ''
+    end
+    COUNTRY3 = values[6]
+    CURRENCY = values[7]
+    LAYOUT = values[8]
+    if LAYOUT ~= '1' then
+      LAYOUT = 'LTR'
+    else
+      LAYOUT = 'RTL'
+    end
+    LCID               = values[9]
+    lcidnum = tonumber('0x'..LCID)
+    lcidnum = lcidnum % 65536
+    divisor = 1024
+    idlang = lcidnum % divisor
+    idsublang = (lcidnum - idlang) / divisor
+    idlang = string.format("0x%02x", idlang)
+    idsublang = string.format("0x%02x", idsublang)
+
+    -- Clean up names
+    -- Replace single apostroph by 2 apostrophes (SQL requirement)
+    NATIVEDISPLAYNAME  = values[10]
+    NATIVEDISPLAYNAME  = NATIVEDISPLAYNAME:gsub("'", "''")
+    NATIVEDISPLAYNAMEHEX  = string.tohex(NATIVEDISPLAYNAME)
+    NATIVELANGUAGENAME = values[11]
+    NATIVELANGUAGENAME = NATIVELANGUAGENAME:gsub("'", "''")
+    NATIVELANGUAGENAMEHEX = string.tohex(NATIVELANGUAGENAME)
+    NATIVECOUNTRYNAME  = values[12]
+    NATIVECOUNTRYNAME  = NATIVECOUNTRYNAME:gsub("'", "''")
+    NATIVECOUNTRYNAMEHEX  = string.tohex(NATIVECOUNTRYNAME)
+
+    -- Replace common characters with accents by the base character
+    -- Needed to generate pure ASCII identifier for wxLANGUAGE symbols
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAME
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("é", "e")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("ô", "o")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("’", "")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("'", "")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("-", "_")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("å", "a")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("ç", "c")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("ã", "a")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("Å", "A")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("ü", "u")
+    ENGLISHLANGUAGENAMEASC = ENGLISHLANGUAGENAMEASC:gsub("ʼ", "")
+
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAME
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("é", "e")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("ô", "o")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("’", "")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("'", "")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("%.", "")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("-", "_")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("å", "a")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("ç", "c")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("ã", "a")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("Å", "A")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("ü", "u")
+    ENGLISHCOUNTRYNAMEASC = ENGLISHCOUNTRYNAMEASC:gsub("í", "i")
+
+    -- Generate wxLANGUAGE symbol
+    locidnew = lcid
+    WXNAME = 'wxLANGUAGE_' .. string.upper(ENGLISHLANGUAGENAMEASC)
+    if #ccid > 0 then
+      locidnew = locidnew .. '_' .. ccid
+      WXNAME = WXNAME .. '_' .. string.upper(ENGLISHCOUNTRYNAMEASC)
+    end
+
+    -- Append script alias to locale identifier
+    if #script > 0 then
+      mappedscript = script2alias[script]
+      if mappedscript ~= nil then
+        scriptold = script
+        script = mappedscript
+      else
+        mappedscript = 'n/a'
+      end
+      locidnew = locidnew .. '@' .. script
+--      print('-- New script: ' .. script .. ' - ' .. mappedscript .. ' ' .. locidnew)
+    end
+
+    -- Replace blank characters by underscore
+    -- Remove parentheses and commas
+    WXNAME  = WXNAME:gsub(" ", "_")
+    WXNAME  = WXNAME:gsub("%(", "")
+    WXNAME  = WXNAME:gsub("%)", "")
+    WXNAME  = WXNAME:gsub("%,", "")
+
+    -- Force specific wxLANGUAGE name for listed locale ids (compatibility with prior wxWidgets versions)
+    mappedwxname = locid2wxid[locidnew]
+    if mappedwxname ~= nil then
+      WXNAME = mappedwxname
+    else
+      mappedwxname = 'n/a'
+    end
+--    print('-- ' .. locidnew .. ' ' .. WXNAME .. ' ' .. mappedwxname)
+
+    -- Force columnar output by appending blank characters
+    SWXNAME = string.format("%-55s", WXNAME)
+    slocid =  string.format("%-14s", locidnew)
+    sidlang = string.format("%-4s", idlang)
+    sidsublang = string.format("%-4s", idsublang)
+
+    fo1:write("insert into win_localetable values ('" .. locid .. "', '" .. locidnew .. "', '" .. WXNAME .. "', '-', '-', '" .. string.tohex(ENGLISHDISPLAYNAME) .. "', '" .. string.tohex(ENGLISHLANGUAGENAME) .. "', '" .. string.tohex(ENGLISHCOUNTRYNAME) .. "', '" .. SCRIPTS .. "', '" .. COUNTRY3 .. "', '" .. script4 .. "', '" .. CURRENCY .. "', '" .. LAYOUT .. "', '" .. sidlang .. "', '" .. sidsublang .. "', '" .. NATIVEDISPLAYNAME .. "', '" .. NATIVELANGUAGENAME .. "', '" .. NATIVECOUNTRYNAME .. "', '" .. NATIVEDISPLAYNAMEHEX .. "', '" .. NATIVELANGUAGENAMEHEX .. "', '" .. NATIVECOUNTRYNAMEHEX .. "');\n")
+
+    fo2:write(SWXNAME .. " " .. slocid .. " " .. sidlang .. " " .. sidsublang .. " " .. LAYOUT .. ' "' .. ENGLISHDISPLAYNAMEHEX .. '","' .. NATIVEDISPLAYNAMEHEX .. '"\n')
+
+    count = count + 1
+  end
+end
+
+-- Add manually entries for compatibility with prior wxWidgets versions
+fo1:write("insert into win_localetable values ('ab', 'ab', 'wxLANGUAGE_ABKHAZIAN', '-', '-', 'Abkhazian', 'Abkhazian', '', '', '', '', '', 'LTR', '-', '-', 'аҧсуа бызшәа', 'аҧсуа бызшәа', '', '" .. string.tohex('аҧсуа бызшәа') .. "', '" .. string.tohex('аҧсуа бызшәа') .. "', '');\n")
+fo1:write("insert into win_localetable values ('ay', 'ay', 'wxLANGUAGE_AYMARA', '-', '-', 'Aymara', 'Aymara', '', '', '', '', '', 'LTR', '-', '-', 'Aymara', 'Aymara', '', 'Aymara', 'Aymara', '');\n")
+fo1:write("insert into win_localetable values ('bh', 'bh', 'wxLANGUAGE_BIHARI', '-', '-', 'Bihari', 'Bihari', '', '', '', '', '', 'LTR', '-', '-', 'Bihari', 'Bihari', '', 'Bihari', 'Bihari', '');\n")
+fo1:write("insert into win_localetable values ('bi', 'bi', 'wxLANGUAGE_BISLAMA', '-', '-', 'Bislama', 'Bislama', '', '', '', '', '', 'LTR', '-', '-', 'Bislama', 'Bislama', 'Vanuatu', 'Bislama', 'Bislama', 'Vanuatu');\n")
+fo1:write("insert into win_localetable values ('en-CB', 'en_CB', 'wxLANGUAGE_ENGLISH_CARIBBEAN_CB', '-', '-', 'English (Caribbean)', 'English (Caribbean)', '', '', '', '', '', 'LTR', '0x09', '0x09', 'English (Caribbean)', 'English (Caribbean)', '', 'English (Caribbean)', 'English (Caribbean)', '');\n")
+
+fo1:write("insert into win_localetable values ('fj', 'fj', 'wxLANGUAGE_FIJI', '-', '-', 'Fiji', 'Fiji', '', '', '', '', '', 'LTR', '-', '-', 'Na Vosa Vakaviti', 'Na Vosa Vakaviti', '', 'Na Vosa Vakaviti', 'Na Vosa Vakaviti', '');\n")
+fo1:write("insert into win_localetable values ('ie', 'ie', 'wxLANGUAGE_INTERLINGUE', '-', '-', 'Interlingue', 'Interlingue', '', '', '', '', '', 'LTR', '-', '-', 'Interlingue', 'Interlingue', '', 'Interlingue', 'Interlingue', '');\n")
+fo1:write("insert into win_localetable values ('ik', 'ik', 'wxLANGUAGE_INUPIAK', '-', '-', 'Inupiak', 'Inupiak', '', '', '', '', '', 'LTR', '-', '-', 'Inupiaq', 'Inupiaq', '', 'Inupiaq', 'Inupiaq', '');\n")
+fo1:write("insert into win_localetable values ('ks-IN', 'ks_IN', 'wxLANGUAGE_KASHMIRI_INDIA', '-', '-', 'Kashmiri (India)', 'Kashmiri', 'India', '', '', '', '', 'RTL', '0x00', '0x04', 'کشور (بھارت)', 'کٲشُر', 'ہِندوستان', '" .. string.tohex('کشور (بھارت)') .."', '" .. string.tohex('کٲشُر') .."', '" .. string.tohex('ہِندوستان') .."');\n")
+fo1:write("insert into win_localetable values ('ku-TR', 'ku_TR', 'wxLANGUAGE_KURDISH', '-', '-', 'Kurdish', 'Kurdish', '', '', '', '', '', 'LTR', '-', '-', 'Kurdî', 'Kurdî', '', '" .. string.tohex('Kurdî') .. "', '" .. string.tohex('Kurdî') .. "', '');\n")
+
+fo1:write("insert into win_localetable values ('la-001', 'la_001', 'wxLANGUAGE_LATIN_WORLD', '-', '-', 'Latin (World)', 'Latin', 'World', '', '', '', '', 'LTR', '0x76', '0x01', 'Latina (World)', 'Latina', 'World', 'Latina (Mundus)', 'Latina', 'Mundus');\n")
+fo1:write("insert into win_localetable values ('mo', 'mo', 'wxLANGUAGE_MOLDAVIAN', '-', '-', 'Moldavian', 'Moldavian', '', '', '', '', '', 'LTR', '-', '-', 'лимба молдовеняскэ', 'лимба молдовеняскэ', '', '" .. string.tohex('лимба молдовеняскэ') .. "', '" .. string.tohex('лимба молдовеняскэ') .. "', '');\n")
+fo1:write("insert into win_localetable values ('na', 'na', 'wxLANGUAGE_NAURU', '-', '-', 'Nauru', 'Nauru', 'Nauru', '', '', '', '', 'LTR', '-', '-', 'Nauru', 'Nauru', 'Nauru', 'Nauru', 'Nauru', 'Nauru');\n")
+fo1:write("insert into win_localetable values ('qu', 'qu', 'wxLANGUAGE_QUECHUA_MACRO', '-', '-', 'Quechua', 'Quechua', '', '', '', '', '', 'LTR', '-', '-', 'Qhichwa', 'Qhichwa', '', 'Qhichwa', 'Qhichwa', '');\n")
+fo1:write("insert into win_localetable values ('sm', 'sm', 'wxLANGUAGE_SAMOAN', '-', '-', 'Samoan', 'Samoan', '', '', '', '', '', 'LTR', '-', '-', 'Samoa', 'Samoa', 'Samoa', 'Samoa', 'Samoa', 'Samoa');\n")
+
+fo1:write("insert into win_localetable values ('sr-RS', 'sr_RS', 'wxLANGUAGE_SERBIAN_SERBIA', '-', '-', 'Serbian (Serbia)', 'Serbian', 'Serbia', '', '', '', '', 'LTR', '0x1a', '0x01', 'srpski (Srbija)', 'srpski', 'Srbija', 'srpski (Srbija)', 'srpski', 'Srbija');\n")
+fo1:write("insert into win_localetable values ('sr-YU', 'sr_YU', 'wxLANGUAGE_SERBIAN_YU', '-', '-', 'Serbian', 'Serbian', '', '', '', '', '', 'LTR', '0x1a', '0x01', 'srpski', 'srpski', 'Srbija', 'srpski', 'srpski', 'Srbija');\n")
+fo1:write("insert into win_localetable values ('sr-Cyrl-YU', 'sr_YU@cyrillic', 'wxLANGUAGE_SERBIAN_CYRILLIC_YU', '-', '-', 'Serbian (Cyrillic)', 'Serbian (Cyrillic)', '', 'Cyrl;', '', '', '', 'LTR', '0x1a', '0x03', 'српски (Србија)', 'српски', 'Србија', '" .. string.tohex('српски (Србија)') .. "', '" .. string.tohex('српски') .. "', '" .. string.tohex('Србија') .. "');\n")
+fo1:write("insert into win_localetable values ('sr-Latn-YU', 'sr_YU@latin', 'wxLANGUAGE_SERBIAN_LATIN_YU', '-', '-', 'Serbian (Latin)', 'Serbian (Latin)', '', 'Latn;', '', '', '', 'LTR', '0x1a', '0x02', 'srpski (latinica)', 'srpski', 'Srbija', 'srpski (latinica)', 'srpski', 'Srbija');\n")
+fo1:write("insert into win_localetable values ('sh', 'sh', 'wxLANGUAGE_SERBO_CROATIAN', '-', '-', 'Serbo-Croatian', 'Serbo-Croatian', '', '', '', '', '', 'LTR', '-', '-', 'srpskohrvatski', 'srpskohrvatski', '', 'srpskohrvatski', 'srpskohrvatski', '');\n")
+
+fo1:write("insert into win_localetable values ('tl-PH', 'tl_PH', 'wxLANGUAGE_TAGALOG', '-', '-', 'Tagalog', 'Tagalog', 'Philippines', '', '', '', '', 'LTR', '-', '-', 'Tagalog', 'Tagalog', 'Pilipinas ', 'Tagalog', 'Tagalog', 'Pilipinas');\n")
+fo1:write("insert into win_localetable values ('tw', 'tw', 'wxLANGUAGE_TWI', '-', '-', 'Twi', 'Twi', '', '', '', '', '', 'LTR', '-', '-', 'Twi', 'Twi', 'Ghana', 'Twi', 'Twi', 'Ghana');\n")
+fo1:write("insert into win_localetable values ('za', 'za', 'wxLANGUAGE_ZHUANG', '-', '-', 'Zhuang', 'Zhuang', '', '', '', '', '', 'LTR', '-', '-', 'Zhuang', 'Zhuang', 'China', 'Zhuang', 'Zhuang', 'China');\n")
+
+-- Update count to reflect manual entries (needs to be updated, if entries are added or removed)
+count = count + 23
+print("Number of locales=" .. count)
+
+fo1:write('commit;\n')
+fo1:close()
+
+fo2:close()
--- a/misc/languages/data/wx/wx_loadlangtabl.lua
+++ b/misc/languages/data/wx/wx_loadlangtabl.lua
@ -0,0 +1,25 @@
+-- Assign script arguments and check validity
+codeFileName = "temp/langtabl_current.txt"
+
+function trim(s)
+   return s:match'^%s*(.*%S)' or ''
+end
+
+fo2 = io.open('temp/wx_loadlangtabl.sql','w')
+fo2:write("create table if not exists wx_langmap (wxlangname char, wxlangid char, wxlocregion, wxversion char, primary key (wxlangname));\n")
+fo2:write('delete from wx_langmap;\nbegin;\n')
+
+count = 0
+for line in io.lines(codeFileName) do
+  wxlangname = trim(string.sub(line,1,55))
+  wxversion = trim(string.sub(line,57,65))
+  wxlangid =  trim(string.sub(line,82,106))
+  wxlocregion =  trim(string.sub(line,108,121))
+
+  fo2:write("insert into wx_langmap values ('" .. wxlangname .. "', '" .. wxlangid .. "', '" .. wxlocregion .. "', '" .. wxversion .. "');\n")
+  count = count + 1
+end
+print("Number of languages =" .. count)
+
+fo2:write('commit;\n')
+fo2:close()
--- a/misc/languages/data/wx/wx_loadsynonymtabl.lua
+++ b/misc/languages/data/wx/wx_loadsynonymtabl.lua
@ -0,0 +1,45 @@
+-- Assign script arguments and check validity
+codeFileName = "temp/synonymtabl_current.txt"
+
+function trim(s)
+   return s:match'^%s*(.*%S)' or ''
+end
+
+function split(str, character)
+  result = {}
+
+  index = 1
+-- (.-),
+--  for s in string.gmatch(str, "[^"..character.."]+") do
+  for s in string.gmatch(str .. character, "(.-)"..character) do
+    result[index] = s
+    index = index + 1
+  end
+
+  return result
+end
+
+fo2 = io.open('temp/wx_loadsynonymtabl.sql','w')
+fo2:write('-- Known syonyms in wx\n-- Date: ' .. os.date("%Y-%m-%d %H:%M") .. '\n\n')
+fo2:write("create table if not exists wx_synonyms (wxnamesyn char, wxnameprim char, winlocid char, wxversion char, primary key (wxnamesyn));\n")
+fo2:write('delete from wx_synonyms;\nbegin;\n')
+
+-- Code|N°|English Name|Nom français|Alias|Age|Date
+
+count = 0
+for line in io.lines(codeFileName) do
+  rem = string.sub(line,1,1)
+  if rem ~= "#" then
+    wxnamesyn = trim(string.sub(line,1,55))
+    wxnameprim = trim(string.sub(line,57,111))
+    winlocid =  trim(string.sub(line,113,126))
+    wxversion =  trim(string.sub(line,128,136))
+
+    fo2:write("insert into wx_synonyms values ('" .. wxnamesyn .. "', '" .. wxnameprim .. "', '" .. winlocid .. "', '" .. wxversion .. "');\n")
+    count = count + 1
+  end
+end
+print("Number of synonyms=" .. count)
+
+fo2:write('commit;\n')
+fo2:close()
--- a/misc/languages/data/wx/wx_readversion.lua
+++ b/misc/languages/data/wx/wx_readversion.lua
@ -0,0 +1,32 @@
+-- Retrieve current wxWidgets version
+codeFileName = "../../../include/wx/version.h"
+
+function trim(s)
+   return s:match'^%s*(.*%S)' or ''
+end
+
+fo2 = io.open('temp/wx_loadversion.sql','w')
+fo2:write('-- Current wxWidgets version\n-- Date: ' .. os.date("%Y-%m-%d %H:%M") .. '\n\n')
+fo2:write("create table if not exists wx_version (version char, primary key (version));\n")
+fo2:write('delete from wx_version;\nbegin;\n')
+
+count = 0
+for line in io.lines(codeFileName) do
+  vkey = trim(string.sub(line,1,28))
+  vval = trim(string.sub(line,30,35))
+  if vkey == "#define wxMAJOR_VERSION" then
+    vmajor = vval
+  elseif vkey == "#define wxMINOR_VERSION" then
+    vminor = vval
+  elseif vkey == "#define wxRELEASE_NUMBER" then
+    vrelno = vval
+  elseif vkey == "#define wxSUBRELEASE_NUMBER" then
+    vsubno = vval
+  end
+end
+wxversion = vmajor .. "." .. vminor .. "." .. vrelno
+print("wx version number=" .. wxversion)
+fo2:write("insert into wx_version values('" .. wxversion .. "');\n")
+
+fo2:write('commit;\n')
+fo2:close()
--- a/misc/languages/genlang.py
+++ b/misc/languages/genlang.py
@ -24,6 +24,19 @@ def ReadScriptTable():
    f.close()
    return scripttable

+def ReadSynonymTable():
+    synonymtable = []
+    try:
+        f = open('misc/languages/synonymtabl.txt')
+    except:
+        print("Did you run the script from top-level wxWidgets directory?")
+        raise
+
+    for i in f.readlines():
+        ispl = i.split()
+        synonymtable.append((ispl[0], ispl[1], ispl[2], ispl[3]))
+    f.close()
+    return synonymtable

 def ReadTable():
    table = []
@ -35,19 +48,14 @@ def ReadTable():

    for i in f.readlines():
        ispl = i.split()
-        table.append((ispl[0], ispl[1], ispl[2], ispl[3], ispl[4], ispl[5], ispl[6], ' '.join(ispl[7:])))
+        table.append((ispl[0], ispl[1], ispl[2], ispl[3], ispl[4], ispl[5], ispl[6], ispl[7], ' '.join(ispl[8:])))
    f.close()
    return table


-def WriteEnum(f, table, scripttable):
+# Kind may be "include" or "interface".
+def WriteEnum(f, table, synonymtable, scripttable, kind = 'include'):
   f.write("""
-/**
-    The languages supported by wxLocale.
-
-    This enum is generated by misc/languages/genlang.py
-    When making changes, please put them into misc/languages/langtabl.txt
-*/
 enum wxLanguage
 {
    /// User's default/preferred language as got from OS.
@ -58,45 +66,46 @@ enum wxLanguage

 """);
   knownLangs = []
+   output = ''
   for i in table:
-       if i[0] not in knownLangs:
-          f.write('    %s,\n' % i[0])
-          knownLangs.append(i[0])
-   f.write("""
+       lang = i[0]
+       wxver = i[1]
+       if lang not in knownLangs:
+          output += '    %s,' % lang
+          if kind == 'interface' and wxver != '-':
+              output += '%s///< @since_wx{%s}' % (' ' * (56 - len(lang)), wxver)
+          output += '\n'
+          knownLangs.append(lang)
+   output += """
    /// For custom, user-defined languages.
    wxLANGUAGE_USER_DEFINED,
+"""

-    /// Synonyms.
-    wxLANGUAGE_AZERI = wxLANGUAGE_AZERBAIJANI,
-    wxLANGUAGE_AZERI_CYRILLIC = wxLANGUAGE_AZERBAIJANI_CYRILLIC,
-    wxLANGUAGE_AZERI_LATIN = wxLANGUAGE_AZERBAIJANI_LATIN,
-    wxLANGUAGE_BENGALI = wxLANGUAGE_BANGLA,
-    wxLANGUAGE_BENGALI_BANGLADESH = wxLANGUAGE_BANGLA_BANGLADESH,
-    wxLANGUAGE_BENGALI_INDIA = wxLANGUAGE_BANGLA_INDIA,
-    wxLANGUAGE_BHUTANI = wxLANGUAGE_DZONGKHA,
-    wxLANGUAGE_CHINESE_SIMPLIFIED = wxLANGUAGE_CHINESE_CHINA,
-    wxLANGUAGE_CHINESE_TRADITIONAL = wxLANGUAGE_CHINESE_TAIWAN,
-    wxLANGUAGE_CHINESE_MACAU = wxLANGUAGE_CHINESE_MACAO,
-    wxLANGUAGE_KANURI_NIGERIA = wxLANGUAGE_KANURI_LATIN_NIGERIA,
-    wxLANGUAGE_KASHMIRI_PERSO_ARABIC = wxLANGUAGE_KASHMIRI_ARABIC,
-    wxLANGUAGE_KASHMIRI_PERSO_ARABIC_INDIA = wxLANGUAGE_KASHMIRI_ARABIC_INDIA,
-    wxLANGUAGE_KERNEWEK = wxLANGUAGE_CORNISH,
-    wxLANGUAGE_KURDISH_PERSO_ARABIC_IRAN  = wxLANGUAGE_KURDISH_ARABIC_IRAN,
-    wxLANGUAGE_MALAY_BRUNEI_DARUSSALAM = wxLANGUAGE_MALAY_BRUNEI,
-    wxLANGUAGE_ORIYA = wxLANGUAGE_ODIA,
-    wxLANGUAGE_ORIYA_INDIA = wxLANGUAGE_ODIA_INDIA,
-    wxLANGUAGE_SPANISH_MODERN = wxLANGUAGE_SPANISH,
-    wxLANGUAGE_UZBEK_PERSO_ARABIC = wxLANGUAGE_UZBEK_ARABIC,
-    wxLANGUAGE_UZBEK_PERSO_ARABIC_AFGHANISTAN = wxLANGUAGE_UZBEK_ARABIC_AFGHANISTAN,
+   if kind == 'include':
+      output += '\n    /// Synonyms.'

-    /// Obsolete synonym.
-    wxLANGUAGE_CAMBODIAN = wxLANGUAGE_KHMER
-};
+   output += '\n'

-""")
+   for i in synonymtable:
+      lang = i[0]
+      synonym = i[1]
+      wxver = i[3]
+      output += '    %s' % lang
+      if kind == 'include':
+         output += ' = %s,\n' % synonym
+      elif kind == 'interface':
+         if wxver != '-':
+           output += ',%s///< Synonym for %s. @since_wx{%s}\n' % (' ' * (42 - len(lang)), synonym, wxver)
+         else:
+           output += ',%s///< Synonym for %s.\n' % (' ' * (42 - len(lang)), synonym)
+      else:
+        print("Unknown kind of generated enum")
+        raise

+   output += '};\n\n'
+   f.write(output)

-def WriteTable(f, table, scripttable):
+def WriteTable(f, table, synonymtable, scripttable):
   sctable = ''
   for i in scripttable:
       scname = '"%s"' % i[0]
@ -106,23 +115,23 @@ def WriteTable(f, table, scripttable):
   lngtable = ''

   for i in table:
-       ibcp47 = '"%s"' % i[1]
-       ican = '"%s"' % i[2]
+       ibcp47 = '"%s"' % i[2]
+       ican = '"%s"' % i[3]
       if ican == '"-"': ican = '""'
-       icanbase = '"%s"' % i[3]
+       icanbase = '"%s"' % i[4]
       if icanbase == '"-"': icanbase = '""'
-       ilang = i[4]
+       ilang = i[5]
       if ilang == '-': ilang = '0'
-       isublang = i[5]
+       isublang = i[6]
       if isublang == '-': isublang = '0'
-       if (i[6] == "LTR") :
+       if (i[7] == "LTR") :
           ilayout = "wxLayout_LeftToRight"
-       elif (i[6] == "RTL"):
+       elif (i[7] == "RTL"):
           ilayout = "wxLayout_RightToLeft"
       else:
           print("ERROR: Invalid value for the layout direction")
       lngtable += '    { %-60s %-17s, %-28s, %-15s, %-4s, %-4s, %s, %s },\n' % \
-                     ((i[0]+','), ibcp47, ican, icanbase, ilang, isublang, ilayout, i[7])
+                     ((i[0]+','), ibcp47, ican, icanbase, ilang, isublang, ilayout, i[8])

   f.write("""
 // The following data tables are generated by misc/languages/genlang.py
@ -221,7 +230,7 @@ def ReplaceGeneratedPartOfFile(fname, func):
                print('Unexpected starting comment.')
            betweenBeginAndEnd = 1
            fout.write(l)
-            func(fout, table, scripttable)
+            func(fout, table, synonymtable, scripttable)
        elif l == '// --- --- --- generated code ends here --- --- ---\n':
            if not betweenBeginAndEnd:
                print('End comment found before the starting one?')
@ -245,6 +254,7 @@ def ReplaceGeneratedPartOfFile(fname, func):

 table = ReadTable()
 scripttable = ReadScriptTable()
+synonymtable = ReadSynonymTable()
 ReplaceGeneratedPartOfFile('include/wx/language.h', WriteEnum)
-ReplaceGeneratedPartOfFile('interface/wx/language.h', WriteEnum)
+ReplaceGeneratedPartOfFile('interface/wx/language.h', lambda f, table, synonymtable, scripttable: WriteEnum(f, table, synonymtable, scripttable, 'interface'))
 ReplaceGeneratedPartOfFile('src/common/languageinfo.cpp', WriteTable)
--- a/misc/languages/langtabl.txt
+++ b/misc/languages/langtabl.txt
--- a/misc/languages/scripttabl.txt
+++ b/misc/languages/scripttabl.txt
@ -23,6 +23,7 @@ Cham cham
 Cher cherokee
 Chrs chorasmian
 Copt coptic
+Cpmn cypro_minoan
 Cprt cypriot
 Cyrl cyrillic
 Deva devanagari
@ -103,6 +104,7 @@ Orkh old_turkic
 Orya oriya
 Osge osage
 Osma osmanya
+Ougr old_uyghur
 Palm palmyrene
 Pauc pau_cin_hau
 Perm old_permic
@ -145,8 +147,11 @@ Thaa thaana
 Thai thai
 Tibt tibetan
 Tirh tirhuta
+Tnsa tangsa
+Toto toto
 Ugar ugaritic
 Vaii vai
+Vith vithkuqi
 Wara warang_citi
 Wcho wancho
 Xpeo old_persian
--- a/misc/languages/synonymtabl.txt
+++ b/misc/languages/synonymtabl.txt
@ -0,0 +1,24 @@
+wxLANGUAGE_AZERI                                        wxLANGUAGE_AZERBAIJANI                                  az             -
+wxLANGUAGE_AZERI_CYRILLIC                               wxLANGUAGE_AZERBAIJANI_CYRILLIC                         az-Cyrl        -
+wxLANGUAGE_AZERI_LATIN                                  wxLANGUAGE_AZERBAIJANI_LATIN                            az-Latn        -
+wxLANGUAGE_BENGALI                                      wxLANGUAGE_BANGLA                                       bn             -
+wxLANGUAGE_BENGALI_BANGLADESH                           wxLANGUAGE_BANGLA_BANGLADESH                            bn-BD          -
+wxLANGUAGE_BENGALI_INDIA                                wxLANGUAGE_BANGLA_INDIA                                 bn-IN          -
+wxLANGUAGE_BHUTANI                                      wxLANGUAGE_DZONGKHA                                     dz             -
+wxLANGUAGE_CAMBODIAN                                    wxLANGUAGE_KHMER                                        km             -
+wxLANGUAGE_CHINESE_MACAU                                wxLANGUAGE_CHINESE_MACAO                                zh-MO          -
+wxLANGUAGE_CHINESE_SIMPLIFIED                           wxLANGUAGE_CHINESE_CHINA                                zh-CN          -
+wxLANGUAGE_CHINESE_TRADITIONAL                          wxLANGUAGE_CHINESE_TAIWAN                               zh-TW          -
+wxLANGUAGE_FARSI                                        wxLANGUAGE_PERSIAN                                      fa             -
+wxLANGUAGE_GREENLANDIC                                  wxLANGUAGE_KALAALLISUT_GREENLAND                        kl_GL          -
+wxLANGUAGE_KANURI_NIGERIA                               wxLANGUAGE_KANURI_LATIN_NIGERIA                         kr-Latn-NG     -
+wxLANGUAGE_KASHMIRI_PERSO_ARABIC                        wxLANGUAGE_KASHMIRI_ARABIC                              ks-Arab        3.1.6
+wxLANGUAGE_KASHMIRI_PERSO_ARABIC_INDIA                  wxLANGUAGE_KASHMIRI_ARABIC_INDIA                        ks-Arab-IN     3.1.6
+wxLANGUAGE_KERNEWEK                                     wxLANGUAGE_CORNISH                                      kw             -
+wxLANGUAGE_KURDISH_PERSO_ARABIC_IRAN                    wxLANGUAGE_KURDISH_ARABIC_IRAN                          ku-Arab-IR     -
+wxLANGUAGE_MALAY_BRUNEI_DARUSSALAM                      wxLANGUAGE_MALAY_BRUNEI                                 ms-BN          -
+wxLANGUAGE_ORIYA                                        wxLANGUAGE_ODIA                                         or             -
+wxLANGUAGE_ORIYA_INDIA                                  wxLANGUAGE_ODIA_INDIA                                   or-IN          -
+wxLANGUAGE_SPANISH_MODERN                               wxLANGUAGE_SPANISH                                      es             -
+wxLANGUAGE_UZBEK_PERSO_ARABIC                           wxLANGUAGE_UZBEK_ARABIC                                 uz-Arab        3.1.6
+wxLANGUAGE_UZBEK_PERSO_ARABIC_AFGHANISTAN               wxLANGUAGE_UZBEK_ARABIC_AFGHANISTAN                     uz-Arab-AF     3.1.6
--- a/misc/languages/util/showlocales.c
+++ b/misc/languages/util/showlocales.c
@ -0,0 +1,81 @@
+#include <windows.h>
+#include <stdio.h>
+
+// All of the LCTYPES new to Windows Vista
+LCTYPE winLocaleTypes[] =
+{
+  LOCALE_SNAME,                   // Tag identifying the locale
+  LOCALE_SENGLISHDISPLAYNAME,     // English locale name (example: "German (Germany)"
+  LOCALE_SENGLISHLANGUAGENAME,    // English language name
+  LOCALE_SENGLISHCOUNTRYNAME,     // English country name
+  LOCALE_SSCRIPTS,                // List of scripts
+  LOCALE_SISO3166CTRYNAME2,
+  LOCALE_SINTLSYMBOL,             // Currency symbol ISO
+  LOCALE_IREADINGLAYOUT,          // Layout direction 0=LTR, 1=RTL, 2=TTB-RTL or LTR, 3=TTB-LTR
+  LOCALE_ILANGUAGE,               // Numeric language code
+  LOCALE_SNATIVEDISPLAYNAME,      // Native locale name (example: "Deutsch (Deutschland)"
+  LOCALE_SNATIVELANGUAGENAME,     // Native language name
+  LOCALE_SNATIVECOUNTRYNAME       // Native country name
+};
+
+// Strings so we can print out the LCTYPES
+LPCWSTR winLocaleTypeNames[] =
+{
+  L"LOCALE_SNAME",                   // Tag identifying the locale
+  L"LOCALE_SENGLISHDISPLAYNAME",     // English locale name (example: "German (Germany)"
+  L"LOCALE_SENGLISHLANGUAGENAME",    // English language name
+  L"LOCALE_SENGLISHCOUNTRYNAME",     // English country name
+  L"LOCALE_SSCRIPTS",                // List of scripts
+  L"LOCALE_SISO3166CTRYNAME2",
+  L"LOCALE_SINTLSYMBOL",             // Currency symbol ISO
+  L"LOCALE_IREADINGLAYOUT",          // Layout direction 0=LTR, 1=RTL, 2=TTB-RTL or LTR, 3=TTB-LTR
+  L"LOCALE_ILANGUAGE",               // Numeric language code
+  L"LOCALE_SNATIVEDISPLAYNAME",      // Native locale name (example: "Deutsch (Deutschland)"
+  L"LOCALE_SNATIVELANGUAGENAME",     // Native language name
+  L"LOCALE_SNATIVECOUNTRYNAME"       // Native country name
+};
+
+// Callback for EnumSystemLocalesEx()
+#define BUFFER_SIZE 512
+BOOL CALLBACK MyFuncLocaleEx(LPWSTR pStr, DWORD dwFlags, LPARAM lparam)
+{
+  if (*pStr == 0) return (TRUE);
+
+  UNREFERENCED_PARAMETER(dwFlags);
+  UNREFERENCED_PARAMETER(lparam);
+  WCHAR wcBuffer[BUFFER_SIZE];
+  char utf8Buffer[BUFFER_SIZE*4];
+
+  int iResult;
+  int i;
+
+  // Loop through all specified LCTYPES and do GetLocaleInfoEx on them
+  for (i = 0; i < sizeof(winLocaleTypes) / sizeof(winLocaleTypes[0]); i++)
+  {
+    // Get this LCTYPE result for this locale
+    iResult = GetLocaleInfoEx(pStr, winLocaleTypes[i], wcBuffer, BUFFER_SIZE);
+
+    // If it succeeds, print it out
+    if (i != 0) printf("|");
+    if (iResult > 0)
+    {
+      WideCharToMultiByte(CP_UTF8, 0, wcBuffer, -1, utf8Buffer, BUFFER_SIZE*4, NULL, NULL);
+      printf(utf8Buffer);
+    }
+    else
+    {
+      //wprintf(L"  %s had error %d\n", NewTypeNames[i], GetLastError());
+    }
+  }
+  printf("\n");
+
+  return (TRUE);
+}
+
+int main(int argc, char* argv[])
+{
+  UNREFERENCED_PARAMETER(argc);
+  // Enumerate the known locales and report on them
+  // LOCALE_ALL = LOCALE_WINDOWS | LOCALE_SUPPLEMENTAL | LOCALE_ALTERNATE_SORTS | LOCALE_NEUTRALDATA
+  EnumSystemLocalesEx( MyFuncLocaleEx, LOCALE_WINDOWS, (LPARAM)argv, NULL);
+}