From 76da8aa47b705d2cd9da9aa92a5dffb21456d1b3 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Tue, 11 Apr 2023 15:00:12 +0200 Subject: [PATCH 01/73] Legacy-free Locale Service, API proposal --- doc/locale_api.md | 95 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 doc/locale_api.md diff --git a/doc/locale_api.md b/doc/locale_api.md new file mode 100644 index 0000000000..398b80591e --- /dev/null +++ b/doc/locale_api.md @@ -0,0 +1,95 @@ +# Legacy-free Locale Service + +> D-Installer currently have a separate Language service, although it's rather +> simplistic. It just allows to set the language of the installed system using +> Yast::Language.Set. And it's quite memory demanding for such an unimpressive +> task. + +> That service would be a nice candidate to be rewritten from scratch with no +> dependencies on YaST or Ruby. It's small enough and could give us a good +> overview on how much can we save. + +Plan: +1. take the systemd APIs as a sensible starting point. +2. deviate only where we add value + +### Language and Keyboard + +#### Systemd + +The systemd API for Locale(Language) and Keyboard is this: +(where the last boolean means Interactive, and the other boolean means Convert) + +``` +org.freedesktop.locale1 service +/org/freedesktop/locale1 object +NAME TYPE SIG RESULT/VALUE +org.freedesktop.locale1 interface - - +.SetLocale method asb - +.SetVConsoleKeyboard method ssbb - +.SetX11Keyboard method ssssbb- +(all properties are read-only and emit PropertiesChanged) +.Locale property as 1 "LANG=en_US.UTF-8" +.VConsoleKeymap property s "cz-lat2-us" +.VConsoleKeymapToggle property s "" +.X11Layout property s "cz,us" +.X11Model property s "pc105" +.X11Options property s "terminate:ctrl_alt_bksp,grp:shift_togg… +.X11Variant property s "qwerty,basic" +``` + +#### Design + +Elementary Layer (lower) + +- Just use the systemd API, don't add any API of our own + +Proposal Layer (uppper) + +- when setting the locale, adjust the proposed package selection and keyboard + accordingly. And timezone. + +Agama.locale1 interface (on Agama/Language1?) +- is a copy of freedesktop.locale1, + forwarding to the elementary layer, + and SetLocale does the additional work + - Agama...Software...todo(...) + - Agama.locale1.SetVConsoleKeyboard(...) + - Agama.locale1.SetX11Keyboard(...) + - Agama...Timezone...todo(...) + +### Timezone + +#### Systemd + +``` +$ busctl --system introspect org.freedesktop.timedate1 /org/freedesktop/timedate1 +NAME TYPE SIG RESULT/VALUE FLAGS +... +org.freedesktop.timedate1 interface - - - +.ListTimezones method - as - +.SetLocalRTC method bbb - - +.SetNTP method bb - - +.SetTime method xbb - - +.SetTimezone method sb - - +(properties are read only) +.CanNTP property b true - +.LocalRTC property b false emits-change +.NTP property b false emits-change +.NTPSynchronized property b false - +.RTCTimeUSec property t 1681214874000000 - +.TimeUSec property t 1681214874046139 - +.Timezone property s "Europe/Prague" emits-change +``` + +"LocalRTC" means "is the local time zone used for the real time clock", +so it's !hwclock_in_UTC + +#### Design + +we will use +.ListTimezones +.SetLocalRTC +.SetTimezone +.LocalRTC +.Timezone From 6b0e10f48eddcbdbba3e13d3d978f67ce7f1dfa7 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Tue, 11 Apr 2023 15:25:57 +0200 Subject: [PATCH 02/73] better formatting --- doc/locale_api.md | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index 398b80591e..fae827fbf3 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -38,13 +38,11 @@ org.freedesktop.locale1 interface - - .X11Variant property s "qwerty,basic" ``` -#### Design - -Elementary Layer (lower) +#### Design of Elementary Layer (lower) - Just use the systemd API, don't add any API of our own -Proposal Layer (uppper) +#### Design of Proposal Layer (uppper) - when setting the locale, adjust the proposed package selection and keyboard accordingly. And timezone. @@ -85,11 +83,15 @@ org.freedesktop.timedate1 interface - - - "LocalRTC" means "is the local time zone used for the real time clock", so it's !hwclock_in_UTC -#### Design +#### Design of Elementary Layer (lower) + +Just use the systemd API, don't add any API of our own. We will use +- .ListTimezones +- .SetLocalRTC +- .SetTimezone +- .LocalRTC +- .Timezone + +#### Design of Proposal Layer (uppper) -we will use -.ListTimezones -.SetLocalRTC -.SetTimezone -.LocalRTC -.Timezone +None needed? From 23272f5670b5fa1b0aa1ab0076585d5f98d903cf Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 12 Apr 2023 17:47:38 +0200 Subject: [PATCH 03/73] Proposal layer for Agama.Locale1, including Priority --- doc/locale_api.md | 119 ++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 111 insertions(+), 8 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index fae827fbf3..2d224d805f 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -42,20 +42,123 @@ org.freedesktop.locale1 interface - - - Just use the systemd API, don't add any API of our own -#### Design of Proposal Layer (uppper) +#### Design of Proposal Layer (upper) - when setting the locale, adjust the proposed package selection and keyboard accordingly. And timezone. -Agama.locale1 interface (on Agama/Language1?) -- is a copy of freedesktop.locale1, - forwarding to the elementary layer, - and SetLocale does the additional work + + +The general design of the proposal layer is + +- declarative, using read-write properties +- setting some properties will make changes in the proposal layer of other + properties of other objects + +So here, setting `Locale` below will set also `X11Keyboard` here and - Agama...Software...todo(...) - - Agama.locale1.SetVConsoleKeyboard(...) - - Agama.locale1.SetX11Keyboard(...) - Agama...Timezone...todo(...) +For the first version of the API, let's keep things simple: + +**LocaleType** is just one string, the value for the `LANG` variable, like +`"cs_CZ.UTF-8"`. + +**X11KeyboardType** is a pair of strings, the Layout and Variant, for example +`("cz", "qwerty")` or `("us", "basic")`. + +We don't expose the console keyboard, instead letting systemd do it via the +_convert_ parameter. + +(The other systemd keyboard settings are X11Model and X11Options, we don't +have UI or data for that) + +``` +# this is gdbus syntax BTW +node ...Agama/Locale1 { + interface ...Agama.Locale1 { + methods: + # FIXME: the method part is unclear to me. anyone better at proposals? + Commit(); + properties: + # NOTE: "as" has different meaning to systemd, + # we have a list of LANG settings, 1st gets passed to systemd, + # others affect package selection + readwrite as Locale = ['cs_CZ.UTF-8', 'de_DE.UTF-8']; + readonly (ss) X11Keyboard = ('cz','qwerty); + }; +}; +``` + +##### Overriding the User's Choice? + +
+ +If setting the Locale proposes the Language, what do we do if the user first +changes the language and _then_ the locale? + + +When Agama UI first shows up, it may show default choices like: + +> Locale: English (US), Keyboard: US + +Then we change the locale to Czech, and the keyboard is adjusted automatically: + +> Locale: Czech, Keyboard: Czech + +We tune the keyboard: + +> Locale: Czech, Keyboard: Czech (qwerty) + +When we then change the locale, the keyboard could stay the same, as we have +already touched it: + +> Locale: German, Keyboard: Czech (qwerty) +
+ +##### Simple Design: Always Repropose + +We can easily afford throwing away the user's choice of keyboard layout and +simply set what we consider a good default for a newly set locale, because: + +1. it is just one setting (as opposed to whole partitioning layout) +2. the change will be visible in the UI, I assume + +##### Detailed Design: Prioritize + +But other cases may not be as simple, so here's a generic design: + +All settings are wrapped in a `Priority` generic type (an Enum in Rust), +meaning, what is the source and importance of the setting: +- `Machine(data)` means the system has proposed it +- `Human(data)` means the user has made the choice + +In D-Bus, it is represented by wrapping the data in a struct, with a leading +byte* tagging the priority. For ease of recognition when watching bus traffic, +special numbers are used: +- `23` means Human, for the number of chromosome pairs +- `42` means Machine, as the famous Answer was given by Deep Thought, a machine + +In the following dump, we see that the locale was set by the user and the +system has adjusted the keyboard. + +``` +node ...Agama/Locale1 { + interface ...Agama.Locale1 { + properties: + readwrite (yas) Locale = (23, ['cs_CZ.UTF-8', 'de_DE.UTF-8']); + readonly (y(ss)) X11Keyboard = (42, ('cz','qwerty)); + }; +}; +``` + +You may know a [similar settings in libzypp][resstatus] where it has 4 levels. + +*: maybe this is a crazy optimization? I am not too opposed to use strings for +this on the bus. + +[resstatus]: https://github.com/openSUSE/libzypp/blob/d441746c59f063b5d54833bfdebc48829b07feb5/zypp/ResStatus.h#L106 + ### Timezone #### Systemd @@ -92,6 +195,6 @@ Just use the systemd API, don't add any API of our own. We will use - .LocalRTC - .Timezone -#### Design of Proposal Layer (uppper) +#### Design of Proposal Layer (upper) None needed? From bf50605937c092c38507a7ad6896c8b0f704190d Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 12 Apr 2023 17:53:13 +0200 Subject: [PATCH 04/73] fixup, accidental readonly by copy-paste --- doc/locale_api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index 2d224d805f..5b1b5e4543 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -85,7 +85,7 @@ node ...Agama/Locale1 { # we have a list of LANG settings, 1st gets passed to systemd, # others affect package selection readwrite as Locale = ['cs_CZ.UTF-8', 'de_DE.UTF-8']; - readonly (ss) X11Keyboard = ('cz','qwerty); + readwrite (ss) X11Keyboard = ('cz','qwerty); }; }; ``` @@ -147,7 +147,7 @@ node ...Agama/Locale1 { interface ...Agama.Locale1 { properties: readwrite (yas) Locale = (23, ['cs_CZ.UTF-8', 'de_DE.UTF-8']); - readonly (y(ss)) X11Keyboard = (42, ('cz','qwerty)); + readwrite (y(ss)) X11Keyboard = (42, ('cz','qwerty)); }; }; ``` From 1db73f26723f89dc6b045cfd3e0316ceee7082a5 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Thu, 13 Apr 2023 11:27:04 +0200 Subject: [PATCH 05/73] Agama.TimeDate1 proposal layer design, just 2 properties --- doc/locale_api.md | 40 ++++++++++++++++++++++++++++++++-------- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index 5b1b5e4543..673c51f607 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -163,16 +163,28 @@ this on the bus. #### Systemd +(I find `gdbus` verbose output better for methods and `busctl` terse output +better for properties) + ``` +$ gdbus introspect -y -d org.freedesktop.timedate1 -o /org/freedesktop/timedate1 +node /org/freedesktop/timedate1 { … + interface org.freedesktop.timedate1 { … + methods: + SetTime(in x usec_utc, + in b relative, + in b interactive); + SetTimezone(in s timezone, + in b interactive); + SetLocalRTC(in b local_rtc, + in b fix_system, + in b interactive); + SetNTP(in b use_ntp, + in b interactive); + ListTimezones(out as timezones); +… $ busctl --system introspect org.freedesktop.timedate1 /org/freedesktop/timedate1 NAME TYPE SIG RESULT/VALUE FLAGS -... -org.freedesktop.timedate1 interface - - - -.ListTimezones method - as - -.SetLocalRTC method bbb - - -.SetNTP method bb - - -.SetTime method xbb - - -.SetTimezone method sb - - (properties are read only) .CanNTP property b true - .LocalRTC property b false emits-change @@ -197,4 +209,16 @@ Just use the systemd API, don't add any API of our own. We will use #### Design of Proposal Layer (upper) -None needed? +(Using the `Priority` design from above) + +``` +node ...Agama/TimeDate1 { + interface ...Agama.TimeDate1 { + methods: + # FIXME: add relevant methods common to all proposals + properties: + readwrite (ys) Timezone = (42, 'Europe/Prague'); + readwrite (yb) LocalRTC = (42, false); + }; +}; +``` From aaadb765943f6ff78685a5e8aca9276daa0e82da Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 13 Apr 2023 15:39:40 +0200 Subject: [PATCH 06/73] initial commit for locale data --- rust/Cargo.lock | 4 ++++ rust/Cargo.toml | 3 ++- rust/agama-locale-data/Cargo.toml | 8 ++++++++ rust/agama-locale-data/src/lib.rs | 14 ++++++++++++++ 4 files changed, 28 insertions(+), 1 deletion(-) create mode 100644 rust/agama-locale-data/Cargo.toml create mode 100644 rust/agama-locale-data/src/lib.rs diff --git a/rust/Cargo.lock b/rust/Cargo.lock index ad2bd8c210..10391831a4 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -44,6 +44,10 @@ dependencies = [ "zbus", ] +[[package]] +name = "agama-locale-data" +version = "0.1.0" + [[package]] name = "ahash" version = "0.8.3" diff --git a/rust/Cargo.toml b/rust/Cargo.toml index 8df317a4b6..cdd8130779 100644 --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -2,5 +2,6 @@ members = [ "agama-lib", "agama-cli", - "agama-derive" + "agama-derive", + "agama-locale-data" ] diff --git a/rust/agama-locale-data/Cargo.toml b/rust/agama-locale-data/Cargo.toml new file mode 100644 index 0000000000..f3ae25c920 --- /dev/null +++ b/rust/agama-locale-data/Cargo.toml @@ -0,0 +1,8 @@ +[package] +name = "agama-locale-data" +version = "0.1.0" +edition = "2021" + +# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html + +[dependencies] diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs new file mode 100644 index 0000000000..7d12d9af81 --- /dev/null +++ b/rust/agama-locale-data/src/lib.rs @@ -0,0 +1,14 @@ +pub fn add(left: usize, right: usize) -> usize { + left + right +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn it_works() { + let result = add(2, 2); + assert_eq!(result, 4); + } +} From ab47c97fd116b34dd67d59c52c88c032b99dc112 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 13 Apr 2023 20:00:22 +0200 Subject: [PATCH 07/73] add first struct for serde deserialization --- rust/Cargo.lock | 3 +++ rust/agama-locale-data/Cargo.toml | 1 + rust/agama-locale-data/src/lib.rs | 25 +++++++++++++++++++++++++ 3 files changed, 29 insertions(+) diff --git a/rust/Cargo.lock b/rust/Cargo.lock index 10391831a4..a1d0c0c5ef 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -47,6 +47,9 @@ dependencies = [ [[package]] name = "agama-locale-data" version = "0.1.0" +dependencies = [ + "serde", +] [[package]] name = "ahash" diff --git a/rust/agama-locale-data/Cargo.toml b/rust/agama-locale-data/Cargo.toml index f3ae25c920..9371391dd4 100644 --- a/rust/agama-locale-data/Cargo.toml +++ b/rust/agama-locale-data/Cargo.toml @@ -6,3 +6,4 @@ edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] +serde = { version = "1.0.152" } diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 7d12d9af81..2204ad6829 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -1,7 +1,32 @@ +use serde::Deserialize; + pub fn add(left: usize, right: usize) -> usize { left + right } +#[derive(Debug, Deserialize)] +pub struct RankedLanguage { + pub id: String, + pub rank: u8 +} + +#[derive(Debug, Deserialize)] +pub struct RankedTerritory { + #[serde(rename(deserialize = "territoryId"))] + pub id: String, + pub rank: u8 +} + +#[derive(Debug, Deserialize)] +pub struct Keyboard { + pub id: String, + pub description: String, + pub ascii: bool, + pub comment: String, + pub languages: Vec, + pub territories: Vec +} + #[cfg(test)] mod tests { use super::*; From 475a75576d2b16344788bfe23328806aa3c566d6 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 14 Apr 2023 10:43:42 +0200 Subject: [PATCH 08/73] first working code to get list of keyboards --- rust/Cargo.lock | 46 +++++++++++++++++++++++++++++++ rust/agama-locale-data/Cargo.toml | 4 ++- rust/agama-locale-data/src/lib.rs | 45 +++++++++++++++++++++++++----- 3 files changed, 87 insertions(+), 8 deletions(-) diff --git a/rust/Cargo.lock b/rust/Cargo.lock index a1d0c0c5ef..6a0b77365d 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -2,6 +2,12 @@ # It is not intended for manual editing. version = 3 +[[package]] +name = "adler" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f26201604c87b1e01bd3d98f8d5d9a8fcbb815e8cedb41ffccbeb4bf593a35fe" + [[package]] name = "agama-cli" version = "1.0.0" @@ -48,6 +54,8 @@ dependencies = [ name = "agama-locale-data" version = "0.1.0" dependencies = [ + "flate2", + "quick-xml", "serde", ] @@ -404,6 +412,15 @@ dependencies = [ "libc", ] +[[package]] +name = "crc32fast" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b540bd8bc810d3885c6ea91e2018302f68baba2129ab3e88f32389ee9370880d" +dependencies = [ + "cfg-if", +] + [[package]] name = "crossbeam-utils" version = "0.8.15" @@ -577,6 +594,16 @@ dependencies = [ "instant", ] +[[package]] +name = "flate2" +version = "1.0.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a8a2db397cb1c8772f31494cb8917e48cd1e64f0fa7efac59fbd741a0a8ce841" +dependencies = [ + "crc32fast", + "miniz_oxide", +] + [[package]] name = "form_urlencoded" version = "1.1.0" @@ -951,6 +978,15 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a" +[[package]] +name = "miniz_oxide" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b275950c28b37e794e8c55d88aeb5e139d0ce23fdbbeda68f8d7174abdf9e8fa" +dependencies = [ + "adler", +] + [[package]] name = "nix" version = "0.26.2" @@ -1228,6 +1264,16 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "quick-xml" +version = "0.28.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ce5e73202a820a31f8a0ee32ada5e21029c81fd9e3ebf668a40832e4219d9d1" +dependencies = [ + "memchr", + "serde", +] + [[package]] name = "quote" version = "1.0.23" diff --git a/rust/agama-locale-data/Cargo.toml b/rust/agama-locale-data/Cargo.toml index 9371391dd4..ddb5b2b1dd 100644 --- a/rust/agama-locale-data/Cargo.toml +++ b/rust/agama-locale-data/Cargo.toml @@ -6,4 +6,6 @@ edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] -serde = { version = "1.0.152" } +serde = { version = "1.0.152", features = ["derive"] } +quick-xml = { version = "0.28.2", features = ["serialize"] } +flate2 = "1.0.25" \ No newline at end of file diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 2204ad6829..6cb5920b15 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -1,4 +1,8 @@ use serde::Deserialize; +use std::fs::File; +use std::io::BufReader; +use quick_xml::de::Deserializer; +use flate2::bufread::GzDecoder; pub fn add(left: usize, right: usize) -> usize { left + right @@ -6,25 +10,50 @@ pub fn add(left: usize, right: usize) -> usize { #[derive(Debug, Deserialize)] pub struct RankedLanguage { + #[serde(rename(deserialize = "languageId"))] pub id: String, - pub rank: u8 + pub rank: u16 +} + +#[derive(Debug, Deserialize)] +pub struct RankedLanguages { + pub language: Option> } #[derive(Debug, Deserialize)] pub struct RankedTerritory { #[serde(rename(deserialize = "territoryId"))] pub id: String, - pub rank: u8 + pub rank: u16 +} + +#[derive(Debug, Deserialize)] +pub struct RankedTerritories { + pub territory: Option> } #[derive(Debug, Deserialize)] pub struct Keyboard { + #[serde(rename(deserialize = "keyboardId"))] pub id: String, pub description: String, pub ascii: bool, - pub comment: String, - pub languages: Vec, - pub territories: Vec + pub comment: Option, + pub languages: RankedLanguages, + pub territories: RankedTerritories +} + +#[derive(Debug, Deserialize)] +pub struct Keyboards { + pub keyboard: Vec +} + +pub fn get_keyboards() -> Keyboards { + const FILE_PATH: &str = "/usr/share/langtable/data/keyboards.xml.gz"; + let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); + let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let mut deserializer = Deserializer::from_reader(reader); + Keyboards::deserialize(&mut deserializer).expect("Failed to deserialize keyboard entry") } #[cfg(test)] @@ -33,7 +62,9 @@ mod tests { #[test] fn it_works() { - let result = add(2, 2); - assert_eq!(result, 4); + let result = get_keyboards(); + assert_eq!(result.keyboard.len(), 247); + let first = result.keyboard.first().expect("no keyboards"); + assert_eq!(first.id, "ad") } } From c19ca2751bd5eba32e25eb3ee08efbbe5e2a03f6 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 14 Apr 2023 13:32:11 +0200 Subject: [PATCH 09/73] structure code a bit better --- rust/agama-locale-data/src/keyboard.rs | 41 ++++++++++++++++++++++ rust/agama-locale-data/src/lib.rs | 48 ++------------------------ 2 files changed, 44 insertions(+), 45 deletions(-) create mode 100644 rust/agama-locale-data/src/keyboard.rs diff --git a/rust/agama-locale-data/src/keyboard.rs b/rust/agama-locale-data/src/keyboard.rs new file mode 100644 index 0000000000..6d575b774e --- /dev/null +++ b/rust/agama-locale-data/src/keyboard.rs @@ -0,0 +1,41 @@ +use serde::Deserialize; + +#[derive(Debug, Deserialize)] +pub struct RankedLanguage { + #[serde(rename(deserialize = "languageId"))] + pub id: String, + pub rank: u16 +} + +#[derive(Debug, Deserialize)] +pub struct RankedLanguages { + pub language: Option> +} + +#[derive(Debug, Deserialize)] +pub struct RankedTerritory { + #[serde(rename(deserialize = "territoryId"))] + pub id: String, + pub rank: u16 +} + +#[derive(Debug, Deserialize)] +pub struct RankedTerritories { + pub territory: Option> +} + +#[derive(Debug, Deserialize)] +pub struct Keyboard { + #[serde(rename(deserialize = "keyboardId"))] + pub id: String, + pub description: String, + pub ascii: bool, + pub comment: Option, + pub languages: RankedLanguages, + pub territories: RankedTerritories +} + +#[derive(Debug, Deserialize)] +pub struct Keyboards { + pub keyboard: Vec +} diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 6cb5920b15..860db5534c 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -4,56 +4,14 @@ use std::io::BufReader; use quick_xml::de::Deserializer; use flate2::bufread::GzDecoder; -pub fn add(left: usize, right: usize) -> usize { - left + right -} - -#[derive(Debug, Deserialize)] -pub struct RankedLanguage { - #[serde(rename(deserialize = "languageId"))] - pub id: String, - pub rank: u16 -} - -#[derive(Debug, Deserialize)] -pub struct RankedLanguages { - pub language: Option> -} - -#[derive(Debug, Deserialize)] -pub struct RankedTerritory { - #[serde(rename(deserialize = "territoryId"))] - pub id: String, - pub rank: u16 -} - -#[derive(Debug, Deserialize)] -pub struct RankedTerritories { - pub territory: Option> -} - -#[derive(Debug, Deserialize)] -pub struct Keyboard { - #[serde(rename(deserialize = "keyboardId"))] - pub id: String, - pub description: String, - pub ascii: bool, - pub comment: Option, - pub languages: RankedLanguages, - pub territories: RankedTerritories -} - -#[derive(Debug, Deserialize)] -pub struct Keyboards { - pub keyboard: Vec -} +pub mod keyboard; -pub fn get_keyboards() -> Keyboards { +pub fn get_keyboards() -> keyboard::Keyboards { const FILE_PATH: &str = "/usr/share/langtable/data/keyboards.xml.gz"; let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); let mut deserializer = Deserializer::from_reader(reader); - Keyboards::deserialize(&mut deserializer).expect("Failed to deserialize keyboard entry") + keyboard::Keyboards::deserialize(&mut deserializer).expect("Failed to deserialize keyboard entry") } #[cfg(test)] From ae17463b4f8cc832de4d937e3d483d4e4adea530 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Sun, 16 Apr 2023 23:25:27 +0200 Subject: [PATCH 10/73] add listing of languages --- rust/agama-locale-data/src/language.rs | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 rust/agama-locale-data/src/language.rs diff --git a/rust/agama-locale-data/src/language.rs b/rust/agama-locale-data/src/language.rs new file mode 100644 index 0000000000..2223c8704e --- /dev/null +++ b/rust/agama-locale-data/src/language.rs @@ -0,0 +1,15 @@ +use serde::Deserialize; + +use crate::keyboard::RankedTerritories; + +#[derive(Debug, Deserialize)] +pub struct Language { + #[serde(rename(deserialize = "languageId"))] + pub id: String, + pub territories: RankedTerritories +} + +#[derive(Debug, Deserialize)] +pub struct Languages { + pub language: Vec +} \ No newline at end of file From 9b054402919c34bf606963462b895c5dcc21cc93 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Fri, 14 Apr 2023 15:34:12 +0200 Subject: [PATCH 11/73] Switch from X11Keyboard to VConsoleKeyboard --- doc/locale_api.md | 54 +++++++++++++++++++++++++++++++---------------- 1 file changed, 36 insertions(+), 18 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index 673c51f607..def2e8760f 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -18,16 +18,26 @@ Plan: #### Systemd The systemd API for Locale(Language) and Keyboard is this: -(where the last boolean means Interactive, and the other boolean means Convert) ``` -org.freedesktop.locale1 service -/org/freedesktop/locale1 object -NAME TYPE SIG RESULT/VALUE -org.freedesktop.locale1 interface - - -.SetLocale method asb - -.SetVConsoleKeyboard method ssbb - -.SetX11Keyboard method ssssbb- +$ gdbus introspect -y -d org.freedesktop.locale1 -o /org/freedesktop/locale1 +node /org/freedesktop/locale1 { + interface org.freedesktop.locale1 { + methods: + SetLocale(in as locale, + in b interactive); + SetVConsoleKeyboard(in s keymap, + in s keymap_toggle, + in b convert, + in b interactive); + SetX11Keyboard(in s layout, + in s model, + in s variant, + in s options, + in b convert, + in b interactive); +… +$ busctl --system introspect org.freedesktop.locale1 /org/freedesktop/locale1 (all properties are read-only and emit PropertiesChanged) .Locale property as 1 "LANG=en_US.UTF-8" .VConsoleKeymap property s "cz-lat2-us" @@ -40,22 +50,30 @@ org.freedesktop.locale1 interface - - #### Design of Elementary Layer (lower) -- Just use the systemd API, don't add any API of our own +The plan was: + +> Just use the systemd API, don't add any API of our own + +But the problem with that is the installer runs in one system (inst-sys, `/`) +and operates on another (target, `/mnt`) and we cannot use the full systemd +API. Instead we rely on `systemd-firstboot`. + +`systemd-firstboot` only has an option for the console keymap, but we have a +way to propagate it to X11, see [bsc#1046436](https://bugzilla.suse.com/show_bug.cgi?id=1046436) + #### Design of Proposal Layer (upper) - when setting the locale, adjust the proposed package selection and keyboard accordingly. And timezone. - - The general design of the proposal layer is - declarative, using read-write properties - setting some properties will make changes in the proposal layer of other properties of other objects -So here, setting `Locale` below will set also `X11Keyboard` here and +So here, setting `Locale` below will set also `VConsoleKeyboard` here and - Agama...Software...todo(...) - Agama...Timezone...todo(...) @@ -64,10 +82,10 @@ For the first version of the API, let's keep things simple: **LocaleType** is just one string, the value for the `LANG` variable, like `"cs_CZ.UTF-8"`. -**X11KeyboardType** is a pair of strings, the Layout and Variant, for example -`("cz", "qwerty")` or `("us", "basic")`. +**VConsoleKeyboardType** is a string, for example +`"cz-qwerty"` or `"us"`. -We don't expose the console keyboard, instead letting systemd do it via the +We don't expose the X11 keyboard, instead letting systemd do it via the _convert_ parameter. (The other systemd keyboard settings are X11Model and X11Options, we don't @@ -85,7 +103,7 @@ node ...Agama/Locale1 { # we have a list of LANG settings, 1st gets passed to systemd, # others affect package selection readwrite as Locale = ['cs_CZ.UTF-8', 'de_DE.UTF-8']; - readwrite (ss) X11Keyboard = ('cz','qwerty); + readwrite s VConsoleKeyboard = 'cz-qwerty'; }; }; ``` @@ -94,8 +112,8 @@ node ...Agama/Locale1 {
-If setting the Locale proposes the Language, what do we do if the user first -changes the language and _then_ the locale? +If setting the locale proposes the keyboard, what do we do if the user first +changes the keyboard and _then_ the locale? When Agama UI first shows up, it may show default choices like: From 28bec0dd00ee81aa703608c29978df45d159374c Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Mon, 17 Apr 2023 22:09:22 +0200 Subject: [PATCH 12/73] add terrioty call and reafctor ranked lists --- rust/agama-locale-data/src/keyboard.rs | 24 +--------------- rust/agama-locale-data/src/language.rs | 2 +- rust/agama-locale-data/src/lib.rs | 37 ++++++++++++++++++++++++- rust/agama-locale-data/src/ranked.rs | 26 +++++++++++++++++ rust/agama-locale-data/src/territory.rs | 13 +++++++++ 5 files changed, 77 insertions(+), 25 deletions(-) create mode 100644 rust/agama-locale-data/src/ranked.rs create mode 100644 rust/agama-locale-data/src/territory.rs diff --git a/rust/agama-locale-data/src/keyboard.rs b/rust/agama-locale-data/src/keyboard.rs index 6d575b774e..e7d3d7b8c7 100644 --- a/rust/agama-locale-data/src/keyboard.rs +++ b/rust/agama-locale-data/src/keyboard.rs @@ -1,28 +1,6 @@ use serde::Deserialize; -#[derive(Debug, Deserialize)] -pub struct RankedLanguage { - #[serde(rename(deserialize = "languageId"))] - pub id: String, - pub rank: u16 -} - -#[derive(Debug, Deserialize)] -pub struct RankedLanguages { - pub language: Option> -} - -#[derive(Debug, Deserialize)] -pub struct RankedTerritory { - #[serde(rename(deserialize = "territoryId"))] - pub id: String, - pub rank: u16 -} - -#[derive(Debug, Deserialize)] -pub struct RankedTerritories { - pub territory: Option> -} +use crate::ranked::{RankedTerritories, RankedLanguages}; #[derive(Debug, Deserialize)] pub struct Keyboard { diff --git a/rust/agama-locale-data/src/language.rs b/rust/agama-locale-data/src/language.rs index 2223c8704e..04801d2082 100644 --- a/rust/agama-locale-data/src/language.rs +++ b/rust/agama-locale-data/src/language.rs @@ -1,6 +1,6 @@ use serde::Deserialize; -use crate::keyboard::RankedTerritories; +use crate::ranked::RankedTerritories; #[derive(Debug, Deserialize)] pub struct Language { diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 860db5534c..58aaca82fb 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -5,6 +5,9 @@ use quick_xml::de::Deserializer; use flate2::bufread::GzDecoder; pub mod keyboard; +pub mod language; +pub mod territory; +pub mod ranked; pub fn get_keyboards() -> keyboard::Keyboards { const FILE_PATH: &str = "/usr/share/langtable/data/keyboards.xml.gz"; @@ -14,15 +17,47 @@ pub fn get_keyboards() -> keyboard::Keyboards { keyboard::Keyboards::deserialize(&mut deserializer).expect("Failed to deserialize keyboard entry") } +pub fn get_languages() -> language::Languages { + const FILE_PATH: &str = "/usr/share/langtable/data/languages.xml.gz"; + let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); + let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let mut deserializer = Deserializer::from_reader(reader); + language::Languages::deserialize(&mut deserializer).expect("Failed to deserialize language entry") +} + +pub fn get_territories() -> territory::Territories { + const FILE_PATH: &str = "/usr/share/langtable/data/territories.xml.gz"; + let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); + let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let mut deserializer = Deserializer::from_reader(reader); + territory::Territories::deserialize(&mut deserializer).expect("Failed to deserialize territory entry") +} + #[cfg(test)] mod tests { use super::*; #[test] - fn it_works() { + fn test_get_keyboards() { let result = get_keyboards(); assert_eq!(result.keyboard.len(), 247); let first = result.keyboard.first().expect("no keyboards"); assert_eq!(first.id, "ad") } + + #[test] + fn test_get_languages() { + let result = get_languages(); + assert_eq!(result.language.len(), 356); + let first = result.language.first().expect("no keyboards"); + assert_eq!(first.id, "aa") + } + + #[test] + fn test_get_territories() { + let result = get_territories(); + assert_eq!(result.territory.len(), 257); + let first = result.territory.first().expect("no keyboards"); + assert_eq!(first.id, "001") // looks strange, but it is meta id for whole world + } } diff --git a/rust/agama-locale-data/src/ranked.rs b/rust/agama-locale-data/src/ranked.rs new file mode 100644 index 0000000000..82b307d8c1 --- /dev/null +++ b/rust/agama-locale-data/src/ranked.rs @@ -0,0 +1,26 @@ +use serde::Deserialize; + + +#[derive(Debug, Deserialize)] +pub struct RankedLanguage { + #[serde(rename(deserialize = "languageId"))] + pub id: String, + pub rank: u16 +} + +#[derive(Debug, Deserialize)] +pub struct RankedLanguages { + pub language: Option> +} + +#[derive(Debug, Deserialize)] +pub struct RankedTerritory { + #[serde(rename(deserialize = "territoryId"))] + pub id: String, + pub rank: u16 +} + +#[derive(Debug, Deserialize)] +pub struct RankedTerritories { + pub territory: Option> +} \ No newline at end of file diff --git a/rust/agama-locale-data/src/territory.rs b/rust/agama-locale-data/src/territory.rs new file mode 100644 index 0000000000..ed752456f1 --- /dev/null +++ b/rust/agama-locale-data/src/territory.rs @@ -0,0 +1,13 @@ +use serde::Deserialize; + +#[derive(Debug, Deserialize)] +pub struct Territory { + #[serde(rename(deserialize = "territoryId"))] + pub id: String, + pub languages: crate::ranked::RankedLanguages +} + +#[derive(Debug, Deserialize)] +pub struct Territories { + pub territory: Vec +} \ No newline at end of file From d6f3fcd89c135fec22aaab534282480a4fe95369 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 18 Apr 2023 10:35:38 +0200 Subject: [PATCH 13/73] add timezone parts --- rust/agama-locale-data/src/lib.rs | 19 ++++++++++++++++++- rust/agama-locale-data/src/timezone_part.rs | 15 +++++++++++++++ 2 files changed, 33 insertions(+), 1 deletion(-) create mode 100644 rust/agama-locale-data/src/timezone_part.rs diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 58aaca82fb..d26804e919 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -7,6 +7,7 @@ use flate2::bufread::GzDecoder; pub mod keyboard; pub mod language; pub mod territory; +pub mod timezone_part; pub mod ranked; pub fn get_keyboards() -> keyboard::Keyboards { @@ -33,6 +34,14 @@ pub fn get_territories() -> territory::Territories { territory::Territories::deserialize(&mut deserializer).expect("Failed to deserialize territory entry") } +pub fn get_timezone_parts() -> timezone_part::TimezoneIdParts { + const FILE_PATH: &str = "/usr/share/langtable/data/timezoneidparts.xml.gz"; + let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); + let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let mut deserializer = Deserializer::from_reader(reader); + timezone_part::TimezoneIdParts::deserialize(&mut deserializer).expect("Failed to deserialize timezone part entry") +} + #[cfg(test)] mod tests { use super::*; @@ -60,4 +69,12 @@ mod tests { let first = result.territory.first().expect("no keyboards"); assert_eq!(first.id, "001") // looks strange, but it is meta id for whole world } -} + + #[test] + fn test_get_timezone_parts() { + let result = get_timezone_parts(); + assert_eq!(result.timezone_part.len(), 441); + let first = result.timezone_part.first().expect("no keyboards"); + assert_eq!(first.id, "Abidjan") + } +} \ No newline at end of file diff --git a/rust/agama-locale-data/src/timezone_part.rs b/rust/agama-locale-data/src/timezone_part.rs new file mode 100644 index 0000000000..beaefa0981 --- /dev/null +++ b/rust/agama-locale-data/src/timezone_part.rs @@ -0,0 +1,15 @@ +use serde::Deserialize; + +#[derive(Debug, Deserialize)] +pub struct TimezoneIdPart { + #[serde(rename(deserialize = "timezoneIdPartId"))] + pub id: String +} + +// Timezone id parts are useful mainly for localization of timezones +// Just search each part of timezone for translation +#[derive(Debug, Deserialize)] +pub struct TimezoneIdParts { + #[serde(rename(deserialize = "timezoneIdPart"))] + pub timezone_part: Vec +} \ No newline at end of file From db43ccf1d0b862139f888a32b15b3e585880f266 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 18 Apr 2023 17:18:22 +0200 Subject: [PATCH 14/73] Adds method to get all timezones --- rust/Cargo.lock | 86 +++++++++++++++++++++++++++++++ rust/agama-locale-data/Cargo.toml | 3 +- rust/agama-locale-data/src/lib.rs | 13 +++++ 3 files changed, 101 insertions(+), 1 deletion(-) diff --git a/rust/Cargo.lock b/rust/Cargo.lock index 6a0b77365d..0d2109c75b 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -54,6 +54,7 @@ dependencies = [ name = "agama-locale-data" version = "0.1.0" dependencies = [ + "chrono-tz", "flate2", "quick-xml", "serde", @@ -335,6 +336,38 @@ version = "1.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd" +[[package]] +name = "chrono" +version = "0.4.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e3c5919066adf22df73762e50cffcde3a758f2a848b113b586d1f86728b673b" +dependencies = [ + "num-integer", + "num-traits", +] + +[[package]] +name = "chrono-tz" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf9cc2b23599e6d7479755f3594285efb3f74a1bdca7a7374948bc831e23a552" +dependencies = [ + "chrono", + "chrono-tz-build", + "phf", +] + +[[package]] +name = "chrono-tz-build" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9998fb9f7e9b2111641485bf8beb32f92945f97f92a3d061f744cfef335f751" +dependencies = [ + "parse-zoneinfo", + "phf", + "phf_codegen", +] + [[package]] name = "clap" version = "4.1.8" @@ -1169,12 +1202,59 @@ dependencies = [ "windows-sys 0.45.0", ] +[[package]] +name = "parse-zoneinfo" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c705f256449c60da65e11ff6626e0c16a0a0b96aaa348de61376b249bc340f41" +dependencies = [ + "regex", +] + [[package]] name = "percent-encoding" version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "478c572c3d73181ff3c2539045f6eb99e5491218eae919370993b890cdbdd98e" +[[package]] +name = "phf" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "928c6535de93548188ef63bb7c4036bd415cd8f36ad25af44b9789b2ee72a48c" +dependencies = [ + "phf_shared", +] + +[[package]] +name = "phf_codegen" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a56ac890c5e3ca598bbdeaa99964edb5b0258a583a9eb6ef4e89fc85d9224770" +dependencies = [ + "phf_generator", + "phf_shared", +] + +[[package]] +name = "phf_generator" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1181c94580fa345f50f19d738aaa39c0ed30a600d95cb2d3e23f94266f14fbf" +dependencies = [ + "phf_shared", + "rand", +] + +[[package]] +name = "phf_shared" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e1fb5f6f826b772a8d4c0394209441e7d37cbbb967ae9c7e0e8134365c9ee676" +dependencies = [ + "siphasher", +] + [[package]] name = "pin-project-lite" version = "0.2.9" @@ -1451,6 +1531,12 @@ dependencies = [ "digest", ] +[[package]] +name = "siphasher" +version = "0.3.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7bd3e3206899af3f8b12af284fafc038cc1dc2b41d1b89dd17297221c5d225de" + [[package]] name = "slab" version = "0.4.8" diff --git a/rust/agama-locale-data/Cargo.toml b/rust/agama-locale-data/Cargo.toml index ddb5b2b1dd..e9ef5f02c1 100644 --- a/rust/agama-locale-data/Cargo.toml +++ b/rust/agama-locale-data/Cargo.toml @@ -8,4 +8,5 @@ edition = "2021" [dependencies] serde = { version = "1.0.152", features = ["derive"] } quick-xml = { version = "0.28.2", features = ["serialize"] } -flate2 = "1.0.25" \ No newline at end of file +flate2 = "1.0.25" +chrono-tz = "0.8.2" \ No newline at end of file diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index d26804e919..38f0a5c4c2 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -42,6 +42,10 @@ pub fn get_timezone_parts() -> timezone_part::TimezoneIdParts { timezone_part::TimezoneIdParts::deserialize(&mut deserializer).expect("Failed to deserialize timezone part entry") } +pub fn get_timezones() -> Vec { + chrono_tz::TZ_VARIANTS.iter().map(|e| e.name().to_string()).collect() +} + #[cfg(test)] mod tests { use super::*; @@ -77,4 +81,13 @@ mod tests { let first = result.timezone_part.first().expect("no keyboards"); assert_eq!(first.id, "Abidjan") } + + + #[test] + fn test_get_timezones() { + let result = get_timezones(); + assert_eq!(result.len(), 596); + let first = result.first().expect("no keyboards"); + assert_eq!(first, "Africa/Abidjan") + } } \ No newline at end of file From 35943255e73d599831993fbc67689e60b68b0f38 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 19 Apr 2023 14:11:36 +0200 Subject: [PATCH 15/73] WIP another try, with langtable backend in mind --- doc/locale_api_WIP.md | 48 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 doc/locale_api_WIP.md diff --git a/doc/locale_api_WIP.md b/doc/locale_api_WIP.md new file mode 100644 index 0000000000..c5c708574d --- /dev/null +++ b/doc/locale_api_WIP.md @@ -0,0 +1,48 @@ + +Note: ListFoo APIs combine the IDs and the translations. +If it's easier to return just the IDs and ask for the translations separately, +let's use such different API. + +### Locales + +``` +ListLocales( + in s display_locale # "en_US.UTF-8" + out a(ss) id_label_pairs +) + +SetLocale(in s locale_id) +``` + +### Keyboards + +``` +ListX11Keyboards( + in s display_locale # "en_US.UTF-8" + out a(ss) id_label_pairs # id like "layout(variant)" +) + +SetX11Keyboard(in s kb_id) + +X11KeyboardForLocale( + in s locale_id + out s kb_id +) +``` + +### Timezones + + +``` +ListTimezones( + in s display_locale # "en_US.UTF-8" + out a(ss) id_label_pairs +) + +SetTimezone(in s tz_id) + +TimezoneForLocale( + in s locale_id + out s tz_id +) +``` From b6ca533a1ea2774147625c3fb89abf1a2c43bd90 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 19 Apr 2023 17:40:37 +0200 Subject: [PATCH 16/73] Add dbus server for locale ( and not only for that ) --- rust/Cargo.lock | 4 ++++ rust/Cargo.toml | 3 ++- rust/agama-dbus-server/Cargo.toml | 8 ++++++++ rust/agama-dbus-server/src/main.rs | 3 +++ 4 files changed, 17 insertions(+), 1 deletion(-) create mode 100644 rust/agama-dbus-server/Cargo.toml create mode 100644 rust/agama-dbus-server/src/main.rs diff --git a/rust/Cargo.lock b/rust/Cargo.lock index 0d2109c75b..9ef8c94ed4 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -23,6 +23,10 @@ dependencies = [ "thiserror", ] +[[package]] +name = "agama-dbus-server" +version = "0.1.0" + [[package]] name = "agama-derive" version = "1.0.0" diff --git a/rust/Cargo.toml b/rust/Cargo.toml index cdd8130779..4143977ed6 100644 --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -3,5 +3,6 @@ members = [ "agama-lib", "agama-cli", "agama-derive", - "agama-locale-data" + "agama-locale-data", + "agama-dbus-server" ] diff --git a/rust/agama-dbus-server/Cargo.toml b/rust/agama-dbus-server/Cargo.toml new file mode 100644 index 0000000000..e0bebebac6 --- /dev/null +++ b/rust/agama-dbus-server/Cargo.toml @@ -0,0 +1,8 @@ +[package] +name = "agama-dbus-server" +version = "0.1.0" +edition = "2021" + +# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html + +[dependencies] diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs new file mode 100644 index 0000000000..e7a11a969c --- /dev/null +++ b/rust/agama-dbus-server/src/main.rs @@ -0,0 +1,3 @@ +fn main() { + println!("Hello, world!"); +} From cd87f2a8278c77a9820237c790fd2f0b5e93035e Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 20 Apr 2023 09:13:53 +0200 Subject: [PATCH 17/73] initial dbus service implementation --- rust/Cargo.lock | 5 +++ rust/agama-dbus-server/Cargo.toml | 3 ++ rust/agama-dbus-server/src/main.rs | 52 ++++++++++++++++++++++++++++-- 3 files changed, 58 insertions(+), 2 deletions(-) diff --git a/rust/Cargo.lock b/rust/Cargo.lock index 9ef8c94ed4..4768124509 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -26,6 +26,11 @@ dependencies = [ [[package]] name = "agama-dbus-server" version = "0.1.0" +dependencies = [ + "agama-locale-data", + "async-std", + "zbus", +] [[package]] name = "agama-derive" diff --git a/rust/agama-dbus-server/Cargo.toml b/rust/agama-dbus-server/Cargo.toml index e0bebebac6..965f88eea0 100644 --- a/rust/agama-dbus-server/Cargo.toml +++ b/rust/agama-dbus-server/Cargo.toml @@ -6,3 +6,6 @@ edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] +agama-locale-data = { path="../agama-locale-data" } +zbus = "3.7.0" +async-std = { version = "1.12.0", features = ["attributes"]} \ No newline at end of file diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index e7a11a969c..1bfb3250e0 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -1,3 +1,51 @@ -fn main() { - println!("Hello, world!"); +use std::{error::Error, future::pending}; +use zbus::{ConnectionBuilder, dbus_interface}; + +struct Locale { + locale_id: String, + keyboard_id: String, + timezone_id: String } + +#[dbus_interface(name = "org.opensuse.Agama.Locale1")] +impl Locale { + // Can be `async` as well. + fn list_locales(&self, locale: &str) -> Vec<(String, String)> { + let locales = agama_locale_data::get_languages(); + // TODO: localization param + return locales.language.iter().map(|l| (l.id.clone(), locale.to_string())).collect() + } + + fn set_locale(&mut self, locale: &str) { + self.locale_id = locale.to_string(); + } + + fn list_x11_keyboards(&self) -> Vec<(String, String)> { + let keyboards = agama_locale_data::get_keyboards(); + return keyboards.keyboard.iter().map(|k| (k.id.clone(), k.description.clone())).collect() + } + + fn set_x11_keyboard(&mut self, keyboard: &str) { + self.keyboard_id = keyboard.to_string(); + } + + fn set_timezone(&mut self, timezone: &str) { + self.timezone_id = timezone.to_string(); + } +} + +// Although we use `async-std` here, you can use any async runtime of choice. +#[async_std::main] +async fn main() -> Result<(), Box> { + let locale = Locale { locale_id: "en".to_string(), keyboard_id: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; + let _conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one + .name("org.opensuse.Agama.Locale1")? + .serve_at("/org/opensuse/Agama/Locale1", locale)? + .build() + .await?; + + // Do other things or go to wait forever + pending::<()>().await; + + Ok(()) +} \ No newline at end of file From c30f0037844c65faf8373d309a11b40cd5833294 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 20 Apr 2023 23:30:43 +0200 Subject: [PATCH 18/73] WIP localized timezones --- rust/agama-dbus-server/src/main.rs | 6 ++ rust/agama-locale-data/src/lib.rs | 84 ++++++++++++++++++++- rust/agama-locale-data/src/localization.rs | 23 ++++++ rust/agama-locale-data/src/timezone_part.rs | 39 +++++++++- 4 files changed, 147 insertions(+), 5 deletions(-) create mode 100644 rust/agama-locale-data/src/localization.rs diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 1bfb3250e0..90fcecb882 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -29,6 +29,12 @@ impl Locale { self.keyboard_id = keyboard.to_string(); } + fn list_timezones(&self, locale: &str) -> Vec<(String, String)> { + let timezones = agama_locale_data::get_timezones(); + let localized = agama_locale_data::get_timezone_parts().localize_timezones(locale, &timezones); + timezones.into_iter().zip(localized.into_iter()).collect() + } + fn set_timezone(&mut self, timezone: &str) { self.timezone_id = timezone.to_string(); } diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 38f0a5c4c2..fd92fc949b 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -6,6 +6,7 @@ use flate2::bufread::GzDecoder; pub mod keyboard; pub mod language; +pub mod localization; pub mod territory; pub mod timezone_part; pub mod ranked; @@ -42,8 +43,75 @@ pub fn get_timezone_parts() -> timezone_part::TimezoneIdParts { timezone_part::TimezoneIdParts::deserialize(&mut deserializer).expect("Failed to deserialize timezone part entry") } +/// List of timezones which are deprecated and langtables missing translations for it +const DEPRECATED_TIMEZONES : &[&str]= &[ + "Africa/Asmera", // replaced by Africa/Asmara + "Africa/Timbuktu", // replaced by Africa/Bamako + "America/Argentina/ComodRivadavia", // replaced by America/Argentina/Catamarca + "America/Atka", // replaced by America/Adak + "America/Ciudad_Juarez", // failed to find replacement + "America/Coral_Harbour", // replaced by America/Atikokan + "America/Ensenada", // replaced by America/Tijuana + "America/Fort_Nelson", + "America/Fort_Wayne", // replaced by America/Indiana/Indianapolis + "America/Knox_IN", // replaced by America/Indiana/Knox + "America/Nuuk", + "America/Porto_Acre", // replaced by America/Rio_Branco + "America/Punta_Arenas", + "America/Rosario", + "America/Virgin", + "Antarctica/Troll", + "Asia/Ashkhabad", // looks like typo/wrong transcript, it should be Asia/Ashgabat + "Asia/Atyrau", + "Asia/Barnaul", + "Asia/Calcutta", // renamed to Asia/Kolkata + "Asia/Chita", + "Asia/Chungking", + "Asia/Dacca", + "Asia/Famagusta", + "Asia/Katmandu", + "Asia/Macao", + "Asia/Qostanay", + "Asia/Saigon", + "Asia/Srednekolymsk", + "Asia/Tel_Aviv", + "Asia/Thimbu", + "Asia/Tomsk", + "Asia/Ujung_Pandang", + "Asia/Ulan_Bator", + "Asia/Yangon", + "Atlantic/Faeroe", + "Atlantic/Jan_Mayen", + "Australia/ACT", + "Australia/Canberra", + "Australia/LHI", + "Australia/NSW", + "Australia/North", + "Australia/Queensland", + "Australia/South", + "Australia/Tasmania", + "Australia/Victoria", + "Australia/West", + "Australia/Yancowinna", + "Brazil/Acre", + "Brazil/DeNoronha", + "Brazil/East", + "Brazil/West", + "CET", + "CST6CDT", + "Canada/Atlantic", + "Canada/Central", + "Canada/Eastern", + "Canada/Mountain", + "Canada/Newfoundland", + "Canada/Pacific", +]; + pub fn get_timezones() -> Vec { - chrono_tz::TZ_VARIANTS.iter().map(|e| e.name().to_string()).collect() + chrono_tz::TZ_VARIANTS.iter() + .filter(|&tz| !DEPRECATED_TIMEZONES.contains(&tz.name())) // Filter out deprecated asmera + .map(|e| e.name().to_string()) + .collect() } #[cfg(test)] @@ -82,12 +150,20 @@ mod tests { assert_eq!(first.id, "Abidjan") } - #[test] fn test_get_timezones() { let result = get_timezones(); - assert_eq!(result.len(), 596); + //assert_eq!(result.len(), 574); let first = result.first().expect("no keyboards"); - assert_eq!(first, "Africa/Abidjan") + assert_eq!(first, "Africa/Abidjan"); + // test that we filter out deprecates Asmera ( there is already recent Asmara) + let asmera = result.iter().find(|&t| *t == "Africa/Asmera".to_string()); + assert_eq!(asmera, None); + let asmara = result.iter().find(|&t| *t == "Africa/Asmara".to_string()); + assert_eq!(asmara, Some(&"Africa/Asmara".to_string())); + // here test that timezones from timezones matches ones in langtable ( as timezones can contain deprecated ones) + let timezones = get_timezones(); + let localized = get_timezone_parts().localize_timezones("de", &timezones); + let _res : Vec<(String, String)> = timezones.into_iter().zip(localized.into_iter()).collect(); } } \ No newline at end of file diff --git a/rust/agama-locale-data/src/localization.rs b/rust/agama-locale-data/src/localization.rs new file mode 100644 index 0000000000..e4bc9c5eba --- /dev/null +++ b/rust/agama-locale-data/src/localization.rs @@ -0,0 +1,23 @@ +use serde::Deserialize; + +#[derive(Debug, Deserialize)] +pub struct Localization { + pub name: Vec +} + +impl Localization { + pub fn name_for(&self, language: &str) -> String { + self.name.iter() + .find(|n| n.language == language) + .expect("Wrong language {language}") + .value.clone() + } +} + +#[derive(Debug, Deserialize)] +pub struct LocalizationEntry { + #[serde(rename(deserialize = "languageId"))] + pub language: String, + #[serde(rename(deserialize = "trName"))] + pub value: String +} \ No newline at end of file diff --git a/rust/agama-locale-data/src/timezone_part.rs b/rust/agama-locale-data/src/timezone_part.rs index beaefa0981..e2790947a4 100644 --- a/rust/agama-locale-data/src/timezone_part.rs +++ b/rust/agama-locale-data/src/timezone_part.rs @@ -1,9 +1,12 @@ +use std::collections::HashMap; + use serde::Deserialize; #[derive(Debug, Deserialize)] pub struct TimezoneIdPart { #[serde(rename(deserialize = "timezoneIdPartId"))] - pub id: String + pub id: String, + pub names: crate::localization::Localization } // Timezone id parts are useful mainly for localization of timezones @@ -12,4 +15,38 @@ pub struct TimezoneIdPart { pub struct TimezoneIdParts { #[serde(rename(deserialize = "timezoneIdPart"))] pub timezone_part: Vec +} + +impl TimezoneIdParts { + /// Localized given list of timezones to given language + /// # Examples + /// + /// ``` + /// let parts = agama_locale_data::get_timezone_parts(); + /// let timezones = vec!["Europe/Prague".to_string(), "Europe/Berlin".to_string()]; + /// let result = vec!["Evropa/Praha".to_string(), "Evropa/Berlín".to_string()]; + /// assert_eq!(parts.localize_timezones("cs", timezones), result); + /// ``` + pub fn localize_timezones(&self, language: &str, timezones: &Vec) -> Vec { + let mapping = self.construct_mapping(language); + timezones.iter().map(|tz| self.translate_timezone(&mapping, tz)).collect() + } + + fn construct_mapping(&self, language: &str) -> HashMap { + let mut res: HashMap = HashMap::with_capacity(self.timezone_part.len()); + self.timezone_part.iter() + .map(|part| (part.id.clone(), part.names.name_for(language))) + .for_each(|tuple| -> () { + res.insert(tuple.0, tuple.1); + } + ); + return res + } + + fn translate_timezone(&self, mapping: &HashMap, timezone: &str) -> String { + eprintln!("{timezone}"); + timezone.split("/") + .map(|tzp| mapping.get(&tzp.to_string()).expect(format!("Unknown timezone part {tzp}").as_str()).to_owned()) + .collect::>().join("/") + } } \ No newline at end of file From 75f473bbff508578b6f02ee41a53fc250bfd88a6 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Fri, 21 Apr 2023 10:11:05 +0200 Subject: [PATCH 19/73] English+native locale labels like YaST; no kb localized labels --- doc/locale_api_WIP.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/doc/locale_api_WIP.md b/doc/locale_api_WIP.md index c5c708574d..43450e8db0 100644 --- a/doc/locale_api_WIP.md +++ b/doc/locale_api_WIP.md @@ -7,8 +7,7 @@ let's use such different API. ``` ListLocales( - in s display_locale # "en_US.UTF-8" - out a(ss) id_label_pairs + out a(sss) id_english_native ) SetLocale(in s locale_id) @@ -18,8 +17,7 @@ SetLocale(in s locale_id) ``` ListX11Keyboards( - in s display_locale # "en_US.UTF-8" - out a(ss) id_label_pairs # id like "layout(variant)" + out as ids # id like "layout(variant)" ) SetX11Keyboard(in s kb_id) From 1af7921e1b0f8f00dbb0ef0b47b4d37a0349d3f8 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 21 Apr 2023 11:20:09 +0200 Subject: [PATCH 20/73] final timezone functionality --- .../src/deprecated_timezones.rs | 173 ++++++++++++++++++ rust/agama-locale-data/src/lib.rs | 68 +------ rust/agama-locale-data/src/timezone_part.rs | 1 - 3 files changed, 176 insertions(+), 66 deletions(-) create mode 100644 rust/agama-locale-data/src/deprecated_timezones.rs diff --git a/rust/agama-locale-data/src/deprecated_timezones.rs b/rust/agama-locale-data/src/deprecated_timezones.rs new file mode 100644 index 0000000000..403f210859 --- /dev/null +++ b/rust/agama-locale-data/src/deprecated_timezones.rs @@ -0,0 +1,173 @@ +/// List of timezones which are deprecated and langtables missing translations for it +/// +/// Filtering it out also helps with returning smaller list of real timezones. +/// Sadly many libraries facing issues with deprecated timezones, see e.g. +/// +pub(crate) const DEPRECATED_TIMEZONES : &[&str]= &[ + "Africa/Asmera", // replaced by Africa/Asmara + "Africa/Timbuktu", // replaced by Africa/Bamako + "America/Argentina/ComodRivadavia", // replaced by America/Argentina/Catamarca + "America/Atka", // replaced by America/Adak + "America/Ciudad_Juarez", // failed to find replacement + "America/Coral_Harbour", // replaced by America/Atikokan + "America/Ensenada", // replaced by America/Tijuana + "America/Fort_Nelson", + "America/Fort_Wayne", // replaced by America/Indiana/Indianapolis + "America/Knox_IN", // replaced by America/Indiana/Knox + "America/Nuuk", + "America/Porto_Acre", // replaced by America/Rio_Branco + "America/Punta_Arenas", + "America/Rosario", + "America/Virgin", + "Antarctica/Troll", + "Asia/Ashkhabad", // looks like typo/wrong transcript, it should be Asia/Ashgabat + "Asia/Atyrau", + "Asia/Barnaul", + "Asia/Calcutta", // renamed to Asia/Kolkata + "Asia/Chita", + "Asia/Chungking", + "Asia/Dacca", + "Asia/Famagusta", + "Asia/Katmandu", + "Asia/Macao", + "Asia/Qostanay", + "Asia/Saigon", + "Asia/Srednekolymsk", + "Asia/Tel_Aviv", + "Asia/Thimbu", + "Asia/Tomsk", + "Asia/Ujung_Pandang", + "Asia/Ulan_Bator", + "Asia/Yangon", + "Atlantic/Faeroe", + "Atlantic/Jan_Mayen", + "Australia/ACT", + "Australia/Canberra", + "Australia/LHI", + "Australia/NSW", + "Australia/North", + "Australia/Queensland", + "Australia/South", + "Australia/Tasmania", + "Australia/Victoria", + "Australia/West", + "Australia/Yancowinna", + "Brazil/Acre", + "Brazil/DeNoronha", + "Brazil/East", + "Brazil/West", + "CET", + "CST6CDT", + "Canada/Atlantic", // all canada TZ was replaced by America ones + "Canada/Central", + "Canada/Eastern", + "Canada/Mountain", + "Canada/Newfoundland", + "Canada/Pacific", + "Canada/Saskatchewan", + "Canada/Yukon", + "Chile/Continental", // all Chile was replaced by continental America tz + "Chile/EasterIsland", + "Cuba", + "EET", + "EST", // not sure why it is not in langtable + "EST5EDT", + "Egypt", + "Eire", + "Etc/GMT", + "Etc/GMT+0", + "Etc/GMT+1", + "Etc/GMT+2", + "Etc/GMT+3", + "Etc/GMT+4", + "Etc/GMT+5", + "Etc/GMT+6", + "Etc/GMT+7", + "Etc/GMT+8", + "Etc/GMT+9", + "Etc/GMT+10", + "Etc/GMT+11", + "Etc/GMT+12", + "Etc/GMT-0", + "Etc/GMT-1", + "Etc/GMT-2", + "Etc/GMT-3", + "Etc/GMT-4", + "Etc/GMT-5", + "Etc/GMT-6", + "Etc/GMT-7", + "Etc/GMT-8", + "Etc/GMT-9", + "Etc/GMT-10", + "Etc/GMT-11", + "Etc/GMT-12", + "Etc/GMT-13", + "Etc/GMT-14", + "Etc/GMT0", + "Etc/Greenwich", + "Etc/UCT", + "Etc/UTC", + "Etc/Universal", + "Etc/Zulu", + "Europe/Astrakhan", + "Europe/Belfast", + "Europe/Kirov", + "Europe/Kyiv", + "Europe/Saratov", + "Europe/Tiraspol", + "Europe/Ulyanovsk", + "GB", + "GB-Eire", + "GMT", + "GMT+0", + "GMT-0", + "GMT0", + "Greenwich", + "HST", + "Hongkong", + "Iceland", + "Iran", + "Israel", + "Jamaica", + "Japan", + "Kwajalein", + "Libya", + "MET", + "Mexico/BajaNorte", + "Mexico/BajaSur", + "Mexico/General", + "MST", + "MST7MDT", + "NZ", + "NZ-CHAT", + "Navajo", + "Pacific/Bougainville", + "Pacific/Kanton", + "Pacific/Ponape", + "Pacific/Samoa", + "Pacific/Truk", + "Pacific/Yap", + "PRC", + "PST8PDT", + "Poland", + "Portugal", + "ROC", + "ROK", + "Singapore", + "Turkey", + "UCT", + "Universal", + "US/Aleutian", // all US/ replaced by America + "US/Central", + "US/East-Indiana", + "US/Eastern", + "US/Hawaii", + "US/Indiana-Starke", + "US/Michigan", + "US/Mountain", + "US/Pacific", + "US/Samoa", + "W-SU", + "WET", + "Zulu", +]; \ No newline at end of file diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index fd92fc949b..22a502da9e 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -10,6 +10,7 @@ pub mod localization; pub mod territory; pub mod timezone_part; pub mod ranked; +pub mod deprecated_timezones; pub fn get_keyboards() -> keyboard::Keyboards { const FILE_PATH: &str = "/usr/share/langtable/data/keyboards.xml.gz"; @@ -43,73 +44,9 @@ pub fn get_timezone_parts() -> timezone_part::TimezoneIdParts { timezone_part::TimezoneIdParts::deserialize(&mut deserializer).expect("Failed to deserialize timezone part entry") } -/// List of timezones which are deprecated and langtables missing translations for it -const DEPRECATED_TIMEZONES : &[&str]= &[ - "Africa/Asmera", // replaced by Africa/Asmara - "Africa/Timbuktu", // replaced by Africa/Bamako - "America/Argentina/ComodRivadavia", // replaced by America/Argentina/Catamarca - "America/Atka", // replaced by America/Adak - "America/Ciudad_Juarez", // failed to find replacement - "America/Coral_Harbour", // replaced by America/Atikokan - "America/Ensenada", // replaced by America/Tijuana - "America/Fort_Nelson", - "America/Fort_Wayne", // replaced by America/Indiana/Indianapolis - "America/Knox_IN", // replaced by America/Indiana/Knox - "America/Nuuk", - "America/Porto_Acre", // replaced by America/Rio_Branco - "America/Punta_Arenas", - "America/Rosario", - "America/Virgin", - "Antarctica/Troll", - "Asia/Ashkhabad", // looks like typo/wrong transcript, it should be Asia/Ashgabat - "Asia/Atyrau", - "Asia/Barnaul", - "Asia/Calcutta", // renamed to Asia/Kolkata - "Asia/Chita", - "Asia/Chungking", - "Asia/Dacca", - "Asia/Famagusta", - "Asia/Katmandu", - "Asia/Macao", - "Asia/Qostanay", - "Asia/Saigon", - "Asia/Srednekolymsk", - "Asia/Tel_Aviv", - "Asia/Thimbu", - "Asia/Tomsk", - "Asia/Ujung_Pandang", - "Asia/Ulan_Bator", - "Asia/Yangon", - "Atlantic/Faeroe", - "Atlantic/Jan_Mayen", - "Australia/ACT", - "Australia/Canberra", - "Australia/LHI", - "Australia/NSW", - "Australia/North", - "Australia/Queensland", - "Australia/South", - "Australia/Tasmania", - "Australia/Victoria", - "Australia/West", - "Australia/Yancowinna", - "Brazil/Acre", - "Brazil/DeNoronha", - "Brazil/East", - "Brazil/West", - "CET", - "CST6CDT", - "Canada/Atlantic", - "Canada/Central", - "Canada/Eastern", - "Canada/Mountain", - "Canada/Newfoundland", - "Canada/Pacific", -]; - pub fn get_timezones() -> Vec { chrono_tz::TZ_VARIANTS.iter() - .filter(|&tz| !DEPRECATED_TIMEZONES.contains(&tz.name())) // Filter out deprecated asmera + .filter(|&tz| !crate::deprecated_timezones::DEPRECATED_TIMEZONES.contains(&tz.name())) // Filter out deprecated asmera .map(|e| e.name().to_string()) .collect() } @@ -162,6 +99,7 @@ mod tests { let asmara = result.iter().find(|&t| *t == "Africa/Asmara".to_string()); assert_eq!(asmara, Some(&"Africa/Asmara".to_string())); // here test that timezones from timezones matches ones in langtable ( as timezones can contain deprecated ones) + // so this test catch if there is new zone that is not translated or if a zone is become deprecated let timezones = get_timezones(); let localized = get_timezone_parts().localize_timezones("de", &timezones); let _res : Vec<(String, String)> = timezones.into_iter().zip(localized.into_iter()).collect(); diff --git a/rust/agama-locale-data/src/timezone_part.rs b/rust/agama-locale-data/src/timezone_part.rs index e2790947a4..c7b2b733b9 100644 --- a/rust/agama-locale-data/src/timezone_part.rs +++ b/rust/agama-locale-data/src/timezone_part.rs @@ -44,7 +44,6 @@ impl TimezoneIdParts { } fn translate_timezone(&self, mapping: &HashMap, timezone: &str) -> String { - eprintln!("{timezone}"); timezone.split("/") .map(|tzp| mapping.get(&tzp.to_string()).expect(format!("Unknown timezone part {tzp}").as_str()).to_owned()) .collect::>().join("/") From a289660248a540eec25d36caaf797681645b533f Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 21 Apr 2023 14:13:36 +0200 Subject: [PATCH 21/73] enable again check of timezones size --- rust/agama-locale-data/src/lib.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 22a502da9e..078fa64e00 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -90,7 +90,7 @@ mod tests { #[test] fn test_get_timezones() { let result = get_timezones(); - //assert_eq!(result.len(), 574); + assert_eq!(result.len(), 430); let first = result.first().expect("no keyboards"); assert_eq!(first, "Africa/Abidjan"); // test that we filter out deprecates Asmera ( there is already recent Asmara) From 367a27744b7153f1b4ca20fdae89add4e941d3d3 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Sat, 22 Apr 2023 09:37:58 +0200 Subject: [PATCH 22/73] Use anyhow::Error to report agama-locale-data errors via D-Bus --- rust/Cargo.lock | 3 +++ rust/agama-dbus-server/Cargo.toml | 4 +++- rust/agama-dbus-server/src/error.rs | 21 +++++++++++++++++++++ rust/agama-dbus-server/src/main.rs | 16 +++++++++++----- rust/agama-locale-data/Cargo.toml | 3 ++- rust/agama-locale-data/src/lib.rs | 12 ++++++++---- 6 files changed, 48 insertions(+), 11 deletions(-) create mode 100644 rust/agama-dbus-server/src/error.rs diff --git a/rust/Cargo.lock b/rust/Cargo.lock index 4768124509..ddc4fe041b 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -28,8 +28,10 @@ name = "agama-dbus-server" version = "0.1.0" dependencies = [ "agama-locale-data", + "anyhow", "async-std", "zbus", + "zbus_macros", ] [[package]] @@ -63,6 +65,7 @@ dependencies = [ name = "agama-locale-data" version = "0.1.0" dependencies = [ + "anyhow", "chrono-tz", "flate2", "quick-xml", diff --git a/rust/agama-dbus-server/Cargo.toml b/rust/agama-dbus-server/Cargo.toml index 965f88eea0..6a38539f7c 100644 --- a/rust/agama-dbus-server/Cargo.toml +++ b/rust/agama-dbus-server/Cargo.toml @@ -6,6 +6,8 @@ edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] +anyhow = "1.0" agama-locale-data = { path="../agama-locale-data" } zbus = "3.7.0" -async-std = { version = "1.12.0", features = ["attributes"]} \ No newline at end of file +zbus_macros = "3.7.0" +async-std = { version = "1.12.0", features = ["attributes"]} diff --git a/rust/agama-dbus-server/src/error.rs b/rust/agama-dbus-server/src/error.rs new file mode 100644 index 0000000000..e2414d045d --- /dev/null +++ b/rust/agama-dbus-server/src/error.rs @@ -0,0 +1,21 @@ +use zbus_macros::DBusError; + +#[derive(DBusError, Debug)] +#[dbus_error(prefix = "org.opensuse.Agama.Locale1")] +pub enum Error { + #[dbus_error(zbus_error)] + ZBus(zbus::Error), + Anyhow(String), +} + +// This would be nice, but using it for a return type +// results in a confusing error message about +// error[E0277]: the trait bound `MyError: Serialize` is not satisfied +//type MyResult = Result; + +impl From for Error { + fn from(e: anyhow::Error) -> Self { + // {:#} includes causes + Self::Anyhow(format!("{:#}", e)) + } +} diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 90fcecb882..83b7287021 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -1,4 +1,6 @@ -use std::{error::Error, future::pending}; +pub mod error; +use crate::error::Error; +use std::future::pending; use zbus::{ConnectionBuilder, dbus_interface}; struct Locale { @@ -10,10 +12,14 @@ struct Locale { #[dbus_interface(name = "org.opensuse.Agama.Locale1")] impl Locale { // Can be `async` as well. - fn list_locales(&self, locale: &str) -> Vec<(String, String)> { + fn list_locales(&self, locale: &str) -> Result, Error> { let locales = agama_locale_data::get_languages(); // TODO: localization param - return locales.language.iter().map(|l| (l.id.clone(), locale.to_string())).collect() + let ret = locales? + .language.iter() + .map(|l| (l.id.clone(), locale.to_string())) + .collect(); + Ok(ret) } fn set_locale(&mut self, locale: &str) { @@ -42,7 +48,7 @@ impl Locale { // Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] -async fn main() -> Result<(), Box> { +async fn main() -> Result<(), Box> { let locale = Locale { locale_id: "en".to_string(), keyboard_id: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; let _conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one .name("org.opensuse.Agama.Locale1")? @@ -54,4 +60,4 @@ async fn main() -> Result<(), Box> { pending::<()>().await; Ok(()) -} \ No newline at end of file +} diff --git a/rust/agama-locale-data/Cargo.toml b/rust/agama-locale-data/Cargo.toml index e9ef5f02c1..48cc219562 100644 --- a/rust/agama-locale-data/Cargo.toml +++ b/rust/agama-locale-data/Cargo.toml @@ -6,7 +6,8 @@ edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] +anyhow = "1.0" serde = { version = "1.0.152", features = ["derive"] } quick-xml = { version = "0.28.2", features = ["serialize"] } flate2 = "1.0.25" -chrono-tz = "0.8.2" \ No newline at end of file +chrono-tz = "0.8.2" diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 078fa64e00..4057524911 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -1,3 +1,4 @@ +use anyhow::Context; use serde::Deserialize; use std::fs::File; use std::io::BufReader; @@ -20,12 +21,15 @@ pub fn get_keyboards() -> keyboard::Keyboards { keyboard::Keyboards::deserialize(&mut deserializer).expect("Failed to deserialize keyboard entry") } -pub fn get_languages() -> language::Languages { +pub fn get_languages() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/languages.xml.gz"; - let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); + let file = File::open(FILE_PATH) + .context("Failed to read langtable-data")?; let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); let mut deserializer = Deserializer::from_reader(reader); - language::Languages::deserialize(&mut deserializer).expect("Failed to deserialize language entry") + let ret = language::Languages::deserialize(&mut deserializer) + .context("Failed to deserialize language entry")?; + Ok(ret) } pub fn get_territories() -> territory::Territories { @@ -104,4 +108,4 @@ mod tests { let localized = get_timezone_parts().localize_timezones("de", &timezones); let _res : Vec<(String, String)> = timezones.into_iter().zip(localized.into_iter()).collect(); } -} \ No newline at end of file +} From f5ec5394b049fcc408e04df6676601df22c10bd1 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Sun, 23 Apr 2023 14:01:53 +0200 Subject: [PATCH 23/73] list_x11_keyboards and list_timezones may return errors too use a file_reader helper --- rust/agama-dbus-server/src/main.rs | 19 +++++--- rust/agama-locale-data/src/keyboard.rs | 2 + rust/agama-locale-data/src/lib.rs | 49 ++++++++++++--------- rust/agama-locale-data/src/ranked.rs | 5 ++- rust/agama-locale-data/src/timezone_part.rs | 2 + 5 files changed, 50 insertions(+), 27 deletions(-) diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 83b7287021..8fb498edbd 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -1,5 +1,6 @@ pub mod error; use crate::error::Error; + use std::future::pending; use zbus::{ConnectionBuilder, dbus_interface}; @@ -26,19 +27,25 @@ impl Locale { self.locale_id = locale.to_string(); } - fn list_x11_keyboards(&self) -> Vec<(String, String)> { - let keyboards = agama_locale_data::get_keyboards(); - return keyboards.keyboard.iter().map(|k| (k.id.clone(), k.description.clone())).collect() + fn list_x11_keyboards(&self) -> Result, Error> { + let keyboards = agama_locale_data::get_keyboards()?; + let ret = keyboards + .keyboard.iter() + .map(|k| (k.id.clone(), k.description.clone())) + .collect(); + Ok(ret) } fn set_x11_keyboard(&mut self, keyboard: &str) { self.keyboard_id = keyboard.to_string(); } - fn list_timezones(&self, locale: &str) -> Vec<(String, String)> { + fn list_timezones(&self, locale: &str) -> Result, Error> { let timezones = agama_locale_data::get_timezones(); - let localized = agama_locale_data::get_timezone_parts().localize_timezones(locale, &timezones); - timezones.into_iter().zip(localized.into_iter()).collect() + let localized = agama_locale_data::get_timezone_parts()? + .localize_timezones(locale, &timezones); + let ret = timezones.into_iter().zip(localized.into_iter()).collect(); + Ok(ret) } fn set_timezone(&mut self, timezone: &str) { diff --git a/rust/agama-locale-data/src/keyboard.rs b/rust/agama-locale-data/src/keyboard.rs index e7d3d7b8c7..e30da4eab3 100644 --- a/rust/agama-locale-data/src/keyboard.rs +++ b/rust/agama-locale-data/src/keyboard.rs @@ -5,7 +5,9 @@ use crate::ranked::{RankedTerritories, RankedLanguages}; #[derive(Debug, Deserialize)] pub struct Keyboard { #[serde(rename(deserialize = "keyboardId"))] + /// like "layout(variant)", for example "us" or "ua(phonetic)" pub id: String, + /// like "Ukrainian (phonetic)" pub description: String, pub ascii: bool, pub comment: Option, diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 4057524911..c5679a521c 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -1,6 +1,7 @@ use anyhow::Context; use serde::Deserialize; use std::fs::File; +use std::io::BufRead; use std::io::BufReader; use quick_xml::de::Deserializer; use flate2::bufread::GzDecoder; @@ -13,39 +14,47 @@ pub mod timezone_part; pub mod ranked; pub mod deprecated_timezones; -pub fn get_keyboards() -> keyboard::Keyboards { +fn file_reader(file_path: &str) -> anyhow::Result { + let file = File::open(file_path) + .with_context(|| format!("Failed to read langtable-data ({})", file_path))?; + let reader = BufReader::new(GzDecoder::new(BufReader::new(file))); + Ok(reader) +} + +pub fn get_keyboards() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/keyboards.xml.gz"; - let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); - let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let reader = file_reader(FILE_PATH)?; let mut deserializer = Deserializer::from_reader(reader); - keyboard::Keyboards::deserialize(&mut deserializer).expect("Failed to deserialize keyboard entry") + let ret = keyboard::Keyboards::deserialize(&mut deserializer) + .context("Failed to deserialize keyboard entry")?; + Ok(ret) } pub fn get_languages() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/languages.xml.gz"; - let file = File::open(FILE_PATH) - .context("Failed to read langtable-data")?; - let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let reader = file_reader(FILE_PATH)?; let mut deserializer = Deserializer::from_reader(reader); let ret = language::Languages::deserialize(&mut deserializer) .context("Failed to deserialize language entry")?; Ok(ret) } -pub fn get_territories() -> territory::Territories { +pub fn get_territories() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/territories.xml.gz"; - let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); - let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let reader = file_reader(FILE_PATH)?; let mut deserializer = Deserializer::from_reader(reader); - territory::Territories::deserialize(&mut deserializer).expect("Failed to deserialize territory entry") + let ret = territory::Territories::deserialize(&mut deserializer) + .context("Failed to deserialize territory entry")?; + Ok(ret) } -pub fn get_timezone_parts() -> timezone_part::TimezoneIdParts { +pub fn get_timezone_parts() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/timezoneidparts.xml.gz"; - let file = File::open(FILE_PATH).expect("Failed to read langtable-data."); - let reader = BufReader::new(GzDecoder::new(BufReader::new(&file))); + let reader = file_reader(FILE_PATH)?; let mut deserializer = Deserializer::from_reader(reader); - timezone_part::TimezoneIdParts::deserialize(&mut deserializer).expect("Failed to deserialize timezone part entry") + let ret = timezone_part::TimezoneIdParts::deserialize(&mut deserializer) + .context("Failed to deserialize timezone part entry")?; + Ok(ret) } pub fn get_timezones() -> Vec { @@ -61,7 +70,7 @@ mod tests { #[test] fn test_get_keyboards() { - let result = get_keyboards(); + let result = get_keyboards().unwrap(); assert_eq!(result.keyboard.len(), 247); let first = result.keyboard.first().expect("no keyboards"); assert_eq!(first.id, "ad") @@ -69,7 +78,7 @@ mod tests { #[test] fn test_get_languages() { - let result = get_languages(); + let result = get_languages().unwrap(); assert_eq!(result.language.len(), 356); let first = result.language.first().expect("no keyboards"); assert_eq!(first.id, "aa") @@ -77,7 +86,7 @@ mod tests { #[test] fn test_get_territories() { - let result = get_territories(); + let result = get_territories().unwrap(); assert_eq!(result.territory.len(), 257); let first = result.territory.first().expect("no keyboards"); assert_eq!(first.id, "001") // looks strange, but it is meta id for whole world @@ -85,7 +94,7 @@ mod tests { #[test] fn test_get_timezone_parts() { - let result = get_timezone_parts(); + let result = get_timezone_parts().unwrap(); assert_eq!(result.timezone_part.len(), 441); let first = result.timezone_part.first().expect("no keyboards"); assert_eq!(first.id, "Abidjan") @@ -105,7 +114,7 @@ mod tests { // here test that timezones from timezones matches ones in langtable ( as timezones can contain deprecated ones) // so this test catch if there is new zone that is not translated or if a zone is become deprecated let timezones = get_timezones(); - let localized = get_timezone_parts().localize_timezones("de", &timezones); + let localized = get_timezone_parts().unwrap().localize_timezones("de", &timezones); let _res : Vec<(String, String)> = timezones.into_iter().zip(localized.into_iter()).collect(); } } diff --git a/rust/agama-locale-data/src/ranked.rs b/rust/agama-locale-data/src/ranked.rs index 82b307d8c1..004af06291 100644 --- a/rust/agama-locale-data/src/ranked.rs +++ b/rust/agama-locale-data/src/ranked.rs @@ -1,10 +1,12 @@ -use serde::Deserialize; +//! Bigger rank means it is more important +use serde::Deserialize; #[derive(Debug, Deserialize)] pub struct RankedLanguage { #[serde(rename(deserialize = "languageId"))] pub id: String, + /// Bigger rank means it is more important pub rank: u16 } @@ -17,6 +19,7 @@ pub struct RankedLanguages { pub struct RankedTerritory { #[serde(rename(deserialize = "territoryId"))] pub id: String, + /// Bigger rank means it is more important pub rank: u16 } diff --git a/rust/agama-locale-data/src/timezone_part.rs b/rust/agama-locale-data/src/timezone_part.rs index c7b2b733b9..47f97f860b 100644 --- a/rust/agama-locale-data/src/timezone_part.rs +++ b/rust/agama-locale-data/src/timezone_part.rs @@ -5,7 +5,9 @@ use serde::Deserialize; #[derive(Debug, Deserialize)] pub struct TimezoneIdPart { #[serde(rename(deserialize = "timezoneIdPartId"))] + /// "Prague" pub id: String, + /// [{language: "cs", value: "Praha"}, {"language": "de", value: "Prag"} ...] pub names: crate::localization::Localization } From 6d47b481c10a2413a0cd7ef6932e74a4d1c4ea58 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Mon, 24 Apr 2023 16:26:58 +0200 Subject: [PATCH 24/73] Update to reflect initial implementation in rust_locale considering langtable, de-emphasizing systemd but not throwing it away just yet because systemd:locale != langtable:(territory+language) well yes but the `+` is a nontrivial operation --- doc/locale_api.md | 267 ++++++++++++++++++++++++------------------ doc/locale_api_WIP.md | 46 -------- 2 files changed, 154 insertions(+), 159 deletions(-) delete mode 100644 doc/locale_api_WIP.md diff --git a/doc/locale_api.md b/doc/locale_api.md index def2e8760f..912decd8c7 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -9,106 +9,39 @@ > dependencies on YaST or Ruby. It's small enough and could give us a good > overview on how much can we save. -Plan: +Original Plan: 1. take the systemd APIs as a sensible starting point. 2. deviate only where we add value -### Language and Keyboard - -#### Systemd - -The systemd API for Locale(Language) and Keyboard is this: - -``` -$ gdbus introspect -y -d org.freedesktop.locale1 -o /org/freedesktop/locale1 -node /org/freedesktop/locale1 { - interface org.freedesktop.locale1 { - methods: - SetLocale(in as locale, - in b interactive); - SetVConsoleKeyboard(in s keymap, - in s keymap_toggle, - in b convert, - in b interactive); - SetX11Keyboard(in s layout, - in s model, - in s variant, - in s options, - in b convert, - in b interactive); -… -$ busctl --system introspect org.freedesktop.locale1 /org/freedesktop/locale1 -(all properties are read-only and emit PropertiesChanged) -.Locale property as 1 "LANG=en_US.UTF-8" -.VConsoleKeymap property s "cz-lat2-us" -.VConsoleKeymapToggle property s "" -.X11Layout property s "cz,us" -.X11Model property s "pc105" -.X11Options property s "terminate:ctrl_alt_bksp,grp:shift_togg… -.X11Variant property s "qwerty,basic" -``` - -#### Design of Elementary Layer (lower) - -The plan was: - -> Just use the systemd API, don't add any API of our own - -But the problem with that is the installer runs in one system (inst-sys, `/`) +The problem with the original plan is that +the installer runs in one system (inst-sys, `/`) and operates on another (target, `/mnt`) and we cannot use the full systemd -API. Instead we rely on `systemd-firstboot`. +API. We may use `systemd-firstboot` instead but its API is much more limited. -`systemd-firstboot` only has an option for the console keymap, but we have a -way to propagate it to X11, see [bsc#1046436](https://bugzilla.suse.com/show_bug.cgi?id=1046436) - - -#### Design of Proposal Layer (upper) - -- when setting the locale, adjust the proposed package selection and keyboard - accordingly. And timezone. - -The general design of the proposal layer is - -- declarative, using read-write properties -- setting some properties will make changes in the proposal layer of other - properties of other objects +## Localization -So here, setting `Locale` below will set also `VConsoleKeyboard` here and - - Agama...Software...todo(...) - - Agama...Timezone...todo(...) +This design includes localized labels in the API. In other contexts that would +be a responsibility of the frontend, but here here the backend has the +information, provided by _langtable_. -For the first version of the API, let's keep things simple: +(Languages, Territories and Timezones have localized names. Keyboards do not.) -**LocaleType** is just one string, the value for the `LANG` variable, like -`"cs_CZ.UTF-8"`. +(Possible alternative: still include localized labels, but in a supplemental +method while the main method only provides the IDs (and English labels)) -**VConsoleKeyboardType** is a string, for example -`"cz-qwerty"` or `"us"`. +## Proposal -We don't expose the X11 keyboard, instead letting systemd do it via the -_convert_ parameter. +A Proposal is what the installer proposes to the user +as settings to be applied to the target system. -(The other systemd keyboard settings are X11Model and X11Options, we don't -have UI or data for that) +For example, when selecting the "German (Germany)" locale, +the timezone will be proposed to "Europe/Berlin". -``` -# this is gdbus syntax BTW -node ...Agama/Locale1 { - interface ...Agama.Locale1 { - methods: - # FIXME: the method part is unclear to me. anyone better at proposals? - Commit(); - properties: - # NOTE: "as" has different meaning to systemd, - # we have a list of LANG settings, 1st gets passed to systemd, - # others affect package selection - readwrite as Locale = ['cs_CZ.UTF-8', 'de_DE.UTF-8']; - readwrite s VConsoleKeyboard = 'cz-qwerty'; - }; -}; -``` +Design decision: put the proposal logic to the antecedent object, that is, +the Locale object will know how to change the Timezone object, +not the other way around (Timezone reacting to Locale value). -##### Overriding the User's Choice? +### Overriding the User's Choice?
@@ -134,7 +67,7 @@ already touched it: > Locale: German, Keyboard: Czech (qwerty)
-##### Simple Design: Always Repropose +### Simple Design: Always Repropose We can easily afford throwing away the user's choice of keyboard layout and simply set what we consider a good default for a newly set locale, because: @@ -142,7 +75,7 @@ simply set what we consider a good default for a newly set locale, because: 1. it is just one setting (as opposed to whole partitioning layout) 2. the change will be visible in the UI, I assume -##### Detailed Design: Prioritize +### Detailed Design: Prioritize But other cases may not be as simple, so here's a generic design: @@ -177,10 +110,141 @@ this on the bus. [resstatus]: https://github.com/openSUSE/libzypp/blob/d441746c59f063b5d54833bfdebc48829b07feb5/zypp/ResStatus.h#L106 + +## Interfaces + +### Language and Keyboard + +- when setting the locale, adjust the proposed package selection and keyboard + accordingly. And timezone. + +The general design of the proposal layer is + +- declarative, using read-write properties +- setting some properties will make changes in the proposal layer of other + properties of other objects + +I don't know: should the proposal be adjusted automatically as part of the property setter, or should it be explicit? + +So here, setting `Locale` below will set also `VConsoleKeyboard` here and + - Agama...Software...todo(...) + - Agama...Timezone...todo(...) + +For the first version of the API, let's keep things simple: + +**LocaleType** is just one string, the value for the `LANG` variable, like +`"cs_CZ.UTF-8"`. + +**VConsoleKeyboardType** is a string, for example +`"cz-qwerty"` or `"us"`. + +`systemd-firstboot` only has an option for the console keymap, but we have a +way to propagate it to X11, see [bsc#1046436](https://bugzilla.suse.com/show_bug.cgi?id=1046436) + +We don't expose the X11 keyboard, instead letting systemd do it via the +_convert_ parameter. + +(The other systemd keyboard settings are X11Model and X11Options, we don't +have UI or data for that) + +**FIXME:** _langtable_ on the other hand only deals with the X11 keyboards. +And legacy YaST has the console keyboard as the primary key. Must resolve this. + +``` +# this is gdbus syntax BTW +node /org/opensuse/Agama/Locale1 { + interface org.opensuse.Agama.Locale1 { + methods: + ListLocales( + out a(sss) id_english_native # [('cs_CZ.UTF-8', 'Czech', 'Čeština'),…] + ) + # langtable has X11 keyboards and no localization of their labels + ListX11Keyboards( + out as ids # id like "layout(variant)" + ) + + # ProposeKeyboard(); # not needed? adjusted automatically, same object + # Sets Agama/TimeDate1's Timezone (but not LocalRTC, that's for Storage to say?) + ProposeTimeDate(); # different object but same service + ProposeSoftware(); # different service + + Commit(); + properties: + # NOTE: "as" has different meaning to systemd, + # we have a list of LANG settings, 1st gets passed to systemd, + # others affect package selection + readwrite as Locale = ['cs_CZ.UTF-8', 'de_DE.UTF-8']; + readwrite s VConsoleKeyboard = 'cz-qwerty'; + }; +}; +``` + +#### Systemd + +
+ +For reference, the systemd API for Locale(Language) and Keyboard is this: + + +``` +$ gdbus introspect -y -d org.freedesktop.locale1 -o /org/freedesktop/locale1 +node /org/freedesktop/locale1 { + interface org.freedesktop.locale1 { + methods: + SetLocale(in as locale, + in b interactive); + SetVConsoleKeyboard(in s keymap, + in s keymap_toggle, + in b convert, + in b interactive); + SetX11Keyboard(in s layout, + in s model, + in s variant, + in s options, + in b convert, + in b interactive); +… +$ busctl --system introspect org.freedesktop.locale1 /org/freedesktop/locale1 +(all properties are read-only and emit PropertiesChanged) +.Locale property as 1 "LANG=en_US.UTF-8" +.VConsoleKeymap property s "cz-lat2-us" +.VConsoleKeymapToggle property s "" +.X11Layout property s "cz,us" +.X11Model property s "pc105" +.X11Options property s "terminate:ctrl_alt_bksp,grp:shift_togg… +.X11Variant property s "qwerty,basic" +``` + +
+ ### Timezone +(Using the `Priority` design from above) + +``` +node /org/opensuse/Agama/TimeDate1 { + interface org.opensuse.Agama.TimeDate1 { + methods: + ListTimezones( + in s display_locale # "de_DE.UTF-8" + out a(ss) id_label_pairs # [('Europe/Prague', 'Europa/Prag')] + ) + # success? do we need a specific return value other than some Error? + Commit(); + properties: + readwrite (ys) Timezone = (42, 'Europe/Prague'); + readwrite (yb) LocalRTC = (42, false); + }; +}; +``` + #### Systemd +
+ +For reference, the systemd API for Time and Timezone is this: + + (I find `gdbus` verbose output better for methods and `busctl` terse output better for properties) @@ -216,27 +280,4 @@ NAME TYPE SIG RESULT/VALUE FLAGS "LocalRTC" means "is the local time zone used for the real time clock", so it's !hwclock_in_UTC -#### Design of Elementary Layer (lower) - -Just use the systemd API, don't add any API of our own. We will use -- .ListTimezones -- .SetLocalRTC -- .SetTimezone -- .LocalRTC -- .Timezone - -#### Design of Proposal Layer (upper) - -(Using the `Priority` design from above) - -``` -node ...Agama/TimeDate1 { - interface ...Agama.TimeDate1 { - methods: - # FIXME: add relevant methods common to all proposals - properties: - readwrite (ys) Timezone = (42, 'Europe/Prague'); - readwrite (yb) LocalRTC = (42, false); - }; -}; -``` +
diff --git a/doc/locale_api_WIP.md b/doc/locale_api_WIP.md deleted file mode 100644 index 43450e8db0..0000000000 --- a/doc/locale_api_WIP.md +++ /dev/null @@ -1,46 +0,0 @@ - -Note: ListFoo APIs combine the IDs and the translations. -If it's easier to return just the IDs and ask for the translations separately, -let's use such different API. - -### Locales - -``` -ListLocales( - out a(sss) id_english_native -) - -SetLocale(in s locale_id) -``` - -### Keyboards - -``` -ListX11Keyboards( - out as ids # id like "layout(variant)" -) - -SetX11Keyboard(in s kb_id) - -X11KeyboardForLocale( - in s locale_id - out s kb_id -) -``` - -### Timezones - - -``` -ListTimezones( - in s display_locale # "en_US.UTF-8" - out a(ss) id_label_pairs -) - -SetTimezone(in s tz_id) - -TimezoneForLocale( - in s locale_id - out s tz_id -) -``` From b336f338d7830cecefe925ada7bbfd2a45c9c849 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 25 Apr 2023 10:06:05 +0200 Subject: [PATCH 25/73] Adapt list of languages and locales method --- rust/agama-dbus-server/src/main.rs | 15 ++++++++------ rust/agama-locale-data/src/language.rs | 6 ++++-- rust/agama-locale-data/src/localization.rs | 12 ++++++----- rust/agama-locale-data/src/ranked.rs | 22 +++++++++++++++++---- rust/agama-locale-data/src/timezone_part.rs | 13 +++++++----- 5 files changed, 46 insertions(+), 22 deletions(-) diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 8fb498edbd..ee581b4844 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -13,13 +13,16 @@ struct Locale { #[dbus_interface(name = "org.opensuse.Agama.Locale1")] impl Locale { // Can be `async` as well. - fn list_locales(&self, locale: &str) -> Result, Error> { + fn list_locales(&self) -> Result)>, Error> { let locales = agama_locale_data::get_languages(); - // TODO: localization param - let ret = locales? - .language.iter() - .map(|l| (l.id.clone(), locale.to_string())) - .collect(); + let res = locales?.language.iter() + .map(|l| (l.id.clone(), l.names.name_for("en").unwrap_or_default(), + l.names.name_for(l.id.as_str()).unwrap_or_default(), + l.locales.locale.iter().map(|l| l.id.to_owned()).collect())) + // filter out missing translations as language that does not translate itself is very minor + .filter(|r| r.1 != "" && r.2 != "") + .collect() + Ok(ret) } diff --git a/rust/agama-locale-data/src/language.rs b/rust/agama-locale-data/src/language.rs index 04801d2082..9f42a0c539 100644 --- a/rust/agama-locale-data/src/language.rs +++ b/rust/agama-locale-data/src/language.rs @@ -1,12 +1,14 @@ use serde::Deserialize; -use crate::ranked::RankedTerritories; +use crate::ranked::{RankedTerritories, RankedLocales}; #[derive(Debug, Deserialize)] pub struct Language { #[serde(rename(deserialize = "languageId"))] pub id: String, - pub territories: RankedTerritories + pub territories: RankedTerritories, + pub locales: RankedLocales, + pub names: crate::localization::Localization } #[derive(Debug, Deserialize)] diff --git a/rust/agama-locale-data/src/localization.rs b/rust/agama-locale-data/src/localization.rs index e4bc9c5eba..bee46104ca 100644 --- a/rust/agama-locale-data/src/localization.rs +++ b/rust/agama-locale-data/src/localization.rs @@ -6,11 +6,13 @@ pub struct Localization { } impl Localization { - pub fn name_for(&self, language: &str) -> String { - self.name.iter() - .find(|n| n.language == language) - .expect("Wrong language {language}") - .value.clone() + pub fn name_for(&self, language: &str) -> Option { + let entry = self.name.iter() + .find(|n| n.language == language); + match entry { + Some(res) => Some(res.value.clone()), + None => None + } } } diff --git a/rust/agama-locale-data/src/ranked.rs b/rust/agama-locale-data/src/ranked.rs index 004af06291..f5d26a2c60 100644 --- a/rust/agama-locale-data/src/ranked.rs +++ b/rust/agama-locale-data/src/ranked.rs @@ -1,5 +1,4 @@ //! Bigger rank means it is more important - use serde::Deserialize; #[derive(Debug, Deserialize)] @@ -12,7 +11,8 @@ pub struct RankedLanguage { #[derive(Debug, Deserialize)] pub struct RankedLanguages { - pub language: Option> + #[serde(default)] + pub language: Vec } #[derive(Debug, Deserialize)] @@ -25,5 +25,19 @@ pub struct RankedTerritory { #[derive(Debug, Deserialize)] pub struct RankedTerritories { - pub territory: Option> -} \ No newline at end of file + #[serde(default)] + pub territory: Vec +} + +#[derive(Debug, Deserialize)] +pub struct RankedLocale { + #[serde(rename(deserialize = "localeId"))] + pub id: String, + pub rank: u16 +} + +#[derive(Debug, Deserialize)] +pub struct RankedLocales { + #[serde(default)] + pub locale: Vec +} diff --git a/rust/agama-locale-data/src/timezone_part.rs b/rust/agama-locale-data/src/timezone_part.rs index 47f97f860b..7948a9f186 100644 --- a/rust/agama-locale-data/src/timezone_part.rs +++ b/rust/agama-locale-data/src/timezone_part.rs @@ -22,12 +22,12 @@ pub struct TimezoneIdParts { impl TimezoneIdParts { /// Localized given list of timezones to given language /// # Examples - /// + /// /// ``` - /// let parts = agama_locale_data::get_timezone_parts(); + /// let parts = agama_locale_data::get_timezone_parts().expect("missing timezone parts"); /// let timezones = vec!["Europe/Prague".to_string(), "Europe/Berlin".to_string()]; /// let result = vec!["Evropa/Praha".to_string(), "Evropa/Berlín".to_string()]; - /// assert_eq!(parts.localize_timezones("cs", timezones), result); + /// assert_eq!(parts.localize_timezones("cs", &timezones), result); /// ``` pub fn localize_timezones(&self, language: &str, timezones: &Vec) -> Vec { let mapping = self.construct_mapping(language); @@ -39,7 +39,10 @@ impl TimezoneIdParts { self.timezone_part.iter() .map(|part| (part.id.clone(), part.names.name_for(language))) .for_each(|tuple| -> () { - res.insert(tuple.0, tuple.1); + // skip missing translations + if let Some(trans) = tuple.1 { + res.insert(tuple.0, trans); + } } ); return res @@ -50,4 +53,4 @@ impl TimezoneIdParts { .map(|tzp| mapping.get(&tzp.to_string()).expect(format!("Unknown timezone part {tzp}").as_str()).to_owned()) .collect::>().join("/") } -} \ No newline at end of file +} From 1656975cea00d38827710e8b8e3c974a0bdc02e1 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 26 Apr 2023 22:02:40 +0200 Subject: [PATCH 26/73] add new dbus methods --- rust/Cargo.lock | 1 + rust/agama-dbus-server/src/main.rs | 46 +++++++++++++------ rust/agama-locale-data/Cargo.toml | 1 + rust/agama-locale-data/src/lib.rs | 43 +++++++++++++++-- rust/agama-locale-data/src/territory.rs | 3 +- .../src/{keyboard.rs => xkeyboard.rs} | 6 +-- 6 files changed, 79 insertions(+), 21 deletions(-) rename rust/agama-locale-data/src/{keyboard.rs => xkeyboard.rs} (85%) diff --git a/rust/Cargo.lock b/rust/Cargo.lock index ddc4fe041b..eecc69875a 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -69,6 +69,7 @@ dependencies = [ "chrono-tz", "flate2", "quick-xml", + "regex", "serde", ] diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index ee581b4844..bf7e814f38 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -2,36 +2,46 @@ pub mod error; use crate::error::Error; use std::future::pending; +use agama_locale_data::{parse_locale}; +use anyhow::Context; use zbus::{ConnectionBuilder, dbus_interface}; struct Locale { locale_id: String, - keyboard_id: String, + keymap: String, timezone_id: String } #[dbus_interface(name = "org.opensuse.Agama.Locale1")] impl Locale { // Can be `async` as well. - fn list_locales(&self) -> Result)>, Error> { - let locales = agama_locale_data::get_languages(); - let res = locales?.language.iter() - .map(|l| (l.id.clone(), l.names.name_for("en").unwrap_or_default(), - l.names.name_for(l.id.as_str()).unwrap_or_default(), - l.locales.locale.iter().map(|l| l.id.to_owned()).collect())) - // filter out missing translations as language that does not translate itself is very minor - .filter(|r| r.1 != "" && r.2 != "") - .collect() + /// get labels for given locale. The first pair is english language and territory + /// and second one is localized one to target language from locale. + /// + fn labels_for_locale(&self, locale: &str) -> Result<((String, String), (String, String)), Error> { + const DEFAULT_LANG : &str = "en"; + let (loc_language, loc_territory) = parse_locale(locale)?; + let languages = agama_locale_data::get_languages()?; + let territories = agama_locale_data::get_territories()?; + let language = languages.language.iter() + .find(|l| l.id == loc_language).context("language for passed locale not found")?; + let territory = territories.territory.iter() + .find(|t| t.id == loc_territory).context("territory for passed locale not found")?; - Ok(ret) + let default_ret = (language.names.name_for(DEFAULT_LANG).context("missing default translation for language")?, + territory.names.name_for(DEFAULT_LANG).context("missing default translation for territory")?); + let localized_ret = (language.names.name_for(language.id.as_str()).context("missing default translation for language")?, + territory.names.name_for(language.id.as_str()).context("missing default translation for territory")?); + Ok((default_ret, localized_ret)) } fn set_locale(&mut self, locale: &str) { self.locale_id = locale.to_string(); } +/* support only keymaps for console for now fn list_x11_keyboards(&self) -> Result, Error> { - let keyboards = agama_locale_data::get_keyboards()?; + let keyboards = agama_locale_data::get_xkeyboards()?; let ret = keyboards .keyboard.iter() .map(|k| (k.id.clone(), k.description.clone())) @@ -42,6 +52,16 @@ impl Locale { fn set_x11_keyboard(&mut self, keyboard: &str) { self.keyboard_id = keyboard.to_string(); } +*/ + + fn list_keyboards(&self) -> Result, Error> { + let res = agama_locale_data::get_key_maps()?; + Ok(res) + } + + fn set_keyboard(&mut self, keyboard: &str) { + self.keymap = keyboard.to_string(); + } fn list_timezones(&self, locale: &str) -> Result, Error> { let timezones = agama_locale_data::get_timezones(); @@ -59,7 +79,7 @@ impl Locale { // Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] async fn main() -> Result<(), Box> { - let locale = Locale { locale_id: "en".to_string(), keyboard_id: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; + let locale = Locale { locale_id: "en".to_string(), keymap: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; let _conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one .name("org.opensuse.Agama.Locale1")? .serve_at("/org/opensuse/Agama/Locale1", locale)? diff --git a/rust/agama-locale-data/Cargo.toml b/rust/agama-locale-data/Cargo.toml index 48cc219562..46380041c5 100644 --- a/rust/agama-locale-data/Cargo.toml +++ b/rust/agama-locale-data/Cargo.toml @@ -11,3 +11,4 @@ serde = { version = "1.0.152", features = ["derive"] } quick-xml = { version = "0.28.2", features = ["serialize"] } flate2 = "1.0.25" chrono-tz = "0.8.2" +regex = "1" diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index c5679a521c..2b1071ef6a 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -3,10 +3,12 @@ use serde::Deserialize; use std::fs::File; use std::io::BufRead; use std::io::BufReader; +use std::process::Command; use quick_xml::de::Deserializer; use flate2::bufread::GzDecoder; +use regex::Regex; -pub mod keyboard; +pub mod xkeyboard; pub mod language; pub mod localization; pub mod territory; @@ -21,15 +23,48 @@ fn file_reader(file_path: &str) -> anyhow::Result { Ok(reader) } -pub fn get_keyboards() -> anyhow::Result { +pub fn get_xkeyboards() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/keyboards.xml.gz"; let reader = file_reader(FILE_PATH)?; let mut deserializer = Deserializer::from_reader(reader); - let ret = keyboard::Keyboards::deserialize(&mut deserializer) + let ret = xkeyboard::XKeyboards::deserialize(&mut deserializer) .context("Failed to deserialize keyboard entry")?; Ok(ret) } +/// Gets list of available keymaps +/// +/// ## Examples +/// +/// ``` +/// let key_maps = agama_locale_data::get_key_maps().unwrap(); +/// assert!(key_maps.contains(&"us".to_string())) +/// ``` +pub fn get_key_maps() -> anyhow::Result> { + const BINARY: &str = "/usr/bin/localectl"; + let output = Command::new(BINARY).arg("list-keymaps") + .output().context("failed to execute localectl list-maps")?.stdout; + let output = String::from_utf8(output).context("Strange localectl output formatting")?; + let ret = output.split('\n').map(|l| l.trim().to_string()).collect(); + + Ok(ret) +} + +/// Parses given locale to language and territory part +/// +/// /// ## Examples +/// +/// ``` +/// let result = agama_locale_data::parse_locale("en_US.UTF-8").unwrap(); +/// assert_eq!(result.0, "en"); +/// assert_eq!(result.1, "US") +/// ``` +pub fn parse_locale(locale: &str) -> anyhow::Result<(&str, &str)> { + let locale_regexp : Regex = Regex::new(r"^([[:alpha:]]+)_([[:alpha:]]+)").unwrap(); + let captures = locale_regexp.captures(locale).context("Failed to parse locale")?; + Ok((captures.get(1).unwrap().as_str(), captures.get(2).unwrap().as_str())) +} + pub fn get_languages() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/languages.xml.gz"; let reader = file_reader(FILE_PATH)?; @@ -70,7 +105,7 @@ mod tests { #[test] fn test_get_keyboards() { - let result = get_keyboards().unwrap(); + let result = get_xkeyboards().unwrap(); assert_eq!(result.keyboard.len(), 247); let first = result.keyboard.first().expect("no keyboards"); assert_eq!(first.id, "ad") diff --git a/rust/agama-locale-data/src/territory.rs b/rust/agama-locale-data/src/territory.rs index ed752456f1..fa5933e390 100644 --- a/rust/agama-locale-data/src/territory.rs +++ b/rust/agama-locale-data/src/territory.rs @@ -4,7 +4,8 @@ use serde::Deserialize; pub struct Territory { #[serde(rename(deserialize = "territoryId"))] pub id: String, - pub languages: crate::ranked::RankedLanguages + pub languages: crate::ranked::RankedLanguages, + pub names: crate::localization::Localization } #[derive(Debug, Deserialize)] diff --git a/rust/agama-locale-data/src/keyboard.rs b/rust/agama-locale-data/src/xkeyboard.rs similarity index 85% rename from rust/agama-locale-data/src/keyboard.rs rename to rust/agama-locale-data/src/xkeyboard.rs index e30da4eab3..b762aead83 100644 --- a/rust/agama-locale-data/src/keyboard.rs +++ b/rust/agama-locale-data/src/xkeyboard.rs @@ -3,7 +3,7 @@ use serde::Deserialize; use crate::ranked::{RankedTerritories, RankedLanguages}; #[derive(Debug, Deserialize)] -pub struct Keyboard { +pub struct XKeyboard { #[serde(rename(deserialize = "keyboardId"))] /// like "layout(variant)", for example "us" or "ua(phonetic)" pub id: String, @@ -16,6 +16,6 @@ pub struct Keyboard { } #[derive(Debug, Deserialize)] -pub struct Keyboards { - pub keyboard: Vec +pub struct XKeyboards { + pub keyboard: Vec } From 825fcb4b33ade0c8a3cc79723bd856305073815d Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 27 Apr 2023 10:08:36 +0200 Subject: [PATCH 27/73] Add commit API to write down localization --- rust/agama-dbus-server/src/main.rs | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index bf7e814f38..08be3e77c7 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -1,13 +1,13 @@ pub mod error; use crate::error::Error; -use std::future::pending; +use std::{future::pending, process::Command}; use agama_locale_data::{parse_locale}; use anyhow::Context; use zbus::{ConnectionBuilder, dbus_interface}; struct Locale { - locale_id: String, + locale: String, keymap: String, timezone_id: String } @@ -36,7 +36,7 @@ impl Locale { } fn set_locale(&mut self, locale: &str) { - self.locale_id = locale.to_string(); + self.locale = locale.to_string(); } /* support only keymaps for console for now @@ -74,12 +74,28 @@ impl Locale { fn set_timezone(&mut self, timezone: &str) { self.timezone_id = timezone.to_string(); } + + // TODO: what should be returned value for commit? + fn commit(&mut self) -> Result<(), Error> { + const ROOT : &str = "/mnt"; + Command::new("/usr/bin/systemd-firstboot") + .args(["root", ROOT, "--locale", self.locale.as_str()]) + .status().context("Failed to execute systemd-firstboot")?; + Command::new("/usr/bin/systemd-firstboot") + .args(["root", ROOT, "--keymap", self.keymap.as_str()]) + .status().context("Failed to execute systemd-firstboot")?; + Command::new("/usr/bin/systemd-firstboot") + .args(["root", ROOT, "--timezone", self.timezone_id.as_str()]) + .status().context("Failed to execute systemd-firstboot")?; + + Ok(()) + } } // Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] async fn main() -> Result<(), Box> { - let locale = Locale { locale_id: "en".to_string(), keymap: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; + let locale = Locale { locale: "en_US.UTF-8".to_string(), keymap: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; let _conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one .name("org.opensuse.Agama.Locale1")? .serve_at("/org/opensuse/Agama/Locale1", locale)? From f44300c4cb103f481c6208de1d37325c4e85e0f9 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 27 Apr 2023 18:32:47 +0200 Subject: [PATCH 28/73] move dbus object to own file --- rust/agama-dbus-server/src/locale.rs | 90 ++++++++++++++++++++++++++ rust/agama-dbus-server/src/main.rs | 95 ++-------------------------- 2 files changed, 94 insertions(+), 91 deletions(-) create mode 100644 rust/agama-dbus-server/src/locale.rs diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs new file mode 100644 index 0000000000..58a25749a9 --- /dev/null +++ b/rust/agama-dbus-server/src/locale.rs @@ -0,0 +1,90 @@ +use crate::error::Error; +use zbus::dbus_interface; +use anyhow::Context; +use std::process::Command; + +pub struct Locale { + pub locale: String, + pub keymap: String, + pub timezone_id: String +} + +#[dbus_interface(name = "org.opensuse.Agama.Locale1")] +impl Locale { + // Can be `async` as well. + /// get labels for given locale. The first pair is english language and territory + /// and second one is localized one to target language from locale. + /// + fn labels_for_locale(&self, locale: &str) -> Result<((String, String), (String, String)), Error> { + const DEFAULT_LANG : &str = "en"; + let (loc_language, loc_territory) = agama_locale_data::parse_locale(locale)?; + let languages = agama_locale_data::get_languages()?; + let territories = agama_locale_data::get_territories()?; + let language = languages.language.iter() + .find(|l| l.id == loc_language).context("language for passed locale not found")?; + let territory = territories.territory.iter() + .find(|t| t.id == loc_territory).context("territory for passed locale not found")?; + + let default_ret = (language.names.name_for(DEFAULT_LANG).context("missing default translation for language")?, + territory.names.name_for(DEFAULT_LANG).context("missing default translation for territory")?); + let localized_ret = (language.names.name_for(language.id.as_str()).context("missing default translation for language")?, + territory.names.name_for(language.id.as_str()).context("missing default translation for territory")?); + Ok((default_ret, localized_ret)) + } + + fn set_locale(&mut self, locale: &str) { + self.locale = locale.to_string(); + } + +/* support only keymaps for console for now + fn list_x11_keyboards(&self) -> Result, Error> { + let keyboards = agama_locale_data::get_xkeyboards()?; + let ret = keyboards + .keyboard.iter() + .map(|k| (k.id.clone(), k.description.clone())) + .collect(); + Ok(ret) + } + + fn set_x11_keyboard(&mut self, keyboard: &str) { + self.keyboard_id = keyboard.to_string(); + } +*/ + + fn list_keyboards(&self) -> Result, Error> { + let res = agama_locale_data::get_key_maps()?; + Ok(res) + } + + fn set_keyboard(&mut self, keyboard: &str) { + self.keymap = keyboard.to_string(); + } + + fn list_timezones(&self, locale: &str) -> Result, Error> { + let timezones = agama_locale_data::get_timezones(); + let localized = agama_locale_data::get_timezone_parts()? + .localize_timezones(locale, &timezones); + let ret = timezones.into_iter().zip(localized.into_iter()).collect(); + Ok(ret) + } + + fn set_timezone(&mut self, timezone: &str) { + self.timezone_id = timezone.to_string(); + } + + // TODO: what should be returned value for commit? + fn commit(&mut self) -> Result<(), Error> { + const ROOT : &str = "/mnt"; + Command::new("/usr/bin/systemd-firstboot") + .args(["root", ROOT, "--locale", self.locale.as_str()]) + .status().context("Failed to execute systemd-firstboot")?; + Command::new("/usr/bin/systemd-firstboot") + .args(["root", ROOT, "--keymap", self.keymap.as_str()]) + .status().context("Failed to execute systemd-firstboot")?; + Command::new("/usr/bin/systemd-firstboot") + .args(["root", ROOT, "--timezone", self.timezone_id.as_str()]) + .status().context("Failed to execute systemd-firstboot")?; + + Ok(()) + } +} diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 08be3e77c7..47df91d0fa 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -1,101 +1,14 @@ pub mod error; -use crate::error::Error; +pub mod locale; -use std::{future::pending, process::Command}; -use agama_locale_data::{parse_locale}; -use anyhow::Context; -use zbus::{ConnectionBuilder, dbus_interface}; +use std::future::pending; -struct Locale { - locale: String, - keymap: String, - timezone_id: String -} - -#[dbus_interface(name = "org.opensuse.Agama.Locale1")] -impl Locale { - // Can be `async` as well. - /// get labels for given locale. The first pair is english language and territory - /// and second one is localized one to target language from locale. - /// - fn labels_for_locale(&self, locale: &str) -> Result<((String, String), (String, String)), Error> { - const DEFAULT_LANG : &str = "en"; - let (loc_language, loc_territory) = parse_locale(locale)?; - let languages = agama_locale_data::get_languages()?; - let territories = agama_locale_data::get_territories()?; - let language = languages.language.iter() - .find(|l| l.id == loc_language).context("language for passed locale not found")?; - let territory = territories.territory.iter() - .find(|t| t.id == loc_territory).context("territory for passed locale not found")?; - - let default_ret = (language.names.name_for(DEFAULT_LANG).context("missing default translation for language")?, - territory.names.name_for(DEFAULT_LANG).context("missing default translation for territory")?); - let localized_ret = (language.names.name_for(language.id.as_str()).context("missing default translation for language")?, - territory.names.name_for(language.id.as_str()).context("missing default translation for territory")?); - Ok((default_ret, localized_ret)) - } - - fn set_locale(&mut self, locale: &str) { - self.locale = locale.to_string(); - } - -/* support only keymaps for console for now - fn list_x11_keyboards(&self) -> Result, Error> { - let keyboards = agama_locale_data::get_xkeyboards()?; - let ret = keyboards - .keyboard.iter() - .map(|k| (k.id.clone(), k.description.clone())) - .collect(); - Ok(ret) - } - - fn set_x11_keyboard(&mut self, keyboard: &str) { - self.keyboard_id = keyboard.to_string(); - } -*/ - - fn list_keyboards(&self) -> Result, Error> { - let res = agama_locale_data::get_key_maps()?; - Ok(res) - } - - fn set_keyboard(&mut self, keyboard: &str) { - self.keymap = keyboard.to_string(); - } - - fn list_timezones(&self, locale: &str) -> Result, Error> { - let timezones = agama_locale_data::get_timezones(); - let localized = agama_locale_data::get_timezone_parts()? - .localize_timezones(locale, &timezones); - let ret = timezones.into_iter().zip(localized.into_iter()).collect(); - Ok(ret) - } - - fn set_timezone(&mut self, timezone: &str) { - self.timezone_id = timezone.to_string(); - } - - // TODO: what should be returned value for commit? - fn commit(&mut self) -> Result<(), Error> { - const ROOT : &str = "/mnt"; - Command::new("/usr/bin/systemd-firstboot") - .args(["root", ROOT, "--locale", self.locale.as_str()]) - .status().context("Failed to execute systemd-firstboot")?; - Command::new("/usr/bin/systemd-firstboot") - .args(["root", ROOT, "--keymap", self.keymap.as_str()]) - .status().context("Failed to execute systemd-firstboot")?; - Command::new("/usr/bin/systemd-firstboot") - .args(["root", ROOT, "--timezone", self.timezone_id.as_str()]) - .status().context("Failed to execute systemd-firstboot")?; - - Ok(()) - } -} +use zbus::ConnectionBuilder; // Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] async fn main() -> Result<(), Box> { - let locale = Locale { locale: "en_US.UTF-8".to_string(), keymap: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; + let locale = crate::locale::Locale { locale: "en_US.UTF-8".to_string(), keymap: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; let _conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one .name("org.opensuse.Agama.Locale1")? .serve_at("/org/opensuse/Agama/Locale1", locale)? From a4819d8c0cf7efd478825b6dd43e071ebdeed7c1 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 27 Apr 2023 20:21:26 +0200 Subject: [PATCH 29/73] Adapt to API proposal --- rust/agama-dbus-server/src/locale.rs | 151 +++++++++++++++++++-------- rust/agama-dbus-server/src/main.rs | 2 +- 2 files changed, 109 insertions(+), 44 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index 58a25749a9..c8867d98df 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -1,12 +1,24 @@ use crate::error::Error; -use zbus::dbus_interface; use anyhow::Context; use std::process::Command; +use zbus::dbus_interface; pub struct Locale { - pub locale: String, - pub keymap: String, - pub timezone_id: String + locales: Vec, + keymap: String, + timezone_id: String, + supported_locales: Vec, +} + +impl Locale { + pub fn new() -> Locale { + Locale { + locales: vec!["en_US.UTF-8".to_string()], + keymap: "us".to_string(), + timezone_id: "America/Los_Angeles".to_string(), + supported_locales: vec!["en_US.UTF-8".to_string()], + } + } } #[dbus_interface(name = "org.opensuse.Agama.Locale1")] @@ -14,76 +26,129 @@ impl Locale { // Can be `async` as well. /// get labels for given locale. The first pair is english language and territory /// and second one is localized one to target language from locale. - /// - fn labels_for_locale(&self, locale: &str) -> Result<((String, String), (String, String)), Error> { - const DEFAULT_LANG : &str = "en"; - let (loc_language, loc_territory) = agama_locale_data::parse_locale(locale)?; - let languages = agama_locale_data::get_languages()?; - let territories = agama_locale_data::get_territories()?; - let language = languages.language.iter() - .find(|l| l.id == loc_language).context("language for passed locale not found")?; - let territory = territories.territory.iter() - .find(|t| t.id == loc_territory).context("territory for passed locale not found")?; - - let default_ret = (language.names.name_for(DEFAULT_LANG).context("missing default translation for language")?, - territory.names.name_for(DEFAULT_LANG).context("missing default translation for territory")?); - let localized_ret = (language.names.name_for(language.id.as_str()).context("missing default translation for language")?, - territory.names.name_for(language.id.as_str()).context("missing default translation for territory")?); - Ok((default_ret, localized_ret)) - } + /// + fn labels_for_locales(&self) -> Result, Error> { + const DEFAULT_LANG: &str = "en"; + let mut res = Vec::with_capacity(self.supported_locales.len()); + for locale in self.supported_locales.as_slice() { + let (loc_language, loc_territory) = agama_locale_data::parse_locale(locale.as_str())?; + let languages = agama_locale_data::get_languages()?; + let territories = agama_locale_data::get_territories()?; + let language = languages + .language + .iter() + .find(|l| l.id == loc_language) + .context("language for passed locale not found")?; + let territory = territories + .territory + .iter() + .find(|t| t.id == loc_territory) + .context("territory for passed locale not found")?; + + let default_ret = ( + language + .names + .name_for(DEFAULT_LANG) + .context("missing default translation for language")?, + territory + .names + .name_for(DEFAULT_LANG) + .context("missing default translation for territory")?, + ); + let localized_ret = ( + language + .names + .name_for(language.id.as_str()) + .context("missing native label for language")?, + territory + .names + .name_for(language.id.as_str()) + .context("missing native label for territory")?, + ); + res.push((default_ret, localized_ret)); + } - fn set_locale(&mut self, locale: &str) { - self.locale = locale.to_string(); + Ok(res) } -/* support only keymaps for console for now - fn list_x11_keyboards(&self) -> Result, Error> { - let keyboards = agama_locale_data::get_xkeyboards()?; - let ret = keyboards - .keyboard.iter() - .map(|k| (k.id.clone(), k.description.clone())) - .collect(); - Ok(ret) + #[dbus_interface(property)] + fn locales(&self) -> Vec { + return self.locales.iter().map(|l| l.clone()).collect(); } - fn set_x11_keyboard(&mut self, keyboard: &str) { - self.keyboard_id = keyboard.to_string(); + #[dbus_interface(property)] + fn set_locales(&mut self, locales: Vec) -> Result<(), zbus::fdo::Error> { + // verify that all locales are supported + self.locales = locales; + Ok(()) } -*/ + /* support only keymaps for console for now + fn list_x11_keyboards(&self) -> Result, Error> { + let keyboards = agama_locale_data::get_xkeyboards()?; + let ret = keyboards + .keyboard.iter() + .map(|k| (k.id.clone(), k.description.clone())) + .collect(); + Ok(ret) + } + + fn set_x11_keyboard(&mut self, keyboard: &str) { + self.keyboard_id = keyboard.to_string(); + } + */ + + #[dbus_interface(name="ListVConsoleKeyboards")] fn list_keyboards(&self) -> Result, Error> { let res = agama_locale_data::get_key_maps()?; Ok(res) } - fn set_keyboard(&mut self, keyboard: &str) { + #[dbus_interface(property, name="VConsoleKeyboard")] + fn keymap(&self) -> &str { + return &self.keymap.as_str(); + } + + #[dbus_interface(property, name="VConsoleKeyboard")] + fn set_keymap(&mut self, keyboard: &str) -> Result<(), zbus::fdo::Error> { self.keymap = keyboard.to_string(); + Ok(()) } fn list_timezones(&self, locale: &str) -> Result, Error> { let timezones = agama_locale_data::get_timezones(); - let localized = agama_locale_data::get_timezone_parts()? - .localize_timezones(locale, &timezones); + let localized = + agama_locale_data::get_timezone_parts()?.localize_timezones(locale, &timezones); let ret = timezones.into_iter().zip(localized.into_iter()).collect(); Ok(ret) } - fn set_timezone(&mut self, timezone: &str) { + #[dbus_interface(property)] + fn timezone(&self) -> &str { + return &self.timezone_id.as_str(); + } + + #[dbus_interface(property)] + fn set_timezone(&mut self, timezone: &str) -> Result<(), zbus::fdo::Error> { // NOTE: cannot use crate::Error as property expect this one self.timezone_id = timezone.to_string(); + Ok(()) } // TODO: what should be returned value for commit? fn commit(&mut self) -> Result<(), Error> { - const ROOT : &str = "/mnt"; + const ROOT: &str = "/mnt"; Command::new("/usr/bin/systemd-firstboot") - .args(["root", ROOT, "--locale", self.locale.as_str()]) - .status().context("Failed to execute systemd-firstboot")?; + .args(["root", ROOT, "--locale", self.locales.first().context("missing locale")?.as_str()]) + .status() + .context("Failed to execute systemd-firstboot")?; Command::new("/usr/bin/systemd-firstboot") .args(["root", ROOT, "--keymap", self.keymap.as_str()]) - .status().context("Failed to execute systemd-firstboot")?; + .status() + .context("Failed to execute systemd-firstboot")?; Command::new("/usr/bin/systemd-firstboot") .args(["root", ROOT, "--timezone", self.timezone_id.as_str()]) - .status().context("Failed to execute systemd-firstboot")?; + .status() + .context("Failed to execute systemd-firstboot")?; Ok(()) } diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 47df91d0fa..1fffe7f9dd 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -8,7 +8,7 @@ use zbus::ConnectionBuilder; // Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] async fn main() -> Result<(), Box> { - let locale = crate::locale::Locale { locale: "en_US.UTF-8".to_string(), keymap: "us".to_string(), timezone_id: "Europe/Prague".to_string() }; + let locale = crate::locale::Locale::new(); let _conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one .name("org.opensuse.Agama.Locale1")? .serve_at("/org/opensuse/Agama/Locale1", locale)? From c5dc09dcfda6c2bcf4b2843e5f45e65352a668ba Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 28 Apr 2023 09:27:54 +0200 Subject: [PATCH 30/73] move to own file also starting of service --- rust/agama-dbus-server/src/locale.rs | 15 +++++++++++++-- rust/agama-dbus-server/src/main.rs | 9 +-------- rust/agama-locale-data/src/lib.rs | 4 ---- rust/agama-locale-data/src/localization.rs | 7 ++----- 4 files changed, 16 insertions(+), 19 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index c8867d98df..3c80fba5b9 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -1,7 +1,7 @@ use crate::error::Error; use anyhow::Context; use std::process::Command; -use zbus::dbus_interface; +use zbus::{dbus_interface, Connection, ConnectionBuilder}; pub struct Locale { locales: Vec, @@ -11,7 +11,7 @@ pub struct Locale { } impl Locale { - pub fn new() -> Locale { + fn new() -> Locale { Locale { locales: vec!["en_US.UTF-8".to_string()], keymap: "us".to_string(), @@ -19,6 +19,17 @@ impl Locale { supported_locales: vec!["en_US.UTF-8".to_string()], } } + + pub async fn start_service() -> Result> { + let locale = Locale::new(); + let conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one + .name("org.opensuse.Agama.Locale1")? + .serve_at("/org/opensuse/Agama/Locale1", locale)? + .build() + .await?; + + Ok(conn) + } } #[dbus_interface(name = "org.opensuse.Agama.Locale1")] diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 1fffe7f9dd..1488c2b845 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -3,17 +3,10 @@ pub mod locale; use std::future::pending; -use zbus::ConnectionBuilder; - // Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] async fn main() -> Result<(), Box> { - let locale = crate::locale::Locale::new(); - let _conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one - .name("org.opensuse.Agama.Locale1")? - .serve_at("/org/opensuse/Agama/Locale1", locale)? - .build() - .await?; + crate::locale::Locale::start_service().await?; // Do other things or go to wait forever pending::<()>().await; diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 2b1071ef6a..49ed014891 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -106,7 +106,6 @@ mod tests { #[test] fn test_get_keyboards() { let result = get_xkeyboards().unwrap(); - assert_eq!(result.keyboard.len(), 247); let first = result.keyboard.first().expect("no keyboards"); assert_eq!(first.id, "ad") } @@ -114,7 +113,6 @@ mod tests { #[test] fn test_get_languages() { let result = get_languages().unwrap(); - assert_eq!(result.language.len(), 356); let first = result.language.first().expect("no keyboards"); assert_eq!(first.id, "aa") } @@ -122,7 +120,6 @@ mod tests { #[test] fn test_get_territories() { let result = get_territories().unwrap(); - assert_eq!(result.territory.len(), 257); let first = result.territory.first().expect("no keyboards"); assert_eq!(first.id, "001") // looks strange, but it is meta id for whole world } @@ -130,7 +127,6 @@ mod tests { #[test] fn test_get_timezone_parts() { let result = get_timezone_parts().unwrap(); - assert_eq!(result.timezone_part.len(), 441); let first = result.timezone_part.first().expect("no keyboards"); assert_eq!(first.id, "Abidjan") } diff --git a/rust/agama-locale-data/src/localization.rs b/rust/agama-locale-data/src/localization.rs index bee46104ca..a3622589a9 100644 --- a/rust/agama-locale-data/src/localization.rs +++ b/rust/agama-locale-data/src/localization.rs @@ -8,11 +8,8 @@ pub struct Localization { impl Localization { pub fn name_for(&self, language: &str) -> Option { let entry = self.name.iter() - .find(|n| n.language == language); - match entry { - Some(res) => Some(res.value.clone()), - None => None - } + .find(|n| n.language == language)?; + Some(entry.value.clone()) } } From da622ffedf78d7d910e034b5a31b477443dcdec7 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 28 Apr 2023 11:40:18 +0200 Subject: [PATCH 31/73] fix attaching to bus --- rust/agama-dbus-server/src/locale.rs | 63 ++++++++++++++++++---------- rust/agama-dbus-server/src/main.rs | 2 +- 2 files changed, 41 insertions(+), 24 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index 3c80fba5b9..bcfb91d315 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -10,34 +10,13 @@ pub struct Locale { supported_locales: Vec, } -impl Locale { - fn new() -> Locale { - Locale { - locales: vec!["en_US.UTF-8".to_string()], - keymap: "us".to_string(), - timezone_id: "America/Los_Angeles".to_string(), - supported_locales: vec!["en_US.UTF-8".to_string()], - } - } - - pub async fn start_service() -> Result> { - let locale = Locale::new(); - let conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one - .name("org.opensuse.Agama.Locale1")? - .serve_at("/org/opensuse/Agama/Locale1", locale)? - .build() - .await?; - - Ok(conn) - } -} - #[dbus_interface(name = "org.opensuse.Agama.Locale1")] impl Locale { // Can be `async` as well. /// get labels for given locale. The first pair is english language and territory /// and second one is localized one to target language from locale. /// + /// Note: check how often it is used and if often, it can be easily cached fn labels_for_locales(&self) -> Result, Error> { const DEFAULT_LANG: &str = "en"; let mut res = Vec::with_capacity(self.supported_locales.len()); @@ -84,7 +63,7 @@ impl Locale { #[dbus_interface(property)] fn locales(&self) -> Vec { - return self.locales.iter().map(|l| l.clone()).collect(); + return self.locales.to_owned(); } #[dbus_interface(property)] @@ -94,6 +73,18 @@ impl Locale { Ok(()) } + #[dbus_interface(property)] + fn supported_locales(&self) -> Vec { + return self.supported_locales.to_owned(); + } + + #[dbus_interface(property)] + fn set_supported_locales(&mut self, locales: Vec) -> Result<(), zbus::fdo::Error> { + self.supported_locales = locales; + // TODO: handle if current selected locale contain something that is no longer supported + Ok(()) + } + /* support only keymaps for console for now fn list_x11_keyboards(&self) -> Result, Error> { let keyboards = agama_locale_data::get_xkeyboards()?; @@ -164,3 +155,29 @@ impl Locale { Ok(()) } } + + +impl Locale { + fn new() -> Locale { + Locale { + locales: vec!["en_US.UTF-8".to_string()], + keymap: "us".to_string(), + timezone_id: "America/Los_Angeles".to_string(), + supported_locales: vec!["en_US.UTF-8".to_string()], + } + } + + pub async fn start_service() -> Result> { + let locale = Locale::new(); + let conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one + .name("org.opensuse.Agama.Locale1")? + .serve_at("/org/opensuse/Agama/Locale1", locale)? + .build() + .await?; + + eprintln!("service started {:?}", conn); + + Ok(conn) + } +} + diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 1488c2b845..2f1e74d434 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -6,7 +6,7 @@ use std::future::pending; // Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] async fn main() -> Result<(), Box> { - crate::locale::Locale::start_service().await?; + let _con = crate::locale::Locale::start_service().await?; // Do other things or go to wait forever pending::<()>().await; From f87274c140307169d8131275b50aca57b3693d86 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Fri, 28 Apr 2023 11:47:23 +0200 Subject: [PATCH 32/73] Improve on "Error: NameTaken" Now: > Error: Requesting name org.opensuse.Agama.Locale1 > > Caused by: > name already taken on the bus --- rust/agama-dbus-server/src/locale.rs | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index bcfb91d315..e1ebdaf1fe 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -168,12 +168,15 @@ impl Locale { } pub async fn start_service() -> Result> { + const SERVICE_NAME: &str = "org.opensuse.Agama.Locale1"; + let locale = Locale::new(); let conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one - .name("org.opensuse.Agama.Locale1")? + .name(SERVICE_NAME)? .serve_at("/org/opensuse/Agama/Locale1", locale)? .build() - .await?; + .await + .context(format!("Requesting name {SERVICE_NAME}"))?; eprintln!("service started {:?}", conn); From 19b3e0d452f94bb5d9e76250dc03f53fa16ae5a7 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 28 Apr 2023 11:48:57 +0200 Subject: [PATCH 33/73] remove debug call --- rust/agama-dbus-server/src/locale.rs | 2 -- 1 file changed, 2 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index e1ebdaf1fe..42f94bcb03 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -178,8 +178,6 @@ impl Locale { .await .context(format!("Requesting name {SERVICE_NAME}"))?; - eprintln!("service started {:?}", conn); - Ok(conn) } } From df5e240088b9490c8a4f0a30874edc89eff0076c Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 28 Apr 2023 22:58:55 +0200 Subject: [PATCH 34/73] first testing setter with validation --- rust/agama-dbus-server/src/locale.rs | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index 42f94bcb03..4e6cda8fde 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -113,6 +113,10 @@ impl Locale { #[dbus_interface(property, name="VConsoleKeyboard")] fn set_keymap(&mut self, keyboard: &str) -> Result<(), zbus::fdo::Error> { + let exist = agama_locale_data::get_key_maps().unwrap().iter().find(|&k| k == keyboard).is_some(); + if !exist { + return Err(zbus::fdo::Error::Failed("Invalid keyboard value".to_string())) + } self.keymap = keyboard.to_string(); Ok(()) } From 5116e8bc13f8001edb1d17b9f2c7c24895e4e131 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 28 Apr 2023 23:25:59 +0200 Subject: [PATCH 35/73] add more validation --- rust/agama-dbus-server/src/locale.rs | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index 4e6cda8fde..7099bf021b 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -68,7 +68,11 @@ impl Locale { #[dbus_interface(property)] fn set_locales(&mut self, locales: Vec) -> Result<(), zbus::fdo::Error> { - // verify that all locales are supported + for loc in &locales { + if !self.supported_locales.contains(&loc) { + return Err(zbus::fdo::Error::Failed(format!("Unsupported locale value '{loc}'").to_string())) + } + } self.locales = locales; Ok(()) } From 7186869bf24b238f096c818b6e152624e421b481 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 2 May 2023 12:11:23 +0200 Subject: [PATCH 36/73] testing spec changes --- rust/package/agama-cli.spec | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/rust/package/agama-cli.spec b/rust/package/agama-cli.spec index baf616ffa7..daaa2a8daf 100644 --- a/rust/package/agama-cli.spec +++ b/rust/package/agama-cli.spec @@ -23,7 +23,7 @@ Summary: Agama command line interface # If you know the license, put it's SPDX string here. # Alternately, you can use cargo lock2rpmprovides to help generate this. License: GPL-2.0-only -Url: https://github.com/yast/agama-cli +Url: https://github.com/opensuse/agama Source0: agama.tar Source1: vendor.tar.zst # Generated by the cargo_vendor OBS service @@ -36,6 +36,17 @@ Requires: lshw %description Command line program to interact with the agama service. +%package -n agama-dbus-server +# This will be set by osc services, that will run after this. +Version: 0 +Release: 0 +Summary: Agama rust dbus service +License: GPL-2.0-only +Url: https://github.com/opensuse/agama + +%description -n agama-dbus-server +DBus service for agama project. It provides so far localization service. + %prep %autosetup -a1 -n agama mkdir .cargo @@ -49,6 +60,7 @@ cp %{SOURCE2} .cargo/config %install install -D -d -m 0755 %{buildroot}%{_bindir} install -m 0755 %{_builddir}/agama/target/release/agama %{buildroot}%{_bindir}/agama +install -m 0755 %{_builddir}/agama/target/release/agama %{buildroot}%{_bindir}/agama-dbus-server install -D -d -m 0755 %{buildroot}%{_datadir}/agama-cli install -m 0644 %{_builddir}/agama/agama-lib/share/profile.schema.json %{buildroot}%{_datadir}/agama-cli @@ -60,4 +72,7 @@ install -m 0644 %{_builddir}/agama/agama-lib/share/profile.schema.json %{buildro %dir %{_datadir}/agama-cli %{_datadir}/agama-cli/profile.schema.json +%files -n agama-dbus-server +%{_bindir}/agama-dbus-server + %changelog From b699cf7939350a6e19b53870a1190bd2fdaf13b7 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Tue, 2 May 2023 15:02:02 +0200 Subject: [PATCH 37/73] API docs updated to reflect code, except Priority Human/Machine Priority is still not implemented --- doc/locale_api.md | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index 912decd8c7..ceff636df2 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -155,12 +155,14 @@ And legacy YaST has the console keyboard as the primary key. Must resolve this. node /org/opensuse/Agama/Locale1 { interface org.opensuse.Agama.Locale1 { methods: - ListLocales( - out a(sss) id_english_native # [('cs_CZ.UTF-8', 'Czech', 'Čeština'),…] + # In the same order as in SupportedLocales, pairs of + # (english_labels, native_labels), where foo_labels + # is a pair of (language, territory) + LabelsForLocales( + out a((ss)(ss)) id_english_native # [(("Spanish", "Spain"), ("Español", "España")), (('English', 'United States'), ('English', 'United States'))] ) - # langtable has X11 keyboards and no localization of their labels - ListX11Keyboards( - out as ids # id like "layout(variant)" + ListVConsoleKeyboards( + out as ids # like ["cz", "cz-qwerty", "gb-intl", "us", "us-dvorak",…] ) # ProposeKeyboard(); # not needed? adjusted automatically, same object @@ -170,11 +172,20 @@ node /org/opensuse/Agama/Locale1 { Commit(); properties: + + # The locale service DOES NOT KNOW which locales are + # available for the product currently selected for installation. + # When the user chooses a product, SupportedLocales should be set. + # It affects the output of LabelsForLocales + # and the valid inputs for Locales. + readwrite as SupportedLocales = ["es_ES.UTF-8", "en_US.UTF-8"]; + # NOTE: "as" has different meaning to systemd, # we have a list of LANG settings, 1st gets passed to systemd, # others affect package selection - readwrite as Locale = ['cs_CZ.UTF-8', 'de_DE.UTF-8']; - readwrite s VConsoleKeyboard = 'cz-qwerty'; + readwrite (yas) Locales = (23, ['cs_CZ.UTF-8', 'de_DE.UTF-8']); + + readwrite (ys) VConsoleKeyboard = (23, 'cz-qwerty'); }; }; ``` From 1ab3f3faf5b5944a08a2aecb7552cb6f791853ea Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 2 May 2023 17:23:47 +0200 Subject: [PATCH 38/73] add missing dependency --- rust/package/agama-cli.spec | 3 +++ 1 file changed, 3 insertions(+) diff --git a/rust/package/agama-cli.spec b/rust/package/agama-cli.spec index daaa2a8daf..0a5fc5628d 100644 --- a/rust/package/agama-cli.spec +++ b/rust/package/agama-cli.spec @@ -30,6 +30,8 @@ Source1: vendor.tar.zst Source2: cargo_config BuildRequires: cargo-packaging BuildRequires: pkgconfig(openssl) +# used in tests for dbus service +BuildRequires: langtable-data Requires: jsonnet Requires: lshw @@ -43,6 +45,7 @@ Release: 0 Summary: Agama rust dbus service License: GPL-2.0-only Url: https://github.com/opensuse/agama +Requires: langtable-data %description -n agama-dbus-server DBus service for agama project. It provides so far localization service. From a989b44cb7f2f84cc468dc05f753a91a2410c3e8 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 2 May 2023 21:48:58 +0200 Subject: [PATCH 39/73] Do not run get maps during spec build --- rust/agama-locale-data/src/lib.rs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 49ed014891..038ff175f5 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -35,8 +35,9 @@ pub fn get_xkeyboards() -> anyhow::Result { /// Gets list of available keymaps /// /// ## Examples +/// Requires working localectl. /// -/// ``` +/// ```no_run /// let key_maps = agama_locale_data::get_key_maps().unwrap(); /// assert!(key_maps.contains(&"us".to_string())) /// ``` From 6aa2246a0ae9d60935a2cf777548f17d6af6560a Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 2 May 2023 22:48:06 +0200 Subject: [PATCH 40/73] add locale dbus service file --- rust/package/agama-cli.spec | 3 +++ rust/share/org.opensuse.Agama.Locale1.service | 4 ++++ 2 files changed, 7 insertions(+) create mode 100644 rust/share/org.opensuse.Agama.Locale1.service diff --git a/rust/package/agama-cli.spec b/rust/package/agama-cli.spec index 0a5fc5628d..59a5bf0f54 100644 --- a/rust/package/agama-cli.spec +++ b/rust/package/agama-cli.spec @@ -66,6 +66,9 @@ install -m 0755 %{_builddir}/agama/target/release/agama %{buildroot}%{_bindir}/a install -m 0755 %{_builddir}/agama/target/release/agama %{buildroot}%{_bindir}/agama-dbus-server install -D -d -m 0755 %{buildroot}%{_datadir}/agama-cli install -m 0644 %{_builddir}/agama/agama-lib/share/profile.schema.json %{buildroot}%{_datadir}/agama-cli +install --directory %{buildroot}%{_datadir}/dbus-1/agama-services +install -m 0644 --target-directory=%{buildroot}%{_datadir}/dbus-1/agama-services %{_builddir}/agama/share/*.service + %check %{cargo_test} diff --git a/rust/share/org.opensuse.Agama.Locale1.service b/rust/share/org.opensuse.Agama.Locale1.service new file mode 100644 index 0000000000..35ed50eef0 --- /dev/null +++ b/rust/share/org.opensuse.Agama.Locale1.service @@ -0,0 +1,4 @@ +[D-BUS Service] +Name=org.opensuse.Agama.Locale1 +Exec=/usr/bin/agama-dbus-server +User=root From f88627ee3444909e26b4de4b087be6b966a5937d Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Tue, 2 May 2023 23:26:01 +0200 Subject: [PATCH 41/73] fix services packaging --- rust/package/agama-cli.spec | 3 +++ 1 file changed, 3 insertions(+) diff --git a/rust/package/agama-cli.spec b/rust/package/agama-cli.spec index 59a5bf0f54..c49a6ac357 100644 --- a/rust/package/agama-cli.spec +++ b/rust/package/agama-cli.spec @@ -32,6 +32,7 @@ BuildRequires: cargo-packaging BuildRequires: pkgconfig(openssl) # used in tests for dbus service BuildRequires: langtable-data +BuildRequires: dbus-1-common Requires: jsonnet Requires: lshw @@ -46,6 +47,7 @@ Summary: Agama rust dbus service License: GPL-2.0-only Url: https://github.com/opensuse/agama Requires: langtable-data +Requires: dbus-1-common %description -n agama-dbus-server DBus service for agama project. It provides so far localization service. @@ -80,5 +82,6 @@ install -m 0644 --target-directory=%{buildroot}%{_datadir}/dbus-1/agama-services %files -n agama-dbus-server %{_bindir}/agama-dbus-server +%{_datadir}/dbus-1/agama-services %changelog From 246a115b78c4d2ed670ff6e675ac2394f51d84b3 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 3 May 2023 09:39:28 +0200 Subject: [PATCH 42/73] update readme --- rust/README.md | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/rust/README.md b/rust/README.md index 565c496b25..4336742900 100644 --- a/rust/README.md +++ b/rust/README.md @@ -1,8 +1,9 @@ -# Agama Command Line Interface +# Agama Command Line and DBus Interface This project aims to build a command-line interface for [Agama](https://github.com/yast/agama), a service-based Linux installer featuring a nice -web interface. +web interface. The second aim is dbus service that does not depend heavily on YaST to +reduce memory consumption and also provide better performance. ## Code organization @@ -15,6 +16,9 @@ three packages: * [agama-cli](./agama-cli): code specific to the command line interface. * [agama-derive](./agama-derive): includes a [procedural macro](https://doc.rust-lang.org/reference/procedural-macros.html) to reduce the boilerplate code. +* [agama-locale-data](./agama-locale-data): specific library to provide data for localization dbus + API +* [agama-dbus-server](./agama-dbus-server): provides dbus API for services implemented in rust ## Status @@ -24,6 +28,8 @@ Agama CLI is still a work in progress, although it is already capable of doing a * Handling the auto-installation profiles. * Triggering the *probing* and the *installation* processes. +Agama DBus API is also a work in progress, but it is already used by Agama. + ## Installation You can grab the [RPM package](https://build.opensuse.org/package/show/YaST:Head:Agama/agama-cli) from @@ -32,14 +38,17 @@ the [YaST:Head:Agama](https://build.opensuse.org/project/show/YaST:Head:Agama) p If you prefer, you can install it from sources with [Cargo](https://doc.rust-lang.org/cargo/): ``` -git clone https://github.com/yast/agama-cli +git clone https://github.com/openSUSE/agama +cd rust cargo install --path . ``` ## Running -Take into account that you need to run `agama-cli` as root when you want to query or change the -Agama configuration. Assuming that the Agama D-Bus service is running, the next command +For DBus API just run as root agama-dbus-server binary and it will properly attach to DBus. + +For CLI take into account that you need to run `agama-cli` as root when you want to query or change +the Agama configuration. Assuming that the Agama D-Bus service is running, the next command prints the current settings using JSON (hint: you can use `jq` to make result look better): ``` @@ -102,3 +111,15 @@ frontend](./agama-cli/doc/backend-for-testing.md)*. ## Caveats * If no product is selected, the `probe` command fails. + +## Packaging + +A packaging files lives in `package` directory. Agama follows +[rust packaging guidelines](https://en.opensuse.org/openSUSE:Packaging_Rust_Software). +For testing changes to spec file use simple `osc branch YaST:Head:Agama agama-cli` +and copy modified spec file to that branch. +If it needs also specific code from git branch, then modify `_service` file and +change git branch name in `` tag. Then run `osc service runall`. +Note: for leap `cargo_audit` does not work with older python, so comment out that +service section for testing build. +For testing build use common `osc build` on modified sources. From 0851a975c1062bf00a378dde96a08bb468f2b521 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 3 May 2023 21:11:34 +0200 Subject: [PATCH 43/73] adapt dbus configuration to locale service --- service/share/dbus.conf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/service/share/dbus.conf b/service/share/dbus.conf index da06b7e009..47a6ef1f4c 100644 --- a/service/share/dbus.conf +++ b/service/share/dbus.conf @@ -36,7 +36,7 @@ - + @@ -44,7 +44,7 @@ - + From 95a3446b955dea3a5fb0bfe8cb55dfd88cf38218 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 3 May 2023 21:12:10 +0200 Subject: [PATCH 44/73] do not start language service anymore --- service/bin/agamactl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/service/bin/agamactl b/service/bin/agamactl index 930a02e4c0..eba447cc6d 100755 --- a/service/bin/agamactl +++ b/service/bin/agamactl @@ -63,7 +63,7 @@ def start_service(name) service_runner.run end -ORDERED_SERVICES = [:questions, :language, :software, :storage, :users, :manager].freeze +ORDERED_SERVICES = [:questions, :software, :storage, :users, :manager].freeze # Normally the services are started by D-Bus activation. # This starts all the services sequentially without relying on that. From c3c32907f883c67e1ab7b48f4dd346c67c6e8dad Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 3 May 2023 21:25:56 +0200 Subject: [PATCH 45/73] use agama bus for locale service --- rust/agama-dbus-server/src/locale.rs | 3 ++- rust/agama-lib/src/lib.rs | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index 7099bf021b..f8fe18bf1a 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -179,7 +179,8 @@ impl Locale { const SERVICE_NAME: &str = "org.opensuse.Agama.Locale1"; let locale = Locale::new(); - let conn = ConnectionBuilder::session()? //TODO: use agama bus instead of session one + const ADDRESS : &str = "unix:path=/run/agama/bus"; + let conn = ConnectionBuilder::address(ADDRESS)? //TODO: use agama bus instead of session one .name(SERVICE_NAME)? .serve_at("/org/opensuse/Agama/Locale1", locale)? .build() diff --git a/rust/agama-lib/src/lib.rs b/rust/agama-lib/src/lib.rs index c4803e1923..ab085a5f7b 100644 --- a/rust/agama-lib/src/lib.rs +++ b/rust/agama-lib/src/lib.rs @@ -16,8 +16,8 @@ use crate::error::ServiceError; use anyhow::Context; pub async fn connection() -> Result { - let path = "/run/agama/bus"; - let address = format!("unix:path={path}"); + const PATH : &str = "/run/agama/bus"; + let address : String = format!("unix:path={PATH}"); let conn = zbus::ConnectionBuilder::address(address.as_str())? .build() .await From 7d470abc4dd8c672998db5a74aa0dacc12587b5c Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 3 May 2023 22:13:14 +0200 Subject: [PATCH 46/73] adapt clients to use locale instead of language --- .../dbus/clients/{language.rb => locale.rb} | 38 ++++++------------- service/lib/agama/dbus/software/manager.rb | 4 +- 2 files changed, 14 insertions(+), 28 deletions(-) rename service/lib/agama/dbus/clients/{language.rb => locale.rb} (58%) diff --git a/service/lib/agama/dbus/clients/language.rb b/service/lib/agama/dbus/clients/locale.rb similarity index 58% rename from service/lib/agama/dbus/clients/language.rb rename to service/lib/agama/dbus/clients/locale.rb index 35a28515e4..ff27aaf19c 100644 --- a/service/lib/agama/dbus/clients/language.rb +++ b/service/lib/agama/dbus/clients/locale.rb @@ -24,54 +24,40 @@ module Agama module DBus module Clients - # D-Bus client for language configuration - class Language < Base + # D-Bus client for locale configuration + class Locale < Base def initialize super - @dbus_object = service.object("/org/opensuse/Agama/Language1") + @dbus_object = service.object("/org/opensuse/Agama/Locale1") @dbus_object.introspect end def service_name - @service_name ||= "org.opensuse.Agama.Language1" + @service_name ||= "org.opensuse.Agama.Locale1" end - # Available languages for the installation + # Sets the supported locales. It can differs per product. # - # @return [Array>] id and name of each language - def available_languages - dbus_object["org.opensuse.Agama.Language1"]["AvailableLanguages"].map { |l| l[0..1] } - end - - # Languages selected to install - # - # @return [Array] ids of the languages - def selected_languages - dbus_object["org.opensuse.Agama.Language1"]["MarkedForInstall"] - end - - # Selects the languages to install - # - # @param ids [Array] - def select_languages(ids) - dbus_object.ToInstall(ids) + # @param locales [Array] + def supported_locales=(locales) + dbus_object.supported_locales = locales end # Finishes the language installation def finish - dbus_object.Finish + dbus_object.Commit end - # Registers a callback to run when the language changes + # Registers a callback to run when the selected locales changes # # @note Signal subscription is done only once. Otherwise, the latest subscription overrides # the previous one. # - # @param block [Proc] Callback to run when a language is selected + # @param block [Proc] Callback to run when a locales are selected def on_language_selected(&block) on_properties_change(dbus_object) do |_, changes, _| - languages = changes["MarkedForInstall"] + languages = changes["Locales"] block.call(languages) end end diff --git a/service/lib/agama/dbus/software/manager.rb b/service/lib/agama/dbus/software/manager.rb index d290f453db..eac0ff69c9 100644 --- a/service/lib/agama/dbus/software/manager.rb +++ b/service/lib/agama/dbus/software/manager.rb @@ -22,7 +22,7 @@ require "dbus" require "agama/dbus/base_object" require "agama/dbus/with_service_status" -require "agama/dbus/clients/language" +require "agama/dbus/clients/locale" require "agama/dbus/clients/network" require "agama/dbus/interfaces/progress" require "agama/dbus/interfaces/service_status" @@ -135,7 +135,7 @@ def finish # Registers callback to be called def register_callbacks - lang_client = Agama::DBus::Clients::Language.new + lang_client = Agama::DBus::Clients::Locale.new lang_client.on_language_selected do |language_ids| backend.languages = language_ids end From 5d61885097358a22763dd1f20f479f861e5317a2 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Thu, 4 May 2023 10:14:06 +0200 Subject: [PATCH 47/73] Separate bus connection from API serving for better error messages ... in case the bus is not running: Before: > Error: Requesting name org.opensuse.Agama.Locale1 > > Caused by: > 0: I/O error: No such file or directory (os error 2) > 1: No such file or directory (os error 2) > After: > Error: Connecting to the Agama bus at unix:path=/run/agama/bus > > Caused by: > 0: I/O error: No such file or directory (os error 2) > 1: No such file or directory (os error 2) > --- rust/agama-dbus-server/src/locale.rs | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index f8fe18bf1a..2bc2e05cd7 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -176,15 +176,26 @@ impl Locale { } pub async fn start_service() -> Result> { + const ADDRESS : &str = "unix:path=/run/agama/bus"; const SERVICE_NAME: &str = "org.opensuse.Agama.Locale1"; + const SERVICE_PATH: &str = "/org/opensuse/Agama/Locale1"; - let locale = Locale::new(); - const ADDRESS : &str = "unix:path=/run/agama/bus"; - let conn = ConnectionBuilder::address(ADDRESS)? //TODO: use agama bus instead of session one - .name(SERVICE_NAME)? - .serve_at("/org/opensuse/Agama/Locale1", locale)? + // First connect to the Agama bus, then serve our API, + // for better error reporting. + let conn = ConnectionBuilder::address(ADDRESS)? .build() .await + .context(format!("Connecting to the Agama bus at {ADDRESS}"))?; + + // When serving, request the service name _after_ exposing the main object + let locale = Locale::new(); + conn + .object_server() + .at(SERVICE_PATH, locale) + .await?; + conn + .request_name(SERVICE_NAME) + .await .context(format!("Requesting name {SERVICE_NAME}"))?; Ok(conn) From 45e454505d86e099ed59a425830196756029e73c Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Thu, 4 May 2023 10:57:38 +0200 Subject: [PATCH 48/73] postpone using Priority --- doc/locale_api.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index ceff636df2..5a967ccb7c 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -77,7 +77,7 @@ simply set what we consider a good default for a newly set locale, because: ### Detailed Design: Prioritize -But other cases may not be as simple, so here's a generic design: +But other cases may not be as simple, so here's a generic design (NOT YET USED): All settings are wrapped in a `Priority` generic type (an Enum in Rust), meaning, what is the source and importance of the setting: @@ -183,9 +183,9 @@ node /org/opensuse/Agama/Locale1 { # NOTE: "as" has different meaning to systemd, # we have a list of LANG settings, 1st gets passed to systemd, # others affect package selection - readwrite (yas) Locales = (23, ['cs_CZ.UTF-8', 'de_DE.UTF-8']); + readwrite as Locales = ['cs_CZ.UTF-8', 'de_DE.UTF-8']; - readwrite (ys) VConsoleKeyboard = (23, 'cz-qwerty'); + readwrite s VConsoleKeyboard = 'cz-qwerty'; }; }; ``` @@ -230,8 +230,6 @@ $ busctl --system introspect org.freedesktop.locale1 /org/freedesktop/locale1 ### Timezone -(Using the `Priority` design from above) - ``` node /org/opensuse/Agama/TimeDate1 { interface org.opensuse.Agama.TimeDate1 { @@ -243,8 +241,8 @@ node /org/opensuse/Agama/TimeDate1 { # success? do we need a specific return value other than some Error? Commit(); properties: - readwrite (ys) Timezone = (42, 'Europe/Prague'); - readwrite (yb) LocalRTC = (42, false); + readwrite s Timezone = 'Europe/Prague'; + readwrite b LocalRTC = false; }; }; ``` From 5f2f314751aff8e93d3d74502df06f8acbaa762f Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 4 May 2023 22:17:44 +0200 Subject: [PATCH 49/73] adapt to renamed langtable-data --- rust/package/agama-cli.spec | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rust/package/agama-cli.spec b/rust/package/agama-cli.spec index c49a6ac357..3726229d3a 100644 --- a/rust/package/agama-cli.spec +++ b/rust/package/agama-cli.spec @@ -31,7 +31,7 @@ Source2: cargo_config BuildRequires: cargo-packaging BuildRequires: pkgconfig(openssl) # used in tests for dbus service -BuildRequires: langtable-data +BuildRequires: python-langtable-data BuildRequires: dbus-1-common Requires: jsonnet Requires: lshw @@ -46,8 +46,8 @@ Release: 0 Summary: Agama rust dbus service License: GPL-2.0-only Url: https://github.com/opensuse/agama -Requires: langtable-data -Requires: dbus-1-common +Requires: python-langtable-data +Requires: dbus-1-common %description -n agama-dbus-server DBus service for agama project. It provides so far localization service. From 9f657e53b05bffea62b593e10ba108af3cfd2c60 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 4 May 2023 22:18:02 +0200 Subject: [PATCH 50/73] first attempt to adapt web ui --- web/src/client/language.js | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/web/src/client/language.js b/web/src/client/language.js index c486c27d43..05418b07a7 100644 --- a/web/src/client/language.js +++ b/web/src/client/language.js @@ -22,9 +22,9 @@ // @ts-check import DBusClient from "./dbus"; -const LANGUAGE_SERVICE = "org.opensuse.Agama.Language1"; -const LANGUAGE_IFACE = "org.opensuse.Agama.Language1"; -const LANGUAGE_PATH = "/org/opensuse/Agama/Language1"; +const LANGUAGE_SERVICE = "org.opensuse.Agama.Locale1"; +const LANGUAGE_IFACE = "org.opensuse.Agama.Locale1"; +const LANGUAGE_PATH = "/org/opensuse/Agama/Locale1"; /** * @typedef {object} Language @@ -50,9 +50,12 @@ class LanguageClient { */ async getLanguages() { const proxy = await this.client.proxy(LANGUAGE_IFACE); - return proxy.AvailableLanguages.map(lang => { - const [id, name] = lang; - return { id, name }; + const locales = proxy.SupportedLocales; + const labels = proxy.LabelsForLocales; + return locales.map((locale, index) => { + // labels structure is [[en_lang, en_territory], [native_lang, native_territory]] + const [[en_lang,], [,]] = labels[index]; + return { locale, en_lang }; }); } @@ -63,7 +66,7 @@ class LanguageClient { */ async getSelectedLanguages() { const proxy = await this.client.proxy(LANGUAGE_IFACE); - return proxy.MarkedForInstall; + return proxy.Locale; } /** @@ -74,7 +77,7 @@ class LanguageClient { */ async setLanguages(langIDs) { const proxy = await this.client.proxy(LANGUAGE_IFACE); - return proxy.ToInstall(langIDs); + proxy.Locale = langIDs; } /** @@ -85,7 +88,7 @@ class LanguageClient { */ onLanguageChange(handler) { return this.client.onObjectChanged(LANGUAGE_PATH, LANGUAGE_IFACE, changes => { - const selected = changes.MarkedForInstall.v[0]; + const selected = changes.Locale.v[0]; handler(selected); }); } From d42ebaaa31309432af74b9c6428ad94d5deb94ad Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 4 May 2023 22:18:17 +0200 Subject: [PATCH 51/73] first attempt to adapt setup-service.sh --- setup-service.sh | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/setup-service.sh b/setup-service.sh index 52417f1b15..2c13b884c5 100755 --- a/setup-service.sh +++ b/setup-service.sh @@ -39,6 +39,12 @@ sudosed() { bundle install ) +# - build also rust service +( + cd $MYDIR/rust + cargo build +) + # - D-Bus configuration $SUDO cp -v $MYDIR/service/share/dbus.conf /usr/share/dbus-1/agama.conf @@ -54,6 +60,20 @@ $SUDO cp -v $MYDIR/service/share/dbus.conf /usr/share/dbus-1/agama.conf done sudosed "s@\(ExecStart\)=/usr/bin/@\1=$MYDIR/service/bin/@" \ systemd.service /usr/lib/systemd/system/agama.service +) + +# and same for rust service +( + cd $MYDIR/rust/share + DBUSDIR=/usr/share/dbus-1/agama-services + for SVC in org.opensuse.Agama*.service; do + # it is intention to use debug here to get more useful debugging output + sudosed "s@\(Exec\)=/usr/bin/@\1=$MYDIR/rust/target/debug/@" $SVC $DBUSDIR/$SVC + done +) + +# systemd reload and start of service +( $SUDO systemctl daemon-reload # Start the separate dbus-daemon for Agama $SUDO systemctl start agama.service From 699979c6218bbe30b2ad3b55e333b11f04d5100c Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Thu, 4 May 2023 22:52:52 +0200 Subject: [PATCH 52/73] more adaptation to new locale service --- service/lib/agama/manager.rb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/service/lib/agama/manager.rb b/service/lib/agama/manager.rb index bbf233a7f6..e9f8eb765a 100644 --- a/service/lib/agama/manager.rb +++ b/service/lib/agama/manager.rb @@ -25,7 +25,7 @@ require "agama/with_progress" require "agama/installation_phase" require "agama/service_status_recorder" -require "agama/dbus/clients/language" +require "agama/dbus/clients/locale" require "agama/dbus/clients/software" require "agama/dbus/clients/storage" require "agama/dbus/clients/users" @@ -124,9 +124,9 @@ def software # Language manager # - # @return [DBus::Clients::Language] + # @return [DBus::Clients::Locale] def language - @language ||= DBus::Clients::Language.new + @language ||= DBus::Clients::Locale.new end # Users client From 0c082adf45f78517185e86d40f6299b195560a2b Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 5 May 2023 22:39:56 +0200 Subject: [PATCH 53/73] more adaptation for new dbus interface --- web/src/client/language.js | 10 +++++----- web/src/components/overview/L10nSection.jsx | 3 +++ 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/web/src/client/language.js b/web/src/client/language.js index 05418b07a7..d8a4e47a6f 100644 --- a/web/src/client/language.js +++ b/web/src/client/language.js @@ -51,11 +51,11 @@ class LanguageClient { async getLanguages() { const proxy = await this.client.proxy(LANGUAGE_IFACE); const locales = proxy.SupportedLocales; - const labels = proxy.LabelsForLocales; + const labels = await proxy.LabelsForLocales(); return locales.map((locale, index) => { // labels structure is [[en_lang, en_territory], [native_lang, native_territory]] const [[en_lang,], [,]] = labels[index]; - return { locale, en_lang }; + return { id: locale, name: en_lang }; }); } @@ -66,7 +66,7 @@ class LanguageClient { */ async getSelectedLanguages() { const proxy = await this.client.proxy(LANGUAGE_IFACE); - return proxy.Locale; + return proxy.Locales; } /** @@ -77,7 +77,7 @@ class LanguageClient { */ async setLanguages(langIDs) { const proxy = await this.client.proxy(LANGUAGE_IFACE); - proxy.Locale = langIDs; + proxy.Locales = langIDs; } /** @@ -88,7 +88,7 @@ class LanguageClient { */ onLanguageChange(handler) { return this.client.onObjectChanged(LANGUAGE_PATH, LANGUAGE_IFACE, changes => { - const selected = changes.Locale.v[0]; + const selected = changes.Locales.v[0]; handler(selected); }); } diff --git a/web/src/components/overview/L10nSection.jsx b/web/src/components/overview/L10nSection.jsx index e72a771304..358f28c726 100644 --- a/web/src/components/overview/L10nSection.jsx +++ b/web/src/components/overview/L10nSection.jsx @@ -63,6 +63,9 @@ export default function L10nSection({ showErrors }) { if (busy) return ; const selected = languages.find(lang => lang.id === language); + console.log(selected); + console.log(language); + console.log(languages); return ( From b833993294a655c29173d9492020e8d3de5be209 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 5 May 2023 22:40:47 +0200 Subject: [PATCH 54/73] remove accidental console logs --- web/src/components/overview/L10nSection.jsx | 3 --- 1 file changed, 3 deletions(-) diff --git a/web/src/components/overview/L10nSection.jsx b/web/src/components/overview/L10nSection.jsx index 358f28c726..e72a771304 100644 --- a/web/src/components/overview/L10nSection.jsx +++ b/web/src/components/overview/L10nSection.jsx @@ -63,9 +63,6 @@ export default function L10nSection({ showErrors }) { if (busy) return ; const selected = languages.find(lang => lang.id === language); - console.log(selected); - console.log(language); - console.log(languages); return ( From 1fdb0eb4856fc242dd8b684d5ceaef2d4e047b8a Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Fri, 5 May 2023 23:01:14 +0200 Subject: [PATCH 55/73] document how to test using container --- README.md | 4 +++ doc/testing_using_container.md | 47 ++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) create mode 100644 doc/testing_using_container.md diff --git a/README.md b/README.md index 74ade8aa26..a28a6ace75 100644 --- a/README.md +++ b/README.md @@ -108,6 +108,10 @@ Alternatively you can run a development server which works as a proxy for the cockpit server. See more details [in the documentation]( web/README.md#using-a-development-server). +Another alternative is to run source checkout inside container so system is not +affected by doing testing run beside real actions really done by installer. +See more details [in the documentation][doc/testing_using_container.md]. + * Start the services: * beware that Agama must run as root (like YaST does) to do hardware probing, partition the disks, install the software and so on. diff --git a/doc/testing_using_container.md b/doc/testing_using_container.md new file mode 100644 index 0000000000..e21d8ef68c --- /dev/null +++ b/doc/testing_using_container.md @@ -0,0 +1,47 @@ +## Testing Using Container + +To test complex change that affects multiple parts of agama it is possible to +run from sources using container that is used to run CI. + +Below is shell script that start container, provides web UI on port 9090 and +also gives root access to container for more testing. + +```sh +# https://build.opensuse.org/package/show/YaST:Head:Containers/agama-testing +CIMAGE=registry.opensuse.org/yast/head/containers/containers_tumbleweed/opensuse/agama-testing:latest +# rename this if you test multiple things +CNAME=agama +# the '?' here will report a shell error +# if you accidentally paste a command without setting the variable first +echo ${CNAME?} + +test -f service/agama.gemspec || echo "You should run this from a checkout of agama" + +# destroy the previous instance, can fail if there is no previous instance +podman stop ${CNAME?} +podman rm ${CNAME?} + +# Update our image +podman pull ${CIMAGE?} + +podman run --name ${CNAME?} \ + --privileged --detach --ipc=host \ + -v .:/checkout \ + -p 9090:9090 \ + ${CIMAGE?} + +# shortcut for the following +CEXEC="podman exec ${CNAME?} bash -c" + +${CEXEC?} "cd /checkout && ./setup.sh" + +# Now the CLI is in the same repo, just symlink it +${CEXEC?} "ln -sfv /checkout/./rust/target/debug/agama /usr/bin/agama" + +# Manually start cockpit as socket activation does not work with port forwarding +${CEXEC?} "systemctl start cockpit" + +# Optional: Interactive shell in the container +podman exec --tty --interactive ${CNAME?} bash + +``` From bb5aeb2e4d7c9cb3b783b7542392450604f94ed7 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 10 May 2023 15:51:07 +0200 Subject: [PATCH 56/73] Fix tests for ruby client proxy (language -> locale) --- .../{language_test.rb => locale_test.rb} | 51 ++++--------------- 1 file changed, 10 insertions(+), 41 deletions(-) rename service/test/agama/dbus/clients/{language_test.rb => locale_test.rb} (57%) diff --git a/service/test/agama/dbus/clients/language_test.rb b/service/test/agama/dbus/clients/locale_test.rb similarity index 57% rename from service/test/agama/dbus/clients/language_test.rb rename to service/test/agama/dbus/clients/locale_test.rb index 93e469d180..21b47bcdce 100644 --- a/service/test/agama/dbus/clients/language_test.rb +++ b/service/test/agama/dbus/clients/locale_test.rb @@ -20,17 +20,17 @@ # find current contact information at www.suse.com. require_relative "../../../test_helper" -require "agama/dbus/clients/language" +require "agama/dbus/clients/locale" require "dbus" -describe Agama::DBus::Clients::Language do +describe Agama::DBus::Clients::Locale do before do allow(Agama::DBus::Bus).to receive(:current).and_return(bus) - allow(bus).to receive(:service).with("org.opensuse.Agama.Language1").and_return(service) - allow(service).to receive(:object).with("/org/opensuse/Agama/Language1") + allow(bus).to receive(:service).with("org.opensuse.Agama.Locale1").and_return(service) + allow(service).to receive(:object).with("/org/opensuse/Agama/Locale1") .and_return(dbus_object) allow(dbus_object).to receive(:introspect) - allow(dbus_object).to receive(:[]).with("org.opensuse.Agama.Language1") + allow(dbus_object).to receive(:[]).with("org.opensuse.Agama.Locale1") .and_return(lang_iface) end @@ -41,44 +41,13 @@ subject { described_class.new } - describe "#available_languages" do - before do - allow(lang_iface).to receive(:[]).with("AvailableLanguages").and_return( - [ - ["en_US", "English (US)", {}], - ["en_GB", "English (UK)", {}], - ["es_ES", "Español", {}] - ] - ) - end - - it "returns the id and name for all available languages" do - expect(subject.available_languages).to contain_exactly( - ["en_US", "English (US)"], - ["en_GB", "English (UK)"], - ["es_ES", "Español"] - ) - end - end - - describe "#selected_languages" do - before do - allow(lang_iface).to receive(:[]).with("MarkedForInstall").and_return(["en_US", "es_ES"]) - end - - it "returns the name of the selected languages" do - expect(subject.selected_languages).to contain_exactly("en_US", "es_ES") - end - end - - describe "#select_languages" do + describe "#supported_locales=" do # Using partial double because methods are dynamically added to the proxy object let(:dbus_object) { double(::DBus::ProxyObject) } - it "selects the given languages" do - expect(dbus_object).to receive(:ToInstall).with(["en_GB"]) - - subject.select_languages(["en_GB"]) + it "calls the D-Bus object" do + expect(dbus_object).to receive(:supported_locales=).with(["no", "se"]) + subject.supported_locales = ["no", "se"] end end @@ -86,7 +55,7 @@ let(:dbus_object) { double(::DBus::ProxyObject) } it "calls the D-Bus finish method" do - expect(dbus_object).to receive(:Finish) + expect(dbus_object).to receive(:Commit) subject.finish end end From 21ccc212005e777e048a510af8ade59b043a9bd8 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 10 May 2023 16:59:22 +0200 Subject: [PATCH 57/73] Fix tests for manager: language -> locale --- service/test/agama/manager_test.rb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/service/test/agama/manager_test.rb b/service/test/agama/manager_test.rb index 69cf6f555f..ea4551a257 100644 --- a/service/test/agama/manager_test.rb +++ b/service/test/agama/manager_test.rb @@ -47,7 +47,7 @@ write: nil, on_service_status_change: nil, valid?: true ) end - let(:language) { instance_double(Agama::DBus::Clients::Language, finish: nil) } + let(:locale) { instance_double(Agama::DBus::Clients::Locale, finish: nil) } let(:network) { instance_double(Agama::Network, install: nil) } let(:storage) do instance_double( @@ -60,7 +60,7 @@ before do allow(Agama::Network).to receive(:new).and_return(network) - allow(Agama::DBus::Clients::Language).to receive(:new).and_return(language) + allow(Agama::DBus::Clients::Locale).to receive(:new).and_return(locale) allow(Agama::DBus::Clients::Software).to receive(:new).and_return(software) allow(Agama::DBus::Clients::Storage).to receive(:new).and_return(storage) allow(Agama::DBus::Clients::Users).to receive(:new).and_return(users) @@ -118,7 +118,7 @@ expect(network).to receive(:install) expect(software).to receive(:install) expect(software).to receive(:finish) - expect(language).to receive(:finish) + expect(locale).to receive(:finish) expect(storage).to receive(:install) expect(storage).to receive(:finish) expect(users).to receive(:write) From 706c35f0930500f5d808fc38fad2f183a219c8b7 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 10 May 2023 17:00:00 +0200 Subject: [PATCH 58/73] Fix tests for Language UI --- web/cspell.json | 1 + web/src/client/language.test.js | 12 ++++++++---- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/web/cspell.json b/web/cspell.json index dec371fe84..c98e662255 100644 --- a/web/cspell.json +++ b/web/cspell.json @@ -25,6 +25,7 @@ "dasd", "dasds", "dbus", + "España", "filecontent", "filename", "fullname", diff --git a/web/src/client/language.test.js b/web/src/client/language.test.js index e23399ed65..ebe05fb712 100644 --- a/web/src/client/language.test.js +++ b/web/src/client/language.test.js @@ -29,9 +29,10 @@ jest.mock("./dbus"); const langProxy = { wait: jest.fn(), - AvailableLanguages: [ - ["cs_CZ", "Cestina", {}] - ] + SupportedLocales: ["es_ES.UTF-8", "en_US.UTF-8"], + LabelsForLocales: jest.fn().mockResolvedValue( + [[["Spanish", "Spain"], ["Español", "España"]], [['English', 'United States'], ['English', 'United States']]] + ), }; jest.mock("./dbus"); @@ -47,6 +48,9 @@ describe("#getLanguages", () => { it("returns the list of available languages", async () => { const client = new LanguageClient(); const availableLanguages = await client.getLanguages(); - expect(availableLanguages).toEqual([{ id: "cs_CZ", name: "Cestina" }]); + expect(availableLanguages).toEqual([ + { id: "es_ES.UTF-8", name: "Spanish" }, + { id: "en_US.UTF-8", name: "English" } + ]); }); }); From 4ce2c17cb8395030090910fec6cf2e5b53d8cd34 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Thu, 11 May 2023 14:59:48 +0200 Subject: [PATCH 59/73] setup.sh: install python-langtable-data this only helps for local testing, not for CI --- setup.sh | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/setup.sh b/setup.sh index 00d47f9b19..f03f437d36 100755 --- a/setup.sh +++ b/setup.sh @@ -20,7 +20,12 @@ fi # Install dependencies +# this repo can be removed once python-language-data reaches Factory +test -f /etc/zypp/repos.d/d_l_python.repo || \ + $SUDO zypper --non-interactive --gpg-auto-import-keys \ + addrepo https://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_Tumbleweed/ d_l_python $SUDO zypper --non-interactive install gcc gcc-c++ make openssl-devel ruby-devel \ + python-langtable-data \ 'npm>=18' git augeas-devel cockpit jemalloc-devel || exit 1 # Backend setup From ce3362a38a15060e12b4a8b5e4476507ffb16372 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Thu, 11 May 2023 15:00:19 +0200 Subject: [PATCH 60/73] FIXME: revert: disable agama-locale-data tests in CI TODO: move this test to integration tests where we can install the python-langtable-data dependency --- .github/workflows/ci.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d74fb02d17..83ac09f421 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -266,7 +266,8 @@ jobs: - name: Build run: cargo build --verbose - name: Run tests - run: cargo test --verbose + # FIXME: omit agama-locale-data until python-langtable-data is installed + run: cargo test --verbose -p agama-cli -p agama-dbus-server -p agama-lib finish: From 33034d4213219d4def53b1f3a7e89b3193f4831a Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Thu, 11 May 2023 15:59:06 +0200 Subject: [PATCH 61/73] CI test: use setup-service.sh instead of racy manual service startup --- .github/workflows/ci.yml | 27 ++++++++------------------- 1 file changed, 8 insertions(+), 19 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 83ac09f421..be73c44d45 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -189,38 +189,27 @@ jobs: - name: start container run: podman run --privileged --detach --name agama --ipc=host -v .:/checkout -v /tmp/log/YaST2:/var/log/YaST2 registry.opensuse.org/yast/head/containers/containers_tumbleweed/opensuse/agama-testing:latest - - name: Install Ruby gems - run: podman exec agama bash -c "cd /checkout/service && bundle config set --local path 'vendor/bundle' && bundle install" - - - name: Install the Agama D-Bus configuration - run: podman exec agama bash -c "cp /checkout/service/share/dbus.conf /usr/share/dbus-1/system.d/org.opensuse.Agama.conf" + - name: Setup service + run: podman exec agama bash -c "cd /checkout; ./setup-service.sh" - name: Set a testing Agama configuration # copy a simplified ALP config file, it skips the product selection at the beginning run: podman exec agama bash -c "cp /checkout/playwright/config/agama.yaml /checkout/service/etc/agama.yaml" - - name: Start NetworkManager - # We need to run it manually as systemd dbus activation looks like failing - run: podman exec agama /usr/sbin/NetworkManager - - - name: Reload the D-Bus service - run: podman exec agama systemctl reload dbus - - - name: Start the Agama D-Bus services - # TODO: here is a potential race condition, but as building the frontend - # takes quite long time it should never happen™ - run: podman exec agama bash -c "cd /checkout/service && (bundle exec bin/agamactl > service.log 2>&1 &)" - - name: Build the frontend run: podman exec agama bash -c "cd /checkout/web && npm install && make" + - name: Show NetworkManager log + run: podman exec agama journalctl -u NetworkManager + - name: Show the D-Bus services log - run: podman exec agama cat /checkout/service/service.log + run: podman exec agama bash -c "journalctl | grep agama" - name: Check DBus socket run: podman exec agama ls -l /var/run/dbus/system_bus_socket - - name: Show journal + - name: Show the complete journal + # maybe not necessary if the above filtered journalctl calls are enough run: podman exec agama journalctl -b || echo "journal failed with $?" - name: Install the frontend From 7fefb8b9aff22da4d98e549bc72f1d663f2b03ce Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Thu, 11 May 2023 16:07:24 +0200 Subject: [PATCH 62/73] agamactl: Break plain `agamactl` up front, instead of later Its child services would fail anyway when they could not reach Locale --- service/bin/agamactl | 27 +++++---------------------- 1 file changed, 5 insertions(+), 22 deletions(-) diff --git a/service/bin/agamactl b/service/bin/agamactl index eba447cc6d..5caee1a663 100755 --- a/service/bin/agamactl +++ b/service/bin/agamactl @@ -65,31 +65,14 @@ end ORDERED_SERVICES = [:questions, :software, :storage, :users, :manager].freeze -# Normally the services are started by D-Bus activation. -# This starts all the services sequentially without relying on that. -# This is useful during development to have their log output on the terminal, -# not mixed with other syslog/journal messages. -# The downside is relying on an arbitrary delay, and a manually maintained ordering. -# -# @return [void] -# @see ORDERED_SERVICES -def start_all_services - ORDERED_SERVICES.map do |name| - puts "Starting #{name}:" - fork { exec("#{__FILE__} #{name}") } - sleep(7) - end -end - dbus_server_manager = Agama::DBus::ServerManager.new if ARGV.empty? - start_all_services - Signal.trap("SIGINT") do - puts "Stopping all services..." - exit - end - Process.wait + puts "ERROR: Using 'agamactl' to start all services no longer works." + puts "NOTE: It had race conditions all along and now there's a Rust service it can't reach too." + puts "NOTE: Use `systemctl start agama.service` instead" + puts "NOTE: which is setup by a) RPMs b) ./setup-service.sh" + exit 1 elsif ["-h", "--help"].include?(ARGV[0]) me = $PROGRAM_NAME puts "Usage:" From 69700e25d74352d881146abe109350e6f6ecb800 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Fri, 12 May 2023 15:03:25 +0200 Subject: [PATCH 63/73] CI: show journal _after_ tests have autostarted the services --- .github/workflows/ci.yml | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index be73c44d45..d4f4513fde 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -208,10 +208,6 @@ jobs: - name: Check DBus socket run: podman exec agama ls -l /var/run/dbus/system_bus_socket - - name: Show the complete journal - # maybe not necessary if the above filtered journalctl calls are enough - run: podman exec agama journalctl -b || echo "journal failed with $?" - - name: Install the frontend run: podman exec agama bash -c "ln -snfv /checkout/web/dist /usr/share/cockpit/agama" @@ -229,6 +225,13 @@ jobs: # run the tests in the Chromium browser run: podman exec agama bash -c "cd /checkout/playwright && SKIP_LOGIN=true playwright test --trace on --project chromium" + - name: Again show the D-Bus services log + run: podman exec agama bash -c "journalctl | grep agama" + + - name: Show the complete journal + # maybe not necessary if the above filtered journalctl calls are enough + run: podman exec agama journalctl -b || echo "journal failed with $?" + - name: Upload the test results uses: actions/upload-artifact@v3 # run even when the previous step fails From 4376b1f018d55be2038815d9fe3850b342ed2bce Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Sun, 21 May 2023 21:35:28 +0200 Subject: [PATCH 64/73] CI: start 'cockpit-ws --local-session' before ./setup-service.sh blocks it that's why we would see the login screen in Playwright tests and that needs the frontend built and installed first --- .github/workflows/ci.yml | 20 +++++++++++--------- setup-service.sh | 6 ++++-- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d4f4513fde..19b2fca7a3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -189,6 +189,17 @@ jobs: - name: start container run: podman run --privileged --detach --name agama --ipc=host -v .:/checkout -v /tmp/log/YaST2:/var/log/YaST2 registry.opensuse.org/yast/head/containers/containers_tumbleweed/opensuse/agama-testing:latest + - name: Build the frontend + run: podman exec agama bash -c "cd /checkout/web && npm install && make" + + - name: Install the frontend + run: podman exec agama bash -c "ln -snfv /checkout/web/dist /usr/share/cockpit/agama" + + # ./setup-service.sh will try setting up cockpit.socket + # which has a login page, so this local-session needs to be first + - name: Start Cockpit service + run: podman exec --detach agama /usr/libexec/cockpit-ws --local-session=/usr/bin/cockpit-bridge + - name: Setup service run: podman exec agama bash -c "cd /checkout; ./setup-service.sh" @@ -196,9 +207,6 @@ jobs: # copy a simplified ALP config file, it skips the product selection at the beginning run: podman exec agama bash -c "cp /checkout/playwright/config/agama.yaml /checkout/service/etc/agama.yaml" - - name: Build the frontend - run: podman exec agama bash -c "cd /checkout/web && npm install && make" - - name: Show NetworkManager log run: podman exec agama journalctl -u NetworkManager @@ -208,12 +216,6 @@ jobs: - name: Check DBus socket run: podman exec agama ls -l /var/run/dbus/system_bus_socket - - name: Install the frontend - run: podman exec agama bash -c "ln -snfv /checkout/web/dist /usr/share/cockpit/agama" - - - name: Start Cockpit service - run: podman exec --detach agama /usr/libexec/cockpit-ws --local-session=/usr/bin/cockpit-bridge - - name: Run the Agama smoke test run: podman exec agama curl http://localhost:9090/cockpit/@localhost/agama/index.html diff --git a/setup-service.sh b/setup-service.sh index 2c13b884c5..e6bf7802f3 100755 --- a/setup-service.sh +++ b/setup-service.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/bin/sh -x # Using a git checkout in the current directory, # set up the service (backend) part of agama @@ -76,7 +76,9 @@ $SUDO cp -v $MYDIR/service/share/dbus.conf /usr/share/dbus-1/agama.conf ( $SUDO systemctl daemon-reload # Start the separate dbus-daemon for Agama - $SUDO systemctl start agama.service + # (in CI we run a custom cockpit-ws which replaces the cockpit.socket + # dependency, continue in that case) + $SUDO systemctl start agama.service || pgrep cockpit-ws ) # - Make sure NetworkManager is running From 88f390fe30f5cab22838deaaa7059c8db096785b Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 24 May 2023 09:48:44 +0200 Subject: [PATCH 65/73] Better spelling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Imobach González Sosa --- rust/package/agama-cli.spec | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust/package/agama-cli.spec b/rust/package/agama-cli.spec index 3726229d3a..b57bca80c1 100644 --- a/rust/package/agama-cli.spec +++ b/rust/package/agama-cli.spec @@ -43,7 +43,7 @@ Command line program to interact with the agama service. # This will be set by osc services, that will run after this. Version: 0 Release: 0 -Summary: Agama rust dbus service +Summary: Agama Rust D-Bus service License: GPL-2.0-only Url: https://github.com/opensuse/agama Requires: python-langtable-data From 7d7350f1261ca15bf1222a825fad3d18c7192d68 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 24 May 2023 09:57:29 +0200 Subject: [PATCH 66/73] Resolve a note about langtable keyboards and other doc fixes thanks to Imo's review --- doc/locale_api.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/doc/locale_api.md b/doc/locale_api.md index 5a967ccb7c..ad8ceeaf4d 100644 --- a/doc/locale_api.md +++ b/doc/locale_api.md @@ -1,6 +1,8 @@ # Legacy-free Locale Service -> D-Installer currently have a separate Language service, although it's rather +Problem statement: (2023-03) + +> Agama currently has a separate Language service, although it's rather > simplistic. It just allows to set the language of the installed system using > Yast::Language.Set. And it's quite memory demanding for such an unimpressive > task. @@ -10,6 +12,7 @@ > overview on how much can we save. Original Plan: + 1. take the systemd APIs as a sensible starting point. 2. deviate only where we add value @@ -21,7 +24,7 @@ API. We may use `systemd-firstboot` instead but its API is much more limited. ## Localization This design includes localized labels in the API. In other contexts that would -be a responsibility of the frontend, but here here the backend has the +be a responsibility of the frontend, but here the backend has the information, provided by _langtable_. (Languages, Territories and Timezones have localized names. Keyboards do not.) @@ -147,8 +150,8 @@ _convert_ parameter. (The other systemd keyboard settings are X11Model and X11Options, we don't have UI or data for that) -**FIXME:** _langtable_ on the other hand only deals with the X11 keyboards. -And legacy YaST has the console keyboard as the primary key. Must resolve this. +NOTE: _langtable_ on the other hand only deals with the X11 keyboards, +linking them to languages and territories. ``` # this is gdbus syntax BTW From 04e5dd61fc6fdbfbbe5409403ca7deaf70e05100 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 24 May 2023 10:00:06 +0200 Subject: [PATCH 67/73] Spell D-Bus properly or else the daemon WILL disconnect you!! ;-) --- rust/README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/rust/README.md b/rust/README.md index 4336742900..58dfd27bc8 100644 --- a/rust/README.md +++ b/rust/README.md @@ -1,8 +1,8 @@ -# Agama Command Line and DBus Interface +# Agama Command Line and D-Bus Interface This project aims to build a command-line interface for [Agama](https://github.com/yast/agama), a service-based Linux installer featuring a nice -web interface. The second aim is dbus service that does not depend heavily on YaST to +web interface. The second aim is D-Bus service that does not depend heavily on YaST to reduce memory consumption and also provide better performance. ## Code organization @@ -11,14 +11,14 @@ We have set up [Cargo workspace](https://doc.rust-lang.org/book/ch14-03-cargo-wo three packages: * [agama-lib](./agama-lib): code that can be reused to access the - [Agama DBus API](https://github.com/yast/agama/blob/master/doc/dbus_api.md) and a + [Agama D-Bus API](https://github.com/yast/agama/blob/master/doc/dbus_api.md) and a model for the configuration settings. * [agama-cli](./agama-cli): code specific to the command line interface. * [agama-derive](./agama-derive): includes a [procedural macro](https://doc.rust-lang.org/reference/procedural-macros.html) to reduce the boilerplate code. -* [agama-locale-data](./agama-locale-data): specific library to provide data for localization dbus +* [agama-locale-data](./agama-locale-data): specific library to provide data for localization D-Bus API -* [agama-dbus-server](./agama-dbus-server): provides dbus API for services implemented in rust +* [agama-dbus-server](./agama-dbus-server): provides D-Bus API for services implemented in rust ## Status @@ -28,7 +28,7 @@ Agama CLI is still a work in progress, although it is already capable of doing a * Handling the auto-installation profiles. * Triggering the *probing* and the *installation* processes. -Agama DBus API is also a work in progress, but it is already used by Agama. +Agama D-Bus API is also a work in progress, but it is already used by Agama. ## Installation @@ -45,7 +45,7 @@ cargo install --path . ## Running -For DBus API just run as root agama-dbus-server binary and it will properly attach to DBus. +For D-Bus API just run as root agama-dbus-server binary and it will properly attach to D-Bus. For CLI take into account that you need to run `agama-cli` as root when you want to query or change the Agama configuration. Assuming that the Agama D-Bus service is running, the next command From 8c042266e30cee7ef2a31ed862ea42a9809c3b40 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 24 May 2023 10:14:05 +0200 Subject: [PATCH 68/73] Improve grammar thanks to Imo and https://www.deepl.com/write Also link to the cargo_audit PR CI skip, docs only --- rust/README.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/rust/README.md b/rust/README.md index 58dfd27bc8..df265fef77 100644 --- a/rust/README.md +++ b/rust/README.md @@ -114,12 +114,15 @@ frontend](./agama-cli/doc/backend-for-testing.md)*. ## Packaging -A packaging files lives in `package` directory. Agama follows -[rust packaging guidelines](https://en.opensuse.org/openSUSE:Packaging_Rust_Software). -For testing changes to spec file use simple `osc branch YaST:Head:Agama agama-cli` -and copy modified spec file to that branch. -If it needs also specific code from git branch, then modify `_service` file and -change git branch name in `` tag. Then run `osc service runall`. -Note: for leap `cargo_audit` does not work with older python, so comment out that -service section for testing build. -For testing build use common `osc build` on modified sources. +Packaging files live in the `package' directory. Agama follows the +[Rust packaging guidelines](https://en.opensuse.org/openSUSE:Packaging_Rust_Software). +To test changes to the spec file, use a simple `osc branch YaST:Head:Agama agama-cli`. +and copy the modified spec file to that branch. +If it also needs specific code from a git branch, then modify `_service' file and +put the git branch name in the `` tag. Then run `osc service runall`. + +Note: for openSUSE Leap, `cargo_audit` [does not work][c_a_bug] with older Python, so comment out that +service section for the test build. +For the test build, use the usual `osc build' on modified sources. + +[c_a_bug]: https://github.com/openSUSE/obs-service-cargo_audit/pull/6 From ac3c4ce6bb6e33d16bbad333b7d4b757f05d7cd0 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 24 May 2023 11:29:33 +0200 Subject: [PATCH 69/73] implement find_by_id --- rust/agama-dbus-server/src/locale.rs | 15 +++++---------- rust/agama-locale-data/src/language.rs | 6 ++++++ rust/agama-locale-data/src/territory.rs | 6 ++++++ 3 files changed, 17 insertions(+), 10 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index 2bc2e05cd7..2e23a592b9 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -20,19 +20,14 @@ impl Locale { fn labels_for_locales(&self) -> Result, Error> { const DEFAULT_LANG: &str = "en"; let mut res = Vec::with_capacity(self.supported_locales.len()); + let languages = agama_locale_data::get_languages()?; + let territories = agama_locale_data::get_territories()?; for locale in self.supported_locales.as_slice() { let (loc_language, loc_territory) = agama_locale_data::parse_locale(locale.as_str())?; - let languages = agama_locale_data::get_languages()?; - let territories = agama_locale_data::get_territories()?; - let language = languages - .language - .iter() - .find(|l| l.id == loc_language) + + let language = languages.find_by_id(loc_language) .context("language for passed locale not found")?; - let territory = territories - .territory - .iter() - .find(|t| t.id == loc_territory) + let territory = territories.find_by_id(loc_territory) .context("territory for passed locale not found")?; let default_ret = ( diff --git a/rust/agama-locale-data/src/language.rs b/rust/agama-locale-data/src/language.rs index 9f42a0c539..8502459307 100644 --- a/rust/agama-locale-data/src/language.rs +++ b/rust/agama-locale-data/src/language.rs @@ -14,4 +14,10 @@ pub struct Language { #[derive(Debug, Deserialize)] pub struct Languages { pub language: Vec +} + +impl Languages { + pub fn find_by_id(&self, id: &str) -> Option<&Language> { + self.language.iter().find(|t| t.id == id) + } } \ No newline at end of file diff --git a/rust/agama-locale-data/src/territory.rs b/rust/agama-locale-data/src/territory.rs index fa5933e390..209afbe2b5 100644 --- a/rust/agama-locale-data/src/territory.rs +++ b/rust/agama-locale-data/src/territory.rs @@ -11,4 +11,10 @@ pub struct Territory { #[derive(Debug, Deserialize)] pub struct Territories { pub territory: Vec +} + +impl Territories { + pub fn find_by_id(&self, id: &str) -> Option<&Territory> { + self.territory.iter().find(|t| t.id == id) + } } \ No newline at end of file From 339d8996c65b2bdf62e9de842899ec9707410a79 Mon Sep 17 00:00:00 2001 From: Josef Reidinger Date: Wed, 24 May 2023 16:53:07 +0200 Subject: [PATCH 70/73] update from review --- rust/agama-dbus-server/src/locale.rs | 8 ++-- rust/agama-dbus-server/src/main.rs | 1 - rust/agama-locale-data/src/lib.rs | 6 +++ rust/agama-locale-data/src/timezone_part.rs | 42 +++++++++++++-------- 4 files changed, 36 insertions(+), 21 deletions(-) diff --git a/rust/agama-dbus-server/src/locale.rs b/rust/agama-dbus-server/src/locale.rs index 2e23a592b9..06ecfa85b7 100644 --- a/rust/agama-dbus-server/src/locale.rs +++ b/rust/agama-dbus-server/src/locale.rs @@ -62,10 +62,10 @@ impl Locale { } #[dbus_interface(property)] - fn set_locales(&mut self, locales: Vec) -> Result<(), zbus::fdo::Error> { + fn set_locales(&mut self, locales: Vec) -> zbus::fdo::Result<()> { for loc in &locales { - if !self.supported_locales.contains(&loc) { - return Err(zbus::fdo::Error::Failed(format!("Unsupported locale value '{loc}'").to_string())) + if !self.supported_locales.contains(loc) { + return Err(zbus::fdo::Error::Failed(format!("Unsupported locale value '{loc}'"))) } } self.locales = locales; @@ -74,7 +74,7 @@ impl Locale { #[dbus_interface(property)] fn supported_locales(&self) -> Vec { - return self.supported_locales.to_owned(); + self.supported_locales.to_owned() } #[dbus_interface(property)] diff --git a/rust/agama-dbus-server/src/main.rs b/rust/agama-dbus-server/src/main.rs index 2f1e74d434..39aa557235 100644 --- a/rust/agama-dbus-server/src/main.rs +++ b/rust/agama-dbus-server/src/main.rs @@ -3,7 +3,6 @@ pub mod locale; use std::future::pending; -// Although we use `async-std` here, you can use any async runtime of choice. #[async_std::main] async fn main() -> Result<(), Box> { let _con = crate::locale::Locale::start_service().await?; diff --git a/rust/agama-locale-data/src/lib.rs b/rust/agama-locale-data/src/lib.rs index 038ff175f5..4a10c2489d 100644 --- a/rust/agama-locale-data/src/lib.rs +++ b/rust/agama-locale-data/src/lib.rs @@ -23,6 +23,7 @@ fn file_reader(file_path: &str) -> anyhow::Result { Ok(reader) } +/// Gets list of X11 keyboards structs pub fn get_xkeyboards() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/keyboards.xml.gz"; let reader = file_reader(FILE_PATH)?; @@ -66,6 +67,7 @@ pub fn parse_locale(locale: &str) -> anyhow::Result<(&str, &str)> { Ok((captures.get(1).unwrap().as_str(), captures.get(2).unwrap().as_str())) } +/// Returns struct which contain list of known languages pub fn get_languages() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/languages.xml.gz"; let reader = file_reader(FILE_PATH)?; @@ -75,6 +77,7 @@ pub fn get_languages() -> anyhow::Result { Ok(ret) } +/// Returns struct which contain list of known territories pub fn get_territories() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/territories.xml.gz"; let reader = file_reader(FILE_PATH)?; @@ -84,6 +87,7 @@ pub fn get_territories() -> anyhow::Result { Ok(ret) } +/// Returns struct which contain list of known parts of timezones. Useful for translation pub fn get_timezone_parts() -> anyhow::Result { const FILE_PATH: &str = "/usr/share/langtable/data/timezoneidparts.xml.gz"; let reader = file_reader(FILE_PATH)?; @@ -93,6 +97,8 @@ pub fn get_timezone_parts() -> anyhow::Result { Ok(ret) } + +/// Gets list of non-deprecated timezones pub fn get_timezones() -> Vec { chrono_tz::TZ_VARIANTS.iter() .filter(|&tz| !crate::deprecated_timezones::DEPRECATED_TIMEZONES.contains(&tz.name())) // Filter out deprecated asmera diff --git a/rust/agama-locale-data/src/timezone_part.rs b/rust/agama-locale-data/src/timezone_part.rs index 7948a9f186..256fd3000c 100644 --- a/rust/agama-locale-data/src/timezone_part.rs +++ b/rust/agama-locale-data/src/timezone_part.rs @@ -8,7 +8,7 @@ pub struct TimezoneIdPart { /// "Prague" pub id: String, /// [{language: "cs", value: "Praha"}, {"language": "de", value: "Prag"} ...] - pub names: crate::localization::Localization + pub names: crate::localization::Localization, } // Timezone id parts are useful mainly for localization of timezones @@ -16,7 +16,7 @@ pub struct TimezoneIdPart { #[derive(Debug, Deserialize)] pub struct TimezoneIdParts { #[serde(rename(deserialize = "timezoneIdPart"))] - pub timezone_part: Vec + pub timezone_part: Vec, } impl TimezoneIdParts { @@ -31,26 +31,36 @@ impl TimezoneIdParts { /// ``` pub fn localize_timezones(&self, language: &str, timezones: &Vec) -> Vec { let mapping = self.construct_mapping(language); - timezones.iter().map(|tz| self.translate_timezone(&mapping, tz)).collect() + timezones + .iter() + .map(|tz| self.translate_timezone(&mapping, tz)) + .collect() } fn construct_mapping(&self, language: &str) -> HashMap { let mut res: HashMap = HashMap::with_capacity(self.timezone_part.len()); - self.timezone_part.iter() - .map(|part| (part.id.clone(), part.names.name_for(language))) - .for_each(|tuple| -> () { - // skip missing translations - if let Some(trans) = tuple.1 { - res.insert(tuple.0, trans); - } - } - ); - return res + self.timezone_part + .iter() + .map(|part| (part.id.clone(), part.names.name_for(language))) + .for_each(|(time_id, names)| -> () { + // skip missing translations + if let Some(trans) = names { + res.insert(time_id, trans); + } + }); + return res; } fn translate_timezone(&self, mapping: &HashMap, timezone: &str) -> String { - timezone.split("/") - .map(|tzp| mapping.get(&tzp.to_string()).expect(format!("Unknown timezone part {tzp}").as_str()).to_owned()) - .collect::>().join("/") + timezone + .split("/") + .map(|tzp| { + mapping + .get(&tzp.to_string()) + .expect(format!("Unknown timezone part {tzp}").as_str()) + .to_owned() + }) + .collect::>() + .join("/") } } From 605ca2c06e17d0b393033b4f7bf463cfc22f64c3 Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 24 May 2023 17:11:14 +0200 Subject: [PATCH 71/73] Re-enable agama-locale-data tests by running `cargo test` in integration_tests job where we do have the langtable dependency With that, integration_tests has swallowed the entire cli_build job. --- .github/workflows/ci.yml | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 25b412d004..de8acf2b8a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -199,6 +199,9 @@ jobs: - name: Setup service run: podman exec agama bash -c "cd /checkout; ./setup-service.sh" + - name: Rust unit tests + run: podman exec agama bash -c "cd /checkout/rust; cargo test --verbose" + - name: Set a testing Agama configuration # copy a simplified ALP config file, it skips the product selection at the beginning run: podman exec agama bash -c "cp /checkout/playwright/config/agama.yaml /checkout/service/etc/agama.yaml" @@ -241,25 +244,6 @@ jobs: playwright/test-results/**/* /tmp/log/YaST2/y2log - cli_build: - runs-on: ubuntu-latest - - env: - CARGO_TERM_COLOR: always - - defaults: - run: - working-directory: ./rust - - steps: - - uses: actions/checkout@v3 - - name: Build - run: cargo build --verbose - - name: Run tests - # FIXME: omit agama-locale-data until python-langtable-data is installed - run: cargo test --verbose -p agama-cli -p agama-dbus-server -p agama-lib - - finish: runs-on: ubuntu-latest From 224520f69f14417ab7d1226951d18dfe43169dea Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 24 May 2023 17:35:06 +0200 Subject: [PATCH 72/73] setup.sh + setup-service.sh: install RPM dependencies in both This makes python-langtable-data available if you only set up the backend I hope I got the split right --- setup-service.sh | 12 +++++++++++- setup.sh | 15 +++++---------- 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/setup-service.sh b/setup-service.sh index e6bf7802f3..7cc63e9f12 100755 --- a/setup-service.sh +++ b/setup-service.sh @@ -32,7 +32,17 @@ sudosed() { sed -e "$1" "$2" | $SUDO tee "$3" > /dev/null } -# - Install the service dependencies +# - Install RPM dependencies + +# this repo can be removed once python-language-data reaches Factory +test -f /etc/zypp/repos.d/d_l_python.repo || \ + $SUDO zypper --non-interactive --gpg-auto-import-keys \ + addrepo https://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_Tumbleweed/ d_l_python +$SUDO zypper --non-interactive install gcc gcc-c++ make openssl-devel ruby-devel \ + python-langtable-data \ + git augeas-devel jemalloc-devel || exit 1 + +# - Install service rubygem dependencies ( cd $MYDIR/service bundle config set --local path 'vendor/bundle' diff --git a/setup.sh b/setup.sh index f03f437d36..00d23d8e19 100755 --- a/setup.sh +++ b/setup.sh @@ -18,20 +18,15 @@ else SUDO="" fi -# Install dependencies - -# this repo can be removed once python-language-data reaches Factory -test -f /etc/zypp/repos.d/d_l_python.repo || \ - $SUDO zypper --non-interactive --gpg-auto-import-keys \ - addrepo https://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_Tumbleweed/ d_l_python -$SUDO zypper --non-interactive install gcc gcc-c++ make openssl-devel ruby-devel \ - python-langtable-data \ - 'npm>=18' git augeas-devel cockpit jemalloc-devel || exit 1 - # Backend setup $MYDIR/setup-service.sh +# Install Frontend dependencies + +$SUDO zypper --non-interactive install \ + make git 'npm>=18' cockpit || exit 1 + # Web Frontend $SUDO systemctl start cockpit From d2264fc5b9c72cf508a4bfe170f255c19c54234b Mon Sep 17 00:00:00 2001 From: Martin Vidner Date: Wed, 24 May 2023 17:58:14 +0200 Subject: [PATCH 73/73] setup.sh + setup-service.sh: use --gpg-auto-import-keys where it matters --- setup-service.sh | 4 ++-- setup.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/setup-service.sh b/setup-service.sh index 7cc63e9f12..d418919279 100755 --- a/setup-service.sh +++ b/setup-service.sh @@ -36,9 +36,9 @@ sudosed() { # this repo can be removed once python-language-data reaches Factory test -f /etc/zypp/repos.d/d_l_python.repo || \ - $SUDO zypper --non-interactive --gpg-auto-import-keys \ + $SUDO zypper --non-interactive \ addrepo https://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_Tumbleweed/ d_l_python -$SUDO zypper --non-interactive install gcc gcc-c++ make openssl-devel ruby-devel \ +$SUDO zypper --non-interactive --gpg-auto-import-keys install gcc gcc-c++ make openssl-devel ruby-devel \ python-langtable-data \ git augeas-devel jemalloc-devel || exit 1 diff --git a/setup.sh b/setup.sh index 00d23d8e19..a4e2c48f8d 100755 --- a/setup.sh +++ b/setup.sh @@ -24,7 +24,7 @@ $MYDIR/setup-service.sh # Install Frontend dependencies -$SUDO zypper --non-interactive install \ +$SUDO zypper --non-interactive --gpg-auto-import-keys install \ make git 'npm>=18' cockpit || exit 1 # Web Frontend