Skip to content

input methods: use GTK3_IM_MODULE_FILE#14417

Closed
ericsagnes wants to merge 2 commits intoNixOS:masterfrom
ericsagnes:fix/gtk-immodules-cache
Closed

input methods: use GTK3_IM_MODULE_FILE#14417
ericsagnes wants to merge 2 commits intoNixOS:masterfrom
ericsagnes:fix/gtk-immodules-cache

Conversation

@ericsagnes
Copy link
Contributor

Work in progress, please do not merge yet.

This is an implementation of #14019 fix.

@ttuegel Could you review this PR code?

A few personal concerns:

  • Using a systemd unit to generate the cache files doesn't feel right, is there a better way to do that? solved in the second commit
  • Having an option (i18n.inputMethod.package) just to pass the input method package doesn't feel right. solved by setting internal = true

Note:

  • Testing this PR will trigger a quite long rebuild of the gtk+3 package.
  • I add commits to keep track of progress, all commits will be merged into one before merging.

@mention-bot
Copy link

By analyzing the blame information on this pull request, we identified @taku0, @ttuegel and @urkud to be potential reviewers

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Environment as a set: Environment.GTK_PATH = "....".

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

If gtk-query-immodules-* doesn't access anything but GTK_PATH then you can run it in a package. Then link results from the package into /etc. I have used this approach for example here.

@ericsagnes
Copy link
Contributor Author

@abbradar Thanks, I added a commit to remove the systemd unit and use mkDerivation instead. It looks cleaner now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add preferLocalBuild = true; allowSubstitutes = false; to those -- they look simple enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I added a commit fixing this.

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

Nice! LGTM, I'll wait for someone else with more GTK expertise to review it but if I understand everything correctly, this should work. Also we may want to port this to stable although it's somewhat mass-rebuildy. @domenkozar, what do you think?

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

Oh, one more thing -- don't sweat over package option, I believe it's our standard way to do this -- just grep for internal = true.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, right -- you should mark this as internal = true, I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat! Thanks, I added a commit.

@ericsagnes
Copy link
Contributor Author

Looking at previous discussions on the topic (#2450), I think that @lethalman might be interested by this PR.

@ttuegel
Copy link
Member

ttuegel commented Apr 3, 2016

@ericsagnes Would you mind patching for the other environment variables GTK 3 reads, the same way you have done for GTK_IM_MODULE_FILE? It needs to be done eventually, and I would hate to incur multiple mass rebuilds.

@ericsagnes
Copy link
Contributor Author

@ttuegel No problems, looking at #5053 I only see GTK_DATA_PREFIX. Are there any other variables that should be patched?

@ttuegel
Copy link
Member

ttuegel commented Apr 3, 2016

We really need to patch every call to g_getenv("GTK_*") because GTK makes no effort to determine if the listed paths are for GTK 2 or GTK 3.

@ericsagnes
Copy link
Contributor Author

@ttuegel Result of a quick grep in gtk3 source:

  • GTK_THEME
  • GTK_DEBUG
  • GTK_OVERLAY_SCROLLING
  • GTK_CSS_DEBUG
  • GTK_DATA_PREFIX
  • GTK_EXE_PREFIX
  • GTK_PATH
  • GTK_STYLE_CONTEXT_WARNING
  • GTK_IM_MODULE
  • GTK_MODULES
  • GTK_TEST_TOUCHSCREEN
  • GTK_INSPECTOR_DISPLAY
  • GTK_WIDGET_ASSERT_COMPONENTS
  • GTK_CSD

Is it fine to patch all these?

@ttuegel
Copy link
Member

ttuegel commented Apr 3, 2016

Yes, that sounds good!

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

From cursory glance, some of these may be unnecessary (for example, GTK_DEBUG or GTK_TEST_TOUCHSCREEN). I'm not sure, however.

@taku0
Copy link
Contributor

taku0 commented Apr 3, 2016

Tested using nixos-rebuild build-vm with a configuration.nix below:
https://gist.github.com/taku0/f988cd6c0f1151f8ee89bc7aa24bdcfb

I got a strange behavior: if GTK_IM_MODULE_FILE is defined and ~/.uim.d/customs/custom-* does not exist, uim crashes with the following message:

Error: in scm_symbol_value: unbound variable: generic-commit-key?

The error does not occur with master branch (GTK_IM_MODULE_FILE is not defined) or if ~/.uim.d/customs/custom-* are created with uim-pref-gtk. The error occurs again if ~/.uim.d/customs/custom-* are removed. This could be a bug of uim or Anthy. I will investigate it later if possible.

@ericsagnes
Copy link
Contributor Author

@ttuegel I added a commit patching all the variables cited in the previous comment.
gtk+3 is compiling fine, but I don't know how to check that all the new GTK3* variables are working correctly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy-paste mistake, maybe?

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

@ericbmerritt Very cool! Maybe we want to try upstreaming this (of course, out of scope of this PR)?

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

Ouch, sorry, auto-completion. I meant @ericsagnes

@ttuegel
Copy link
Member

ttuegel commented Apr 3, 2016

@ericsagnes I agree now that the variables which are testing for NULLity should not have GTK3_ variants. It breaks the logic of these debug options, I think. Although, I'm not sure anything is harmed.

@ericsagnes
Copy link
Contributor Author

@abbradar Good suggestion, I added a ticket upstream.

I agree now that the variables which are testing for NULLity should not have GTK3_ variants. It breaks the logic of these debug options, I think. Although, I'm not sure anything is harmed.

@ttuegel I can revert that if you think it is better, should I?

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

@ericbmerritt Thanks! Let's see what upstream thinks.

From another look (brief, though) those variables can be actually useful to have GTK3_ variants:

  • GTK_IM_MODULE_FILE
  • GTK_THEME
  • GTK_DATA_PREFIX
  • GTK_PATH
  • GTK_IM_MODULE

Notice that I haven't included GTK_EXE_PREFIX because it's visible in the patch that GTK uses it with /gtk-3.0 postfix, so it should be fine even when co-used with GTK2. Maybe I'm missing something, though.

Others are not really important and/or mostly for debugging, I think.

Notice that I don't have any knowledge on GTK topics and this is just me reading the patch and concluding what a variable does from its name!

@ttuegel
Copy link
Member

ttuegel commented Apr 3, 2016

From another look (brief, though) those variables can be actually useful to have GTK3_ variants:

GTK_IM_MODULE_FILE
GTK_THEME
GTK_DATA_PREFIX
GTK_PATH
GTK_IM_MODULE

Notice that I haven't included GTK_EXE_PREFIX because it's visible in the patch that GTK uses it with /gtk-3.0 postfix, so it should be fine even when co-used with GTK2. Maybe I'm missing something, though.

I think the problem with GTK_EXE_PREFIX as things stand now is that it must be shared with GTK 2. I would prefer to have separate variables; in worst-case scenario, they always take the same value. Otherwise, I agree with you list of variables that need separate prefixes. Thanks for looking into this!

@abbradar
Copy link
Member

abbradar commented Apr 3, 2016

@ttuegel Hm, indeed, this would be more convenient (allows us to separate GTK2 and 3 configuration, enable their support separately etc). So I agree, let's patch _EXE_PREFIX too.

@ericsagnes
Copy link
Contributor Author

@abbradar I added a commit to remove the patches of the following variables:

  • GTK_DEBUG
  • GTK_OVERLAY_SCROLLING
  • GTK_CSS_DEBUG
  • GTK_STYLE_CONTEXT_WARNING
  • GTK_TEST_TOUCHSCREEN
  • GTK_INSPECTOR_DISPLAY
  • GTK_WIDGET_ASSERT_COMPONENTS
  • GTK_CSD

So now, the only patched variables are:

  • GTK_IM_MODULE_FILE
  • GTK_THEME
  • GTK_DATA_PREFIX
  • GTK_PATH
  • GTK_IM_MODULE
  • GTK_EXE_PREFIX

@abbradar
Copy link
Member

abbradar commented Apr 4, 2016

Thank you! This looks ready to me now.

Nikolay.

@abbradar abbradar removed the 0.kind: enhancement Add something new or improve an existing system. label Apr 4, 2016
@ericsagnes
Copy link
Contributor Author

I arranged the commits so the PR is ready to merge.

@taku0 Have you found the cause of the uim crash? I tested the PR with uim on a test machine and couldn't reproduce the issue.

@ericsagnes ericsagnes changed the title [WIP] input methods: use GTK3_IM_MODULE_FILE input methods: use GTK3_IM_MODULE_FILE Apr 4, 2016
@domenkozar
Copy link
Member

@abbradar we can eventually backport to 16.03, there's no staging for it. But for mass rebuilds, it should first be committed to master and tested for a few weeks. Then open a PR to backport to stable with arguments why it's feasible to do so.

@abbradar
Copy link
Member

abbradar commented Apr 4, 2016

@domenkozar Got it, thanks

Let's wait for @taku0's response to ensure it isn't this patch's fault and merge.

@taku0
Copy link
Contributor

taku0 commented Apr 4, 2016

It crashes when I try to input to an text field of uim-pref-gtk or uim-pref-gtk3 but not crashes on firefox-bin or emacs. So it would be fine for daily use.

@ericsagnes
Copy link
Contributor Author

@taku0 I would like to check if the same happens on my test machine, can you tell which text field of uim-gtk-pref and the steps to reproduce the crash?

@taku0
Copy link
Contributor

taku0 commented Apr 4, 2016

  1. Download https://gist.github.com/taku0/f988cd6c0f1151f8ee89bc7aa24bdcfb to a temporary directory.
  2. git fetch --force upstream pull/14417/head:pull_14417 in nixpkgs directory.
  3. git checkout pull_14417 -b tmp
  4. git rebase master (optional)
  5. nixos-rebuild build-vm -I nixpkgs=~/src/nixpkgs -I nixos-config=./configuration.nix where ~/src/nixpkgs and ./configuration.nix should be adjusted.
  6. ./result/bin/run-nixos-vm
  7. Login as alice with password alice.
  8. (Ignore KDE error message)
  9. Open konsole from start menu.
  10. uim-pref-gtk
  11. Open “SKK 辞書” (SKK dictionaries) group.
  12. Focus “個人辞書ファイル” (Personal dictionary file) text field.
  13. Ensure Anthy is enabled (leftmost button on the uim toolbar)
  14. Switch Anthy input mode (2nd button from the left on the uim toolbar) to “ひらがな” mode.
  15. Type some words into the text field, e.g. ka.
  16. Error messages is dumped to the konsole.

screenshot

@abbradar
Copy link
Member

abbradar commented Apr 4, 2016

Seeing the upstream reaction to this patch, I wonder why won't we just patch our default paths into GTK instead? So it would expect IM cache in /etc/gtk-*/immodules.cache, have data prefix /var/lib/current-system/sw etc. by default. What would the disadvantages be?

@ttuegel
Copy link
Member

ttuegel commented Apr 4, 2016

What would the disadvantages be?

GTK applications won't work outside NixOS. I don't have a problem with that; given upstream's obstructionist attitude, we should drop all GTK packages anyway. That's not going to happen, so I would say: go with the patch we already have.

@abbradar
Copy link
Member

abbradar commented Apr 4, 2016

GTK applications won't work outside NixOS.

A-ha, indeed: I haven't considered that. Thanks for the clarification!

@ericsagnes
Copy link
Contributor Author

@taku0 Thanks for the detailed steps! I could reproduce the errors in the vm, but it is working fine on my physical test machine.
It might be related to the vm, do you have a way to test on a physical machine to see if it is working any better there?

@taku0
Copy link
Contributor

taku0 commented Apr 6, 2016

I will try to test on this weekend.

taku0 pushed a commit to taku0/nixpkgs that referenced this pull request Apr 6, 2016
This patch the following environment variables logic, enabling to use a
`GTK3_` version as an override:

- GTK_IM_MODULE_FILE
- GTK_THEME
- GTK_DATA_PREFIX
- GTK_PATH
- GTK_IM_MODULE
- GTK_EXE_PREFIX

Details at NixOS#14417
@ericsagnes
Copy link
Contributor Author

Update from the upstream bug report:

  • GTK_* environment variables shouldn't be used and some of them are deprecated (GTK_IM_MODULE_FILE and GTK_THEME)
  • Instead of patching environment variables, they prefer patches to fix default behavior of gtk; So it is very unlikely they will use this solution.

From the bug report:

I would be much more interested in patches that fix cache file locations like that or just make gtk work without the cache than I am in adding extra environment variables as a 'quick fix'.
The long term vision for this is that we want to move away from loading modules for these things altogether. GTK+ should just use the preferred input method framework of the platform it is running on (for Linux, that would be ibus), not let the use switch between dozens of unmaintained and half-working alternatives.

So shall we merge the PR an keep this patch until upstream fixes the default functionality or try a different approach?

@abbradar
Copy link
Member

abbradar commented Apr 7, 2016

I say let's merge -- while upstream fixes this, we want to have a working solution for our users. I don't think we as distribution maintainers need a better fix (of course, we as (GTK) users may want one, but that's another story). I also propose to move discussion about @taku0's problem to another issue. If noone objects, I'll hit the button in several days (feel free to beat me to it, of course!).

(EDIT to clarify: by "GTK users" I meant mainly developers)

@astsmtl
Copy link
Contributor

astsmtl commented Apr 7, 2016

Thank you, this commit fixed my UIM. :)

@astsmtl
Copy link
Contributor

astsmtl commented Apr 7, 2016

...and broke pasystray. Backtrace shows that pasystray's GTK+3 loads UIM plugin for GTK+2 and gets SIGABRT later. We need to test and fix upgrade path for GTK+3 apps before merging this.

@ericsagnes
Copy link
Contributor Author

@astsmtl I tried pasystray with this PR and couldn't reproduce your issue.
I checked pasystray source to see if it was using any of the variables patched and couldn't find any reference.
Is there any special action that trigger the bug?

@abbradar
Copy link
Member

abbradar commented Apr 8, 2016

@astsmtl Wild guess -- maybe you use pasystray from nix-env without having it rebuilt? This patch would break all such software that uses old (unpatched) GTK+3.

@taku0
Copy link
Contributor

taku0 commented Apr 8, 2016

or try a different approach?

Another option is reverting a91161a partially for gtkPlugins as ttuegel suggests (disclaimer: I'm the author of gtkPlugins: #3275), though I'm not against GTK3_IM_MODULE_FILE.

@astsmtl
Copy link
Contributor

astsmtl commented Apr 8, 2016

Of course it was pasystray from my profile. But is this a good excuse for breaking? Maybe we can also patch GTK+2 to use GTK2_XXX variables?

@ttuegel
Copy link
Member

ttuegel commented Apr 8, 2016

Of course it was pasystray from my profile. But is this a good excuse for breaking?

This is a fair point.

One way we could fix this issue without breaking existing packages is to patch GTK 2 and 3 to search NIX_PROFILES for an immodules.cache file. I patched Qt 5 this way for its plugins. We could have GTK search each profile for a file $PROFILE/etc/gtk-{2,3}.0/immodules.cache. This simultaneously solves the backwards-compatibility problem, it allows users to override the system immodules.cache, and it works on non-NixOS systems.

EDIT: There's one other option, which upstream alluded to: hardcode the choice of input method at compile time and ship a binary cache with I-Bus as the default. (Realistically, I-Bus is the only mature, modern input method on Linux.) I think we should seriously consider this option. As a distribution, it's our job to be software integrators for our users, i.e. to provide a distribution not a mere binary cache. I think this point is usually lost in Nixpkgs. The "throw all possible options at the user" approach isn't responsible. This is not Gentoo or LFS.

@taku0
Copy link
Contributor

taku0 commented Apr 8, 2016

Ideally, the following three GTK+ versions must match:

  • A. the version used to build the application
  • B. the version used to build the input method (or other plugins)
  • C. the version used at runtime

The GTK3_IM_MODULE_FILE approach fixes B system-wide, so A-B and B-C may be different.
The gtkPlugins approach fixes B and C system-wide, so A-B and A-C may be different.
The wrapping-all-GTK+-applications approach suggested by @lethalman on #2450 fixes none of them, so all versions match, but requires to wrap all existing and future GTK+ applications. To make matters worse, if a non-privilege user installs an input method, it will effective only for applications installed by that user but not for system packages.

This is an unsolved problem of Nix.

@taku0
Copy link
Contributor

taku0 commented Apr 9, 2016

@ericsagnes
It fails on my physical x86_64 machine.

uim-pref-gtk: fails
uim-pref-gtk3: fails
uim-pref-qt4: works
firefox: works
emacs: works
inkscape: works

@ericsagnes
Copy link
Contributor Author

One way we could fix this issue without breaking existing packages is to patch GTK 2 and 3 to search NIX_PROFILES for an immodules.cache file. I patched Qt 5 this way for its plugins. We could have GTK search each profile for a file $PROFILE/etc/gtk-{2,3}.0/immodules.cache. This simultaneously solves the backwards-compatibility problem, it allows users to override the system immodules.cache, and it works on non-NixOS systems.

@ttuegel That sounds like a cleaner approach, I made an implementation here. It is working fine with gedit (gtk3) and geany (gtk2). It trigger long mass rebuilds.
This fixes #14019 but does nothing for #5053.
Should I make a new PR with the NIX_PROFILES version and close this one?

@taku0 thanks for testing, I have no idea about what can be the cause as other gtk apps are working. Does the NIX_PROFILES version work any better?

@ttuegel
Copy link
Member

ttuegel commented Apr 10, 2016

Should I make a new PR with the NIX_PROFILES version and close this one?

Yes, I think that would be preferable.

@ericsagnes
Copy link
Contributor Author

closing in favor of #14568.

@ericsagnes ericsagnes closed this Apr 10, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0.kind: bug Something is broken 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 9.needs: port to stable A PR needs a backport to the stable release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants