Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Projectile known projects file randomly being wiped #185

Closed
umi2002 opened this issue Feb 3, 2025 · 38 comments · Fixed by #187
Closed

Projectile known projects file randomly being wiped #185

umi2002 opened this issue Feb 3, 2025 · 38 comments · Fixed by #187
Labels

Comments

@umi2002
Copy link

umi2002 commented Feb 3, 2025

There are 2 problems that I am running into:

  1. Until I open a project, helm-projectile-switch-project shows an empty list. I checked the contents of the known projects file manually and it shows correctly the projects that were discovered.
  2. (Seemingly) randomly, my known projects file will be wiped. I can't really seem to find any correlation with what I do, it just happens at random intervals.

In both cases, whenever I open a fresh Emacs client, I can't instantly switch to a project which is very annoying. Here is the relevant part of my config:

(use-package projectile)
(use-package helm-projectile
  :after (helm projectile))

(with-eval-after-load 'projectile
  (projectile-global-mode)
  (setq projectile-indexing-method 'hybrid)
  (setq projectile-enable-caching t)
  (setq projectile-git-command "git ls-files -zco --exclude-standard")
  (setq projectile-project-search-path '(("~/code/" . 3) "~/dotfiles/"))
  (setq projectile-generic-command "fd . -0 --type f --color=never"))

(with-eval-after-load 'helm-projectile
  (helm-projectile-on))

I am using projectile MELPA 2.9.0 and helm-projectile 1.1.0

@ohamov
Copy link

ohamov commented Feb 4, 2025

+1
I see the same issue. The reproduce rate on my side is 100%. Every time I open Emacs, the list of projects is empty, even though it wasn't empty when I closed Emacs

@bbatsov bbatsov added the bug label Feb 4, 2025
@bbatsov
Copy link
Owner

bbatsov commented Feb 4, 2025

Hmm, that's likely caused by helm-projectile, as I can't reproduce the problem in Projectile (and I know that it doesn't cleanup projects automatically). I haven't used Helm in years, so I don't remember the code here very well, but a quick grep doesn't show anything that would strip the known projects.

Did this issue start recently or it has been around for a while?

@umi2002
Copy link
Author

umi2002 commented Feb 4, 2025

I'm a new Emacs user so I can't say it has been around for a while, but I will say that when I was configuring projectile and helm-projectile for the first time, it worked fine. I continued configuring other stuff and it suddenly broke.

@bbatsov
Copy link
Owner

bbatsov commented Feb 4, 2025

@Thaodan Might this be caused by 9db7976 ?

@ohamov
Copy link

ohamov commented Feb 4, 2025

@bbatsov the projectile standalone works as expected as far as I can judge, so, yes, I think your guess about helm-projectile is correct. The issue is new, I'd say it wasn't there a week or two ago

Update:

I've tried using an older version of helm-projectile (e2e3882), but the issue is still reproducible

@Thaodan
Copy link
Contributor

Thaodan commented Feb 4, 2025 via email

@ohamov
Copy link

ohamov commented Feb 4, 2025

Interesting that helm-projectile-switch-project returns an empty list until projectile-switch-project is called (which returns the list of projects correctlly). All subsequent calls to helm-projectile-swtich-projects work as expected until Emacs is restarted

@umi2002
Copy link
Author

umi2002 commented Feb 4, 2025

I don't know if anyone was able to reproduce it, but not all subsequent calls work for me. However, the behavior is slightly different for subsequent calls. For the first call after restarting Emacs, the contents of the projectile-known-projects-file is still there but the list is empty. For subsequent calls, there is a chance that the file gets wiped, which returns an empty list in turn. As long as the contents of the file are still there, it works as expected.

@ohamov
Copy link

ohamov commented Feb 4, 2025

At this point I'm not sure why helm-projectile package is needed: I removed helm-projectile and added (setq projectile-completion-system 'helm) to my config instead. This works just fine for me, projectile gets invoked with helm as a completion backend. The empty list of projects issue doesn't reproduce anymore with this approach

  (use-package projectile
    :ensure t
    :config
    (setq projectile-completion-system 'helm)
    (setq profectile-sort-order 'recently-active)
    (define-key projectile-mode-map (kbd "C-c p") #'projectile-command-map)
    (projectile-mode))

  (use-package helm
    :ensure t
    :init
    (setq helm-buffers-truncate-lines nil)
    :config
    (global-set-key (kbd "M-x") #'helm-M-x)
    (global-set-key (kbd "C-x C-f") #'helm-find-files)
    (global-set-key (kbd "C-x C-b") #'helm-resume)
    (global-set-key (kbd "M-y") #'helm-show-kill-ring)
    (global-set-key (kbd "C-x b") #'helm-buffers-list)
    (add-hook 'eshell-mode-hook
	      (lambda ()
		(define-key ehsell-mode-map (kdb "C-c C-l") #'helm-eshell-history)))
    (define-key projectile-command-map (kbd "n") #'helm-do-ag-this-file)
    (define-key projectile-command-map (kbd "o") #'helm-occur)
    (helm-mode))

@umi2002
Copy link
Author

umi2002 commented Feb 4, 2025

This solution works for the most part as in it fixes my initial problems, but the projectile commands don't have the same fuzzy matching as helm-projectile.

@ohamov
Copy link

ohamov commented Feb 4, 2025

Right, so I guess a proper fix needed. I have the same config on my other laptop, which doesn't have this issue. I can check which projectile/helm/helm-projectile versions are installed there. I think it can be an integration issue

Update: projectile-2.8.0, helm-projectile-1.0.0, helm-4.0 work fine

@bbatsov
Copy link
Owner

bbatsov commented Feb 4, 2025

I found the source of the issue in helm-projectile and I'll have it fixed soon.

@bbatsov bbatsov closed this as completed in acde352 Feb 4, 2025
@ohamov
Copy link

ohamov commented Feb 4, 2025

@bbatsov unfortunately the fix didn't help in my case

@umi2002
Copy link
Author

umi2002 commented Feb 4, 2025

Same here, just updated (snapshot 7a9497c) and I am observing the same behavior.

@Thaodan
Copy link
Contributor

Thaodan commented Feb 5, 2025 via email

@bbatsov
Copy link
Owner

bbatsov commented Feb 5, 2025

It used to matter in the past, but now it doesn't as Projectile initializes the known projects only when you trigger some project switching action, and that's what caused the problem for helm-projectile - it's checking directly the value of projectile-known-projects, assuming that projectile-mode has loaded it.

I assume for most people the way to use helm-projectile is by invoking this command command directly (and that's what I fixed yesterday), but I see there's also a know projects helm source that I need to update.

@bbatsov
Copy link
Owner

bbatsov commented Feb 5, 2025

Please test my second attempt to fix this. I hope I got it right this time around.

@umi2002
Copy link
Author

umi2002 commented Feb 5, 2025

Fixed it for me. Thanks!

@ohamov
Copy link

ohamov commented Feb 5, 2025

@bbatsov The fix still doesn't help. I can reproduce the issue, but now it manifests somehow differently. Also, I noticed that projectile-bookmarks.eld gets corrupted. Here is what it looks like (I opened two projects: emacs and awesome)
("~/.config/awesome/" #("~/.config/emacs" 0 15 (help-echo "/home/ohamov/.config/emacs" match-part #1 face helm-ff-directory)) #("~/.config/awesome" 0 17 (help-echo "/home/ohamov/.config/awesome" match-part #1 face helm-ff-directory)))

I also created manually the projectile-bookmarks.eld, launched emacs and invoked helm-projectile-switch-project. After I selected the project from the list, the bookmarks file got immediately corrupted. Of course, on the next invocation I couldn't get the list of project - it was empty because of the corruption of the bookmarks. Hope this helps

@bbatsov
Copy link
Owner

bbatsov commented Feb 5, 2025

Hmm, I'm out of ideas then. I'm still fairly certain there's some issue with helm-projectile-projects-source's definition, but I've forgotten pretty much everything about Helm's APIs.

I can't imagine how those help-echo entries got into you bookmarks file, though. That looks super weird.

@Thaodan
Copy link
Contributor

Thaodan commented Feb 5, 2025 via email

@ohamov
Copy link

ohamov commented Feb 5, 2025

I did some bisecting by installing old packages one by one:

Firstly, I installed helm-projectile 1.0.0 from melpa-stable + applied this patch 5813f72, as helm-projectile is broken in the stable repository.

After this step I see no corruption anymore, but the list of projects is always empty

Secondly, I installed projectile 2.8.0.

After this step, I see no issues at all (just like on my other laptop where those packages are also installed). So, the combination of projectile 2.8.0, helm-projectile 1.0.0 and the latest version of helm (20250204.1208) works as expected

@bbatsov
Copy link
Owner

bbatsov commented Feb 6, 2025

A bit of extra context - I'm pretty sure this change bbatsov/projectile@7d5e816 caused the breakage here, but I think my recent changes to helm-projectile should address it, because I see now other places where it was using projectile-known-projects directly.

I'm guessing one bullet-proof fix would be to be to have (projectile-load-known-projects) in your init.el, but this should be needed IMO. As for how weird entries like #("~/.config/emacs" 0 15 (help-echo "/home/ohamov/.config/emacs" match-part #1 face helm-ff-directory)) get added your bookmarks - that's extremely puzzling to me. I think the only way to figure this out would be to place a breakpoint in projectile-add-known-project and see what happens there.

See this for how to use the debugger https://emacsredux.com/blog/2025/02/03/debugging-emacs-commands/

@bbatsov bbatsov reopened this Feb 6, 2025
@bbatsov
Copy link
Owner

bbatsov commented Feb 6, 2025

It seems the problems is that Helm inserts some project paths fontified (no idea how exactly), which is very weird and I'm not sure how exactly is this happening and what triggered it. I have an idea about the fix.

Normally Emacs treats fontified strings as strings, though, so this should work in theory regardless that in looks weird in its serialized form.

@bbatsov
Copy link
Owner

bbatsov commented Feb 6, 2025

@ohamov Can you test with the commit before this one 9db7976 ? (probably you'll need to add (projectile-load-known-projects) somewhere in your config as well for things to work properly) I know you tried this already, but without the manual project loading. I want to narrow down what's causing the font-locking issue.

I'm like 99% this weird fontification happens in helm-projectile, and this seems like the only recent changes that might have caused it.

@bbatsov
Copy link
Owner

bbatsov commented Feb 6, 2025

I'm linking here #186, as it seems another person has the issue with font-locked entries corrupting their project bookmarks.

@ohamov
Copy link

ohamov commented Feb 6, 2025

@bbatsov I tested the parent commit of 9db7976 and added (projectile-load-known-projects) just before turning on the projectile mode.

I see no issues so far: the bookmarks file haven't got corrupted yet and the list of projects is also ok

@bbatsov
Copy link
Owner

bbatsov commented Feb 6, 2025

@ohamov Great news! @Thaodan, so it seems that something in the source definition change messed up how known projects are saved for some people, but I'm struggling to understand how project names get saved straight from the helm buffer with some fontification in place.

@Thaodan
Copy link
Contributor

Thaodan commented Feb 6, 2025 via email

@bbatsov
Copy link
Owner

bbatsov commented Feb 6, 2025

Same here, but it's definitely happening somewhere, otherwise people won't be getting those weird fontified helm-ff-directory properties in their known projects. It's like known projects get added to list straight from the helm buffer somehow.

pkryger added a commit to pkryger/helm-projectile that referenced this issue Feb 7, 2025
I am not sure if this is the correct way to fix, yet this workaround seems to
work in my case. I had very similar observations as others claimed in bbatsov#185 when
trying to debug this issue. Namely, right after executing
`helm-projectile-swithc-project` I observed that `projectile-known-projects`
had names changed with fontification.

Fixes bbatsov#185
@pkryger
Copy link
Contributor

pkryger commented Feb 7, 2025

I have observed very similar issues. Namely, that the list of know projects started to disappear and noticed

Error during file deserialization: (invalid-read-syntax "#")

in *Messages* buffer. Also the ~/.emacs.d/projectile-bookmarks.eld had strings with properties stored. I managed to narrow it to helm-type-file added to helm-projectile-projects-source: when I took it out, and evaluated the buffer the issue disappeared - I could change the project with helm-projectile-switch-project).

I proposed #187 as a fix: create a separate list of candidates. I am not sure if this is the best way to fix it. Perhaps @thierryvolpiatto could weight in on why the list uses as a source is updated (likely, but not necessarily) by helm.

@thierryvolpiatto
Copy link

thierryvolpiatto commented Feb 7, 2025 via email

@bbatsov
Copy link
Owner

bbatsov commented Feb 7, 2025

But from what I understand helm-projectile is listing projects right?
If it adds text properties to these candidates (face, display, anything
else..) with e.g. add-text-properties, it modifies the candidates of
this list directly by side effect. If it then tries to save this list
of projects (by printing the list presumably) it tries to save
candidates like #("foo" 0 3 (face font-lock-warning-face)) and fails
with "invalid read syntax #" as expected.

Yeah, the problematic part of the code is just a Helm source linked to a list of Projectile projects that get modified when we didn't expect them to. But from what you wrote it seems these modification of the underlying source from Helm is expected. Do I get this right? What are we supposed to do if we want to avoid it.

@thierryvolpiatto
Copy link

thierryvolpiatto commented Feb 8, 2025 via email

@Thaodan
Copy link
Contributor

Thaodan commented Feb 8, 2025 via email

@bbatsov
Copy link
Owner

bbatsov commented Feb 13, 2025

Strip out text properties before saving or using a copy of list (like in
#187) ?

@thierryvolpiatto I was thinking of stripping them, but I'm just not sure what exactly gets triggered by Helm here, as there are no direct references to projectile-add-known-project (the function that updates the project bookmarks) in helm-projectile.el. I might add it there to be on safe side, but I've been trying to understand how the change of the Helm source in #181 resulted in this.

@thierryvolpiatto
Copy link

thierryvolpiatto commented Feb 13, 2025 via email

@thierryvolpiatto
Copy link

thierryvolpiatto commented Feb 19, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants