fix: entity detection prefers git repo directory names over README content

[tavaresgmg]

Summary

Fixes #97 — mempalace init on a folder of git repositories now detects repo directory names as project candidates instead of surfacing generic words scraped from README files.

Problem: Running mempalace init <folder-of-repos> where the folder contains ~40 sibling git repositories produces useless entity suggestions like "Code (21x), Typescript (10x), Node (5x)" instead of the actual repo names (acme-dashboard, acme-chess, etc.).

Root cause: detect_entities() only scans file content for capitalized proper nouns. Directory names — the strongest signal for project identity — are completely ignored.

Changes

• detect_directory_projects(project_dir) — new function that scans for immediate child directories containing .git (supports both directories and worktree files) and returns them as high-confidence (0.95) project entities
• detect_entities(file_paths, base_dir=None) — new base_dir parameter triggers directory-based detection and merges results with content-based detection (deduplicating by name)
• 50+ tech stopwords added — common programming terms (Code, Typescript, Node, Plugin, Icon, React, Docker, etc.) that appear capitalized in READMEs but are not real entities
• onboarding.py updated — _auto_detect() now returns both detected people and directory-detected projects, and run_onboarding() presents them separately during the interactive flow
• CLI entry point updated — entity_detector.py __main__ now passes base_dir to detect_entities()

Test plan

• 29 new tests in tests/test_entity_detector.py (all passing)
• detect_directory_projects: finds git repos, skips non-git/hidden/SKIP_DIRS, handles worktrees, returns empty for nonexistent dirs
• detect_entities with base_dir: includes directory projects, merges without duplicates, sorted by confidence
• Stopwords: all issue-reported words filtered, real entity names preserved
• score_entity/classify_entity: person and project signal detection verified
• scan_for_detection: prose files found, .git skipped, fallback to readable files
• Full test suite passes (126/128 — 2 pre-existing failures in test_dialect.py unrelated to this PR)
• ruff check and ruff format clean

[bgauryy]

PR Review: fix: entity detection prefers git repo directory names over README content

Executive Summary

Aspect	Value
PR Goal	Fix mempalace init on folders of git repos to detect repo directory names instead of generic README words
Files Changed	3 (2 modified, 1 new test file)
Risk Level	🟢 LOW — backward-compatible parameter addition, well-isolated new function, solid test coverage
Review Effort	2/5 — focused bug fix with clear scope
Recommendation	✅ APPROVE

Affected Areas: mempalace/entity_detector.py (core detection logic), mempalace/onboarding.py (interactive setup), tests/test_entity_detector.py (new)

Business Impact: Users running mempalace init on a folder of git repositories will now see actual project names (e.g. acme-dashboard) instead of generic words like "Code (21x), Typescript (10x)".

Flow Changes: detect_entities() gains an optional base_dir parameter that triggers directory-based project scanning. _auto_detect() in onboarding now returns a tuple (people, dir_projects) instead of a flat list. The onboarding flow presents directory-detected projects separately before name candidates.

Ratings

Aspect	Score
Correctness	5/5
Security	5/5
Performance	4/5
Maintainability	4/5

PR Health

• Has clear description
• References ticket/issue (Entity detection ignores directory names, surfaces generic words from READMEs #97)
• Appropriate size (423 additions, but 266 are new tests)
• Has relevant tests (comprehensive — 8 test classes, covers edge cases)

Medium Priority Issues

🎨 #1: Unused prose_dir fixture parameter in test

Location: tests/test_entity_detector.py — test_merges_content_and_directory_projects | Confidence: ✅ HIGH

The prose_dir fixture is injected as a parameter but never referenced in the test body. The test creates its own content directly in repos_dir. This adds a misleading dependency and triggers an unnecessary tmp_path fixture setup.

- def test_merges_content_and_directory_projects(self, repos_dir, prose_dir):
+ def test_merges_content_and_directory_projects(self, repos_dir):

---

🎨 #2: Conditional ternary in _auto_detect can be simplified

Location: mempalace/onboarding.py — _auto_detect() | Confidence: ✅ HIGH

The if files / else ternary calls detect_entities with the same base_dir in both branches — the only difference is whether files is passed or []. Since detect_entities already handles empty file_paths gracefully (it produces an empty combined_text and no candidates), the conditional is unnecessary.

-        detected = (
-            detect_entities(files, base_dir=directory)
-            if files
-            else detect_entities([], base_dir=directory)
-        )
+        detected = detect_entities(files or [], base_dir=directory)

Even simpler: scan_for_detection already returns a list, so files is always a list (possibly empty). The entire ternary can be just detect_entities(files, base_dir=directory).

---

Low Priority Issues

🎨 #3: Framework names in stopwords could suppress legitimate project entities

Location: mempalace/entity_detector.py:396-459 | Confidence: ❓ LOW

Adding "react", "vue", "svelte", "angular", "docker", "figma" to STOPWORDS prevents them from ever being detected as project entities — even for repos actually named "React" or "Docker". In the README-scanning context this is the right trade-off (they're almost always generic references), but worth noting that a project literally named "React" would be invisible to content-based detection. Directory-based detection (detect_directory_projects) would still catch it if it's a git repo child directory.

No code change needed — just documenting the design trade-off.

---

🎨 #4: Double dedup of directory projects

Location: mempalace/onboarding.py:395-405 | Confidence: ✅ HIGH

detect_entities() already deduplicates directory projects against content-detected projects (case-insensitive). Then run_onboarding() deduplicates again against known_project_names. The second dedup isn't wrong — it adds a layer against projects already entered manually by the user — but it's worth a brief comment explaining that the first dedup is content vs. directory, while the second is all-detected vs. user-provided.

No code change required — the defense-in-depth is fine, just slightly opaque.

---

Flow Impact Analysis

BEFORE:
  mempalace init <dir>
    └── _auto_detect(dir, people) → list[people]
         └── detect_entities(files) → {people, projects, uncertain}
              └── (content-based only: README words like "Code", "Typescript")

AFTER:
  mempalace init <dir>
    └── _auto_detect(dir, people) → tuple[people, dir_projects]
         └── detect_entities(files, base_dir=dir) → {people, projects, uncertain}
              ├── content-based detection (with expanded stopwords)
              └── detect_directory_projects(dir) → git repo child dirs at 0.95 confidence

Backward compatibility: The base_dir=None default means all existing callers are unaffected. The only callers are onboarding.py (updated) and the __main__ block (updated).

---

Created by Octocode MCP https://octocode.ai

[bensig]

This conflicts with main. The entity detection improvements are being addressed in #507 (NLP providers). If there are specific fixes not covered there, a rebased focused PR would be welcome.