Feature/advanced data pipeline v2.1#27
Merged
SanghunYun95 merged 71 commits intomainfrom Mar 28, 2026
Merged
Conversation
…and observer cleanup
…-deps, and update docs
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthrough문서 기반 협업 규칙·RAG 고도화 계획·보안 지침 문서 추가, 로컬 HuggingFace 임베딩 전환, Supabase 클라이언트 팩토리화, 메타데이터 업데이트 스크립트 추가, 다수의 Project Gutenberg 텍스트 자산 추가 및 헬스체크 워크플로 삭제가 포함된 변경입니다. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~40 minutes Possibly related PRs
시 🐰
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
backend/verify_and_clear.py (1)
10-15:⚠️ Potential issue | 🟡 Minor클라이언트 초기화 예외가 try 블록 밖에 있어 오류 진단 메시지가 누락됩니다.
Line 11의
get_client()실패는 아래 예외 처리 흐름에 포함되지 않아, 실행 시 원인 파악이 어려워집니다.backend/scripts/check_db.py처럼 초기화도 try 내부에 두는 편이 안전합니다.🛠️ 제안 변경
def verify_and_clear(): - supabase_client = get_client() print("Clearing 'documents' table...") # Delete all rows try: + supabase_client = get_client() total_deleted = 0 while True:🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/verify_and_clear.py` around lines 10 - 15, The client initialization get_client() is outside the try so failures aren't caught; move the call to get_client() inside the existing try in verify_and_clear(), wrap it so any exceptions during initialization are caught, log or print the exception with context (e.g., "Failed to initialize supabase client: {err}") and then re-raise or return; ensure the rest of the function (deleting rows and using total_deleted) still runs only if get_client() succeeded.backend/app/services/embedding.py (1)
9-10:⚠️ Potential issue | 🟡 MinorREADME 문서와 임베딩 모델 불일치 수정 필요
README.md (36번, 143번 줄)에서는
jhgan/ko-sroberta-multitask를 임베딩 모델로 명시하고 있으나, 실제 코드(backend/app/services/embedding.py 9번 줄)에서는sentence-transformers/all-MiniLM-L6-v2를 사용합니다. README를 현재 구현과 일치하도록 업데이트해주세요.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/app/services/embedding.py` around lines 9 - 10, The README references the wrong embedding model; align documentation with the code by updating README occurrences of the model to match the constants in backend/app/services/embedding.py: replace any `jhgan/ko-sroberta-multitask` entries with the current MODEL_NAME value `sentence-transformers/all-MiniLM-L6-v2` and ensure any listed embedding dimension matches EXPECTED_EMBEDDING_DIM (384), so docs reflect the actual MODEL_NAME and embedding size used by the code.
🧹 Nitpick comments (8)
data/As a man thinketh by James Allen.txt (1)
1-27: RAG 품질을 위해 Gutenberg 머리말/라이선스 구간 분리를 권장합니다.현재 파일은 원문 보존 목적에는 적절하지만, 전/후면 boilerplate(헤더·라이선스)가 본문 임베딩에 섞이면 검색 품질이 떨어질 수 있습니다. 수집 단계에서 본문 범위만 추출하거나,
raw/clean파일을 분리해 관리하는 구성이 더 안정적입니다.Also applies to: 988-1338
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@data/As` a man thinketh by James Allen.txt around lines 1 - 27, Split the Gutenberg boilerplate from the main text by detecting the known boundary marker "*** START OF THE PROJECT GUTENBERG EBOOK AS A MAN THINKETH ***" (and corresponding END markers) and produce two managed artifacts: a raw file preserving the full original (including header/license) and a clean file containing only the main body between those markers (excluding header/footer/license). Update ingestion/collection code or pipeline to prefer the clean file for embeddings/search and keep the raw file for archival/attribution; if automated, add extraction logic that trims everything before the START marker and after the END marker (also apply same treatment for the similar block ranges noted, e.g., 988-1338) and store metadata (source, license) alongside the clean text.data/How to Live on 24 Hours a Day by Arnold Bennett.txt (1)
1-1: Line 1 BOM 문자 제거를 권장합니다.파일 시작의 보이지 않는 BOM 문자(
)는 전처리/해시/토큰화 결과를 미세하게 바꿀 수 있습니다. UTF-8 without BOM으로 정규화해 두는 편이 안전합니다.정리 예시(diff)
-The Project Gutenberg eBook of How to Live on 24 Hours a Day +The Project Gutenberg eBook of How to Live on 24 Hours a Day🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@data/How` to Live on 24 Hours a Day by Arnold Bennett.txt at line 1, The file begins with a UTF-8 BOM character (U+FEFF) before "The Project Gutenberg eBook of How to Live on 24 Hours a Day"; remove that invisible BOM from the start of the file and re-save the file as UTF-8 without BOM so tokenization/hashing is stable, and if you have repository hooks or preprocessors (e.g., a normalize-encoding or pre-commit hook), ensure they enforce UTF-8 without BOM for this file name.data/Dhammapada a Collection of Verses Being One of the Canonical Books of the Buddhists.txt (1)
1-1: UTF-8 BOM 제거 권장Line 1 시작에 BOM 문자가 포함되어 있습니다. 데이터 적재/해시/전처리 일관성을 위해 BOM 없는 UTF-8(가능하면 LF 통일)로 저장하는 편이 안전합니다.
제안 수정
-The Project Gutenberg eBook of Dhammapada, a Collection of Verses; Being One of the Canonical Books of the Buddhists +The Project Gutenberg eBook of Dhammapada, a Collection of Verses; Being One of the Canonical Books of the Buddhists🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@data/Dhammapada` a Collection of Verses Being One of the Canonical Books of the Buddhists.txt at line 1, The file begins with a UTF-8 BOM (the invisible U+FEFF at the start of "The Project Gutenberg eBook..."); remove the BOM so the file is saved as plain UTF-8 without BOM and normalize line endings to LF (ensure any CRLF are converted to LF) to keep hashing and preprocessing consistent—open the file, strip any leading BOM from the first byte/character, convert all CRLF to LF, and re-save as UTF-8 (no BOM).data/Index of the Project Gutenberg Works of Lucius Annaeus Seneca by Lucius Annaeus Seneca.txt (1)
1-1: 데이터 파이프라인에서 BOM 처리 방안 재검토 필요이 파일의 첫 번째 줄에 UTF-8 BOM 문자(
)가 포함되어 있습니다. 그러나 검증 결과, 데이터 디렉토리의 모든 430개 이상의 .txt 파일이 동일하게 BOM 문자를 포함하고 있습니다. 이는 개별 파일 문제가 아닌 Project Gutenberg 데이터 소스의 일관된 특성입니다.BOM이 텍스트 처리 시스템에서 문제를 일으킨다면, 단일 파일 수정이 아닌 다음 중 하나의 방식으로 해결해야 합니다:
- 데이터 파이프라인 단계에서 모든 파일의 BOM을 제거
- 데이터 로딩 시 BOM을 자동으로 무시하도록 처리
현재 파이프라인이 BOM을 정상적으로 처리한다면 추가 조치는 필요하지 않습니다.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@data/Index` of the Project Gutenberg Works of Lucius Annaeus Seneca by Lucius Annaeus Seneca.txt at line 1, 파일 "Index of the Project Gutenberg Works of Lucius Annaeus Seneca.txt"의 첫 줄에 UTF-8 BOM 문자()가 포함되어 있으며 동일한 BOM이 데이터 디렉토리의 430+ .txt 파일에 일관되게 존재하므로 개별 파일 수정이 아닌 파이프라인 단에서 해결해야 합니다; 수정 방법은 데이터 파이프라인의 파일 읽기/로딩 단계(예: 텍스트 로더 함수)에서 파일 바디를 읽기 전 또는 직후에 BOM 문자를 제거하도록 처리하거나, 파일 디코딩 시 BOM을 자동으로 무시하도록 설정하여 모든 입력 파일에서 선행 BOM을 일괄 제거·무시하도록 구현하면 됩니다..agent/documents/stories/001.advanced_rag_system.md (1)
47-47: 문서 내 고정 라인 번호 참조는 유지보수 시 빠르게 부정확해집니다.
llm.py L89-91같은 표기는 코드 변경 직후 깨질 가능성이 높습니다. 파일 경로 + 함수/심볼 기준으로 참조하는 형태가 더 안정적입니다.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.agent/documents/stories/001.advanced_rag_system.md at line 47, Replace the brittle line-number reference "llm.py L89-91" with a stable symbol-based reference: point to the file and the relevant function/constant name that implements the Strict Instruction (e.g., reference "llm.py" and the function or constant that defines the system prompt such as the symbol implementing Strict Instruction), so documentation shows "llm.py: <symbol_name>" instead of line numbers; update the mention near "시스템 프롬프트에 `Strict Instruction` 추가" to use that file+symbol format and, if known, include a short parenthetical of how to find it (e.g., function or class name) for maintainability.backend/download_books.py (1)
183-185: 다운로드 목표 수는 하드코딩보다 설정값으로 분리하는 편이 안전합니다.
target_count = 300고정값은 환경별 실행 제어가 어려워 운영/테스트 전환 시 매번 코드 수정이 필요합니다.♻️ 제안 변경
-def main(): +def main(target_count: int = 300): @@ - downloaded_count = 0 - target_count = 300 + downloaded_count = 0 @@ -if __name__ == '__main__': - main() +if __name__ == '__main__': + target_count = int(os.getenv("TARGET_COUNT", "300")) + main(target_count=target_count)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/download_books.py` around lines 183 - 185, Replace the hardcoded download target by reading a configurable setting (e.g., from environment or existing config) instead of setting target_count = 300; update the code around downloaded_count and target_count in download_books.py to obtain the target from a config key like DOWNLOAD_TARGET (with a sensible default) or a settings object, validate/convert it to int, and use that value so the download target can be changed per environment without editing code.backend/scripts/ingest_data.py (1)
18-21: 불필요한 래퍼 함수 간소화 권장
get_supabase_client()함수는get_client()를 단순히 호출만 하므로 불필요한 추상화입니다. 직접get_client()를 사용하면 코드가 더 간결해집니다.♻️ 간소화 제안
from app.services.database import get_client from langchain.text_splitter import RecursiveCharacterTextSplitter -def get_supabase_client(): - return get_client() - -supabase_client = get_supabase_client() +supabase_client = get_client()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/scripts/ingest_data.py` around lines 18 - 21, The get_supabase_client wrapper is redundant because it only calls get_client(); remove the get_supabase_client function and replace its usages by calling get_client() directly (e.g., set supabase_client = get_client()), ensuring any imports or references to get_supabase_client are updated to use get_client instead.backend/scripts/update_metadata.py (1)
115-116: 광범위한 예외 처리 및 로깅 개선정적 분석 도구가 지적한 대로, 구체적인 예외 타입을 지정하고
str(e)대신!r변환 플래그를 사용하면 더 나은 디버깅 정보를 제공합니다.♻️ 개선 제안
- except Exception as e: - logger.error(f"[{i+1}/{total}] Error updating {title}: {str(e)}") + except Exception as e: + logger.error(f"[{i+1}/{total}] Error updating {title}: {e!r}")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/scripts/update_metadata.py` around lines 115 - 116, The current broad except block around the metadata update loop (the except Exception as e that logs logger.error(f"[{i+1}/{total}] Error updating {title}: {str(e)}")) should be narrowed to concrete exception types relevant to the operations in this routine (e.g., ValueError, KeyError, IOError/OSError, requests.exceptions.RequestException, or whatever specific exceptions your update functions raise) and handle them separately as appropriate; also change the log call to include the exception repr (use {!r} or %r) so you get traceback-friendly info (e.g., logger.error(f"[{i+1}/{total}] Error updating {title}: {e!r}")). Ensure you keep the loop context (i, total, title) in the message and only fall back to a final broad except if you re-raise after logging.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.agent/documents/bmad.md:
- Around line 1-3: The document currently references "Contract Ledger" in the
title and opening paragraph but this repo is for "Philo-RAG"; update the title
line "# Contract Ledger BMAD-METHOD 통합 가이드라인" and any occurrences of the string
"Contract Ledger" in the file to "Philo-RAG" (or the correct project name), and
scan the document for other leftover project-specific names to replace so the
guide matches the Philo-RAG project context.
In `@backend/scripts/check_db.py`:
- Around line 4-5: The script currently uses sys.path.append(...) to add the
project root which can let an installed package with the same name shadow the
local app; change the path insertion to add the computed project root at the
front of sys.path (use sys.path.insert(0, project_root)) so the local package is
resolved first—update the line that calls
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) in
backend/scripts/check_db.py accordingly and ensure the inserted value is the
same computed path.
In `@backend/scripts/update_metadata.py`:
- Around line 100-112: The current update builds update_data from
res.data[0]["metadata"] and applies it to all matching rows via
supabase.table("documents").update(...).eq(...).execute(), which overwrites
per-document fields like chunk_index; instead, for each document matched by
eq("metadata->book_info->>title", title) merge only the new keys
("kr_title","thumbnail","link") into each row's metadata (preserving existing
metadata.chunk_index) — either by iterating over the matched rows and calling
update with a merged metadata per-row, or by issuing a server-side JSONB merge
(jsonb_set or jsonb ||) via raw SQL/RPC so the update does not replace the
entire metadata object; update the logic around update_data and the
supabase.table(...).update(...) call accordingly.
In `@data/A` Pickle for the Knowing Ones by Timothy Dexter.txt:
- Line 1: The file "data/A Pickle for the Knowing Ones by Timothy Dexter.txt"
contains a UTF-8 BOM (U+FEFF) at the very start (Line 1); remove the BOM so
downstream token/character comparisons aren't contaminated by the invisible
character—either re-save the file without BOM or update the ingestion logic to
strip a leading '\uFEFF' (check for and remove the BOM before processing) so the
first character/token is the real content.
In `@data/As` a man thinketh by James Allen.txt:
- Line 1: 파일 시작 부분의 보이지 않는 BOM 문자 (U+FEFF) 때문에 첫 토큰 불일치가 발생할 수 있으니 파일 "The
Project Gutenberg eBook of As a man thinketh"의 첫 줄 앞에 있는 BOM(\uFEFF)을 제거하고 UTF-8
without BOM로 저장하세요; 즉 파일의 첫 문자가 "The Project Gutenberg eBook of As a man
thinketh"로 바로 시작하도록 편집기나 스크립(예: 텍스트 인코딩 설정 또는 스트립 전처리 단계)으로 BOM을 삭제하도록 수정하세요.
In `@data/Critias` by Plato.txt:
- Line 1: The file "Critias by Plato.txt" contains a leading UTF-8 BOM character
(U+FEFF) at Line 1; remove the BOM by deleting the leading invisible character
from the start of the file (or replace the leading U+FEFF if present) and
re-save the file as UTF-8 without BOM so downstream text-preprocessing/embedding
pipelines receive clean text.
In `@data/Euthyphro` by Plato.txt:
- Line 1: 파일 시작에 포함된 UTF-8 BOM 문자 (U+FEFF, Zero Width No-Break Space)를 제거하고 파일을
UTF-8 without BOM 형식으로 재저장한 뒤 텍스트의 줄종결자를 LF로 정규화하세요; 즉 현재 파일의 선두 문자(U+FEFF)를
삭제하고 인코딩을 UTF-8(no BOM)으로 설정한 다음 모든 CRLF를 LF로 변환하여 인덱싱/청킹/해시 일관성이 유지되도록 하세요.
- Around line 1-1508: The file contains excessive Project Gutenberg
header/footer and full license text that will pollute embeddings; update
ingestion to separate raw vs clean text by extracting only the dialogue between
the "*** START OF THE PROJECT GUTENBERG EBOOK" and "*** END OF THE PROJECT
GUTENBERG EBOOK ***" markers (or the specific "*** START ... ***" / "*** END ...
***" boundaries) for indexing, store the full file as raw archival content, and
ensure your ingestion pipeline (the reader/parser step) strips
header/meta/license blocks before tokenization/embedding so search uses only the
clean body text.
In `@data/Index` of the Project Gutenberg Works of Friedrich Nietzsche by
Friedrich Wilhelm Nietzsche.txt:
- Line 1: The file begins with a UTF-8 BOM (bytes 0xEF 0xBB 0xBF) at Line 1
which must be removed; open "Index of the Project Gutenberg Works of Friedrich
Nietzsche by Friedrich Wilhelm Nietzsche.txt", strip the leading BOM bytes so
the file is saved as plain UTF-8 without BOM, and re-save/commit the normalized
file to prevent hash/chunking/string-matching mismatches caused by the BOM.
In `@data/Index` of the Project Gutenberg Works of Immanuel Kant by Immanuel
Kant.txt:
- Around line 108-121: The file contains corrupted replacement characters (�) in
the text (e.g., the substring "First Part�TRANSCENDENTAL AESTHETIC" and other
occurrences around "SECTION I. OF SPACE") which degrades corpus quality; replace
the damaged lines with the original UTF-8 text by re-ingesting or restoring the
source for those passages (ensuring proper UTF-8 encoding) so that all instances
of "�" are removed and the phrases such as "First Part — TRANSCENDENTAL
AESTHETIC", "SECTION I. OF SPACE", "SECTION II. OF TIME", etc., are correct and
intact.
In `@data/Ion` by Plato.txt:
- Line 1: The file "Ion by Plato.txt" begins with a BOM character ('\uFEFF')
which can corrupt title/first-token parsing; remove the BOM by re-saving the
file as UTF-8 without BOM or add a preprocessing step in the text loader (e.g.,
the function that reads text files / preprocessText) to strip any leading
'\uFEFF' (check and trimStart the character) so downstream parsers receive clean
input.
In `@data/The` Enchiridion by Epictetus.txt:
- Line 1: Remove the leading UTF-8 BOM (U+FEFF) from the start of the file so
the initial string "The Project Gutenberg eBook of The Enchiridion" is not
prefixed by a hidden character, and save the file as UTF-8 without BOM; also
normalize line endings to a consistent format (prefer LF) across the file to
avoid tokenization/embedding issues.
- Around line 3-9: Add a repository-level dataset manifest/README that records
source (Project Gutenberg), jurisdictional constraints (US-centric license with
"check local laws" note), redistribution conditions (Project Gutenberg License
text or link), permitted use cases and disallowed uses, and a per-file index
covering the affected documents (those referenced as 1411-1497); update the data
ingestion/docs so the manifest is referenced wherever the dataset is consumed
and ensure the README/manifest includes contact/owner, SPDX-like license
identifier, and a short guidance section for non-US users on compliance steps.
In `@README.md`:
- Around line 223-224: Remove the duplicate instructional line "Open
`http://localhost:3000` to start using the system." from the README so it only
appears once; search for that exact sentence and delete the redundant instance,
ensuring surrounding paragraphs and headings remain unchanged.
---
Outside diff comments:
In `@backend/app/services/embedding.py`:
- Around line 9-10: The README references the wrong embedding model; align
documentation with the code by updating README occurrences of the model to match
the constants in backend/app/services/embedding.py: replace any
`jhgan/ko-sroberta-multitask` entries with the current MODEL_NAME value
`sentence-transformers/all-MiniLM-L6-v2` and ensure any listed embedding
dimension matches EXPECTED_EMBEDDING_DIM (384), so docs reflect the actual
MODEL_NAME and embedding size used by the code.
In `@backend/verify_and_clear.py`:
- Around line 10-15: The client initialization get_client() is outside the try
so failures aren't caught; move the call to get_client() inside the existing try
in verify_and_clear(), wrap it so any exceptions during initialization are
caught, log or print the exception with context (e.g., "Failed to initialize
supabase client: {err}") and then re-raise or return; ensure the rest of the
function (deleting rows and using total_deleted) still runs only if get_client()
succeeded.
---
Nitpick comments:
In @.agent/documents/stories/001.advanced_rag_system.md:
- Line 47: Replace the brittle line-number reference "llm.py L89-91" with a
stable symbol-based reference: point to the file and the relevant
function/constant name that implements the Strict Instruction (e.g., reference
"llm.py" and the function or constant that defines the system prompt such as the
symbol implementing Strict Instruction), so documentation shows "llm.py:
<symbol_name>" instead of line numbers; update the mention near "시스템 프롬프트에
`Strict Instruction` 추가" to use that file+symbol format and, if known, include a
short parenthetical of how to find it (e.g., function or class name) for
maintainability.
In `@backend/download_books.py`:
- Around line 183-185: Replace the hardcoded download target by reading a
configurable setting (e.g., from environment or existing config) instead of
setting target_count = 300; update the code around downloaded_count and
target_count in download_books.py to obtain the target from a config key like
DOWNLOAD_TARGET (with a sensible default) or a settings object, validate/convert
it to int, and use that value so the download target can be changed per
environment without editing code.
In `@backend/scripts/ingest_data.py`:
- Around line 18-21: The get_supabase_client wrapper is redundant because it
only calls get_client(); remove the get_supabase_client function and replace its
usages by calling get_client() directly (e.g., set supabase_client =
get_client()), ensuring any imports or references to get_supabase_client are
updated to use get_client instead.
In `@backend/scripts/update_metadata.py`:
- Around line 115-116: The current broad except block around the metadata update
loop (the except Exception as e that logs logger.error(f"[{i+1}/{total}] Error
updating {title}: {str(e)}")) should be narrowed to concrete exception types
relevant to the operations in this routine (e.g., ValueError, KeyError,
IOError/OSError, requests.exceptions.RequestException, or whatever specific
exceptions your update functions raise) and handle them separately as
appropriate; also change the log call to include the exception repr (use {!r} or
%r) so you get traceback-friendly info (e.g., logger.error(f"[{i+1}/{total}]
Error updating {title}: {e!r}")). Ensure you keep the loop context (i, total,
title) in the message and only fall back to a final broad except if you re-raise
after logging.
In `@data/As` a man thinketh by James Allen.txt:
- Around line 1-27: Split the Gutenberg boilerplate from the main text by
detecting the known boundary marker "*** START OF THE PROJECT GUTENBERG EBOOK AS
A MAN THINKETH ***" (and corresponding END markers) and produce two managed
artifacts: a raw file preserving the full original (including header/license)
and a clean file containing only the main body between those markers (excluding
header/footer/license). Update ingestion/collection code or pipeline to prefer
the clean file for embeddings/search and keep the raw file for
archival/attribution; if automated, add extraction logic that trims everything
before the START marker and after the END marker (also apply same treatment for
the similar block ranges noted, e.g., 988-1338) and store metadata (source,
license) alongside the clean text.
In `@data/Dhammapada` a Collection of Verses Being One of the Canonical Books of
the Buddhists.txt:
- Line 1: The file begins with a UTF-8 BOM (the invisible U+FEFF at the start of
"The Project Gutenberg eBook..."); remove the BOM so the file is saved as plain
UTF-8 without BOM and normalize line endings to LF (ensure any CRLF are
converted to LF) to keep hashing and preprocessing consistent—open the file,
strip any leading BOM from the first byte/character, convert all CRLF to LF, and
re-save as UTF-8 (no BOM).
In `@data/How` to Live on 24 Hours a Day by Arnold Bennett.txt:
- Line 1: The file begins with a UTF-8 BOM character (U+FEFF) before "The
Project Gutenberg eBook of How to Live on 24 Hours a Day"; remove that invisible
BOM from the start of the file and re-save the file as UTF-8 without BOM so
tokenization/hashing is stable, and if you have repository hooks or
preprocessors (e.g., a normalize-encoding or pre-commit hook), ensure they
enforce UTF-8 without BOM for this file name.
In `@data/Index` of the Project Gutenberg Works of Lucius Annaeus Seneca by Lucius
Annaeus Seneca.txt:
- Line 1: 파일 "Index of the Project Gutenberg Works of Lucius Annaeus
Seneca.txt"의 첫 줄에 UTF-8 BOM 문자()가 포함되어 있으며 동일한 BOM이 데이터 디렉토리의 430+ .txt 파일에
일관되게 존재하므로 개별 파일 수정이 아닌 파이프라인 단에서 해결해야 합니다; 수정 방법은 데이터 파이프라인의 파일 읽기/로딩 단계(예: 텍스트
로더 함수)에서 파일 바디를 읽기 전 또는 직후에 BOM 문자를 제거하도록 처리하거나, 파일 디코딩 시 BOM을 자동으로 무시하도록 설정하여
모든 입력 파일에서 선행 BOM을 일괄 제거·무시하도록 구현하면 됩니다.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 956be787-3ca7-4ef2-bcfd-e3a4bdb1ec3e
📒 Files selected for processing (300)
.agent/documents/bmad.md.agent/documents/improvement_plan.md.agent/documents/stories/001.advanced_rag_system.md.agent/rules/security_guideline.md.agent/rules/service.md.agent/skills/frontend-design.md.agent/skills/react-best-practice.md.github/workflows/keep-alive.ymlREADME.mdbackend/app/services/embedding.pybackend/download_books.pybackend/scripts/check_db.pybackend/scripts/ingest_data.pybackend/scripts/update_metadata.pybackend/verify_and_clear.pydata/A Biographical Dictionary of Freethinkers of All Ages and Nations by J M Wheeler.txtdata/A Budget of Paradoxes Volume I by Augustus De Morgan.txtdata/A Budget of Paradoxes Volume II by Augustus De Morgan.txtdata/A Commentary to Kants Critique of Pure Reason by Norman Kemp Smith.txtdata/A Contribution to the Critique of Political Economy by Karl Marx.txtdata/A Defence of Poetry and Other Essays by Percy Bysshe Shelley.txtdata/A Discourse Upon the Origin and the Foundation of the Inequality Among Mankind by Jean-Jacques Rousseau.txtdata/A Pickle for the Knowing Ones by Timothy Dexter.txtdata/A Selection from the Discourses of Epictetus with the Encheiridion by Epictetus.txtdata/A Short History of Freethought Ancient and Modern Volume 1 of 2 by J M Robertson.txtdata/A Short History of Freethought Ancient and Modern Volume 2 of 2 by J M Robertson.txtdata/A System of Logic Ratiocinative and Inductive by John Stuart Mill.txtdata/A Treatise of Human Nature by David Hume.txtdata/A Vindication of the Rights of Woman by Mary Wollstonecraft.txtdata/Aesthetical Essays of Friedrich Schiller by Friedrich Schiller.txtdata/Also sprach Zarathustra Ein Buch für Alle und Keinen German by Friedrich Wilhelm Nietzsche.txtdata/Amiels Journal The Journal Intime of Henri-Frédéric Amiel by Henri Frédéric Amiel.txtdata/An Enquiry Concerning Human Understanding by David Hume.txtdata/An Enquiry Concerning the Principles of Morals by David Hume.txtdata/An Essay Concerning Humane Understanding Volume 1 by John Locke.txtdata/An Essay on Man Moral Essays and Satires by Alexander Pope.txtdata/An Essay on the Development of Christian Doctrine by John Henry Newman.txtdata/Aphorismen zur Lebensweisheit German by Arthur Schopenhauer.txtdata/Apology Crito and Phaedo of Socrates by Plato.txtdata/Apology by Plato.txtdata/Areopagitica by John Milton.txtdata/Aristotle by George Grote.txtdata/As a man thinketh by James Allen.txtdata/Autobiography by John Stuart Mill.txtdata/Bacons Essays and Wisdom of the Ancients by Francis Bacon.txtdata/Balder the Beautiful Volume I by James George Frazer.txtdata/Beyond Good and Evil by Friedrich Wilhelm Nietzsche.txtdata/Bushido the Soul of Japan by Inazo Nitobe.txtdata/Character by Samuel Smiles.txtdata/Chaucers Works Volume 2 Boethius and Troilus by Geoffrey Chaucer.txtdata/Chuang Tzu Mystic Moralist and Social Reformer by Zhuangzi.txtdata/Ciceros Tusculan Disputations by Marcus Tullius Cicero.txtdata/Confessiones Latin by Saint of Hippo Augustine.txtdata/Considerations on Representative Government by John Stuart Mill.txtdata/Cratylus by Plato.txtdata/Creative Evolution by Henri Bergson.txtdata/Critias by Plato.txtdata/Daemonologie by King of England James I.txtdata/Dante and the early astronomers by M A Orr.txtdata/De Officiis Latin by Marcus Tullius Cicero.txtdata/Democracy and Education An Introduction to the Philosophy of Education by John Dewey.txtdata/Democracy in America Volume 2 by Alexis de Tocqueville.txtdata/Demonology and Devil-lore by Moncure Daniel Conway.txtdata/Der Wille zur Macht Eine Auslegung alles Geschehens German by Friedrich Wilhelm Nietzsche.txtdata/Devotions Upon Emergent Occasions Together with Deaths Duel by John Donne.txtdata/Dhammapada a Collection of Verses Being One of the Canonical Books of the Buddhists.txtdata/Dialogues Concerning Natural Religion by David Hume.txtdata/Discourse on the Method of Rightly Conducting Ones Reason and of Seeking Truth in the Sciences by René Descartes.txtdata/Discourses on the First Decade of Titus Livius by Niccolò Machiavelli.txtdata/Early Greek Philosophy Other Essays by Friedrich Wilhelm Nietzsche.txtdata/Early Greek philosophy by John Burnet.txtdata/Ecce Homo by Friedrich Wilhelm Nietzsche.txtdata/Emile by Jean-Jacques Rousseau.txtdata/Essays First Series by Ralph Waldo Emerson.txtdata/Essays by Ralph Waldo Emerson by Ralph Waldo Emerson.txtdata/Essays of Schopenhauer by Arthur Schopenhauer.txtdata/Essays of an Americanist by Daniel G Brinton.txtdata/Ethics by Benedictus de Spinoza.txtdata/Etiquette by Emily Post.txtdata/Euthyphro by Plato.txtdata/Friedrich Nietzsche by Georg Brandes.txtdata/Fundamental Principles of the Metaphysic of Morals by Immanuel Kant.txtdata/Galen On the Natural Faculties Ancient Greek to 1453 by Galen.txtdata/Goethes Theory of Colours by Johann Wolfgang von Goethe.txtdata/Gorgias by Plato.txtdata/Hegels Lectures on the History of Philosophy Volume 1 of 3 by Georg Wilhelm Friedrich Hegel.txtdata/Heretics by G K Chesterton.txtdata/Hidden Symbolism of Alchemy and the Occult Arts by Herbert Silberer.txtdata/History of European Morals From Augustus to Charlemagne Vol 1 of 2 by William Edward Hartpole Lecky.txtdata/Hours with the Mystics A Contribution to the History of Religious Opinion by Robert Alfred Vaughan.txtdata/How We Think by John Dewey.txtdata/How to Live on 24 Hours a Day by Arnold Bennett.txtdata/Human All Too Human A Book for Free Spirits by Friedrich Wilhelm Nietzsche.txtdata/Human All-Too-Human A Book for Free Spirits Part 1 by Friedrich Wilhelm Nietzsche.txtdata/Human All-Too-Human A Book for Free Spirits Part 2 by Friedrich Wilhelm Nietzsche.txtdata/Human Nature and Conduct An introduction to social psychology by John Dewey.txtdata/Iamblichus Life of Pythagoras or Pythagoric Life by Iamblichus.txtdata/Index of the Project Gutenberg Works of Friedrich Nietzsche by Friedrich Wilhelm Nietzsche.txtdata/Index of the Project Gutenberg Works of Immanuel Kant by Immanuel Kant.txtdata/Index of the Project Gutenberg Works of Lucius Annaeus Seneca by Lucius Annaeus Seneca.txtdata/Institutes of the Christian Religion Vol 1 of 2 by Jean Calvin.txtdata/Ion by Plato.txtdata/Isis unveiled Volume 1 of 2 Science A master-key to mysteries of ancient and modern science and theology by H P Blavatsky.txtdata/Isis unveiled Volume 2 of 2 Theology A master-key to the mysteries of ancient and modern science and theology by H P Blavatsky.txtdata/Journal 01 1837-1846 by Henry David Thoreau.txtdata/Kants Critique of Judgement by Immanuel Kant.txtdata/Kants Prolegomena to Any Future Metaphysics by Immanuel Kant.txtdata/Kants gesammelte Schriften Band V Kritik der Urtheilskraft German by Immanuel Kant.txtdata/L Annaeus Seneca on Benefits by Lucius Annaeus Seneca.txtdata/Laughter An Essay on the Meaning of the Comic by Henri Bergson.txtdata/Laws by Plato.txtdata/Leviathan by Thomas Hobbes.txtdata/Life and Correspondence of David Hume Volume 1 by John Hill Burton.txtdata/Lucifer A Theosophical Magazine Volume I September 1887-February 1888 by Various.txtdata/Machiavelli Volume I by Niccolò Machiavelli.txtdata/Man a Machine by Julien Offray de La Mettrie.txtdata/Man and Superman A Comedy and a Philosophy by Bernard Shaw.txtdata/Manifesto of the Communist Party by Karl Marx and Friedrich Engels.txtdata/Maxims and Reflections by Johann Wolfgang von Goethe.txtdata/Meditations by Emperor of Rome Marcus Aurelius.txtdata/Meno by Plato.txtdata/Minor Dialogues Together With the Dialogue on Clemency by Lucius Annaeus Seneca.txtdata/Moral Theology by Charles J Callan and John A McHugh.txtdata/Morals and Dogma of the Ancient and Accepted Scottish Rite of Freemasonry by Albert Pike.txtdata/Mysterious Psychic Forces by Camille Flammarion.txtdata/Nature by Ralph Waldo Emerson.txtdata/Nietzsche and Other Exponents of Individualism by Paul Carus.txtdata/Novum organum or True suggestions for the interpretation of nature by Francis Bacon.txtdata/On Heroes Hero-Worship and the Heroic in History by Thomas Carlyle.txtdata/On Liberty by John Stuart Mill.txtdata/On Love by Stendhal.txtdata/On War by Carl von Clausewitz.txtdata/On the Duty of Civil Disobedience by Henry David Thoreau.txtdata/On the Fourfold Root of the Principle of Sufficient Reason and On the Will in Nature Two Essays revised edition by Arthur Schopenhauer.txtdata/On the Nature of Things by Titus Lucretius Carus.txtdata/On the Sublime by active 1st century Longinus.txtdata/Orthodoxy by G K Chesterton.txtdata/Outlines of Educational Doctrine by Johann Friedrich Herbart.txtdata/Pascals Pensées by Blaise Pascal.txtdata/Perpetual Peace A Philosophical Essay by Immanuel Kant.txtdata/Phaedo by Plato.txtdata/Phaedrus by Plato.txtdata/Philosophiae Naturalis Principia Mathematica Latin by Isaac Newton.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 1 by George Grote.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 2 by George Grote.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 3 by George Grote.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 4 by George Grote.txtdata/Plutarchs Morals by Plutarch.txtdata/Politics A Treatise on Government by Aristotle.txtdata/Pragmatism A New Name for Some Old Ways of Thinking by William James.txtdata/Protagoras by Plato.txtdata/Psychology of the Unconscious by C G Jung.txtdata/Rasselas Prince of Abyssinia by Samuel Johnson.txtdata/Reflections or Sentences and Moral Maxims by François duc de La Rochefoucauld.txtdata/Revelations of Divine Love by of Norwich Julian.txtdata/Ritual of the Order of the Eastern Star by Order of the Eastern Star General Grand Chapter.txtdata/Roman Stoicism by Edward Vernon Arnold.txtdata/Rousseau Volume 1 and 2 by John Morley.txtdata/Rousseau and Romanticism by Irving Babbitt.txtdata/Sartor Resartus The Life and Opinions of Herr Teufelsdröckh by Thomas Carlyle.txtdata/Sartor Resartus and On Heroes Hero-Worship and the Heroic in History by Thomas Carlyle.txtdata/Second Treatise of Government by John Locke.txtdata/Selected Works of Voltairine de Cleyre by Voltairine De Cleyre.txtdata/Selections from the Speeches and Writings of Edmund Burke by Edmund Burke.txtdata/Selections from the Writings of Kierkegaard by Søren Kierkegaard.txtdata/Self Help with Illustrations of Conduct and Perseverance by Samuel Smiles.txtdata/Self Knowledge and Guide to Sex Instruction Vital Facts of Life for All Ages by Thomas W Shannon.txtdata/Senecas Morals of a Happy Life Benefits Anger and Clemency by Lucius Annaeus Seneca.txtdata/Siddhartha by Hermann Hesse.txtdata/Sophist by Plato.txtdata/Summa Theologica Part I Prima Pars by Saint Aquinas Thomas.txtdata/Summa Theologica Part I-II Pars Prima Secundae by Saint Aquinas Thomas.txtdata/Summa Theologica Part II-II Secunda Secundae by Saint Aquinas Thomas.txtdata/Sun Tzŭ on the Art of War The Oldest Military Treatise in the World by active 6th century BC Sunzi.txtdata/Symposium by Plato.txtdata/The Academic Questions Treatise De Finibus and Tusculan Disputations of MT Cicero With a Sketch of the Greek Philosophers Mentioned by Cicero by Marcus Tullius Cicero.txtdata/The Advancement of Learning by Francis Bacon.txtdata/The Analects of Confucius from the Chinese Classics by Confucius.txtdata/The Analysis of Mind by Bertrand Russell.txtdata/The Anatomy of Melancholy by Robert Burton.txtdata/The Antichrist by Friedrich Wilhelm Nietzsche.txtdata/The Augsburg Confession by Philipp Melanchthon.txtdata/The Basis of Morality by Arthur Schopenhauer.txtdata/The Birth of Tragedy or Hellenism and Pessimism by Friedrich Wilhelm Nietzsche.txtdata/The Book of Tea by Kakuzo Okakura.txtdata/The Book of the Damned by Charles Fort.txtdata/The Case of Wagner Nietzsche Contra Wagner and Selected Aphorisms by Friedrich Wilhelm Nietzsche.txtdata/The Catholic World Vol 19 April 1874September 1874 by Various.txtdata/The Catholic World Vol 26 October 1877 to March 1878 by Various.txtdata/The City of God Volume I by Saint of Hippo Augustine.txtdata/The City of God Volume II by Saint of Hippo Augustine.txtdata/The Communist Manifesto by Karl Marx and Friedrich Engels.txtdata/The Concept of Nature by Alfred North Whitehead.txtdata/The Confessions of Jean Jacques Rousseau Complete by Jean-Jacques Rousseau.txtdata/The Confessions of St Augustine by Saint of Hippo Augustine.txtdata/The Consolation of Philosophy by Boethius.txtdata/The Critique of Practical Reason by Immanuel Kant.txtdata/The Critique of Pure Reason by Immanuel Kant.txtdata/The Crown of Wild Olive by John Ruskin.txtdata/The Dawn of Day by Friedrich Wilhelm Nietzsche.txtdata/The Declaration of Independence of the United States of America by Thomas Jefferson.txtdata/The Diamond Sutra Chin-Kang-Ching or Prajna-Paramita by Unknown.txtdata/The Discoverie of Witchcraft by Reginald Scot.txtdata/The Economist by Xenophon.txtdata/The Enchiridion by Epictetus.txtdata/The Essays of Arthur Schopenhauer Counsels and Maxims by Arthur Schopenhauer.txtdata/The Essays of Arthur Schopenhauer Studies in Pessimism by Arthur Schopenhauer.txtdata/The Essays of Arthur Schopenhauer the Art of Controversy by Arthur Schopenhauer.txtdata/The Essays of Arthur Schopenhauer the Wisdom of Life by Arthur Schopenhauer.txtdata/The Essays or Counsels Civil and Moral by Francis Bacon.txtdata/The Ethics of Aristotle by Aristotle.txtdata/The Everlasting Man by G K Chesterton.txtdata/The Existence and Attributes of God Volumes 1 and 2 by Stephen Charnock.txtdata/The Fable of the Bees Or Private Vices Public Benefits by Bernard Mandeville.txtdata/The First Blast of the Trumpet against the monstrous regiment of Women by John Knox.txtdata/The Five Great Philosophies of Life by William De Witt Hyde.txtdata/The Genealogy of Morals by Friedrich Wilhelm Nietzsche.txtdata/The Golden Bough A Study in Magic and Religion Third Edition Vol 09 of 12 by James George Frazer.txtdata/The Grand Inquisitor by Fyodor Dostoyevsky.txtdata/The Greek Philosophers Vol 2 of 2 by Alfred William Benn.txtdata/The History of Freedom and Other Essays by Baron John Emerich Edward Dalberg Acton Acton.txtdata/The Idea of a University Defined and Illustrated by John Henry Newman.txtdata/The Illustrated Key to the Tarot The Veil of Divination by L W De Laurence.txtdata/The Imitation of Christ by à Kempis Thomas.txtdata/The Instruction of Ptah-Hotep and the Instruction of KeGemni by Ptahhotep and Kagemna.txtdata/The Itching Palm A Study of the Habit of Tipping in America by William R Scott.txtdata/The Japanese Spirit by Yoshisaburo Okakura.txtdata/The Joyful Wisdom La Gaya Scienza by Friedrich Wilhelm Nietzsche.txtdata/The Kama Sutra of Vatsyayana by Vatsyayana.txtdata/The Kingdom of God Is Within You by graf Leo Tolstoy.txtdata/The Kybalion by Three Initiates.txtdata/The Letters of William James Vol 2 by William James.txtdata/The Life of Reason The Phases of Human Progress by George Santayana.txtdata/The Lives and Opinions of Eminent Philosophers by Diogenes Laertius.txtdata/The Logic of Hegel by Georg Wilhelm Friedrich Hegel.txtdata/The Magic and Science of Jewels and Stones by Isidore Kozminsky.txtdata/The Man Who Was Thursday A Nightmare by G K Chesterton.txtdata/The Man of Genius by Cesare Lombroso.txtdata/The Marriage of Heaven and Hell by William Blake.txtdata/The Meaning of Truth by William James.txtdata/The Mediaeval Mind Volume 2 of 2 by Henry Osborn Taylor.txtdata/The Meditations of the Emperor Marcus Aurelius Antoninus by Emperor of Rome Marcus Aurelius.txtdata/The Memorabilia by Xenophon.txtdata/The Methods of Ethics by Henry Sidgwick.txtdata/The New Jerusalem by G K Chesterton.txtdata/The Note-Books of Samuel Butler by Samuel Butler.txtdata/The Philosophy of Friedrich Nietzsche by H L Mencken.txtdata/The Poetics of Aristotle by Aristotle.txtdata/The Prince by Niccolò Machiavelli.txtdata/The Principles of Psychology Volume 1 of 2 by William James.txtdata/The Problem of China by Bertrand Russell.txtdata/The Problems of Philosophy by Bertrand Russell.txtdata/The Prophet by Kahlil Gibran.txtdata/The Republic by Plato.txtdata/The Republic of Plato by Plato.txtdata/The Rubaiyat of Omar Khayyam by Omar Khayyam.txtdata/The Ruins Or Meditation on the Revolutions of Empires and the Law of Nature by C-F Volney.txtdata/The Secret Doctrine Vol 1 of 4 by H P Blavatsky.txtdata/The Secret Doctrine Vol 2 of 4 by H P Blavatsky.txtdata/The Secret Doctrine Vol 3 of 4 by H P Blavatsky.txtdata/The Seven Lamps of Architecture by John Ruskin.txtdata/The Sexual Life of Our Time in Its Relations to Modern Civilization by Iwan Bloch.txtdata/The Song Celestial Or Bhagavad-Gîtâ from the Mahâbhârata.txtdata/The Subjection of Women by John Stuart Mill.txtdata/The Tao Teh King or the Tao and its Characteristics by Laozi.txtdata/The Theory of Moral Sentiments by Adam Smith.txtdata/The Twilight of the Idols or How to Philosophize with the Hammer The Antichrist by Friedrich Wilhelm Nietzsche.txtdata/The Varieties of Religious Experience A Study in Human Nature by William James.txtdata/The Will to Believe and Other Essays in Popular Philosophy by William James.txtdata/The Will to Power An Attempted Transvaluation of All Values Book I and II by Friedrich Wilhelm Nietzsche.txtdata/The Will to Power An Attempted Transvaluation of All Values Book III and IV by Friedrich Wilhelm Nietzsche.txtdata/The Works of George Berkeley Vol 1 of 4 Philosophical Works 1705-21 by George Berkeley.txtdata/The Works of Robert G Ingersoll Complete Contents by Robert Green Ingersoll.txtdata/The Works of the Emperor Julian Vol 1 by Emperor of Rome Julian.txtdata/The Works of the Right Honourable Edmund Burke Vol 01 of 12 by Edmund Burke.txtdata/The World as Will and Idea Vol 1 of 3 by Arthur Schopenhauer.txtdata/The World as Will and Idea Vol 2 of 3 by Arthur Schopenhauer.txtdata/The World as Will and Idea Vol 3 of 3 by Arthur Schopenhauer.txtdata/The decline of the West Volume 1 Form and actuality by Oswald Spengler.txtdata/The guide for the perplexed by Moses Maimonides.txtdata/The history of magic including a clear and precise exposition of its procedure its rites and its mysteries by Éliphas Lévi.txtdata/The life of Friedrich Nietzsche by Daniel Halévy.txtdata/The origin and development of the moral ideas by Edward Westermarck.txtdata/The principles of science a treatise on logic and scientific method by William Stanley Jevons.txtdata/The quest of the historical Jesus a critical study of its progress from Reimarus to Wrede by Albert Schweitzer.txtdata/The republic of Cicero by Marcus Tullius Cicero.txtdata/The social contract discourses by Jean-Jacques Rousseau.txtdata/The symbolism of Freemasonry Illustrating and explaining its science and philosophy its legends myths and symbols by Albert Gallatin Mackey.txtdata/The writings of Clement of Alexandria Vol 1 of 2 by Saint of Alexandria Clement.txtdata/Theaetetus by Plato.txtdata/Theodicy by Freiherr von Gottfried Wilhelm Leibniz.txtdata/Thought-Forms by Annie Besant and C W Leadbeater.txtdata/Thoughts of Marcus Aurelius Antoninus by Emperor of Rome Marcus Aurelius.txtdata/Thus Spake Zarathustra A Book for All and None by Friedrich Wilhelm Nietzsche.txtdata/Timaeus by Plato.txtdata/Time and Free Will An Essay on the Immediate Data of Consciousness by Henri Bergson.txtdata/Tragic Sense Of Life by Miguel de Unamuno.txtdata/Treatises on Friendship and Old Age by Marcus Tullius Cicero.txtdata/Utilitarianism by John Stuart Mill.txtdata/Utopia by Saint Thomas More.txt
💤 Files with no reviewable changes (2)
- .github/workflows/keep-alive.yml
- .agent/rules/service.md
Comment on lines
+4
to
+5
| # Ensure we can import app modules | ||
| sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) |
There was a problem hiding this comment.
모듈 경로를 append로 추가하면 잘못된 app 패키지를 참조할 수 있습니다.
Line 5는 경로를 뒤에 추가하므로, 환경에 설치된 동명 패키지가 먼저 해석될 수 있습니다. 로컬 경로를 우선하도록 앞에 삽입하는 것이 안전합니다.
🔧 제안 변경
-sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # Ensure we can import app modules | |
| sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) | |
| # Ensure we can import app modules | |
| sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/scripts/check_db.py` around lines 4 - 5, The script currently uses
sys.path.append(...) to add the project root which can let an installed package
with the same name shadow the local app; change the path insertion to add the
computed project root at the front of sys.path (use sys.path.insert(0,
project_root)) so the local package is resolved first—update the line that calls
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) in
backend/scripts/check_db.py accordingly and ensure the inserted value is the
same computed path.
| @@ -0,0 +1,1678 @@ | |||
| The Project Gutenberg eBook of A Pickle for the Knowing Ones | |||
There was a problem hiding this comment.
BOM 문자 제거 권장
Line 1의 파일 시작에 UTF-8 BOM()이 포함되어 있습니다. 데이터 파이프라인에서 첫 토큰/첫 문자 비교를 오염시킬 수 있어 제거하는 편이 안전합니다.
제안 diff
-The Project Gutenberg eBook of A Pickle for the Knowing Ones
+The Project Gutenberg eBook of A Pickle for the Knowing Ones📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| The Project Gutenberg eBook of A Pickle for the Knowing Ones | |
| The Project Gutenberg eBook of A Pickle for the Knowing Ones |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/A` Pickle for the Knowing Ones by Timothy Dexter.txt at line 1, The file
"data/A Pickle for the Knowing Ones by Timothy Dexter.txt" contains a UTF-8 BOM
(U+FEFF) at the very start (Line 1); remove the BOM so downstream
token/character comparisons aren't contaminated by the invisible
character—either re-save the file without BOM or update the ingestion logic to
strip a leading '\uFEFF' (check for and remove the BOM before processing) so the
first character/token is the real content.
| @@ -0,0 +1,1338 @@ | |||
| The Project Gutenberg eBook of As a man thinketh | |||
There was a problem hiding this comment.
BOM 문자 제거를 권장합니다.
Line 1 시작의 보이지 않는 BOM(\uFEFF)은 일부 파서/청킹 로직에서 첫 토큰 불일치나 전처리 누락을 유발할 수 있습니다. UTF-8 without BOM으로 저장해 두는 편이 안전합니다.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/As` a man thinketh by James Allen.txt at line 1, 파일 시작 부분의 보이지 않는 BOM 문자
(U+FEFF) 때문에 첫 토큰 불일치가 발생할 수 있으니 파일 "The Project Gutenberg eBook of As a man
thinketh"의 첫 줄 앞에 있는 BOM(\uFEFF)을 제거하고 UTF-8 without BOM로 저장하세요; 즉 파일의 첫 문자가
"The Project Gutenberg eBook of As a man thinketh"로 바로 시작하도록 편집기나 스크립(예: 텍스트 인코딩
설정 또는 스트립 전처리 단계)으로 BOM을 삭제하도록 수정하세요.
Comment on lines
+108
to
+121
| First Part�TRANSCENDENTAL AESTHETIC | ||
| � 1. Introductory | ||
| SECTION I. OF SPACE | ||
| � 2. Metaphysical Exposition of this Conception. | ||
| � 3. Transcendental Exposition of the Conception of Space. | ||
| � 4. Conclusions from the foregoing Conceptions. | ||
| SECTION II. OF TIME | ||
| � 5. Metaphysical Exposition of this Conception. | ||
| � 6. Transcendental Exposition of the Conception of Time. | ||
| � 7. Conclusions from the above Conceptions. | ||
| � 8. Elucidation. | ||
| � 9. General Remarks on Transcendental Aesthetic. | ||
| � 10. Conclusion of the Transcendental Aesthetic. | ||
| Second Part�TRANSCENDENTAL LOGIC |
There was a problem hiding this comment.
텍스트 인코딩 손상 문자가 포함되어 코퍼스 품질이 저하됩니다.
Line 108 부근의 �(replacement character)처럼 원문이 깨진 문자가 포함되어 있어 임베딩/검색 정합도가 떨어질 수 있습니다. 손상 없는 UTF-8 원문으로 재수집/교체해 주세요.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/Index` of the Project Gutenberg Works of Immanuel Kant by Immanuel
Kant.txt around lines 108 - 121, The file contains corrupted replacement
characters (�) in the text (e.g., the substring "First Part�TRANSCENDENTAL
AESTHETIC" and other occurrences around "SECTION I. OF SPACE") which degrades
corpus quality; replace the damaged lines with the original UTF-8 text by
re-ingesting or restoring the source for those passages (ensuring proper UTF-8
encoding) so that all instances of "�" are removed and the phrases such as
"First Part — TRANSCENDENTAL AESTHETIC", "SECTION I. OF SPACE", "SECTION II. OF
TIME", etc., are correct and intact.
| @@ -0,0 +1,1200 @@ | |||
| The Project Gutenberg eBook of Ion | |||
There was a problem hiding this comment.
파일 시작 BOM 문자 제거 권장 (수집/전처리 안정성)
Line 1 시작에 BOM(\uFEFF)이 포함되어 있어, 파이프라인에서 첫 토큰/제목 파싱 시 잡음이 섞일 수 있습니다. BOM 없이 UTF-8로 저장하는 편이 안전합니다.
제안 변경 예시
-The Project Gutenberg eBook of Ion
+The Project Gutenberg eBook of Ion📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| The Project Gutenberg eBook of Ion | |
| The Project Gutenberg eBook of Ion |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/Ion` by Plato.txt at line 1, The file "Ion by Plato.txt" begins with a
BOM character ('\uFEFF') which can corrupt title/first-token parsing; remove the
BOM by re-saving the file as UTF-8 without BOM or add a preprocessing step in
the text loader (e.g., the function that reads text files / preprocessText) to
strip any leading '\uFEFF' (check and trimStart the character) so downstream
parsers receive clean input.
| @@ -0,0 +1,1756 @@ | |||
| The Project Gutenberg eBook of The Enchiridion | |||
There was a problem hiding this comment.
UTF-8 BOM 제거를 권장합니다.
파일 시작 BOM(U+FEFF)은 텍스트 임베딩/토크나이징 시 첫 토큰 오염을 유발할 수 있습니다. BOM 없는 UTF-8 + 일관된 줄바꿈으로 정규화해 주세요.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/The` Enchiridion by Epictetus.txt at line 1, Remove the leading UTF-8
BOM (U+FEFF) from the start of the file so the initial string "The Project
Gutenberg eBook of The Enchiridion" is not prefixed by a hidden character, and
save the file as UTF-8 without BOM; also normalize line endings to a consistent
format (prefer LF) across the file to avoid tokenization/embedding issues.
Comment on lines
+3
to
+9
| This eBook is for the use of anyone anywhere in the United States and | ||
| most other parts of the world at no cost and with almost no restrictions | ||
| whatsoever. You may copy it, give it away or re-use it under the terms | ||
| of the Project Gutenberg License included with this eBook or online | ||
| at www.gutenberg.org. If you are not located in the United States, | ||
| you will have to check the laws of the country where you are located | ||
| before using this eBook. |
There was a problem hiding this comment.
비미국권 사용 조건에 대한 저장소 차원의 컴플라이언스 가드가 필요합니다.
본문이 “미국 외 지역은 현지법 확인 필요”를 명시하고 있어, 공개 저장소/배포 파이프라인에서 지역별 저작권 준수 정책이 없으면 법적 리스크가 생깁니다. 데이터셋 매니페스트(출처, 관할, 재배포 조건, 허용 사용범위)를 저장소 레벨에 명시해 주세요.
권장 보완안 (예: data/README.md 추가)
+## The Enchiridion by Epictetus (Project Gutenberg `#45109`)
+
+- Source: https://www.gutenberg.org/ebooks/45109
+- Jurisdiction note: Public-domain handling is stated for the U.S.; non-U.S. use must follow local copyright law.
+- Redistribution requirement: Keep Project Gutenberg license/notice text attached.
+- Intended use in this repo: Research/processing under repository data policy.Also applies to: 1411-1497
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/The` Enchiridion by Epictetus.txt around lines 3 - 9, Add a
repository-level dataset manifest/README that records source (Project
Gutenberg), jurisdictional constraints (US-centric license with "check local
laws" note), redistribution conditions (Project Gutenberg License text or link),
permitted use cases and disallowed uses, and a per-file index covering the
affected documents (those referenced as 1411-1497); update the data
ingestion/docs so the manifest is referenced wherever the dataset is consumed
and ensure the README/manifest includes contact/owner, SPDX-like license
identifier, and a short guidance section for non-US users on compliance steps.
Comment on lines
+223
to
+224
|
|
||
| Open `http://localhost:3000` to start using the system. |
There was a problem hiding this comment.
중복된 라인 제거 필요
동일한 안내 문구가 두 번 반복되어 있습니다.
🔧 수정 제안
Open `http://localhost:3000` to start using the system.
-
-Open `http://localhost:3000` to start using the system.
-🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@README.md` around lines 223 - 224, Remove the duplicate instructional line
"Open `http://localhost:3000` to start using the system." from the README so it
only appears once; search for that exact sentence and delete the redundant
instance, ensuring surrounding paragraphs and headings remain unchanged.
…ling, metadata update fix)
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/scripts/update_metadata.py`:
- Around line 101-116: The current loop updates chunks for a single book with
multiple supabase.table("documents").update(...) calls inside the for doc in
res.data loop and swallows exceptions in the except block, causing partial
commits if one update fails; change this to perform an atomic book-level update
by either calling a server-side JSONB merge/RPC that updates all rows for a
given book id in one request (instead of per-doc updates) or collect all per-doc
update payloads and execute a single batch/transactional update, and if any
update fails, surface the error (raise or exit non-zero) rather than only
logging it; update references: the for doc in res.data loop, update_data,
supabase.table("documents").update(...).eq("id", doc["id"]).execute(), and the
except Exception as e handler to implement the atomic/transactional behavior and
proper error propagation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e8fb7b21-5987-479f-a6b0-392ff8bd1291
📒 Files selected for processing (292)
.agent/documents/bmad.md.agent/documents/stories/001.advanced_rag_system.mdREADME.mdbackend/download_books.pybackend/scripts/ingest_data.pybackend/scripts/update_metadata.pybackend/verify_and_clear.pydata/A Biographical Dictionary of Freethinkers of All Ages and Nations by J M Wheeler.txtdata/A Budget of Paradoxes Volume I by Augustus De Morgan.txtdata/A Budget of Paradoxes Volume II by Augustus De Morgan.txtdata/A Commentary to Kants Critique of Pure Reason by Norman Kemp Smith.txtdata/A Contribution to the Critique of Political Economy by Karl Marx.txtdata/A Defence of Poetry and Other Essays by Percy Bysshe Shelley.txtdata/A Discourse Upon the Origin and the Foundation of the Inequality Among Mankind by Jean-Jacques Rousseau.txtdata/A Pickle for the Knowing Ones by Timothy Dexter.txtdata/A Selection from the Discourses of Epictetus with the Encheiridion by Epictetus.txtdata/A Short History of Freethought Ancient and Modern Volume 1 of 2 by J M Robertson.txtdata/A Short History of Freethought Ancient and Modern Volume 2 of 2 by J M Robertson.txtdata/A System of Logic Ratiocinative and Inductive by John Stuart Mill.txtdata/A Treatise of Human Nature by David Hume.txtdata/A Vindication of the Rights of Woman by Mary Wollstonecraft.txtdata/Aesthetical Essays of Friedrich Schiller by Friedrich Schiller.txtdata/Also sprach Zarathustra Ein Buch für Alle und Keinen German by Friedrich Wilhelm Nietzsche.txtdata/Amiels Journal The Journal Intime of Henri-Frédéric Amiel by Henri Frédéric Amiel.txtdata/An Enquiry Concerning Human Understanding by David Hume.txtdata/An Enquiry Concerning the Principles of Morals by David Hume.txtdata/An Essay Concerning Humane Understanding Volume 1 by John Locke.txtdata/An Essay on Man Moral Essays and Satires by Alexander Pope.txtdata/An Essay on the Development of Christian Doctrine by John Henry Newman.txtdata/Aphorismen zur Lebensweisheit German by Arthur Schopenhauer.txtdata/Apology Crito and Phaedo of Socrates by Plato.txtdata/Apology by Plato.txtdata/Areopagitica by John Milton.txtdata/Aristotle by George Grote.txtdata/As a man thinketh by James Allen.txtdata/Autobiography by John Stuart Mill.txtdata/Bacons Essays and Wisdom of the Ancients by Francis Bacon.txtdata/Balder the Beautiful Volume I by James George Frazer.txtdata/Beyond Good and Evil by Friedrich Wilhelm Nietzsche.txtdata/Bushido the Soul of Japan by Inazo Nitobe.txtdata/Character by Samuel Smiles.txtdata/Chaucers Works Volume 2 Boethius and Troilus by Geoffrey Chaucer.txtdata/Chuang Tzu Mystic Moralist and Social Reformer by Zhuangzi.txtdata/Ciceros Tusculan Disputations by Marcus Tullius Cicero.txtdata/Confessiones Latin by Saint of Hippo Augustine.txtdata/Considerations on Representative Government by John Stuart Mill.txtdata/Cratylus by Plato.txtdata/Creative Evolution by Henri Bergson.txtdata/Critias by Plato.txtdata/Daemonologie by King of England James I.txtdata/Dante and the early astronomers by M A Orr.txtdata/De Officiis Latin by Marcus Tullius Cicero.txtdata/Democracy and Education An Introduction to the Philosophy of Education by John Dewey.txtdata/Democracy in America Volume 2 by Alexis de Tocqueville.txtdata/Demonology and Devil-lore by Moncure Daniel Conway.txtdata/Der Wille zur Macht Eine Auslegung alles Geschehens German by Friedrich Wilhelm Nietzsche.txtdata/Devotions Upon Emergent Occasions Together with Deaths Duel by John Donne.txtdata/Dhammapada a Collection of Verses Being One of the Canonical Books of the Buddhists.txtdata/Dialogues Concerning Natural Religion by David Hume.txtdata/Discourse on the Method of Rightly Conducting Ones Reason and of Seeking Truth in the Sciences by René Descartes.txtdata/Discourses on the First Decade of Titus Livius by Niccolò Machiavelli.txtdata/Early Greek Philosophy Other Essays by Friedrich Wilhelm Nietzsche.txtdata/Early Greek philosophy by John Burnet.txtdata/Ecce Homo by Friedrich Wilhelm Nietzsche.txtdata/Emile by Jean-Jacques Rousseau.txtdata/Essays First Series by Ralph Waldo Emerson.txtdata/Essays by Ralph Waldo Emerson by Ralph Waldo Emerson.txtdata/Essays of Schopenhauer by Arthur Schopenhauer.txtdata/Essays of an Americanist by Daniel G Brinton.txtdata/Ethics by Benedictus de Spinoza.txtdata/Etiquette by Emily Post.txtdata/Euthyphro by Plato.txtdata/Friedrich Nietzsche by Georg Brandes.txtdata/Fundamental Principles of the Metaphysic of Morals by Immanuel Kant.txtdata/Galen On the Natural Faculties Ancient Greek to 1453 by Galen.txtdata/Goethes Theory of Colours by Johann Wolfgang von Goethe.txtdata/Gorgias by Plato.txtdata/Hegels Lectures on the History of Philosophy Volume 1 of 3 by Georg Wilhelm Friedrich Hegel.txtdata/Heretics by G K Chesterton.txtdata/Hidden Symbolism of Alchemy and the Occult Arts by Herbert Silberer.txtdata/History of European Morals From Augustus to Charlemagne Vol 1 of 2 by William Edward Hartpole Lecky.txtdata/Hours with the Mystics A Contribution to the History of Religious Opinion by Robert Alfred Vaughan.txtdata/How We Think by John Dewey.txtdata/How to Live on 24 Hours a Day by Arnold Bennett.txtdata/Human All Too Human A Book for Free Spirits by Friedrich Wilhelm Nietzsche.txtdata/Human All-Too-Human A Book for Free Spirits Part 1 by Friedrich Wilhelm Nietzsche.txtdata/Human All-Too-Human A Book for Free Spirits Part 2 by Friedrich Wilhelm Nietzsche.txtdata/Human Nature and Conduct An introduction to social psychology by John Dewey.txtdata/Iamblichus Life of Pythagoras or Pythagoric Life by Iamblichus.txtdata/Index of the Project Gutenberg Works of Friedrich Nietzsche by Friedrich Wilhelm Nietzsche.txtdata/Index of the Project Gutenberg Works of Immanuel Kant by Immanuel Kant.txtdata/Index of the Project Gutenberg Works of Lucius Annaeus Seneca by Lucius Annaeus Seneca.txtdata/Institutes of the Christian Religion Vol 1 of 2 by Jean Calvin.txtdata/Ion by Plato.txtdata/Isis unveiled Volume 1 of 2 Science A master-key to mysteries of ancient and modern science and theology by H P Blavatsky.txtdata/Isis unveiled Volume 2 of 2 Theology A master-key to the mysteries of ancient and modern science and theology by H P Blavatsky.txtdata/Journal 01 1837-1846 by Henry David Thoreau.txtdata/Kants Critique of Judgement by Immanuel Kant.txtdata/Kants Prolegomena to Any Future Metaphysics by Immanuel Kant.txtdata/Kants gesammelte Schriften Band V Kritik der Urtheilskraft German by Immanuel Kant.txtdata/L Annaeus Seneca on Benefits by Lucius Annaeus Seneca.txtdata/Laughter An Essay on the Meaning of the Comic by Henri Bergson.txtdata/Laws by Plato.txtdata/Leviathan by Thomas Hobbes.txtdata/Life and Correspondence of David Hume Volume 1 by John Hill Burton.txtdata/Lucifer A Theosophical Magazine Volume I September 1887-February 1888 by Various.txtdata/Machiavelli Volume I by Niccolò Machiavelli.txtdata/Man a Machine by Julien Offray de La Mettrie.txtdata/Man and Superman A Comedy and a Philosophy by Bernard Shaw.txtdata/Manifesto of the Communist Party by Karl Marx and Friedrich Engels.txtdata/Maxims and Reflections by Johann Wolfgang von Goethe.txtdata/Meditations by Emperor of Rome Marcus Aurelius.txtdata/Meno by Plato.txtdata/Minor Dialogues Together With the Dialogue on Clemency by Lucius Annaeus Seneca.txtdata/Moral Theology by Charles J Callan and John A McHugh.txtdata/Morals and Dogma of the Ancient and Accepted Scottish Rite of Freemasonry by Albert Pike.txtdata/Mysterious Psychic Forces by Camille Flammarion.txtdata/Nature by Ralph Waldo Emerson.txtdata/Nietzsche and Other Exponents of Individualism by Paul Carus.txtdata/Novum organum or True suggestions for the interpretation of nature by Francis Bacon.txtdata/On Heroes Hero-Worship and the Heroic in History by Thomas Carlyle.txtdata/On Liberty by John Stuart Mill.txtdata/On Love by Stendhal.txtdata/On War by Carl von Clausewitz.txtdata/On the Duty of Civil Disobedience by Henry David Thoreau.txtdata/On the Fourfold Root of the Principle of Sufficient Reason and On the Will in Nature Two Essays revised edition by Arthur Schopenhauer.txtdata/On the Nature of Things by Titus Lucretius Carus.txtdata/On the Sublime by active 1st century Longinus.txtdata/Orthodoxy by G K Chesterton.txtdata/Outlines of Educational Doctrine by Johann Friedrich Herbart.txtdata/Pascals Pensées by Blaise Pascal.txtdata/Perpetual Peace A Philosophical Essay by Immanuel Kant.txtdata/Phaedo by Plato.txtdata/Phaedrus by Plato.txtdata/Philosophiae Naturalis Principia Mathematica Latin by Isaac Newton.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 1 by George Grote.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 2 by George Grote.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 3 by George Grote.txtdata/Plato and the Other Companions of Sokrates 3rd ed Volume 4 by George Grote.txtdata/Plutarchs Morals by Plutarch.txtdata/Politics A Treatise on Government by Aristotle.txtdata/Pragmatism A New Name for Some Old Ways of Thinking by William James.txtdata/Protagoras by Plato.txtdata/Psychology of the Unconscious by C G Jung.txtdata/Rasselas Prince of Abyssinia by Samuel Johnson.txtdata/Reflections or Sentences and Moral Maxims by François duc de La Rochefoucauld.txtdata/Revelations of Divine Love by of Norwich Julian.txtdata/Ritual of the Order of the Eastern Star by Order of the Eastern Star General Grand Chapter.txtdata/Roman Stoicism by Edward Vernon Arnold.txtdata/Rousseau Volume 1 and 2 by John Morley.txtdata/Rousseau and Romanticism by Irving Babbitt.txtdata/Sartor Resartus The Life and Opinions of Herr Teufelsdröckh by Thomas Carlyle.txtdata/Sartor Resartus and On Heroes Hero-Worship and the Heroic in History by Thomas Carlyle.txtdata/Second Treatise of Government by John Locke.txtdata/Selected Works of Voltairine de Cleyre by Voltairine De Cleyre.txtdata/Selections from the Speeches and Writings of Edmund Burke by Edmund Burke.txtdata/Selections from the Writings of Kierkegaard by Søren Kierkegaard.txtdata/Self Help with Illustrations of Conduct and Perseverance by Samuel Smiles.txtdata/Self Knowledge and Guide to Sex Instruction Vital Facts of Life for All Ages by Thomas W Shannon.txtdata/Senecas Morals of a Happy Life Benefits Anger and Clemency by Lucius Annaeus Seneca.txtdata/Siddhartha by Hermann Hesse.txtdata/Sophist by Plato.txtdata/Summa Theologica Part I Prima Pars by Saint Aquinas Thomas.txtdata/Summa Theologica Part I-II Pars Prima Secundae by Saint Aquinas Thomas.txtdata/Summa Theologica Part II-II Secunda Secundae by Saint Aquinas Thomas.txtdata/Sun Tzŭ on the Art of War The Oldest Military Treatise in the World by active 6th century BC Sunzi.txtdata/Symposium by Plato.txtdata/The Academic Questions Treatise De Finibus and Tusculan Disputations of MT Cicero With a Sketch of the Greek Philosophers Mentioned by Cicero by Marcus Tullius Cicero.txtdata/The Advancement of Learning by Francis Bacon.txtdata/The Analects of Confucius from the Chinese Classics by Confucius.txtdata/The Analysis of Mind by Bertrand Russell.txtdata/The Anatomy of Melancholy by Robert Burton.txtdata/The Antichrist by Friedrich Wilhelm Nietzsche.txtdata/The Augsburg Confession by Philipp Melanchthon.txtdata/The Basis of Morality by Arthur Schopenhauer.txtdata/The Birth of Tragedy or Hellenism and Pessimism by Friedrich Wilhelm Nietzsche.txtdata/The Book of Tea by Kakuzo Okakura.txtdata/The Book of the Damned by Charles Fort.txtdata/The Case of Wagner Nietzsche Contra Wagner and Selected Aphorisms by Friedrich Wilhelm Nietzsche.txtdata/The Catholic World Vol 19 April 1874September 1874 by Various.txtdata/The Catholic World Vol 26 October 1877 to March 1878 by Various.txtdata/The City of God Volume I by Saint of Hippo Augustine.txtdata/The City of God Volume II by Saint of Hippo Augustine.txtdata/The Communist Manifesto by Karl Marx and Friedrich Engels.txtdata/The Concept of Nature by Alfred North Whitehead.txtdata/The Confessions of Jean Jacques Rousseau Complete by Jean-Jacques Rousseau.txtdata/The Confessions of St Augustine by Saint of Hippo Augustine.txtdata/The Consolation of Philosophy by Boethius.txtdata/The Critique of Practical Reason by Immanuel Kant.txtdata/The Critique of Pure Reason by Immanuel Kant.txtdata/The Crown of Wild Olive by John Ruskin.txtdata/The Dawn of Day by Friedrich Wilhelm Nietzsche.txtdata/The Declaration of Independence of the United States of America by Thomas Jefferson.txtdata/The Diamond Sutra Chin-Kang-Ching or Prajna-Paramita by Unknown.txtdata/The Discoverie of Witchcraft by Reginald Scot.txtdata/The Economist by Xenophon.txtdata/The Enchiridion by Epictetus.txtdata/The Essays of Arthur Schopenhauer Counsels and Maxims by Arthur Schopenhauer.txtdata/The Essays of Arthur Schopenhauer Studies in Pessimism by Arthur Schopenhauer.txtdata/The Essays of Arthur Schopenhauer the Art of Controversy by Arthur Schopenhauer.txtdata/The Essays of Arthur Schopenhauer the Wisdom of Life by Arthur Schopenhauer.txtdata/The Essays or Counsels Civil and Moral by Francis Bacon.txtdata/The Ethics of Aristotle by Aristotle.txtdata/The Everlasting Man by G K Chesterton.txtdata/The Existence and Attributes of God Volumes 1 and 2 by Stephen Charnock.txtdata/The Fable of the Bees Or Private Vices Public Benefits by Bernard Mandeville.txtdata/The First Blast of the Trumpet against the monstrous regiment of Women by John Knox.txtdata/The Five Great Philosophies of Life by William De Witt Hyde.txtdata/The Genealogy of Morals by Friedrich Wilhelm Nietzsche.txtdata/The Golden Bough A Study in Magic and Religion Third Edition Vol 09 of 12 by James George Frazer.txtdata/The Grand Inquisitor by Fyodor Dostoyevsky.txtdata/The Greek Philosophers Vol 2 of 2 by Alfred William Benn.txtdata/The History of Freedom and Other Essays by Baron John Emerich Edward Dalberg Acton Acton.txtdata/The Idea of a University Defined and Illustrated by John Henry Newman.txtdata/The Illustrated Key to the Tarot The Veil of Divination by L W De Laurence.txtdata/The Imitation of Christ by à Kempis Thomas.txtdata/The Instruction of Ptah-Hotep and the Instruction of KeGemni by Ptahhotep and Kagemna.txtdata/The Itching Palm A Study of the Habit of Tipping in America by William R Scott.txtdata/The Japanese Spirit by Yoshisaburo Okakura.txtdata/The Joyful Wisdom La Gaya Scienza by Friedrich Wilhelm Nietzsche.txtdata/The Kama Sutra of Vatsyayana by Vatsyayana.txtdata/The Kingdom of God Is Within You by graf Leo Tolstoy.txtdata/The Kybalion by Three Initiates.txtdata/The Letters of William James Vol 2 by William James.txtdata/The Life of Reason The Phases of Human Progress by George Santayana.txtdata/The Lives and Opinions of Eminent Philosophers by Diogenes Laertius.txtdata/The Logic of Hegel by Georg Wilhelm Friedrich Hegel.txtdata/The Magic and Science of Jewels and Stones by Isidore Kozminsky.txtdata/The Man Who Was Thursday A Nightmare by G K Chesterton.txtdata/The Man of Genius by Cesare Lombroso.txtdata/The Marriage of Heaven and Hell by William Blake.txtdata/The Meaning of Truth by William James.txtdata/The Mediaeval Mind Volume 2 of 2 by Henry Osborn Taylor.txtdata/The Meditations of the Emperor Marcus Aurelius Antoninus by Emperor of Rome Marcus Aurelius.txtdata/The Memorabilia by Xenophon.txtdata/The Methods of Ethics by Henry Sidgwick.txtdata/The New Jerusalem by G K Chesterton.txtdata/The Note-Books of Samuel Butler by Samuel Butler.txtdata/The Philosophy of Friedrich Nietzsche by H L Mencken.txtdata/The Poetics of Aristotle by Aristotle.txtdata/The Prince by Niccolò Machiavelli.txtdata/The Principles of Psychology Volume 1 of 2 by William James.txtdata/The Problem of China by Bertrand Russell.txtdata/The Problems of Philosophy by Bertrand Russell.txtdata/The Prophet by Kahlil Gibran.txtdata/The Republic by Plato.txtdata/The Republic of Plato by Plato.txtdata/The Rubaiyat of Omar Khayyam by Omar Khayyam.txtdata/The Ruins Or Meditation on the Revolutions of Empires and the Law of Nature by C-F Volney.txtdata/The Secret Doctrine Vol 1 of 4 by H P Blavatsky.txtdata/The Secret Doctrine Vol 2 of 4 by H P Blavatsky.txtdata/The Secret Doctrine Vol 3 of 4 by H P Blavatsky.txtdata/The Seven Lamps of Architecture by John Ruskin.txtdata/The Sexual Life of Our Time in Its Relations to Modern Civilization by Iwan Bloch.txtdata/The Song Celestial Or Bhagavad-Gîtâ from the Mahâbhârata.txtdata/The Subjection of Women by John Stuart Mill.txtdata/The Tao Teh King or the Tao and its Characteristics by Laozi.txtdata/The Theory of Moral Sentiments by Adam Smith.txtdata/The Twilight of the Idols or How to Philosophize with the Hammer The Antichrist by Friedrich Wilhelm Nietzsche.txtdata/The Varieties of Religious Experience A Study in Human Nature by William James.txtdata/The Will to Believe and Other Essays in Popular Philosophy by William James.txtdata/The Will to Power An Attempted Transvaluation of All Values Book I and II by Friedrich Wilhelm Nietzsche.txtdata/The Will to Power An Attempted Transvaluation of All Values Book III and IV by Friedrich Wilhelm Nietzsche.txtdata/The Works of George Berkeley Vol 1 of 4 Philosophical Works 1705-21 by George Berkeley.txtdata/The Works of Robert G Ingersoll Complete Contents by Robert Green Ingersoll.txtdata/The Works of the Emperor Julian Vol 1 by Emperor of Rome Julian.txtdata/The Works of the Right Honourable Edmund Burke Vol 01 of 12 by Edmund Burke.txtdata/The World as Will and Idea Vol 1 of 3 by Arthur Schopenhauer.txtdata/The World as Will and Idea Vol 2 of 3 by Arthur Schopenhauer.txtdata/The World as Will and Idea Vol 3 of 3 by Arthur Schopenhauer.txtdata/The decline of the West Volume 1 Form and actuality by Oswald Spengler.txtdata/The guide for the perplexed by Moses Maimonides.txtdata/The history of magic including a clear and precise exposition of its procedure its rites and its mysteries by Éliphas Lévi.txtdata/The life of Friedrich Nietzsche by Daniel Halévy.txtdata/The origin and development of the moral ideas by Edward Westermarck.txtdata/The principles of science a treatise on logic and scientific method by William Stanley Jevons.txtdata/The quest of the historical Jesus a critical study of its progress from Reimarus to Wrede by Albert Schweitzer.txtdata/The republic of Cicero by Marcus Tullius Cicero.txtdata/The social contract discourses by Jean-Jacques Rousseau.txtdata/The symbolism of Freemasonry Illustrating and explaining its science and philosophy its legends myths and symbols by Albert Gallatin Mackey.txtdata/The writings of Clement of Alexandria Vol 1 of 2 by Saint of Alexandria Clement.txtdata/Theaetetus by Plato.txtdata/Theodicy by Freiherr von Gottfried Wilhelm Leibniz.txtdata/Thought-Forms by Annie Besant and C W Leadbeater.txtdata/Thoughts of Marcus Aurelius Antoninus by Emperor of Rome Marcus Aurelius.txtdata/Thus Spake Zarathustra A Book for All and None by Friedrich Wilhelm Nietzsche.txtdata/Timaeus by Plato.txtdata/Time and Free Will An Essay on the Immediate Data of Consciousness by Henri Bergson.txtdata/Tragic Sense Of Life by Miguel de Unamuno.txtdata/Treatises on Friendship and Old Age by Marcus Tullius Cicero.txtdata/Utilitarianism by John Stuart Mill.txtdata/Utopia by Saint Thomas More.txt
✅ Files skipped from review due to trivial changes (17)
- data/On the Duty of Civil Disobedience by Henry David Thoreau.txt
- data/As a man thinketh by James Allen.txt
- data/The Declaration of Independence of the United States of America by Thomas Jefferson.txt
- backend/download_books.py
- data/Euthyphro by Plato.txt
- data/Critias by Plato.txt
- data/Dhammapada a Collection of Verses Being One of the Canonical Books of the Buddhists.txt
- data/The Enchiridion by Epictetus.txt
- data/Ion by Plato.txt
- data/How to Live on 24 Hours a Day by Arnold Bennett.txt
- data/Index of the Project Gutenberg Works of Lucius Annaeus Seneca by Lucius Annaeus Seneca.txt
- data/The Communist Manifesto by Karl Marx and Friedrich Engels.txt
- .agent/documents/stories/001.advanced_rag_system.md
- data/A Pickle for the Knowing Ones by Timothy Dexter.txt
- .agent/documents/bmad.md
- data/Index of the Project Gutenberg Works of Immanuel Kant by Immanuel Kant.txt
- data/Index of the Project Gutenberg Works of Friedrich Nietzsche by Friedrich Wilhelm Nietzsche.txt
🚧 Files skipped from review as they are similar to previous changes (3)
- backend/verify_and_clear.py
- backend/scripts/ingest_data.py
- README.md
There was a problem hiding this comment.
♻️ Duplicate comments (1)
backend/scripts/update_metadata.py (1)
101-111:⚠️ Potential issue | 🟠 Major책 단위로는 아직 부분 반영이 남습니다.
Line 102-111에서 청크별 UPDATE를 순차 호출하고 있어서 중간 한 건만 실패해도 앞선 청크는 이미 반영된 상태로 남습니다. Line 117-118의 재-raise는 종료 코드는 보장하지만 롤백은 못 하므로, 같은 책 안에
kr_title/thumbnail/link가 섞인 채 끝날 수 있습니다. 책 단위로 서버 측 JSONB merge/RPC 한 번에 처리하거나, 최소한 같은 책의 변경이 원자적으로 적용되도록 바꾸는 편이 안전합니다.Also applies to: 115-118
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 986d56de-522d-4601-9e65-3a54bd5991e2
📒 Files selected for processing (1)
backend/scripts/update_metadata.py
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/scripts/update_metadata.py`:
- Line 94: The SQL JSONB path in the supabase query is malformed; update the
.select/.eq string that currently uses metadata->book_info->>title to use proper
PostgreSQL JSONB syntax with quoted keys and operators (e.g.,
metadata->'book_info'->>'title') so it matches other files like ingest_data.py
and generate_sql_updates.py; locate the supabase.table(...).select("id, content,
embedding, metadata").eq("metadata->book_info->>title", title).execute() call
and replace the JSON path accordingly.
- Line 94: 현재 res = supabase.table("documents").select("id, content, embedding,
metadata").eq("metadata->book_info->>title", title).execute()에서 불필요하게 content와
embedding을 조회하고 있어 성능 저하가 발생합니다; select(...) 호출에서 content와 embedding을 제거하여 "id,
metadata"만 조회하도록 바꾸고, 이후 upsert용으로 생성되는 batch_data(102-113번에 해당하는 batch_data 구성
코드)에서 content/embedding 필드를 기대하거나 포함하지 않도록 수정하세요 — Supabase upsert는 제공된 필드만
업데이트하므로 생략된 필드는 그대로 유지됩니다.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ae43e3dd-e945-49fc-af38-3897b9ca2fdc
📒 Files selected for processing (1)
backend/scripts/update_metadata.py
This was referenced Mar 28, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
새로운 기능
개선사항
문서
기타