Skip to content

Read model from cache if offline#269

Merged
JosephMarinier merged 2 commits intomainfrom
joseph/transformers-offline
Oct 17, 2022
Merged

Read model from cache if offline#269
JosephMarinier merged 2 commits intomainfrom
joseph/transformers-offline

Conversation

@JosephMarinier
Copy link
Copy Markdown
Contributor

@JosephMarinier JosephMarinier commented Oct 14, 2022

Description:

Loading the sentence transformer currently fails without Internet access. Here is a simple workaround that uses sentence-transformers' own environment variables. I don't understand why it doesn't already work like that, so I opened a ticket on their end: huggingface/sentence-transformers#1725.

Checklist:

You should check all boxes before the PR is ready. If a box does not apply, check it to acknowledge it.

  • ISSUE NUMBER. You linked the issue number (Ex: Resolve #XXX).
  • PRE-COMMIT. You ran pre-commit on all commits, or else, you
    ran pre-commit run --all-files at the end.
  • USER CHANGES. The changes are added to CHANGELOG.md and the documentation, if they impact
    our users.
  • DEV CHANGES.
    • Update the documentation if this PR changes how to develop/launch on the app.
    • Update the README files and our wiki for any big design decisions, if relevant.
    • Add unit tests, docstrings, typing and comments for complex sections.

@JosephMarinier JosephMarinier self-assigned this Oct 14, 2022
with FileLock(pjoin(self.cache_dir, "st.lock")):
self.encoder = SentenceTransformer(self.config.similarity.faiss_encoder)
model = self.config.similarity.faiss_encoder
if os.environ.get("TRANSFORMERS_OFFLINE"):
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative to checking TRANSFORMERS_OFFLINE would be to check if the local cache exists and load it if so, or even try to load the SentenceTransformer with the local cache and catch if there is an error to then load it online. I guess it depends on if it is likely to be updated online (and we want the latest) or not. Any thoughts?

@JosephMarinier JosephMarinier force-pushed the joseph/transformers-offline branch from e4c3627 to e653d17 Compare October 14, 2022 20:13
Copy link
Copy Markdown
Contributor

@gabegma gabegma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!

@JosephMarinier JosephMarinier merged commit 99447f9 into main Oct 17, 2022
@JosephMarinier JosephMarinier deleted the joseph/transformers-offline branch October 17, 2022 16:19
@JosephMarinier
Copy link
Copy Markdown
Contributor Author

This could be reverted if the issue huggingface/sentence-transformers#1725 is resolved. Let's see if the PR huggingface/sentence-transformers#1845 gets accepted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants