Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

suppress Beautiful Soup warnings for URLs during sync_git #20

Open
Pomax opened this issue Aug 18, 2021 · 1 comment
Open

suppress Beautiful Soup warnings for URLs during sync_git #20

Pomax opened this issue Aug 18, 2021 · 1 comment

Comments

@Pomax
Copy link

Pomax commented Aug 18, 2021

Any translation field with values that look like URLs currently cause the sync_git command to print warnings to the console like:

INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/day-unite-against-hate-speech-and-misinformation.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/209304/facebook" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/209305/twitter" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/join-mozilla-telling-major-companies-stophateforprofit.po'
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-ethical-ai-the-choice-is-not-yours-sometimes.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/xoue4-ohk1Y?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-how-dating-apps-might-be-keeping-you-single.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/YTlOrapai-M?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-what-is-ai.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/P-iiN0c2uic?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-why-did-i-watch-that.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/pt9YCVX7VOk?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/S0pE87hADUU?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-why-does-youtube-recommend-conspiracy-theory-videos.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/r98HbsaN9nw?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/new-iphone-update-how-to-opt-out-of-cross-app-tracking.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217446/twitter" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217447/facebook" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217448/email" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning

And while true, these warnings are also essentially meaningless, as sync_git is not concerned with validation, it just needs to sync data, whatever that data is. Could the command be updated so that even if it still uses BS4, with loglevel=error rather than what it's using atm?

@zerolab zerolab transferred this issue from wagtail/wagtail-localize Dec 2, 2021
@zerolab
Copy link
Collaborator

zerolab commented Jun 27, 2023

from bs4 import MarkupResemblesLocatorWarning


warnings.filterwarnings("ignore", category=MarkupResemblesLocatorWarning, module=r"^wagtail_localize_git\..+")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants