Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: Which Endpoint #2100

Open
tim-moody opened this issue Nov 12, 2024 · 4 comments
Open

Question: Which Endpoint #2100

tim-moody opened this issue Nov 12, 2024 · 4 comments

Comments

@tim-moody
Copy link
Contributor

  • If run against a WMF wiki, mwoffliner by default uses the endpoint /api/rest_v1, which stock mediawiki does not support.
  • mwoffliner:dev with --forceRender="RestApi" uses the endpoint /w/rest.php/v1/page, which both WMF and stock mediawikis support.
  • It looks like the ZimFarm currently uses mwoffliner:dev. Is it also possible to specify --forceRender="RestApi" in a recipe?
  • mwoffliner:dev --help does not explicitly mention RestApi as a value for --forceRender, but mwoffliner:dev accepts it and then uses the rest.php endpoint
  • Can we assume that RestApi is to be a supported value for --forceRender going forward?
@kelson42
Copy link
Collaborator

kelson42 commented Nov 13, 2024

Many things written are wrong/false but I don't have time right now to go and check each of them again.

@audiodude
Copy link
Member

  • If run against a WMF wiki, mwoffliner by default uses the endpoint /api/rest_v1, which stock mediawiki does not support.

The tool has mechanisms for detecting which APIs are supported, as well as a preferential order for selecting one.

  • mwoffliner:dev with --forceRender="RestApi" uses the endpoint /w/rest.php/v1/page, which both WMF and stock mediawikis support.

Correct.

  • It looks like the ZimFarm currently uses mwoffliner:dev. Is it also possible to specify --forceRender="RestApi" in a recipe?

Some recipes are using mwoffliner:dev, for testing, however it has not been widely adopted yet. I believe using that parameter should be acceptable for that mwoffliner version.

  • mwoffliner:dev --help does not explicitly mention RestApi as a value for --forceRender, but mwoffliner:dev accepts it and then uses the rest.php endpoint

#2102

  • Can we assume that RestApi is to be a supported value for --forceRender going forward?

Yes.

@kelson42 kelson42 reopened this Nov 15, 2024
@kelson42
Copy link
Collaborator

kelson42 commented Nov 15, 2024

On Wikimedia, MWoffliner should use "WikimediaMobile" for articles per default, if not, this is a bug. Mobile is the strategy of MWoffliner since... always (so nore than 10 years).

For the API, the doc is here https://github.com/openzim/mwoffliner/wiki/API-end%E2%80%90points, it's important to read faq and doc before asking.

Each recipe of the Zimfarm decides which scraper versions and all recipes of MWoffliner are currently paudes. All of this has already been said and is clearly indicated on each Zimfarm recipe.

@tim-moody
Copy link
Contributor Author

For the API, the doc is here https://github.com/openzim/mwoffliner/wiki/API-end%E2%80%90points, it's important to read faq and doc before asking.

I have read this too many times. I don't see anything relevant in https://kiwix.org/en/frequently-asked-questions/

I have also read https://www.mediawiki.org/wiki/API:REST_API (see the comparison at the bottom)

The API endpoints doc is not consistent with either the latest or dev code or the help.

Many things written are wrong/false but I don't have time right now to go and check each of them again.

I find it more incomplete than false. There is no mention of version of mwoffliner or forceRender or the default or the algorithm by which the endpoint is probed and selected. One might think that ActionParse is a parameter or a value, but if it is I haven't found it. Same for RestApi if we are not talking about dev. I understand that there are many things that compete with reworking documentation.

Can we assume that RestApi is to be a supported value for --forceRender going forward?

Yes

Thanks and I see this is now in 1.14.0-dev --help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants