Skip to content

Conversation

@christianlupus
Copy link
Collaborator

This PR should reduce the complexity of the RecipeService and pull out some of the functionality in smaller chunks that might also simplify debugging.

For now, the recipeParseHtml() method is only deprecated but should be removed soon.

@christianlupus christianlupus added the Backend Issue or PR related to the backend code label Jan 23, 2021
@codecov
Copy link

codecov bot commented Jan 23, 2021

Codecov Report

Merging #553 (dda8eb2) into master (1b97ad5) will increase coverage by 8.39%.
The diff coverage is 94.65%.

❗ Current head dda8eb2 differs from pull request most recent head 8651abd. Consider uploading reports for the commit 8651abd to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #553      +/-   ##
==========================================
+ Coverage   32.26%   40.65%   +8.39%     
==========================================
  Files          32       42      +10     
  Lines        1621     1852     +231     
==========================================
+ Hits          523      753     +230     
- Misses       1098     1099       +1     
Flag Coverage Δ
integration 4.58% <0.00%> (-0.66%) ⬇️
unittests 36.06% <94.65%> (+9.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
lib/Service/RecipeService.php 0.00% <0.00%> (ø)
lib/Helper/HtmlToDomParser.php 95.16% <95.16%> (ø)
lib/Exception/HtmlParsingException.php 100.00% <100.00%> (ø)
lib/Exception/ImportException.php 100.00% <100.00%> (ø)
lib/Helper/HTMLFilter/HtmlEntityDecodeFilter.php 100.00% <100.00%> (ø)
lib/Helper/HTMLParser/AbstractHtmlParser.php 100.00% <100.00%> (ø)
...b/Helper/HTMLParser/AttributeNotFoundException.php 100.00% <100.00%> (ø)
lib/Helper/HTMLParser/HttpJsonLdParser.php 100.00% <100.00%> (ø)
lib/Helper/HTMLParser/HttpMicrodataParser.php 100.00% <100.00%> (ø)
lib/Service/HtmlDownloadService.php 100.00% <100.00%> (ø)
... and 1 more

@christianlupus christianlupus force-pushed the dev/html-parser-service branch from 6b7feb6 to 4660ebb Compare January 23, 2021 17:24
@christianlupus
Copy link
Collaborator Author

@seyfeb I tried here to start the codebase refractory and pull some functionality into its own service. I know this is quite a bigger pack of changes which makes it hard to check line by line. What do you think, how should we proceed here? First writing tests for all changes involved here or building the tests later? I tried to make it possible to test the code but did not write the tests yet.

If we want to do this with everything tested, I would have to solve #537 first, I assume.

Any further remarks?

@seyfeb
Copy link
Collaborator

seyfeb commented Jan 24, 2021

I’m not sure either. We might get some regressions when doing this without tests, so this might be a good moment to start writing tests. However, I’m not sure if, with our limited time to work on this, it might just be okay to deal with them later and first get the better structure in the code and hope for the best (keeping in mind to add tests once we have #537).

@christianlupus christianlupus marked this pull request as draft January 25, 2021 07:03
@christianlupus
Copy link
Collaborator Author

OK, then we wait for #537, rebase to have these changes, and try to implement some tests afterward. I converted to draft to make this obvious.

@christianlupus
Copy link
Collaborator Author

Closes #740

@christianlupus christianlupus linked an issue Jun 29, 2021 that may be closed by this pull request
@christianlupus christianlupus force-pushed the dev/html-parser-service branch from 4660ebb to 74f199a Compare August 10, 2021 11:31
@christianlupus christianlupus force-pushed the dev/html-parser-service branch from 74f199a to dbe1ee0 Compare August 26, 2021 14:40
@github-actions
Copy link

github-actions bot commented Aug 26, 2021

Unit Test Results

     24 files       24 suites   8m 44s ⏱️
   186 tests    186 ✔️ 0 💤 0
2 232 runs  2 232 ✔️ 0 💤 0

Results for commit 8651abd.

♻️ This comment has been updated with latest results.

@christianlupus christianlupus marked this pull request as ready for review September 2, 2021 11:56
@christianlupus
Copy link
Collaborator Author

OK, I added tests for all new classes except for the RecipeService. This special class is not yet refactored and is not in a testable shape.

Also, no integration tests were written so far. This is related to the fact that no clear idea of what sites should be taken as examples for these tests.

@christianlupus christianlupus force-pushed the dev/html-parser-service branch 2 times, most recently from 09013d1 to aa2bbb5 Compare September 11, 2021 14:33
@christianlupus christianlupus linked an issue Oct 17, 2021 that may be closed by this pull request
@christianlupus christianlupus force-pushed the dev/html-parser-service branch from 66cac34 to 2146636 Compare October 18, 2021 16:16
@christianlupus christianlupus linked an issue Oct 20, 2021 that may be closed by this pull request
@christianlupus christianlupus force-pushed the dev/html-parser-service branch from 22e492d to eb2624c Compare May 7, 2022 18:17
@christianlupus christianlupus requested a review from seyfeb May 7, 2022 18:35
@christianlupus christianlupus force-pushed the dev/html-parser-service branch 2 times, most recently from 075a5f1 to 50a6caa Compare May 14, 2022 15:31
Copy link
Collaborator

@seyfeb seyfeb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this with a recipe and it worked as expected. I did not go through all the test cases though..

christianlupus and others added 25 commits May 24, 2022 20:02
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
This has shown one bug that was fixed as a byproduct.

Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Co-authored-by: Sebastian Fey <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
Signed-off-by: Christian Wolf <[email protected]>
@christianlupus christianlupus force-pushed the dev/html-parser-service branch from d55dfc6 to 8651abd Compare May 24, 2022 18:05
@christianlupus
Copy link
Collaborator Author

I am going to merge this now. This will allow breaking the work into smaller chunks that can be handled more easily. Especially adding more test and integration tests can be done separately. See also #1017.

@christianlupus christianlupus merged commit 072b8d4 into master May 24, 2022
@delete-merged-branch delete-merged-branch bot deleted the dev/html-parser-service branch May 24, 2022 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backend Issue or PR related to the backend code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTML parser seems to ignore https://schema.org/Recipe Failing URL-import: ichkoche.at SeriousEats.com not supported: Type as array

3 participants