Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement tests #11

Merged
merged 1 commit into from
Jun 19, 2024
Merged

feat: Implement tests #11

merged 1 commit into from
Jun 19, 2024

Conversation

martinalbert
Copy link
Contributor

No description provided.

@martinalbert martinalbert marked this pull request as ready for review May 5, 2024 19:48
@martinalbert martinalbert force-pushed the feat/implement-tests branch from e989622 to 807382d Compare May 6, 2024 12:11
@martinalbert martinalbert force-pushed the feat/implement-continuous-delivery branch 2 times, most recently from f4a3e79 to 1972c84 Compare May 7, 2024 07:25
@martinalbert martinalbert force-pushed the feat/implement-tests branch from 8a0a773 to 68d1654 Compare May 7, 2024 07:37
butter_cms/content_field.py Outdated Show resolved Hide resolved
butter_cms/content_field.py Outdated Show resolved Hide resolved
.github/workflows/test.yml Outdated Show resolved Hide resolved
@martinalbert martinalbert requested a review from Toreno96 May 7, 2024 09:49
Base automatically changed from feat/implement-continuous-delivery to master May 7, 2024 13:19
@martinalbert martinalbert force-pushed the feat/implement-tests branch from d64725e to 77c3fd9 Compare May 8, 2024 14:18
@martinalbert martinalbert requested a review from Toreno96 May 9, 2024 06:59
README.md Outdated Show resolved Hide resolved
requirements.txt Show resolved Hide resolved
Copy link

@Toreno96 Toreno96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to finally review the unit tests themselves, and I have bad news: I think all of those require reimplementation.

Let's see an example, TestListAuthor 👇

We mock Author.all (the function on the highest possible level of abstraction) to return list_authors():

@patch('butter_cms.author.Author.all')
def test_all(self, mock_all):
mock_all.return_value = list_authors()
author = Author('<demo-token>')

So we don't really test any internals of the Author class other than the initialization of the class, but even that is not much because the only thing the initialization does is assigning/defining a few member variables, which we then don't use at all because of the mock.

So:

    @patch('butter_cms.author.Author.all')
    def test_all(self, mock_all):
        mock_all.return_value = list_authors()
        author = Author('<demo-token>')
        response = author.all()
        …

This is effectively the same as:

    def test_all(self):
        response = list_authors()
        …

Furthermore, the whole test is effectively the same as:

    def test_all(self):
        response = {
            'data': [
                {
                    'first_name': 'John', 
                    'last_name': 'Doe', 
                    'slug': 'john-doe'
                },
                {
                    'first_name': 'Jane', 
                    'last_name': 'Doe', 
                    'slug': 'jane-doe'
                }
            ]
        }

        self.assertIsNotNone(response)
        self.assertIn('data', response)

        data = response['data']
        self.assertIsNotNone(data)
        self.assertIsInstance(data, list)
        self.assertGreater(len(data), 0)

        author_data = data[0]
        self.assertEqual('John', author_data.get('first_name'))
        self.assertIsNone(author_data.get('recent_posts'))

I think at this point you can see, that this is kind of the same as assert true == true; which is not a good unit test 😅

As such, I think all the current tests have to be removed and rewritten.

I was thinking what could be a good approach. We can't mock all() or get() or similar because of the above reasons.
I don't think we can mock Client.api_get() either because in a significant number of cases all() or get() is just passing through the result of api_get, and only in some cases it does a bit of additional work, so mocking api_get() would be the same as mocking all() or get() for the most part.

I believe a good approach could be to rely on https://pypi.org/project/responses/ to mock responses from request library (which we use inside the api_get) without mocking any method of Client and derivatives themselves. Something like (pseudocode):

    def test_all(self):
        # mock an appropriate response(s) using `responses`# then
        author = Author('<demo-token>')
        response = author.all()
        …

This is not perfect either because unit tests aren't supposed to know that the tested function is using library X or Y internally, but I don't see any better solution, unfortunately.

@martinalbert WDYT?

@martinalbert
Copy link
Contributor Author

martinalbert commented May 15, 2024

@Toreno96 thank you for the review, I understand your concerns and I don't like this approach as well.

I tried the responses package as well as vcr and I'm thinking if it wouldn't make more sense to use the VCR while we're at it. I tested it on one test and you can see the results in this commit: 62f5980

@martinalbert martinalbert force-pushed the feat/implement-tests branch from 6dad0f9 to 62f5980 Compare May 15, 2024 10:42
@martinalbert
Copy link
Contributor Author

@Toreno96 Just to let you know why I tried also the VCR along the responses

  • They both solve the mentioned concern of superficial unit tests - intercepting underlying HTTP traffic instead of method calls,
  • It feels like similar amount of work would be needed to make the tests better,
  • VCR removes the need to create mocks manually and uses more realistic data

@martinalbert martinalbert requested a review from Toreno96 May 15, 2024 11:10
Copy link

@Toreno96 Toreno96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much better now!

Please see:
#11 (comment)
#11 (comment)

Also, I see that in a few places you're defining variables like f = data['navigation_menu_item'], d1 = data[0], etc.

Could you rename them all to something more meaningful?
Single-letter variables are unclear and difficult to search, and it's not like we have a character limit or anything.

Copy link

@Toreno96 Toreno96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

feat: implement tests for Author

feat: implement tests for Category

feat: implement tests for Feed

feat: implement tests for ContentField

feat: implement tests for Page

feat: implement tests for Post

feat: implement tests for Tag

chore: polishing

feat: update dependencies and tests to support different python versions

feat: use test workflow in publish and release-please workflows

chore: remove old tests

fix: review fixes

fix: review fixes

feat: add support for testing python versions 3.8 - 3.12

feat!: remove support for python version lesser than 3

docs: mention official support in readme

chore: remove last-release-sha as already bootstrapped
- release-please will find latest release and use that commit as a last-release-sha

feat: add new workflow for running tests outside master branch

fix: add missing requirement for build job in publish workflow

docs: update readme

wip: feat: use vcr for mocking responses

wip: feat: use responses for mocking responses

fix test using responses package

feat: use vcr with tests

fix(content_field): update get_collection method
- to use correct slug
- to use correct mapping of fields

feat: update tests

feat: add fixtures for tests

feat: update requirements.txt to include vcrpy

test: make tests more strict

fix: rename test variables

feat: block outgoing http traffic in tests

fix: review fixes

fix: move tests to seperate from main module

fix: update test command in CI/CD

fix: review fixes
@martinalbert martinalbert force-pushed the feat/implement-tests branch from 0644e26 to 01838ec Compare June 18, 2024 12:49
@ViolanteCodes ViolanteCodes merged commit 9301df8 into master Jun 19, 2024
5 checks passed
@ViolanteCodes ViolanteCodes deleted the feat/implement-tests branch June 19, 2024 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants