Mock network requests for faster unit testing #13 #84

hydrosquall · 2017-10-23T02:11:45Z

Description of the change:

Addresses the HTTP Requests part of Create unit tests #13. I wasn't sure how to mock the DB requests, so I left those alone.
Uses VCR.py to record the HTTP interactions once, and stores the results in a tests/fixtures folder. I redacted the API key that was used to get this data, and used unittest.patch to path the HTTP requests so that the cassettes could be reused later.

Reason for the change:

Tests run more slowly when they need to make live network requests
Testing can't happen if the developer doesn't have internet connection
Adding mocked fixtures fixes this.

Checklist

Make sure all of these items are complete, or else the PR will be ineligible for a code review.

Code passes all existing tests
Any new functionality added includes new unit tests in tests/test.py
Create or update example code to show the new functionality in action.
All code, branch, and git naming and style conventions are followed (see CONTRIBUTING.md)
Feature branch has been rebased off of the latest master branch. ( see CONTRIBUTING.md ).

This way, changes to the local config won't affect the tests.

thinkingserious · 2017-10-31T00:46:52Z

Hi @hydrosquall,

This is awesome! I think we will create a new issue to deal with the DB mocks. I'm not sure the best way to do that either. Perhaps if we query a few people in this repos, we might find a great solution.

With regards to the failing tests, it seems we need to update the requirements files to include the missing dependencies. Could you please update those?

With Best Regards,

Elmer

hydrosquall · 2017-11-03T19:17:36Z

Hey @thinkingserious : How would you feel about dropping support for Python 3.2?

Otherwise, I'll need to figure out a way to install the library below only for the build that happens on version 3.2, since mock has been in the standard library since 3.3.

https://pypi.python.org/pypi/mock/

thinkingserious · 2017-11-03T20:14:14Z

HI @hydrosquall,

I'm not opposed to requiring that dependency for 3.2 development.

hydrosquall · 2017-11-03T21:46:07Z

@thinkingserious : I think my latest changes should fix the build. In the future, we may want to write some separate tests that are able to test just the HTTP requests, without having the dependency on the DB mocks. As it stands, I think all of the tests might depend on the database with the exception of the "send email" test.

thinkingserious · 2017-11-03T23:49:43Z

Thanks @hydrosquall!

hydrosquall · 2017-11-05T17:30:29Z

Hey @thinkingserious , it looks like VCR.py fails to install on just the python 3.2 build :(.

thinkingserious · 2017-11-09T06:13:33Z

Hi @hydrosquall,

Thanks for the further detail! Once this issue pops up on our backlog, I'll dig deeper.

With Best Regards,

Elmer

42B · 2018-10-28T02:29:57Z

test/test.py

+                number_of_watchers=0,
+                number_of_stargazers=0,
+                number_of_forks=0
+            )
            res = self.db.add_data(github_data_import)
            self.assertTrue(isinstance(res, GitHubData))


self.assertIsInstance ?

https://docs.python.org/3/library/unittest.html?#unittest.TestCase.assertIsInstance

@42B thanks for catching these items - with that said, these lines fall outside the scope of the topics for this active PR, if you're interested, maybe those changes could go into a separate PR

42B · 2018-10-28T02:30:18Z

test/test.py

+                php_downloads=0,
+                python_downloads=0,
+                ruby_downloads=0
+            )
            res = self.db.add_data(packagedata)
            self.assertTrue(isinstance(res, PackageManagerData))


self.assertIsInstance ?

https://docs.python.org/3/library/unittest.html?#unittest.TestCase.assertIsInstance

42B · 2018-10-28T02:30:40Z

test/test.py

-                                                self.config.github_repos[0])
+            config = MOCK_DATA['github_config']
+            res = self.github.update_library_data(config['user'],
+                                                  config['repositories'][0])
            self.assertTrue(isinstance(res, GitHubData))


self.assertIsInstance ?

https://docs.python.org/3/library/unittest.html?#unittest.TestCase.assertIsInstance

42B · 2018-10-28T02:30:52Z

test/test.py

    def test_update_package_manager_data(self):
        if os.environ.get('TRAVIS') == None:
            res = self.pm.update_package_manager_data(
-                self.config.package_manager_urls)
+                MOCK_DATA['package_urls'])
            self.assertTrue(isinstance(res, PackageManagerData))


self.assertIsInstance ?

https://docs.python.org/3/library/unittest.html?#unittest.TestCase.assertIsInstance

hydrosquall added 2 commits October 22, 2017 22:04

Use Mock Config Object instead of actual config file

d7a0772

This way, changes to the local config won't affect the tests.

Use VCR.py to mock HTTP requests

d384f99

thinkingserious added status: code review request difficulty: medium hacktoberfest labels Oct 23, 2017

apigirl added hacktoberfest and removed hacktoberfest labels Oct 31, 2017

Install developer dependencies on travis

58e9d25

hydrosquall force-pushed the feature/ISSUE-13_mock-network-requests branch from 57aee5c to 58e9d25 Compare November 3, 2017 21:35

Adjust imports to accommodate python 3.2

113b5a0

hydrosquall force-pushed the feature/ISSUE-13_mock-network-requests branch from 16d33f1 to 113b5a0 Compare November 3, 2017 21:42

thinkingserious added the type: community enhancement label Mar 6, 2018

thinkingserious removed the hacktoberfest label Sep 28, 2018

42B reviewed Oct 28, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mock network requests for faster unit testing #13 #84

Mock network requests for faster unit testing #13 #84

hydrosquall commented Oct 23, 2017 •

edited

Loading

thinkingserious commented Oct 31, 2017

hydrosquall commented Nov 3, 2017 •

edited

Loading

thinkingserious commented Nov 3, 2017

hydrosquall commented Nov 3, 2017

thinkingserious commented Nov 3, 2017

hydrosquall commented Nov 5, 2017

thinkingserious commented Nov 9, 2017

42B Oct 28, 2018

hydrosquall Oct 28, 2018

42B Oct 28, 2018

42B Oct 28, 2018

42B Oct 28, 2018

Mock network requests for faster unit testing #13 #84

Are you sure you want to change the base?

Mock network requests for faster unit testing #13 #84

Conversation

hydrosquall commented Oct 23, 2017 • edited Loading

Checklist

thinkingserious commented Oct 31, 2017

hydrosquall commented Nov 3, 2017 • edited Loading

thinkingserious commented Nov 3, 2017

hydrosquall commented Nov 3, 2017

thinkingserious commented Nov 3, 2017

hydrosquall commented Nov 5, 2017

thinkingserious commented Nov 9, 2017

42B Oct 28, 2018

Choose a reason for hiding this comment

hydrosquall Oct 28, 2018

Choose a reason for hiding this comment

42B Oct 28, 2018

Choose a reason for hiding this comment

42B Oct 28, 2018

Choose a reason for hiding this comment

42B Oct 28, 2018

Choose a reason for hiding this comment

hydrosquall commented Oct 23, 2017 •

edited

Loading

hydrosquall commented Nov 3, 2017 •

edited

Loading