Unit tests

[Oberon00]

Based on #29.

Adds unit tests using the unittest standard library module, executes them in tox and runs pylint and a basic mypy configuration on the test code.

[lzchen]

Nit: python -m unittest is equivalent to python -m unittest discover

[Oberon00]

Yeah, I saw that in the unittest docs, but went with "explicit is better than implicit"

[lzchen]

I think we should have the unit tests run before the lint/mypy. When developing, I usually fix the lint issues at the end once all the code has been completed and finalized. If the build were to fail on the tests with perfect lint, after the tests are fixed, the lint might be broken due to the changes.

[c24t]

I think it's better in general to fail fast. If our main use case here is CI I'd like to see errors as soon as possible.

In any case the ordering doesn't matter if we run the tests in parallel (with -p auto), and you can always run the unit tests alone with -e py37-test.

[Oberon00]

True, the order would be defined with the envlist at the top.

[carlosalberto]

I also think unit tests should run before the lint/mypy step ;)

[reyang]

Me too, personally I think the ordering in OpenCensus makes sense.

1. unit test
2. integration test
3. code coverage
4. lint
5. document, packaging

[Oberon00]

I sometimes got better error messages from pylint than from my failed unit test about e.g. some non-existent variable. On the other hand I agree that spending time on making the line-breaks nice before the unit test runs through is futile. So personally I think my ordering would be:

1. mypy (what this finds are not style issues but mostly real bugs, in type annotations or code)
2. maybe pylint
3. tests (would it make sense to just run the tests including coverage here, or is that too slow?)
4. pylint if not run in step 2
5. flake8 (this one does not find any bugs, only style issues)
6. coverage (when making changes due to previous steps code coverage is likely to change)

In any case, like @c24t said, the ordering does not matter that much, so I'm really fine with anything.

[c24t]

It'd be great if we could run multiple python versions, and avoid changing PYTHONPATH by hand, but otherwise it LGTM.

[c24t]

How about running the tests in all supported python versions (#25 notwithstanding)?

Suggested change

	envlist = py37-{lint,mypy,test}, docs
	envlist = py{34,35,36,37}-test, py37-{lint,mypy}, docs

Also note that this turns up a type annotation in the loader as unsupported in 3.4.

[Oberon00]

That's definitely something we should do but when I first turned that on I run into "mypy does not support Python 3.4 and decided to do that later and reopen #25 first. I'll take another stab at fixing the 3.4 issues.

[c24t]

If you're doing this to get opentelemetry on the path for tests: would installing opentelemetry-api in the testenv work instead? usedevelop may work for this.

[Oberon00]

I believe it would. Why do we use skipsdist anyway?

[c24t]

IIRC because we've got multiple packages under separate directories so tox didn't know what to install.

[c24t]

It looks like pylint doesn't care about blank lines... do you think it'd be overkill to run flake8 with lint too?

[c24t]

For your consideration: #46. 😄

[Oberon00]

I'm merging this as-is for now, as other PRs are already based on it. Since I'm on holiday for a longer time now anyway, it's yours to adapt (I'll just add my opinion to the lint/test ordering discussion).