Skip to content

Commit 55a1946

Browse files
committed
Merge remote-tracking branch 'upstream/master' into feature/xUnitXmlPublisher_AssertionFailure
2 parents 81504f2 + 9dfff05 commit 55a1946

File tree

152 files changed

+2043
-266
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

152 files changed

+2043
-266
lines changed

.gitignore

+8-1
Original file line numberDiff line numberDiff line change
@@ -369,4 +369,11 @@ healthchecksdb
369369
MigrationBackup/
370370

371371
# Ionide (cross platform F# VS Code tools) working folder
372-
.ionide/
372+
.ionide/
373+
374+
# Jekyll
375+
_site
376+
.sass-cache
377+
.jekyll-cache
378+
.jekyll-metadata
379+
vendor

CONTRIBUTING.md

+5-6
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@ We love your input! We want to make contributing to this project as easy and tra
1010
## We Develop with Github
1111
We use github to host code, to track issues and feature requests, as well as accept pull requests.
1212

13-
## We Use [Github Flow](https://guides.github.com/introduction/flow/index.html), So All Code Changes Happen Through Pull Requests
14-
Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://guides.github.com/introduction/flow/index.html)). We actively welcome your pull requests:
13+
## We Use [Github Flow](https://docs.github.com/en/get-started/quickstart/github-flow), So All Code Changes Happen Through Pull Requests
14+
Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://docs.github.com/en/get-started/quickstart/github-flow)). We actively welcome your pull requests:
1515

1616
1. Fork the repo and create your branch from `master`.
1717
2. If you've added new functionality, it's necessary to add tests. For this there is a separate test project called [TcUnit-Verifier](https://github.com/tcunit/TcUnit/tree/master/TcUnit-Verifier). In that project there are several test suites defined to test the different functionality of TcUnit. Please read the [README.MD](https://github.com/tcunit/TcUnit/blob/master/TcUnit-Verifier/README.md) in that project for further instructions. **No new functionality will be accepted without any proper tests**.
@@ -38,14 +38,13 @@ We use GitHub issues to track public bugs. Report a bug by [opening a new issue]
3838
- Notes (possibly including why you think this might be happening, or stuff you tried that didn't work)
3939

4040
## Use a Consistent Coding Style
41-
* Make sure to edit the project with the same version of Visual Studio as the master branch. All software (TcUnit and TcUnit-Verifier) has been developed using Visual Studio 2013. Note that VS2013 shell (provided with TwinCAT 3.1.4022.x) can't open the TcUnit-Verifier_DotNet (as it's a .NET/C# project). Instead it's recommended to use [VS2013 community edition](https://visualstudio.microsoft.com/vs/older-downloads/) which can be used both for TwinCAT and .NET/C# projects
42-
* Make sure to edit the project with the same build of TwinCAT XAE as the master branch. Which build of TwinCAT that is currently used can be deduced from the [TcUnit.tsproj](https://github.com/tcunit/TcUnit/blob/master/TcUnit/TcUnit.tsproj)-file. The "TcVersion"-attribute will give you the version of TwinCAT that was lastly used to edit the project. The build is the third number from the left (for example 4020, 4022 or 4024)
43-
* Make sure that your TwinCAT development environment uses spaces instead of tabs. The default behaviour of the TwinCAT development environment is to use tabs so it needs to be changed, which can be done according to [this guide](https://alltwincat.com/2017/04/14/replace-tabs-with-whitespaces/)
41+
* Make sure to edit the project with the same version of Visual Studio as the master branch. All software (TcUnit and TcUnit-Verifier) has been developed using Visual Studio 2019. Note that the TwinCAT XAE Shell (provided with TwinCAT XAE installer) can't open the TcUnit-Verifier_DotNet (as it's a .NET/C# project). Instead it's recommended to use [VS2019 community edition](https://visualstudio.microsoft.com/vs/older-downloads/) which can be used both for TwinCAT and .NET/C# projects
42+
* Make sure to use TwinCAT XAE version 3.1.4024.44 or later
4443
* The prefixes of naming of function blocks/variables/etc from the [Beckhoff TwinCAT3 identifier/name conventions](https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_plc_intro/18014401873267083.html&id=) are ignored as a modern integrated development environment (as Visual Studio) gives all the hints/information of data types etc
4544
* Make sure to set your TwinCAT development environment to use Separate LineIDs. This is available in the TwinCAT XAE under **Tools→Options→TwinCAT→PLC Environment→Write options→Separate LineIDs** (set this to TRUE, more information is available [here](https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_userinterface/18014403202147467.html&id=))
4645

4746
## License
48-
By contributing, you agree that your contributions will be licensed under its MIT License.
47+
By contributing, you agree that your contributions will be licensed under the MIT License.
4948

5049
## References
5150
This document was adapted from briandk's excellent [contribution guidelines template](https://gist.github.com/briandk/3d2e8b3ec8daf5a27a62).

README.md

+7-4
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
1-
# TcUnit - TwinCAT unit testing framework
1+
![TcUnit logo](https://github.com/tcunit/TcUnit/blob/master/img/tcunit-logo.png)
2+
[![GitHub license](https://img.shields.io/github/license/Naereen/StrapDown.js.svg)](https://github.com/tcunit/TcUnit/blob/master/LICENSE)
3+
[![Open Source? Yes!](https://badgen.net/badge/Open%20Source%20%3F/Yes%21/blue?icon=github)](https://github.com/TcUnit/TcUnit)
24

35

4-
Welcome to the documentation TcUnit - an xUnit testing framework for [Beckhoff's TwinCAT3](https://www.beckhoff.com/twincat3/).
6+
# TcUnit - TwinCAT unit testing framework
57

6-
![TwinCAT logo](https://github.com/tcunit/TcUnit/blob/master/img/TcUnit-logo.jpg)
8+
Welcome to the documentation TcUnit - an xUnit testing framework for [Beckhoff's TwinCAT3](https://www.beckhoff.com/twincat3/).
79

810
**Main documentation site is available on:**
911
**[www.tcunit.org](https://www.tcunit.org)**
@@ -27,6 +29,7 @@ Check out the [TcUnit-Runner project](https://github.com/tcunit/TcUnit-Runner).
2729
That's fantastic! But please read the [CONTRIBUTING](CONTRIBUTING.md) first.
2830

2931
**Have any questions? Found a bug or want to discuss an idea?**
30-
Check the [F.A.Q](https://tcunit.org/frequently-asked-questions/). Check the [open](https://github.com/tcunit/TcUnit/issues) and [closed](https://github.com/tcunit/TcUnit/issues?q=is%3Aissue+is%3Aclosed) issues.
32+
Check the [F.A.Q](https://tcunit.org/frequently-asked-questions/).
33+
Check the [open](https://github.com/tcunit/TcUnit/issues) and [closed](https://github.com/tcunit/TcUnit/issues?q=is%3Aissue+is%3Aclosed) issues.
3134
If your issue does not already exist, create a [new](https://github.com/tcunit/TcUnit/issues/new).
3235
For general ideas/discussions, use the [discussions](https://github.com/tcunit/TcUnit/discussions).

TcUnit-Verifier/README.md

+42-71
Original file line numberDiff line numberDiff line change
@@ -1,42 +1,31 @@
11
# TcUnit-Verifier
2-
This project is can be used to verify the different functions of
3-
[TcUnit](https://www.github.com/tcunit/TcUnit). It has various test suites to
4-
test different functions in TcUnit. It consists of two separate programs:
2+
This project is can be used to verify the different functions of [TcUnit](https://www.github.com/tcunit/TcUnit).
3+
It has various test suites to test different functions in TcUnit.
4+
It consists of two separate programs:
55
- TcUnit-Verifier_TwinCAT
66
- TcUnit-Verifier_DotNet
77

8-
When verifying that TcUnit works as expected a developer needs to add tests to
9-
both of these programs.
8+
When verifying that TcUnit works as expected a developer needs to add tests to both of these programs.
109

1110
![TcUnit-Verifier concept](img/TcUnit-Verifier_Concept_1280.png)
1211

1312
## TcUnit-Verifier_TwinCAT
1413
TcUnit-Verifier_TwinCAT (TcVT) is as the name implies a TwinCAT project.
15-
It holds a reference to and instantiates the TcUnit framework. It is defined by
16-
several test suites (function blocks) that make use of TcUnit and its different
17-
functionalitites. An example of a test is to verify that TcUnit prints a message
18-
if an assertion of an INT has failed (expected value not equals to
19-
asserted value). This means per definition that the verifier will have failed
20-
tests as results, because that is what we want to test. Everytime a test suite
21-
is created in TcVT that tests the functionality of TcUnit it is expected
22-
that either:
23-
- Some specific output is created by TcUnit (error-log), for example asserting a
24-
value that differs from the expected value
25-
- Some specific output is **not** created by TcUnit (error-log), for example
26-
asserting a value that is equal to the expected value
14+
It holds a reference to and instantiates the TcUnit framework.
15+
It is defined by several test suites (function blocks) that make use of TcUnit and its different functionalitites.
16+
An example of a test is to verify that TcUnit prints a message if an assertion of an INT has failed (expected value not equals to asserted value).
17+
This means per definition that the verifier will have failed tests as results, because that is what we want to test.
18+
Everytime a test suite is created in TcVT that tests the functionality of TcUnit it is expected that either:
19+
- Some specific output is created by TcUnit (error-log), for example asserting a value that differs from the expected value
20+
- Some specific output is **not** created by TcUnit (error-log), for example asserting a value that is equal to the expected value
2721

28-
Creating a test suite for TcUnit thus requires that the developer needs to know
29-
what TcUnit should print (or not print). This means that running this TwinCAT
30-
project is not enough to know whether TcUnit behaves as expected, as it would
31-
require to manually check all the output from TcVT. Because we don't manually
32-
want to check that the output provided by TcUnit-Verifier_TwinCAT everytime it's
33-
running, another program is necessary that verifies that the output of the TcVT
34-
is as expected.
22+
Creating a test suite for TcUnit thus requires that the developer needs to know what TcUnit should print (or not print).
23+
This means that running this TwinCAT project is not enough to know whether TcUnit behaves as expected, as it would require to manually check all the output from TcVT.
24+
Because we don't manually want to check that the output provided by TcUnit-Verifier_TwinCAT everytime it's running, another program is necessary that verifies that the output of the TcVT is as expected.
3525

3626
## TcUnit-Verifier_DotNet
37-
The TcUnit-Verifier_DotNet (TcVD) is a (Visual Studio 2013) C# program that
38-
opens and runs the TcUnit-Verifier_TwinCAT project by the usage of the TwinCAT
39-
automation interface. It basically does the following:
27+
The TcUnit-Verifier_DotNet (TcVD) is a C# program that opens and runs the TcUnit-Verifier_TwinCAT project by the usage of the TwinCAT automation interface.
28+
It basically does the following:
4029
- Starts Visual Studio (using the same version that was used developing TcVT)
4130
- Opens TcVT
4231
- Does a clean/build of TcVT
@@ -47,16 +36,11 @@ automation interface. It basically does the following:
4736
- Collects all results from the error list in Visual Studio
4837
- Checks that the output is as defined/expected
4938

50-
The different tests in TcVD follow the same naming-schema as for the TcVT tests
51-
so for example you can find a structured text (ST) function block (FB)
52-
`FB_PrimitiveTypes.TcPOU` in TcVT and the accompanying C#-class
53-
`FB_PrimitiveTypes.cs`. The C# classes verify and test that the output from the
54-
ST function blocks is correct. Thus a complete test of a specific function in
55-
TcUnit needs to be developed in pair, both a FB of tests needs to be added to
56-
TcVT as well a class of tests needs to be added to TcVD.
39+
The different tests in TcVD follow the same naming-schema as for the TcVT tests, so for example you can find a structured text (ST) function block (FB) `FB_PrimitiveTypes.TcPOU` in TcVT and the accompanying C#-class `FB_PrimitiveTypes.cs`.
40+
The C# classes verify and test that the output from the ST function blocks is correct.
41+
Thus a complete test of a specific function in TcUnit needs to be developed in pair, both a FB of tests needs to be added to TcVT as well a class of tests needs to be added to TcVD.
5742

58-
All test classes are instantiated in the class `Program.cs` starting from the
59-
lines:
43+
All test classes are instantiated in the class `Program.cs` starting from the lines:
6044
```
6145
/* Insert the test classes here */
6246
new FB_PrimitiveTypes(errors);
@@ -67,15 +51,10 @@ new FB_AssertEveryFailedTestTwice(errors);
6751
...
6852
```
6953

70-
To create a new test class and make sure that it will be running all that is
71-
necessary is to make sure to instantiate it with the argument `errors`
72-
(just as above). If you have added a test in TcVT that is supposed to fail, and
73-
thus adding an additional failed test to the output, you need to increment the
74-
variable `expectedNumberOfFailedTests` in TcVD by one for every failed test
75-
that you have added.
54+
To create a new test class and make sure that it will be running all that is necessary is to make sure to instantiate it with the argument `errors` (just as above).
55+
If you have added a test in TcVT that is supposed to fail, and thus adding an additional failed test to the output, you need to increment the variable `expectedNumberOfFailedTests` in TcVD by one for every failed test that you have added.
7656

77-
For example, if we in the PRG_TEST-program of TcVT have a function block
78-
instantiated in this way:
57+
For example, if we in the PRG_TEST-program of TcVT have a function block instantiated in this way:
7958
```
8059
PROGRAM PRG_TEST
8160
VAR
@@ -91,31 +70,23 @@ This is an example of how it can look running the TcUnit-Verifier_DotNet:
9170

9271
```
9372
C:\Code\GitHub_TcUnit\TcUnit\TcUnit-Verifier\TcUnit-Verifier_DotNet\TcUnit-Verifier\bin\Debug>TcUnit-Verifier.exe -v "C:\Code\GitHub_TcUnit\TcUnit\TcUnit-Verifier\TcUnit-Verifier_TwinCAT\TcUnit-Verifier_TwinCAT.sln"
94-
2022-01-07 18:58:05 [INFO ] - Starting TcUnit-Verifier...
95-
2022-01-07 18:58:05 [INFO ] - In Visual Studio solution file, found visual studio version 12.0
96-
2022-01-07 18:58:05 [INFO ] - Loading the Visual Studio Development Tools Environment (DTE)...
97-
2022-01-07 18:58:11 [INFO ] - Cleaning and building TcUnit-Verifier_TwinCAT solution...
98-
2022-01-07 18:58:11 [INFO ] - Generating TcUnit-Verifier_TwinCAT boot project...
99-
2022-01-07 18:58:25 [INFO ] - Activating TcUnit-Verifier_TwinCAT configuration...
100-
2022-01-07 18:58:28 [INFO ] - Restarting TwinCAT...
101-
2022-01-07 18:58:28 [INFO ] - Waiting for TcUnit-Verifier_TwinCAT to finish running tests...
102-
2022-01-07 18:58:38 [INFO ] - ... got 409 report lines so far.
103-
2022-01-07 18:58:48 [INFO ] - ... got 934 report lines so far.
104-
2022-01-07 18:58:59 [INFO ] - ... got 1357 report lines so far.
105-
2022-01-07 18:59:01 [INFO ] - Asserting results...
106-
2022-01-07 18:59:01 [INFO ] - Done.
107-
2022-01-07 18:59:01 [INFO ] - Closing the Visual Studio Development Tools Environment (DTE), please wait...
108-
2022-01-07 18:59:23 [INFO ] - Exiting application...
73+
2023-04-29 22:15:29 [INFO ] - Starting TcUnit-Verifier...
74+
2023-04-29 22:15:29 [INFO ] - In Visual Studio solution file, found visual studio version 16.0
75+
2023-04-29 22:15:29 [INFO ] - Loading the Visual Studio Development Tools Environment (DTE)...
76+
2023-04-29 22:15:41 [INFO ] - Cleaning and building TcUnit-Verifier_TwinCAT solution...
77+
2023-04-29 22:15:41 [INFO ] - Generating TcUnit-Verifier_TwinCAT boot project...
78+
2023-04-29 22:15:54 [INFO ] - Activating TcUnit-Verifier_TwinCAT configuration...
79+
2023-04-29 22:15:57 [INFO ] - Restarting TwinCAT...
80+
2023-04-29 22:15:57 [INFO ] - Waiting for TcUnit-Verifier_TwinCAT to finish running tests...
81+
2023-04-29 22:16:07 [INFO ] - ... got 347 report lines so far.
82+
2023-04-29 22:16:17 [INFO ] - ... got 872 report lines so far.
83+
2023-04-29 22:16:28 [INFO ] - ... got 1398 report lines so far.
84+
2023-04-29 22:16:39 [INFO ] - ... got 1618 report lines so far.
85+
2023-04-29 22:16:41 [INFO ] - Asserting results...
86+
2023-04-29 22:16:42 [INFO ] - Done.
87+
2023-04-29 22:16:42 [INFO ] - Closing the Visual Studio Development Tools Environment (DTE), please wait...
88+
2023-04-29 22:17:02 [INFO ] - Exiting application...
10989
```
110-
If there was an error in the TcUnit framework this would be shown between the
111-
lines `Asserting results...` and `Done.`. If nothing is shown between these
112-
lines (like in the example) it means that TcUnit behaves according to the tests
113-
defined in TcUnit-Verifier. As TcVD starts the TwinCAT runtime it is required
114-
that a (trial) runtime license for TwinCAT is activated on the PC of where TcVD
115-
and TcVT will be running.
116-
117-
To compile the project you will also need to install
118-
[log4net](https://logging.apache.org/log4net/).
119-
Open the project with VS2013. Go to
120-
`TOOLS->NuGet Package Manager->Package Manager Console`.
121-
In the console enter `Install-Package log4net -Version 2.0.14`.
90+
If there was an error in the TcUnit framework this would be shown between the lines `Asserting results...` and `Done.`.
91+
If nothing is shown between these lines (like in the example) it means that TcUnit behaves according to the tests defined in TcUnit-Verifier.
92+
As TcVD starts the TwinCAT runtime it is required that a (trial) runtime license for TwinCAT is activated on the PC of where TcVD and TcVT will be running.

TcUnit-Verifier/TcUnit-Verifier_DotNet/TcUnit-Verifier.sln

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11

22
Microsoft Visual Studio Solution File, Format Version 12.00
3-
# Visual Studio 2013
4-
VisualStudioVersion = 12.0.40629.0
3+
# Visual Studio Version 16
4+
VisualStudioVersion = 16.0.33529.622
55
MinimumVisualStudioVersion = 10.0.40219.1
66
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TcUnit-Verifier", "TcUnit-Verifier\TcUnit-Verifier.csproj", "{13F197A8-6EF6-4799-A221-37B97BA42100}"
77
EndProject
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<?xml version="1.0" encoding="utf-8"?>
22
<configuration>
33
<startup>
4-
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.1"/>
4+
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2"/>
55
</startup>
66
</configuration>

TcUnit-Verifier/TcUnit-Verifier_DotNet/TcUnit-Verifier/FB_ExtendedTestInformation.cs

+6
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ public FB_ExtendedTestInformation(IEnumerable<ErrorList.Error> errors, string te
99
{
1010
Check_TestSuite_Statistics();
1111
AssertTestClassName();
12+
AssertTestDuration();
1213
Test_BOOL_AssertFailed();
1314
Test_BOOL_AssertSuccess();
1415
Test_BYTE_TwoFailedAsserts();
@@ -36,6 +37,11 @@ private void AssertTestClassName()
3637
AssertMessageCount(className, 5, EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
3738
}
3839

40+
private void AssertTestDuration()
41+
{
42+
string duration = "Test duration=%f";
43+
AssertMessageCount(duration, 260, EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
44+
}
3945

4046
private void Test_BOOL_AssertFailed()
4147
{

TcUnit-Verifier/TcUnit-Verifier_DotNet/TcUnit-Verifier/FB_TestNumberOfAssertionsCalculation.cs

+5-5
Original file line numberDiff line numberDiff line change
@@ -16,27 +16,27 @@ public FB_TestNumberOfAssertionsCalculation(IEnumerable<ErrorList.Error> errors,
1616

1717
private void TestMixed33SuccessulAnd9FailedAssertions()
1818
{
19-
AssertContainsMessage("| Test status=FAIL, number of asserts=42", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
19+
AssertContainsMessage("| Test status=FAIL, number of asserts=42, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
2020
}
2121

2222
private void TestWith43SuccessfulAssertions()
2323
{
24-
AssertContainsMessage("| Test status=PASS, number of asserts=43", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
24+
AssertContainsMessage("| Test status=PASS, number of asserts=43, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
2525
}
2626

2727
private void TestWith44FailedAssertions()
2828
{
29-
AssertContainsMessage("| Test status=FAIL, number of asserts=44", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
29+
AssertContainsMessage("| Test status=FAIL, number of asserts=44, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
3030
}
3131

3232
private void TestWith45SuccessfulArrayAssertions()
3333
{
34-
AssertContainsMessage("| Test status=PASS, number of asserts=45", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
34+
AssertContainsMessage("| Test status=PASS, number of asserts=45, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
3535
}
3636

3737
private void TestWith46FailedArrayAssertions()
3838
{
39-
AssertContainsMessage("| Test status=FAIL, number of asserts=46", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
39+
AssertContainsMessage("| Test status=FAIL, number of asserts=46, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
4040
}
4141

4242
}

0 commit comments

Comments
 (0)