-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Made the imitation_game start reporting its results in the README.md. #193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
2b327ec
Made the imitation game start reporting its results in the README.md.
gaaclarke 3a90d34
fixed analyzer errors
gaaclarke eb321d0
ran the formatter
gaaclarke 1a3e2a7
moved things around to make CI happy, made it more like a package
gaaclarke fa3d6fd
incremented dart version
gaaclarke e63f695
ran formatter
gaaclarke 3a7b5b3
formatter patch from ci
gaaclarke cd3d03a
updated readme
gaaclarke File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| String readmeTemplate = """# Imitation Game | ||
|
|
||
| ## Description | ||
|
|
||
| `imitation_game` is a platform for performing automated tests to compare the | ||
| performance of different UI frameworks. For example, how much memory does the | ||
| same app written in Flutter and UIKit take? | ||
|
|
||
| ## Running all the tests | ||
|
|
||
| You need a mobile device plugged into your computer and setup for development. | ||
| The mobile device and the computer need to be on the same network, one that | ||
| allows communication between computers since that's how the mobile phone will | ||
| report its results to the computer. | ||
|
|
||
| ```sh | ||
| dart imitation_game.dart | ||
| ``` | ||
|
|
||
| ## Dependencies | ||
|
|
||
| In order to run the tests you will need the union of all the platforms being | ||
| tested. As new tests are added please add to this list: | ||
|
|
||
| ### iOS | ||
|
|
||
| - Flutter | ||
| - Xcode | ||
| - [ios_deploy](https://github.com/ios-control/ios-deploy) - used to launch apps | ||
| on the attached iOS device. | ||
|
|
||
| ## Example File Layout | ||
|
|
||
| ```text | ||
| ./ | ||
| ├─ imitation_game.dart | ||
| └─ tests/ | ||
| ├─ smiley/ | ||
| │ ├─ README.md | ||
| │ ├─ flutter/ | ||
| │ │ ├─ run_ios.sh | ||
| │ │ └─ <flutter project files> | ||
| │ └─ uikit/ | ||
| │ ├─ run_ios.sh | ||
| │ └─ <uikit project files> | ||
| └─ memory/ | ||
| ├─ README.md | ||
| ├─ flutter/ | ||
| │ ├─ run_ios.sh | ||
| │ └─ <flutter project files> | ||
| └─ uikit/ | ||
| ├─ run_ios.sh | ||
| └─ <uikit project files> | ||
| ``` | ||
|
|
||
| Here there are 2 different tests with 2 different platform implementations. The | ||
| tests are named `smiley` and `memory`, they are both implemented on the | ||
| platforms `flutter` and `uikit`. | ||
|
|
||
| ### Adding a test | ||
|
|
||
| Tests should comprise of implementations on one or more platform. The directory | ||
| for the test should be added to `./tests`. Inside that directory there should | ||
| be a directory of implementations and a `README.md` file that explains the test. | ||
|
|
||
| ### Adding an implementation to a test | ||
|
|
||
| An implementation has to follow these rules: | ||
|
|
||
| - It needs to perform the same operations as the other implementations and | ||
| follow the description in the test's `README.md`. | ||
| - It needs to contain a `run_ios.sh` script that will build and launch the test | ||
| on the connected device. | ||
| - It should contain a file named `ip.txt` which will be overwritten by | ||
| `imitation_game.dart` with the ip address and port that should be used to | ||
| report results to. | ||
| - It needs to report its results to the ip and port in the `ip.txt` via an HTTP | ||
| POST of JSON data. | ||
|
|
||
| ## Data format for results | ||
|
|
||
| ```json | ||
| { | ||
| "test": "name_of_test", | ||
| "platform": "name_of_platform", | ||
| "results": { | ||
| "some_result_name": 1.23, | ||
| "some_result_name2": 4.56, | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| A single test run can report multiple numbers. | ||
|
|
||
| ## Results | ||
| Date created: {{date}} | ||
|
|
||
| {{#tests}} | ||
| - {{name}} | ||
| {{#platforms}} | ||
| - {{name}} | ||
| {{#measurements}} | ||
| - {{name}}: {{value}}s | ||
| {{/measurements}} | ||
| {{/platforms}} | ||
| {{/tests}} | ||
|
|
||
| """; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| name: imitation_game | ||
| dependencies: | ||
| mustache: ^1.1.1 |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ultra nit: just let this be a .md file and read it to string for the
Templatecall? Might be easier to read for contributors.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about it but reading from disk has some issues i can avoid by embedding the template into the binary. Unfortunately there doesn't seem to be a way to embed assets in dart that keeps it as a md file.