[MetaSchedule][M4b] Testcases for TensorRT builder/runner #10055
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sunggg
requested review from
areusch,
comaniac,
icemelon,
jroesch,
junrushao,
merrymercy,
tqchen and
yzhliu
as code owners
Member
|
CC @zxybazh would love you guys to review each other’s code :-) |
62 tasks
junrushao
changed the title
junrushao
changed the title
junrushao
reviewed
Jan 28, 2022
Comment on lines
30
to
32
| # from tvm import script | ||
| # from tvm._ffi import register_func | ||
| # from tvm.runtime import Module |
| ): | ||
| if use_meta_sched: | ||
| # With meta_schedule | ||
| dev = "nvidia/geforce-rtx-2080" |
Member
There was a problem hiding this comment.
Suggested change
| dev = "nvidia/geforce-rtx-2080" | |
| dev = "cuda" |
Comment on lines
102
to
115
| def relay_build_with_tensorrt( | ||
| mod: Module, | ||
| target: Target, | ||
| params: dict, | ||
| ) -> List[BuilderResult]: | ||
| from tvm.relay.op.contrib.tensorrt import partition_for_tensorrt | ||
|
|
||
| mod, config = partition_for_tensorrt(mod, params) | ||
| with tvm.transform.PassContext( | ||
| opt_level=3, config={"relay.ext.tensorrt.options": config} | ||
| ): | ||
| return tvm.relay.build_module._build_module_no_factory( | ||
| mod, "cuda", "llvm", params | ||
| ) |
Member
There was a problem hiding this comment.
maybe we should refactor these functions, put them under python/tvm/meta_schedule/testing/byoc_trt.py, so that others could conveniently reuse these cool stuff
Member
There was a problem hiding this comment.
| target: Target, | ||
| params: dict, | ||
| ) -> List[BuilderResult]: | ||
| # @Sung: Weird. Cannot pass keyword arg |
Member
There was a problem hiding this comment.
if you have time, you may add a proxy function to _build_module_no_factory to allow kwargs
@register_func("tvm.relay.build")
def _build_module_no_factory_impl(mod, target, target_host, params, mod_name):
target, target_host = Target.check_and_update_host_consist(target, target_host)
return build(mod, target, params=params, mod_name=mod_name).module
def _build_module_no_factory(mod, target=None, target_host=None, params=None, mod_name="default"):
"""A wrapper around build which discards the Python GraphFactoryRuntime.
This wrapper is suitable to be used from other programming languages as
the runtime::Module can be freely passed between language boundaries.
"""
return _build_module_no_factory_impl(mod, target, target_host, params, mod_name)| "model_name", | ||
| ["resnet-50", "mobilenet"], | ||
| ) | ||
| @pytest.mark.parametrize("batch_size", [1, 8, 16]) |
Member
There was a problem hiding this comment.
Suggested change
| @pytest.mark.parametrize("batch_size", [1, 8, 16]) | |
| @pytest.mark.parametrize("batch_size", [1]) |
| ) | ||
|
|
||
|
|
||
| # @sunggg: memory verification error at test_relay_model("resnet-50", 1, use_meta_sched=False, use_trt=True) |
Member
There was a problem hiding this comment.
cannot reproduce this, so let's double confirm :-) If there is no problem, let's remove this line
junrushao
force-pushed
the
meta-trt-testcase
branch
from
8f9b886 to
eb21e55
Compare
…0010) Adding interfaces into Pipeline Executor to "run", "stop","set input", and "get input" from the pipeline executor, In this patch, we also implemented the "BackendRuntime" structure to wrap the graph runtime interface in order to support pipeline executor interface and implement data copy method. This method is used to transfer data between two backend runtimes.
…che#10036) * introduce profile_all_alignments option * add profile_all_alignment option to API * wip * fixed dynamic case * black * update gen_gemm too * minor improvement * fix * all tests work * add doc * fixed for sm = 75 case * fix typo * remove unused import * profile_all -> find_first_valid * fix
…10049) * Add ApplyHisotryBest. Co-authored-by: Junru Shao <[email protected]> Co-authored-by: Bohan Hou <[email protected]> Co-authored-by: Ruihang Lai <[email protected]> Co-authored-by: Hongyi Jin <[email protected]> Co-authored-by: Wuwei Lin <[email protected]> Co-authored-by: Siyuan Feng <[email protected]> * Retrigger CI. * Update integration.py Co-authored-by: Junru Shao <[email protected]> Co-authored-by: Bohan Hou <[email protected]> Co-authored-by: Ruihang Lai <[email protected]> Co-authored-by: Hongyi Jin <[email protected]> Co-authored-by: Wuwei Lin <[email protected]> Co-authored-by: Siyuan Feng <[email protected]>
* mutate-unroll * mutate-unroll
* WIP * WIP * WIP * test cases * add examples * lint * Amend co-authors information Co-authored-by: Siyuan Feng <[email protected]> Co-authored-by: Bohan Hou <[email protected]> Co-authored-by: Hongyi Jin <[email protected]> Co-authored-by: Ruihang Lai <[email protected]> Co-authored-by: Junru Shao <[email protected]> Co-authored-by: Xiyou Zhou <[email protected]> * WIP * address comments and changed tensorized comparator * update * nit * fix example * lint * lint * lint * remove unused * trigger ci * clang-format * fix * rebase Co-authored-by: Siyuan Feng <[email protected]> Co-authored-by: Bohan Hou <[email protected]> Co-authored-by: Hongyi Jin <[email protected]> Co-authored-by: Ruihang Lai <[email protected]> Co-authored-by: Junru Shao <[email protected]> Co-authored-by: Xiyou Zhou <[email protected]>
…ardware (apache#9993) * Add env variable to micro tflite tutorial * Address @gromero comments * address @areusch comment * fix scope * trigger * trigger
This commit introduces BaseAddress ObjectRef to determine base addresses in the codegen for microNPU. This is required when multiple memory pools become available. Thus, base addresses could not be statically determined in the source module.
…ets with (apache#9637) sse4.1 support
… than call node (apache#10069) Co-authored-by: pranav jonnalagadda-SJ1 Eng_ML <[email protected]>
…pache#9999) Co-authored-by: wangjiuyang <[email protected]>
* multi level tiling * remove tensor core related code * pylint * fix Co-authored-by: Junru Shao <[email protected]>
…)" (apache#10072) Because of the failure of LSTM conversion from Pytorch
…ort tensoflow 2.6 (apache#9978) On tensorflow 2.4 the test is expected to fail as the generated graph is not forzen. On tensorflow 2.6 the generated graph is identified as frozen, therefore the test is not needed
) Co-authored-by: Junru Shao <[email protected]> Co-authored-by: Xiyou Zhou <[email protected]> Co-authored-by: Bohan Hou <[email protected]> Co-authored-by: Ruihang Lai <[email protected]> Co-authored-by: Hongyi Jin <[email protected]> Co-authored-by: Wuwei Lin <[email protected]> Co-authored-by: Junru Shao <[email protected]> Co-authored-by: Xiyou Zhou <[email protected]> Co-authored-by: Bohan Hou <[email protected]> Co-authored-by: Ruihang Lai <[email protected]> Co-authored-by: Hongyi Jin <[email protected]> Co-authored-by: Wuwei Lin <[email protected]>
sunggg
requested review from
a team,
ZihengJiang,
mbaret,
mbrookhart,
siju-samuel,
slyubomirsky,
srkreddy1238,
tmoreau89,
trevor-m,
vinx13,
wweic and
zhiics
as code owners
junrushao
force-pushed
the
meta-trt-testcase
branch
from
e38b24d to
68fd695
Compare
junrushao
approved these changes
Jan 29, 2022
Member
|
Thanks @sunggg! It's finally merged :-) |
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Feb 16, 2022
Co-authored-by: Siyuan Feng <[email protected]> Co-authored-by: Bohan Hou <[email protected]> Co-authored-by: Hongyi Jin <[email protected]> Co-authored-by: Ruihang Lai <[email protected]> Co-authored-by: Junru Shao <[email protected]> Co-authored-by: Xiyou Zhou <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
24 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR includes BYOC builder/runner infra and its test case for TensorRT.
Thanks for your time to review this.
cc: @junrushao1994
Please note that previous PR is closed due to the overlap with previous merge.