Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add bazel tools to generate tao build version info #5

Open
wants to merge 1 commit into
base: features/aicompiler_rebase_20220216
Choose a base branch
from
Open

add bazel tools to generate tao build version info #5

wants to merge 1 commit into from

Conversation

Orion34-lanbo
Copy link

No description provided.

tanyokwok
tanyokwok reviewed Mar 22, 2022
View reviewed changes
tensorflow/workspace2.bzl
@@ -3,6 +3,7 @@
# Import third party config rules.
load("//tensorflow:version_check.bzl", "check_bazel_version_at_least")
load("//third_party/gpus:cuda_configure.bzl", "cuda_configure")
load("//third_party/blade_helper:blade_helper_configure.bzl", "blade_helper_configure")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blade->baldedisc?

qiuxiafei
qiuxiafei approved these changes Mar 22, 2022
View reviewed changes
Copy link
Collaborator

@qiuxiafei qiuxiafei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qiuxiafei qiuxiafei self-requested a review March 22, 2022 14:04
@qiuxiafei
Copy link
Collaborator

After taking a second look, I wonder the necessity to add these configurations. Ideally, tao bridge should be compiled without anything in this repo (aka tf_community), even though it contains code of RAL lned to tf_community directory.

@qiuxiafei
Copy link
Collaborator

How about there two options:

  1. Remove dependencies to TF bazel ruls in tensorflow/compiler/mlir/xla/ral/BUILD. There're only 4 effective rules imported from tensorflow.bzl: tf_cc_shared_object / tf_cc_test / tf_gpu_kernel_library / tf_gpu_library. If they were replaced with standard bazel rules, the whole BUILD file become neutral to both side.
  2. Move all source RAL source files to another directory and expose filegroup targets only. Both bridge and compiler side reference the files and build with their own rule set. One shortcoming is that two nearly identical BUILD files will be maintained.

@Orion34-lanbo
Copy link
Author

After taking a second look, I wonder the necessity to add these configurations. Ideally, tao bridge should be compiled without anything in this repo (aka tf_community), even though it contains code of RAL lned to tf_community directory.

These bazel rules are for compiler side build... bridge side compilation uses gpu related rules from tf community, using these rules may not be a problem

bladedisc pushed a commit that referenced this pull request Aug 16, 2022
@tensorflower-gardener @mihaimaruseac
Merged commit includes the following changes:
49f97f1 
467408627  by A. Unique TensorFlower<[email protected]>:

    Update sqlite version in TF

--
467380418  by A. Unique TensorFlower<[email protected]>:

    compat: Update forward compatibility horizon to 2022-08-13

--
467378663  by A. Unique TensorFlower<[email protected]>:

    Update GraphDef version to 1222.

--
467363891  by A. Unique TensorFlower<[email protected]>:

    Update TFRT dependency to use revision
    https://github.com/tensorflow/runtime/commit/b750bc2999cf02abca6ad9eeff0a04ec7bf3b683.

--
467363622  by A. Unique TensorFlower<[email protected]>:

    [xla:runtime] NFC: Move constraints documentation from jitrt to xla/runtime/constraints

--
467362586  by A. Unique TensorFlower<[email protected]>:

    [xla:runtime] NFC: Extract JitCompilationContext library from jitrt and move it to xla/runtime

--
467361314  by A. Unique TensorFlower<[email protected]>:

    Update TFRT dependency to use revision
    https://github.com/tensorflow/runtime/commit/0a042cbb5275e6ff9a3a7c2748c74df6dcede09e.

--
467360160  by A. Unique TensorFlower<[email protected]>:

    [xla:runtime] NFC: Extract calling_convention library from jitrt and move it to xla/runtime

--
467341954  by A. Unique TensorFlower<[email protected]>:

    Op documentation update.
    	update of g3doc/_includes/tf_passes.md

--
467341426  by A. Unique TensorFlower<[email protected]>:

    Refactor SELECT_V2 in preparation for porting to TFLM.

--
467340678  by A. Unique TensorFlower<[email protected]>:

    Create some global stat tracking for CompilationEnvironments. This tracking can be used to help debug cases in which multiple CompilationEnvironments are used to compile a single HloModule (which should not happen).

--
467339870  by A. Unique TensorFlower<[email protected]>:
    Automated rollback of changelist 467224197.

467339756  by A. Unique TensorFlower<[email protected]>:

    Update TFRT dependency to use revision
    https://github.com/tensorflow/runtime/commit/b20ec05d272477fa6223213687bb22145df92674.

--
467339529  by A. Unique TensorFlower<[email protected]>:

    [XLA] Bugfix for gather index parallel partitioning where the sharded non-parallel dims in indices are not handled.

--
467337900  by A. Unique TensorFlower<[email protected]>:

    [XLA] Minor renamings, refactorings, checks.

--
467337622  by A. Unique TensorFlower<[email protected]>:

    Remove unneeded dependency.

--
467337170  by A. Unique TensorFlower<[email protected]>:

    Integrate LLVM at llvm/llvm-project@2c3ca3b684bb

    Updates LLVM usage to match
    [2c3ca3b684bb](llvm/llvm-project@2c3ca3b684bb)

--
467335264  by A. Unique TensorFlower<[email protected]>:

    [SavedModel Fingerprinting] Add hash #5, which represents the checkpoint.

    The `checkpoint_hash` is a hash of the serialized .index file, which is the metadata file of the TensorBundle containing a string-string table
    of the name of a tensor to its serialized BundleEntryProto. The BundleEntryProto contains a crc32 hash of the tensor contents, but not the contents of the tensor itself.

    RFC: tensorflow/community#415

--
467334010  by A. Unique TensorFlower<[email protected]>:

    Update TFRT dependency to use revision
    https://github.com/tensorflow/runtime/commit/76b3fea4cc9d5e7cb8a85798e41a61a55c301578.

--
467332094  by A. Unique TensorFlower<[email protected]>:

    [xla:runtime] NFC: Extract executable library from jitrt and move it to xla/runtime

--
467324078  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467322782  by A. Unique TensorFlower<[email protected]>:

    PR #57137: [oneDNN] Skip appending kernel registration to log message for MKL ops

    Imported from GitHub PR tensorflow/tensorflow#57137

    This PR skips printing kernel registrations for MKL ops since it leads to performance drop for some eager models caused by this commit tensorflow/tensorflow@c04f65d This is a temporary fix and the condition will be removed when support for block format is removed as a more permanent fix.
    Copybara import of the project:

    --
    89c4c20fd138a0e992599ac333e1b41417bdd7a5 by Kanvi Khanna <[email protected]>:

    [oneDNN] Skip appending kernel registration to log message for MKL ops

    Merging this change closes #57137

--
467322425  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <memory> for 'std::unique_ptr'

--
467321561  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467321058  by A. Unique TensorFlower<[email protected]>:

    PR #57089: [TF-TRT] Adjusting Conv2D Test Tolerance

    Imported from GitHub PR tensorflow/tensorflow#57089

    This PR adjusts & fixes the unittest tolerance for the test `Conv2DStridedNCHWTest` in INT8 mode.

    Copybara import of the project:

    --
    13e4bff8394aaa58f2cdf07b6495f6793a1d2324 by DEKHTIARJonathan <[email protected]>:

    [TF-TRT] Adjusting Conv2D Test Tolerance

    Merging this change closes #57089

--
467320826  by A. Unique TensorFlower<[email protected]>:

    PR #55804: [TF-TRT] Various Cleanups & Python Debugging Assertion Improvements

    Imported from GitHub PR tensorflow/tensorflow#55804

    This PR cleans a few spots in the code base, improves the debuggability of assertion messages in unittests. And replace `distutils.version.LooseVersion` (deprecated) with `packaging.version.Version` (new recommended API).
    Copybara import of the project:

    --
    a4d15ef2cbdcefbf634228b9a5a58a559a5206e5 by DEKHTIARJonathan <[email protected]>:

    [TF-TRT] Various Cleanups & Python Debugging Assertion Improvements

    Merging this change closes #55804

--
467320083  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <ostream> for 'std::ostream'

--
467319094  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467318151  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <iterator> for 'std::back_inserter'

--
467316931  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    using decl 'IsSubsetOf' is unused

--
467316097  by A. Unique TensorFlower<[email protected]>:

    Move passes under tensorflow/compiler/mlir/tensorflow/.

--
467315812  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <memory> for 'std::unique_ptr'

--
467314236  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <memory> for 'std::unique_ptr'

--
467313254  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'
    missing #include <memory> for 'std::make_unique'

--
467312293  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    using decl 'RangeSquareDataset' is unused

--
467311309  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467310637  by A. Unique TensorFlower<[email protected]>:

    PR #57013: [TF-TRT] Add LogSoftmax Support for TF-TRT

    Imported from GitHub PR tensorflow/tensorflow#57013

    This PR adds TF-TRT support to `tf.nn.log_softmax` operation.  This is performed using the formula `logsoftmax = logits - log(reduce_sum(exp(logits), axis=-1))` .  The implemented TRT layers are fused into a single op.

    @DEKHTIARJonathan @tfeher : Please review the changes.
    Copybara import of the project:

    --
    1a8eb9a28dc919f815cbf9c58e0c383992faa08b by Pavani Majety <[email protected]>:

    Add LogSoftmax conversion

    Fix Softmax comments

    [TF-TRT] Move LogSoftmax to use OpConverterBase

    Fix compiler errors

    clang-format

    Undo changes to convert_nodes.cc

    Fix comments

    Merging this change closes #57013

--
467310335  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <array> for 'std::array'

--
467310313  by A. Unique TensorFlower<[email protected]>:

    Update test config in cross device ops

--
467309032  by A. Unique TensorFlower<[email protected]>:

    Update TFRT dependency to use revision
    https://github.com/tensorflow/runtime/commit/eba528ef667653c3554984e5c05573b152c9893b.

--
467308765  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467307702  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467306473  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467306092  by A. Unique TensorFlower<[email protected]>:

    PR #56771: Add return_index_map argument in ssim()

    Imported from GitHub PR tensorflow/tensorflow#56771

    Closes #53115
    Copybara import of the project:

    --
    8f5a1b1e8c10ae0ff705d95f605df41e674b9102 by CohenAriel <[email protected]>:

    Add return_index_map argument in ssim()

    Merging this change closes #56771

--
467305190  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <unordered_map> for 'std::unordered_map'
    missing #include <vector> for 'std::vector'
    missing #include <memory> for 'std::shared_ptr'

--
467304747  by A. Unique TensorFlower<[email protected]>:

    [tfrt:jitrt] NFC: Remove Executable::KernelContext

    It was added before runtime::KernelContext and is not used anywhere. Remove it to avoid confusion. In the future we should reuse runtime::KernelContext as an extension point for user-defined memory allocation etc.

--
467303335  by A. Unique TensorFlower<[email protected]>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467301808  by A. Unique TensorFlower<[email protected]>:

    Changes all local `State` or `TaskState` enum in coordination service into `CoordinatedTaskState` enum in proto.

--
467300580  by A. Unique TensorFlower<[email protected]>:

    lite: enable variable freezing in tf_tfl_translate tester

--
467298890  by A. Unique TensorFlower<[email protected]>:

    Update TFRT dependency to use revision
    https://github.com/tensorflow/runtime/commit/9bb23f7d1ee0e9a55d26c7168790667e5266a74c.

--
467292686  by A. Unique TensorFlower<[email protected]>:

    [xla:runtime] NFC: Move execution_engine library from jitrt to xla/runtime

--
467280901  by A. Unique TensorFlower<[email protected]>:

    [GML] Add tests for concat in the GML tiling and fusion pipeline

--
467276349  by A. Unique TensorFlower<[email protected]>:

    [GML] Implement dim-based shape reification for concat

--
467273958  by A. Unique TensorFlower<[email protected]>:

    Change mutexes under stream_executor/gpu to use absl::Mutex and absl::MutexLock instead of tensorflow::mutex and tensorflow::mutex_lock. Change instance of absl::make_unique to std::make_unique

--
467272897  by A. Unique TensorFlower<[email protected]>:

    [tf.data] Prepend `/bufferedio/` for all paths passed to LoadDataset op.

--

PiperOrigin-RevId: 467408627
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanyokwok tanyokwok tanyokwok left review comments

@qiuxiafei qiuxiafei Awaiting requested review from qiuxiafei

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Orion34-lanbo @qiuxiafei @tanyokwok