Skip to content

TF package size fix #3983

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
harishsk merged 9 commits into from
Jul 10, 2019
Merged

TF package size fix #3983

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8749a10
Merge pull request #1 from dotnet/master
harishsk Jun 20, 2019
fe25bf6
Fixed build errors resulting from upgrade to VS2019 compilers
harishsk Jun 21, 2019
cb446be
Added additional message describing the previous fix
harishsk Jun 21, 2019
b5ee220
Merge pull request #2 from dotnet/master
harishsk Jun 21, 2019
b9a7471
Merge pull request #3 from dotnet/master
harishsk Jun 24, 2019
80e238d
Merge pull request #4 from dotnet/master
harishsk Jun 27, 2019
2ef424d
Merge pull request #5 from dotnet/master
harishsk Jul 9, 2019
fd25a45
Reduced list of TF binaries to only the essential ones
harishsk Jul 10, 2019
40b47ab
Fixed formatting
harishsk Jul 10, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions Directory.Build.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
</ItemGroup>

<ItemGroup>
<NativeAssemblyReferenceWithMajorVersion Include="@(NativeAssemblyReference->HasMetadata('MajorVersion'))">
<NativeAssemblyReferenceWithMajorVersion>
<!-- Tensorflow has a different naming scheme for v1.14.0. Those binaries need to be copied along with the standard names -->
<AssemblyPathWithMajorVersion Condition="'$(OS)' != 'Windows_NT'">$(NativeOutputPath)$(LibPrefix)%(NativeAssemblyReference.Identity)$(LibExtension).%(NativeAssemblyReference.MajorVersion)</AssemblyPathWithMajorVersion>
<AssemblyPathWithMajorVersion Condition="$([MSBuild]::IsOSPlatform('osx'))">$(NativeOutputPath)$(LibPrefix)%(NativeAssemblyReference.Identity).%(NativeAssemblyReference.MajorVersion)$(LibExtension)</AssemblyPathWithMajorVersion>
<AssemblyPathWithMajorVersion Condition="'$(OS)' != 'Windows_NT'">$(NativeOutputPath)$(LibPrefix)%(NativeAssemblyReferenceWithMajorVersion.Identity)$(LibExtension).%(NativeAssemblyReferenceWithMajorVersion.MajorVersion)</AssemblyPathWithMajorVersion>
<AssemblyPathWithMajorVersion Condition="$([MSBuild]::IsOSPlatform('osx'))">$(NativeOutputPath)$(LibPrefix)%(NativeAssemblyReferenceWithMajorVersion.Identity).%(NativeAssemblyReferenceWithMajorVersion.MajorVersion)$(LibExtension)</AssemblyPathWithMajorVersion>
</NativeAssemblyReferenceWithMajorVersion>
</ItemGroup>

Expand Down
8 changes: 3 additions & 5 deletions src/Microsoft.ML.DnnAnalyzer/Microsoft.ML.DnnAnalyzer/Microsoft.ML.DnnAnalyzer.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,10 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReferenceWithMajorVersion Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</NativeAssemblyReferenceWithMajorVersion>
</ItemGroup>

</Project>
8 changes: 0 additions & 8 deletions src/Redist/Microsoft.ML.TensorFlow.Redist/Microsoft.ML.TensorFlow.Redist.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,12 @@
<TensorFlowConfig Include="windows" FileExtension=".zip" FilesFromArchive="lib\tensorflow.dll;include\tensorflow\c\LICENSE" Runtime="win-x64"/>

@artidoro artidoro Jul 10, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could you format the windows config the same way you did below for linux and mac?
Not important if you are not doing another iteration.

<TensorFlowConfig Include="linux" FileExtension=".tar.gz"
FilesFromArchive="lib\libtensorflow.so;
lib\libtensorflow.so.$(TensorFlowMajorVersion);

@artidoro artidoro Jul 10, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all these dynamic libraries not necessary then?

@harishsk harishsk Jul 10, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The binaries deleted are identical to the ones remaining. They are symlinked in the tar file. But when they get to the nuget package, they become copies.
This change keeps only the ones that are used.

lib\libtensorflow.so.$(TensorFlowVersion);
lib\libtensorflow_framework.so;
lib\libtensorflow_framework.so.$(TensorFlowMajorVersion);
lib\libtensorflow_framework.so.$(TensorFlowVersion);
include\tensorflow\c\LICENSE"
Runtime="linux-x64" />
<TensorFlowConfig Include="darwin" FileExtension=".tar.gz"
FilesFromArchive="lib\libtensorflow.dylib;
lib\libtensorflow.$(TensorFlowMajorVersion).dylib;
lib\libtensorflow.$(TensorFlowVersion).dylib;
lib\libtensorflow_framework.dylib;
lib\libtensorflow_framework.$(TensorFlowMajorVersion).dylib;
lib\libtensorflow_framework.$(TensorFlowVersion).dylib;
include\tensorflow\c\LICENSE"
Runtime="osx-x64" />

Expand Down
8 changes: 3 additions & 5 deletions test/Microsoft.ML.Core.Tests/Microsoft.ML.Core.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,10 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReferenceWithMajorVersion Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</NativeAssemblyReferenceWithMajorVersion>
</ItemGroup>

</Project>
8 changes: 3 additions & 5 deletions test/Microsoft.ML.Functional.Tests/Microsoft.ML.Functional.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,10 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReferenceWithMajorVersion Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</NativeAssemblyReferenceWithMajorVersion>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.ML.TensorFlow.TestModels" Version="$(MicrosoftMLTensorFlowTestModelsVersion)" />
Expand Down
8 changes: 3 additions & 5 deletions test/Microsoft.ML.Tests/Microsoft.ML.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,10 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReferenceWithMajorVersion Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</NativeAssemblyReferenceWithMajorVersion>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.ML.TensorFlow.TestModels" Version="$(MicrosoftMLTensorFlowTestModelsVersion)" />
Expand Down