Skip to content

Create a shorter temp file name for model loading. #397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TomFinley merged 4 commits into from
Jun 27, 2018
Merged

Create a shorter temp file name for model loading. #397

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6ec24f3
Create a shorter temp file name for model loading.
yaeldMS Jun 22, 2018
35e7a6c
Address PR comments.
yaeldMS Jun 25, 2018
35864ad
Address PR comments.
yaeldMS Jun 26, 2018
cf27789
Trigger build.
yaeldMS Jun 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion src/Microsoft.ML.Data/Model/Repository.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,13 +109,35 @@ internal Repository(bool needDir, IExceptionContext ectx)
_open = new List<Entry>();
if (needDir)
{
DirTemp = GetTempPath();
DirTemp = GetShortTempPath();
Directory.CreateDirectory(DirTemp);
}
else
GC.SuppressFinalize(this);
}

private static string GetShortTempPath()
{
Guid guid = Guid.NewGuid();
var guidName = guid.ToString();
int index = guidName.IndexOf('-');
if (index < 0)
index = Math.Min(8, guidName.Length);
guidName = guidName.Substring(0, index);
var path = Path.Combine(Path.GetTempPath(), "TLC_" + guidName);
while (Directory.Exists(path))

@Ivanidzo4ka Ivanidzo4ka Jun 22, 2018 •
edited by yaeldekel
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while (Directory.Exists(path)) [](start = 12, length = 30)

you can just make do while loop #Resolved

@glebuk glebuk Jun 25, 2018 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exists [](start = 29, length = 6)

why even bother with GUId?, why not just take a random int and convert to hex? That's the shortest and will collide only once in a million times on average (assuming you randomize properly.) #Closed

{
guid = Guid.NewGuid();
guidName = guid.ToString();
index = guidName.IndexOf('-');
if (index < 0)
index = Math.Min(8, guidName.Length);
guidName = guidName.Substring(0, index);
path = Path.Combine(Path.GetTempPath(), "TLC_" + guidName);
}
return Path.GetFullPath(path);

@TomFinley TomFinley Jun 26, 2018 •
edited by yaeldekel
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Back when we were using a GUID this was less likely, but now the likelihood of a collision is much higher, I start to worry a lot more about the inherent race condition in this code. :P With a 128-bit GUID, the chance of collision was on the order of 1/2^64... now it's 31 bits, it's on the order of 1/2^16.

Can we restructure this code a bit so that the creation of the directory and the generation of a supposedly safe directory name happen at the same time? We can handle this by creating it in a non-lazy fashion. If we want the creation to be "lazy" as it currently is, then I do not see how the currently scheme of a protected readonly string DirTemp can be allowed to stand. #Resolved

}

// REVIEW: This should use host environment functionality.
private static string GetTempPath()
{
Expand Down