Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ENH]: add purging log query and property test #2020

Merged
merged 10 commits into from
Apr 24, 2024
Merged

[ENH]: add purging log query and property test #2020

merged 10 commits into from
Apr 24, 2024

Conversation

nicolasgere
Copy link
Contributor

@nicolasgere nicolasgere commented Apr 16, 2024 •
edited
Loading

Description of changes

Summarize the changes made by this PR.

  • Improvements & Bug fixes
    • Make collection ordering asc
    • Add log purge query
    • Improve property/model testing

@nicolasgere nicolasgere changed the title add purging log query and property test [enh]: add purging log query and property test Apr 16, 2024
@nicolasgere Graphite App
Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @nicolasgere and the rest of your teammates on Graphite Graphite

@nicolasgere nicolasgere marked this pull request as ready for review April 16, 2024 22:19
@github-actions GitHub Actions
Copy link

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

@github-actions GitHub Actions
Copy link

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@github-actions GitHub Actions
Copy link

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@github-actions GitHub Actions
Copy link

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@nicolasgere nicolasgere changed the title [enh]: add purging log query and property test [ENH]: add purging log query and property test Apr 17, 2024
@nicolasgere nicolasgere requested a review from Ishiihara April 17, 2024 20:43
Ishiihara
Ishiihara approved these changes Apr 24, 2024
View reviewed changes
go/database/log/queries/queries.sql Outdated
@@ -29,3 +29,6 @@ UPDATE collection set record_enumeration_offset_position = $2 where id = $1;

-- name: InsertCollection :one
INSERT INTO collection (id, record_enumeration_offset_position, record_compaction_offset_position) values($1, $2, $3) returning *;

-- name: PurgeRecords :exec
DELETE FROM record_log r using collection c where r.collection_id = c.id and r.offset < c.record_compaction_offset_position;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline

go/pkg/metastore/db/dao/test_utils.go Outdated
@@ -138,7 +138,8 @@ func CreateTestCollection(db *gorm.DB, collectionName string, dimension int32, d
return "", err
}
}

// Avoid to have the same create time for a collection
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you share the context on the need to sleep here?

go/pkg/log/server/property_test.go
})
if err != nil {
t.Fatal(err)
}
// Verify that the number of records returned is correct
if len(suite.model.CollectionData[c]) > int(startOffset) {
if len(suite.model.CollectionData[c])-int(startOffset) < int(batchSize) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can we combine the if and else condition?

go/pkg/log/server/property_test.go
}
}
// Verify that the first record offset is correct
if len(response.Records) > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Do we also want to verify the other offsets?

@nicolasgere nicolasgere merged commit fa44ff2 into main Apr 24, 2024
120 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Ishiihara Ishiihara Ishiihara approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nicolasgere @Ishiihara