Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/344 functorch #378

Merged
merged 10 commits into from
Jul 26, 2023
Merged

Conversation

schroedk
Copy link
Collaborator

@schroedk schroedk commented Jul 14, 2023

Changes

  • modify the hessian computation to use torch.func

Checklist

  • Wrote Unit tests (if necessary)
  • Updated Documentation (if necessary)
  • Updated Changelog
    - [ ] If notebooks were added/changed, added boilerplate cells are tagged with "nbsphinx":"hidden"

Copy link
Collaborator

@AnesBenmerzoug AnesBenmerzoug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docstrings should use the same rst style.

y = hessian_vp(x) + hessian_perturbation * x
return cp.from_dlpack(to_dlpack(y))

eigen_vals, eigen_vecs = eigsh(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of repetition and return points in this function. I think it should be simplified by making a single call to eigsh and defining the different variables and functions in the if and else branches, along with an extra function to convert the result to torch tensors.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, done

@@ -21,7 +21,7 @@
],
setup_requires=["wheel"],
tests_require=["pytest"],
extras_require={"influence": ["torch"]},
extras_require={"influence": ["torch>=2.0.0"], "cupy": ["cupy-cuda11x>=12.1.0"]},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cupy is only used with influence functions, right? What would be the disadvantage of adding it to the influence extras instead of creating a separate one just for it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On mac, there is no wheel cupy-cuda11x. If we include it into the influence extra, then no mac user could install the influence extra

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, we can leave it for now.
I will create a separate issue to consider using environment markers for this.

@AnesBenmerzoug
Copy link
Collaborator

@schroedk could you please merge develop into your branch and mark this PR as ready so that CI can run?

@schroedk schroedk changed the title Draft: Feature/344 functorch Feature/344 functorch Jul 25, 2023
@schroedk
Copy link
Collaborator Author

@AnesBenmerzoug as this PR is not into develop, I would first merge into into to the target feature branch and run the CI there

@schroedk schroedk merged commit 51fb652 into feature/low_rank_approx Jul 26, 2023
@schroedk schroedk deleted the feature/344-functorch branch July 26, 2023 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants