Skip to content

moved IndexIVFPQ and IndexPQ to dynamic dispatch#4291

Closed
mdouze wants to merge 4 commits intofacebookresearch:mainfrom
mdouze:export-D72937709
Closed

moved IndexIVFPQ and IndexPQ to dynamic dispatch#4291
mdouze wants to merge 4 commits intofacebookresearch:mainfrom
mdouze:export-D72937709

Conversation

@mdouze
Copy link
Contributor

@mdouze mdouze commented Apr 14, 2025

Summary:
moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular, this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709

mdouze and others added 4 commits April 13, 2025 14:41
Differential Revision: D72937710
Differential Revision: D72937708
Differential Revision: D72937711
Summary:
moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular, this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D72937709

@mdouze mdouze marked this pull request as draft April 14, 2025 05:42
@mdouze
Copy link
Contributor Author

mdouze commented Apr 14, 2025

This export is not functional. It is mainly for external visibility, on how dynamic dispatching will work.

@mdouze
Copy link
Contributor Author

mdouze commented Apr 14, 2025

The dynamic dispatching requires:

  1. several implementations of a function / class to co-exist. The SIMDLevel object can be used as a template parameter to differentiate these functions
  2. the code to be in several compile units that use different compilation flags (ie. -mavx2, etc.)

#ifdef __x86_64__
{
unsigned int eax, ebx, ecx, edx;
asm volatile("cpuid"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! How portable is the cpuid.h header?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you would recommend to make all the distance functions pointers ?
That's also a possibility but wondering about the performance impact...

@mdouze mdouze closed this Apr 15, 2025
mdouze added a commit to mdouze/faiss that referenced this pull request Apr 24, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
mdouze added a commit to mdouze/faiss that referenced this pull request May 5, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
mdouze added a commit to mdouze/faiss that referenced this pull request May 6, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
algoriddle pushed a commit to algoriddle/faiss that referenced this pull request May 7, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
mdouze added a commit to mdouze/faiss that referenced this pull request Jun 4, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 26, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 26, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 26, 2025
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 26, 2025
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 26, 2025
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 26, 2025
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 26, 2025
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 27, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 27, 2025
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Reviewed By: mnorris11

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 28, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Aug 28, 2025
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Reviewed By: mnorris11

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Sep 9, 2025
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Jan 21, 2026
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Jan 23, 2026
Summary:
Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Differential Revision: D72937709
subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request Jan 23, 2026
Summary:
Pull Request resolved: facebookresearch#4555

Pull Request resolved: facebookresearch#4291

moved IndexIVFPQ and IndexPQ to dynamic dispatch.
Since the code was already quite modular (thanks Alex!), this boils down to make independent cpp files for the different SIMD versions.

Reviewed By: mnorris11

Differential Revision: D72937709
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants