Skip to content

Use bit arrays for predicate matching in search. #8684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

isoos
Copy link
Collaborator

@isoos isoos commented Apr 1, 2025

  • Package search index improvements (tracking issue) #8671
  • There are two further potential improvements, but I would implement them in a subsequent PRs: (a) reusing the bitarray instance, and (b) not creating/using a PackageScore instance when the BitArray would suffice.
  • Measured around 5-10% improvement for predicate-only search, and around 1-4% for more complex text matching + predicates.

@isoos isoos requested review from jonasfj and sigurdm April 1, 2025 10:49
@isoos
Copy link
Collaborator Author

isoos commented Apr 16, 2025

I've modified and benchmarked the 64-bit version of BitArray and couldn't point out much different performance to the current 32-bit implementation. With that in mind, I've vendored the library as-is, with minor updates (e.g. removing the part of and adding imports). I'll also update package:bit_array to reflect these changes, so next time the update will be straightforward file copy.

Should we also vendor some of the tests too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants