chore: performing sparse search with query expansion in langchain

[bruvduroiu]

Problem

Showcase the use of Pinecone sparse embeddings and sparse indexes for in situations where dense vector search might miss specific technical terms.

Solution

• use Pinecone sparse embeddings
• use query expansion technique
• allow Langchain agent to search for specific technical terms from general queries ("Which tech companies were top performers this week?" transforms into vector search about "GOOGL performance", "NVIDIA Performance", ...

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update
• Infrastructure change (CI configs, etc)
• Non-code change (docs, etc)
• None of the above: (updates to documentation)

Test Plan

Describe specific steps for validating this change.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB