-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement multi-sv detection #2510
Conversation
{c.Parts.map((p) => { | ||
return ( | ||
<li key={p.QueryPart}> | ||
[{p.QueryPart}] | ||
<ul> | ||
{p.SV.length === p.CosineScore.length && | ||
p.SV.map((sv, i) => { | ||
return ( | ||
<li key={i}> | ||
{sv} ({p.CosineScore[i]}) | ||
</li> | ||
); | ||
})} | ||
</ul> | ||
</li> | ||
); | ||
})} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: make a helper component to wrap this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
nl_server/embeddings.py
Outdated
|
||
TEMP_DIR = '/tmp/' | ||
MODEL_NAME = 'all-MiniLM-L6-v2' | ||
|
||
# A value higher than the highest score. | ||
_INIT_SCORE = 1.1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe have _HIGHEST_SCORE and use (_HIGHEST_SCORE + 0.1) for _INIT_SCORE so the purpose is more apparent?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
svs: List[str] | ||
scores: List[float] | ||
sv2sentences: Dict[str, List[str]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might not worth a change for now, but why not having a list of {sv, score, sentences} struct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It has been that way from before, so not changing it now.
nl_server/embeddings.py
Outdated
class MultiVarCandidate: | ||
parts: List[MultiVarCandidatePart] | ||
# Aggregate score | ||
score: float |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use "aggregate_score" explicitly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
nl_server/embeddings.py
Outdated
@dataclass | ||
class MultiVarCandidates: | ||
candidates: List[MultiVarCandidate] | ||
sv2sentences: Dict[str, List[str]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why sv2sentences is not in MultiVarCandidate? what is they key (one stat var) mean in a multiVarCandidates scenario.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sv2sentences is a debug map of sentences that we matched for each SV in the index. But it is not used at the moment, so let me remove and add it in later PR.
scores_sorted = [v for (_, v) in sv2score_sorted] | ||
|
||
sv_index_sorted = [sv2index[k] for (k, _) in sv2score_sorted] | ||
query2sv2sentence2score: Dict[str, Dict[str, Dict[str, float]]] = {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
have a comment with an example of query2sv2sentence2score?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Described it.
|
||
query2result: Dict[str, VarCandidates] = {} | ||
|
||
for q, sv2score in query2sv2score.items(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have a comment for this and the next large for loop on what it processes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
'SV_to_Sentences': all_svs_sentences, | ||
'SV': result_monovar.svs, | ||
'CosineScore': result_monovar.scores, | ||
'SV_to_Sentences': result_monovar.sv2sentences, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use CamelCase too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is from before, and will touch a few unrelated places, so can change in a follow on PR. Added TODO
# | ||
# TODO: Come up with a better ranking function. | ||
# | ||
for qs in querysets: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some examples of "querysets" is helpful, since its fields are very deep
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, PTAL.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review Bo!
nl_server/embeddings.py
Outdated
|
||
TEMP_DIR = '/tmp/' | ||
MODEL_NAME = 'all-MiniLM-L6-v2' | ||
|
||
# A value higher than the highest score. | ||
_INIT_SCORE = 1.1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
svs: List[str] | ||
scores: List[float] | ||
sv2sentences: Dict[str, List[str]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It has been that way from before, so not changing it now.
nl_server/embeddings.py
Outdated
class MultiVarCandidate: | ||
parts: List[MultiVarCandidatePart] | ||
# Aggregate score | ||
score: float |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
nl_server/embeddings.py
Outdated
@dataclass | ||
class MultiVarCandidates: | ||
candidates: List[MultiVarCandidate] | ||
sv2sentences: Dict[str, List[str]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sv2sentences is a debug map of sentences that we matched for each SV in the index. But it is not used at the moment, so let me remove and add it in later PR.
'SV_to_Sentences': all_svs_sentences, | ||
'SV': result_monovar.svs, | ||
'CosineScore': result_monovar.scores, | ||
'SV_to_Sentences': result_monovar.sv2sentences, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is from before, and will touch a few unrelated places, so can change in a follow on PR. Added TODO
{c.Parts.map((p) => { | ||
return ( | ||
<li key={p.QueryPart}> | ||
[{p.QueryPart}] | ||
<ul> | ||
{p.SV.length === p.CosineScore.length && | ||
p.SV.map((sv, i) => { | ||
return ( | ||
<li key={i}> | ||
{sv} ({p.CosineScore[i]}) | ||
</li> | ||
); | ||
})} | ||
</ul> | ||
</li> | ||
); | ||
})} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
scores_sorted = [v for (_, v) in sv2score_sorted] | ||
|
||
sv_index_sorted = [sv2index[k] for (k, _) in sv2score_sorted] | ||
query2sv2sentence2score: Dict[str, Dict[str, Dict[str, float]]] = {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Described it.
|
||
query2result: Dict[str, VarCandidates] = {} | ||
|
||
for q, sv2score in query2sv2score.items(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
# | ||
# TODO: Come up with a better ranking function. | ||
# | ||
for qs in querysets: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, PTAL.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apologies for the delay. Some comments which might be good to discuss
# Given a list of queries, searches the in-memory embeddings index | ||
# and returns a map of candidates keyed by input queries. | ||
# | ||
def _search_embeddings(self, queries: List[str]) -> Dict[str, VarCandidates]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question: why do the refactor of this function in the same PR as the introduction of a new function for detecting multi svs? Is there some functionality that was missing which this refactor addresses?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The refactor takes multiple queries and returns results, which was needed for the multi SV case.
combinations: List[QuerySplit] | ||
|
||
|
||
# |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be great to document this with some examples. With the recursive calls inside this function, highly recommended extensive documentation because it is fairly hairy to figure out what's going on. I also recommend more comments to help step line by line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm adding comments and unit tests in a follow on PR.
# | ||
# Prepare a combination of query-sets. | ||
# | ||
querysets = query_util.prepare_multivar_querysets(query) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reading the comments below about a queryset, wouldn't it be better to use this combinatorial API/function built in to python? https://www.geeksforgeeks.org/itertools-combinations-module-python-print-possible-combinations/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good idea. doing in the follow on PR.
Fold in the great feedback from @jehangiramjad on #2510. This also fixes a bug in previous implementation.
Bumps [redis](https://github.com/redis/redis-py) from 3.5.3 to 4.5.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/redis/redis-py/releases">redis's releases</a>.</em></p> <blockquote> <h2>4.5.3</h2> <h1>Changes</h1> <p>Update urgency: HIGH: There is a critical bug that may affect a subset of users. Upgrade!</p> <h2>🐛 Bug Fixes</h2> <ul> <li><a href="https://cwe.mitre.org/data/definitions/404.html">CWE-404</a> AsyncIO Race Condition Fix (<a href="https://github.com/redis/redis-py/issues/2624">#2624</a>, <a href="https://github.com/redis/redis-py/issues/2579">#2579</a>)</li> </ul> <h2>4.5.2</h2> <h1>Changes</h1> <h2>🚀 New Features</h2> <ul> <li>Introduce AbstractConnection so that UnixDomainSocketConnection can call super().<strong>init</strong> (<a href="https://github.com/redis/redis-py/issues/2588">#2588</a>)</li> <li>Added queue_class to REDIS_ALLOWED_KEYS (<a href="https://github.com/redis/redis-py/issues/2577">#2577</a>)</li> <li>Made search document subscriptable (<a href="https://github.com/redis/redis-py/issues/2615">#2615</a>)</li> <li>Sped up the protocol parsing (<a href="https://github.com/redis/redis-py/issues/2596">#2596</a>)</li> </ul> <h2>🐛 Bug Fixes</h2> <ul> <li>Fix behaviour of async PythonParser to match RedisParser as for issue <a href="https://github.com/redis/redis-py/issues/2349">#2349</a> (<a href="https://github.com/redis/redis-py/issues/2582">#2582</a>)</li> <li>Replace async_timeout by asyncio.timeout (<a href="https://github.com/redis/redis-py/issues/2602">#2602</a>)</li> <li>Update json().arrindex() default values (<a href="https://github.com/redis/redis-py/issues/2611">#2611</a>)</li> </ul> <h2>🧰 Maintenance</h2> <ul> <li>Coverage for pypy-3.9 (<a href="https://github.com/redis/redis-py/issues/2608">#2608</a>)</li> <li>Developer Experience: Adding redis version compatibility details to the README (<a href="https://github.com/redis/redis-py/issues/2621">#2621</a>)</li> <li>Remove redundant assignment to RedisCluster.nodes_manager. (<a href="https://github.com/redis/redis-py/issues/2620">#2620</a>)</li> <li>Developer Experience: [types] update return type of smismember to list[int] (<a href="https://github.com/redis/redis-py/issues/2617">#2617</a>)</li> <li>Developer Experience: [docs] ConnectionPool SSL example (<a href="https://github.com/redis/redis-py/issues/2605">#2605</a>)</li> <li>Developer Experience: Fixed CredentialsProvider examples (<a href="https://github.com/redis/redis-py/issues/2587">#2587</a>)</li> <li>Developer Experience: Update README to make pip install copy-pastable on zsh (<a href="https://github.com/redis/redis-py/issues/2584">#2584</a>)</li> <li>Developer Experience: Fix for <code>lpop</code> and <code>rpop</code> return typing (<a href="https://github.com/redis/redis-py/issues/2590">#2590</a>)</li> </ul> <h2>Contributors</h2> <p>We'd like to thank all the contributors who worked on this release!</p> <p><a href="https://github.com/CrimsonGlory"><code>@CrimsonGlory</code></a>, <a href="https://github.com/Galtozzy"><code>@Galtozzy</code></a>, <a href="https://github.com/aksinha334"><code>@aksinha334</code></a>, <a href="https://github.com/barshaul"><code>@barshaul</code></a>, <a href="https://github.com/chayim"><code>@chayim</code></a>, <a href="https://github.com/davemcphee"><code>@davemcphee</code></a>, <a href="https://github.com/dvora-h"><code>@dvora-h</code></a>, <a href="https://github.com/kristjanvalur"><code>@kristjanvalur</code></a>, <a href="https://github.com/ryin1"><code>@ryin1</code></a>, <a href="https://github.com/sileht"><code>@sileht</code></a>, <a href="https://github.com/thebarbershop"><code>@thebarbershop</code></a>, <a href="https://github.com/uglide"><code>@uglide</code></a>, <a href="https://github.com/woutdenolf"><code>@woutdenolf</code></a> and <a href="https://github.com/zakaf"><code>@zakaf</code></a></p> <h2>4.5.1</h2> <h1>Changes</h1> <h2>🐛 Bug Fixes</h2> <ul> <li>Fix <a href="https://github.com/redis/redis-py/issues/2581">#2581</a> <code>UnixDomainSocketConnection</code> object has no attribute <code>_command_packer</code> (<a href="https://github.com/redis/redis-py/issues/2583">#2583</a>)</li> </ul> <h2>Contributors</h2> <p>We'd like to thank all the contributors who worked on this release!</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/redis/redis-py/blob/master/CHANGES">redis's changelog</a>.</em></p> <blockquote> <pre><code>* Allow data to drain from async PythonParser when reading during a disconnect() * Use asyncio.timeout() instead of async_timeout.timeout() for python >= 3.11 ([datacommonsorg#2602](redis/redis-py#2602)) * Add test and fix async HiredisParser when reading during a disconnect() ([datacommonsorg#2349](redis/redis-py#2349)) * Use hiredis-py pack_command if available. * Support `.unlink()` in ClusterPipeline * Simplify synchronous SocketBuffer state management * Fix string cleanse in Redis Graph * Make PythonParser resumable in case of error ([datacommonsorg#2510](redis/redis-py#2510)) * Add `timeout=None` in `SentinelConnectionManager.read_response` * Documentation fix: password protected socket connection ([datacommonsorg#2374](redis/redis-py#2374)) * Allow `timeout=None` in `PubSub.get_message()` to wait forever * add `nowait` flag to `asyncio.Connection.disconnect()` * Update README.md links * Fix timezone handling for datetime to unixtime conversions * Fix start_id type for XAUTOCLAIM * Remove verbose logging from cluster.py * Add retry mechanism to async version of Connection * Compare commands case-insensitively in the asyncio command parser * Allow negative `retries` for `Retry` class to retry forever * Add `items` parameter to `hset` signature * Create codeql-analysis.yml ([datacommonsorg#1988](redis/redis-py#1988)). Thanks @chayim * Add limited support for Lua scripting with RedisCluster * Implement `.lock()` method on RedisCluster * Fix cursor returned by SCAN for RedisCluster & change default target to PRIMARIES * Fix scan_iter for RedisCluster * Remove verbose logging when initializing ClusterPubSub, ClusterPipeline or RedisCluster * Fix broken connection writer lock-up for asyncio ([datacommonsorg#2065](redis/redis-py#2065)) * Fix auth bug when provided with no username ([datacommonsorg#2086](redis/redis-py#2086)) * Fix missing ClusterPipeline._lock ([datacommonsorg#2189](redis/redis-py#2189)) * Added dynaminc_startup_nodes configuration to RedisCluster * Fix reusing the old nodes' connections when cluster topology refresh is being done * Fix RedisCluster to immediately raise AuthenticationError without a retry * ClusterPipeline Doesn't Handle ConnectionError for Dead Hosts ([datacommonsorg#2225](redis/redis-py#2225)) * Remove compatibility code for old versions of Hiredis, drop Packaging dependency * The `deprecated` library is no longer a dependency * Failover handling improvements for RedisCluster and Async RedisCluster ([datacommonsorg#2377](redis/redis-py#2377)) * Fixed "cannot pickle '_thread.lock' object" bug ([datacommonsorg#2354](redis/redis-py#2354), [datacommonsorg#2297](redis/redis-py#2297)) * Added CredentialsProvider class to support password rotation * Enable Lock for asyncio cluster mode * Fix Sentinel.execute_command doesn't execute across the entire sentinel cluster bug ([datacommonsorg#2458](redis/redis-py#2458)) * Added a replacement for the default cluster node in the event of failure ([datacommonsorg#2463](redis/redis-py#2463)) * Fix for Unhandled exception related to self.host with unix socket ([datacommonsorg#2496](redis/redis-py#2496)) </code></pre> <ul> <li>4.1.3 (Feb 8, 2022) <ul> <li>Fix flushdb and flushall (<a href="https://github.com/redis/redis-py/issues/1926">#1926</a>)</li> <li>Add redis5 and redis4 dockers (<a href="https://github.com/redis/redis-py/issues/1871">#1871</a>)</li> <li>Change json.clear test multi to be up to date with redisjson (<a href="https://github.com/redis/redis-py/issues/1922">#1922</a>)</li> <li>Fixing volume for unstable_cluster docker (<a href="https://github.com/redis/redis-py/issues/1914">#1914</a>)</li> <li>Update changes file with changes since 4.0.0-beta2 (<a href="https://github.com/redis/redis-py/issues/1915">#1915</a>)</li> </ul> </li> <li>4.1.2 (Jan 27, 2022)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/redis/redis-py/commit/66a4d6b2a493dd3a20cc299ab5fef3c14baad965"><code>66a4d6b</code></a> AsyncIO Race Condition Fix (<a href="https://github.com/redis/redis-py/issues/2641">#2641</a>)</li> <li><a href="https://github.com/redis/redis-py/commit/318b114f4da9846a2a7c150e1fb702e9bebd9fdf"><code>318b114</code></a> Version 4.5.2 (<a href="https://github.com/redis/redis-py/issues/2627">#2627</a>)</li> <li><a href="https://github.com/redis/redis-py/commit/1b2f408259405d412d7530291902f9e0c8bd34b3"><code>1b2f408</code></a> Fix behaviour of async PythonParser to match RedisParser as for issue <a href="https://github.com/redis/redis-py/issues/2349">#2349</a> (...</li> <li><a href="https://github.com/redis/redis-py/commit/7d474f90453c7b90bd06c94e0250b618120a599d"><code>7d474f9</code></a> introduce AbstractConnection so that UnixDomainSocketConnection can call supe...</li> <li><a href="https://github.com/redis/redis-py/commit/c87172347584301f453c601c483126e4800257b7"><code>c871723</code></a> pypy-3.9 CI (<a href="https://github.com/redis/redis-py/issues/2608">#2608</a>)</li> <li><a href="https://github.com/redis/redis-py/commit/d63313bf6080acaf18d61e072c78303adc0d4166"><code>d63313b</code></a> add queue_class to REDIS_ALLOWED_KEYS (<a href="https://github.com/redis/redis-py/issues/2577">#2577</a>)</li> <li><a href="https://github.com/redis/redis-py/commit/c61eeb2e3b5dff1f01eb1e665f424c7e75354f56"><code>c61eeb2</code></a> Adding supported redis/library details (<a href="https://github.com/redis/redis-py/issues/2621">#2621</a>)</li> <li><a href="https://github.com/redis/redis-py/commit/25e85e51e57b7aae9eb8fc77cfb0a45a07a501a7"><code>25e85e5</code></a> fix: replace async_timeout by asyncio.timeout (<a href="https://github.com/redis/redis-py/issues/2602">#2602</a>)</li> <li><a href="https://github.com/redis/redis-py/commit/91ab12a0f1bdf0e433131e1a51578e9fa2f89718"><code>91ab12a</code></a> Remove redundant assignment. (<a href="https://github.com/redis/redis-py/issues/2620">#2620</a>)</li> <li><a href="https://github.com/redis/redis-py/commit/8bfd492240fd33489a86cd3d353e3ece1fc94c10"><code>8bfd492</code></a> Making search document subscriptable (<a href="https://github.com/redis/redis-py/issues/2615">#2615</a>)</li> <li>Additional commits viewable in <a href="https://github.com/redis/redis-py/compare/3.5.3...v4.5.3">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=redis&package-manager=pip&previous-version=3.5.3&new-version=4.5.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/datacommonsorg/website/network/alerts). </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Bo Xu <[email protected]>
Roughly as described in this doc. Not hooked up to fulfillment yet. Propagated to debug info (screenshot below).
This PR moves the stop-word removal done for single-sv detection from website server to nl server. This is so that multi-sv detection can be done with the stop-words present.
BUG=#2397