Conversation
|
I like this idea! I definitely think we should fork important deps that are abandoned. Suggestion: To make this more in line with our styles, run |
|
@kjbekkelund I'd like to get the original files committed without changes first, and then I can follow up with a commit that restyles them. |
|
cc @simianhacker for a heads up |
I think that warrants some more discussion. Flot is only a part of the code-base, an implementation detail for the most part. There is still much d3 too. Both had advantages and disadvantages. Choosing on a charting library (flot, d3, plotly, ...) is imho not that fundamental of a decision if we want more chart reuse. |
|
@thomasneirynck Yeah, sorry... I don't even know why I described it in that way. My goal here is strictly to move an abandoned dependency into Kibana so that we can remove unnecessary stuff from it and own it for as long as it is an important tool for some features in Kibana. This shouldn't in any way mean we're planning to use this as our visualizations baseline going forward. I'll update the message accordingly. |
The flot-charts project has been abandoned since 2014, but we use it pretty extensively in Kibana. By forking it, we can fix bugs and improve on the library in ways that we need to for Kibana.
|
I just want to note a point which @pickypg has made, which is that we should explore incorporating our charting tool into the UI Framework (once we conclude the discussion @thomasneirynck mentioned and settle on a single, common charting tool). |
|
I think it makes a lot of sense to fork Flot, so we are able to take control over a key dependency, and fix issues we have with it. But wouldn't it be better to create a fork independent of Kibana, that the community can use and contribute to (without pulling in the entire Kibana world?). Actually: If we contact the original maintainer(s) of Flot they might even give us commit rights to the current repo, and we can take over ownership - this way the community doesn't have to go find a new package to get updates a project they are currently using. Is that something we have looked into? |
|
@sqren If we were committed to maintaining flot as a standalone, general purpose visualizations library for the long term, then an independent fork would be the way to go. We're still unsure what we want to do with flot, and it's entirely possible we'll move away from it entirely. We have to take creating new open source projects really seriously because maintaining open source projects is a large amount of effort, and abandoning them after we've created them isn't fair to any of the users that come to rely on them. |
|
@epixa It was my impression that you wanted to revive the Flot project. I can see how we don't want to be official maintainers of a library that we are not 100% committed to. |
thomasneirynck
left a comment
thomasneirynck
left a comment
There was a problem hiding this comment.
thanks, tested with timelion, tsvb, works!
The flot-charts project has been abandoned since 2014, but we use it pretty extensively in Kibana. By forking it, we can fix bugs and improve on the library in ways that we need to for Kibana.
|
6.x/6.1 7799851 |
The flot-charts project has been abandoned since 2014, but we use it pretty extensively in Kibana. By forking it, we can fix bugs and improve on the library in ways that we need to for Kibana.
…lastic#14363) Land the natural-language search API over SML, replacing the transitional bool.should mash with a proper RRF retriever and introducing the trust-boundary split between runtime-imposed scoping and agent-discoverable filters. Decisions per the implementation prompt §6: - §6.1 Retrieval: RRF over BM25 (title^2, description, content via best_fields multi_match) and semantic (unified_semantic). Filters mirrored to each child retriever so RRF can't pull unauthorized docs into the fused top-k. Defaults rank_constant=60, rank_window_size=50. - §6.2 Filter merging: option (2). Service signature splits `scoping?: SmlSearchScoping` (runtime-imposed, unconditional) from `filters?: SmlSearchFilters` (caller refinement). Trust boundary visible at the API layer; merge happens in one place. Sean's per-type id-allowlist shape (elastic#267333) survives unchanged as `SmlSearchScoping`. - §6.3 Agent-supplied dimensions: `types?: string[]`, `tags?: string[]`, each lowering into a `terms` clause. Time-range and free-form payload filters deferred. - §6.4 Compact LLM response: SmlSearchResult drops the `content` blob, full `payload`, dates, and discovery_labels; gains `more_content` (true when the indexed record has non-empty content worth fetching via sml_read). `permissions` retained internally for post-hoc filtering, stripped from the HTTP response. - §6.5 Reference resolution: URI strings only — SML does not dereference. Inline-expansion deferred until eval data justifies it (Reading 1; Sean's "SML shouldn't turn pointers into blobs"). - §6.6 Tool description rewrite with when-to-use guidance and four worked examples (plain query, types+tags filter, connector restriction, wildcard inventory). Runtime scoping is not exposed in the LLM input schema. Fixes the `total` bug: post-permission-filter `results.length` no longer overwrites ES `hits.total.value`. Frontend / HTTP route updates: - POST /sml/_search body accepts `scoping?` and `filters?` (replaces the old `filters` per-type id-allowlist field and `skip_content`). - POST /sml/_autocomplete renames `filters` → `scoping`; no agent-discoverable filters on this route (no LLM in the path). - Frontend SmlService, useSmlSearch, useSmlAutocomplete, usePrefetchSml, queryKeys, and buildSmlScopingFromAgent (formerly buildSmlFiltersFromAgent) all follow the new naming. Tests: - New unit tests for RRF body, filter threading through child retrievers, vocabulary-mismatch over the semantic leg, more_content on/off, compact result mapping, agent-filter builder, and tool wrapper scoping/filters mapping. - FTR + Scout integration tests updated to drop content/skip_content assertions and lock in the no-content invariant. Out of scope (deferred per epic): reranker, kneedle, per-type retriever profiles, HyDE/query expansion, reference inline-expansion, telemetry instrumentation (elastic#14366), recrawl/versioning (elastic#14367). Open coordination points (not blocking this PR): - Inference model on unified_semantic still relies on defaults (Kathleen Mar 19: pin Jina V5). Belongs to A. - Kathleen review on A's PR proposes splitting unified_semantic into per-field semantic_text fields. If A reverts unified, the semantic leg in this PR fans out from 1 to 2-3 child retrievers — small retriever-only change. - Pierre's configuration_override-aware tool handler context (elastic#267333 review) — wrapper currently reads agentConfiguration from the tool handler context, which `runner.ts` already resolves with overrides applied via `resolveConfiguration`. Note in code. Stacked on top of apmats/sml-autocomplete-followup (elastic#14364). Once autocomplete merges, this PR rebases to main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 participants
The flot-charts project has been abandoned since 2014, but we use it
pretty extensively in Kibana. By forking it, we can fix bugs and improve
on the library in ways that we need to for Kibana.
I've wanted to do this for a long time, but I was recently inspired by #14312
Notes:
directly from the flot git repo. I did, however, remove a bunch of files that
we don't need nor want, like example code.
dependencies. I'll follow up with a PR to move that over to this instead.