Yong Studtandt8y2 98342184af feat(elasticsearch): translate SELECT * WHERE to ES DSL in-process (#910) ...

* feat(elasticsearch): translate SELECT * WHERE to ES DSL in-process

`SELECT *` with clauses our hand-written parser doesn't cover (WHERE, IN, BETWEEN, LIKE, IS NULL, ...) is now parsed with sqlparser-rs and translated to a /_search body locally — we no longer hand it off to ES's _sql endpoint.

Why: _sql refuses several common shapes — LIKE on a text field with no .keyword sub-field (the typical filebeat / log-shipper mapping), and SELECT * over docs containing an array field like host.ip. Both translate cleanly to raw DSL. Going through _sql/translate doesn't help either — that lives in the same ES SQL engine and inherits the same restrictions.

Translation:
- field = 'v'                  → term
- field LIKE 'prefix%'         → prefix (optimised)
- field LIKE '%x%'             → wildcard, case_insensitive
- field IN ('a','b')           → terms
- field BETWEEN a AND b        → range gte/lte
- field >/<>=/<=               → range
- field IS NULL / IS NOT NULL  → bool.must_not.exists / exists
- A AND B AND C                → flattened bool.must
- A OR B                       → bool.should, minimum_should_match: 1
- NOT A                        → bool.must_not
- ORDER BY f ASC|DESC          → sort
- LIMIT N OFFSET M             → size / from

SQL LIKE patterns: % → *, _ → ?, backslash escapes preserved, user-written * / ? in patterns escaped back to literals.

Input runs through the existing adapt_elasticsearch_sql_query first so hyphenated indices (filebeat-7.17.1-…) and @timestamp-style identifiers reach sqlparser as double-quoted identifiers.

Also: 0-hit _search bodies now surface as an empty grid (with an _id column placeholder) instead of falling back to the raw status/response JSON view — `.filter(|h| !h.is_empty())` was masking the empty-result case.

Adds docs/screenshot-es-sql-where.png demonstrating SELECT * with WHERE log.offset = N on a long field — would 400 through _sql, works through the in-process translator.

Follows up on #874.

* chore(test): fill ConnectionConfig::idle_timeout_secs in test fixtures

After the new pub idle_timeout_secs: u64 field was added to ConnectionConfig, 11 #[cfg(test)] / tests fixture constructors still built the struct without it, so `cargo test --workspace --locked` and `cargo clippy --all-targets` would fail with E0063 against the test profile. `cargo check` alone passed because the lib path doesn't compile tests.

Filled in with default_idle_timeout_secs() (matches the field's own #[serde(default = …)]) so future bumps to the default flow through automatically.

---------

Co-authored-by: t8y2 <1156263951@qq.com>

2026-06-09 11:59:49 +08:00

77 KiB

1598x805px

Raw Permalink History