4 posts tagged with "Observability"

Spice v2.4.0-rc.2 is now available! 🔥

Spice v2.4.0-rc.2 is the second release candidate of v2.4.0 for early testing and feedback. It introduces a new OpenAI-compatible POST /v1/decisions API backed by an OpenAI decision model, TypeSafe Jev, or any other LLM in the Spicepod that can answer decisions. Five new SQL functions bring the same AI classification, scoring, and decisions into SQL.

This release also adds Iceberg reads without a catalog and improves acceleration snapshots. Cayenne gains runtime-wide adaptive tuning and applies acceleration configuration changes on restart.

Highlights in this release:

• AI Decisions API and decisions in SQL: An OpenAI-compatible /v1/decisions API, plus five SQL functions for filtering, classifying, and scoring data with models.
• Iceberg without a catalog: Read a table directly from its root directory or pin a metadata file to read a fixed version.
• Shared runtime state: Startup validation and a common location for cluster state, rate control, cache warmup, and acceleration snapshots.
• SQL results cache: Cache entries expire by TTL and remain available across data refreshes.
• Cayenne acceleration: Improvements to primary-key loading, secondary indexes, and configuration updates.
• OpenTelemetry log export: Send runtime logs to any OTLP endpoint as structured log records, and choose which failed queries are logged.

These notes cover changes since v2.4.0-rc.1. Review Breaking Changes before upgrading; upgrades from earlier versions also require the rc.1 migration guidance.

What's New in v2.4.0-rc.2​

AI Decisions API and decisions in SQL​

Spice now supports POST /v1/decisions, which is compatible with the OpenAI Decisions API. A decision request asks typed questions about an input. Each answer comes back with its probabilities and a confidence.

Any model in the Spicepod can answer a decision request:

OpenAI decision models use the native OpenAI Decisions API. gpt-6-luna also supports chat and /v1/nsql. TypeSafe Jev returns calibrated probabilities and answers decisions only. Other LLMs answer through chat completions and return their own uncalibrated estimates.

models:
  - from: openai:gpt-6-luna
    name: luna
    params:
      openai_api_key: ${secrets:OPENAI_API_KEY}
  - from: typesafe:jev
    name: jev
    params:
      typesafe_api_key: ${secrets:TYPESAFE_API_KEY}

Five new SQL functions make the same decisions available in queries:

SELECT
    id,
    ai_classify(body, ['billing', 'technical', 'account'], model => 'support_model' ) AS team,
    ai_probability(body, 'The customer is requesting a refund', model => 'support_model') AS refund_probability
FROM tickets
WHERE status = 'open';

See AI functions for more examples.

Iceberg Without a Catalog​

Read an Iceberg table directly from object storage or the local filesystem, without configuring a catalog. Set from to the table root using file://, s3://, s3a://, gs://, or gcs://:

datasets:
  - from: iceberg:s3://my-bucket/warehouse/db/taxis
    name: taxis
    params:
      iceberg_s3_region: us-east-1

Spice resolves the metadata file on each query and refresh. It supports metadata/version-hint.text as well as a fixed version (a path relative to the table root or a full URL):

datasets:
  - from: iceberg:s3://my-bucket/warehouse/db/taxis
    name: taxis
    params:
      iceberg_s3_region: us-east-1
      iceberg_metadata_path: /metadata/00003-8f2c.metadata.json

Fixed tables are read-only. The earlier metadata_path parameter remains supported with a deprecation warning; see Breaking Changes.

Shared Runtime State​

runtime.state provides a shared, object-store location for cluster state, rate control, SQL results-cache warmup, and acceleration snapshots. Each feature uses its own subdirectory within the provided location.

Acceleration snapshots now default to the snapshots/ subdirectory when runtime.state is configured.

runtime:
  state:
    location: s3://my-bucket/spice/

datasets:
  - from: postgres:orders
    name: orders
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      snapshots: enabled

A dataset can override the snapshot location, its settings, and the bootstrap behavior on its own acceleration:

datasets:
  - from: postgres:events
    name: events
    acceleration:
      snapshots: enabled
      snapshots_location: s3://archive-bucket/snapshots/
      snapshots_params:
        s3_region: eu-west-1
      snapshots_bootstrap_on_failure_behavior: fallback

Other changes to shared state:

• Snapshot queues: An SQS queue that covers the whole runtime.state prefix no longer causes an ERROR log for each scheduler heartbeat. The snapshot reader now deletes messages for the other state folders without a log.

SQL Results Cache​

The SQL results cache now expires entries by TTL, independent of acceleration refreshes, CDC batches, and retention passes no longer invalidate cached results.

Review item_ttl and stale_while_revalidate_ttl when upgrading: a successful refresh no longer makes cached results reflect the refreshed data immediately. See Breaking Changes.

Cayenne Acceleration​

Adaptive tuning is configured once for the runtime. Enable it with runtime.adaptive_tuning and set targets under runtime.params:

runtime:
  adaptive_tuning: enabled
  params:
    target_freshness: 30s

Supported targets are target_replication_lag, target_freshness, target_query_latency, target_convergence_window, and target_qph. Adaptive tuning defaults to disabled. See Breaking Changes and the runtime YAML documentation.

Other Cayenne improvements include:

• Primary-key loading: Keyed tables build their primary-key filter during the initial load and bulk-load without per-row routing or a whole-table fold.
• Position deletes: cayenne_deletion_mode: auto can select position deletes for small primary-key tables on local storage on Linux systems with at least eight physical cores.
• Secondary indexes: Index runs are persisted, indexes remain complete across acceleration snapshots, and CDC maintains indexes for each operation type. Hash joins read each file once when narrowing a scan through an index.
• Partition handling: Cayenne rejects stored data with an incompatible partition_by, protects keyed appends to partitioned tables during publication, and prepares every partition after a refresh.
• Storage and query correctness: Cayenne reclaims snapshot directories superseded by a reload, corrects pruning for NaN values, and fixes deletion-index bounds and reload fencing.
• Snapshot storage growth: A writer with snapshots_trigger: refresh_complete kept one more copy of the table for each refresh. Each snapshot held file deletions, so Cayenne skipped each cleanup pass. Cayenne now runs the cleanup when a snapshot releases its hold. In a 10-minute test with a refresh every 60 seconds, each dataset kept 1 snapshot directory instead of 10.
• Configuration changes on restart: A change to write settings only, such as cayenne_sort_columns or cayenne_compression_strategy, applies to the existing acceleration. A change to primary_key or partition_by rebuilds the acceleration from the source only under specific conditions. See Cayenne Configuration Updates.

OpenTelemetry Log Export​

Spice can now send its logs to an OpenTelemetry Collector, or any OTLP endpoint, as structured OTLP log records. Logs use the same otel_exporter that already sends metrics. A new runtime.query.log_errors setting chooses which failed queries are logged.

runtime:
  telemetry:
    otel_exporter:
      endpoint: otel-collector:4317
      signals: [metrics, logs] # new; default [metrics]
  query:
    log_errors: execution # new; none (default) | execution | all

• Structured records: Each log line becomes an OTLP log record with its severity, the message as the body, typed fields as attributes, and the logging module as the instrumentation scope. Records share the resource attributes of exported metrics.
• Trace correlation: A record logged during a query carries the query's trace ID, the same ID returned in spice-trace-id and stored in runtime.task_history.
• Delivery: Records are batched on a dedicated export runtime, separate from query and HTTP traffic, and flushed at shutdown so the final error and Goodbye! reach the endpoint. For OTLP/HTTP endpoints, logs are sent to <base>/v1/logs.
• Cluster executors: Executors export their logs with the scheduler's telemetry settings, including headers resolved from the scheduler's secrets.

runtime.query.log_errors controls which failed queries are logged:

Each record reads Query failed: <error>, with error_code and datasets fields and the query's trace ID. The SQL text is never logged.

Unknown-column errors are now reported with the correct error_code instead of InternalError, in logs, runtime.task_history, and the query_failures metric.

See OpenTelemetry Logs Export, Exporting logs, and runtime.query.log_errors.

Fixes and Improvements​

AI APIs and Models​

• Responses API: /v1/responses executes Spice tools and honors tool_recursion_limit when streaming. It returns Spice tool calls as mcp_call items in output, allowing clients that manage their own conversation history to replay them.
• Model controls: /v1/nsql accepts reasoning_effort, and rerank() uses evaluation models with a strategy selected for each query.
• Anthropic: Streaming handles responses that begin with a thinking block, empty text blocks are dropped, and anthropic_workspace_id is available as a model parameter.
• OpenAI: Models with an unknown service tier can load and serve requests.
• Luna: gpt-6-luna and its snapshots support chat, /v1/nsql, and native decisions.

Acceleration​

• Caching point lookups: Lookups on a refresh_mode: caching dataset failed with HTTP 400. This occurred when Cayenne accelerated the dataset, the primary key covered the request columns, and the table held two protected snapshots. The point-lookup plan did not combine the partitions of the Cayenne scan. Lookups now return the cached row or fetch it from the source.

Connectors​

• MySQL timestamps: Timestamp literals in queries that the runtime sends to MySQL now keep their fractional seconds.

Dependency Updates​

Breaking Changes​

Shared State Directory Layout and Startup Checks​

State written at the runtime.state root by rc.1 is no longer read from that location. Move existing state objects into the appropriate subdirectories before upgrading to retain them:

Azure locations (abfs:// and abfss://) need one more step. In rc.1, the runtime ignored the path in an Azure URL and wrote state at the container root. For example, with abfs://container/spice/, rc.1 wrote cluster.json at the container root. This release writes spice/cluster/cluster.json. Move Azure state from the container root into the configured path and then into the feature subdirectory.

An explicit runtime.scheduler.state_location retains its earlier layout. Without migration, Spice starts without the state previously stored at the shared root.

The configured store must accept writes and conditional writes. Spice does not become ready until startup validation succeeds and refuses stores with a newer state format. Check storage permissions and compatibility before upgrading.

SQL Results Cache Freshness​

A refresh no longer invalidates SQL results-cache entries. Queries can return data from before the latest refresh until item_ttl expires, plus stale_while_revalidate_ttl when configured. Reduce item_ttl if results need to follow source refreshes more closely.

Cayenne Tuning Configuration​

Replace the Cayenne tuning parameters with runtime settings that apply across datasets and catalogs:

runtime.adaptive_tuning accepts only enabled or disabled. Other values, including true, fail the Spicepod load.

Pinned Iceberg Tables​

Replace the deprecated metadata_path parameter with iceberg_metadata_path. The two settings resolve paths differently. metadata_path uses its value as given. iceberg_metadata_path resolves a value without a URL scheme relative to the table root in from. Convert an absolute local path to a full file:// URL or to a path relative to the table root.

Datasets pinned to a .json metadata file are read-only and fail to load with access: read_write. Remove that setting from pinned datasets before upgrading.

Cayenne Configuration Updates​

A restart now applies changed settings to an existing Cayenne acceleration. The result depends on the setting and the dataset:

• Write settings only: A change to only cayenne_sort_columns, cayenne_cluster_by, cayenne_compression_strategy, cayenne_delta_encoding, or cayenne_shard_key_columns applies to the existing acceleration. Stored rows keep their layout until a refresh or compaction rewrites them.
• Identity settings: A change to primary_key, partition_by, cayenne_pk_conflict_detection from none to auto, or the data location rebuilds the acceleration from the source. The runtime rebuilds only a dataset with refresh_mode: full, mode: file or file_update, and write_mode: write_through. The source must not be sink:, and must be readable with the schema of the dataset.
• All other datasets: The acceleration keeps all of its stored settings. This includes write settings that changed in the same restart. The runtime logs a warning that names the change and the fix.

While the runtime cannot read the source, a dataset that qualifies for a rebuild keeps its existing acceleration. A partition_by change or a conflict-detection change cannot use the stored rows, so it fails the dataset instead. Revert the setting, or restart when the source is readable.

A restart alone does not migrate a dataset that does not meet these conditions. Read the startup warning for the dataset before you delete stored data. Allow time for a rebuild when the conditions apply.

Upgrading​

Once release artifacts are available, select v2.4.0-rc.2 explicitly with the CLI, Docker tag, or Helm chart version.

CLI​

spice upgrade v2.4.0-rc.2

Docker​

docker pull spiceai/spiceai:2.4.0-rc.2

See Docker Hub for available tags.

Helm​

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.4.0-rc.2

Homebrew and AWS Marketplace​

Homebrew and AWS Marketplace follow their published versions. To select this release candidate, use an explicit version through the CLI or Docker.

Spice is also available in the AWS Marketplace.

Contributors​

• @bjchambers
• @ewgenius
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov
• @spiceemma

Changelog​

See the full changelog from v2.4.0-rc.1 to v2.4.0-rc.2.

• feat(connector-huggingface): Hugging Face datasets data connector (hf://datasets/...) by @lukekim in #14877
• fix(iceberg): Iceberg REST clients read the real table or get an error, never an empty one by @lukekim in #14588
• fix(federation): ask the executor for the SELECT 1 placeholder of an empty projection by @krinart in #14851
• fix(cayenne): float statistics must not leave NaN out of pruning or metadata answers (fixes #14719) by @claudespice in #14792
• fix: Update tpch benchmark snapshots for accelerated/s3[parquet]-duckdb[file]-2gib.yaml by @github-actions in #14850
• Update openapi.json by @github-actions in #14858
• test: strengthen assertions and enforce sign-off coverage by @lukekim in #14740
• Reapply "fix(sqlite, duckdb): keep a decimal AVG, and on SQLite a decimal SUM, out of the federated plan (fixes #14492)" by @krinart in #14888
• fix(caching): a GET and a POST to one HTTP path no longer share a cache entry (fixes #14768) by @claudespice in #14884
• fix(anthropic): drop empty text blocks from assistant messages by @Jeadie in #14834
• fix(cayenne): ask the table whether it takes a versioned append, instead of predicting it by @lukekim in #14882
• test(sqlite): guard the Date32 literal cast through the SQLite acceleration's own scan (fixes #14491) by @grokspice in #14703
• fix(cayenne): read each file once when a hash join narrows an indexed table's scan by @lukekim in #14934
• fix(sharepoint): list drive items when the listing contains a folder (fixes #2797) by @phillipleblanc in #14946
• fix(runtime): log each failing dataset's load failure (fixes #14918) by @phillipleblanc in #14939
• fix(deps): move the DataFusion pin past spiceai/datafusion#256, so FlightSQL Substrait plans read string columns (fixes #14935) by @lukekim in #14944
• fix(cayenne): publish a keyed partition's append as a protected overlay (fixes #14947) by @lukekim in #14955
• fix(acceleration): a failed acceleration build no longer keeps the dataset down until restart (fixes #14917) by @phillipleblanc in #14942
• fix(postgres): reject a changes dataset without a primary key when it loads by @phillipleblanc in #14960
• feat(spicepod): publish pull request reviews by @lukekim in #14964
• fix(ci): bound the REPL expect-script waits on silence, not total response time (fixes #13711) by @grokspice in #14963
• fix(cluster): treat secrets the scheduler declines as not found on executors by @phillipleblanc in #14957
• fix(flight): return query and data errors as InvalidArgument, not Internal by @phillipleblanc in #14956
• fix(duckdb,sqlite): correct pushdown for string_agg ORDER BY, concat NULLs, and unsupported fns by @lukekim in #14798
• fix(ci): provision DuckLake for the ducklake[catalog] schema test, which has never passed by @grokspice in #14952
• fix(llms): stream Anthropic responses that open with a thinking block (fixes #14909) by @krinart in #14920
• fix(arrow): serve lookups from enabled indexes; a repeated key disables the index with a warning by @sgrebnov in #14968
• feat(nsql): accept reasoning_effort on POST /v1/nsql by @lukekim in #14852
• fix(listing): scan file_extension '*' tables through ListingTable, so Glue joins broadcast again (fixes #14890) by @krinart in #14972
• fix(llms): load and serve an OpenAI model when a reply names an unknown service tier (fixes #14916) by @krinart in #14927
• feat(cayenne): always persist secondary index runs by @sgrebnov in #14979
• fix(cache): expire results-cache entries in insertion order instead of walking every LRU list by @bjchambers in #14919
• feat: decisions in SQL (ai_if, ai_probability, ai_classify, ai_score, ai_decide) and an OpenAI-compatible /v1/decisions by @lukekim in #14838
• fix(rrf): keep named arguments on nested search calls by @phillipleblanc in #14975
• fix(rate-control): hold instances that set different limits for one origin to the lowest (fixes #14913) by @krinart in #14932
• fix(vortex): keep temporal casts Vortex can't evaluate out of the scan by @sgrebnov in #14985
• fix(acceleration): recommend mode: file for CDC on_conflict migrations to Cayenne by @bjchambers in #14928
• test(huggingface): unblock the merge queue: read imdb's parquet conversion through @~parquet and fail fast on a vanished Hub pin by @grokspice in #14998
• test(cayenne): link the integration tests into one test binary by @bjchambers in #14976
• test(vortex): compile the integration tests into a binary the sign-off gate runs by @bjchambers in #14987
• fix(http): reject a request_path that URL parsing would rewrite by @bjchambers in #14965
• fix(clickhouse): load LowCardinality, DateTime64, Array, Map, Tuple, Enum, 128-bit integer and IP columns by @phillipleblanc in #14953
• Add anthropic_workspace_id param for Anthropic models by @Jeadie in #14829
• fix(acceleration): keep retention-evicted rows out of the on_zero_results source fallback by @lukekim in #14783
• test: run federated source tests on every supported LTS release line by @lukekim in #14936
• fix(drasi): forward CDC creates and snapshots as upserts by @ewgenius in #14791
• fix(datafusion-optimizer-rules): answer only plain partition columns from the listing by @lukekim in #14806
• fix(connector-huggingface): retry Hub API responses that break off part way, and harden the live-Hub check by @lukekim in #15009
• Improve types methods in runtime-rate-control by @Jeadie in #14573
• fix(models): /v1/responses runs Spice tools and honors tool_recursion_limit when streaming (fixes #14905, #14631) by @krinart in #14930
• Update openapi.json by @github-actions in #14908
• test(cayenne): run the heap-measuring tests in one isolated test binary by @bjchambers in #15010
• test(vortex): run vortex-datafusion's integration tests in the sign-off gate by @bjchambers in #15001
• fix(cayenne): refuse acceleration data stored under a different partition_by by @phillipleblanc in #15019
• Update openapi.json by @github-actions in #15018
• fix(ci): keep the OpenAPI regeneration PR current instead of skipping while one is open by @phillipleblanc in #15023
• fix: Cayenne deletion-index bounds and reload fencing, scan-statistics single-flight, UDF filter pushdown, Server-Timing, results-cache determinism, and view loading by @lukekim in #14993
• fix(cayenne): load write-back without on_conflict, and refuse pk_conflict_detection: none where it loses data (fixes #14886, #14889) by @bjchambers in #14891
• chore: Update Turso crate to 0.8.2 by @spiceemma in #15022
• feat(cayenne): runtime-wide adaptive_tuning and target_* settings by @peasee in #14844
• ci: apply whole-binary nextest overrides to the module test sets they select by @bjchambers in #15011
• fix(snapshots): give each snapshot publish its own object name, so publishes in the same second stop overwriting each other by @phillipleblanc in #15028
• test(data_components): compile the integration tests into one test binary by @bjchambers in #14991
• fix(glue): load Iceberg tables from S3 buckets with dots in the name by @phillipleblanc in #15053
• fix(runtime): give each runtime.state feature its own folder by @phillipleblanc in #15036
• test(data_components): make the HTTP provider snapshots independent of the machine by @bjchambers in #15000
• fix(json): reject incomplete arrays and avoid repeated parsing by @phillipleblanc in #15054
• test(runtime): drive the cdc_cayenne tests through Cayenne's change sink by @bjchambers in #15003
• ci(object_store_occ): run the WAL E2E suite on self-hosted runners against disk, MinIO, RustFS and spiceio by @lukekim in #15031
• test(llms): compile the credential-free tests into the integration binary by @bjchambers in #14990
• fix(cayenne): keep secondary indexes complete across acceleration snapshots by @sgrebnov in #15068
• ci(setup-minio): name the missing secret instead of failing in mc's argument parser by @grokspice in #15073
• ci: run Substrait Mode B on the Linux runners that can reach the trunk spiced builds by @grokspice in #15074
• feat(rerank): use evaluation models (TypeSafe) in rerank(), with per-query strategy by @Jeadie in #14782
• build(deps): bump the github-actions-dependencies group across 5 directories with 7 updates by @dependabot in #15027
• fix(cayenne): reclaim the snapshot directories a snapshot reader's reload supersedes by @sgrebnov in #15069
• feat(iceberg): read a table from a pinned metadata file without a catalog by @sgrebnov in #15081
• Fixes for MCP server by @Jeadie in #15041
• fix(search): vector_search returns its top k rows by score on chunked and multi-vector columns (fixes #14819) by @claudespice in #15056
• fix(postgres): keep applying CDC changes after a source column type change, and never store a rounded value (refs #14718) by @claudespice in #15037
• perf(iceberg): decode Iceberg scans at the session batch_size by @bjchambers in #15072
• fix(rate-control): admit waiting requests in arrival order at the charge in force (fixes #14912) by @krinart in #14931
• fix(rate-control): compare shared-origin settings as they resolve (fixes #14914) by @krinart in #14937
• fix(decisions): Fixes for /v1/decisions by @Jeadie in #15040
• perf(cayenne): bulk-load a keyed table without per-row routing or a whole-table fold by @bjchambers in #15076
• fix(cayenne): list a new snapshot's files once, when it is published by @bjchambers in #15077
• test(runtime,spiced): compile integration tests into declared binaries, run the ones no CI job ran, and remove the uncalled supersede_initial_load by @bjchambers in #15005
• fix(duckdb): unparse string literals with backslash-quote exactly by @lukekim in #15083
• ci(e2e): don't run rust-cache on the macOS build, whose save deletes cargo from sibling runners by @grokspice in #15100
• feat(testoperator): record per-query search metrics by @Jeadie in #15090
• fix(vortex): keep casts Vortex cannot evaluate out of the scan by @phillipleblanc in #15096
• feat(index): operation-aware CDC index maintenance via IndexChangeSet by @Jeadie in #14340
• ci: run merge-queue Mode A TPC-H and LLM integration tests on spiceai-macos-large by @lukekim in #15116
• docs(cayenne): the per-file fold merges no source files, but may roll by @bjchambers in #15101
• build(deps): bump async-openai to the fork's merge of upstream main by @krinart in #14980
• feat(iceberg): read a table root without a catalog, finding its current metadata on every query by @sgrebnov in #15095
• fix(runtime): fail recovery of an unsaved running query by @phillipleblanc in #15092
• perf(cayenne): build a keyed table's primary-key filter during its first load by @phillipleblanc in #15099
• test(cayenne): guard per-file statistics sharing for listings outside a scan view by @bjchambers in #15103
• feat(cache): age SQL results on TTL instead of invalidating them on refresh by @bjchambers in #15033
• fix(deps): estimate selective IN lists and OR-of-equalities from NDV by @bjchambers in #14140
• Change codeql runner from spiceai-macos to spiceai-macos-large by @lukekim in #15120
• fix(flightsql): prepared-statement GetFlightInfo schema matches the stream when parameters are bound by @lukekim in #15106
• fix(telemetry): report the configured CPU count as host.cpu.count by @lukekim in #15108
• fix(runtime-table): prepare every partition of a partitioned Cayenne table after a refresh by @bjchambers in #15111
• fix: Update Search integration test snapshots by @github-actions in #15050
• test(iceberg): read the Glue fixture unpinned and check the picked metadata file by @sgrebnov in #15115
• test(cayenne): differential and plan coverage for secondary indexes on partitioned accelerations by @bjchambers in #15113
• test: answer results-cache misses from the acceleration while the source is down by @bjchambers in #15112
• ci(lint): fail on test files no target compiles, and document the integration test layout by @bjchambers in #15109
• feat(snapshots): store snapshots under runtime.state, built by the shared state-location builder by @sgrebnov in #15117
• fix(federation): match NULL rows in pushed-down INTERSECT and EXCEPT (fixes #14941) by @sgrebnov in #15110
• fix(models): allowed_tools shape, provider error statuses, decision response model (fixes #14910) by @krinart in #14977
• feat(cayenne): let deletion_mode auto pick position deletes for small primary-key tables by @lukekim in #15042
• fix(iceberg): keep nested child namespaces when loading an Iceberg catalog by @lukekim in #15105
• fix(flight): authenticate before binding sessions, and bound TLS handshakes by @phillipleblanc in #15128
• fix(test): wait for pkd_ctl to load before reading its bootstrap row in the write-back pk_conflict test by @grokspice in #15102
• fix(models): return Spice tool calls in /v1/responses output so clients can replay them by @krinart in #15126
• test(mysql): skip the SQL results cache in the binlog replication end-to-end tests by @grokspice in #15141
• feat(runtime): check runtime.state at startup, and record its state format by @phillipleblanc in #15138
• feat(acceleration): apply changed acceleration settings on restart by @lukekim in #15094
• fix(models): enable Luna chat and NSQL support by @krinart in #15166
• fix(acceleration): read every accelerator partition in a caching point lookup by @phillipleblanc in #15177
• fix(cayenne): run snapshot-directory sweeps when a snapshot releases its file-deletion hold by @sgrebnov in #15179
• feat(telemetry): export runtime logs over OTLP and log failed queries by level by @lukekim in #15176

Spice v2.2.0 is now available! 🚀

Spice v2.2.0 focuses on real-time data, performance, and stability. Cloud Connect links self-hosted runtimes to Spice.ai Cloud for management and observability. MySQL datasets and PostgreSQL catalogs can now stream source changes into accelerated datasets. Debezium sources can send CDC events directly without Kafka. Warm in-memory indexes improve vector and full-text search performance.

Highlights in v2.2.0 include:

• Cloud Connect — link self-hosted runtimes for BYOC (Bring Your Own Cloud) to Spice.ai Cloud for management and observability
• MySQL CDC — MySQL tables now stay in sync in real time using MySQL binlog replication, with no Kafka or Debezium infrastructure and no scheduled refreshes
• PostgreSQL Catalog CDC — accelerate an entire PostgreSQL database in real time with a few lines of configuration and no per-table setup
• Faster Search — vector and full-text search can now serve from an in-memory index by default, with no configuration change

What's New in v2.2.0​

Cloud Connect​

Spice Cloud Connect is a feature of Spice.ai Cloud.

Spice Cloud Connect links self-hosted runtimes for BYOC (Bring Your Own Cloud) to Spice.ai Cloud for management and observability. Run spice cloud link to enroll a standalone runtime.

Try it with the Cloud Connect on a Development Machine recipe.

MySQL Change Data Capture​

The MySQL Data Connector now supports refresh_mode: changes. The runtime loads an initial snapshot of the table. It then applies inserts, updates, and deletes from the MySQL binlog as they commit. The acceleration stays current without scheduled refreshes.

datasets:
  - from: mysql:orders
    name: orders
    params:
      mysql_host: localhost
      mysql_db: mydb
      mysql_user: replicator
      mysql_pass: ${secrets:mysql_pass}
    acceleration:
      engine: cayenne
      refresh_mode: changes

• Failover-safe GTID positions: When the source server has GTID enabled, the runtime tracks the replication position as a GTID set. A GTID position stays valid across a replica promotion, so replication survives a failover to a new primary without a rebuild.

PostgreSQL Catalog CDC​

The PostgreSQL Catalog Connector is now Beta with new Catalog CDC acceleration support at Alpha.

A PostgreSQL catalog can now use refresh_mode: changes. One configuration replicates every table that the include patterns match. Queries then read fresh PostgreSQL data with no per-table setup.

catalogs:
  - from: pg
    name: pg
    include:
      - 'public.*'
    params:
      pg_host: localhost
      pg_db: mydb
      pg_user: postgres
      pg_pass: ${secrets:pg_pass}
    acceleration:
      refresh_mode: changes

Known limitations while catalog CDC acceleration is Alpha: configuration may change; a table dropped and recreated in the source can serve rows captured before it was recreated (#12110); and a durable catalog acceleration can come back empty after a restart (#12729). Feedback is welcome in #11850.

Debezium CDC Without Kafka​

Any Debezium source plugin can now stream change events directly into Spice, with no Kafka bus in between. A dataset with from: cdc:… accepts change events at POST /v1/datasets/{name}/cdc, in JSON or Avro. The existing Kafka path (from: debezium:…) is unchanged.

Faster Search with Warm In-Memory Indexes​

Search adds a warm in-memory tier for vector and full-text indexes. The runtime writes each change to the warm tier and to the durable store together. Queries read the warm tier first and fall back to the durable store. The warm tier covers vector indexes, full-text indexes, .vectors datasets, views, and chunked Elasticsearch vector columns.

More search improvements:

• cosine_distance uses SIMD instructions through the simsimd library.
• Full-text search applies stemming by default.
• Full-text search pushes SQL filters down into the tantivy index.
• A delete now removes the document from full-text and vector indexes. BM25 statistics no longer count superseded documents.
• The runtime loads local rerankers from text-embeddings-inference models.
• The runtime validates vector search parameters before it runs the SQL query.

Operations & Observability​

• Query timeout: The new runtime.query.timeout setting bounds the duration of every query. An expired query fails with HTTP 504 or gRPC DEADLINE_EXCEEDED.

runtime:
  query:
    timeout: 30s

• CPU sizing for Kubernetes pods without a CPU limit: A Spice runtime pod with a CPU request and no CPU limit now sizes itself to twice the request instead of every core on the node. Set SPICE_CPU_CORES=all (or runtime.cpu.cores: all) to burst to all cores, for example when a 0.5-core request runs on a 24-core node.
• Trace IDs in logs: Every log record from a query carries the query's trace ID. Filter the logs by one ID to see everything that query did.
• Improved dashboards: The Grafana and Datadog dashboards add new and improved panels, and support multi-replica and Kubernetes deployments.
• Helm: The Helm chart supports a custom Deployment strategy and StatefulSet updateStrategy.
• Dataset status: A non-accelerated dataset now shows the Error state when its source is unavailable, instead of appearing healthy while queries fail.

Other Notable Improvements and Bug Fixes​

• Drasi (Alpha): The runtime forwards CDC changes and runtime tables to a Drasi source.
• Microsoft SQL Server: Kerberos integrated authentication now works on Unix.
• HTTP connector: OAuth2 client-credentials authentication.
• Oracle DATE: Values lost their time of day. They now map to a timestamp.
• Catalog exclude patterns: The Glue and Cayenne catalogs ignored exclude patterns. Both pattern lists now apply.
• Spice Cayenne schema changes: Partitioned accelerations could report a source schema change as applied while partitions kept the old schema. This mismatch caused lossy casts or append failures. Cayenne now rejects in-place evolution for partitioned accelerations. The configured schema-change policy then handles the change.
• Snowflake NUMBER values: The connector could remove fractional digits during schema discovery. It treated numeric metadata as absent and defaulted the scale to zero. It now preserves the source precision and scale.

See the Changelog for the full list of fixes.

SDK Updates​

Updated SDKs release alongside v2.2.0:

• spice.js v3.2.0 — adds query cancellation (listActiveQueries/cancelActiveQuery), mTLS client certificates, HTTP fallback when Flight is unavailable, and typed parameter binding via Flight SQL prepared statements. Also fixes nsql() and search() response handling and named parameters over HTTP.
• spicepy v4.0.0 — adds streaming of large results, natural-language queries (nsql), vector and hybrid search, and query cancellation. Spice.ai Cloud users need this update: the legacy hostnames are retired in favor of region-specific endpoints.
• spice-rs v4.0.0 — adds search, NSQL, async queries, query cancellation, and mTLS.

Dependency Updates​

New Contributors​

• @boittega made their first contribution in #11509
• @lesbass made their first contribution in #12253

Contributors​

• @bjchambers
• @boittega
• @ewgenius
• @Jeadie
• @krinart
• @lesbass
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov
• @v1gnesh

Breaking Changes​

• CPU sizing for Kubernetes pods without a CPU limit: A Spice runtime pod with a CPU request and no CPU limit now sizes itself to twice the request instead of every core on the node. Set SPICE_CPU_CORES=all (or runtime.cpu.cores: all) to keep the previous behavior, or set runtime.cpu.cores to a specific core count.
• Partitioned DuckDB accelerations are removed: The DuckDB accelerator now rejects partition_by. Use the Cayenne or Arrow accelerator for partitioned datasets.
• Adaptive Cayenne tuning requires an explicit opt-in: An unset cayenne_tuning now resolves to auto. Set cayenne_tuning: adaptive to enable the closed-loop controller.
• ONNX ML inference is removed: The runtime no longer loads ONNX models, and it no longer serves the /v1/predict endpoints. Use an LLM model provider instead.
• One process-wide Vortex segment cache: Cayenne tables now share one segment cache instead of one cache per table. Set cayenne_segment_cache_mb under runtime.params to size it. When unset, the cache takes 1/64 of the memory entitlement, clamped to 256 MiB - 2 GiB.
• The Pingora cache engine is now a Spice.ai Enterprise feature: An OSS build with engine: pingora degrades to the Moka engine and logs the substitution.
• Deprecated settings: pg_replication_temporary_slot is deprecated. cayenne_segment_cache_mb at the table level is deprecated — set it under runtime.params instead. This release renames the MongoDB num_docs_to_infer_schema setting to schema_infer_max_records. The old name still works and warns.

Cookbook Updates​

• Added a new recipe for MySQL CDC (Binlog Replication): stream real-time changes from a MySQL table into Spice with refresh_mode: changes.
• Added a new recipe for Aurora MySQL CDC (Binlog Replication): the same CDC setup for an AWS-hosted Amazon Aurora MySQL cluster.
• Added a new recipe for PostgreSQL Catalog CDC: replicate every table of a PostgreSQL database with one catalog configuration.
• Added a new recipe for Cloud Connect on a Development Machine: connect a local Spice instance to Spice.ai Cloud, deploy live changes, and deliver a secret.

The Spice Cookbook includes more than 104 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.2.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.2.0 image:

docker pull spiceai/spiceai:2.2.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.2.0

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• feat(testoperator): --prepare-only / --skip-prepare for htap source reuse by @bjchambers in #11486
• test(cluster): de-flake scheduler_failover (drop unreliable scheduler_node assertion) by @phillipleblanc in #11518
• feat(cayenne): wire orphaned-DV cleanup knob to spicepod params + doc sync by @bjchambers in #11523
• fix(cayenne): warn (not panic) on benign manifest/listing race; re-enable now-passing ignored tests by @lukekim in #11511
• fix(datafusion): accurate projected scan byte size so hash joins build the smaller side by @sgrebnov in #11503
• fix(cayenne): seed persisted num_rows for hash-join sizing by @sgrebnov in #11515
• fix(cayenne): user-visible DELETE WHERE pk IN (...) reports the real row count by @lukekim in #11514
• test(cayenne): CDC convergence/resurrection fuzz harness by @bjchambers in #11502
• fix(cayenne): correctness & memory_limit fixes from perf audit (2 P0, 3 P1) by @lukekim in #11516
• perf(cayenne): compaction IO-hygiene follow-ups — input-page evict + O_DIRECT writer + bench by @lukekim in #11498
• Update end_game.md by @bjchambers in #11535
• chore: Vendor ttf-parser, lopdf, pdf-extract by @peasee in #11521
• fix: Update tpch benchmark snapshots for federated/mysql[catalog].yaml by @app/github-actions in #11531
• Fix s3 vectors API by @krinart in #11536
• Update from spiceai/datafusion#177 by @Jeadie in #11541
• perf(cayenne): off-write_lock N>1 mem-tier checkpoint + adaptive query-admission throttle by @lukekim in #11538
• fix(flightsql): propagate bearer token to per-endpoint clients by @boittega in #11509
• fix: Placeholder table initialization lock swap by @peasee in #11540
• chore(cluster): bump ballista pin for the null-aware anti-join fix by @phillipleblanc in #11544
• fix(runtime-tools): fix memory table identifier validation rejecting valid names by @Jeadie in #11546
• test(chbench): enable 4-way mem-tier PK sharding on SF1000 memory-durability configurations by @sgrebnov in #11553
• fix(reorder): reorder inner-join islands inside EXISTS / NOT EXISTS by @sgrebnov in #11552
• feat(cayenne): enable orphaned-DV cleanup by default (per-table threshold 20) by @bjchambers in #11533
• fix(postgres): delete old key on CDC primary-key updates by @phillipleblanc in #11551
• Properly exclude vendored crates from make lint-rust-fix by @krinart in #11560
• fix: arrow IndexedMemTable serves stale results after DML/retention/sync (fixes #11262) by @claudespice in #11532
• perf(cayenne): concurrent per-shard mem-tier checkpoint encode (drain parallelism) by @lukekim in #11558
• fix(physical-plan): estimate col=literal selectivity from NDV, not the flat 20% default (Datafusion) by @sgrebnov in #11562
• fix: Read Postgres DB source error when available by @peasee in #11566
• feat(cayenne): cold object-store tier with read-optimized clustering by @lukekim in #11543
• Generic CDC replication-lag metric by @krinart in #11554
• fix: ignore transitive quick-xml advisories (RUSTSEC-2026-0194/0195) in cargo-deny by @krinart in #11571
• test(chbench): add cold-object-store-tier SF-1000 HTAP run; cap SF-1000 mem-tier at 1 GiB by @lukekim in #11573
• fix: surface silent Turso read-path conversion failures (fixes #11276) by @claudespice in #11570
• feat(testoperator): emit P99 + Max replication lag and staleness telemetry for HTAP by @sgrebnov in #11572
• feat(runtime): cost-based eager aggregation, enabled by default (datafusion spiceai-54) by @bjchambers in #11446
• Bump datafusion to include spiceai/datafusion#181 by @Jeadie in #11563
• fix(cayenne): drain in-flight maintenance before in-process reopen (mutation_property flake) by @bjchambers in #11578
• Unified JSON nesting by @krinart in #11555
• feat(testoperator): capture under-load EXPLAIN plans as HTAP artifacts by @sgrebnov in #11576
• feat(cayenne): freshness-SLO-driven adaptive mem-tier shrink (+ freshness diagnostics) by @lukekim in #11574
• fix(cayenne): fold un-checkpointed mem-tier keys into the cold keyset rebuild by @lukekim in #11592
• perf(cayenne): reuse the PK RowConverter across the sharded CDC apply by @lukekim in #11590
• Add labels to spiceai-dev by @krinart in #11591
• Rename mongo num_docs_to_infer_schema (deprecated) -> schema_infer_max_records by @krinart in #11593
• fix(postgres): don't ACK replication slot past uncommitted envelopes on empty transactions by @bjchambers in #11582
• refactor(connectors): extract abfs, adbc, cosmosdb, ducklake, gcs, git, github, glue, kafka, snowflake-native, spiceai into data-connectors crates by @Jeadie in #11407
• Revert "Generic CDC replication-lag metric" by @lukekim in #11600
• refactor(cayenne): remove orphaned-DV cleanup spicepod param, make it always-on by @bjchambers in #11575
• fix(mongodb): substitute delete when UpdateLookup finds no document by @bjchambers in #11583
• ci(chbench): SF1000 tuned+adaptive every 3h, other configs once daily by @lukekim in #11601
• feat(cayenne): add integer support to maintained AVG by @bjchambers in #11564
• Refresh spidapter Spice Cloud token from client credentials by @krinart in #11594
• fix: cluster filtered COUNT(*) on append-only tables returns unfiltered total (fixes #11599) by @claudespice in #11603
• fix(scylladb): error on lossy CQL decimal conversion instead of silent truncation/NULL (fixes #11267) by @claudespice in #11604
• ci(chbench): drop postgres-arrow and tuned (non-memory) from scheduled HTAP runs by @lukekim in #11606
• ci(chbench): lower tuned SF1000 mem-tier max age to 15s for scheduled HTAP runs by @lukekim in #11618
• fix(cayenne): fold the un-checkpointed mem-tier into the persisted-bloom PK-index rebuild (SF-100 over-count) by @lukekim in #11609
• Potential fix for 1 code quality finding by @lukekim in #11614
• fix(postgres): shared replication slot drops WAL for partitioned tables (fixes #11290) by @claudespice in #11607
• bench(chbench): add 4-slot CDC variant pod for SF-1000 memory (separate from default) by @bjchambers in #11605
• fix(cayenne): don't serve drifted memory-CDC row count as Exact to COUNT(*) by @bjchambers in #11602
• chore(deps): vendor pgwire-replication 0.3.2 by @bjchambers in #11617
• Add SQL warehouse ID to integration workflow by @Jeadie in #11561
• fix(search): cosine_distance returns NULL not NaN for zero-magnitude vectors by @claudespice in #11621
• feat(spidapter): tuned + adaptive MongoDB CDC spicepods with scenario selection by @bjchambers in #11579
• Potential fixes for 3 code quality findings by @lukekim in #11613
• Potential fixes for 2 code quality findings by @lukekim in #11611
• Potential fixes for 2 code quality findings by @lukekim in #11615
• Potential fix for 1 code quality finding by @lukekim in #11612
• feat(cayenne): split mem-tier into ingestion + immutable pieces; advance the replication slot on a cheap periodic seal (default 2s) by @lukekim in #11622
• fix(mcp): encode catalog-qualified tool names without '/' (fixes #10894) by @claudespice in #11629
• fix(refresh): apply refresh_sql override on manual refresh when source unchanged (fixes #11353) by @claudespice in #11626
• fix: report empty accelerated schema instead of misleading missing-column error (fixes #10920) by @claudespice in #11625
• fix(imap): store message date as millisecond Timestamp, not seconds in Date64 (fixes #11547) by @claudespice in #11624
• test(bucket): guard frozen partition-hash output against silent ahash drift (addresses #11277) by @claudespice in #11623
• Bump datafusion (spiceai/datafusion#182) and datafusion-table-providers (#27): fix q16 CollectLeft planning error and SQLite q6 wrong revenue by @Jeadie in #11598
• fix(cluster): prevent executor flaps and distributed query stalls by @phillipleblanc in #11565
• fix(cli): allow spice trace on dynamic MCP/custom tool_use tasks (fixes #10995) by @claudespice in #11630
• docs: add clang/lld to arm64 Linux build prerequisites (fixes #11120) by @claudespice in #11632
• perf(cayenne): mem-tier visible-batch memo + maintained MIN/MAX IVM by @lukekim in #11631
• docs: consolidate agent instructions and fix stale references by @lukekim in #11634
• build(deps): bump the github-actions-dependencies group across 2 directories with 7 updates by @app/dependabot in #11627
• perf(flight): offload query-result IPC encoding off the IO runtime by @lukekim in #11608
• feat(runtime): reservation-aware Cayenne CDC query-memory default (75%→70%) by @lukekim in #11641
• Revert "ci(chbench): lower tuned SF1000 mem-tier max age to 15s for scheduled HTAP runs" by @lukekim in #11648
• fix(cayenne): don't bump mem-tier version on seal (QPH regression from #11622) by @lukekim in #11649
• feat(cayenne): vendor versioned row_converter module (byte-compatible with arrow-row 58.3.0) by @phillipleblanc in #11654
• fix(flightsql): advertise analyzed query schemas by @phillipleblanc in #11656
• feat(cayenne): bounded append freshness — segmented stream publishing + publish-triggered executor stats broadcast by @phillipleblanc in #11620
• fix(postgres): decouple replication keepalive/feedback from apply backpressure by @bjchambers in #11653
• fix: bump DataFusion for eager decimal SUM schema by @phillipleblanc in #11657
• fix(cayenne): stop mem-tier snapshot storms under memory pressure and byte-range splitting of small snapshot scans by @sgrebnov in #11645
• fix(ducklake): add opt-in automatic_migration to attach older catalogs (fixes #10899) by @claudespice in #11635
• ci(e2e): tolerate empty-type DuckDB variant of the parquet-rename race by @claudespice in #11636
• build(deps): bump the aws-sdk group with 3 updates by @app/dependabot in #11628
• ci(chbench): FIFO-queue HTAP runs via concurrency queue: max; simplify dispatcher by @bjchambers in #11659
• fix(bucket): guard against ahash AES-path re-bucketing of partition data (fixes #11277) by @claudespice in #11665
• fix(cayenne): gate metastore reinsert_sequence schema behind a user_version (fixes #11291) by @claudespice in #11651
• fix(cayenne): durable CDC delete keeps the pk IN (...) fast path (fixes #11633) by @lukekim in #11642
• ci(chbench): make scheduled HTAP runs adaptive-only (retire tuned, convert cold tier to adaptive) by @lukekim in #11677
• fix(imap): decode text body into content instead of storing raw MIME (fixes #11549) by @claudespice in #11667
• feat(cayenne): end-to-end integrity checksums for WAL records and Vortex data files (fixes #11639) by @claudespice in #11646
• feat(spidapter): date-prefix per-run MongoDB CDC databases by @bjchambers in #11658
• test(cayenne): fuzz mem-tier CDC + seq-prefix bake; harden durable-path count exactness by @bjchambers in #11643
• deps: bump spiceai/datafusion to pick up ORDER BY alias unparser fix by @Jeadie in #11655
• fix(postgres_replication): size CDC change-batch builders to the transaction row count by @sgrebnov in #11675
• Release notes for v2.1.0 by @Jeadie in #11429
• perf(pgwire-replication): incremental zero-copy framing (FrameReader) + bounded message size by @lukekim in #11668
• fix(cdc): memory-mode changes stream no longer fatally stops on a transient deferred-commit queue (fixes #11644) by @bjchambers in #11678
• test: update snapshots from job 84685029883 by @Jeadie in #11569
• fix: Distinguish no filters vs lit(false) for RefreshSql distributed accelerations by @peasee in #11596
• Fix cargo-deny crossbeam-epoch advisory by @Jeadie in #11681
• fix: Update benchmark snapshots for trunk run by @app/github-actions in #11666
• chore(deps): upgrade Vortex to 0.76.0 by @lukekim in #11682
• refactor: extract runtime-metrics crate, move ComponentType to runtime-api-types by @Jeadie in #11524
• feat(cloud-connect): standalone instance adoption by @lukekim in #11060
• docs: keep the feature set constant across a session (incremental-build hygiene) by @bjchambers in #11679
• feat(mysql): binlog replication for refresh_mode: changes by @lukekim in #11672
• feat(cdc): cap keys per durable delete plan + absorb fall-through metrics (#11673) by @bjchambers in #11680
• ci(chbench): run scheduled SF1000-adaptive HTAP bench every 6h instead of 3h by @lukekim in #11698
• fix: MySQL connector fails to convert unsigned integer types (fixes #8364) by @claudespice in #11691
• observability(cdc): localize CDC back-pressure across the ingest pipeline by @bjchambers in #11610
• fix(cache): cache empty (zero-row) SQL result sets by @bjchambers in #11699
• Update spicepod.schema.json by @Jeadie in #11706
• Add support for MysSQL in Ch-Bench by @krinart in #11700
• observability(chbench): limit CH-BenCHmark logs to info and above by @bjchambers in #11711
• perf(cdc): cache per-dataset metric labels to avoid per-event string copies by @bjchambers in #11712
• feat(postgres): pgoutput binary-format CDC decoding + zero-copy decoder by @bjchambers in #11702
• Update integration_models.yml by @Jeadie in #11715
• fix(cayenne): zero-row append refresh fails with "No such file or directory" by @sgrebnov in #11710
• perf(cdc): cut shared-slot Postgres replication pump per-event overhead by @bjchambers in #11709
• fix(spicepod): reject unknown fields on columns[] config (fixes #10972) by @claudespice in #11719
• feat(cayenne): tier-gated O_DIRECT compaction writer (EBS/NAS network storage) by @lukekim in #11696
• fix(cayenne): harden Vortex compaction writes by @phillipleblanc in #11692
• perf(vortex): push boolean NOT filters into the Vortex scan by @lukekim in #11721
• chore(chbench): cap scheduled HTAP runs at 250K tpmC (rate 9250) by @sgrebnov in #11723
• MySQL CH-benCHmark HTAP: SF1 + SF1000 adaptive configs by @krinart in #11722
• feat(testoperator): TPC-DS SF100 federated cluster-bench spicepod + default spicehq nodegroup by @phillipleblanc in #11716
• refactor(runtime): move AccelerationSource and Acceleration into runtime-acceleration by @Jeadie in #11530
• remove(ml): delete model_components crate and ONNX/Tract ML inference by @Jeadie in #11684
• ci: dedupe testoperator query_overrides run steps by @Jeadie in #11690
• Update search snapshots by @Jeadie in #11714
• refactor: use inherent is::<T>() on dyn TableProvider/dyn ExecutionPlan for type checks by @Jeadie in #11585
• chore: remove unused BlueOak license allowance by @phillipleblanc in #11762
• perf(vortex): remove per-file and per-segment allocations on the scan path by @lukekim in #11733
• fix(ci): use GraphQL API to check isLatest in spiced_docker workflow by @Jeadie in #11735
• fix(release): use release notes heading as GitHub release title by @Jeadie in #11744
• feat(chbench): headline run summary + adaptive slot-topology variants by @bjchambers in #11734
• fix(ci): update e2e CLI tests for Windows-no-runtime and chat error wording by @Jeadie in #11717
• build(rust): upgrade toolchain to 1.96.1 by @lukekim in #11743
• fix(deps): enable reqwest "query" feature for isolated crate builds by @claudespice in #11760
• build: define clippy lints in [workspace.lints], opt in every crate by @bjchambers in #11676
• refactor(runtime): parameterize table-provider unwrapping by inner-fn set by @Jeadie in #11759
• fix(cayenne): retain source deletion index in scan memos to close ABA hazard (fixes #11303) by @claudespice in #11739
• fix(mysql): refuse binlog resume across source layout drift by @phillipleblanc in #11761
• perf: avoid unnecessary allocations on search hot paths by @phillipleblanc in #11767
• feat(spidapter): support cluster_name for private nodegroup targeting by @phillipleblanc in #11740
• feat(cayenne): finalize datalake (cold) tier v1 UX, credentials, and e2e integration test by @sgrebnov in #11731
• perf(vortex): push CAST(CASE ... END) filters into the Vortex scan by @lukekim in #11729
• docs: lead README with real-time analytics-node + CDC messaging (Spice 2.0) by @lukekim in #11774
• chore(chbench): extend scheduled HTAP runs to 15m by @lukekim in #11773
• perf(cayenne): lazy NDV — compute on file spill, off the inline hot loop by @bjchambers in #11741
• ci(rust): install 1.96.1 in workflows that pin the toolchain explicitly by @lukekim in #11769
• perf(cdc): deferred parsing — move Postgres CDC decode + build off the shared pump by @bjchambers in #11732
• docs: correct sort-kernel SIMD note (AVX2, not AVX-512) by @lukekim in #11771
• fix(cayenne): roll cold-tier files at cayenne_cold_target_file_size_mb by @sgrebnov in #11746
• docs(cayenne): move the Cayenne technical reference into the repo by @bjchambers in #11775
• perf(cayenne): unify upsert PK-key hashing on prehashed XXH3-128 digests by @bjchambers in #11736
• perf: avoid unnecessary allocations on query hot paths by @phillipleblanc in #11768
• Extended schema inference for mysql by @krinart in #11742
• chore(hash-index): add SBBF probe/insert baseline benchmark by @lukekim in #11770
• feat(cayenne): incremental datalake promotion, per-file PK blooms, physical GC by @sgrebnov in #11745
• test(cayenne): limit property-test concurrency by @phillipleblanc in #11784
• perf(cdc): lock-free per-member ack slots + coalescing for shared Postgres replication by @bjchambers in #11779
• perf(chbench): speed up MySQL and Postgres seed loading by @bjchambers in #11782
• fix(ci): download MinIO client from public-data mirror by @phillipleblanc in #11790
• ci: developer sign-off attestation gating the merge queue by @lukekim in #11776
• fix(tests): synchronize chat expect prompts correctly by @phillipleblanc in #11802
• perf(chbench): default adaptive pod to 4-slot grouping + deep prefetch by @bjchambers in #11799
• refactor(cli): replace open crate with in-tree system-open by @phillipleblanc in #11791
• refactor(runtime): drop direct rustls-pemfile dependency by @phillipleblanc in #11792
• Cache ListingTable file statistics to avoid per-query footer re-parse by @phillipleblanc in #11793
• ci: inherit sign-off across clean base merges by @phillipleblanc in #11804
• perf(cayenne): hash PK once and skip OwnedRow clone on present path by @bjchambers in #11805
• fix(postgres catalog): register partitioned parent, not child partitions by @bjchambers in #11798
• fix(postgres catalog): quote foreign-key target identifiers by @bjchambers in #11796
• refactor(cayenne): rename remaining cold_* params to cayenne_datalake_* by @sgrebnov in #11795
• build: add release-profiling Cargo profile for CPU profiling by @bjchambers in #11807
• fix(ci): make CH-benCH template restore robust to leftover replication slots by @sgrebnov in #11811
• perf(cayenne): remove eager-NDV escape hatch, speed up per-value NDV hashing by @bjchambers in #11806
• Auto-refresh the Attestation check after signoff by @bjchambers in #11815
• refactor(duckdb): remove partitioned DuckDB accelerator modes by @lukekim in #11808
• feat(testoperator): add option to skip the HTAP analytic gate by @bjchambers in #11810
• fix: prevent executor S3 region poisoning before object-store bind by @phillipleblanc in #11766
• perf(cayenne): reuse per-batch scratch allocations in KeyBasedDeletionFilterStream by @bjchambers in #11817
• Update openapi.json by @app/github-actions in #11816
• build(deps): bump the github-actions-dependencies group across 2 directories with 5 updates by @app/dependabot in #11819
• fix(task_history): capture ExplainAnalyze metrics from the executed plan by @phillipleblanc in #11794
• feat: generic spiced env-var passthrough for HTAP testoperator runs by @bjchambers in #11824
• feat(cayenne): datalake tier hardening + row-capped PK-bloom-backed promotion by @sgrebnov in #11812
• chore: release finalization for v2.2.0 by @Jeadie in #11738
• feat(cayenne): add memory mode (mode: memory) — fully in-RAM accelerator by @lukekim in #11720
• Fix secrets in .github/workflows/integration_models.yml. by @Jeadie in #11781
• feat(runtime): add runtime.query.timeout parameter by @sgrebnov in #11822
• feat(cloud-connect): align proto to canonical, fix enrollment ordering, add TLS e2e by @lukekim in #11829
• docs(criteria): mark PostgreSQL Catalog Connector as Alpha by @bjchambers in #11785
• test(cayenne): add regression tests for the mid-pass overwrite guard by @sgrebnov in #11844
• perf(cayenne): light delta encoding + higher CDC coalescing defaults by @bjchambers in #11826
• ci: run Cayenne doc PDF build only on trunk merges by @bjchambers in #11860
• fix(cayenne): drain staged Stage-B publishes before cold promotion; add cold-tier fuzz coverage by @sgrebnov in #11847
• feat(cayenne): atomic cross-partition append + delete/on-conflict atomicity by @lukekim in #11803
• feat(cayenne): min/max maintained aggregates + N>1 CDC retract by @lukekim in #11862
• perf(chbench): CSV-based seed loading for MySQL and Postgres by @bjchambers in #11843
• fix(tests): re-enable prop_concurrent_cold_sqlite cayenne cold-tier fuzz test by @sgrebnov in #11871
• fix(deps): replace yanked spin releases by @phillipleblanc in #11878
• fix(cayenne): correct stale mem_checkpoint_lock comment by @bjchambers in #11872
• Add run links to testoperator_dispatch.yml by @Jeadie in #11747
• Cayenne serializable transactions: gated writes, per-key OCC, multi-table, FlightSQL, durable write-back by @phillipleblanc in #11870
• fix(cayenne): partitioned datasets deadlock against the global encode budget and never become ready by @Jeadie in #11825
• test(runtime): re-runnable turso file cleanup + bump turso to 0.7.0 by @lukekim in #11783
• Use liteparse for PDF document parsing by @Jeadie in #11522
• ci: add remote signoff workflow by @Jeadie in #11864
• fix(cli): populate org in spice cloud apps --output json (fixes #11041) by @claudespice in #11867
• fix(embeddings): restore params broken by #10853 by @Jeadie in #11788
• Move cargo advisory checks to scheduled workflow by @Jeadie in #11879
• perf(cayenne): bound cold-promotion Z-order sort into streaming byte-capped runs by @sgrebnov in #11890
• fix(cayenne): write-back transaction atomicity — stage (not publish) and read mem-tier rows by @phillipleblanc in #11889
• fix(runtime-table-partition): restore sound modulo inequality partition pruning by @Jeadie in #11891
• Replace dotenvy with in-repo dotenv crate by @phillipleblanc in #11894
• Remove unnecessary allocations from results-cache and hot conversion paths by @phillipleblanc in #11895
• Increase ready_wait timeout for chbench sf1000 cold-tier configs by @sgrebnov in #11900
• fix(mysql): retry binlog checkpoint upsert on transient accelerator write lock by @krinart in #11876
• fix(cayenne): gate freshness mem-tier shrink on apply backlog by @lukekim in #11893
• fix(postgres catalog): honor unsupported_type_action for catalog-discovered tables by @bjchambers in #11875
• fix(postgres): make TPC-H/TPC-DS benchmark CI actually run at SF1/SF10/SF100 by @bjchambers in #11883
• feat(search): writethrough compound SearchIndex/VectorIndex with optional fallback by @Jeadie in #11892
• Revert "perf(cayenne): light delta encoding + higher CDC coalescing defaults" by @bjchambers in #11910
• perf(vss): SIMD-accelerate cosine_distance via simsimd by @Jeadie in #11748
• Add run-name to signoff.yml by @Jeadie in #11912
• fix(cayenne): serialize cold-tier mem-tier checkpoint by @sgrebnov in #11907
• fix(postgres catalog): discover materialized views and foreign tables by @bjchambers in #11874
• test(chbench): stronger content fingerprint + MySQL sidecar template caching by @krinart in #11901
• test(chbench): align money columns to canonical DECIMAL schema (CMU BenchBase) by @sgrebnov in #11921
• fix(cayenne): close per-key OCC missed-conflict holes and fused-txn IVM staleness by @lukekim in #11916
• ci(attestation): fast-track pure reverts past developer sign-off by @bjchambers in #11913
• fix(cloud-connect): accept the portal's 5-char adoption-code segments by @phillipleblanc in #11926
• docs(layering): codify crate tiers + workspace layering guard by @bjchambers in #11919
• refactor(cayenne): datalake tiering UX — param rename + tracing by @sgrebnov in #11920
• feat(params): typed component params via #[derive(TypedParams)] — embeddings pilot by @Jeadie in #11809
• Fix SQlite round type error. by @Jeadie in #11927
• fix(signoff): let remote sign-off refresh the Attestation check in CI by @bjchambers in #11929
• feat(testoperator): print rows around CH-benCH analytical-gate mismatches by @bjchambers in #11923
• fix(postgres catalog): don't abort the whole catalog on one schema's discovery failure by @bjchambers in #11873
• ci(signoff): target-lint changed crates before full local/remote gate by @lukekim in #11909
• feat(runtime)!: always-on schema inference; remove schema_inference config by @lukekim in #11880
• refactor(layering): extract ClickHouse into connector-clickhouse; add restricted_deps guard by @bjchambers in #11931
• perf: remove unnecessary allocations on hot query, search, and metrics paths by @phillipleblanc in #11918
• Release notes for v2.1.1 by @Jeadie in #11906
• refactor(layering): prepare DynamoDB for extraction from runtime by @bjchambers in #11935
• Coalesce Null primary keys in RRF. by @Jeadie in #11519
• ci(e2e): strip ANSI before duckdb_append graceful-shutdown log whitelist by @phillipleblanc in #11937
• feat(catalog): PostgreSQL catalog-level CDC acceleration (changes mode) by @bjchambers in #11897
• chbench(mysql): parallel analytical gate, decimal-comparison fix, and reseed/CI hardening by @krinart in #11946
• fix: array_any_value panic when output is hash-repartitioned (empty list elements) by @bjchambers in #11952
• feat(layering): checkpoint sidecar as per-engine crates + DI the dynamodb connector by @bjchambers in #11938
• chore(cluster): pin Ballista to upstream 54 merge tip by @phillipleblanc in #11941
• feat(vortex): upgrade fork pins to Vortex 0.79.0 by @lukekim in #11950
• chbench(htap): run mysql configs at the shared 9250 rate; 2x converge wait by @sgrebnov in #11956
• fix(testoperator): HTAP drain gate false-fails on stale probe observations — final snapshot decides convergence (fixes #11953) by @claudespice in #11966
• feat(catalog): replica-identity-aware eligibility for PostgreSQL catalog CDC by @bjchambers in #11951
• refactor(layering): extract DynamoDB into connector-dynamodb by @bjchambers in #11960
• Add MemoryVectorIndex: in-memory external-store VectorIndex with brute-force exact k-NN by @Jeadie in #11908
• ci(signoff): prefer lab SSH for remote sign-off; skip Rust when no .rs changes by @lukekim in #11977
• Reapply #11826: light delta encoding + CDC coalescing (SF1000 convergence lever) — gated on #11943 + SF1000 fingerprint run by @lukekim in #11944
• docs: align component statuses with signed release criteria by @lukekim in #11970
• fix: Spidapter flight auth passthrough by @peasee in #11976
• Unified CDC config by @krinart in #11777
• feat(cdc): Debezium plugin push ingest to Spice without Kafka by @lukekim in #11955
• refactor(connectors): unify registration on the linkme slice; restore schema coverage by @bjchambers in #11972
• GTID-based MySQL CDC by @krinart in #11813
• test(mysql): ignore mysql_binlog_replication_end_to_end_cayenne by @sgrebnov in #11986
• fix(cayenne): support Decimal128 in maintained SUM/AVG aggregates (fixes #11933) by @claudespice in #11979
• feat(params): typed params for VectorStore and FtsStore engines by @Jeadie in #11954
• chore(layering): remove orphaned dead code from connector extractions by @bjchambers in #11988
• feat(cayenne): default-on adaptive cold layout from observed filters (F4) by @lukekim in #11973
• fix: Fairly allocate partitions from scheduler assignment cycle only by @peasee in #11853
• fix(cli): avoid escaped ANSI version notification by @ewgenius in #11996
• MySQL Shared binlog connection by @krinart in #11814
• feat(catalog): fail-loud + metrics + by-kind reporting for PostgreSQL catalog CDC (#11850) by @bjchambers in #11983
• fix(cayenne): P1 audit — subset compact, SMJ 2.5× HT, mem-tier pool account by @lukekim in #11991
• Add OAuth2 client-credentials grant and configurable auth header to HTTP connector by @krinart in #11981
• fix(telemetry): validate runtime.telemetry.metric_prefix against OTel name syntax by @ewgenius in #12002
• fix(cayenne): purge CDC mem-tier on delete-all/TRUNCATE by @sgrebnov in #12009
• Update signoff.yml. by @Jeadie in #11998
• feat(dev): add scripts/signoff mine — attestation status across your open PRs by @Jeadie in #11978
• refactor(layering): extract MongoDB provider into connector-mongodb by @bjchambers in #11982
• perf: remove three unnecessary allocations on the query hot path by @phillipleblanc in #12029
• refactor(layering): evacuate cosmosdb/graphql/github/sharepoint providers into their connector crates by @bjchambers in #12024
• fix(mysql-cdc): detect source reset on GTID resume by @sgrebnov in #12023
• fix(smb): exclude final SESSION_SETUP response from preauth integrity hash (fixes #11148) by @claudespice in #12003
• fix(postgres): validate pg_replication_slot names before CDC refresh by @ewgenius in #12001
• fix(postgres): honor pg_connection_string for CDC (closes #11994) by @ewgenius in #12000
• feat: Support OTLP Histogram ingest, unix nanos time by @peasee in #11992
• feat(search): write-through warm + fallback compound index for FTS (#11886) by @Jeadie in #11971
• docs: PR-description and code-comment conventions by @bjchambers in #12032
• refactor(postgres cdc): consolidate every dataset onto the shared pump by @bjchambers in #12028
• feat(cloud-connect): Standalone instance adoption connection by @peasee in #11980
• fix(cdc): don't force the durable path on zero-row readiness heartbeats (fixes #12007) by @claudespice in #12030
• Upgrade DataFusion to 54.1.0 by @krinart in #11974
• perf(github): parallelize serial fetches (commits + workflow logs) by @lukekim in #12017
• refactor(layering): evacuate odbc/scylladb/imap/git providers into their connector crates by @bjchambers in #11993
• refactor(layering): carve runtime-component out of runtime, break the ArcRuntime cycle by @bjchambers in #12031
• feat(mssql): enable tiberius integrated-auth-gssapi for Kerberos integrated auth on Unix by @v1gnesh in #11386
• build(deps): bump the aws-sdk group across 1 directory with 5 updates by @app/dependabot in #11821
• fix(cayenne): clear a committed CDC upsert's staging WAL at finalize (#12027) by @bjchambers in #12034
• feat(catalog): deterministic instance-independent replication slot + fail-loud + restart recovery (#11850) by @bjchambers in #12026
• Auto-cap DuckDB accelerator memory to prevent startup over-commit by @lukekim in #11985
• perf(refresh): speed up dataset refresh at startup by @lukekim in #12015
• fix(cayenne): don't create stray file: directory in memory mode (fixes #11922) by @claudespice in #11940
• fix(aws): accept aws_session_token for temporary credentials (fixes #10932) by @claudespice in #12041
• fix(mysql): verify an adopted layout against the event's own column types (fixes #11764) by @claudespice in #12048
• fix(mysql): take the binlog rotate target from the ROTATE event, not its header offset (fixes #12042) by @claudespice in #12044
• fix(search): check dataset readiness before embedding the query (fixes #10956) by @claudespice in #12043
• fix(databricks): pin the authentication mode with databricks_auth_mode so an auto-loaded client secret can't switch U2M to M2M (fixes #11508) by @claudespice in #12040
• fix(imap): narrow scans with IMAP SEARCH instead of refetching the mailbox (fixes #11548) by @claudespice in #12039
• fix(cayenne): rank inference-derived sort columns below observed filter columns by @lukekim in #12049
• perf(cayenne): warm-subset compaction + mem-tier memory_limit honesty by @lukekim in #12035
• fix(cayenne): purge the mem-tier on delete-all for a table without a primary key (fixes #12072) by @claudespice in #12073
• fix(accelerator): let an index declare a finalize failure fatal so a stale index can't report a successful refresh (fixes #12038) by @claudespice in #12050
• fix(http): restore a join's embedded projection in HTTP subquery pushdown (fixes #11009) by @claudespice in #12054
• fix(e2e): report a REPL that exits mid-script instead of passing the step (fixes #12057) by @claudespice in #12059
• fix(imap): fetch the raw message only when a scan reads content (fixes #12045) by @claudespice in #12060
• chore: Update Turso crate to 0.7.1 by @claudespice in #12062
• feat(chbench): in-memory _bench_ts watermarks for the MySQL HTAP staleness probe by @sgrebnov in #12055
• test(chbench): default MySQL adaptive spicepods to one shared binlog dump by @sgrebnov in #12076
• feat(search): warm in-memory writethrough/fallback index for .vectors datasets and views by @Jeadie in #11914
• fix(search): expunge superseded documents from full-text BM25 statistics (fixes #12053) by @claudespice in #12056
• fix(write-back): gate durable write-back on a safe source delivery primitive (fixes #11915) by @claudespice in #12051
• fix(cayenne): keep protected snapshots a position-delete rewrite never folded in (fixes #11477) by @claudespice in #12052
• ci(signoff): fast-track Attestation for Dependabot and non-Rust PRs, and cut the local gate by @lukekim in #12081
• fix(search): reach a compound's inner full-text tier when a change stream attaches (fixes #12061) by @claudespice in #12063
• fix(accelerator): report PK equality pushdown Inexact so a point lookup can't return the whole table (fixes #12070) by @claudespice in #12071
• fix(github): paginate issues and pull_requests on an immutable sort key so a row touched mid-scan isn't dropped (fixes #12067) by @claudespice in #12069
• fix(otel): scope reserved metric column names to the data-point shape (fixes #12064) by @claudespice in #12065
• fix(cayenne): sample the subset compaction append fence before the listing it guards (fixes #12074) by @claudespice in #12075
• fix(search): drop a search hit whose row is not in the base table (fixes #12089) by @claudespice in #12094
• feat(datasets): mark non-accelerated datasets Error when their source is unavailable by @krinart in #12079
• fix(models): restore spice.ai/spiceai as a model source by @lukekim in #12092
• fix(cayenne): bound the cold PK-index rebuild; build shard views in one pass by @sgrebnov in #12078
• fix: Update Search integration test snapshots by @Jeadie in #12077
• feat(catalog): view warnings, docs-linked messages, and clearer startup summary for PostgreSQL catalog CDC (#11850) by @bjchambers in #12022
• fix(postgres): log the cumulative delivery wait in the sink-stall warning by @sgrebnov in #12093
• refactor(cloud-connect): single-source the sealed-secret wire crypto in a shared crate by @peasee in #12124
• build(deps): bump quinn-proto from 0.11.14 to 0.11.16 by @app/dependabot in #12037
• perf(cayenne): demand-driven scan-view cache with access-based freshness (alternative to #11948) by @bjchambers in #12005
• build: exclude libnfs from the nextest workspace run by @phillipleblanc in #12131
• build(deps): bump the aws-sdk group across 1 directory with 3 updates by @app/dependabot in #12068
• fix(search): keep CDC changes streams working when embeddings wrap a CDC source by @Jeadie in #12086
• fix(cli): report a truncated release download as a failed download (fixes #12120) by @claudespice in #12121
• fix(postgres): honor inline PEM pg_sslrootcert on the replication path by @phillipleblanc in #12128
• refactor(layering): extract RuntimeStatus into a runtime-status crate by @bjchambers in #12115
• fix(libnfs): model AUTH as an opaque type so its layout assertion holds by @phillipleblanc in #12132
• feat(duckdb): on_full_refresh: replace_file — full refresh into a new database file, atomically replaced by @lukekim in #12135
• Add v2.1.2 release notes by @sgrebnov in #12146
• feat(duckdb): 'on_full_refresh: checkpoint_file' to bound acceleration file growth by @sgrebnov in #12139
• fix(lint): remove unused DuckDBTableWriter import in duckdb accelerator tests by @sgrebnov in #12158
• fix(duckdb): drop a file-replacement test assertion that can never hold by @bjchambers in #12178
• fix(cayenne): Remove redundant EmptyExec within UnionExec used by CayenneTableProvider::scan by @Jeadie in #12126
• build(deps): bump async-openai for aggregated rate-limit logging by @phillipleblanc in #12149
• perf(mysql): fast row-image decoder for CDC change builds by @sgrebnov in #12122
• feat(catalog): pre-flight replication-slot-capacity check for PostgreSQL catalog CDC (#11850) by @bjchambers in #12114
• fix(oracle): map DATE to a timestamp so the time-of-day is not silently truncated (fixes #12096) by @claudespice in #12097
• fix(search): skip the warm vector index when nothing hydrates it (fixes #12101) by @claudespice in #12103
• refactor(layering): extract the DataAccelerator contract into data-accelerator-api by @bjchambers in #12099
• fix(cli): surface the registry error behind a failed spice add instead of blaming the archive (fixes #12116) by @claudespice in #12119
• fix(ci): gate Rust checks on the lint config the gate itself reads (fixes #12111) by @claudespice in #12112
• fix: Ensure executors report table statistics for distributed plan ordering by @peasee in #11854
• fix: EMBED_UDF_NAME without models feature by @krinart in #12136
• fix(openai): openai_responses_tools: web_search uses web_search, not web_search_preview by @Jeadie in #12142
• fix(ci): cancel workflow runs left behind by superseded merge-queue batches (fixes #12170) by @claudespice in #12177
• fix(runtime): return from a readiness wait when the runtime shuts down (fixes #12125) by @claudespice in #12165
• perf(postgres): coalesce shared-slot CDC envelopes so a slow sink stops throttling the walsender by @bjchambers in #12147
• fix(ci): make cargo_deny_advisories.yml parseable so the Rust advisory scan can run (fixes #12181) by @claudespice in #12182
• fix(cluster): run GetTaskHistory under read-only SQL validation by @lukekim in #12174
• fix(duckdb): count accelerated views in the coordinated memory budget (fixes #12123) by @claudespice in #12161
• fix(ci): run the PR hygiene check on a GitHub-hosted runner by @claudespice in #12163
• fix(cayenne): warm PK existence caches at first write; bound keyset memory during bulk load by @sgrebnov in #12133
• ci: remote sign-off dispatches GitHub Actions only, never SSH to lab hosts by @lukekim in #12205
• Standalone instances: spice connect enroll-only split + connect surface by @krinart in #12143
• fix(search): Fix ordering to correct NDCG@k by @Jeadie in #12191
• refactor(layering): carry component configuration into connectors and extract data-connector-api by @bjchambers in #12157
• fix(logging): mute the candle embedding backend's GeLU notice below -vv by @phillipleblanc in #12194
• test: redact connection context in federated explain snapshots by @bjchambers in #11800
• Enable deletes for Index, SearchIndex, and VectorIndex by @Jeadie in #11961
• feat(testoperator): trace probe latency breaches and percentiles under HTAP load by @sgrebnov in #12209
• Add v2.1.1 and 2.1.2 as supported in SECURITY.md by @sgrebnov in #12190
• fix(ci): seed the TPC-H Postgres benches from the fleet's dataset and schedule the catalog schema tests by @bjchambers in #12214
• fix(cluster): restrict ExpandSecret to spicepod-referenced keys by @lukekim in #12156
• test: redact endpoint-URL compute contexts in federated explain snapshots by @bjchambers in #12215
• fix(testoperator): a fully-censored table floors worst-P99 at the discard cap by @lukekim in #12196
• ci(e2e): tolerate any garbage type token in the duckdb_append parquet-rename race by @bjchambers in #12207
• refactor(cloud-connect): update the proto contract for evolvability and versioning by @peasee in #12153
• chore(cloud-connect): use port 443 for gateway addresses in the doc example and test fixtures by @phillipleblanc in #12252
• fix(ci): authenticate the management API integration suite with secrets that exist (fixes #12184) by @claudespice in #12185
• feat(search): add Recall@k, MRR@k, and Precision@k retrieval metrics by @Jeadie in #12193
• fix(search): enable stemming by default for full-text search by @Jeadie in #12220
• fix: Report query metrics when task history is disabled by @sgrebnov in #12227
• fix(runtime): count accelerated views when gating the Cayenne compaction pool (fixes #12164) by @claudespice in #12171
• fix(secrets): snapshot the store registry instead of holding its lock across awaits (fixes #12127) by @claudespice in #12166
• fix(search): address Elasticsearch deletes by document _id (fixes #12267) by @claudespice in #12273
• fix(postgres): deprecate pg_replication_temporary_slot, which could never work (fixes #12213) by @claudespice in #12265
• fix(cayenne): always materialize a snapshot directory the catalog may reference (fixes #12208) by @claudespice in #12262
• fix(duckdb): size the connection pool for accelerated views too (fixes #12160) by @claudespice in #12169
• fix(ci): link brew formulas that are installed but unlinked in setup-cc by @claudespice in #12286
• fix(runtime): size memory budgets from the process's own cgroup limit by @lukekim in #12263
• Harden dataset availability checks against idle connection resets by @krinart in #12180
• test(mysql): harden binlog CDC with dump-reconnect and full type-matrix coverage by @sgrebnov in #12264
• fix: send enc_pubkey_pem on /renew for standalone runtime by @phillipleblanc in #12312
• Connection-scale client modes for testoperator throughput and load tests by @lukekim in #12280
• fix(search): remove all chunk documents when a chunked Elasticsearch index deletes a row (fixes #12088) by @claudespice in #12268
• fix(duckdb): run spice_sys DuckDB sidecar writes on the blocking pool (fixes #12175) by @claudespice in #12204
• fix(cayenne): the write-concurrency raise must respect the memory brake by @lukekim in #12317
• feat(runtime): expose the memory numbers that explain an OOM as gauges by @lukekim in #12195
• fix(flightsql): mark stamped statistics inexact when a LIMIT is pushed to the remote scan (fixes #12292) by @claudespice in #12293
• fix(deps): bump wasmtime to 47.0.3 to clear RUSTSEC-2026-0222 (fixes #12295) by @claudespice in #12298
• fix(telemetry): read the cgroup CPU quota along the whole cgroup path (fixes #12299) by @claudespice in #12300
• refactor(layering): move the DataFusion helpers below runtime and derive the federation deny-list from a function registry by @bjchambers in #12210
• fix(search): reject a persisted full-text index whose schema no longer matches the configuration (fixes #12274) by @claudespice in #12275
• fix(spiced): keep dependency logging and chat progress alive when task history is off (fixes #12279) by @claudespice in #12281
• fix(runtime): count a mid-stream response failure as a failure in the HTTP metrics (fixes #12284) by @claudespice in #12291
• fix(cluster): mark a coordinator leaf scan's cached executor statistics inexact (fixes #12303) by @claudespice in #12304
• fix(runtime): report a memory-pool refusal as ResourcesExhausted and answer it with 503 (fixes #12282) by @claudespice in #12289
• fix(cayenne): enforce the sharded PK keyset byte budget; correct per-entry accounting by @lukekim in #12192
• fix(runtime): carve the Cayenne compaction memory pool only when a dataset can compact into it (fixes #12320) by @claudespice in #12326
• feat(spiced): report fatal signals before exit by @sgrebnov in #12334
• bug: Filters on _match fail during planning by @Jeadie in #12247
• bug: S3 metadata-filter conversion silently drops unconvertible AND/OR operands by @Jeadie in #12248
• fix(ci): reject a failed sign-off on the head commit (fixes #12357) by @claudespice in #12362
• ci: let signoff.yml sign off fork PRs by @Jeadie in #12025
• fix(search): resolve an append stream through a vector scan (fixes #12313) by @claudespice in #12314
• docs: Spice v2.1.3 release notes by @sgrebnov in #12378
• bug: MemoryVectorIndex leaves external-store entries on delete. by @Jeadie in #12246
• fix(cluster): keep the executor's cluster-service channel alive while idle (fixes #12301) by @claudespice in #12302
• fix(https): reject OAuth2 params on structured HTTP file datasets (fixes #12315) by @claudespice in #12321
• fix(scheduler): drive the interval-timing tests on a virtual clock (fixes #12323) by @claudespice in #12329
• fix(ci): do not report a shutdown-cancelled sidecar task as a failure (fixes #12322) by @claudespice in #12331
• fix(logging): let log colour follow the output sink (fixes #12327) by @claudespice in #12335
• fix(cayenne): report the auto-tuned config once per resolution (fixes #12330) by @claudespice in #12341
• chore: Update Turso crate to 0.7.2 by @claudespice in #12344
• fix(runtime): stop retrying a dataset configuration failure that no retry can clear (fixes #12339) by @claudespice in #12345
• build(deps): bump the github-actions-dependencies group across 3 directories with 9 updates by @app/dependabot in #12359
• feat(llms): GLM 5.2 across 3+ nodes, context_length + paged_attention params, MXFP4 for DeepSeek-V4 by @lukekim in #11990
• feat(runtime): size every CPU-derived pool from the CPU entitlement by @bjchambers in #12276
• chore(ci): bump the spiceio setup action to v0.5.9 by @lukekim in #12391
• feat(testoperator): serve /health and /v1/ready for the HTAP run by @lukekim in #12373
• feat(catalog): durable storage modes for PostgreSQL catalog CDC, and let the catalog own its accelerated tables by @bjchambers in #12222
• fix(runtime): correctly classify a memory-pool refusal as ResourcesExhausted (fixes #12380) by @sgrebnov in #12382
• fix(ci): budget the sign-off run below the pool's job wall (fixes #12340) by @claudespice in #12343
• fix(bench): install make before the seeded-database check; surface catalog error causes by @bjchambers in #12372
• Clean up SQLite sidecar files in cold bloom catalog test by @sgrebnov in #12404
• fix(ci): name an incoherent Cargo.lock before every cargo job fails on it (fixes #12375) by @grokspice in #12423
• test(search): exercise the S3 Vectors warm-tier fallback path by @Jeadie in #12197
• fix(ci): stop the spiced child E2E cleanup was leaking, scoped to this runner (fixes #12058) by @grokspice in #12431
• perf(cayenne): give refresh_mode full its own write profile by @lukekim in #12338
• fix(ci): skip the Attestation refresh when the sign-off SHA is no longer the PR head (fixes #12360) by @grokspice in #12433
• fix(monitoring): correct p99 queries in the Datadog dashboard by @sgrebnov in #12444
• fix(ci): tell an out-of-disk sign-off apart from a failing branch (fixes #12412) by @grokspice in #12426
• feat(search): warm in-memory tier for chunked Elasticsearch vector columns by @Jeadie in #12082
• Update spicepod.schema.json by @app/github-actions in #12451
• fix(search): preserve field names with capitals and '.' by @Jeadie in #12389
• fix(ci): size the cayenne property-test timeout ceiling to its measured runtime (fixes #12336) by @grokspice in #12438
• fix(models): pass the pinned revision to a HuggingFace embedding model (fixes #12430) by @grokspice in #12446
• fix(ci): make the out-of-disk sign-off verdict survive being out of disk (fixes #12427) by @grokspice in #12461
• fix(ci): stop Enforce Pulls with Spice racing itself, and let a Dependabot PR pass it (fixes #12377) by @grokspice in #12466
• docs: update v2.1.3 release notes for the Iceberg timestamptz fix (trunk sync) by @phillipleblanc in #12459
• fix(ci): install protoc before building the spice CLI in the E2E CLI workflow by @phillipleblanc in #12449
• fix(ci): reject an if: naming a context GitHub does not provide there (refs #12396) by @grokspice in #12468
• fix(monitoring): align Datadog dashboard with OTel service instance identity by @ewgenius in #12454
• chore(ci): bump the spiceio setup action to v0.5.10 by @lukekim in #12490
• test(cayenne): query result correctness vs standalone engines and Spice accelerators by @lukekim in #12098
• fix(elasticsearch): keep the response body out of errors on the row-data request paths (fixes #12409) by @grokspice in #12456
• fix(ci): key the sign-off concurrency group on the commit, not the dispatch input (fixes #12472) by @grokspice in #12474
• perf(ci): run the workspace unit-test gate in one cargo invocation by @bjchambers in #12437
• feat(monitoring): Runtime Resources observability on Grafana dashboard by @ewgenius in #12441
• fix(ci): report cache health on the build failures it explains (refs #12420) by @grokspice in #12471
• fix: resolve fixed-offset timezones when writing Vortex files by @phillipleblanc in #12463
• fix(search): stop re-parsing a dotted index key as relation.column (fixes #12462) by @grokspice in #12464
• fix(ci): build the ADBC BigQuery driver with the Go version its go.mod requires by @grokspice in #12470
• fix(llms): keep the Hugging Face cache token and harden the model E2E job by @grokspice in #12422
• fix(ci): stop a cancelled Remote Sign-off from claiming the checks failed (fixes #12424) by @grokspice in #12425
• fix(runtime): fail the write when an index cannot prepare its write window (fixes #12421) by @grokspice in #12448
• fix(runtime): stop retrying a permanent configuration failure on the catalog and dataset load paths (fixes #12417) by @grokspice in #12443
• chore(ci): bump spiceio setup action to v0.6.0 by @lukekim in #12522
• fix(ci): say what was unready when a spiced readiness wait times out (refs #12473) by @grokspice in #12484
• feat(cayenne): drive incremental vacuum from the maintenance tick by @lukekim in #12435
• feat(ci): allow additional cargo features for spiceai-dev Docker builds by @ewgenius in #12198
• fix(cli): authenticate the SQL REPL's nql line like the session's SQL (fixes #12491) by @grokspice in #12496
• fix(search): keep the row's primary key out of Elasticsearch bulk-index failures (fixes #12370) by @claudespice in #12410
• fix(cpu-budget): state a low CPU request plainly, and only below half the cores by @bjchambers in #12440
• fix(models): reject a pinned Model2Vec revision instead of 401ing on it (fixes #12445) by @grokspice in #12475
• fix(cli): keep a login credential on the origin it was minted for (fixes #12505) by @grokspice in #12508
• bug: S3 vector indexing succeeds when the embedding column is absent by @Jeadie in #12249
• bug: S3 Vectors advertises unsupported metadata predicates as exact pushdown (fixes #12243) by @Jeadie in #12250
• fix(cli): treat a blank --api-key as no key at all (fixes #12498) by @grokspice in #12501
• task: Add #[derive(TypedParams)] for secret stores. by @Jeadie in #12154
• fix(runtime): report a deferred dataset that cannot build its connector (fixes #12414) by @grokspice in #12483
• fix(mysql): pre-flight CDC replication privileges with an actionable error (fixes #11967) by @grokspice in #12485
• fix(testoperator): reject a client fleet the HTTP executors cannot bound (fixes #12348) by @claudespice in #12349
• fix(ci): commit the catalog schema snapshot baselines and stop the daily PR churn by @phillipleblanc in #12525
• refactor(runtime): describe every table-provider wrapper layer once in a layer table by @phillipleblanc in #12200
• Add testoperator dispatch support for search benchmarks and replace benchmarks_search.yml by @Jeadie in #12224
• bug: RRF discards a candidate stream after an empty first batch (fixes #12239) by @Jeadie in #12384
• fix(cli): stop the SQL REPL asking a runtime its queries never went to (fixes #12493) by @grokspice in #12494
• test(runtime): run the metrics tests that are built but never run by @sgrebnov in #12510
• fix(tests): assert the search cache status instead of wall-clock timings (fixes #12487) by @grokspice in #12488
• fix(search): resolve a chunked index's entries from the authoritative store, not the read listing (fixes #12266) by @claudespice in #12411
• fix(search): Improvements for search harness by @Jeadie in #12226
• test(cayenne): cover the selective PK join and the inline/file tier boundary by @lukekim in #12517
• fix(runtime): surface the unknown-connector suggestion, and report it once (fixes #12415) by @grokspice in #12469
• fix(ci): publish no sign-off verdict when the run was signalled (fixes #12518) by @grokspice in #12544
• refactor(layering): move the modules the accelerated table shares below runtime by @bjchambers in #12219
• chore(ci): upgrade spiceio setup action to v0.7.0 by @lukekim in #12611
• fix(runtime): keep the leading slash on an inferred local Iceberg warehouse root (fixes #12533) by @grokspice in #12540
• fix(ci): let a cancelled sign-off correct only the failure it posted (fixes #12428) by @grokspice in #12432
• fix(runtime): compare an append high-water mark inclusively on a day-granular time column (fixes #12492) by @grokspice in #12500
• fix(runtime): report a dataset connector load failure once, not once per site (fixes #12365) by @grokspice in #12526
• fix(workers): give the weighted-router test a noise margin it can survive (fixes #12537) by @grokspice in #12538
• fix(runtime): reject a Hadoop table URL too short to name a namespace (fixes #12539) by @grokspice in #12542
• fix(ci): probe a pinned Hugging Face revision at a URL the Hub has (fixes #12553) by @grokspice in #12554
• feat(testoperator): pin the load phase to a target query rate by @lukekim in #12519
• fix(cli): follow only same-origin redirects, so the API key cannot leave the origin by @grokspice in #12503
• fix(ci): use spiceio and sccache for E2E macOS aarch64 builds by @lukekim in #12590
• perf(cdc): build a drained CDC burst with one blocking-pool handoff by @sgrebnov in #12514
• feat(cloud-connect): complete the spice connect surface — install, codeless connect, remove by @peasee in #12159
• fix(scripts): distinguish merge-queue PRs in signoff mine by @Jeadie in #12383
• bug: HTTP RRF truncates each candidate leg before fusion by @Jeadie in #12386
• fix(cayenne): pair the cold manifest with the warm snapshot a scan captured by @phillipleblanc in #12577
• fix(deps): bump the Vortex pin and guard the task-cancellation regression by @phillipleblanc in #12548
• chore(deps): bump datafusion and table-providers for the DuckDB timezone and retention fixes by @phillipleblanc in #12545
• fix(cayenne): surface the real error for partitioned Cayenne writes by @Jeadie in #12535
• fix(postgres): re-snapshot in-memory Cayenne on slot resume, and match the slot's lifetime to the accelerator's by @bjchambers in #12221
• fix(cli): bound the Spice.ai login poll instead of retrying forever (fixes #12506) by @grokspice in #12523
• fix(runtime): name a UTC timestamp column's zone in a spelling DuckDB knows (fixes #12528) by @grokspice in #12534
• fix(mysql): raise net_write_timeout on the shared binlog dump session (fixes #12527) by @claudespice in #12586
• fix(google-genai): read a Gemini SSE stream as bytes, so a split character survives (fixes #12597) by @claudespice in #12601
• fix(mssql): follow an availability group's read-only routing redirect (fixes #11453) by @claudespice in #12607
• test(cayenne): measure small-file fan-out against the seeded appends, not a listed count (fixes #12602) by @claudespice in #12613
• fix(ci): keep the build cache's S3 endpoint out of the runtime under test (fixes #12624) by @claudespice in #12626
• perf(ci): verify the CLI binary instead of building it again by @bjchambers in #12486
• fix(postgres): publish a member's held CDC envelope before a bulk transaction (fixes #12311) by @claudespice in #12408
• fix(cayenne): consolidate the two partition creators, fixing the accelerator's missing directory sync (fixes #12212) by @claudespice in #12619
• fix(cayenne): run a backend-parameterized test on a stack its plan fits in (fixes #12436) by @grokspice in #12561
• fix(ci): let only the merge queue pass Attestation without a sign-off (fixes #12679) by @grokspice in #12681
• fix(ci): take the trusted sign-off helpers from the workflow's own commit (fixes #12657) by @claudespice in #12662
• fix(ci): name the E2E jobs that did not succeed (fixes #12643) by @grokspice in #12649
• fix(ci): give the openai model job a ceiling above its own setup (fixes #12644) by @grokspice in #12650
• fix(ci): recognise the Flight bind wording so a foreign ready cannot pass (fixes #12642) by @grokspice in #12648
• fix(ci): call an unreachable compiler cache infrastructure, not a failing branch (fixes #12556) by @grokspice in #12557
• fix(search): clear a replacing index instead of keeping rows the source dropped (fixes #12066) by @grokspice in #12564
• fix(ci): retry the sign-off status post, and say so when it never lands (fixes #12701) by @grokspice in #12704
• build(deps): bump aws-smithy-runtime from 1.12.0 to 1.12.1 in the aws-sdk group across 1 directory by @app/dependabot in #12358
• task: Add #[derive(TypedParams)] for LLMs. (fixes #12150) by @Jeadie in #12155
• fix(ci): let a nightly's later test suites survive an earlier one's failure (fixes #12625) by @claudespice in #12627
• fix(tests): wait for a partition to persist before snapshotting its plan (fixes #12645) by @grokspice in #12652
• fix(object-store): bound an FTP and SFTP connection attempt end to end (fixes #12647) by @grokspice in #12655
• test(iceberg): decide the Hadoop catalog test's backends by environment (fixes #12646) by @grokspice in #12665
• fix(ci): give each E2E model job its own spiced ports (fixes #12419) by @grokspice in #12685
• fix(cache): evict the Pingora cache down to max_size instead of only recording it (fixes #12688) by @grokspice in #12694
• fix(cayenne): bound protected-snapshot compaction by the compaction memory pool by @sgrebnov in #12541
• fix(ci): leave an incomplete sign-off pending, not failed (fixes #12741) by @grokspice in #12742
• fix(telemetry): resolve duration histograms below a millisecond (fixes #12693) by @grokspice in #12699
• fix(ci): bound every E2E job, so a wedged build cannot hold the queue (fixes #12717) by @grokspice in #12719
• feat(testoperator): add new search benchmark datasets by @Jeadie in #12237
• test(cayenne): name the small-file compaction tests after the path they take (fixes #12612) by @grokspice in #12740
• fix(cli): bound an inference call by silence, not by total duration (fixes #12583) by @claudespice in #12589
• fix(turso): store and read back the same set of types (fixes #12628) by @claudespice in #12633
• fix(catalogs): keep a Glue database whose tables the include patterns select (fixes #12630) by @claudespice in #12638
• fix(catalogs): apply a catalog's exclude patterns, not just its include (fixes #12636) by @claudespice in #12641
• fix(cayenne): build a carry-forward rewrite from the classified manifest rows (fixes #12708) by @claudespice in #12711
• fix(runtime): subtract an append dedup window as a multiset, not a set (fixes #12499) by @grokspice in #12513
• bug: FTS exec drops absent columns and violates its declared schema (fixes #12228) by @Jeadie in #12245
• bug: rrf() linear recency decay divides the document age by the decay window twice (fixes #12232) by @Jeadie in #12385
• chore(deps): bump datafusion rev to spiceai-54 with outer-join unparser fix by @Jeadie in #12683
• fix(ci): recognise sccache's startup-timeout wording as an unusable cache (fixes #12622) by @claudespice in #12788
• fix(spiced): open the log with the startup banner by @bjchambers in #12615
• fix(metrics): resolve S3 Vectors latency below a hundred milliseconds (fixes #12698) by @grokspice in #12702
• fix(google-genai): bound one unterminated SSE event so a stalled endpoint cannot grow spiced (fixes #12600) by @grokspice in #12690
• feat(cpu-budget): size a burstable pod from its CPU request, with an all-cores opt-out by @bjchambers in #12581
• fix(ci): refuse a sign-off the branch's Makefile cannot run (fixes #12813) by @claudespice in #12815
• test(postgres): say which readiness wait timed out and what it saw (fixes #12730) by @grokspice in #12731
• fix(mssql): bound one connection attempt, so a stalled peer cannot pin a pool slot (fixes #12606) by @grokspice in #12733
• fix(ci): ask whether the disk watcher runs before make writes to it (fixes #12734) by @grokspice in #12736
• fix: delete three source files that no crate root declares (fixes #12735) by @grokspice in #12738
• fix(telemetry): keep a startup-recorded gauge on the operator meter, so it reaches /metrics (fixes #12667) by @grokspice in #12754
• fix(cloud-connect): pin rustls where the release call presents its identity (fixes #12760) by @grokspice in #12764
• test(cloud-client): assert the redirect policy by behaviour against a live server (fixes #12509) by @grokspice in #12767
• fix(search): decline a warm vector tier the accelerator cannot refill (fixes #12102) by @grokspice in #12768
• fix(ci): build without the compiler cache when it cannot be reached (fixes #12770) by @grokspice in #12771
• fix(json): end an array element where serde did, so bare scalars read (fixes #12782) by @grokspice in #12785
• fix(cache): count an invalidation as an eviction, and export the counters before one fires (fixes #12687) by @grokspice in #12791
• fix(ci): ask whether the runner can hold the build before starting one (fixes #12798) by @grokspice in #12799
• fix(ci): size the cayenne property-test ceiling above the pool, not at it (fixes #12811) by @grokspice in #12814
• fix(ci): bound every merge-queue-required job with timeout-minutes by @grokspice in #12817
• deps: bump datafusion-table-providers to 894e279 (fixes #12585) by @bjchambers in #12663
• fix(ci): take stale build output off the runners, not just report a full one (fixes #12800) by @grokspice in #12801
• fix(cayenne): give the Turso metastore sidecars one pool over cayenne.db (fixes #12727) by @grokspice in #12804
• fix(deps): drop the unreferenced ctor dev-dependency from data_components (fixes #12664) by @grokspice in #12819
• fix(ci): decline a verdict when the sign-off run itself was signalled (fixes #12710) by @grokspice in #12724
• fix(postgres): hold a resuming shared slot's ack floor for tables with no member by @bjchambers in #12676
• refactor(layering): split the table-provider layers out of runtime by @bjchambers in #12661
• build(deps): bump the github-actions-dependencies group across 3 directories with 7 updates by @app/dependabot in #12842
• fix(spiced): make the crash handler more robust by @sgrebnov in #12834
• fix(catalogs): apply the Cayenne catalog's include and exclude patterns (fixes #12766) by @grokspice in #12833
• fix(search): decide a compound index's write fatality per half, not by OR (fixes #12826) by @grokspice in #12828
• feat(cloud): organization context for spice cloud commands by @lukekim in #12515
• perf(postgres catalog): skip metadata queries for schemas no include pattern can reach by @bjchambers in #12651
• bug: S3 vector metadata conversion silently drops filterable metadata (fixes #12240) by @Jeadie in #12387
• build(deps): bump nvidia/cuda from 13.3.0-cudnn-runtime-ubuntu24.04 to 13.3.1-cudnn-runtime-ubuntu24.04 in the docker-dependencies group by @app/dependabot in #12356
• fix(runtime): keep serving when the working directory cannot be walked (fixes #6301) by @claudespice in #12803
• fix(connectors): report an object-store timeout as a timeout, not as bad credentials (fixes #12793) by @claudespice in #12797
• test(runtime): say which components were not ready when the wait times out (refs #12396) by @grokspice in #12827
• fix(cayenne): measure tuner memory pressure as unreclaimable demand (fixes #12531) by @sgrebnov in #12623
• Include v2.1.3 and 2.1.4 in SECURITY.md by @sgrebnov in #12497
• fix(search): leave the stored rows in place when a delete cannot be applied (fixes #12822) by @grokspice in #12825
• fix(google-genai): resume the SSE scan where it stopped, not at the buffer start (fixes #12689) by @grokspice in #12713
• fix(json): report a malformed JSON array element instead of dropping the rest of the file (fixes #12755) by @claudespice in #12777
• fix(runtime): honour a declared type's precision instead of narrowing it (fixes #12756) by @claudespice in #12774
• fix(cayenne): drop the partition-value guard that key encoding made unreachable (fixes #12616) by @claudespice in #12753
• fix(runtime): pin the catalog-acceleration contract in both feature configurations (fixes #12743) by @claudespice in #12752
• fix(cli): report an answer the model stopped early, instead of printing it as whole (fixes #12596) by @grokspice in #12715
• fix(runtime): scope query jobs and active queries to the principal that submitted them by @phillipleblanc in #12841
• Compute all metrics@k for all 0 < k <= n during search quality metrics. by @Jeadie in #12521
• task: Add a truncate to huggingface and file embedding providers by @Jeadie in #12620
• fix(cache): never serve a result whose tables changed after it read them by @bjchambers in #12703
• test(cayenne): cover datalake-tier statistics, pruning, and promotion triggers by @sgrebnov in #12847
• docs(ci): warn that re-running enforce-pull-with-spice replays a stale payload (fixes #12809) by @grokspice in #12810
• fix(cayenne): bound the maintained-aggregate index by memory budget, and let a stale registry recover by @lukekim in #12573
• fix(runtime): only initialize accelerators for the datasets a reload applies by @phillipleblanc in #12872
• feat: promote BYOC and Cloud Connect changes to trunk by @phillipleblanc in #12852
• fix(flight): resolve per-request settings from the live app by @phillipleblanc in #12873
• fix(cli): keep polling when the token exchange has not seen the auth code yet by @phillipleblanc in #12871
• feat(cayenne): inline a small whole-table refresh into the metastore by @lukekim in #12367
• ci: upgrade spiceio setup action to v0.8.0 by @lukekim in #12885
• fix(runtime): measure the CDC prefetch backlog, and stop its byte counter wrapping by @lukekim in #12675
• fix(turso): keep a reloaded file-accelerated dataset queryable by @phillipleblanc in #12882
• fix(query): preserve expression ordering across partition wrappers by @Jeadie in #12854
• fix(runtime): report every start-time-only runtime.* section a reload changes by @phillipleblanc in #12874
• fix(search): handle null embeddings in vector writes and JIT search by @Jeadie in #12855
• test(embeddings): truncate over-length inputs in the MiniLM embed spicepods by @Jeadie in #12856
• fix(search): Improve search benchmarking DX and fix some vector index names by @Jeadie in #12892
• test(search): dispatch all MTEB search variants by @Jeadie in #12850
• fix(runtime): stop a superseded scheduler incarnation from claiming the heartbeat key by @phillipleblanc in #12851
• fix: Ensure OTel ingest materializes missing/null dimension columns by @peasee in #12087
• feat(cloud-connect): apply a component-only Spicepod deployment in place by @phillipleblanc in #12895
• feat(cli): typed login sessions and explicit connect organization selection by @phillipleblanc in #12912
• refactor: make each table wrapper a typed layer that answers where a walk goes by @bjchambers in #12891
• feat(runtime): put every query's trace id on its log records by @lukekim in #12610
• feat(cayenne): bound the SUM of the per-table PK keyset caches by @lukekim in #12802
• fix(cache): read and expire a Pingora entry under one hold of its shard (fixes #12832) by @grokspice in #12839
• fix: Update Search integration test snapshots by @Jeadie in #12707
• fix(runtime): invalidate localpod children's cached results on parent refresh by @krinart in #12897
• fix(cayenne): use valid snapshot IDs for deferred appends by @Jeadie in #12853
• fix(ci): New branch for each push-snap-changes (only affects integration_search.yml) by @Jeadie in #12684
• fix(search): make chunked embedding and offset columns nullable (#12778) by @Jeadie in #12783
• feat(cayenne): charge scan materialization to the query memory pool by @lukekim in #12759
• fix(cache): run the Pingora invalidation scan off the calling runtime worker (fixes #12806) by @grokspice in #12808
• fix(cache): count the removals the Pingora engine performs itself (fixes #12792) by @grokspice in #12830
• fix(cayenne): reserve append sequence for partitioned upsert appends (#12779) by @Jeadie in #12784
• test(cayenne): prove the keyset degrade releases each shard as it converts by @lukekim in #12790
• perf(postgres catalog): resolve a schema's table schemas in one query (#12106) by @bjchambers in #12886
• fix(vortex): expose segment cache metrics on scrape by @phillipleblanc in #12944
• refactor: point connector imports at the crates that actually own them by @bjchambers in #12925
• fix(postgres): rebuild the acceleration when a replication slot's history is gone by @bjchambers in #12922
• fix(cayenne)!: only an explicit cayenne_tuning: adaptive enables the closed loop by @lukekim in #12949
• fix(dev-tools): show merge-queue and conflict state as labels in scripts/signoff mine by @Jeadie in #12924
• test: cover warm-index delete paths across backends (#11964) by @Jeadie in #12635
• fix(cayenne): fractional proptest scaling + batch timestamp-partition test inserts by @Jeadie in #12796
• Helpful tool to track what users/bots have got done lately by @Jeadie in #12603
• Release notes housekeeping by @krinart in #12977
• build(deps): bump serde_with from 3.20.0 to 3.21.0 by @app/dependabot in #11969
• refactor: make runtime's re-export shims crate-visible by @bjchambers in #12969
• fix(runtime): surface OTel metric tables with duplicate columns, and count unsupported metric types as rejected by @peasee in #12923
• fix: Update Search integration test snapshots by @app/github-actions in #12979
• Make signoff.yml more searchable. by @Jeadie in #12392
• bug: Support search UDTFs in SQL function components by @Jeadie in #12919
• docs: document the stacked-PR workflow, with a tested restack helper by @bjchambers in #12951
• fix(llms): fall back to pytorch_model.bin for embeddings without safetensors by @Jeadie in #12971
• feat(monitoring): improve the Datadog dashboard for multi-replica and Kubernetes deployments by @sgrebnov in #12972
• Charge the query memory pool for concurrent Vortex split decodes by @lukekim in #12940
• docs(release-notes): write release notes in Simplified Technical English by @lukekim in #12947
• refactor: narrow the DataConnector trait off the runtime's types by @bjchambers in #12993
• fix(cayenne): compact a DDL-created partition on an interval, not only on write (fixes #12617) by @grokspice in #12757
• fix(cayenne): hand out a pooled Turso connection in autocommit (refs #12820) by @grokspice in #12821
• fix: fail the lint gate on a source file no crate root declares (fixes #12737) by @grokspice in #12744
• fix(search): stage a full refresh of the memory vector tier, so dropped rows leave it (refs #12413) by @grokspice in #12823
• fix(graphql): keep the configured schema when a page has no rows (fixes #13004) by @claudespice in #13016
• fix(runtime): register task_history before embeddings/rerankers load by @Jeadie in #12978
• test(postgres): converge the catalog on source DDL, and warn when it selects nothing by @bjchambers in #12988
• fix(cayenne): budget for catalogs in the Cayenne memory classification by @bjchambers in #13027
• fix(cayenne): hold primary keys committed while the existence index is checked out by @lukekim in #13019
• feat(search): push SQL filters down into tantivy full-text search by @Jeadie in #12812
• fix(cayenne): serve the maintained count Inexact while its delta is queued (fixes #12824) by @grokspice in #12829
• fix: Update Search integration test snapshots by @app/github-actions in #13060
• fix(testoperator benchmarks): benchmark fixes by @krinart in #13012
• fix(runtime): stop query_active_count drifting up when queries share a request (fixes #12883) by @claudespice in #13034
• chore(deps): bump model2vec-rs for fast WordPiece tokenizer by @Jeadie in #12986
• ci(cayenne-doc): render diagrams locally with mermaid-cli, and fix the diagram that has been failing the build for a month by @bjchambers in #13063
• feat(monitoring): improve results cache panels on the Datadog dashboard by @sgrebnov in #13066
• feat(cayenne)!: one process-wide Vortex segment cache instead of a cache per table by @bjchambers in #12983
• fix(connectors): keep the projected schema when a query returns no rows (fixes #13015) by @claudespice in #13078
• fix(deps): bump the datafusion pin past the merged unparser fixes (fixes #12406) by @claudespice in #13083
• fix(cache): compact a sliced result before caching it (fixes #12921) by @claudespice in #13043
• refactor(cdc): give the sidecar checkpoint stores a below-runtime interface by @bjchambers in #13045
• fix(cayenne): re-baseline the maintained row count when a promotion folds a delete tombstone (fixes #12846) by @claudespice in #12998
• fix(acceleration): stop an engine-required type rewrite reading as a stale acceleration schema (fixes #13014) by @claudespice in #13074
• fix(udfs): return NULL for a vector distance that is not defined (refs #11263) by @claudespice in #13091
• fix(cayenne): refuse a data directory that contains the metastore (fixes #13055) by @claudespice in #13101
• fix(json): report content after a JSON array instead of dropping it (fixes #12786) by @claudespice in #13097
• fix(runtime): bound the spicepod apply so an unloadable dataset cannot wedge it (fixes #12862) by @claudespice in #13095
• fix(catalogs): apply the Glue catalog's exclude patterns (fixes #12634) by @claudespice in #13103
• fix(metrics): make component metrics and tests more robust by @sgrebnov in #13094
• feat(cache)!: ship the Pingora cache engine as an Enterprise-only feature by @lukekim in #13000
• fix: Update benchmark snapshots by @app/github-actions in #13056
• fix(postgres): compare CDC positions against what the slot can stream, not what it retains by @bjchambers in #12990
• docs(cayenne): compress the changelog, and give it a rule for what earns a row by @bjchambers in #13067
• test: remove a test's container when it is done with it, including on failure by @bjchambers in #13123
• fix(flight): record one metric sample per Flight RPC (fixes #12844) by @claudespice in #13112
• feat(drasi): forward CDC changes and runtime tables to a Drasi source (Alpha) by @lukekim in #12653
• fix(bench): time the engine instead of the results cache, and validate ClickBench by @lukekim in #13150
• refactor(connector): retype the DataConnector surface to DatasetSpec by @bjchambers in #13096
• bug: SQL-tier function body does not support named arguments (=>) for search UDTFs (fixes #12898) by @Jeadie in #12920
• fix(cache): compact the sliced and wide results that compaction previously skipped by @sgrebnov in #13176
• docs(stacked-prs): correct when a restack forfeits the sign-off by @bjchambers in #13130
• fix(postgres): load a provably empty CDC acceleration through its snapshot bootstrap by @bjchambers in #13175
• fix(cayenne): keep Arrow timestamp units (Vortex supports ns) by @lukekim in #13180
• Cloud Connect: enrollment and identity foundation by @phillipleblanc in #13181
• refactor(model): give the model-provider contracts a home below llms by @bjchambers in #13201
• Fix runtime lint with zero features by @krinart in #13208
• fix(benchmarks): BigQuery + Snowflake fixes + updated snapshots by @krinart in #13136
• Revert "fix(search): decline a warm vector tier the accelerator cannot refill (fixes #12102)" (#12768) by @Jeadie in #13041
• fix(snapshots): don't bootstrap the snapshot file_create just discarded by @lukekim in #13179
• Declare the CPU entitlement to Vortex at startup by @sgrebnov in #13173
• fix(mysql): fix net_write_timeout session syntax and extend transient replication errors by @ewgenius in #13178
• fix(postgres): classify EOF and server termination errors as transient in CDC replication by @ewgenius in #13163
• feat(testoperator): support customer-supplied datasets in run search by @Jeadie in #12982
• Add PDF page splitter and FinanceBench staging job (#12858) by @Jeadie in #12984
• fix(cayenne): frame the persisted PK bloom with the probe function that filled it (fixes #13137) by @claudespice in #13153
• refactor(layering): move the connectors below the runtime by @bjchambers in #13166
• feat(cli): manage the Cloud Connect service with launchd on macOS by @phillipleblanc in #13203
• Support local rerankers from text-embeddings-inference by @Jeadie in #12929
• fix(catalog): make the PostgreSQL catalog's errors and warnings actionable by @bjchambers in #13199
• fix(cloud-connect): look again when a lock create reports the entry missing by @phillipleblanc in #13231
• Skip DynamoDB in scheduled benchmarks by @ewgenius in #13222
• Populate the file metadata cache with Vortex footers at write time by @sgrebnov in #13228
• perf(cayenne): give the PK filter a cache-line layout, and the bit count it asks for by @bjchambers in #13219
• fix(mysql): rebuild a purged-position acceleration atomically instead of emptying it (fixes #12967) by @claudespice in #13023
• fix(cloud-connect): heartbeat restart-required, attachment by app id, and this crate's integration tests in the gate by @phillipleblanc in #13234
• fix(search): join all of a dataset's vector indexes on one VectorScanTableProvider by @krinart in #13209
• Validate vector search parameters before SQL query by @lesbass in #12253
• feat(helm): support customizing Deployment strategy and StatefulSet updateStrategy by @sgrebnov in #13249
• fix(postgres): recover from a replication slot lost while streaming, and state what each slot costs by @bjchambers in #13221
• fix(cayenne): stop a partitioned acceleration reporting a schema change it never applied (fixes #12999) by @claudespice in #13057
• feat(cloud-connect): name the local spicepod a cloud-managed instance serves or ignores by @phillipleblanc in #13262
• fix(snowflake): keep NUMBER precision and scale during schema discovery by @phillipleblanc in #13272
• fix(secrets): check secret references where the components resolve them by @peasee in #13265
• docs(criteria): sign off the PostgreSQL Catalog Connector at Beta by @bjchambers in #13235
• fix(cloud-connect): resolve locked service state through the retained directory descriptor (fixes #13204) by @claudespice in #13292
• chore(deps): upgrade iceberg-rust to v0.10.0 by @krinart in #13189
• fix(cloud-connect): fix Windows build of identity state-file helpers by @sgrebnov in #13308
• fix(cayenne): normalize append de-duplication types by @ewgenius in #13200
• feat(cli): unify Spice Cloud enrollment and service workflows by @phillipleblanc in #13326
• feat(postgres): native upsert delivery for durable write-back by @Jeadie in #13323
• feat(cli): create an unattached Cloud Connect project by @phillipleblanc in #13333
• fix(cayenne): exclude protected snapshots above the delete fence from subset compaction by @krinart in #13343
• fix(cli): resolve a Cloud Connect project's data-plane region from its config by @krinart in #13370
• fix(cli): keep a granted Spice Cloud login when the identity endpoint is silent by @bjchambers in #13376
• fix(duckdb): fix concurrent-query failures after a replace_file swap with multiple DuckDB files by @Jeadie in #13383

*Full Changelog: https://github.com/spiceai/spiceai/compare/v2.1.0...v2.2.0

Announcing the release of Spice v2.0-rc.3! ⚡

v2.0.0-rc.3 is the third release candidate for advanced testing of v2.0, building on v2.0.0-rc.2.

Highlights in this release candidate include:

• HTTP Connector Enhancements with OAuth2 refresh-token authentication, query-parameter pagination, and map-to-array conversion for broader API compatibility
• Databricks and Unity Catalog Reliability Improvements with resilience controls, improved UC-awareness, permission checks, and structured error reporting
• Snowflake and ADBC Registration Performance Improvements with better observability during dataset registration
• OpenTelemetry Exporter Improvements with exporter fixes and support for authenticated metrics export
• Kafka, GitHub, and HTTP Connector Fixes including Kafka reliability improvements, GitHub GraphQL resilience updates, and HTTP JSON union/reload fixes

What's New in v2.0.0-rc.3​

HTTP Connector Enhancements​

The HTTP connector now supports more authentication and API response patterns, making it easier to integrate with modern REST APIs.

Key improvements:

• OAuth2 Refresh-Token Authentication: Added support for OAuth2 refresh-token flows for APIs that issue short-lived access tokens.
• Query-Parameter Pagination: Added pagination support using query parameters for APIs that expose page or cursor controls in the URL.
• Map-to-Array Conversion: Added response transformation support for APIs that return map-shaped payloads that need to be normalized into arrays.
• Improved JSON Union Handling: Better handling for heterogeneous JSON payloads during ingestion.
• More Reliable Reloads: Fixed runtime behavior for HTTP-backed datasets during spicepod reloads.

Example configuration of an HTTP connector using the OAuth2 refresh token flow:

datasets:
  - from: https://api.example.com
    name: secure_data
    params:
      file_format: json
      allowed_request_paths: '/v1/**'
      auth_token_url: https://auth.example.com/oauth/token
      http_auth_refresh_token: ${secrets:my_refresh_token}
      http_auth_client_id: ${secrets:my_client_id}
      http_auth_client_secret: ${secrets:my_client_secret}
      auth_scopes: 'read:data offline_access'

Databricks and Unity Catalog Reliability Improvements​

Databricks and Unity Catalog integrations are now more resilient and provide clearer behavior in permission-constrained environments.

Key improvements:

• Resilience Controls: Added controls to improve reliability when interacting with Databricks services.
• Unity Catalog Awareness: Improved handling for Unity Catalog-specific behaviors and mixed deployment configurations.
• Permission Prechecks: Databricks UC permission checks now distinguish explicit denials from ambiguous cases.
• Structured Error Reporting: Advisory permission failures now surface with more actionable structured errors.
• Classic SQL Warehouse Compatibility: Improved handling for foreign table scenarios with Classic SQL Warehouse combinations.
• Task History Instrumentation: Added instrumentation to improve observability for Databricks-related operations.

Snowflake and ADBC Improvements​

Snowflake and ADBC-backed dataset registration is now faster and easier to observe.

Key improvements:

• Faster Dataset Registration: Improved registration performance for Snowflake and ADBC datasets.
• Better Observability: Added better instrumentation and visibility into registration workflows.
• ADBC Alignment: Updated ADBC dependencies and integration points for improved compatibility.
• Search Schema Fix: Fixed a full-text search schema mismatch issue with the ADBC connector.

OpenTelemetry and Observability Improvements​

Spice improves telemetry export reliability and authenticated metrics delivery for the OpenTelemetry integration.

Key improvements:

• OTEL Exporter Fixes: Fixed issues in the OpenTelemetry exporter.
• Authenticated Metrics Export: Added support for authorization headers in the OTEL metrics exporter.
• Reduced Startup Noise: Suppressed unnecessary AWS SDK noise and improved OTEL-related initialization behavior.
• Connector Initialization Reliability: Fixed issues that could block connector initialization in telemetry-related code paths.

Dependency and Toolchain Updates​

Other Improvements​

• Improved Query Pushdown: Expanded sort and limit pushdown, including improved pushdown behavior for Oracle and MSSQL connectors.
• Partitioned Query Planning Improvements: Improved PartitionedTableScanRewrite handling for ORDER BY, partition expressions, and fully qualified table references, while preventing incorrect bucketing partition pushdown to executors.
• MongoDB SRV Support: Upgraded datafusion-table-providers with MongoDB SRV support.
• Tantivy Logging: Search logging now defaults to warn unless very verbose logging is enabled.
• Kafka Connector Fixes: Improved reliability for the Kafka connector behavior.
• GitHub Connector Resilience: Improved commit fetching for dynamic and slash refs, and reduced GraphQL page sizes on gateway errors for the GitHub connector.
• GitHub API Efficiency: Lowered default comment fetch sizes to reduce pressure on GitHub GraphQL APIs.
• Embedding Validation: Added validation for embedding row_id columns during dataset initialization.
• View Cache Invalidation: Cached plans are now cleared when views are updated.
• Refresh SQL Dedup Fix: Fixed append refresh deduplication when refresh_sql selects a subset of columns.

Contributors​

• @ewgenius
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @sgrebnov

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.0.0-rc.3, use one of the following methods:

CLI:

spice upgrade v2.0.0-rc.3

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.0.0-rc.3 image:

docker pull spiceai/spiceai:2.0.0-rc.3

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.0.0-rc.3

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix: Full Text Search schema mismatch with ADBC connector by @lukekim in #10235
• docs: Update v2.0.0-rc.2 release notes with latest changes by @lukekim in #10238
• Fix append refresh dedup failure when refresh_sql selects column subset by @sgrebnov in #10225
• Revert "Properly mark dataset as Ready on Scheduler (#10215)" by @sgrebnov in #10242
• Fix failing merge conflicts for benchmarks by @krinart in #10247
• fix(github): fetch commits for dynamic and slash refs by @lukekim in #10233
• Upgrade DataFusion to v52.5.0-rc1 by @lukekim in #10249
• Merge develop to trunk (2026-04-09) by @claudespice in #10248
• fix: Validate embedding row_id columns during dataset init (fixes #8226) by @claudespice in #10208
• fix: Update tpch benchmark snapshots for federated/glue[csv].yaml by @app/github-actions in #10244
• feat(databricks): add resilience controls, UC awareness, and task history instrumentation by @lukekim in #10246
• fix: Make PartitionManager resilient to bare vs fully qualified table references by @sgrebnov in #10257
• fix: Update tpch benchmark snapshots for accelerated/s3[parquet]-cayenne[file].yaml by @app/github-actions in #10256
• Merge develop to trunk (2026-04-10) by @claudespice in #10251
• Improve Snowflake/ADBC dataset registration performance and observability by @lukekim in #10266
• Fixes for kafka connector by @krinart in #10263
• fix(runtime): gate otel code tags, suppress aws sdk noise, and unblock connector init by @lukekim in #10260
• fix(runtime): avoid regionless AWS SDK loads by @lukekim in #10271
• Add versioned release install workflow coverage by @lukekim in #10276
• fix(runtime): handle HTTP JSON unions and spicepod reloads by @lukekim in #10277
• Databricks UC permission prechecks: explicit denial as permanent error, ambiguous cases advisory by @lukekim in #10274
• Revert component status changes re-introduced by develop merge (#10248) by @sgrebnov in #10293
• Fix broken CI workflows by @ewgenius in #10294
• Group dependabot updates by ecosystem by @lukekim in #10296
• fix(tests): Replace flaky S3 Vectors snapshot tests with structural validation by @lukekim in #10301
• Update test_github_workflows snapshot by @lukekim in #10304
• fix(ci): fix Bedrock runner mismatch and snapshot auto-merge failure by @ewgenius in #10306
• feat(http): Add map-to-array conversion and query-parameter pagination by @lukekim in #10295
• New crate: datafusion-ddl by @Jeadie in #10205
• Make Databricks UC permission checks advisory with structured error reporting by @lukekim in #10283
• build(deps): bump the github-actions-dependencies group with 4 updates by @app/dependabot in #10298
• fix: Clear cached plans on view updates by @peasee in #10312
• build(deps): bump the aws-sdk group with 7 updates by @app/dependabot in #10299
• Code out of runtime. by @Jeadie in #10178
• fix: Respect function registry denies for accelerated table filter pushdown by @peasee in #10311
• fix: Don't block heartbeat when all slots acquired by @peasee in #10322
• fix: strip only outer parens in get_table_partition_expr_from_ctx by @Jeadie in #10323
• Upgrade datafusion-table-providers with MongoDB SRV support by @lukekim in #10317
• fix: Avoid pushing down bucketing partition expressions into executors by @peasee in #10324
• Upgrade datafusion-table-providers to d1b911a5 and bump adbc to 0.23 by @lukekim in #10329
• fix: Update Search integration test snapshots by @app/github-actions in #10308
• Handle foreign table + Classic sql warehouse combination gracefully by @krinart in #10318
• New crate datafusion-flightsql by @Jeadie in #10201
• Set tantivy=warn unless very verbose logging by @Jeadie in #10338
• Remove image registry and image name options from spidapter by @ewgenius in #10241
• build(deps): bump sysinfo from 0.37.2 to 0.38.4 by @app/dependabot in #10291
• build(deps): bump futures from 0.3.31 to 0.3.32 by @app/dependabot in #10289
• New crate 'datafusion-dml' by @Jeadie in #10334
• Jeadie/26 04 16/spice sql by @Jeadie in #10343
• Add Teraswitch/Pittsburgh apt mirrors + retry config for CI runners by @lukekim in #10349
• Implement sort pushdown and fix pushdown gaps across providers by @lukekim in #10337
• Merge develop to trunk (2026-04-16) by @claudespice in #10345
• Update candle and mistral.rs lock-step pins by @lukekim in #10278
• docs: fix status badges in README by @lukekim in #10350
• Migrate secrets to vars by @krinart in #10354
• Add limit pushdown and improve sort pushdown for Oracle and MSSQL by @sgrebnov in #10351
• Fix ubuntu mirror configuration by @ewgenius in #10359
• fix: Increase throughput test default ready_wait from 30s to 300s (fixes #8207) by @claudespice in #10344
• Add auth headers support to OTEL metrics exporter by @lukekim in #10347
• fix(github): shrink GraphQL page size on gateway errors; lower comment defaults by @lukekim in #10355
• Relax apt mirror substitution failure to warning in CI action by @ewgenius in #10361
• feat(http): Add OAuth2 refresh-token auth to HTTP connector by @lukekim in #10348
• Upgrade Rust toolchain to 1.94.1 by @lukekim in #10353
• Handle order by and sort in PartitionedTableScanRewrite by @Jeadie in #9656
• Fix OTEL Exporter by @krinart in #10363
• Pin spiceai candle / TEI forks to merged revs; drop local [patch] overrides by @lukekim in #10362

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.0.0-rc.2...v2.0.0-rc.3

Announcing the release of Spice v0.9-alpha! 🧙‍♂️

The v0.9 release adds several data connectors including the Spice data connector for the ability to connect to other Spice instances. Improved observability for the Spice runtime has been added with the new /metrics endpoint for monitoring deployed instances.

Highlights in v0.9-alpha​

Arrow Flight SQL endpoint: The Arrow Flight endpoint now supports Flight SQL, including JDBC, ODBC, and ADBC enabling database clients like DBeaver or BI applications like Tableau to connect to and query the Spice runtime.

Spice.ai data connector: Use other Spice runtime instances as data connectors for federated SQL query across Spice deployments and for chaining Spice runtimes.

Keyring secret store: Use the operating system native credential store, like macOS keychain for storing secrets used by the Spice runtime.

PostgreSQL data connector: PostgreSQL can now be used as both a data store for acceleration and as a connector for federated SQL query.

Databricks data connector: Databricks as a connector for federated SQL query across Delta Lake tables.

S3 data connector: S3 as a connector for federated SQL query across Parquet files stored in S3.

Metrics endpoint: Added new /metrics endpoint for Spice runtime observability and monitoring with the following metrics:

- spiced_runtime_http_server_start counter
- spiced_runtime_flight_server_start counter
- datasets_count gauge
- load_dataset summary
- load_secrets summary
- datasets/load_error counter
- datasets/count counter
- models/load_error counter
- models/count counter

Contributors​

• @phillipleblanc
• @mitchdevenport
• @Jeadie
• @ewgenius
• @sgrebnov
• @Sevenannn
• @y-f-u
• @digadeesh
• @lukekim

New in this release​

• Adds Keyring secret store (keyring).
• Adds PostgreSQL data connector (postgres).
• Adds Spice.ai data connector (spiceai).
• Adds Arrow Flight SQL (JDBC/ODBC/ADBC) support.
• Adds Databricks data connector (databricks) - Delta Lake support.
• Adds S3 data connector (s3) - Parquet support.
• Adds /v1/models API.
• Adds /v1/status API.
• Adds /metrics API.

Resources​

• Getting started with Spice.ai
• Documentation

Community​

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

• Twitter: @spice_ai
• Slack: spiceai.org/slack
• Telegram: Spice AI Discussion
• Reddit: https://www.reddit.com/r/spiceai
• Email: hey@spice.ai