10 posts tagged with "PostgreSQL"

PostgreSQL related topics and usage.

View All Tags

Spice v2.2.0 (Aug 25, 2026)

August 25, 2026 · 104 min read

Viktor Yershov

Senior Software Engineer at Spice AI

Spice v2.2.0 is now available! 🚀

Spice v2.2.0 focuses on real-time data, performance, and stability. Cloud Connect links self-hosted runtimes to Spice.ai Cloud for management and observability. MySQL datasets and PostgreSQL catalogs can now stream source changes into accelerated datasets. Debezium sources can send CDC events directly without Kafka. Warm in-memory indexes improve vector and full-text search performance.

Highlights in v2.2.0 include:

Cloud Connect — link self-hosted runtimes for BYOC (Bring Your Own Cloud) to Spice.ai Cloud for management and observability
MySQL CDC — MySQL tables now stay in sync in real time using MySQL binlog replication, with no Kafka or Debezium infrastructure and no scheduled refreshes
PostgreSQL Catalog CDC — accelerate an entire PostgreSQL database in real time with a few lines of configuration and no per-table setup
Faster Search — vector and full-text search can now serve from an in-memory index by default, with no configuration change

What's New in v2.2.0

Cloud Connect

Spice Cloud Connect is a feature of Spice.ai Cloud.

Spice Cloud Connect links self-hosted runtimes for BYOC (Bring Your Own Cloud) to Spice.ai Cloud for management and observability. Run spice cloud link to enroll a standalone runtime.

Try it with the Cloud Connect on a Development Machine recipe.

MySQL Change Data Capture

The MySQL Data Connector now supports refresh_mode: changes. The runtime loads an initial snapshot of the table. It then applies inserts, updates, and deletes from the MySQL binlog as they commit. The acceleration stays current without scheduled refreshes.

datasets:
  - from: mysql:orders
    name: orders
    params:
      mysql_host: localhost
      mysql_db: mydb
      mysql_user: replicator
      mysql_pass: ${secrets:mysql_pass}
    acceleration:
      engine: cayenne
      refresh_mode: changes

Failover-safe GTID positions: When the source server has GTID enabled, the runtime tracks the replication position as a GTID set. A GTID position stays valid across a replica promotion, so replication survives a failover to a new primary without a rebuild.

PostgreSQL Catalog CDC

The PostgreSQL Catalog Connector is now Beta with new Catalog CDC acceleration support at Alpha.

A PostgreSQL catalog can now use refresh_mode: changes. One configuration replicates every table that the include patterns match. Queries then read fresh PostgreSQL data with no per-table setup.

catalogs:
  - from: pg
    name: pg
    include:
      - 'public.*'
    params:
      pg_host: localhost
      pg_db: mydb
      pg_user: postgres
      pg_pass: ${secrets:pg_pass}
    acceleration:
      refresh_mode: changes

Known limitations while catalog CDC acceleration is Alpha: configuration may change; a table dropped and recreated in the source can serve rows captured before it was recreated (#12110); and a durable catalog acceleration can come back empty after a restart (#12729). Feedback is welcome in #11850.

Debezium CDC Without Kafka

Any Debezium source plugin can now stream change events directly into Spice, with no Kafka bus in between. A dataset with from: cdc:… accepts change events at POST /v1/datasets/{name}/cdc, in JSON or Avro. The existing Kafka path (from: debezium:…) is unchanged.

Faster Search with Warm In-Memory Indexes

Search adds a warm in-memory tier for vector and full-text indexes. The runtime writes each change to the warm tier and to the durable store together. Queries read the warm tier first and fall back to the durable store. The warm tier covers vector indexes, full-text indexes, .vectors datasets, views, and chunked Elasticsearch vector columns.

More search improvements:

cosine_distance uses SIMD instructions through the simsimd library.
Full-text search applies stemming by default.
Full-text search pushes SQL filters down into the tantivy index.
A delete now removes the document from full-text and vector indexes. BM25 statistics no longer count superseded documents.
The runtime loads local rerankers from text-embeddings-inference models.
The runtime validates vector search parameters before it runs the SQL query.

Operations & Observability

Query timeout: The new runtime.query.timeout setting bounds the duration of every query. An expired query fails with HTTP 504 or gRPC DEADLINE_EXCEEDED.

runtime:
  query:
    timeout: 30s

CPU sizing for Kubernetes pods without a CPU limit: A Spice runtime pod with a CPU request and no CPU limit now sizes itself to twice the request instead of every core on the node. Set SPICE_CPU_CORES=all (or runtime.cpu.cores: all) to burst to all cores, for example when a 0.5-core request runs on a 24-core node.
Trace IDs in logs: Every log record from a query carries the query's trace ID. Filter the logs by one ID to see everything that query did.
Improved dashboards: The Grafana and Datadog dashboards add new and improved panels, and support multi-replica and Kubernetes deployments.
Helm: The Helm chart supports a custom Deployment strategy and StatefulSet updateStrategy.
Dataset status: A non-accelerated dataset now shows the Error state when its source is unavailable, instead of appearing healthy while queries fail.

Other Notable Improvements and Bug Fixes

Drasi (Alpha): The runtime forwards CDC changes and runtime tables to a Drasi source.
Microsoft SQL Server: Kerberos integrated authentication now works on Unix.
HTTP connector: OAuth2 client-credentials authentication.
Oracle DATE: Values lost their time of day. They now map to a timestamp.
Catalog exclude patterns: The Glue and Cayenne catalogs ignored exclude patterns. Both pattern lists now apply.
Spice Cayenne schema changes: Partitioned accelerations could report a source schema change as applied while partitions kept the old schema. This mismatch caused lossy casts or append failures. Cayenne now rejects in-place evolution for partitioned accelerations. The configured schema-change policy then handles the change.
Snowflake NUMBER values: The connector could remove fractional digits during schema discovery. It treated numeric metadata as absent and defaulted the scale to zero. It now preserves the source precision and scale.

See the Changelog for the full list of fixes.

SDK Updates

Updated SDKs release alongside v2.2.0:

spice.js v3.2.0 — adds query cancellation (listActiveQueries/cancelActiveQuery), mTLS client certificates, HTTP fallback when Flight is unavailable, and typed parameter binding via Flight SQL prepared statements. Also fixes nsql() and search() response handling and named parameters over HTTP.
spicepy v4.0.0 — adds streaming of large results, natural-language queries (nsql), vector and hybrid search, and query cancellation. Spice.ai Cloud users need this update: the legacy hostnames are retired in favor of region-specific endpoints.
spice-rs v3.2.0 — adds search, NSQL, async queries, query cancellation, and mTLS.

Dependency Updates

Dependency / Component	Version
DataFusion	v54.1
iceberg-rust	v0.10.0
Turso	v0.7.2
Rust toolchain	v1.96.1

New Contributors

@boittega made their first contribution in #11509
@lesbass made their first contribution in #12253

Contributors

Breaking Changes

CPU sizing for Kubernetes pods without a CPU limit: A Spice runtime pod with a CPU request and no CPU limit now sizes itself to twice the request instead of every core on the node. Set SPICE_CPU_CORES=all (or runtime.cpu.cores: all) to keep the previous behavior, or set runtime.cpu.cores to a specific core count.
Partitioned DuckDB accelerations are removed: The DuckDB accelerator now rejects partition_by. Use the Cayenne or Arrow accelerator for partitioned datasets.
Adaptive Cayenne tuning requires an explicit opt-in: An unset cayenne_tuning now resolves to auto. Set cayenne_tuning: adaptive to enable the closed-loop controller.
ONNX ML inference is removed: The runtime no longer loads ONNX models, and it no longer serves the /v1/predict endpoints. Use an LLM model provider instead.
One process-wide Vortex segment cache: Cayenne tables now share one segment cache instead of one cache per table. Set cayenne_segment_cache_mb under runtime.params to size it. When unset, the cache takes 1/64 of the memory entitlement, clamped to 256 MiB - 2 GiB.
The Pingora cache engine is now a Spice.ai Enterprise feature: An OSS build with engine: pingora degrades to the Moka engine and logs the substitution.
Deprecated settings: pg_replication_temporary_slot is deprecated. cayenne_segment_cache_mb at the table level is deprecated — set it under runtime.params instead. This release renames the MongoDB num_docs_to_infer_schema setting to schema_infer_max_records. The old name still works and warns.

Cookbook Updates

Added a new recipe for MySQL CDC (Binlog Replication): stream real-time changes from a MySQL table into Spice with refresh_mode: changes.
Added a new recipe for Aurora MySQL CDC (Binlog Replication): the same CDC setup for an AWS-hosted Amazon Aurora MySQL cluster.
Added a new recipe for PostgreSQL Catalog CDC: replicate every table of a PostgreSQL database with one catalog configuration.
Added a new recipe for Cloud Connect on a Development Machine: connect a local Spice instance to Spice.ai Cloud, deploy live changes, and deliver a secret.

The Spice Cookbook includes more than 104 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v2.2.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.2.0 image:

docker pull spiceai/spiceai:2.2.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.2.0

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed

Changelog

feat(testoperator): --prepare-only / --skip-prepare for htap source reuse by @bjchambers in #11486
test(cluster): de-flake scheduler_failover (drop unreliable scheduler_node assertion) by @phillipleblanc in #11518
feat(cayenne): wire orphaned-DV cleanup knob to spicepod params + doc sync by @bjchambers in #11523
fix(cayenne): warn (not panic) on benign manifest/listing race; re-enable now-passing ignored tests by @lukekim in #11511
fix(datafusion): accurate projected scan byte size so hash joins build the smaller side by @sgrebnov in #11503
fix(cayenne): seed persisted num_rows for hash-join sizing by @sgrebnov in #11515
fix(cayenne): user-visible DELETE WHERE pk IN (...) reports the real row count by @lukekim in #11514
test(cayenne): CDC convergence/resurrection fuzz harness by @bjchambers in #11502
fix(cayenne): correctness & memory_limit fixes from perf audit (2 P0, 3 P1) by @lukekim in #11516
perf(cayenne): compaction IO-hygiene follow-ups — input-page evict + O_DIRECT writer + bench by @lukekim in #11498
Update end_game.md by @bjchambers in #11535
chore: Vendor ttf-parser, lopdf, pdf-extract by @peasee in #11521
fix: Update tpch benchmark snapshots for federated/mysql[catalog].yaml by @app/github-actions in #11531
Fix s3 vectors API by @krinart in #11536
Update from spiceai/datafusion#177 by @Jeadie in #11541
perf(cayenne): off-write_lock N>1 mem-tier checkpoint + adaptive query-admission throttle by @lukekim in #11538
fix(flightsql): propagate bearer token to per-endpoint clients by @boittega in #11509
fix: Placeholder table initialization lock swap by @peasee in #11540
chore(cluster): bump ballista pin for the null-aware anti-join fix by @phillipleblanc in #11544
fix(runtime-tools): fix memory table identifier validation rejecting valid names by @Jeadie in #11546
test(chbench): enable 4-way mem-tier PK sharding on SF1000 memory-durability configurations by @sgrebnov in #11553
fix(reorder): reorder inner-join islands inside EXISTS / NOT EXISTS by @sgrebnov in #11552
feat(cayenne): enable orphaned-DV cleanup by default (per-table threshold 20) by @bjchambers in #11533
fix(postgres): delete old key on CDC primary-key updates by @phillipleblanc in #11551
Properly exclude vendored crates from make lint-rust-fix by @krinart in #11560
fix: arrow IndexedMemTable serves stale results after DML/retention/sync (fixes #11262) by @claudespice in #11532
perf(cayenne): concurrent per-shard mem-tier checkpoint encode (drain parallelism) by @lukekim in #11558
fix(physical-plan): estimate col=literal selectivity from NDV, not the flat 20% default (Datafusion) by @sgrebnov in #11562
fix: Read Postgres DB source error when available by @peasee in #11566
feat(cayenne): cold object-store tier with read-optimized clustering by @lukekim in #11543
Generic CDC replication-lag metric by @krinart in #11554
fix: ignore transitive quick-xml advisories (RUSTSEC-2026-0194/0195) in cargo-deny by @krinart in #11571
test(chbench): add cold-object-store-tier SF-1000 HTAP run; cap SF-1000 mem-tier at 1 GiB by @lukekim in #11573
fix: surface silent Turso read-path conversion failures (fixes #11276) by @claudespice in #11570
feat(testoperator): emit P99 + Max replication lag and staleness telemetry for HTAP by @sgrebnov in #11572
feat(runtime): cost-based eager aggregation, enabled by default (datafusion spiceai-54) by @bjchambers in #11446
Bump datafusion to include spiceai/datafusion#181 by @Jeadie in #11563
fix(cayenne): drain in-flight maintenance before in-process reopen (mutation_property flake) by @bjchambers in #11578
Unified JSON nesting by @krinart in #11555
feat(testoperator): capture under-load EXPLAIN plans as HTAP artifacts by @sgrebnov in #11576
feat(cayenne): freshness-SLO-driven adaptive mem-tier shrink (+ freshness diagnostics) by @lukekim in #11574
fix(cayenne): fold un-checkpointed mem-tier keys into the cold keyset rebuild by @lukekim in #11592
perf(cayenne): reuse the PK RowConverter across the sharded CDC apply by @lukekim in #11590
Add labels to spiceai-dev by @krinart in #11591
Rename mongo num_docs_to_infer_schema (deprecated) -> schema_infer_max_records by @krinart in #11593
fix(postgres): don't ACK replication slot past uncommitted envelopes on empty transactions by @bjchambers in #11582
refactor(connectors): extract abfs, adbc, cosmosdb, ducklake, gcs, git, github, glue, kafka, snowflake-native, spiceai into data-connectors crates by @Jeadie in #11407
Revert "Generic CDC replication-lag metric" by @lukekim in #11600
refactor(cayenne): remove orphaned-DV cleanup spicepod param, make it always-on by @bjchambers in #11575
fix(mongodb): substitute delete when UpdateLookup finds no document by @bjchambers in #11583
ci(chbench): SF1000 tuned+adaptive every 3h, other configs once daily by @lukekim in #11601
feat(cayenne): add integer support to maintained AVG by @bjchambers in #11564
Refresh spidapter Spice Cloud token from client credentials by @krinart in #11594
fix: cluster filtered COUNT(*) on append-only tables returns unfiltered total (fixes #11599) by @claudespice in #11603
fix(scylladb): error on lossy CQL decimal conversion instead of silent truncation/NULL (fixes #11267) by @claudespice in #11604
ci(chbench): drop postgres-arrow and tuned (non-memory) from scheduled HTAP runs by @lukekim in #11606
ci(chbench): lower tuned SF1000 mem-tier max age to 15s for scheduled HTAP runs by @lukekim in #11618
fix(cayenne): fold the un-checkpointed mem-tier into the persisted-bloom PK-index rebuild (SF-100 over-count) by @lukekim in #11609
Potential fix for 1 code quality finding by @lukekim in #11614
fix(postgres): shared replication slot drops WAL for partitioned tables (fixes #11290) by @claudespice in #11607
bench(chbench): add 4-slot CDC variant pod for SF-1000 memory (separate from default) by @bjchambers in #11605
fix(cayenne): don't serve drifted memory-CDC row count as Exact to COUNT(*) by @bjchambers in #11602
chore(deps): vendor pgwire-replication 0.3.2 by @bjchambers in #11617
Add SQL warehouse ID to integration workflow by @Jeadie in #11561
fix(search): cosine_distance returns NULL not NaN for zero-magnitude vectors by @claudespice in #11621
feat(spidapter): tuned + adaptive MongoDB CDC spicepods with scenario selection by @bjchambers in #11579
Potential fixes for 3 code quality findings by @lukekim in #11613
Potential fixes for 2 code quality findings by @lukekim in #11611
Potential fixes for 2 code quality findings by @lukekim in #11615
Potential fix for 1 code quality finding by @lukekim in #11612
feat(cayenne): split mem-tier into ingestion + immutable pieces; advance the replication slot on a cheap periodic seal (default 2s) by @lukekim in #11622
fix(mcp): encode catalog-qualified tool names without '/' (fixes #10894) by @claudespice in #11629
fix(refresh): apply refresh_sql override on manual refresh when source unchanged (fixes #11353) by @claudespice in #11626
fix: report empty accelerated schema instead of misleading missing-column error (fixes #10920) by @claudespice in #11625
fix(imap): store message date as millisecond Timestamp, not seconds in Date64 (fixes #11547) by @claudespice in #11624
test(bucket): guard frozen partition-hash output against silent ahash drift (addresses #11277) by @claudespice in #11623
Bump datafusion (spiceai/datafusion#182) and datafusion-table-providers (#27): fix q16 CollectLeft planning error and SQLite q6 wrong revenue by @Jeadie in #11598
fix(cluster): prevent executor flaps and distributed query stalls by @phillipleblanc in #11565
fix(cli): allow spice trace on dynamic MCP/custom tool_use tasks (fixes #10995) by @claudespice in #11630
docs: add clang/lld to arm64 Linux build prerequisites (fixes #11120) by @claudespice in #11632
perf(cayenne): mem-tier visible-batch memo + maintained MIN/MAX IVM by @lukekim in #11631
docs: consolidate agent instructions and fix stale references by @lukekim in #11634
build(deps): bump the github-actions-dependencies group across 2 directories with 7 updates by @app/dependabot in #11627
perf(flight): offload query-result IPC encoding off the IO runtime by @lukekim in #11608
feat(runtime): reservation-aware Cayenne CDC query-memory default (75%→70%) by @lukekim in #11641
Revert "ci(chbench): lower tuned SF1000 mem-tier max age to 15s for scheduled HTAP runs" by @lukekim in #11648
fix(cayenne): don't bump mem-tier version on seal (QPH regression from #11622) by @lukekim in #11649
feat(cayenne): vendor versioned row_converter module (byte-compatible with arrow-row 58.3.0) by @phillipleblanc in #11654
fix(flightsql): advertise analyzed query schemas by @phillipleblanc in #11656
feat(cayenne): bounded append freshness — segmented stream publishing + publish-triggered executor stats broadcast by @phillipleblanc in #11620
fix(postgres): decouple replication keepalive/feedback from apply backpressure by @bjchambers in #11653
fix: bump DataFusion for eager decimal SUM schema by @phillipleblanc in #11657
fix(cayenne): stop mem-tier snapshot storms under memory pressure and byte-range splitting of small snapshot scans by @sgrebnov in #11645
fix(ducklake): add opt-in automatic_migration to attach older catalogs (fixes #10899) by @claudespice in #11635
ci(e2e): tolerate empty-type DuckDB variant of the parquet-rename race by @claudespice in #11636
build(deps): bump the aws-sdk group with 3 updates by @app/dependabot in #11628
ci(chbench): FIFO-queue HTAP runs via concurrency queue: max; simplify dispatcher by @bjchambers in #11659
fix(bucket): guard against ahash AES-path re-bucketing of partition data (fixes #11277) by @claudespice in #11665
fix(cayenne): gate metastore reinsert_sequence schema behind a user_version (fixes #11291) by @claudespice in #11651
fix(cayenne): durable CDC delete keeps the pk IN (...) fast path (fixes #11633) by @lukekim in #11642
ci(chbench): make scheduled HTAP runs adaptive-only (retire tuned, convert cold tier to adaptive) by @lukekim in #11677
fix(imap): decode text body into content instead of storing raw MIME (fixes #11549) by @claudespice in #11667
feat(cayenne): end-to-end integrity checksums for WAL records and Vortex data files (fixes #11639) by @claudespice in #11646
feat(spidapter): date-prefix per-run MongoDB CDC databases by @bjchambers in #11658
test(cayenne): fuzz mem-tier CDC + seq-prefix bake; harden durable-path count exactness by @bjchambers in #11643
deps: bump spiceai/datafusion to pick up ORDER BY alias unparser fix by @Jeadie in #11655
fix(postgres_replication): size CDC change-batch builders to the transaction row count by @sgrebnov in #11675
Release notes for v2.1.0 by @Jeadie in #11429
perf(pgwire-replication): incremental zero-copy framing (FrameReader) + bounded message size by @lukekim in #11668
fix(cdc): memory-mode changes stream no longer fatally stops on a transient deferred-commit queue (fixes #11644) by @bjchambers in #11678
test: update snapshots from job 84685029883 by @Jeadie in #11569
fix: Distinguish no filters vs lit(false) for RefreshSql distributed accelerations by @peasee in #11596
Fix cargo-deny crossbeam-epoch advisory by @Jeadie in #11681
fix: Update benchmark snapshots for trunk run by @app/github-actions in #11666
chore(deps): upgrade Vortex to 0.76.0 by @lukekim in #11682
refactor: extract runtime-metrics crate, move ComponentType to runtime-api-types by @Jeadie in #11524
feat(cloud-connect): standalone instance adoption by @lukekim in #11060
docs: keep the feature set constant across a session (incremental-build hygiene) by @bjchambers in #11679
feat(mysql): binlog replication for refresh_mode: changes by @lukekim in #11672
feat(cdc): cap keys per durable delete plan + absorb fall-through metrics (#11673) by @bjchambers in #11680
ci(chbench): run scheduled SF1000-adaptive HTAP bench every 6h instead of 3h by @lukekim in #11698
fix: MySQL connector fails to convert unsigned integer types (fixes #8364) by @claudespice in #11691
observability(cdc): localize CDC back-pressure across the ingest pipeline by @bjchambers in #11610
fix(cache): cache empty (zero-row) SQL result sets by @bjchambers in #11699
Update spicepod.schema.json by @Jeadie in #11706
Add support for MysSQL in Ch-Bench by @krinart in #11700
observability(chbench): limit CH-BenCHmark logs to info and above by @bjchambers in #11711
perf(cdc): cache per-dataset metric labels to avoid per-event string copies by @bjchambers in #11712
feat(postgres): pgoutput binary-format CDC decoding + zero-copy decoder by @bjchambers in #11702
Update integration_models.yml by @Jeadie in #11715
fix(cayenne): zero-row append refresh fails with "No such file or directory" by @sgrebnov in #11710
perf(cdc): cut shared-slot Postgres replication pump per-event overhead by @bjchambers in #11709
fix(spicepod): reject unknown fields on columns[] config (fixes #10972) by @claudespice in #11719
feat(cayenne): tier-gated O_DIRECT compaction writer (EBS/NAS network storage) by @lukekim in #11696
fix(cayenne): harden Vortex compaction writes by @phillipleblanc in #11692
perf(vortex): push boolean NOT filters into the Vortex scan by @lukekim in #11721
chore(chbench): cap scheduled HTAP runs at 250K tpmC (rate 9250) by @sgrebnov in #11723
MySQL CH-benCHmark HTAP: SF1 + SF1000 adaptive configs by @krinart in #11722
feat(testoperator): TPC-DS SF100 federated cluster-bench spicepod + default spicehq nodegroup by @phillipleblanc in #11716
refactor(runtime): move AccelerationSource and Acceleration into runtime-acceleration by @Jeadie in #11530
remove(ml): delete model_components crate and ONNX/Tract ML inference by @Jeadie in #11684
ci: dedupe testoperator query_overrides run steps by @Jeadie in #11690
Update search snapshots by @Jeadie in #11714
refactor: use inherent is::<T>() on dyn TableProvider/dyn ExecutionPlan for type checks by @Jeadie in #11585
chore: remove unused BlueOak license allowance by @phillipleblanc in #11762
perf(vortex): remove per-file and per-segment allocations on the scan path by @lukekim in #11733
fix(ci): use GraphQL API to check isLatest in spiced_docker workflow by @Jeadie in #11735
fix(release): use release notes heading as GitHub release title by @Jeadie in #11744
feat(chbench): headline run summary + adaptive slot-topology variants by @bjchambers in #11734
fix(ci): update e2e CLI tests for Windows-no-runtime and chat error wording by @Jeadie in #11717
build(rust): upgrade toolchain to 1.96.1 by @lukekim in #11743
fix(deps): enable reqwest "query" feature for isolated crate builds by @claudespice in #11760
build: define clippy lints in [workspace.lints], opt in every crate by @bjchambers in #11676
refactor(runtime): parameterize table-provider unwrapping by inner-fn set by @Jeadie in #11759
fix(cayenne): retain source deletion index in scan memos to close ABA hazard (fixes #11303) by @claudespice in #11739
fix(mysql): refuse binlog resume across source layout drift by @phillipleblanc in #11761
perf: avoid unnecessary allocations on search hot paths by @phillipleblanc in #11767
feat(spidapter): support cluster_name for private nodegroup targeting by @phillipleblanc in #11740
feat(cayenne): finalize datalake (cold) tier v1 UX, credentials, and e2e integration test by @sgrebnov in #11731
perf(vortex): push CAST(CASE ... END) filters into the Vortex scan by @lukekim in #11729
docs: lead README with real-time analytics-node + CDC messaging (Spice 2.0) by @lukekim in #11774
chore(chbench): extend scheduled HTAP runs to 15m by @lukekim in #11773
perf(cayenne): lazy NDV — compute on file spill, off the inline hot loop by @bjchambers in #11741
ci(rust): install 1.96.1 in workflows that pin the toolchain explicitly by @lukekim in #11769
perf(cdc): deferred parsing — move Postgres CDC decode + build off the shared pump by @bjchambers in #11732
docs: correct sort-kernel SIMD note (AVX2, not AVX-512) by @lukekim in #11771
fix(cayenne): roll cold-tier files at cayenne_cold_target_file_size_mb by @sgrebnov in #11746
docs(cayenne): move the Cayenne technical reference into the repo by @bjchambers in #11775
perf(cayenne): unify upsert PK-key hashing on prehashed XXH3-128 digests by @bjchambers in #11736
perf: avoid unnecessary allocations on query hot paths by @phillipleblanc in #11768
Extended schema inference for mysql by @krinart in #11742
chore(hash-index): add SBBF probe/insert baseline benchmark by @lukekim in #11770
feat(cayenne): incremental datalake promotion, per-file PK blooms, physical GC by @sgrebnov in #11745
test(cayenne): limit property-test concurrency by @phillipleblanc in #11784
perf(cdc): lock-free per-member ack slots + coalescing for shared Postgres replication by @bjchambers in #11779
perf(chbench): speed up MySQL and Postgres seed loading by @bjchambers in #11782
fix(ci): download MinIO client from public-data mirror by @phillipleblanc in #11790
ci: developer sign-off attestation gating the merge queue by @lukekim in #11776
fix(tests): synchronize chat expect prompts correctly by @phillipleblanc in #11802
perf(chbench): default adaptive pod to 4-slot grouping + deep prefetch by @bjchambers in #11799
refactor(cli): replace open crate with in-tree system-open by @phillipleblanc in #11791
refactor(runtime): drop direct rustls-pemfile dependency by @phillipleblanc in #11792
Cache ListingTable file statistics to avoid per-query footer re-parse by @phillipleblanc in #11793
ci: inherit sign-off across clean base merges by @phillipleblanc in #11804
perf(cayenne): hash PK once and skip OwnedRow clone on present path by @bjchambers in #11805
fix(postgres catalog): register partitioned parent, not child partitions by @bjchambers in #11798
fix(postgres catalog): quote foreign-key target identifiers by @bjchambers in #11796
refactor(cayenne): rename remaining cold_* params to cayenne_datalake_* by @sgrebnov in #11795
build: add release-profiling Cargo profile for CPU profiling by @bjchambers in #11807
fix(ci): make CH-benCH template restore robust to leftover replication slots by @sgrebnov in #11811
perf(cayenne): remove eager-NDV escape hatch, speed up per-value NDV hashing by @bjchambers in #11806
Auto-refresh the Attestation check after signoff by @bjchambers in #11815
refactor(duckdb): remove partitioned DuckDB accelerator modes by @lukekim in #11808
feat(testoperator): add option to skip the HTAP analytic gate by @bjchambers in #11810
fix: prevent executor S3 region poisoning before object-store bind by @phillipleblanc in #11766
perf(cayenne): reuse per-batch scratch allocations in KeyBasedDeletionFilterStream by @bjchambers in #11817
Update openapi.json by @app/github-actions in #11816
build(deps): bump the github-actions-dependencies group across 2 directories with 5 updates by @app/dependabot in #11819
fix(task_history): capture ExplainAnalyze metrics from the executed plan by @phillipleblanc in #11794
feat: generic spiced env-var passthrough for HTAP testoperator runs by @bjchambers in #11824
feat(cayenne): datalake tier hardening + row-capped PK-bloom-backed promotion by @sgrebnov in #11812
chore: release finalization for v2.2.0 by @Jeadie in #11738
feat(cayenne): add memory mode (mode: memory) — fully in-RAM accelerator by @lukekim in #11720
Fix secrets in .github/workflows/integration_models.yml. by @Jeadie in #11781
feat(runtime): add runtime.query.timeout parameter by @sgrebnov in #11822
feat(cloud-connect): align proto to canonical, fix enrollment ordering, add TLS e2e by @lukekim in #11829
docs(criteria): mark PostgreSQL Catalog Connector as Alpha by @bjchambers in #11785
test(cayenne): add regression tests for the mid-pass overwrite guard by @sgrebnov in #11844
perf(cayenne): light delta encoding + higher CDC coalescing defaults by @bjchambers in #11826
ci: run Cayenne doc PDF build only on trunk merges by @bjchambers in #11860
fix(cayenne): drain staged Stage-B publishes before cold promotion; add cold-tier fuzz coverage by @sgrebnov in #11847
feat(cayenne): atomic cross-partition append + delete/on-conflict atomicity by @lukekim in #11803
feat(cayenne): min/max maintained aggregates + N>1 CDC retract by @lukekim in #11862
perf(chbench): CSV-based seed loading for MySQL and Postgres by @bjchambers in #11843
fix(tests): re-enable prop_concurrent_cold_sqlite cayenne cold-tier fuzz test by @sgrebnov in #11871
fix(deps): replace yanked spin releases by @phillipleblanc in #11878
fix(cayenne): correct stale mem_checkpoint_lock comment by @bjchambers in #11872
Add run links to testoperator_dispatch.yml by @Jeadie in #11747
Cayenne serializable transactions: gated writes, per-key OCC, multi-table, FlightSQL, durable write-back by @phillipleblanc in #11870
fix(cayenne): partitioned datasets deadlock against the global encode budget and never become ready by @Jeadie in #11825
test(runtime): re-runnable turso file cleanup + bump turso to 0.7.0 by @lukekim in #11783
Use liteparse for PDF document parsing by @Jeadie in #11522
ci: add remote signoff workflow by @Jeadie in #11864
fix(cli): populate org in spice cloud apps --output json (fixes #11041) by @claudespice in #11867
fix(embeddings): restore params broken by #10853 by @Jeadie in #11788
Move cargo advisory checks to scheduled workflow by @Jeadie in #11879
perf(cayenne): bound cold-promotion Z-order sort into streaming byte-capped runs by @sgrebnov in #11890
fix(cayenne): write-back transaction atomicity — stage (not publish) and read mem-tier rows by @phillipleblanc in #11889
fix(runtime-table-partition): restore sound modulo inequality partition pruning by @Jeadie in #11891
Replace dotenvy with in-repo dotenv crate by @phillipleblanc in #11894
Remove unnecessary allocations from results-cache and hot conversion paths by @phillipleblanc in #11895
Increase ready_wait timeout for chbench sf1000 cold-tier configs by @sgrebnov in #11900
fix(mysql): retry binlog checkpoint upsert on transient accelerator write lock by @krinart in #11876
fix(cayenne): gate freshness mem-tier shrink on apply backlog by @lukekim in #11893
fix(postgres catalog): honor unsupported_type_action for catalog-discovered tables by @bjchambers in #11875
fix(postgres): make TPC-H/TPC-DS benchmark CI actually run at SF1/SF10/SF100 by @bjchambers in #11883
feat(search): writethrough compound SearchIndex/VectorIndex with optional fallback by @Jeadie in #11892
Revert "perf(cayenne): light delta encoding + higher CDC coalescing defaults" by @bjchambers in #11910
perf(vss): SIMD-accelerate cosine_distance via simsimd by @Jeadie in #11748
Add run-name to signoff.yml by @Jeadie in #11912
fix(cayenne): serialize cold-tier mem-tier checkpoint by @sgrebnov in #11907
fix(postgres catalog): discover materialized views and foreign tables by @bjchambers in #11874
test(chbench): stronger content fingerprint + MySQL sidecar template caching by @krinart in #11901
test(chbench): align money columns to canonical DECIMAL schema (CMU BenchBase) by @sgrebnov in #11921
fix(cayenne): close per-key OCC missed-conflict holes and fused-txn IVM staleness by @lukekim in #11916
ci(attestation): fast-track pure reverts past developer sign-off by @bjchambers in #11913
fix(cloud-connect): accept the portal's 5-char adoption-code segments by @phillipleblanc in #11926
docs(layering): codify crate tiers + workspace layering guard by @bjchambers in #11919
refactor(cayenne): datalake tiering UX — param rename + tracing by @sgrebnov in #11920
feat(params): typed component params via #[derive(TypedParams)] — embeddings pilot by @Jeadie in #11809
Fix SQlite round type error. by @Jeadie in #11927
fix(signoff): let remote sign-off refresh the Attestation check in CI by @bjchambers in #11929
feat(testoperator): print rows around CH-benCH analytical-gate mismatches by @bjchambers in #11923
fix(postgres catalog): don't abort the whole catalog on one schema's discovery failure by @bjchambers in #11873
ci(signoff): target-lint changed crates before full local/remote gate by @lukekim in #11909
feat(runtime)!: always-on schema inference; remove schema_inference config by @lukekim in #11880
refactor(layering): extract ClickHouse into connector-clickhouse; add restricted_deps guard by @bjchambers in #11931
perf: remove unnecessary allocations on hot query, search, and metrics paths by @phillipleblanc in #11918
Release notes for v2.1.1 by @Jeadie in #11906
refactor(layering): prepare DynamoDB for extraction from runtime by @bjchambers in #11935
Coalesce Null primary keys in RRF. by @Jeadie in #11519
ci(e2e): strip ANSI before duckdb_append graceful-shutdown log whitelist by @phillipleblanc in #11937
feat(catalog): PostgreSQL catalog-level CDC acceleration (changes mode) by @bjchambers in #11897
chbench(mysql): parallel analytical gate, decimal-comparison fix, and reseed/CI hardening by @krinart in #11946
fix: array_any_value panic when output is hash-repartitioned (empty list elements) by @bjchambers in #11952
feat(layering): checkpoint sidecar as per-engine crates + DI the dynamodb connector by @bjchambers in #11938
chore(cluster): pin Ballista to upstream 54 merge tip by @phillipleblanc in #11941
feat(vortex): upgrade fork pins to Vortex 0.79.0 by @lukekim in #11950
chbench(htap): run mysql configs at the shared 9250 rate; 2x converge wait by @sgrebnov in #11956
fix(testoperator): HTAP drain gate false-fails on stale probe observations — final snapshot decides convergence (fixes #11953) by @claudespice in #11966
feat(catalog): replica-identity-aware eligibility for PostgreSQL catalog CDC by @bjchambers in #11951
refactor(layering): extract DynamoDB into connector-dynamodb by @bjchambers in #11960
Add MemoryVectorIndex: in-memory external-store VectorIndex with brute-force exact k-NN by @Jeadie in #11908
ci(signoff): prefer lab SSH for remote sign-off; skip Rust when no .rs changes by @lukekim in #11977
Reapply #11826: light delta encoding + CDC coalescing (SF1000 convergence lever) — gated on #11943 + SF1000 fingerprint run by @lukekim in #11944
docs: align component statuses with signed release criteria by @lukekim in #11970
fix: Spidapter flight auth passthrough by @peasee in #11976
Unified CDC config by @krinart in #11777
feat(cdc): Debezium plugin push ingest to Spice without Kafka by @lukekim in #11955
refactor(connectors): unify registration on the linkme slice; restore schema coverage by @bjchambers in #11972
GTID-based MySQL CDC by @krinart in #11813
test(mysql): ignore mysql_binlog_replication_end_to_end_cayenne by @sgrebnov in #11986
fix(cayenne): support Decimal128 in maintained SUM/AVG aggregates (fixes #11933) by @claudespice in #11979
feat(params): typed params for VectorStore and FtsStore engines by @Jeadie in #11954
chore(layering): remove orphaned dead code from connector extractions by @bjchambers in #11988
feat(cayenne): default-on adaptive cold layout from observed filters (F4) by @lukekim in #11973
fix: Fairly allocate partitions from scheduler assignment cycle only by @peasee in #11853
fix(cli): avoid escaped ANSI version notification by @ewgenius in #11996
MySQL Shared binlog connection by @krinart in #11814
feat(catalog): fail-loud + metrics + by-kind reporting for PostgreSQL catalog CDC (#11850) by @bjchambers in #11983
fix(cayenne): P1 audit — subset compact, SMJ 2.5× HT, mem-tier pool account by @lukekim in #11991
Add OAuth2 client-credentials grant and configurable auth header to HTTP connector by @krinart in #11981
fix(telemetry): validate runtime.telemetry.metric_prefix against OTel name syntax by @ewgenius in #12002
fix(cayenne): purge CDC mem-tier on delete-all/TRUNCATE by @sgrebnov in #12009
Update signoff.yml. by @Jeadie in #11998
feat(dev): add scripts/signoff mine — attestation status across your open PRs by @Jeadie in #11978
refactor(layering): extract MongoDB provider into connector-mongodb by @bjchambers in #11982
perf: remove three unnecessary allocations on the query hot path by @phillipleblanc in #12029
refactor(layering): evacuate cosmosdb/graphql/github/sharepoint providers into their connector crates by @bjchambers in #12024
fix(mysql-cdc): detect source reset on GTID resume by @sgrebnov in #12023
fix(smb): exclude final SESSION_SETUP response from preauth integrity hash (fixes #11148) by @claudespice in #12003
fix(postgres): validate pg_replication_slot names before CDC refresh by @ewgenius in #12001
fix(postgres): honor pg_connection_string for CDC (closes #11994) by @ewgenius in #12000
feat: Support OTLP Histogram ingest, unix nanos time by @peasee in #11992
feat(search): write-through warm + fallback compound index for FTS (#11886) by @Jeadie in #11971
docs: PR-description and code-comment conventions by @bjchambers in #12032
refactor(postgres cdc): consolidate every dataset onto the shared pump by @bjchambers in #12028
feat(cloud-connect): Standalone instance adoption connection by @peasee in #11980
fix(cdc): don't force the durable path on zero-row readiness heartbeats (fixes #12007) by @claudespice in #12030
Upgrade DataFusion to 54.1.0 by @krinart in #11974
perf(github): parallelize serial fetches (commits + workflow logs) by @lukekim in #12017
refactor(layering): evacuate odbc/scylladb/imap/git providers into their connector crates by @bjchambers in #11993
refactor(layering): carve runtime-component out of runtime, break the ArcRuntime cycle by @bjchambers in #12031
feat(mssql): enable tiberius integrated-auth-gssapi for Kerberos integrated auth on Unix by @v1gnesh in #11386
build(deps): bump the aws-sdk group across 1 directory with 5 updates by @app/dependabot in #11821
fix(cayenne): clear a committed CDC upsert's staging WAL at finalize (#12027) by @bjchambers in #12034
feat(catalog): deterministic instance-independent replication slot + fail-loud + restart recovery (#11850) by @bjchambers in #12026
Auto-cap DuckDB accelerator memory to prevent startup over-commit by @lukekim in #11985
perf(refresh): speed up dataset refresh at startup by @lukekim in #12015
fix(cayenne): don't create stray file: directory in memory mode (fixes #11922) by @claudespice in #11940
fix(aws): accept aws_session_token for temporary credentials (fixes #10932) by @claudespice in #12041
fix(mysql): verify an adopted layout against the event's own column types (fixes #11764) by @claudespice in #12048
fix(mysql): take the binlog rotate target from the ROTATE event, not its header offset (fixes #12042) by @claudespice in #12044
fix(search): check dataset readiness before embedding the query (fixes #10956) by @claudespice in #12043
fix(databricks): pin the authentication mode with databricks_auth_mode so an auto-loaded client secret can't switch U2M to M2M (fixes #11508) by @claudespice in #12040
fix(imap): narrow scans with IMAP SEARCH instead of refetching the mailbox (fixes #11548) by @claudespice in #12039
fix(cayenne): rank inference-derived sort columns below observed filter columns by @lukekim in #12049
perf(cayenne): warm-subset compaction + mem-tier memory_limit honesty by @lukekim in #12035
fix(cayenne): purge the mem-tier on delete-all for a table without a primary key (fixes #12072) by @claudespice in #12073
fix(accelerator): let an index declare a finalize failure fatal so a stale index can't report a successful refresh (fixes #12038) by @claudespice in #12050
fix(http): restore a join's embedded projection in HTTP subquery pushdown (fixes #11009) by @claudespice in #12054
fix(e2e): report a REPL that exits mid-script instead of passing the step (fixes #12057) by @claudespice in #12059
fix(imap): fetch the raw message only when a scan reads content (fixes #12045) by @claudespice in #12060
chore: Update Turso crate to 0.7.1 by @claudespice in #12062
feat(chbench): in-memory _bench_ts watermarks for the MySQL HTAP staleness probe by @sgrebnov in #12055
test(chbench): default MySQL adaptive spicepods to one shared binlog dump by @sgrebnov in #12076
feat(search): warm in-memory writethrough/fallback index for .vectors datasets and views by @Jeadie in #11914
fix(search): expunge superseded documents from full-text BM25 statistics (fixes #12053) by @claudespice in #12056
fix(write-back): gate durable write-back on a safe source delivery primitive (fixes #11915) by @claudespice in #12051
fix(cayenne): keep protected snapshots a position-delete rewrite never folded in (fixes #11477) by @claudespice in #12052
ci(signoff): fast-track Attestation for Dependabot and non-Rust PRs, and cut the local gate by @lukekim in #12081
fix(search): reach a compound's inner full-text tier when a change stream attaches (fixes #12061) by @claudespice in #12063
fix(accelerator): report PK equality pushdown Inexact so a point lookup can't return the whole table (fixes #12070) by @claudespice in #12071
fix(github): paginate issues and pull_requests on an immutable sort key so a row touched mid-scan isn't dropped (fixes #12067) by @claudespice in #12069
fix(otel): scope reserved metric column names to the data-point shape (fixes #12064) by @claudespice in #12065
fix(cayenne): sample the subset compaction append fence before the listing it guards (fixes #12074) by @claudespice in #12075
fix(search): drop a search hit whose row is not in the base table (fixes #12089) by @claudespice in #12094
feat(datasets): mark non-accelerated datasets Error when their source is unavailable by @krinart in #12079
fix(models): restore spice.ai/spiceai as a model source by @lukekim in #12092
fix(cayenne): bound the cold PK-index rebuild; build shard views in one pass by @sgrebnov in #12078
fix: Update Search integration test snapshots by @Jeadie in #12077
feat(catalog): view warnings, docs-linked messages, and clearer startup summary for PostgreSQL catalog CDC (#11850) by @bjchambers in #12022
fix(postgres): log the cumulative delivery wait in the sink-stall warning by @sgrebnov in #12093
refactor(cloud-connect): single-source the sealed-secret wire crypto in a shared crate by @peasee in #12124
build(deps): bump quinn-proto from 0.11.14 to 0.11.16 by @app/dependabot in #12037
perf(cayenne): demand-driven scan-view cache with access-based freshness (alternative to #11948) by @bjchambers in #12005
build: exclude libnfs from the nextest workspace run by @phillipleblanc in #12131
build(deps): bump the aws-sdk group across 1 directory with 3 updates by @app/dependabot in #12068
fix(search): keep CDC changes streams working when embeddings wrap a CDC source by @Jeadie in #12086
fix(cli): report a truncated release download as a failed download (fixes #12120) by @claudespice in #12121
fix(postgres): honor inline PEM pg_sslrootcert on the replication path by @phillipleblanc in #12128
refactor(layering): extract RuntimeStatus into a runtime-status crate by @bjchambers in #12115
fix(libnfs): model AUTH as an opaque type so its layout assertion holds by @phillipleblanc in #12132
feat(duckdb): on_full_refresh: replace_file — full refresh into a new database file, atomically replaced by @lukekim in #12135
Add v2.1.2 release notes by @sgrebnov in #12146
feat(duckdb): 'on_full_refresh: checkpoint_file' to bound acceleration file growth by @sgrebnov in #12139
fix(lint): remove unused DuckDBTableWriter import in duckdb accelerator tests by @sgrebnov in #12158
fix(duckdb): drop a file-replacement test assertion that can never hold by @bjchambers in #12178
fix(cayenne): Remove redundant EmptyExec within UnionExec used by CayenneTableProvider::scan by @Jeadie in #12126
build(deps): bump async-openai for aggregated rate-limit logging by @phillipleblanc in #12149
perf(mysql): fast row-image decoder for CDC change builds by @sgrebnov in #12122
feat(catalog): pre-flight replication-slot-capacity check for PostgreSQL catalog CDC (#11850) by @bjchambers in #12114
fix(oracle): map DATE to a timestamp so the time-of-day is not silently truncated (fixes #12096) by @claudespice in #12097
fix(search): skip the warm vector index when nothing hydrates it (fixes #12101) by @claudespice in #12103
refactor(layering): extract the DataAccelerator contract into data-accelerator-api by @bjchambers in #12099
fix(cli): surface the registry error behind a failed spice add instead of blaming the archive (fixes #12116) by @claudespice in #12119
fix(ci): gate Rust checks on the lint config the gate itself reads (fixes #12111) by @claudespice in #12112
fix: Ensure executors report table statistics for distributed plan ordering by @peasee in #11854
fix: EMBED_UDF_NAME without models feature by @krinart in #12136
fix(openai): openai_responses_tools: web_search uses web_search, not web_search_preview by @Jeadie in #12142
fix(ci): cancel workflow runs left behind by superseded merge-queue batches (fixes #12170) by @claudespice in #12177
fix(runtime): return from a readiness wait when the runtime shuts down (fixes #12125) by @claudespice in #12165
perf(postgres): coalesce shared-slot CDC envelopes so a slow sink stops throttling the walsender by @bjchambers in #12147
fix(ci): make cargo_deny_advisories.yml parseable so the Rust advisory scan can run (fixes #12181) by @claudespice in #12182
fix(cluster): run GetTaskHistory under read-only SQL validation by @lukekim in #12174
fix(duckdb): count accelerated views in the coordinated memory budget (fixes #12123) by @claudespice in #12161
fix(ci): run the PR hygiene check on a GitHub-hosted runner by @claudespice in #12163
fix(cayenne): warm PK existence caches at first write; bound keyset memory during bulk load by @sgrebnov in #12133
ci: remote sign-off dispatches GitHub Actions only, never SSH to lab hosts by @lukekim in #12205
Standalone instances: spice connect enroll-only split + connect surface by @krinart in #12143
fix(search): Fix ordering to correct NDCG@k by @Jeadie in #12191
refactor(layering): carry component configuration into connectors and extract data-connector-api by @bjchambers in #12157
fix(logging): mute the candle embedding backend's GeLU notice below -vv by @phillipleblanc in #12194
test: redact connection context in federated explain snapshots by @bjchambers in #11800
Enable deletes for Index, SearchIndex, and VectorIndex by @Jeadie in #11961
feat(testoperator): trace probe latency breaches and percentiles under HTAP load by @sgrebnov in #12209
Add v2.1.1 and 2.1.2 as supported in SECURITY.md by @sgrebnov in #12190
fix(ci): seed the TPC-H Postgres benches from the fleet's dataset and schedule the catalog schema tests by @bjchambers in #12214
fix(cluster): restrict ExpandSecret to spicepod-referenced keys by @lukekim in #12156
test: redact endpoint-URL compute contexts in federated explain snapshots by @bjchambers in #12215
fix(testoperator): a fully-censored table floors worst-P99 at the discard cap by @lukekim in #12196
ci(e2e): tolerate any garbage type token in the duckdb_append parquet-rename race by @bjchambers in #12207
refactor(cloud-connect): update the proto contract for evolvability and versioning by @peasee in #12153
chore(cloud-connect): use port 443 for gateway addresses in the doc example and test fixtures by @phillipleblanc in #12252
fix(ci): authenticate the management API integration suite with secrets that exist (fixes #12184) by @claudespice in #12185
feat(search): add Recall@k, MRR@k, and Precision@k retrieval metrics by @Jeadie in #12193
fix(search): enable stemming by default for full-text search by @Jeadie in #12220
fix: Report query metrics when task history is disabled by @sgrebnov in #12227
fix(runtime): count accelerated views when gating the Cayenne compaction pool (fixes #12164) by @claudespice in #12171
fix(secrets): snapshot the store registry instead of holding its lock across awaits (fixes #12127) by @claudespice in #12166
fix(search): address Elasticsearch deletes by document _id (fixes #12267) by @claudespice in #12273
fix(postgres): deprecate pg_replication_temporary_slot, which could never work (fixes #12213) by @claudespice in #12265
fix(cayenne): always materialize a snapshot directory the catalog may reference (fixes #12208) by @claudespice in #12262
fix(duckdb): size the connection pool for accelerated views too (fixes #12160) by @claudespice in #12169
fix(ci): link brew formulas that are installed but unlinked in setup-cc by @claudespice in #12286
fix(runtime): size memory budgets from the process's own cgroup limit by @lukekim in #12263
Harden dataset availability checks against idle connection resets by @krinart in #12180
test(mysql): harden binlog CDC with dump-reconnect and full type-matrix coverage by @sgrebnov in #12264
fix: send enc_pubkey_pem on /renew for standalone runtime by @phillipleblanc in #12312
Connection-scale client modes for testoperator throughput and load tests by @lukekim in #12280
fix(search): remove all chunk documents when a chunked Elasticsearch index deletes a row (fixes #12088) by @claudespice in #12268
fix(duckdb): run spice_sys DuckDB sidecar writes on the blocking pool (fixes #12175) by @claudespice in #12204
fix(cayenne): the write-concurrency raise must respect the memory brake by @lukekim in #12317
feat(runtime): expose the memory numbers that explain an OOM as gauges by @lukekim in #12195
fix(flightsql): mark stamped statistics inexact when a LIMIT is pushed to the remote scan (fixes #12292) by @claudespice in #12293
fix(deps): bump wasmtime to 47.0.3 to clear RUSTSEC-2026-0222 (fixes #12295) by @claudespice in #12298
fix(telemetry): read the cgroup CPU quota along the whole cgroup path (fixes #12299) by @claudespice in #12300
refactor(layering): move the DataFusion helpers below runtime and derive the federation deny-list from a function registry by @bjchambers in #12210
fix(search): reject a persisted full-text index whose schema no longer matches the configuration (fixes #12274) by @claudespice in #12275
fix(spiced): keep dependency logging and chat progress alive when task history is off (fixes #12279) by @claudespice in #12281
fix(runtime): count a mid-stream response failure as a failure in the HTTP metrics (fixes #12284) by @claudespice in #12291
fix(cluster): mark a coordinator leaf scan's cached executor statistics inexact (fixes #12303) by @claudespice in #12304
fix(runtime): report a memory-pool refusal as ResourcesExhausted and answer it with 503 (fixes #12282) by @claudespice in #12289
fix(cayenne): enforce the sharded PK keyset byte budget; correct per-entry accounting by @lukekim in #12192
fix(runtime): carve the Cayenne compaction memory pool only when a dataset can compact into it (fixes #12320) by @claudespice in #12326
feat(spiced): report fatal signals before exit by @sgrebnov in #12334
bug: Filters on _match fail during planning by @Jeadie in #12247
bug: S3 metadata-filter conversion silently drops unconvertible AND/OR operands by @Jeadie in #12248
fix(ci): reject a failed sign-off on the head commit (fixes #12357) by @claudespice in #12362
ci: let signoff.yml sign off fork PRs by @Jeadie in #12025
fix(search): resolve an append stream through a vector scan (fixes #12313) by @claudespice in #12314
docs: Spice v2.1.3 release notes by @sgrebnov in #12378
bug: MemoryVectorIndex leaves external-store entries on delete. by @Jeadie in #12246
fix(cluster): keep the executor's cluster-service channel alive while idle (fixes #12301) by @claudespice in #12302
fix(https): reject OAuth2 params on structured HTTP file datasets (fixes #12315) by @claudespice in #12321
fix(scheduler): drive the interval-timing tests on a virtual clock (fixes #12323) by @claudespice in #12329
fix(ci): do not report a shutdown-cancelled sidecar task as a failure (fixes #12322) by @claudespice in #12331
fix(logging): let log colour follow the output sink (fixes #12327) by @claudespice in #12335
fix(cayenne): report the auto-tuned config once per resolution (fixes #12330) by @claudespice in #12341
chore: Update Turso crate to 0.7.2 by @claudespice in #12344
fix(runtime): stop retrying a dataset configuration failure that no retry can clear (fixes #12339) by @claudespice in #12345
build(deps): bump the github-actions-dependencies group across 3 directories with 9 updates by @app/dependabot in #12359
feat(llms): GLM 5.2 across 3+ nodes, context_length + paged_attention params, MXFP4 for DeepSeek-V4 by @lukekim in #11990
feat(runtime): size every CPU-derived pool from the CPU entitlement by @bjchambers in #12276
chore(ci): bump the spiceio setup action to v0.5.9 by @lukekim in #12391
feat(testoperator): serve /health and /v1/ready for the HTAP run by @lukekim in #12373
feat(catalog): durable storage modes for PostgreSQL catalog CDC, and let the catalog own its accelerated tables by @bjchambers in #12222
fix(runtime): correctly classify a memory-pool refusal as ResourcesExhausted (fixes #12380) by @sgrebnov in #12382
fix(ci): budget the sign-off run below the pool's job wall (fixes #12340) by @claudespice in #12343
fix(bench): install make before the seeded-database check; surface catalog error causes by @bjchambers in #12372
Clean up SQLite sidecar files in cold bloom catalog test by @sgrebnov in #12404
fix(ci): name an incoherent Cargo.lock before every cargo job fails on it (fixes #12375) by @grokspice in #12423
test(search): exercise the S3 Vectors warm-tier fallback path by @Jeadie in #12197
fix(ci): stop the spiced child E2E cleanup was leaking, scoped to this runner (fixes #12058) by @grokspice in #12431
perf(cayenne): give refresh_mode full its own write profile by @lukekim in #12338
fix(ci): skip the Attestation refresh when the sign-off SHA is no longer the PR head (fixes #12360) by @grokspice in #12433
fix(monitoring): correct p99 queries in the Datadog dashboard by @sgrebnov in #12444
fix(ci): tell an out-of-disk sign-off apart from a failing branch (fixes #12412) by @grokspice in #12426
feat(search): warm in-memory tier for chunked Elasticsearch vector columns by @Jeadie in #12082
Update spicepod.schema.json by @app/github-actions in #12451
fix(search): preserve field names with capitals and '.' by @Jeadie in #12389
fix(ci): size the cayenne property-test timeout ceiling to its measured runtime (fixes #12336) by @grokspice in #12438
fix(models): pass the pinned revision to a HuggingFace embedding model (fixes #12430) by @grokspice in #12446
fix(ci): make the out-of-disk sign-off verdict survive being out of disk (fixes #12427) by @grokspice in #12461
fix(ci): stop Enforce Pulls with Spice racing itself, and let a Dependabot PR pass it (fixes #12377) by @grokspice in #12466
docs: update v2.1.3 release notes for the Iceberg timestamptz fix (trunk sync) by @phillipleblanc in #12459
fix(ci): install protoc before building the spice CLI in the E2E CLI workflow by @phillipleblanc in #12449
fix(ci): reject an if: naming a context GitHub does not provide there (refs #12396) by @grokspice in #12468
fix(monitoring): align Datadog dashboard with OTel service instance identity by @ewgenius in #12454
chore(ci): bump the spiceio setup action to v0.5.10 by @lukekim in #12490
test(cayenne): query result correctness vs standalone engines and Spice accelerators by @lukekim in #12098
fix(elasticsearch): keep the response body out of errors on the row-data request paths (fixes #12409) by @grokspice in #12456
fix(ci): key the sign-off concurrency group on the commit, not the dispatch input (fixes #12472) by @grokspice in #12474
perf(ci): run the workspace unit-test gate in one cargo invocation by @bjchambers in #12437
feat(monitoring): Runtime Resources observability on Grafana dashboard by @ewgenius in #12441
fix(ci): report cache health on the build failures it explains (refs #12420) by @grokspice in #12471
fix: resolve fixed-offset timezones when writing Vortex files by @phillipleblanc in #12463
fix(search): stop re-parsing a dotted index key as relation.column (fixes #12462) by @grokspice in #12464
fix(ci): build the ADBC BigQuery driver with the Go version its go.mod requires by @grokspice in #12470
fix(llms): keep the Hugging Face cache token and harden the model E2E job by @grokspice in #12422
fix(ci): stop a cancelled Remote Sign-off from claiming the checks failed (fixes #12424) by @grokspice in #12425
fix(runtime): fail the write when an index cannot prepare its write window (fixes #12421) by @grokspice in #12448
fix(runtime): stop retrying a permanent configuration failure on the catalog and dataset load paths (fixes #12417) by @grokspice in #12443
chore(ci): bump spiceio setup action to v0.6.0 by @lukekim in #12522
fix(ci): say what was unready when a spiced readiness wait times out (refs #12473) by @grokspice in #12484
feat(cayenne): drive incremental vacuum from the maintenance tick by @lukekim in #12435
feat(ci): allow additional cargo features for spiceai-dev Docker builds by @ewgenius in #12198
fix(cli): authenticate the SQL REPL's nql line like the session's SQL (fixes #12491) by @grokspice in #12496
fix(search): keep the row's primary key out of Elasticsearch bulk-index failures (fixes #12370) by @claudespice in #12410
fix(cpu-budget): state a low CPU request plainly, and only below half the cores by @bjchambers in #12440
fix(models): reject a pinned Model2Vec revision instead of 401ing on it (fixes #12445) by @grokspice in #12475
fix(cli): keep a login credential on the origin it was minted for (fixes #12505) by @grokspice in #12508
bug: S3 vector indexing succeeds when the embedding column is absent by @Jeadie in #12249
bug: S3 Vectors advertises unsupported metadata predicates as exact pushdown (fixes #12243) by @Jeadie in #12250
fix(cli): treat a blank --api-key as no key at all (fixes #12498) by @grokspice in #12501
task: Add #[derive(TypedParams)] for secret stores. by @Jeadie in #12154
fix(runtime): report a deferred dataset that cannot build its connector (fixes #12414) by @grokspice in #12483
fix(mysql): pre-flight CDC replication privileges with an actionable error (fixes #11967) by @grokspice in #12485
fix(testoperator): reject a client fleet the HTTP executors cannot bound (fixes #12348) by @claudespice in #12349
fix(ci): commit the catalog schema snapshot baselines and stop the daily PR churn by @phillipleblanc in #12525
refactor(runtime): describe every table-provider wrapper layer once in a layer table by @phillipleblanc in #12200
Add testoperator dispatch support for search benchmarks and replace benchmarks_search.yml by @Jeadie in #12224
bug: RRF discards a candidate stream after an empty first batch (fixes #12239) by @Jeadie in #12384
fix(cli): stop the SQL REPL asking a runtime its queries never went to (fixes #12493) by @grokspice in #12494
test(runtime): run the metrics tests that are built but never run by @sgrebnov in #12510
fix(tests): assert the search cache status instead of wall-clock timings (fixes #12487) by @grokspice in #12488
fix(search): resolve a chunked index's entries from the authoritative store, not the read listing (fixes #12266) by @claudespice in #12411
fix(search): Improvements for search harness by @Jeadie in #12226
test(cayenne): cover the selective PK join and the inline/file tier boundary by @lukekim in #12517
fix(runtime): surface the unknown-connector suggestion, and report it once (fixes #12415) by @grokspice in #12469
fix(ci): publish no sign-off verdict when the run was signalled (fixes #12518) by @grokspice in #12544
refactor(layering): move the modules the accelerated table shares below runtime by @bjchambers in #12219
chore(ci): upgrade spiceio setup action to v0.7.0 by @lukekim in #12611
fix(runtime): keep the leading slash on an inferred local Iceberg warehouse root (fixes #12533) by @grokspice in #12540
fix(ci): let a cancelled sign-off correct only the failure it posted (fixes #12428) by @grokspice in #12432
fix(runtime): compare an append high-water mark inclusively on a day-granular time column (fixes #12492) by @grokspice in #12500
fix(runtime): report a dataset connector load failure once, not once per site (fixes #12365) by @grokspice in #12526
fix(workers): give the weighted-router test a noise margin it can survive (fixes #12537) by @grokspice in #12538
fix(runtime): reject a Hadoop table URL too short to name a namespace (fixes #12539) by @grokspice in #12542
fix(ci): probe a pinned Hugging Face revision at a URL the Hub has (fixes #12553) by @grokspice in #12554
feat(testoperator): pin the load phase to a target query rate by @lukekim in #12519
fix(cli): follow only same-origin redirects, so the API key cannot leave the origin by @grokspice in #12503
fix(ci): use spiceio and sccache for E2E macOS aarch64 builds by @lukekim in #12590
perf(cdc): build a drained CDC burst with one blocking-pool handoff by @sgrebnov in #12514
feat(cloud-connect): complete the spice connect surface — install, codeless connect, remove by @peasee in #12159
fix(scripts): distinguish merge-queue PRs in signoff mine by @Jeadie in #12383
bug: HTTP RRF truncates each candidate leg before fusion by @Jeadie in #12386
fix(cayenne): pair the cold manifest with the warm snapshot a scan captured by @phillipleblanc in #12577
fix(deps): bump the Vortex pin and guard the task-cancellation regression by @phillipleblanc in #12548
chore(deps): bump datafusion and table-providers for the DuckDB timezone and retention fixes by @phillipleblanc in #12545
fix(cayenne): surface the real error for partitioned Cayenne writes by @Jeadie in #12535
fix(postgres): re-snapshot in-memory Cayenne on slot resume, and match the slot's lifetime to the accelerator's by @bjchambers in #12221
fix(cli): bound the Spice.ai login poll instead of retrying forever (fixes #12506) by @grokspice in #12523
fix(runtime): name a UTC timestamp column's zone in a spelling DuckDB knows (fixes #12528) by @grokspice in #12534
fix(mysql): raise net_write_timeout on the shared binlog dump session (fixes #12527) by @claudespice in #12586
fix(google-genai): read a Gemini SSE stream as bytes, so a split character survives (fixes #12597) by @claudespice in #12601
fix(mssql): follow an availability group's read-only routing redirect (fixes #11453) by @claudespice in #12607
test(cayenne): measure small-file fan-out against the seeded appends, not a listed count (fixes #12602) by @claudespice in #12613
fix(ci): keep the build cache's S3 endpoint out of the runtime under test (fixes #12624) by @claudespice in #12626
perf(ci): verify the CLI binary instead of building it again by @bjchambers in #12486
fix(postgres): publish a member's held CDC envelope before a bulk transaction (fixes #12311) by @claudespice in #12408
fix(cayenne): consolidate the two partition creators, fixing the accelerator's missing directory sync (fixes #12212) by @claudespice in #12619
fix(cayenne): run a backend-parameterized test on a stack its plan fits in (fixes #12436) by @grokspice in #12561
fix(ci): let only the merge queue pass Attestation without a sign-off (fixes #12679) by @grokspice in #12681
fix(ci): take the trusted sign-off helpers from the workflow's own commit (fixes #12657) by @claudespice in #12662
fix(ci): name the E2E jobs that did not succeed (fixes #12643) by @grokspice in #12649
fix(ci): give the openai model job a ceiling above its own setup (fixes #12644) by @grokspice in #12650
fix(ci): recognise the Flight bind wording so a foreign ready cannot pass (fixes #12642) by @grokspice in #12648
fix(ci): call an unreachable compiler cache infrastructure, not a failing branch (fixes #12556) by @grokspice in #12557
fix(search): clear a replacing index instead of keeping rows the source dropped (fixes #12066) by @grokspice in #12564
fix(ci): retry the sign-off status post, and say so when it never lands (fixes #12701) by @grokspice in #12704
build(deps): bump aws-smithy-runtime from 1.12.0 to 1.12.1 in the aws-sdk group across 1 directory by @app/dependabot in #12358
task: Add #[derive(TypedParams)] for LLMs. (fixes #12150) by @Jeadie in #12155
fix(ci): let a nightly's later test suites survive an earlier one's failure (fixes #12625) by @claudespice in #12627
fix(tests): wait for a partition to persist before snapshotting its plan (fixes #12645) by @grokspice in #12652
fix(object-store): bound an FTP and SFTP connection attempt end to end (fixes #12647) by @grokspice in #12655
test(iceberg): decide the Hadoop catalog test's backends by environment (fixes #12646) by @grokspice in #12665
fix(ci): give each E2E model job its own spiced ports (fixes #12419) by @grokspice in #12685
fix(cache): evict the Pingora cache down to max_size instead of only recording it (fixes #12688) by @grokspice in #12694
fix(cayenne): bound protected-snapshot compaction by the compaction memory pool by @sgrebnov in #12541
fix(ci): leave an incomplete sign-off pending, not failed (fixes #12741) by @grokspice in #12742
fix(telemetry): resolve duration histograms below a millisecond (fixes #12693) by @grokspice in #12699
fix(ci): bound every E2E job, so a wedged build cannot hold the queue (fixes #12717) by @grokspice in #12719
feat(testoperator): add new search benchmark datasets by @Jeadie in #12237
test(cayenne): name the small-file compaction tests after the path they take (fixes #12612) by @grokspice in #12740
fix(cli): bound an inference call by silence, not by total duration (fixes #12583) by @claudespice in #12589
fix(turso): store and read back the same set of types (fixes #12628) by @claudespice in #12633
fix(catalogs): keep a Glue database whose tables the include patterns select (fixes #12630) by @claudespice in #12638
fix(catalogs): apply a catalog's exclude patterns, not just its include (fixes #12636) by @claudespice in #12641
fix(cayenne): build a carry-forward rewrite from the classified manifest rows (fixes #12708) by @claudespice in #12711
fix(runtime): subtract an append dedup window as a multiset, not a set (fixes #12499) by @grokspice in #12513
bug: FTS exec drops absent columns and violates its declared schema (fixes #12228) by @Jeadie in #12245
bug: rrf() linear recency decay divides the document age by the decay window twice (fixes #12232) by @Jeadie in #12385
chore(deps): bump datafusion rev to spiceai-54 with outer-join unparser fix by @Jeadie in #12683
fix(ci): recognise sccache's startup-timeout wording as an unusable cache (fixes #12622) by @claudespice in #12788
fix(spiced): open the log with the startup banner by @bjchambers in #12615
fix(metrics): resolve S3 Vectors latency below a hundred milliseconds (fixes #12698) by @grokspice in #12702
fix(google-genai): bound one unterminated SSE event so a stalled endpoint cannot grow spiced (fixes #12600) by @grokspice in #12690
feat(cpu-budget): size a burstable pod from its CPU request, with an all-cores opt-out by @bjchambers in #12581
fix(ci): refuse a sign-off the branch's Makefile cannot run (fixes #12813) by @claudespice in #12815
test(postgres): say which readiness wait timed out and what it saw (fixes #12730) by @grokspice in #12731
fix(mssql): bound one connection attempt, so a stalled peer cannot pin a pool slot (fixes #12606) by @grokspice in #12733
fix(ci): ask whether the disk watcher runs before make writes to it (fixes #12734) by @grokspice in #12736
fix: delete three source files that no crate root declares (fixes #12735) by @grokspice in #12738
fix(telemetry): keep a startup-recorded gauge on the operator meter, so it reaches /metrics (fixes #12667) by @grokspice in #12754
fix(cloud-connect): pin rustls where the release call presents its identity (fixes #12760) by @grokspice in #12764
test(cloud-client): assert the redirect policy by behaviour against a live server (fixes #12509) by @grokspice in #12767
fix(search): decline a warm vector tier the accelerator cannot refill (fixes #12102) by @grokspice in #12768
fix(ci): build without the compiler cache when it cannot be reached (fixes #12770) by @grokspice in #12771
fix(json): end an array element where serde did, so bare scalars read (fixes #12782) by @grokspice in #12785
fix(cache): count an invalidation as an eviction, and export the counters before one fires (fixes #12687) by @grokspice in #12791
fix(ci): ask whether the runner can hold the build before starting one (fixes #12798) by @grokspice in #12799
fix(ci): size the cayenne property-test ceiling above the pool, not at it (fixes #12811) by @grokspice in #12814
fix(ci): bound every merge-queue-required job with timeout-minutes by @grokspice in #12817
deps: bump datafusion-table-providers to 894e279 (fixes #12585) by @bjchambers in #12663
fix(ci): take stale build output off the runners, not just report a full one (fixes #12800) by @grokspice in #12801
fix(cayenne): give the Turso metastore sidecars one pool over cayenne.db (fixes #12727) by @grokspice in #12804
fix(deps): drop the unreferenced ctor dev-dependency from data_components (fixes #12664) by @grokspice in #12819
fix(ci): decline a verdict when the sign-off run itself was signalled (fixes #12710) by @grokspice in #12724
fix(postgres): hold a resuming shared slot's ack floor for tables with no member by @bjchambers in #12676
refactor(layering): split the table-provider layers out of runtime by @bjchambers in #12661
build(deps): bump the github-actions-dependencies group across 3 directories with 7 updates by @app/dependabot in #12842
fix(spiced): make the crash handler more robust by @sgrebnov in #12834
fix(catalogs): apply the Cayenne catalog's include and exclude patterns (fixes #12766) by @grokspice in #12833
fix(search): decide a compound index's write fatality per half, not by OR (fixes #12826) by @grokspice in #12828
feat(cloud): organization context for spice cloud commands by @lukekim in #12515
perf(postgres catalog): skip metadata queries for schemas no include pattern can reach by @bjchambers in #12651
bug: S3 vector metadata conversion silently drops filterable metadata (fixes #12240) by @Jeadie in #12387
build(deps): bump nvidia/cuda from 13.3.0-cudnn-runtime-ubuntu24.04 to 13.3.1-cudnn-runtime-ubuntu24.04 in the docker-dependencies group by @app/dependabot in #12356
fix(runtime): keep serving when the working directory cannot be walked (fixes #6301) by @claudespice in #12803
fix(connectors): report an object-store timeout as a timeout, not as bad credentials (fixes #12793) by @claudespice in #12797
test(runtime): say which components were not ready when the wait times out (refs #12396) by @grokspice in #12827
fix(cayenne): measure tuner memory pressure as unreclaimable demand (fixes #12531) by @sgrebnov in #12623
Include v2.1.3 and 2.1.4 in SECURITY.md by @sgrebnov in #12497
fix(search): leave the stored rows in place when a delete cannot be applied (fixes #12822) by @grokspice in #12825
fix(google-genai): resume the SSE scan where it stopped, not at the buffer start (fixes #12689) by @grokspice in #12713
fix(json): report a malformed JSON array element instead of dropping the rest of the file (fixes #12755) by @claudespice in #12777
fix(runtime): honour a declared type's precision instead of narrowing it (fixes #12756) by @claudespice in #12774
fix(cayenne): drop the partition-value guard that key encoding made unreachable (fixes #12616) by @claudespice in #12753
fix(runtime): pin the catalog-acceleration contract in both feature configurations (fixes #12743) by @claudespice in #12752
fix(cli): report an answer the model stopped early, instead of printing it as whole (fixes #12596) by @grokspice in #12715
fix(runtime): scope query jobs and active queries to the principal that submitted them by @phillipleblanc in #12841
Compute all metrics@k for all 0 < k <= n during search quality metrics. by @Jeadie in #12521
task: Add a truncate to huggingface and file embedding providers by @Jeadie in #12620
fix(cache): never serve a result whose tables changed after it read them by @bjchambers in #12703
test(cayenne): cover datalake-tier statistics, pruning, and promotion triggers by @sgrebnov in #12847
docs(ci): warn that re-running enforce-pull-with-spice replays a stale payload (fixes #12809) by @grokspice in #12810
fix(cayenne): bound the maintained-aggregate index by memory budget, and let a stale registry recover by @lukekim in #12573
fix(runtime): only initialize accelerators for the datasets a reload applies by @phillipleblanc in #12872
feat: promote BYOC and Cloud Connect changes to trunk by @phillipleblanc in #12852
fix(flight): resolve per-request settings from the live app by @phillipleblanc in #12873
fix(cli): keep polling when the token exchange has not seen the auth code yet by @phillipleblanc in #12871
feat(cayenne): inline a small whole-table refresh into the metastore by @lukekim in #12367
ci: upgrade spiceio setup action to v0.8.0 by @lukekim in #12885
fix(runtime): measure the CDC prefetch backlog, and stop its byte counter wrapping by @lukekim in #12675
fix(turso): keep a reloaded file-accelerated dataset queryable by @phillipleblanc in #12882
fix(query): preserve expression ordering across partition wrappers by @Jeadie in #12854
fix(runtime): report every start-time-only runtime.* section a reload changes by @phillipleblanc in #12874
fix(search): handle null embeddings in vector writes and JIT search by @Jeadie in #12855
test(embeddings): truncate over-length inputs in the MiniLM embed spicepods by @Jeadie in #12856
fix(search): Improve search benchmarking DX and fix some vector index names by @Jeadie in #12892
test(search): dispatch all MTEB search variants by @Jeadie in #12850
fix(runtime): stop a superseded scheduler incarnation from claiming the heartbeat key by @phillipleblanc in #12851
fix: Ensure OTel ingest materializes missing/null dimension columns by @peasee in #12087
feat(cloud-connect): apply a component-only Spicepod deployment in place by @phillipleblanc in #12895
feat(cli): typed login sessions and explicit connect organization selection by @phillipleblanc in #12912
refactor: make each table wrapper a typed layer that answers where a walk goes by @bjchambers in #12891
feat(runtime): put every query's trace id on its log records by @lukekim in #12610
feat(cayenne): bound the SUM of the per-table PK keyset caches by @lukekim in #12802
fix(cache): read and expire a Pingora entry under one hold of its shard (fixes #12832) by @grokspice in #12839
fix: Update Search integration test snapshots by @Jeadie in #12707
fix(runtime): invalidate localpod children's cached results on parent refresh by @krinart in #12897
fix(cayenne): use valid snapshot IDs for deferred appends by @Jeadie in #12853
fix(ci): New branch for each push-snap-changes (only affects integration_search.yml) by @Jeadie in #12684
fix(search): make chunked embedding and offset columns nullable (#12778) by @Jeadie in #12783
feat(cayenne): charge scan materialization to the query memory pool by @lukekim in #12759
fix(cache): run the Pingora invalidation scan off the calling runtime worker (fixes #12806) by @grokspice in #12808
fix(cache): count the removals the Pingora engine performs itself (fixes #12792) by @grokspice in #12830
fix(cayenne): reserve append sequence for partitioned upsert appends (#12779) by @Jeadie in #12784
test(cayenne): prove the keyset degrade releases each shard as it converts by @lukekim in #12790
perf(postgres catalog): resolve a schema's table schemas in one query (#12106) by @bjchambers in #12886
fix(vortex): expose segment cache metrics on scrape by @phillipleblanc in #12944
refactor: point connector imports at the crates that actually own them by @bjchambers in #12925
fix(postgres): rebuild the acceleration when a replication slot's history is gone by @bjchambers in #12922
fix(cayenne)!: only an explicit cayenne_tuning: adaptive enables the closed loop by @lukekim in #12949
fix(dev-tools): show merge-queue and conflict state as labels in scripts/signoff mine by @Jeadie in #12924
test: cover warm-index delete paths across backends (#11964) by @Jeadie in #12635
fix(cayenne): fractional proptest scaling + batch timestamp-partition test inserts by @Jeadie in #12796
Helpful tool to track what users/bots have got done lately by @Jeadie in #12603
Release notes housekeeping by @krinart in #12977
build(deps): bump serde_with from 3.20.0 to 3.21.0 by @app/dependabot in #11969
refactor: make runtime's re-export shims crate-visible by @bjchambers in #12969
fix(runtime): surface OTel metric tables with duplicate columns, and count unsupported metric types as rejected by @peasee in #12923
fix: Update Search integration test snapshots by @app/github-actions in #12979
Make signoff.yml more searchable. by @Jeadie in #12392
bug: Support search UDTFs in SQL function components by @Jeadie in #12919
docs: document the stacked-PR workflow, with a tested restack helper by @bjchambers in #12951
fix(llms): fall back to pytorch_model.bin for embeddings without safetensors by @Jeadie in #12971
feat(monitoring): improve the Datadog dashboard for multi-replica and Kubernetes deployments by @sgrebnov in #12972
Charge the query memory pool for concurrent Vortex split decodes by @lukekim in #12940
docs(release-notes): write release notes in Simplified Technical English by @lukekim in #12947
refactor: narrow the DataConnector trait off the runtime's types by @bjchambers in #12993
fix(cayenne): compact a DDL-created partition on an interval, not only on write (fixes #12617) by @grokspice in #12757
fix(cayenne): hand out a pooled Turso connection in autocommit (refs #12820) by @grokspice in #12821
fix: fail the lint gate on a source file no crate root declares (fixes #12737) by @grokspice in #12744
fix(search): stage a full refresh of the memory vector tier, so dropped rows leave it (refs #12413) by @grokspice in #12823
fix(graphql): keep the configured schema when a page has no rows (fixes #13004) by @claudespice in #13016
fix(runtime): register task_history before embeddings/rerankers load by @Jeadie in #12978
test(postgres): converge the catalog on source DDL, and warn when it selects nothing by @bjchambers in #12988
fix(cayenne): budget for catalogs in the Cayenne memory classification by @bjchambers in #13027
fix(cayenne): hold primary keys committed while the existence index is checked out by @lukekim in #13019
feat(search): push SQL filters down into tantivy full-text search by @Jeadie in #12812
fix(cayenne): serve the maintained count Inexact while its delta is queued (fixes #12824) by @grokspice in #12829
fix: Update Search integration test snapshots by @app/github-actions in #13060
fix(testoperator benchmarks): benchmark fixes by @krinart in #13012
fix(runtime): stop query_active_count drifting up when queries share a request (fixes #12883) by @claudespice in #13034
chore(deps): bump model2vec-rs for fast WordPiece tokenizer by @Jeadie in #12986
ci(cayenne-doc): render diagrams locally with mermaid-cli, and fix the diagram that has been failing the build for a month by @bjchambers in #13063
feat(monitoring): improve results cache panels on the Datadog dashboard by @sgrebnov in #13066
feat(cayenne)!: one process-wide Vortex segment cache instead of a cache per table by @bjchambers in #12983
fix(connectors): keep the projected schema when a query returns no rows (fixes #13015) by @claudespice in #13078
fix(deps): bump the datafusion pin past the merged unparser fixes (fixes #12406) by @claudespice in #13083
fix(cache): compact a sliced result before caching it (fixes #12921) by @claudespice in #13043
refactor(cdc): give the sidecar checkpoint stores a below-runtime interface by @bjchambers in #13045
fix(cayenne): re-baseline the maintained row count when a promotion folds a delete tombstone (fixes #12846) by @claudespice in #12998
fix(acceleration): stop an engine-required type rewrite reading as a stale acceleration schema (fixes #13014) by @claudespice in #13074
fix(udfs): return NULL for a vector distance that is not defined (refs #11263) by @claudespice in #13091
fix(cayenne): refuse a data directory that contains the metastore (fixes #13055) by @claudespice in #13101
fix(json): report content after a JSON array instead of dropping it (fixes #12786) by @claudespice in #13097
fix(runtime): bound the spicepod apply so an unloadable dataset cannot wedge it (fixes #12862) by @claudespice in #13095
fix(catalogs): apply the Glue catalog's exclude patterns (fixes #12634) by @claudespice in #13103
fix(metrics): make component metrics and tests more robust by @sgrebnov in #13094
feat(cache)!: ship the Pingora cache engine as an Enterprise-only feature by @lukekim in #13000
fix: Update benchmark snapshots by @app/github-actions in #13056
fix(postgres): compare CDC positions against what the slot can stream, not what it retains by @bjchambers in #12990
docs(cayenne): compress the changelog, and give it a rule for what earns a row by @bjchambers in #13067
test: remove a test's container when it is done with it, including on failure by @bjchambers in #13123
fix(flight): record one metric sample per Flight RPC (fixes #12844) by @claudespice in #13112
feat(drasi): forward CDC changes and runtime tables to a Drasi source (Alpha) by @lukekim in #12653
fix(bench): time the engine instead of the results cache, and validate ClickBench by @lukekim in #13150
refactor(connector): retype the DataConnector surface to DatasetSpec by @bjchambers in #13096
bug: SQL-tier function body does not support named arguments (=>) for search UDTFs (fixes #12898) by @Jeadie in #12920
fix(cache): compact the sliced and wide results that compaction previously skipped by @sgrebnov in #13176
docs(stacked-prs): correct when a restack forfeits the sign-off by @bjchambers in #13130
fix(postgres): load a provably empty CDC acceleration through its snapshot bootstrap by @bjchambers in #13175
fix(cayenne): keep Arrow timestamp units (Vortex supports ns) by @lukekim in #13180
Cloud Connect: enrollment and identity foundation by @phillipleblanc in #13181
refactor(model): give the model-provider contracts a home below llms by @bjchambers in #13201
Fix runtime lint with zero features by @krinart in #13208
fix(benchmarks): BigQuery + Snowflake fixes + updated snapshots by @krinart in #13136
Revert "fix(search): decline a warm vector tier the accelerator cannot refill (fixes #12102)" (#12768) by @Jeadie in #13041
fix(snapshots): don't bootstrap the snapshot file_create just discarded by @lukekim in #13179
Declare the CPU entitlement to Vortex at startup by @sgrebnov in #13173
fix(mysql): fix net_write_timeout session syntax and extend transient replication errors by @ewgenius in #13178
fix(postgres): classify EOF and server termination errors as transient in CDC replication by @ewgenius in #13163
feat(testoperator): support customer-supplied datasets in run search by @Jeadie in #12982
Add PDF page splitter and FinanceBench staging job (#12858) by @Jeadie in #12984
fix(cayenne): frame the persisted PK bloom with the probe function that filled it (fixes #13137) by @claudespice in #13153
refactor(layering): move the connectors below the runtime by @bjchambers in #13166
feat(cli): manage the Cloud Connect service with launchd on macOS by @phillipleblanc in #13203
Support local rerankers from text-embeddings-inference by @Jeadie in #12929
fix(catalog): make the PostgreSQL catalog's errors and warnings actionable by @bjchambers in #13199
fix(cloud-connect): look again when a lock create reports the entry missing by @phillipleblanc in #13231
Skip DynamoDB in scheduled benchmarks by @ewgenius in #13222
Populate the file metadata cache with Vortex footers at write time by @sgrebnov in #13228
perf(cayenne): give the PK filter a cache-line layout, and the bit count it asks for by @bjchambers in #13219
fix(mysql): rebuild a purged-position acceleration atomically instead of emptying it (fixes #12967) by @claudespice in #13023
fix(cloud-connect): heartbeat restart-required, attachment by app id, and this crate's integration tests in the gate by @phillipleblanc in #13234
fix(search): join all of a dataset's vector indexes on one VectorScanTableProvider by @krinart in #13209
Validate vector search parameters before SQL query by @lesbass in #12253
feat(helm): support customizing Deployment strategy and StatefulSet updateStrategy by @sgrebnov in #13249
fix(postgres): recover from a replication slot lost while streaming, and state what each slot costs by @bjchambers in #13221
fix(cayenne): stop a partitioned acceleration reporting a schema change it never applied (fixes #12999) by @claudespice in #13057
feat(cloud-connect): name the local spicepod a cloud-managed instance serves or ignores by @phillipleblanc in #13262
fix(snowflake): keep NUMBER precision and scale during schema discovery by @phillipleblanc in #13272
fix(secrets): check secret references where the components resolve them by @peasee in #13265
docs(criteria): sign off the PostgreSQL Catalog Connector at Beta by @bjchambers in #13235
fix(cloud-connect): resolve locked service state through the retained directory descriptor (fixes #13204) by @claudespice in #13292
chore(deps): upgrade iceberg-rust to v0.10.0 by @krinart in #13189
fix(cloud-connect): fix Windows build of identity state-file helpers by @sgrebnov in #13308
fix(cayenne): normalize append de-duplication types by @ewgenius in #13200
feat(cli): unify Spice Cloud enrollment and service workflows by @phillipleblanc in #13326
feat(postgres): native upsert delivery for durable write-back by @Jeadie in #13323
feat(cli): create an unattached Cloud Connect project by @phillipleblanc in #13333
fix(cayenne): exclude protected snapshots above the delete fence from subset compaction by @krinart in #13343
fix(cli): resolve a Cloud Connect project's data-plane region from its config by @krinart in #13370
fix(cli): keep a granted Spice Cloud login when the identity endpoint is silent by @bjchambers in #13376
fix(duckdb): fix concurrent-query failures after a replace_file swap with multiple DuckDB files by @Jeadie in #13383

*Full Changelog: https://github.com/spiceai/spiceai/compare/v2.1.0...v2.2.0

Spice v1.11.5 (Apr 1, 2026)

April 1, 2026 · 4 min read

Sergei Grebnov

Senior Software Engineer at Spice AI

Announcing the release of Spice v1.11.5! 🛠️

Spice v1.11.5 is a patch release improving on_zero_results: use_source fallback performance, Delta Lake timestamp predicate data skipping, S3 Parquet read performance, PostgreSQL partitioned table support, Cayenne target file size handling, and preparing the CLI for v2.0 runtime upgrades.

What's New in v1.11.5

`on_zero_results: use_source` Fallback Performance Improvement

Improved the on_zero_results: use_source fallback path to run DataFusion's physical optimizer on the federated scan plan (#9927). The fallback path now runs SessionState::physical_optimizers() rules on the federated scan plan before execution, enabling parallel file group scanning and other optimizations. This results in significantly faster fallback queries on multi-core machines, particularly for file-based data sources like Delta Lake.

Delta Lake: Improved Data Skipping for `>=` Timestamp Predicates

Delta Lake table scans with >= timestamp filters now correctly prune files that do not match the predicate (#9932), improving query performance through more effective data skipping (file-level pruning).

PostgreSQL: Partitioned Tables Support

The PostgreSQL data connector now supports partitioned tables (#9997) for both federated and accelerated queries.

S3 Parquet Read Performance Improvement

Improved parquet read performance from S3 and other object stores (#10064), particularly for tables with many columns. Column data ranges are now coalesced into fewer, larger requests instead of being fetched individually, reducing the number of HTTP round-trips.

Cayenne: Ensure Target File Size is Respected

The Cayenne accelerator now correctly respects the configured target file size (#10071). Previously, Cayenne could produce many small, fragmented Vortex files; with this fix, files are written at the expected target size, improving storage efficiency and query performance.

CLI: Support for v2.0 Runtime Upgrades

The Spice CLI can now upgrade to v2.0 runtime versions. This enables upgrading to v2.0 release candidates and, once released, the v2.0 stable runtime.

spice upgrade v2.0.0-rc.1

Running spice upgrade without a version will upgrade to the latest stable version, including v2.0 once released.

Note: Native Windows runtime builds will no longer be provided in v2.0. Use WSL for local development instead.

Contributors

Breaking Changes

No breaking changes.

Cookbook Updates

No new cookbook recipes.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v1.11.5, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.11.5 image:

docker pull spiceai/spiceai:1.11.5

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.11.5

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed

Changelog

fix(runtime): Run physical optimizer on FallbackOnZeroResultsScanExec fallback plan by @sgrebnov in #9927
fix(delta_lake): Fix data skipping for >= timestamp predicates by @sgrebnov in #9932
fix(PostgreSQL): Fix schema discovery for PostgreSQL partitioned tables by @sgrebnov in #9997
fix(cli): Skip models variant download for v2+ in upgrade/install by @lukekim and @sgrebnov in #10052
perf(s3): Improve Parquet read performance by @sgrebnov in #10064
fix(cayenne): Ensure Cayenne respects target file size by @krinart in #10071

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.11.4...v1.11.5

Spice v1.0-rc.4 (Jan 6, 2025)

January 6, 2025 · 5 min read

Phillip LeBlanc

Co-Founder and CTO of Spice AI

Happy New Year 🎆!

Announcing the release of Spice v1.0-rc.4 🌟

Spice v1.0.0-rc.4 is the fourth release candidate for the first major version of Spice.ai OSS. This release continues the focus on production readiness. In addition, xAI has been added as a model provider.

Highlights in v1.0-rc.4

xAI Model Provider: Adds support for xAI hosted models.

models:
  - from: xai:grok2-latest
    name: xai
    params:
      xai_api_key: ${secrets:SPICE_XAI_API_KEY}

Spicepod Spec Version: Spicepod spec version v1 is now by default. v1beta1 will continue to work.

version: v1
kind: Spicepod
name: my_pod

GitHub Data Connector: Graduated to Stable.
PostgreSQL Data Accelerator: Graduated to Release Candidate (RC).

Cookbook

Added xAI model provider recipe.

Dependencies

No major dependency changes.

Contributors

@lukekim
@phillipleblanc
@peasee
@karifabri
@sgrebnov
@Jeadie
@ewgenius

What's Changed

- Update openapi.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4087>
- Update Helm chart for v1.0.0-rc.3 (v0.2.2) by @lukekim in <https://github.com/spiceai/spiceai/pull/4088>
- Rev version to v1.0.0-rc.4 by @lukekim in <https://github.com/spiceai/spiceai/pull/4090>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4089>
- Fix OpenAI Models Integration tests by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4084>
- fix: Update Postgres TPCDS and ClickBench queries by @peasee in <https://github.com/spiceai/spiceai/pull/4092>
- fix: Check Postgres acceleration schema on insert by @peasee in <https://github.com/spiceai/spiceai/pull/4094>
- Update v1.0.0-rc.3.md by @karifabri in <https://github.com/spiceai/spiceai/pull/4096>
- Update openapi.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4093>
- First-class TSV for file data connector by @lukekim in <https://github.com/spiceai/spiceai/pull/4098>
- Allow Flight DoPut only for write api-keys by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4010>
- Only create tables `eval.runs` and `eval.results` when an eval is defined by @Jeadie in <https://github.com/spiceai/spiceai/pull/4099>
- Update Copyright year to include 2025 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4100>
- feat: add postgres clickbench accelerator, release postgres accelerator by @peasee in <https://github.com/spiceai/spiceai/pull/4111>
- Add spice binaries with metal to releases; detect metal device in `spice install/upgrade`. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4097>
- docs: Clarify connector release criteria by @peasee in <https://github.com/spiceai/spiceai/pull/4112>
- Update datafusion-federation to fix LIMIT with OFFSET handling in logical plan rewrite by @ewgenius in <https://github.com/spiceai/spiceai/pull/4115>
- Support Grok AI. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4113>
- Fix `spice chat` usage bar. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4119>
- Set unified max encoding and decoding message size for all flight client configurations across runtime by @ewgenius in <https://github.com/spiceai/spiceai/pull/4116>
- feat: Add the file connector as an appendable benchmark connector by @peasee in <https://github.com/spiceai/spiceai/pull/4120>
- Add `spice eval` command by @lukekim in <https://github.com/spiceai/spiceai/pull/4118>
- Support multi-level table nesting for Dremio by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4129>
- feat: run append TPCH benchmarks in workflow (Arrow, DuckDB) by @peasee in <https://github.com/spiceai/spiceai/pull/4131>
- Fix bug in Iceberg tables selecting a subset of columns by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4132>
- feat: Run append TPCDS benchmarks in workflow (Arrow, DuckDB) by @peasee in <https://github.com/spiceai/spiceai/pull/4141>
- Setup spice.ai clickbench by @ewgenius in <https://github.com/spiceai/spiceai/pull/4134>
- Data is streamed when reading from the GitHub connector (GraphQL tables) by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4142>
- Mark the GitHub Data Connector as Stable by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4143>
- Fix table quoting for Databricks Spark connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4145>
- Extend flight compute context for spice.ai connector with org and app names, to fix federated queries from different spice.ai data sources by @ewgenius in <https://github.com/spiceai/spiceai/pull/4144>
- Enforce Flight DoPut policies: Rate Limiting, Read Timeout, and Max Records per Batch by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4117>
- Fix bug Changes in catalog.yaml would require saving in spicepod.yaml to apply by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4147>
- Update benchmark snapshots by @github-actions in <https://github.com/spiceai/spiceai/pull/4137>
- Add `test-framework` crate to contain all common benchmark, E2E, integration testing logic. by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4157>
- Fix `platform_option` variable in `build_and_release.yml`. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4154>
- feat: Add Clickbench append benchmark for DuckDB and Arrow by @peasee in <https://github.com/spiceai/spiceai/pull/4160>
- Upload artifacts to Minio on build_and_release by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4159>
- feat: add on zero results benchmark by @peasee in <https://github.com/spiceai/spiceai/pull/4164>
- Update spice.ai connector tests by @ewgenius in <https://github.com/spiceai/spiceai/pull/4161>

**Full Changelog**: <https://github.com/spiceai/spiceai/compare/v1.0.0-rc.3...v1.0.0-rc.4>
```text

## Resources

- [Getting started with Spice.ai](https://docs.spiceai.org/getting-started/)
- [Documentation](https://docs.spiceai.org/)

## Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

- Twitter: [@spice_ai](https://twitter.com/spice_ai)
- Slack: [spiceai.org/slack](https://spiceai.org/slack)
- Telegram: [Spice AI Discussion](https://t.me/spiceaichat)
- Reddit: [https://www.reddit.com/r/spiceai](https://www.reddit.com/r/spiceai)
- Email: [hey@spice.ai](mailto:hey@spice.ai)

Spice v0.20-beta (Nov 4, 2024)

November 4, 2024 · 4 min read

Phillip LeBlanc

Co-Founder and CTO of Spice AI

Announcing the release of Spice v0.20-beta 🧩

Spice v0.20.0-beta improves federated query performance with column pruning and adds support for Metal (Apple Silicon) and CUDA (NVidia) accelerators. The S3, PostgreSQL, MySQL, and GitHub Data Connectors have graduated from Beta to Release Candidates. The Arrow, DuckDB, and SQLite Data Accelerators have graduated from Alpha to Beta.

Highlights in v0.20.0-beta

Data Connectors: The S3, PostgreSQL, MySQL, and GitHub Data Connectors have graduated from beta to release candidate.

Data Accelerators: The Arrow, DuckDB, and SQLite Data Accelerators have graduated from alpha to beta.

Metal and CUDA Support: Added support for Metal (Apple Silicon) and CUDA (NVidia) for AI/ML workloads including embeddings and local LLM inference.

For instructions on compiling a Meta or CUDA binary, see the Installation Docs.

Breaking Changes

The ODBC Data Connector now requires ODBC drivers specified in connection strings are registered in the system ODBC driver manager.

Example invalid connection string:

DRIVER={/path/to/driver.so};SERVER=localhost;DATABASE=master

Example valid connection string:

DRIVER={My ODBC Driver};SERVER=localhost;DATABASE=master

Where My ODBC Driver is the name of an ODBC driver registered in the ODBC driver manager.

Contributors

@ewgenius
@peasee
@phillipleblanc
@sgrebnov
@Jeadie
@barracudarin
@Sevenannn

What's Changed

- Update Helm for v0.19.4-beta and add release notes by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3310>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/3311>
- `metal` & `cuda` flags for spice by @Jeadie in <https://github.com/spiceai/spiceai/pull/3212>
- Promote postgres connector to RC quality by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3305>
- docs: Update ROADMAP.md by @peasee in <https://github.com/spiceai/spiceai/pull/3322>
- feat: Enable federation for in-memory accelerators by @peasee in <https://github.com/spiceai/spiceai/pull/3325>
- fix: Only allow env files from the current dir by @peasee in <https://github.com/spiceai/spiceai/pull/3327>
- Always read TimezoneTZ from PostgreSQL as UTC by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3330>
- For multi-sink acceleration refreshes, ensure parent table completes before the children. by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3329>
- Update TPC-DS Q49 (Decimal to Float) to match SQLite's type system by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3323>
- Enable parquet pushdown in Spice by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3245>
- Use spice object_store fork to fix S3 ambiguous error by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3304>
- Don't mix commented out queries for s3 connectors and accelerators by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3331>
- Allow only valid WHERE conditions in vector searches by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3335>
- fix: Allow only ODBC profiles by @peasee in <https://github.com/spiceai/spiceai/pull/3324>
- Track how many times an acceleration falls back during initialization by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3339>
- Anthropic model regex and fix tool parsing aggregation bug by @Jeadie in <https://github.com/spiceai/spiceai/pull/3334>
- Upgrade runtime along with CLI on `spice upgrade` by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3341>
- Update upcoming Roadmap by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3343>
- fix: Prevent acceleration files outside of working directory by @peasee in <https://github.com/spiceai/spiceai/pull/3340>
- Document S3 connector limitations by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3333>
- Update Object Store Patch by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3361>
- Promote SQLite Data Accelerator to Beta by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3365>
- Promote S3 connector to RC quality by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3362>
- Revert "fix: Only allow env files from the current dir" by @peasee in <https://github.com/spiceai/spiceai/pull/3368>
- docs: Fix typo for S3 release status in README.md by @peasee in <https://github.com/spiceai/spiceai/pull/3370>
- Include unnecessary columns pruning step during federated plan creation by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3363>

**Full Changelog**: <https://github.com/spiceai/spiceai/compare/v0.19.4-beta...v0.20.0-beta>

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: hey@spice.ai

Spice v0.17.2-beta (August 26, 2024)

August 26, 2024 · 8 min read

Phillip LeBlanc

Co-Founder and CTO of Spice AI

Announcing the release of Spice v0.17.2-beta 🏄

The v0.17.2-beta release focuses on improving data accelerator compatibility, stability, and performance. Expanded data type support for DuckDB, SQLite, and PostgreSQL data accelerators (and data connectors) enables significantly more data types to be accelerated. Error handling and logging has also been improved along with several bugs.

Highlights in v0.17.2-beta

Expanded Data Type Support for Data Accelerators: DuckDB, SQLite, and PostgreSQL Data Accelerators now support a wider range of data types, enabling acceleration of more diverse datasets.

Enhanced Error Handling and Logging: Improvements have been made to aid in troubleshooting and debugging.

Anonymous Usage Telemetry: Optional, anonymous, aggregated telemetry has been added to help improve Spice. This feature can be disabled. For details about collected data, see the telemetry documentation.

To opt out of telemetry:

Using the CLI flag:
```
spice run -- --telemetry-enabled false
```
Add configuration to spicepod.yaml:
```
runtime:
  telemetry:
    enabled: false
```

Improved Benchmarking: A suite of performance benchmarking tests have been added to the project, helping to maintain and improve runtime performance; a top priority for the project.

Breaking Changes

None.

Contributors

@Jeadie
@y-f-u
@phillipleblanc
@sgrebnov
@Sevenannn
@peasee
@ewgenius

What's Changed

Dependencies

DataFusion: Upgraded from v40 to v41

Commits

- Pin actions/upload-artifact to v4.3.4 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2200>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/2202>
- Update to next release version, `v0.17.2-beta` by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2203>
- add accelerator beta criteria by @y-f-u in <https://github.com/spiceai/spiceai/pull/2201>
- update helm chart to 0.17.1-beta by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2205>
- add dockerignore to avoid copy target and test folder by @y-f-u in <https://github.com/spiceai/spiceai/pull/2206>
- add client timeout for deltalake connector by @y-f-u in <https://github.com/spiceai/spiceai/pull/2208>
- Upgrade tonic and opentelemetry-proto by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2223>
- Add index and resource tuning for postgres ghcr image to support postgres benchmark in sf1 by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2196>
- Remove embedding columns from `retrieved_primary_keys` in v1/search by @Jeadie in <https://github.com/spiceai/spiceai/pull/2176>
- use file as db_path_param as the param prefix is trimmed by @y-f-u in <https://github.com/spiceai/spiceai/pull/2230>
- use file for sqlite db path param by @y-f-u in <https://github.com/spiceai/spiceai/pull/2231>
- docs: Clarify the global requirement for local_infile when loading TPCH by @peasee in <https://github.com/spiceai/spiceai/pull/2228>
- Revert pinning actions/upload-artifact@v4 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2232>
- Runtime tools to chat models by @Jeadie in <https://github.com/spiceai/spiceai/pull/2207>
- Create `runtime.task_history` table for queries, and embeddings by @Jeadie in <https://github.com/spiceai/spiceai/pull/2191>
- chore: Update Databricks ODBC Bench to use TPCH SF1 by @peasee in <https://github.com/spiceai/spiceai/pull/2238>
- Replace `metrics-rs` with OpenTelemetry Metrics by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2240>
- fix: Remove dead code by @peasee in <https://github.com/spiceai/spiceai/pull/2249>
- Improve tool quality and add vector search tool by @Jeadie in <https://github.com/spiceai/spiceai/pull/2250>
- fix missing partition cols in delta lake by @y-f-u in <https://github.com/spiceai/spiceai/pull/2253>
- download file from remote for delta testing by @y-f-u in <https://github.com/spiceai/spiceai/pull/2254>
- feat: Set SQLite DB path to .spice/data by @peasee in <https://github.com/spiceai/spiceai/pull/2242>
- Support tools for chat completions in streaming mode by @ewgenius in <https://github.com/spiceai/spiceai/pull/2255>
- Load component `description` field from spicepod.yaml and include in LLM context by @ewgenius in <https://github.com/spiceai/spiceai/pull/2261>
- Add parameter for `connection_pool_size` in the Postgres Data Connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2251>
- Add primary keys to response of `DocumentSimilarityTool` by @Jeadie in <https://github.com/spiceai/spiceai/pull/2263>
- run queries bash script by @y-f-u in <https://github.com/spiceai/spiceai/pull/2262>
- Run benchmark test on schedule by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2277>
- feat: Add a reference to originating App for a Dataset by @peasee in <https://github.com/spiceai/spiceai/pull/2283>
- Tool use & telemetry productionisation. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2286>
- Fix cron in benchmarks.yml by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2288>
- Upgrade to DataFusion v41 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2290>
- Chat completions adjustments and fixes by @ewgenius in <https://github.com/spiceai/spiceai/pull/2292>
- Define the new metrics Arrow schema based on Open Telemetry by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2295>
- OpenTelemetry Metrics Arrow exporter to `runtime.metrics` table by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2296>
- Calculate summary metrics from histograms for Prometheus endpoint by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2302>
- Add back Spice DF runtime_env during SessionContext construction by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2304>
- Add integration test for S3 data connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2305>
- Fix `secrets.inject_secrets` when secret not found. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2306>
- Intra-table federation query on duckdb accelerated table by @y-f-u in <https://github.com/spiceai/spiceai/pull/2299>
- Postgres federation on acceleration by @y-f-u in <https://github.com/spiceai/spiceai/pull/2309>
- sqlite intra table federation on acceleration by @y-f-u in <https://github.com/spiceai/spiceai/pull/2308>
- feat: Add `DataAccelerator::init()` for SQLite acceleration federation by @peasee in <https://github.com/spiceai/spiceai/pull/2293>
- Initial framework for collecting anonymous usage telemetry by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2310>
- Add gRPC action to trigger accelerated dataset refresh by @sgrebnov in <https://github.com/spiceai/spiceai/pull/2316>
- add `disable_query_push_down` option to acceleration settings by @y-f-u in <https://github.com/spiceai/spiceai/pull/2327>
- Remove `v1/assist` by @Jeadie in <https://github.com/spiceai/spiceai/pull/2312>
- bump table provider version to set the correct dialect for postgres writer by @y-f-u in <https://github.com/spiceai/spiceai/pull/2329>
- Send telemetry on startup by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2331>
- Calculate resource IDs for telemetry by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2332>
- Refactor `v1/search`: include WHERE condition, allow extra columns in projection. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2328>
- Add integration test for gRPC dataset refresh action by @sgrebnov in <https://github.com/spiceai/spiceai/pull/2330>
- Propagate errors through all `task_history` nested spans by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2337>
- Improve tools by @Jeadie in <https://github.com/spiceai/spiceai/pull/2338>
- update duckdb rs version to support more types: interval/duration/etc by @y-f-u in <https://github.com/spiceai/spiceai/pull/2336>
- feat: Add DuckDB accelerator init, attach databases for federation by @peasee in <https://github.com/spiceai/spiceai/pull/2335>
- Add query telemetry metrics by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2333>
- Add system prompts for LLMs; system prompts for tool using models. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2342>
- Fix benchmark test to keep running when there's failed queries by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2347>
- Tools as a spicepod first class citizen. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2344>
- Add `bytes_processed` telemetry metric by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2343>
- fix misaligned columns from delta lake by @y-f-u in <https://github.com/spiceai/spiceai/pull/2356>
- Emit telemetry metrics to `runtime.metrics`/Prometheus as well by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2352>
- Use UTC timezone for telemetry timestamps by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2354>
- Fix MetricType deserialization by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2358>
- Add dataset details to tool using LLMs; early check tables in vector search by @Jeadie in <https://github.com/spiceai/spiceai/pull/2353>
- Bump datafusion-federation/datafusion-table-providers dependencies by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2360>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/2362>
- fix: Disable DuckDB and SQLite federation by @peasee in <https://github.com/spiceai/spiceai/pull/2371>
- Fix system prompt in ToolUsingChat, fix builtin registration by @Jeadie in <https://github.com/spiceai/spiceai/pull/2367>
- fix: Use --profile release for benchmarks by @peasee in <https://github.com/spiceai/spiceai/pull/2372>
- nql parameter 'use' -> 'model' by @Jeadie in <https://github.com/spiceai/spiceai/pull/2366>

**Full Changelog**: <https://github.com/spiceai/spiceai/compare/v0.17.1-beta...v0.17.2-beta>

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: hey@spice.ai

Spice v0.15.2-alpha (July 15, 2024)

July 15, 2024 · 5 min read

Luke Kim

Founder and CEO of Spice AI

The v0.15.2-alpha minor release focuses on enhancing stability, performance, and introduces Catalog Providers for streamlined access to Data Catalog tables. Unity Catalog, Databricks Unity Catalog, and the Spice.ai Cloud Platform Catalog are supported in v0.15.2-alpha. The reliability of federated query push-down has also been improved for the MySQL, PostgreSQL, ODBC, S3, Databricks, and Spice.ai Cloud Platform data connectors.

Highlights in v0.15.2-alpha

Catalog Providers: Catalog Providers streamline access to Data Catalog tables. Initial catalog providers supported are Databricks Unity Catalog, Unity Catalog and Spice.ai Cloud Platform Catalog.

For example, to configure Spice to connect to tpch tables in the Spice.ai Cloud Platform Catalog use the new catalogs: section in the spicepod.yml:

catalogs:
  - name: spiceai
    from: spiceai
    include:
      - tpch.*

sql> show tables
+---------------+--------------+---------------+------------+
| table_catalog | table_schema | table_name    | table_type |
+---------------+--------------+---------------+------------+
| spiceai       | tpch         | region        | BASE TABLE |
| spiceai       | tpch         | part          | BASE TABLE |
| spiceai       | tpch         | customer      | BASE TABLE |
| spiceai       | tpch         | lineitem      | BASE TABLE |
| spiceai       | tpch         | partsupp      | BASE TABLE |
| spiceai       | tpch         | supplier      | BASE TABLE |
| spiceai       | tpch         | nation        | BASE TABLE |
| spiceai       | tpch         | orders        | BASE TABLE |
| spice         | runtime      | query_history | BASE TABLE |
+---------------+--------------+---------------+------------+

Time: 0.001866958 seconds. 9 rows.

ODBC Data Connector Push-Down: The ODBC Data Connector now supports query push-down for joins, improving performance for joined datasets configured with the same odbc_connection_string.

Improved Spicepod Validation Improved spicepod.yml validation has been added, including warnings when loading resources with duplicate names (datasets, views, models, embeddings).

Breaking Changes

None.

Contributors

@phillipleblanc
@peasee
@y-f-u
@ewgenius
@Sevenannn
@sgrebnov
@lukekim

What's Changed

Dependencies

Upgraded Apache DataFusion to v40.0.0.

Commits

Update to next release version v0.15.2-alpha by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1901
release: Update helm 0.15.1-alpha by @peasee in https://github.com/spiceai/spiceai/pull/1902
fix: Detect and error on duplicate component names on spiced (re)load by @peasee in https://github.com/spiceai/spiceai/pull/1905
fix: flaky test - test_refresh_status_change_to_ready by @y-f-u in https://github.com/spiceai/spiceai/pull/1908
Add support for parsing catalog from Spicepod. by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1903
Add catalog component to Runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1906
Adds a RuntimeBuilder and make most items on Runtime private by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1913
Bump zerovec-derive from 0.10.2 to 0.10.3 by @dependabot in https://github.com/spiceai/spiceai/pull/1914
Add separate tagged image with enabled models feature by @ewgenius in https://github.com/spiceai/spiceai/pull/1909
Update datafusion-table-providers to use newest head by @Sevenannn in https://github.com/spiceai/spiceai/pull/1927
Add MySQL support for TPC-H test data generation script by @sgrebnov in https://github.com/spiceai/spiceai/pull/1932
fix: Expose ODBC task errors if error is before data stream begins by @peasee in https://github.com/spiceai/spiceai/pull/1924
Use public.ecr.aws/docker/library/{postgres/mysql}:latest for integration test images by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1934
Implement spice.ai CatalogProvider by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1925
fix: validate time column and time format when constructing accelerated table refresh by @y-f-u in https://github.com/spiceai/spiceai/pull/1926
Add support for filtering tables included by a catalog by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1933
Add UnityCatalog catalog provider by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1940
Implement Databricks catalog provider by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1941
Copy params into dataset_params by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1947
Make integration tests more stable by using logged-in registry during CI by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1955
Add integration test for Spice.ai catalog provider by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1956
Add GET /v1/catalogs API and catalogs CMD by @lukekim in https://github.com/spiceai/spiceai/pull/1957
feat: Enable ODBC JoinPushDown with hashed connection string by @peasee in https://github.com/spiceai/spiceai/pull/1954
Fix bug: arrow acceleration reports zero results during refresh by @sgrebnov in https://github.com/spiceai/spiceai/pull/1962
Revert "fix: validate time column and time format when constructing accelerated table refresh" by @y-f-u in https://github.com/spiceai/spiceai/pull/1964
fix: Update arrow-odbc to use our fork for pending fixes by @peasee in https://github.com/spiceai/spiceai/pull/1965
Upgrade to DataFusion 40 by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1963
Do exchange shouldn't require table to be writable by @Sevenannn in https://github.com/spiceai/spiceai/pull/1958
Use custom dialect rule for flight federated request by @y-f-u in https://github.com/spiceai/spiceai/pull/1946
upgrade datafusion federation to have the table rewrite fix for tpch-q9 by @y-f-u in https://github.com/spiceai/spiceai/pull/1970
Create v0.15.2-alpha.md Release notes by @digadeesh in https://github.com/spiceai/spiceai/pull/1969
Fix Unity Catalog API response for Azure Databricks by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1973
Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/1976

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.15.1-alpha...v0.15.2-alpha

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: hey@spice.ai

Spice.ai v0.10.1-alpha

April 1, 2024 · 2 min read

Luke Kim

Founder and CEO of Spice AI

Announcing the release of Spice v0.10.1-alpha! 🔥

The v0.10.1-alpha release focuses on stability, bug fixes, and usability by improving error messages when using SQLite data accelerators, improving the PostgreSQL support, and adding a basic Helm chart.

Highlights in v0.10.1-alpha

Improved PostgreSQL support for Data Connectors TLS is now supported with PostgreSQL Data Connectors and there is improved VARCHAR and BPCHAR conversions through Spice.

Improved Error messages Simplified error messages from Spice when propagating errors from Data Connectors and Accelerator Engines.

Spice Pods Command The spice pods command can give you quick statistics about models, dependencies, and datasets that are loaded by the Spice runtime.

Kubernetes Helm Deployment

Spice.ai can be deployed to Kubernetes using Helm. Here's a quick guide to get started:

Step 1. (Optional) Start a local kind cluster:

go install sigs.k8s.io/kind@v0.22.0
kind create cluster

Step 2. Install Spice in your Kubernetes cluster using Helm:

helm repo add spiceai https://helm.spiceai.org
helm install spiceai spiceai/spiceai

Step 3. Verify that the Spice pods are running:

kubectl get pods
kubectl logs deploy/spiceai

Step 4. Run the Spice SQL REPL inside the running pod:

kubectl exec -it deploy/spiceai -- spiced --repl

Learn more about deploying Spice.ai to Kubernetes

Contributors

@phillipleblanc
@mitchdevenport
@ewgenius
@sgrebnov
@lukekim
@digadeesh

New in this release

Adds Basic Helm Chart for spiceai (https://github.com/spiceai/spiceai/pull/1002)
Adds Support for spice login in environments with no browser. (https://github.com/spiceai/spiceai/pull/994)
Adds TLS support in Postgres connector. (https://github.com/spiceai/spiceai/pull/970)
Fixes Improve Postgres VARCHAR and BPCHAR conversion. (https://github.com/spiceai/spiceai/pull/993)
Fixes spice pods Returns incorrect counts. (https://github.com/spiceai/spiceai/pull/998)
Fixes Return friendly error messages for unsupported types in sqlite. (https://github.com/spiceai/spiceai/pull/982)
Fixes Pass Tonic errors when receiving errors from dependencies. (https://github.com/spiceai/spiceai/pull/995)

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: hey@spice.ai

Spice.ai v0.10-alpha

March 27, 2024 · 2 min read

Phillip LeBlanc

Co-Founder and CTO of Spice AI

Announcing the release of Spice v0.10-alpha! 🧙‍♂️

The Spice.ai v0.10-alpha release focused on additions and updates to improve stability, usability, and the overall Spice developer experience.

Highlights in v0.10-alpha

Public Bucket Support for S3 Data Connector: The S3 Data Connector now supports public buckets in addition to buckets requiring an access id and key.

JDBC-Client Connectivity: Improved connectivity for JDBC clients, like Tableau.

User Experience Improvements:

Friendlier error messages across the board to make debugging and development better.
Added a spice login postgres command, streamlining the process for connecting to PostgreSQL databases.
Added PostgreSQL connection verification and connection string support, enhancing usability for PostgreSQL users.

Grafana Dashboard: Improving the ability to monitor Spice deployments, a standard Grafana dashboard is now available.

Contributors

@phillipleblanc
@mitchdevenport
@Jeadie
@ewgenius
@sgrebnov
@y-f-u
@lukekim
@digadeesh

New in this release

Fixes Gracefully handle Arrow Flight DoExchange connection resets
Adds Grafana Dashboard
Adds Flight SQL CommandGetTableTypes Command support (improves JDBC-client connectivity)
Adds Friendlier error messages
Adds spice login postgres command
Adds PostgreSQL connection verification
Adds PostgreSQL connection string support
Adds Linux aarch64 build
Updates Improves spice status with dataset metrics
Updates CLI REPL improved show tables output
Updates CLI REPL limit output to 500 rows
Updates Improved README.md with architecture diagram updates
Updates Improved CI run time.
Updates Use macOS hosted Actions runner

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: hey@spice.ai

Spice.ai v0.9-alpha

March 15, 2024 · 2 min read

Luke Kim

Founder and CEO of Spice AI

Announcing the release of Spice v0.9-alpha! 🧙‍♂️

The v0.9 release adds several data connectors including the Spice data connector for the ability to connect to other Spice instances. Improved observability for the Spice runtime has been added with the new /metrics endpoint for monitoring deployed instances.

Highlights in v0.9-alpha

Arrow Flight SQL endpoint: The Arrow Flight endpoint now supports Flight SQL, including JDBC, ODBC, and ADBC enabling database clients like DBeaver or BI applications like Tableau to connect to and query the Spice runtime.

Spice.ai data connector: Use other Spice runtime instances as data connectors for federated SQL query across Spice deployments and for chaining Spice runtimes.

Keyring secret store: Use the operating system native credential store, like macOS keychain for storing secrets used by the Spice runtime.

PostgreSQL data connector: PostgreSQL can now be used as both a data store for acceleration and as a connector for federated SQL query.

Databricks data connector: Databricks as a connector for federated SQL query across Delta Lake tables.

S3 data connector: S3 as a connector for federated SQL query across Parquet files stored in S3.

Metrics endpoint: Added new /metrics endpoint for Spice runtime observability and monitoring with the following metrics:

- spiced_runtime_http_server_start counter
- spiced_runtime_flight_server_start counter
- datasets_count gauge
- load_dataset summary
- load_secrets summary
- datasets/load_error counter
- datasets/count counter
- models/load_error counter
- models/count counter

Contributors

New in this release

Adds Keyring secret store (keyring).
Adds PostgreSQL data connector (postgres).
Adds Spice.ai data connector (spiceai).
Adds Arrow Flight SQL (JDBC/ODBC/ADBC) support.
Adds Databricks data connector (databricks) - Delta Lake support.
Adds S3 data connector (s3) - Parquet support.
Adds /v1/models API.
Adds /v1/status API.
Adds /metrics API.

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: hey@spice.ai

Spice.ai v0.8-alpha

March 8, 2024 · One min read

Luke Kim

Founder and CEO of Spice AI

Announcing the release of Spice v0.8-alpha! 🏹

This is a minor release that builds on the new Rust-based runtime, adding stability and a preview of new features for the first major release.

Highlights in v0.8-alpha

Secrets management: Spice 0.8 runtime can now configure and retrieve secrets from local environment variables and in a Kubernetes cluster.

Data tables can be locally accelerated using PostgreSQL

New in this release

Adds Secrets management in local environment variables and Kubernetes clusters.
Adds (Preview) PostgreSQL as a data table acceleration engine.

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: hey@spice.ai

What's New in v2.2.0​

Cloud Connect​

MySQL Change Data Capture​

PostgreSQL Catalog CDC​

Debezium CDC Without Kafka​

Faster Search with Warm In-Memory Indexes​

Operations & Observability​

Other Notable Improvements and Bug Fixes​

SDK Updates​

Dependency Updates​

New Contributors​

Contributors​

Breaking Changes​

Cookbook Updates​

Upgrading​

What's Changed​

Changelog​

What's New in v1.11.5​

on_zero_results: use_source Fallback Performance Improvement​

Delta Lake: Improved Data Skipping for >= Timestamp Predicates​

PostgreSQL: Partitioned Tables Support​

S3 Parquet Read Performance Improvement​

Cayenne: Ensure Target File Size is Respected​

CLI: Support for v2.0 Runtime Upgrades​

Contributors​

Breaking Changes​

Cookbook Updates​

Upgrading​

What's Changed​

Changelog​

Highlights in v1.0-rc.4​

Cookbook​

Dependencies​

Contributors​

What's Changed​

Highlights in v0.20.0-beta​

Breaking Changes​

Contributors​

What's Changed​

Resources​

Community​

Highlights in v0.17.2-beta​

Breaking Changes​

Contributors​

What's Changed​

Dependencies​

Commits​

Resources​

Community​

Highlights in v0.15.2-alpha​

Breaking Changes​

Contributors​

What's Changed​

Dependencies​

Commits​

Resources​

Community​

Highlights in v0.10.1-alpha​

Kubernetes Helm Deployment​

Contributors​

New in this release​

Resources​

Community​

Highlights in v0.10-alpha​

Contributors​

New in this release​

Resources​

Community​

Highlights in v0.9-alpha​

Contributors​

New in this release​

Resources​

Community​

Highlights in v0.8-alpha​

New in this release​

Resources​

Community​

What's New in v2.2.0

Cloud Connect

MySQL Change Data Capture

PostgreSQL Catalog CDC

Debezium CDC Without Kafka

Faster Search with Warm In-Memory Indexes

Operations & Observability

Other Notable Improvements and Bug Fixes

SDK Updates

Dependency Updates

New Contributors

Contributors

Breaking Changes

Cookbook Updates

Upgrading

What's Changed

Changelog

What's New in v1.11.5

`on_zero_results: use_source` Fallback Performance Improvement

Delta Lake: Improved Data Skipping for `>=` Timestamp Predicates

PostgreSQL: Partitioned Tables Support

S3 Parquet Read Performance Improvement

Cayenne: Ensure Target File Size is Respected

CLI: Support for v2.0 Runtime Upgrades

Contributors

Breaking Changes

Cookbook Updates

Upgrading

What's Changed

Changelog

Highlights in v1.0-rc.4

Cookbook

Dependencies

Contributors

What's Changed

Highlights in v0.20.0-beta

Breaking Changes

Contributors

What's Changed

Resources

Community

Highlights in v0.17.2-beta

Breaking Changes

Contributors

What's Changed

Dependencies

Commits

Resources

Community

Highlights in v0.15.2-alpha

Breaking Changes

Contributors

What's Changed

Dependencies

Commits

Resources

Community

Highlights in v0.10.1-alpha

Kubernetes Helm Deployment

Contributors

New in this release

Resources

Community

Highlights in v0.10-alpha

Contributors

New in this release

Resources

Community

Highlights in v0.9-alpha

Contributors

New in this release

Resources

Community

Highlights in v0.8-alpha

New in this release

Resources

Community