Spice v1.10.0-rc.1 (Dec 2, 2025)
Announcing the release of Spice v1.10.0-rc.1! ⚡
v1.10.0-rc1 is a release candidate for early testing of v1.10 features including an all new caching acceleration mode, tiny_lfu caching policy, a new DynamoDB Streams connector (Preview), improvements to the DynamoDB connector, faster distributed query execution, S3 connector improvements, and security hardening for v1.10.0-stable.
What's New in v1.10.0-rc1
Caching Acceleration Mode with SWR and TinyLFU
This release introduces a new caching acceleration mode that implements the stale-while-revalidate (SWR) pattern using Data Accelerators such as DuckDB or Cayenne, enabling queries to return file-persisted cached results immediately while asynchronously refreshing data in the background. Combined with the new TinyLFU cache eviction policy, Spice can now maintain higher cache hit rates while keeping memory usage predictable.
Key Features:
- Stale-While-Revalidate (SWR): Returns cached data immediately while refreshing in the background
- Data Accelerator Support: Cached accelerators can persist data to disk using DuckDB, SQLite, or Cayenne file modes.
- TinyLFU Cache Policy: Probabilistic cache admission policy that maintains high hit rates with minimal overhead
- Predictable Memory Usage: Configurable memory limits with automatic eviction of less frequently used entries
Example Spicepod.yml configuration:
runtime:
caching:
sql_results:
enabled: true
eviction_policy: tiny_lfu # default lru
datasets:
- from: s3://my-bucket/data.parquet
name: cached_data
acceleration:
enabled: true
engine: duckdb
mode: file # Persist cache to disk
refresh_mode: caching
refresh_check_interval: 10m
For more details, refer to the Data Acceleration Documentation and Caching Documentation.
DynamoDB Streams Data Connector in Preview
DynamoDB Connector now integrates with DynamoDB Streams which enables real-time streaming with support for both table bootstrapping and continuous change data capture (CDC). This connector automatically detects changes in DynamoDB tables and streams them into Spice for real-time query, search, and LLM-inference.
Key Features:
- Real-Time CDC: Automatically captures inserts, updates, and deletes from DynamoDB tables
- Table Bootstrapping: Initial full table load before streaming changes
Example Spicepod.yml configuration:
datasets:
- from: dynamodb:my_table
name: orders_stream
acceleration:
enabled: true
refresh_mode: changes
For more details, refer to the DynamoDB Connector Documentation.
Cayenne Accelerator Enhancements
The Cayenne data accelerator now supports:
- Sort Columns Configuration: Optimize inserts by pre-sorting data on specified columns for improved query performance
Example Spicepod.yml configuration:
datasets:
- from: s3://my-bucket/data.parquet
name: sorted_data
acceleration:
enabled: true
engine: cayenne
mode: file_create
params:
sort_columns: timestamp,region
For more details, refer to the Cayenne Documentation.
S3 Connector Improvements
S3 Location Predicate Pruning: The S3 data connector now supports location-based predicate pruning, dramatically reducing data scanned by pushing down predicates to S3 listing operations. This optimization is especially effective for partitioned datasets stored in S3.
AWS S3 Tables Write Support: Full read/write capability for AWS S3 Tables, enabling fast integration with AWS's table format for S3.
For more details, refer to the S3 Tables Data Connector Documentation and Glue Data Connection Documentation.
Faster Distributed Query Execution
Distributed query planning and execution have been significantly improved:
- Fixed executor registration in cluster mode for more reliable distributed deployments
- Improved hostname resolution for Flight server binding, enabling better executor discovery
- Distributed accelerator registration: Data accelerators now properly register in distributed mode
- Optimized query planning:
DistributeFileScanOptimizerimprovements for faster planning with large datasets
For more details, refer to the Distributed Query Documentation.
Search Improvements
Search capabilities have been improved with several performance and reliability enhancements:
- Fixed FTS query blocking: Full-text search queries no longer block unnecessarily, improving query responsiveness
- Optimized vector index operations: Eliminated unnecessary
list_vectorscalls for better performance - Improved limit pushdown:
IndexerExecnow properly handles limit pushdown for more efficient searches
For more details, refer to the Search Documentation.
Security Hardening
Multiple security improvements have been implemented:
- SQL identifier quoting: Hardened SQL identifier quoting across all connectors to prevent injection attacks
- Token redaction: Sensitive tokens are now fully redacted in debug output to prevent credential leakage
- Path traversal prevention: Fixed tar extraction to prevent path traversal vulnerabilities
- Input sanitization: Added validation for
top_n_sampleorder_by parsing - Improved credential handling: Improved credential management in Glue connector
Developer Experience Improvements
- Health probe metrics: Added health probe latency metrics for better observability
- CLI improvements: Fixed
.clear historycommand in the REPL to fully clear persisted history
Contributors
Breaking Changes
No breaking changes.
Cookbook Updates
No major cookbook updates. The Spice Cookbook still offers 82+ recipes to help you prototype quickly.
Upgrading
To try v1.10.0-rc1, use one of the following methods:
CLI:
spice upgrade --version 1.10.0-rc1
Homebrew:
brew upgrade spiceai/spiceai/spice
Docker:
Pull the spiceai/spiceai:1.10.0-rc1 image:
docker pull spiceai/spiceai:1.10.0-rc1
For available tags, see DockerHub.
Helm:
helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.10.0-rc1
AWS Marketplace:
🎉 Spice is available in the AWS Marketplace.
What's Changed
Changelog
- Test-operator: Add tpcds_q8 to the default row-count validation skip list by @sgrebnov in #8185
- fix: Remove unwrap_used from test by @peasee in #8212
- Run glue_iceberg_integration_test_catalog as part of main integration tests by @sgrebnov in #8222
- Add TPCH sf100 testoperator spicepods with dispatch by @Jeadie in #8192
- Build with CPU native flags by @lukekim in #8224
- Make copilot check for empty copyright header by @krinart in #8245
- fix: Apply assertion clippy in CI/Makefile only by @peasee in #8229
- feat: Support running queries only in testoperator by @peasee in #8211
- DuckDB query planning: aggregate pushdown by @mach-kernel in #8174
- install.sh improvements by @lukekim in #8252
- Fix
.clear historyby @lukekim in #8254 - fix: Pushdown dynamic filters to partition scans by @peasee in #8240
- Harden SQL identifier quoting in connectors by @phillipleblanc in #8276
- Cayenne
sort_columnson insert by @lukekim in #8091 - Redact token debug output by @phillipleblanc in #8280
- fix: Cayenne configuration options by @lukekim in #8281
- Prevent path traversal in untar by @phillipleblanc in #8284
- Fix cluster mode executor registration by @mach-kernel in #8292
- Unignore s3_vectors_kafka_stream test by @Jeadie in #8289
- Post-release house keeping by @krinart in #8293
- Improve generate_changelog script by @krinart in #8273
- Acceleration mode
cachingby @lukekim in #8237 - Sanitization and security checks by @lukekim in #7854
- Add health probe latency metric by @phillipleblanc in #8300
- Add distributed registration for data accelerators by @phillipleblanc in #8299
- Pass IndexedTableProvider down in 'changes_stream' and 'append_stream' by @Jeadie in #8295
- Add
dynamodb-streamscrate by @krinart in #8283 - Distributed query: resolve executor hostname when determining Flight server binding by @mach-kernel in #8304
- Return computed embeddings from index for partitioned S3Vectors by @Jeadie in #8306
- [DDB Streams] Skeleton for DynamoDB Streams by @krinart in #8296
- DistributeFileScanOptimizer: Improve planning performance by @mach-kernel in #8305
- feat: Add an ExactLeftAccumulator implementation by @peasee in #8302
- deps: Upgrade Vortex to 0.56 by @peasee in #8311
- DynamoDB table bootstrapping + streaming by @krinart in #8312
- Avoid calling S3Vector
list_vectors(or equivalent) when indexing intoVectorIndexs by @Jeadie in #8282 - Add on_conflict testing support to append benchmark by @sgrebnov in #8314
- docker: Add valid home directory to fix duckdb extension loading issue by @phillipleblanc in #8318
- Add GH Workflow to run Append benchmark test by @sgrebnov in #8321
- Exclude MySQL SF100 from test-operator dispatch by @sgrebnov in #8320
- feat: Update clippy lints by @peasee in #8317
- Add S3 location predicate pruning to listing connector by @phillipleblanc
- Review feedback for caching mode accelerator by @phillipleblanc in #8326
- Also include Dockerfile home changes for release build by @phillipleblanc in #8327
- Change communication channel from Discord to Slack by @Jeadie in #8330
- Replace Discord link with Slack link in README by @Jeadie in #8331
- fix(glue): Prevent OpenDAL from automatic loading of AWS credentials from environment by @sgrebnov in #8337
- Block on index read for FTS queries by @Jeadie in #8339
- Fix search query provider by @Jeadie in #8343
- Support for writing into AWS S3 Tables by @sgrebnov in #8344
- Acceleration file_create mode by @lukekim in #8347
- Don't block on lock in FTS query path by @Jeadie in #8348
- feat: Add an optimizer rule to replace join accumulator for Cayenne by @peasee in #8316
- S3 Vectors limit updates by @lukekim in #8352
- Sanitize top_n_sample order_by parsing by @phillipleblanc in #8356
- Update version to v1.10.0-rc.1 by @ewgenius in #8362
- Improve
IndexerExecto properly handle limit pushdown by @sgrebnov in #8366 - Fix Cayenne partition_by metadata flaky integration test by @phillipleblanc in #8367
- Rework caching accelerator to use the
stale-while-revalidatepattern. by @phillipleblanc in #8365 - Add TinyLFU caching policy by @lukekim in #8370
