ScalarDL 3.13 introduces enhanced namespace support, giving organizations a powerful way to logically group and isolate assets, credentials, contracts, and functions within a single ScalarDL deployment. Whether you’re running a multi-tenant SaaS platform, managing data across departments, or optimizing infrastructure costs, namespaces give you the flexibility to do more with a single cluster.

Overview

In ScalarDL, a namespace is a logical boundary that contains its own set of resources:

• Assets: The immutable, tamper-evident data stored in the ledger.
• Credentials: Certificates or secret keys used for authentication.
• Contracts: The business logic that manages assets in the ledger.
• Functions: The business logic that works alongside contracts to manage mutable records in an external database.

By default, all resources live in a namespace called “default”. With the namespace enhancements in 3.13, you can now create additional namespaces and control how your application interacts with them.

How namespace support works

ScalarDL 3.13 introduces two access models for namespaces: the cross-namespace access model and the restricted access model. Each is designed for a different set of operational needs.

Cross-namespace access model

In the cross-namespace access model, a single application accesses assets across multiple namespaces. This is useful when you want logical data separation without requiring strict isolation between namespaces.

To enable cross-namespace access, you implement contracts by using namespace-aware APIs and register those contracts in the “default” namespace. Contracts registered in the “default” namespace have a global access scope, which means they can get, put, and scan assets in any namespace.

Namespace-aware contract APIs accept a namespace parameter in get, put, and scan calls, giving a single contract the ability to read and write assets across multiple namespaces. For implementation details, see Manage assets with a namespace.

Restricted access model

In the restricted access model, each namespace is independently managed and completely inaccessible from other namespaces. This is ideal for multi-tenant deployments where strict data isolation is a requirement. ScalarDL enforces this isolation through port-based access control: administrators manage namespaces and register credentials through a privileged port, while tenants connect through a separate non-privileged port and can only access their own namespace.

Setting up restricted access involves three steps:

1. Create a namespace by using the privileged administrative port.
2. Register tenant credentials (certificates or secret keys) to that namespace.
3. Configure the tenant client to target the namespace.

Because tenant credentials are registered to a specific namespace and the client is configured with a matching namespace property, all operations are automatically scoped to that namespace — tenants cannot access assets or execute contracts outside their assigned namespace.

Unlike in the cross-namespace model, contracts registered in a restricted-access namespace have only a local access scope — they cannot read or write assets in any other namespace.

To learn how to set up restricted access, see Access namespaces in a restricted manner.

Use cases

Namespaces in ScalarDL are flexible enough to support a range of organizational needs. Here are some common scenarios where they add the most value.

Use case 1: Multi-tenant and multi-application isolation

If you’re building a SaaS platform, you can host multiple customers on a single ScalarDL cluster while ensuring that each customer’s data and contracts are fully isolated. The same model works equally well for internal deployments — different departments or business units can each operate within their own namespace, reducing infrastructure costs while keeping data and contracts completely separate.

Use case 2: Data lifecycle management

For data that must be retained for a fixed period — such as financial records subject to regulatory requirements — namespaces let you organize assets by time period (for example, by year). When a retention period expires, you can bulk-delete an entire namespace rather than identifying and deleting individual records, greatly simplifying data governance.

Use case 3: Cost optimization through storage tiering

Not all data needs to live on the same storage system. By creating namespaces backed by storage systems with different performance and cost characteristics, you can place frequently accessed or high-value data on faster, more expensive storage while moving older or lower-priority data to cheaper tiers. Since contracts in the “default” namespace can access any namespace transparently, this can be done without changing application logic.

Best practices

• Plan your namespace structure before deploying: Changing namespace boundaries after data is stored can be disruptive. Take time upfront to design your namespace layout based on your access patterns, retention requirements, and organizational structure.
• Restrict access to the privileged port: The privileged port is used for namespace management and credential registration. In production environments, only administrators should have access to it. Tenants should connect exclusively through the non-privileged port.
• Use dedicated credentials per namespace: Never share credentials across namespaces in a restricted access setup. Each tenant should have their own certificate or secret key registered to their specific namespace.
• Register cross-namespace contracts in “default”: If you need contracts that span multiple namespaces, register them in the “default” namespace so they have global access scope. Contracts in non-default namespaces are limited to their own namespace.

Wrap-up

The enhanced namespace support in ScalarDL 3.13 makes it significantly easier to build scalable, secure, multi-tenant applications on a single ScalarDL infrastructure. With the cross-namespace and restricted access models, you have fine-grained control over how data is isolated, accessed, and governed — without sacrificing the tamper-evidence and Byzantine fault detection that ScalarDL provides.

To learn more about ScalarDL, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDL documentation.

If you are considering using ScalarDL, please contact us.

Securing and isolating data by using namespaces in ScalarDL was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Version 3.18 includes new features and improvements, including:

• Token-based access control with OpenID Connect (OIDC) for ScalarDB Cluster
• Google Spanner support in ScalarDB
• Authentication and authorization for ScalarDB Analytics

In this post, we’ll introduce these new features.

Token-based access control with OIDC for ScalarDB Cluster

As enterprise data infrastructure grows increasingly distributed, organizations need a way to manage who can access which data across multiple databases — ideally without maintaining a separate set of credentials for each system. ScalarDB Cluster 3.18 addresses this by adding support for token-based access control by using JSON Web Tokens (JWTs) issued by OIDC-compatible identity providers (IdPs).

With this feature, clients can authenticate to ScalarDB Cluster by using the access tokens they already obtain from their corporate IdP — such as Azure Active Directory, Okta, or any OIDC-compatible provider. ScalarDB Cluster validates the presented token and grants access based on the permissions associated with the corresponding user, without requiring separate credential management inside ScalarDB itself. Organizations can now leverage their existing identity infrastructure to manage data access across ScalarDB, simplifying administration and improving security posture.

For more details, see Control User Access via OIDC-Based JWT Access Tokens.

Google Spanner support in ScalarDB

ScalarDB 3.18 adds support for Google Spanner through the JDBC adapter, using the Spanner PostgreSQL dialect. This means Spanner can now be used as a backend database in ScalarDB, alongside the many other databases ScalarDB already supports.

Spanner is a fully managed, globally distributed relational database that offers strong consistency and high availability. Many organizations already rely on Spanner as part of their cloud-native infrastructure. By adding Spanner support to ScalarDB, organizations can now include their Spanner databases in multi-database transactions — for example, running a single ACID transaction that spans data stored in both Spanner and PostgreSQL, without custom two-phase commit logic in the application.

For more information on supported databases and how to configure them, see Requirements and Database Configurations.

Authentication and authorization for ScalarDB Analytics

ScalarDB Analytics 3.18 introduces authentication and authorization capabilities to the analytics layer, extending security governance beyond transactional operations to analytical queries as well.

Previously, ScalarDB Analytics connected to underlying data sources without enforcing user-level access controls at the ScalarDB Analytics layer. Organizations running analytical workloads across multiple databases now have a consistent way to control who can query which data sources through ScalarDB Analytics, using the same user and privilege model familiar from ScalarDB Cluster.

When the data being analyzed resides under ScalarDB Cluster, ScalarDB Analytics can optionally delegate authentication and authorization decisions to ScalarDB Cluster. This means access control policies defined once in ScalarDB Cluster automatically apply to both transactional and analytical access paths — making it easier to maintain a coherent security posture across your entire data infrastructure without duplicating policy configuration.

For more information, see the Authenticate and Authorize Users in ScalarDB Analytics.

Wrap-up

We have released ScalarDB 3.18, which includes the new features we mentioned in this blog post. For details about additional changes, please refer to the ScalarDB 3.18 release notes.

To learn more about ScalarDB, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDB documentation.

If you are considering using ScalarDB, please contact us.

ScalarDB 3.18 has been released! was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Version 3.13 includes various improvements, including the following new features and improvements:

• Namespace support for multi-tenant operations and logical data separation
• Java 21 compatibility for enhanced performance and stability

In this post, we’ll introduce these new features.

Namespace support for multi-tenant operations

ScalarDL has supported namespaces as a way to logically group and isolate assets, credentials, contracts, and functions within a ledger. In ScalarDL 3.13, namespace functionality is enhanced (currently as a Public Preview) to provide more flexible and powerful ways to manage and isolate data, applications, and access controls — especially for multi-tenant and large-scale environments.

Previously, operating multiple applications, departments, or customers on a single ScalarDL deployment required complex workarounds to achieve secure isolation and governance. It was difficult to enforce clear boundaries, which increased operational risk and often forced organizations to deploy separate ScalarDL clusters for each use case — raising infrastructure and maintenance costs.

With the enhanced namespace capabilities in 3.13, organizations can now more easily and securely manage different organizational units, customers, or use cases within a single ScalarDL infrastructure. The new access models and management options allow for true logical separation, making it possible to create isolated environments for departments, business units, or external customers, while simplifying operations and governance.

Key benefits of the enhanced namespace support include:

• Multi-tenant architecture: Host multiple customers or business units on a shared ScalarDL cluster while maintaining complete data isolation between tenants.
• Organizational separation: Separate data and applications by department, project, or organizational boundary.
• Lifecycle management: Organize data by retention periods or usage purposes for more efficient data management.
• Cost optimization: Consolidate multiple applications onto a single infrastructure while maintaining security boundaries.

For more details on namespace functionality, please refer to Manage namespaces and Access namespaces in a restricted manner.

Java 21 compatibility

ScalarDL 3.13 now supports Java 21, bringing the benefits of the latest Java runtime environment to your tamper-evident applications.

Java 21 is a Long Term Support (LTS) release that provides significant performance improvements, enhanced security features, and better memory management compared to earlier versions. By supporting Java 21, ScalarDL ensures that organizations can take advantage of these improvements while maintaining the robust tamper-evidence and Byzantine fault detection capabilities that ScalarDL is known for.

This compatibility upgrade enables organizations to:

• Leverage modern Java features: Access the latest language features and APIs for more efficient development.
• Improve application performance: Benefit from JVM optimizations and performance enhancements in Java 21.
• Enhance security posture: Take advantage of the latest security updates and improvements.
• Future-proof deployments: Ensure compatibility with modern Java-based toolchains and frameworks.

Together with the enhanced namespace capabilities, Java 21 compatibility significantly strengthens the position of ScalarDL as an enterprise-grade distributed ledger platform, enabling organizations to build more scalable, secure, and efficient enterprise systems while meeting demanding governance and compliance requirements.

For more information on Java version compatibility, see Requirements.

Other improvements

ScalarDL 3.13 also includes various performance optimizations, security enhancements, and stability improvements that contribute to a more robust and reliable platform for enterprise deployments.

For more information on ScalarDL features, see the following documentation:

• ScalarDL Overview
• Getting Started with ScalarDL
• How to Write Contracts
• Authentication Guide

Wrap-up

We have released ScalarDL 3.13, which includes the new features that we mentioned in this blog post. For details about additional changes, including the new features mentioned in this post, please refer to the ScalarDL 3.13 release notes.

To learn more about ScalarDL, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDL documentation.

If you are considering using ScalarDL, please contact us.

ScalarDL 3.13 has been released! was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Overview

HashStore and TableStore are high-level abstractions built on top of the low-level ledger abstraction of ScalarDL. Both eliminate the complexity of writing custom contracts while maintaining full tamper-evidence guarantees and Byzantine fault detection capabilities.

• HashStore specializes in digital evidence preservation by managing hash values of objects and tracking collection membership. Instead of storing entire files in the ledger, it stores cryptographic hash values with metadata, enabling you to prove that external objects haven’t been tampered with — ideal for compliance scenarios and audit trails.
• TableStore provides a SQL-based interface for managing structured data with complete audit trails. By using a PartiQL-based SQL dialect, it supports familiar operations like CREATE TABLE, INSERT, UPDATE, and SELECT while automatically maintaining tamper-evident version history.

Both use predefined contracts registered during a simple bootstrap process, allowing you to start building applications immediately without learning contract programming.

Choosing between HashStore and TableStore

Understanding when to use HashStore versus TableStore depends on your application requirements and data model.

Use HashStore when:

• You need to verify the authenticity of external files, documents, or objects without storing them in the ledger.
• You’re building compliance or audit systems that track evidence packages.
• Storage efficiency is critical and you only need to prove data hasn’t been tampered with.
• You need to manage collections of related objects with tamper-evident membership tracking.

Use TableStore when:

• You need to store and query structured data with complete audit trails.
• Your application requires SQL-like querying capabilities with joins and complex filters.
• You need to track the complete version history of individual records.
• You want to integrate tamper-evident storage with existing SQL-based applications and tools.

How HashStore works

HashStore manages the authenticity of objects — such as files, audit logs, or documents — without storing the actual data in the ledger. For a hands-on tutorial, see Get Started with ScalarDL HashStore.

Object authenticity management

Object authenticity management allows you to store and verify hash values of individual objects along with optional metadata. You calculate your object’s hash, store it with a unique object ID, then verify authenticity by comparing the current hash with stored values. HashStore maintains complete hash history, allowing you to verify any version and identify tampering.

Collection authenticity management

Collection authenticity management tracks which objects belong to a collection with an immutable history of all changes. This proves that specific document sets existed together — valuable for audit sets and compliance packages. All additions and removals are permanently recorded, preventing malicious actors from hiding evidence by quietly removing objects.

Synchronization with ScalarDB

HashStore integrates with ScalarDB, allowing you to synchronize object state between the tamper-evident ledger and a ScalarDB table for more flexible querying while maintaining the ledger as the authoritative source of truth.

How TableStore works

TableStore offers SQL-based operations for managing structured data by using a PartiQL-based dialect that supports CREATE TABLE, INSERT, UPDATE, and SELECT while automatically maintaining tamper-evident version history. For a hands-on tutorial, see Get Started with ScalarDL TableStore.

SQL-based data management

Tables are flexible and schemaless — you must specify a primary key, but can add optional index columns. Since TableStore treats records as JSON objects, you can include any fields beyond what you specify at table creation. Unlike traditional databases, every change is preserved in an immutable audit trail.

Advanced querying and version history

TableStore supports complex SQL queries including filtering, joining, and projection across multiple tables. The unique history() function returns complete version history for records, providing full audit trails showing how data evolved over time. This is invaluable for compliance scenarios where you must demonstrate not only the current state but also historical changes.

Use cases

HashStore and TableStore enable a wide range of real-world applications across industries. The following are some practical examples that demonstrate how each abstraction solves specific business challenges.

Regulatory compliance and document verification (HashStore)

A hypothetical example is using HashStore to support regulatory compliance and document verification. For instance, an organization managing sensitive documents — such as clinical trial materials — could store cryptographic hash values along with metadata to create tamper-evident records. When verification is required, document integrity can be confirmed by comparing current hashes with stored values. The collection feature of HashStore can be used to define and protect a set of document identifiers required for an audit or submission, ensuring that the set is complete and has not been modified, without storing the documents themselves.

Supply chain traceability (TableStore)

As an illustrative use case, TableStore can be applied to supply chain traceability scenarios. For example, a system may record shipment events such as temperature readings, custody transfers, and inspection results in a relational form. The SQL interface of TableStore enables querying the current state and joining related records across tables.

Financial transaction records (TableStore)

Another representative example is maintaining tamper-evident financial transaction records. In this scenario, transactional data is recorded in an append-only manner and accessed through standard SQL for operational queries and audit-oriented inspections. The history() function provides complete transaction histories with cryptographic proof, making it easier to implement logging, traceability, and post-incident investigation in compliance-sensitive systems.

Best practices

• Design for immutability: Although TableStore provides a powerful abstraction that lets you treat data as a relational model, you should design your ledger data to be as compact as possible. In particular, clearly identify which data truly requires authenticity guarantees, since storing data in an append-only, immutable manner directly impacts performance and storage efficiency.
• Use indexes strategically: In TableStore, index frequently queried fields but avoid over-indexing to maintain optimal performance.
• Leverage integrations: Use ScalarDB integration for high-performance queries with HashStore.

Additional resources

To dive deeper into HashStore and TableStore, explore the following documentation:

• HashStore
  – Get Started with ScalarDL HashStore
  – Write a ScalarDL Application with the HashStore Abstraction
  – ScalarDL HashStore Command Reference
• TableStore
  – Get Started with ScalarDL TableStore
  – Write a ScalarDL Application with the TableStore Abstraction
  – ScalarDL TableStore Command Reference
  – ScalarDL TableStore SQL Grammar

Wrap-up

ScalarDL HashStore and TableStore make tamper-evident data management accessible by eliminating custom contract development. HashStore excels at digital evidence preservation and compliance scenarios, while TableStore provides SQL-based structured data management with complete version history.

Whether you’re building regulatory compliance systems, supply chain traceability, or financial audit trails, these abstractions bring the power of the tamper-evident ledger of ScalarDL to your applications without traditional complexity.

Ready to get started? Explore the tutorials in the ScalarDL HashStore getting started documentation and ScalarDL TableStore getting started documentation. To learn more, visit the Scalar Engineering Blog and ScalarDL documentation.

If you are considering using ScalarDL, please contact us.

Simplifying tamper-evident applications by using ScalarDL HashStore and TableStore was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

ScalarDB Cluster is now available as a bring-your-own-cloud (BYOC) offering on Google Cloud through Google Cloud Marketplace. This means you can spin up a fully managed ScalarDB Cluster environment in your cloud account without ever touching a Kubernetes manifest or Helm Chart. If you’ve been deploying ScalarDB Cluster by using the container and the Helm Chart, or if you’re new to ScalarDB Cluster and want to get started quickly, the offering is worth considering.

Previously, deploying ScalarDB Cluster meant setting up and managing your own Kubernetes cluster manually, configuring Helm Charts, and handling all the operational overhead that comes with it. The new offering changes that by giving you a managed service that automatically handles the infrastructure provisioning for you in your cloud account.

What is BYOC?

BYOC is a deployment model that lets you run managed software solutions directly within your own cloud account, giving you more flexibility and control over your resources, security, and compliance. In the context of ScalarDB Cluster, the BYOC control plane automates the provisioning and management, enabling you to focus on your application and data instead of handling infrastructure-related tasks like Kubernetes or Helm Charts.

How to start using ScalarDB Cluster on Google Cloud

The deployment flow on Google Cloud is designed to be straightforward. Here’s the high-level experience:

1. Subscribe through Google Cloud Marketplace. You visit the ScalarDB Cluster listing on Google Cloud Marketplace and subscribe to the edition that fits your needs — either Enterprise Standard or Enterprise Premium. For details, see Figure 2.
2. Get approved and access the Scalar portal. After we approve your subscription request, you’re directed to the Scalar portal, which is your centralized management interface for ScalarDB Cluster deployments.
3. Register your Google Cloud project. You link your Google Cloud project to the Scalar portal. A one-time setup script (run in Google Cloud Shell) configures the necessary APIs, service accounts, IAM roles, and workload identity settings automatically.
4. Create a deployment instance. Through the Scalar portal, you specify your region, replica count, database properties, and other configuration options. The deployment — including the underlying GKE cluster — is then automatically provisioned for you. For details, see Figure 3.
5. Start using your cluster. Once the instance is running, you get endpoint and Grafana dashboard connectivity information directly from the portal.

With a self-hosted deployment, you’re responsible for provisioning a Kubernetes cluster, installing and configuring Helm Charts, managing licensing, and handling scaling and availability. However, the offering takes care of all these tasks for you, so you can focus on using ScalarDB Cluster instead of managing infrastructure.

For the full step-by-step deployment guide, refer to Deploy ScalarDB Cluster Through Google Cloud Marketplace.

Use cases

The offering fits naturally into two common scenarios that we hear from teams considering ScalarDB Cluster.

Getting started quickly

For teams evaluating ScalarDB Cluster or building a new application that needs distributed transactions across multiple databases, the offering removes the setup barrier entirely. Instead of spending time configuring Kubernetes and Helm, you can subscribe, deploy, and start experimenting with ScalarDB Cluster in a much shorter window.

Manage deployments through a GUI

The Scalar portal provides a user-friendly GUI for managing your ScalarDB Cluster deployments. Instead of working with command-line tools or editing YAML files, you can configure, monitor, and scale your clusters through an intuitive dashboard. This GUI streamlines common tasks such as creating new instances, adjusting replica counts, viewing cluster health, and accessing endpoint information, making deployment and management accessible to both developers and operations teams.

Wrap-up

With ScalarDB Cluster on Google Cloud through Google Cloud Marketplace, you have a managed service that doesn’t require you to have hands-on Kubernetes expertise. If you’re exploring ScalarDB Cluster for the first time, the offering lets you focus on your application rather than on infrastructure.

To learn more about ScalarDB Cluster, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDB Cluster documentation.

If you are considering using ScalarDB Cluster, please contact us.

Introducing ScalarDB Cluster as a BYOC offering through Google Cloud Marketplace was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

With the release of ScalarDB MCP Server, we’re bringing the power of AI-assisted database management to the ScalarDB ecosystem through the Model Context Protocol (MCP).

Overview

ScalarDB MCP Server is an MCP implementation that enables large language models (LLMs) to interact with your databases by using natural language. Instead of writing SQL queries or learning database-specific APIs, you can simply ask questions and make requests in plain language.

What makes ScalarDB MCP Server particularly powerful is its foundation on the multi-storage capabilities of ScalarDB. While traditional MCP servers often target one specific database technology at a time, ScalarDB MCP Server provides unified access to all your databases — whether PostgreSQL, MySQL, Azure Cosmos DB, Amazon DynamoDB, or other supported databases — through a single server instance. This eliminates the complexity of managing multiple MCP servers and provides a consistent interface for working with heterogeneous database environments.

How ScalarDB MCP Server works

ScalarDB MCP Server acts as a bridge between LLMs and your database infrastructure, translating natural language requests into appropriate database operations while leveraging the transaction management and multi-storage capabilities of ScalarDB.

Natural language to database operations

When you make a request by using natural language through an MCP-compatible client (like Claude Desktop), the MCP server automatically determines and then executes the appropriate operations (either through SQL mode or CRUD mode) across your databases.

The following are some examples of what ScalarDB MCP Server can do:

• Single-table queries: Requests like “Show all customers from the APAC region” are parsed to identify the target table and filter conditions, then converted directly into basic SELECT statements or CRUD read operations.
• Cross-database joins: Complex queries such as “Show customers who made purchases in the last week” require the system to identify relationships between tables across different databases, perform join operations, and coordinate transactions across multiple storage systems.
• Analytical aggregations: Requests like “What’s the average order value by customer region for the last quarter” involve parsing time ranges, grouping criteria, and aggregate functions, then generating queries that can efficiently compute results across large datasets.

ACID transaction support

ACID transactions are particularly important when operations span multiple databases. If you request an operation that updates records in both PostgreSQL and Cosmos DB, the distributed transaction capabilities of ScalarDB ensure that either both updates succeed or both are rolled back, maintaining consistency across your entire data landscape.

Configuration options

ScalarDB MCP Server must be configured to connect to your ScalarDB deployment, with operational parameters specified.

For details, see ScalarDB MCP Server configurations.

Available tools

ScalarDB MCP Server exposes database operations through MCP tools that the LLM invokes based on your natural language requests.

For details about tools, see the ScalarDB MCP Server Tools Reference.

Best practices

Based on the capabilities and architecture of ScalarDB MCP Server, here are recommendations for effective usage:

• Be specific in your requests: While the LLM can interpret natural language, more specific requests generally produce more accurate results. Instead of “Show me some customers,” try “Show me the first 10 customers in order of the date in which they registered.” This helps the LLM generate more precise queries.
• Start with read-only operations: When first exploring ScalarDB MCP Server, focus on read operations to familiarize yourself with how the LLM interprets natural language requests and generates database operations. This allows you to build confidence in the system before making data modifications.
• Verify critical operations: For operations that modify data, especially in production environments, verify the generated operations before execution. Many MCP clients allow you to review the operations the LLM plans to execute before confirming.
• Monitor resource usage: Natural language database access can lead to unexpected query patterns or complexity. Monitor your database and ScalarDB Cluster resource usage to ensure the generated operations perform efficiently and don’t impact other workloads.

Wrap-up

ScalarDB MCP Server brings AI-assisted database management to the ScalarDB ecosystem, making it easier for developers, analysts, and business users to interact with data across heterogeneous database systems. By combining the Model Context Protocol with the multi-storage transaction capabilities of ScalarDB, ScalarDB MCP Server provides a unified natural language interface that eliminates the complexity of managing multiple database systems.

Whether you’re exploring data for business insights, experimenting with data access through natural language, or interacting with existing databases without having deep knowledge of their schemas, ScalarDB MCP Server reduces the barriers between you and your data while maintaining the ACID guarantees required for production use.

To learn more about ScalarDB, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDB documentation.

If you are considering using ScalarDB, please contact us.

Querying Multiple Databases by Using Natural Language with ScalarDB MCP Server was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Version 3.17 includes various enhancements and improvements, including:

• Transaction metadata decoupling
• Expanded database and storage support
• SQL aggregation with GROUP BY support
• Enhanced access control with role-based permissions
• Red Hat OpenShift support
• Performance improvements
  – Batch operations
  – Piggybacking
  – Write buffering

In this post, we’ll introduce these new features and improvements.

Transaction metadata decoupling

One of the most significant enhancements in ScalarDB 3.17 is transaction metadata decoupling, which enables organizations to adopt ScalarDB without modifying their existing database schemas.

Previously, when integrating ScalarDB with existing databases, transaction metadata had to be embedded directly within the database records. This requirement often meant altering existing table schemas, which could be a barrier for organizations with established production systems.

With transaction metadata decoupling, ScalarDB now manages transaction metadata separately from your application data in an independent storage area. This separation means you can integrate ScalarDB into your existing infrastructure while maintaining your original database schemas completely unchanged. This capability is particularly valuable for organizations looking to gradually modernize their database infrastructure or implement phased migration strategies while preserving existing system assets.

The feature significantly reduces implementation risks and costs by eliminating the need for schema modifications, making ScalarDB adoption more accessible for production environments with existing data models.

For more information about transaction metadata decoupling, please see Consensus Commit Protocol.

Expanded database and storage support

ScalarDB 3.17 extends its already comprehensive database support to include even more data sources.

Newly supported databases and storages:

• TiDB: A distributed NewSQL database that provides horizontal scalability
• AlloyDB: Google Cloud’s fully managed PostgreSQL-compatible database service optimized for demanding enterprise workloads
• Amazon S3: AWS’s object storage solution
• Azure Blob Storage: Microsoft Azure’s object storage solution
• Google Cloud Storage: Google Cloud’s object storage solution

This expanded support reinforces ScalarDB as a universal hybrid transaction/analytical processing (HTAP) engine, enabling you to build applications that span diverse database technologies and cloud platforms. Whether you’re working with traditional relational databases, modern NewSQL systems, or cloud object storage, ScalarDB provides a unified transactional interface.

For more information about the newly supported databases, please see Databases.

SQL aggregation with GROUP BY support

ScalarDB 3.17 enhances its SQL capabilities by adding support for GROUP BY aggregation operations. The implementation includes support for the following aggregate functions: COUNT, SUM, MIN, MAX, and AVG. These functions enable you to perform common operations directly within your SQL queries, such as calculating totals, averages, and finding minimum or maximum values across grouped data sets.

Previously, applications that used the ScalarDB SQL interface had to retrieve detailed records and perform aggregation logic in the application layer. With native GROUP BY support, you can now push aggregation operations down to ScalarDB Cluster, thereby eliminating the need to implement aggregation logic in the application, which reduces the processing load on the client side.

This enhancement makes the ScalarDB SQL interface more complete and powerful, enabling more complex queries while maintaining the benefits of distributed transactions across multiple databases.

For more information about GROUP BY, please see the Select section in the ScalarDB SQL Grammar documentation.

Enhanced access control with role-based permissions

ScalarDB 3.17 improves its security capabilities with enhanced access control features, including role-based access control (RBAC) that supplements the existing user-level permissions.

With role-based permissions, you can define access policies based on organizational roles rather than managing permissions for individual users. This approach simplifies security administration in enterprise environments where multiple users share similar access requirements.

These enhancements make ScalarDB more suitable for enterprise deployments with complex security and compliance requirements.

For more information about RBAC, please see Authenticate and Authorize Users.

Red Hat OpenShift support

ScalarDB 3.17 also includes support for Red Hat OpenShift, one of the leading enterprise Kubernetes platforms. This addition enables organizations to deploy ScalarDB in highly regulated on-premises environments and enterprise cloud deployments that standardize on OpenShift.

OpenShift support expands the deployment flexibility of ScalarDB, making it suitable for a wider range of enterprise container platform strategies.

For more information about Open Hat RedShift, please see the Kubernetes section of the ScalarDB Requirements documentation.

Performance improvements

ScalarDB 3.17 introduces several performance optimizations that reduce network overhead and improve transaction throughput.

Batch operations

The new batch operation capability allows you to execute multiple operations — including Get, Scan, Put, Insert, Upsert, Update, and Delete — in a single request. By bundling multiple operations together, you can significantly reduce network round trips between your application and ScalarDB Cluster.

This feature is especially beneficial for workloads that need to perform multiple related operations, as it minimizes latency and improves overall transaction performance.

For more information on batch operations, please see Batch operation.

Piggybacking

ScalarDB 3.17 introduces piggybacking, an optimization technique that allows transaction control operations like BEGIN and COMMIT to ride alongside other database operations. Instead of sending these control operations as separate network requests, they can now be combined with your actual data operations.

This optimization further reduces network overhead and improves transaction processing efficiency, particularly for transactions with few operations where the overhead of separate control messages would be more noticeable.

For more information on piggybacking, please see the cluster.client.piggyback_begin.enabled configuration.

Write buffering

ScalarDB 3.17 introduces write buffering, another performance optimization that intelligently batches non-conditional write operations to reduce the number of remote procedure calls (RPCs) between your application and ScalarDB Cluster.

Write buffering automatically buffers operations like Insert, Upsert, and unconditional Put, Update, and Delete operations, then executes them in batches rather than sending each operation individually. This batching approach significantly reduces network overhead, especially for applications that perform many write operations in sequence.

For more information, please see the clusterclientwrite_bufferingenabled configuration.

Other improvements

This release also includes various bug fixes, performance optimizations, and documentation enhancements that improve the overall stability and usability of ScalarDB.

For more information on ScalarDB features, see the following documentation:

• ScalarDB Java API Guide
• ScalarDB SQL Grammar
• Getting Started with ScalarDB Cluster for Vector Search

Wrap-up

We have released ScalarDB 3.17, which includes the new features that we mentioned in this blog post. For details about additional changes, including the new features mentioned in this post, please refer to the ScalarDB 3.17 release notes.

To learn more about ScalarDB, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDB documentation.

If you are considering using ScalarDB, please contact us.

ScalarDB 3.17 has been released! was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

With the release of ScalarDB 3.16, we’ve introduced remote replication (in private preview), which is a powerful feature that addresses these challenges head-on.

Overview

Remote replication provides near-real-time data replication between primary and backup sites with a recovery point objective (RPO) of 0 seconds and a recovery time objective (RTO) of just seconds. This feature is particularly valuable for financial services, retail, manufacturing, and other mission-critical enterprise systems where data loss is simply unacceptable.

What sets remote replication in ScalarDB Cluster apart is its cloud-agnostic and database-agnostic architecture. Built on top of ScalarDB Cluster, you can replicate data from one database in one cloud service provider to a completely different database in another cloud service provider. This flexibility means you’re not locked into a single vendor’s replication solution and can design your disaster recovery strategy based on your specific business requirements rather than technical limitations.

The feature uses a hybrid replication approach that combines synchronous and asynchronous processing. During transaction commit at the primary site, write operations are synchronously captured and stored in a replication database. Then, asynchronously, these operations are applied to the backup site databases. This hybrid approach achieves zero data loss while minimizing the performance impact on your primary site operations.

How remote replication works

Understanding the architecture of remote replication helps you appreciate how it achieves both zero data loss and minimal performance impact. The system consists of three main sites: the primary site, the backup site, and shared components that span both.

Primary site components

At the primary site, your application continues to interact with ScalarDB Cluster as usual. However, when remote replication is enabled, each ScalarDB Cluster node includes a LogWriter module that captures transaction operations during commit. Here’s what happens:

1. When your application commits a transaction, ScalarDB Cluster writes the data to the primary site database.
2. Simultaneously, LogWriter captures all write operations from the transaction and stores them in the replication database.
3. The transaction commit succeeds only after both the primary site database write and the replication database write complete successfully.

This synchronous phase ensures that every committed transaction is safely stored in the replication database, guaranteeing zero data loss even if the primary site fails immediately after commit.

Shared components

The Coordinator database and replication database serve as shared components between the primary and backup sites. The Coordinator database manages transaction states across sites, while the replication database stores transaction groups that contain write operations from the primary site.

These shared components are critical to the overall system availability. If they become unavailable, the primary site cannot commit new transactions, and replication processing halts. Therefore, you should use highly available databases for these components — such as Azure Cosmos DB or YugabyteDB.

The replication database uses a partitioned table design for performance and scalability. Write operations are distributed evenly across partitions, and you can configure the partition count based on your expected write volume and potential backup site downtime duration.

Backup site components

At the backup site, ScalarDB Cluster nodes run with LogApplier modules instead of LogWriter. LogApplier continuously processes write operations from the replication database and applies them to the backup site tables:

1. LogApplier scans the replication database for new transaction groups.
2. It checks the Coordinator database to verify transaction completion.
3. It calculates dependencies between operations at the record level by using replication record metadata tables.
4. It applies operations to the backup site database tables in the correct order to maintain consistency.
5. It updates the replication record metadata tables to track progress and removes processed operations from the replication database.

This asynchronous processing happens continuously in the background, keeping the backup site synchronized with minimal latency — typically within seconds under normal conditions.

The hybrid approach in action

The beauty of this hybrid approach is that it separates the concerns of durability and replication. The synchronous phase guarantees durability by ensuring every committed transaction is captured, while the asynchronous phase handles the complexity of applying operations to the backup site without blocking primary site operations.

Consider a scenario where your backup site database experiences temporary issues. With traditional synchronous replication, this would impact your primary site performance or availability. With the hybrid approach of ScalarDB, your primary site continues operating normally. Operations accumulate in the replication database until the backup site recovers, at which point LogApplier processes the backlog and brings the backup site up-to-date.

Use cases

Remote replication addresses several critical use cases for enterprise organizations.

Use case 1: Disaster recovery with zero data loss

Financial institutions and other regulated industries often have strict requirements around data loss prevention. Traditional backup strategies may provide recovery, but they cannot guarantee zero data loss because backups are point-in-time snapshots.

With remote replication, every committed transaction is guaranteed to be replicated. If your primary site suffers a catastrophic failure — whether due to natural disaster, hardware failure, or other causes — you can promote the backup site to serve application traffic with confidence that no committed data was lost. The RPO of 0 means you meet even the strictest regulatory requirements for data durability.

Use case 2: Cross-region and cross-cloud deployments

Organizations with global operations often need to maintain data availability across multiple regions or even multiple cloud service providers. You might run your primary operations in AWS in one region while maintaining a disaster recovery site in Azure in a different geographic region.

Remote replication in ScalarDB makes this straightforward because it’s both cloud-agnostic and database-agnostic. You could replicate from PostgreSQL on AWS RDS to MySQL on Azure Database for MySQL, or from Cosmos DB in one region to DynamoDB in another region. This flexibility allows you to design your disaster recovery strategy based on business requirements rather than being constrained by vendor-specific replication technologies.

Use case 3: Offloading analytical workloads

While remote replication is primarily designed for disaster recovery, the backup site can also serve as a read replica for certain types of queries. The backup site supports read-only transactions with read-committed isolation, making it suitable for offloading reporting, business intelligence, and analytical workloads from your primary site.

This use case is particularly valuable when you want to run resource-intensive analytical queries without impacting your primary OLTP workload. The backup site provides a view of your data that’s typically just seconds behind the primary site, which is acceptable for many analytical use cases.

Best practices

Based on our experience with remote replication, here are some recommendations for optimal usage:

• Ensure high availability for shared infrastructure: Use databases with built-in redundancy across multiple zones or regions for critical shared components to maintain system reliability.
• Plan for capacity during outages: Size your infrastructure to handle data accumulation when the backup site is unavailable, accounting for typical write volumes and potential downtime periods.
• Optimize for network conditions: If you experience latency between sites, adjust batching settings to group operations together and improve overall performance.
• Monitor system health: Track replication lag metrics to detect when the backup site falls behind or encounters issues that need attention.
• Validate disaster recovery readiness: Regularly test your failover process to ensure you can successfully switch to the backup site when necessary.
• Account for geographic distribution: When components span multiple regions, configure batching appropriately to minimize the impact of network latency on system throughput.
• Manage schema updates carefully: Coordinate database schema changes across both sites during planned maintenance windows, as these updates require manual synchronization.

Wrap-up

Remote replication, which is currently in private preview in ScalarDB 3.16, provides a robust solution for high availability and disaster recovery with zero data loss. The hybrid replication approach achieves RPO of 0 while minimizing performance impact on your primary site, and the cloud-agnostic, database-agnostic architecture gives you flexibility in designing your disaster recovery strategy.

Whether you need to meet strict regulatory requirements for data durability, distribute your operations across multiple cloud service providers, or offload analytical workloads to a read replica, remote replication provides the capabilities you need. The extensive configuration options allow you to tune performance for your specific workload, while the comprehensive monitoring capabilities ensure you can maintain confidence in your replication health.

To learn more about ScalarDB, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDB documentation.

If you are considering using ScalarDB, please contact us.

Remote replication in ScalarDB Cluster was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

This version includes new features and improvements, including:

• New abstractions
  – ScalarDL HashStore for object and collection authenticity management
  – ScalarDL TableStore with SQL interface for building tamper-evident applications
• Simplified ledger validation

In this post, we’ll introduce these new updates.

New abstractions

ScalarDL 3.12 introduces two new abstractions — HashStore and TableStore — that run on top of the primitive ledger and are designed for different use cases. These data stores provide high-level interfaces that make it easier to build tamper-evident applications without writing custom contracts.

ScalarDL HashStore

ScalarDL HashStore offers two powerful functionalities: object authenticity management and collection authenticity management. By using HashStore, developers can develop authenticity management applications for file services, audit logging systems, or any scenario requiring evidence preservation.

Core capabilities

HashStore provides two main functionalities:

• Object authenticity management: Manage the authenticity of any kind of object, such as files, audit logs, and directories in your file or object storage. The workflow involves storing hash values with unique object IDs, and validating authenticity by comparing current hash values with stored values.
• Collection authenticity management: Manage which objects exist in a collection, particularly valuable for audit sets — collections of objects that must be validated in an auditing process. Create collections, add and remove objects in a tamper-evident manner, and track complete history of collection changes.

Key benefits

HashStore is particularly valuable for organizations that need to:

• Maintain tamper-evident records of critical documents and files.
• Comply with regulations requiring data authenticity (such as GDPR, CCPA/CPRA, or financial regulations).
• Preserve intellectual property evidence with cryptographic proof.
• Track document changes with complete audit trails.
• Manage audit sets that must remain immutable.

Pre-defined contracts foundation

HashStore is built on pre-defined contracts that provide out-of-the-box functionality without requiring developers to write custom contract code. This approach makes authenticity management accessible to development teams while maintaining the Byzantine-fault detection and tamper-evident properties of ScalarDL.

For more information, see Get Started with ScalarDL HashStore.

ScalarDL TableStore

ScalarDL TableStore offers a SQL interface that enables developers to build versatile, tamper-evident applications with familiar data models and interfaces. With TableStore, developers can create applications by using standard SQL operations, eliminating the previous requirement of developing Java contracts for basic data operations.

Core capabilities

Developers can perform common database operations using familiar SQL syntax, including:

• Creating tables with primary keys and secondary indexes
• Inserting and updating records
• Querying data with WHERE clauses and projections
• Joining multiple tables
• Retrieving complete record histories with special functions like history() for tamper-evident audit trails

Key benefits

TableStore is particularly valuable for organizations that need to:

• Migrate from traditional ledger databases with minimal code changes.
• Build financial systems, inventory management, or applications that benefit from relational data structures.
• Reduce development cycles by using familiar SQL instead of writing custom contracts.
• Maintain complete audit trails with built-in history tracking for compliance and regulatory requirements.
• Simplify application architecture with a proven relational data model.

Pre-defined contracts foundation

This SQL interface is built on top of pre-defined contracts that provide a relational data model while maintaining the core tamper-evident properties of ScalarDL. These pre-defined contracts eliminate the need for developers to write custom Java contract code, dramatically reducing programming overhead and development cycles.

For more details, including SQL syntax examples and getting started guides, see Get Started with ScalarDL TableStore.

Simplified ledger validation

ScalarDL 3.12 simplifies the validation process by bundling the ValidateLedger contract directly into the client SDK. Previously, developers needed to separately register this contract before they could validate ledger data with the Auditor configuration. Now, this essential validation functionality is available out of the box.

This improvement streamlines the validation workflow:

• Reduced setup complexity: No need to manually register the validation contract.
• Faster onboarding: New developers can start validating ledger data immediately.
• Consistent validation: Ensures all applications use the same, tested validation logic.
• Lower maintenance overhead: One less component to manage and update.

Ledger validation is a critical operation in ScalarDL, allowing you to cryptographically verify that data has not been tampered with. By making this functionality more accessible, ScalarDL 3.12 encourages best practices around data integrity verification.

Wrap-up

We have released ScalarDL 3.12, which includes the new features that we mentioned in this blog post. For details about additional changes, including the new features mentioned in this post, refer to the ScalarDL 3.12 release notes.

To learn more about ScalarDL, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDL documentation.

If you are considering using ScalarDL, please contact us.

ScalarDL 3.12 has been released! was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Version 3.16 includes various enhancements and improvements, including:

• Remote replication feature for enhanced data availability
• Scanner API in transaction abstraction for iterative result retrieval
• Enhanced transaction configurations
  – Support for enabling/disabling the transaction feature
  – Support for read-only transaction mode
  – Support for the read-committed isolation-level
• Support for IBM Db2
• Performance enhancement in transaction processing
• ScalarDB Analytics enhancements
  – Support for Databricks and Snowflake
  – Pay-as-you-go pricing model

In this post, we’ll introduce these updates.

Remote replication feature

ScalarDB 3.16 introduces a powerful remote replication feature that enhances data availability and disaster recovery capabilities for your applications.

This near real-time database replication mechanism enables data replication between primary and secondary sites with a recovery point objective (RPO) of 0 seconds and recovery time objective (RTO) of just seconds. The remote replication feature supports cross-cloud and cross-database replication configurations, giving you flexibility in maintaining data availability across different environments.

Key benefits include improved fault tolerance, geographic data distribution for compliance requirements, and enhanced business continuity. Organizations can now maintain business continuity with zero data loss, making this particularly valuable for financial services, retail, manufacturing, and other mission-critical enterprise systems.

For more details on remote replication, please refer to Remote Replication for High Availability.

Scanner API in transaction abstraction

ScalarDB 3.16 introduces the Scanner API within the transaction abstraction layer, providing developers with efficient tools for iteratively retrieving large result sets without loading entire datasets into memory.

This enhancement addresses common stability challenges when working with large-scale data operations by allowing applications to process query results in manageable chunks. The Scanner API maintains transaction consistency while providing streaming-like access to data, significantly reducing memory consumption and improving application responsiveness.

The Scanner API supports filtered scanning, range queries, and ordered retrieval, making it ideal for analytics workloads, batch processing operations, and any scenario where you need to process large volumes of data within transactional boundaries.

For more information, see Scan operation (when executing transactions) and Execute Scan operation (without beginning or starting a transaction).

Enhanced transaction configurations

ScalarDB 3.16 provides greater flexibility in transaction management through new configuration options that allow enhanced control over transaction behavior.

Transaction enable/disable configuration

The new scalar.db.transaction.enabled configuration option for ScalarDB Cluster allows you to enable or disable the transaction feature at the application level. This provides operational flexibility for scenarios where you may need to temporarily disable transactions for maintenance, testing, or performance optimization purposes.

For more details, see the configuration details for scalar.db.transaction.enabled.

Read-only transaction mode

ScalarDB now supports beginning transactions in read-only mode, which optimizes performance for read-heavy workloads while maintaining transaction isolation guarantees. Read-only transactions eliminate the overhead of write locks and conflict detection, resulting in improved throughput for analytics and reporting applications.

For more details, see Begin or start a transaction in read-only mode.

Read-committed isolation level

ScalarDB 3.16 adds support for the read-committed isolation level, providing a performance-optimized option for applications that can benefit from relaxed consistency guarantees. This isolation level reduces locking overhead while maintaining sufficient consistency for many real-world scenarios, resulting in improved performance for high-throughput applications.

For details, see Isolation levels.

Support for IBM Db2

ScalarDB 3.16 now includes native support for IBM Db2, expanding the ecosystem of supported databases and enabling organizations already invested in IBM’s database technology to take advantage of the ACID transaction capabilities of ScalarDB.

This integration allows you to use IBM Db2 as an underlying storage system while maintaining all the benefits of the database-agnostic transaction management of ScalarDB. The implementation supports IBM Db2 for Linux, Unix, and Windows (LUW).

The Db2 integration provides full CRUD operation support, transaction management, and schema management capabilities.

For more information on IBM Db2, refer to supported databases in ScalarDB.

Performance enhancement in transaction processing

ScalarDB traditionally abstracted various types of databases to provide consistent transaction processing independent of the underlying database implementation. While this approach offered flexibility, it wasn’t always optimal in terms of processing efficiency.

ScalarDB 3.16 introduces a new execution mechanism within this abstraction layer that considers the processing characteristics of each database. This allows for “pushing down” (actively delegating processing) a portion of transaction processing to the underlying database, enabling a larger part of the processing to be completed on the database side. As a result, ScalarDB 3.16 significantly reduces transaction processing overhead, leading to an improvement in processing performance.

ScalarDB Analytics enhancements

ScalarDB Analytics 3.16 introduces significant improvements to support modern data lakehouse architectures and flexible consumption models.

Databricks and Snowflake support

ScalarDB Analytics now provides native integration with Databricks and Snowflake. This enhancement enables organizations to perform advanced analytics on data lakehouse architectures.

The integration supports cross-source data analysis that spans both structured transactional data and semi-structured analytical data, enabling real-time decision-making workflows. This capability allows organizations to support modern analytical workloads across hybrid data architectures.

For details on Databricks and Snowflake support in ScalarDB Analytics, see ScalarDB Analytics Data Source Reference.

Pay-as-you-go pricing model

ScalarDB Analytics 3.16 introduces a flexible pay-as-you-go pricing model available through AWS Marketplace, providing CPU resource-based billing that scales with actual usage. This pricing approach removes traditional barriers to adoption and allows organizations to align costs with consumption patterns.

The new pricing model is particularly beneficial for organizations with variable analytical workloads or those seeking to evaluate ScalarDB Analytics without significant upfront investment commitments.

For details on how to subscribe to ScalarDB Analytics in AWS Marketplace, see Subscribe to Scalar products from AWS Marketplace.

Other improvements

This release also includes various bug fixes, performance optimizations, and documentation enhancements that improve the overall stability and usability of ScalarDB.

For more information on ScalarDB features, see the following documentation:

• ScalarDB Java API Guide
• ScalarDB Core Configurations
• ScalarDB Cluster Configurations
• Getting Started with ScalarDB

Wrap-up

We have released ScalarDB 3.16, which includes the new features that we mentioned in this blog post. For details about additional changes, including the new features mentioned in this post, please refer to the ScalarDB 3.16 release notes.

To learn more about ScalarDB, please take a look at our other posts on the Scalar Engineering Blog and at the ScalarDB documentation.

If you are considering using ScalarDB, please contact us.

ScalarDB 3.16 has been released! was originally published in Scalar Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.