Frequently Asked Questions

What is SymmetricDS?

SymmetricDS is software that replicates relational database tables between multiple databases. It can also be used to replicate files and directories between multiple hosts. It uses a light-weight, web-based protocol to send and receive data, which makes it easy to work with firewalls. Replication is done in the background asynchronously, allowing data changes in offline mode. It supports most commercial and open source database platforms.

What is SymmetricDS Pro?

SymmetricDS Pro is commercial software that extends the community version of SymmetricDS with enterprise features for security, performance, and management. In additional to the core data replication features, it provides a web interface, additional connectors, log mining, clustering, bulk loading, data validation, auditing, monitoring, and remote management.

How does it work?

Triggers are installed in the database to guarantee that data changes are captured. The triggers are written to be as small and efficient as possible. SymmetricDS Pro can also use log mining, change tracking, or logical streaming to capture changes. This means that applications continue to use the database as usual without any special driver software. Routing and syncing of data is done outside of the database in the SymmetricDS process.

Is replication using timestamps?

No, timestamps are not used. Data is captured as it is changed, and each change is given a sequence number. Data changes are grouped together and given a batch number to route through the system. Data changes within a transactional unit are preserved together within a batch.

Can partial tables be replicated?

Yes, subsets of data can be synchronized to other databases. The subset can be: vertical, selecting only some columns to sync; horizontal, selecting only some rows to sync, or both. Different subsets of data can be sent to different databases. For example, a retail store chain can have a central database containing data for all stores, while each retail store database contains just its subset.

Can table names and column names be re-mapped?

Yes, the source table can be mapped to a different table name and different column names at the target database. It is also possible to add new columns and ignore existing columns.

Is there data transformation support?

Data transformation is a built-in core feature. A source table is mapped column by column to a destination table. Data can be transformed using one of several built-in transformers or specifying a Beanshell script. Multiple tables can be mapped together to merge data, and data can be looked up with SQL queries on either the source or target side to supplement data.

Does replication require a fast network connection?

No, the network connection can be slow and even unreliable for periods of time. SymmetricDS makes efficient use of the network with low overhead, batching of data, and data compression. It is designed to withstand periods of network outage by tracking changes that were unsuccessful and retrying.

How quickly are databases replicated?

Asynchronous replication allows the data capture to happen immediately, allowing the source system to continue its activity without waiting. The target system is synchronized in the background separately. The user configures a delay in milliseconds for when the changes are replicated to the target system.

How many replicas are supported?

SymmetricDS is designed to scale to thousands of replicated databases. To scale vertically, configuration properties allow adding more resources to a single node. To scale horizontally, more nodes can be added and configured in cluster mode. A large installation of SymmetricDS at a central database can have multiple instances running together.

Is replication one-way or two-way?

Both one-way and two-way replication configurations are possible. Configuration is at the table level, so table A can be replicated one-way, while table B is replicated two-way. One-way replication, sometimes called primary-backup or master-slave, means changes are captured at the source system and replicated to the target system. Two-way replication, sometimes called multi-primary or multi-master, means changes are captured at both source and target systems and replicated to both.

How is data replicated securely?

Data can be encrypted by configuring node addresses to use HTTPS (SSL over HTTP) instead of the default HTTP. By default, connections between instances of SymmetricDS require password authentication. The setup between SymmetricDS instances is called registration, which initiates the password generation and transfer. The registration process can also be customized to use other methods of key exchange.

Can multiple instances be run on the same machine?

Yes, by specifying a different port number for each instance. You can also run multiple nodes in the same instance, which gives each node a different URL to contact.

Can I embed SymmetricDS in my application?

Yes, an application can include SymmetricDS as a library, which allows it to have more control and feedback of replication. This type of application is popular when using a small database system like H2, HSQLDB, Apache Derby, or SQLite, which gives the overall application a small footprint on memory. SymmetricDS has libraries for embedding with Java, while SymmetricDS Pro has libraries for embedding with Java, Android, iOS, and native code (C version).

I’m a developer. What are some advanced ways I can use SymmetricDS?

SymmetricDS has published APIs that allow for extension, embedding into applications, and remote management. The extension API allows a developer to write custom code that is used by the server at runtime. Extensions can change how different subsystems behave, including registration, routing, data loading, transformation, and acknowledgements. The embedding API allows a developer to include SymmetricDS in an application so it can control when replication happens and give feedback to the user. With SymmetricDS Pro, embedding is used to sync data on mobile platforms like Android and iOS. The remote management API is available through JMX and command line tools. SymmetricDS Pro adds a remote management API through REST. Remote management lets you perform operational tasks like registering new databases, sending data loads, and querying the system status.

What is origin of SymmetricDS?

SymmetricDS is a “scratch your own itch” project that originated from the needs of a large retailer in Columbus, Ohio. After deploying a vendor’s Point of Sale (POS) system across a test pilot of a few hundred store locations, it became clear that data movement did not meet the retailer’s requirements. The system was already straining to transfer data, with transfer errors needing manual intervention. Sales transaction were posting in an “end of day” process, but the retailer needed more timely data. Adding new fields to the data required cumbersome software modifications. In response, the developers started an open source project to replicate data across heterogeneous databases, scale to support thousands of databases, work across low-bandwidth networks, and survive network outages. From there, the project grew with new features, more data platforms, and more use cases from other industries.

What is the license for SymmetricDS?

SymmetricDS is free software licensed under the GNU General Public License (GPL) version 3.0. SymmetricDS Pro is commercial software licensed from Jumpmind as part of a support subscription or independent software vendor (ISV) program.

How do I troubleshoot errors?

When an error is encountered during replication, it is written to the log file at the ERROR level with details, using standard Log4J logging. On the source side, the database has an outgoing batch entry with an error status and the captured data that was unable to sync. On the target side, the database has an incoming batch entry with an error status along with the source system’s error code and message. Data errors are retried, so if the underlying problem is resolved, such as a missing parent record, then data replication will continue automatically.