Data sources

Data sources are reusable implementations of common patterns that can be used by the other extension types. They could be thought of as pluggable functions that can encapsulate anything such as accessing a database, accessing an external API, interacting with a legacy system or calling into an LLM.

Extensions running on Synapse call into a data source using its unique name defined in the configuration. See Data sources development documentation for more details about the request and response formats.

Built-in sqldb data source

Synapse ships with a built-in sqldb datasource that can be used to query Litestream, MySQL, Postgres or SQLite databases.

The query field for the lookup request should be a string containing an SQL query. The query could contain named parameters prefixed by a colon (:) and the values for those parameters should be passed in the queryParameters field as an object whose keys are the parameter names. If cacheOptions is defined, the result is cached in the Synapse cache.

The result field of the response would be one of the following:

null if the query returned no results
Queries that return a single row: a JSON object with column names as the keys.
Queries that return multiple rows: a JSON array containing objects with column names as the keys.

Litestream

Synapse can fetch a Litestream backup from any of the standard Litestream replication destinations and enable read-only querying of the retrieved copy. Note that VFS support is not currently available. Therefore, in order to fetch a new copy of the database, Synapse needs to be restarted. Similar to the standard Litestream service, credentials required for accessing the replica are read from the environment or through URL parameters. Refer to https://litestream.io/guides/#replica-guides for details.

extensions:
  dataSources:
    myLitestreamDB: (1)
      extension:
        extensionURL: system://sqldb
        configuration:
          litestream: (2)
            replicationURL: "s3://litestream/fruits.db" (3)

1	Unique name for the data source.
2	Use the Litestream backend with `sqldb`
3	URL of the replica. See https://litestream.io/guides/#replica-guides for more information.

MySQL

Provides access to a MySQL database. The connection string must be in the following format.

[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]

See https://github.com/go-sql-driver/mysql#dsn-data-source-name for the list of supported parameters. Environment variables can be used to provide sensitive values such as passwords.

extensions:
  dataSources:
    myMySQLDB: (1)
      extension:
        extensionURL: system://sqldb
        configuration:
          mysql: (2)
            connectionString: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos" (3)
            connectionPool: (4)
              maxLifeTime: 600s
              maxIdleTime: 300s
              maxOpen: 5
              maxIdle: 1

1	Unique name for the data source
2	Use the MySQL backend with `sqldb`
3	MySQL connection string
4	Connection pool settings (optional)

If your MySQL server requires TLS or if you want to use RSA key pair-based password exchange, you can configure those settings as follows:

TLS certificates

extensions:
  dataSources:
    myMySQLDB:
      extension:
        extensionURL: system://sqldb
        configuration:
          mysql:
            connectionString: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos?tls=mysecuretls"
            tls:
              mysecuretls:
                caCert: /path/to/ca_certificate.crt
                cert: /path/to/certificate.crt
                key: /path/to/private.key

Server public key

extensions:
  dataSources:
    myMySQLDB:
      extension:
        extensionURL: system://sqldb
        configuration:
          mysql:
            connectionString: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos?serverPubKey=mypubkey"
            serverPubKey:
              mypubkey: /path/to/server_public_key.pem

PostgreSQL

extensions:
  dataSources:
    myPostgresDB: (1)
      extension:
        extensionURL: system://sqldb
        configuration:
          postgresql: (2)
            connectionString: "postgres://${PG_USER}:${PG_PASSWORD}@localhost:5432/postgres?sslmode=disable&search_path=cerbos" (3)
            connectionPool: (4)
              maxLifeTime: 600s
              maxIdleTime: 300s
              maxOpen: 5
              maxIdle: 1

1	Unique name for the data source
2	Use the postgres backend with `sqldb`
3	Connection string for the database. See Postgres connstring documentation for more information.
4	Connection pool settings (optional)

SQLite

extensions:
  dataSources:
    mySQLiteDB: (1)
      extension:
        extensionURL: system://sqldb
        configuration:
          sqlite: (2)
            connectionString: "file:/tmp/cerbos.sqlite?mode=rwc&cache=shared&_fk=true" (3)

1	Unique name for the data source
2	Use the sqlite backend with `sqldb`
3	Connection string for the database. See https://sqlite.org/c3ref/open.html.

Custom data sources

extensions:
  dataSources:
    myAwesomeDataSource: (1)
      extension:
        extensionURL: "/extensions/awesome.wasm" (2)
        configuration: (3)
          environment: staging

1	Unique name for the data source
2	URL to fetch the extension from. See extension URL format for more information.
3	Optional configuration values for the extension

Building a custom data source

See Building custom Synapse extensions.