Developing Starlark extensions

Starlark is a lightweight Python dialect that can be used to drive the Synapse extension logic. They are easy to write and test because they do not require any intermediate compilation stages or specialised tools.

To use a Starlark script as the extension implementation, set the extensionURL configuration field to a valid extension URL in one of the following forms:

  • /path/to/script.star (File name must have a .star extension)

  • starlark+http://scripts.example.com/synapse/envoy?checksum=sha256:677cce2788330f16f27981130eaa50aa4189beab2121903bcea5b3c4c918dccc (Protocol of the URL must be prefixed with starlark+)

All Synapse extension points support running Starlark scripts. Each request is served by a fresh instance of a script and there’s no shared global state. Each extension type requires a specific set of functions that must be implemented by the script. See the extensions section below for details about these API contracts.

REPL

Synapse ships with a REPL (read, evaluate, print loop) to help with debugging Starlark scripts.

Synapse starlark repl [--exec] [SCRIPT_FILE]

Without any arguments, the command starts the REPL where you can write expressions and evaluate them. If started with a script file as the argument, it executes the script and populates the globals with symbols exported by the script (variables, function definitions and so on). If the --exec argument is provided, the REPL exits after executing the script and printing its outputs.

Press Ctrl+D to exit the REPL. You can load script files from disk using the load function.

>>> load("my_script.star", "my_script")
>>> my_script.foo("bar") # Executes the `foo` function exported by the script.

Standard environment

In addition to the built-in language constructs, the following variables and functions are available to use by any script.

Name Description

cerbos.cache_delete(key)

Delete the cached value from Synapse cache

cerbos.cache_get(key)

Return the cached value from Synapse cache

cerbos.cache_set(key, value, expiry?, if_not_exists?)

Save a value to shared Synapse cache with optional expiry and existence check

cerbos.check_resources(req)

Do a CheckResources call to the PDP. Use the struct function to construct the request

cerbos.data_source_lookup(datasource, query, query_parameters?, cache_options?)

Do a lookup using one of the configured data sources

cerbos.plan_resources(req)

Do a PlanResources call to the PDP. Use the struct function to construct the request

context.extension_config

The optional configuration map attached to the extension definition in the Synapse configuration file

context.extension_kind

The kind of the extension. One of datasource, envoy, proxy or route

context.extension_name

The name of this extension as defined in the Synapse configuration file

math.ceil(x)

Ceiling of x

math.fabs(x)

Absolute value of x as a float

math.floor(x)

Floor of x

math.mod(x, y)

Value of x modulo y

math.pow(x, y)

Value of x raised to the power of y

math.remainder(x, y)

Remainder of x/y

math.round(x)

Round x to nearest integer

struct()

Create structs. E.g. struct(k1 = "foo", k2 = "bar)

time.from_timestamp(sec, nsec?)

Converts the given Unix time corresponding to the number of seconds and (optionally) nanoseconds since January 1, 1970 UTC into an object of type Time

time.hour

A constant representing a duration of one hour

time.is_valid_timezone(loc)

Reports whether loc is a valid time zone name

time.microsecond

A constant representing a duration of one microsecond

time.millisecond

A constant representing a duration of one millisecond

time.minute

A constant representing a duration of one minute

time.nanosecond

A constant representing a duration of one nanosecond

time.now()

Returns the current local time

time.parse_duration(d)

Parses the given duration string. For more details, refer to https://pkg.go.dev/time#ParseDuration

time.parse_time(x, format?, location?)

Parses the given time string using a specific time format and location. The expected arguments are a time string (mandatory), a time format (optional, set to RFC3339 by default, e.g. "2021-03-22T23:20:50.52Z") and a name of location (optional, set to UTC by default). For more details, refer to https://pkg.go.dev/time#Parse and https://pkg.go.dev/time#ParseInLocation.

time.second

A constant representing a duration of one second

time.time(year?, month?, day?, hour?, minute?, second?, nanosecond?, location?)

Returns the Time corresponding to yyyy-mm-dd hh:mm:ss + nsec nanoseconds in the appropriate zone for that time in the given location. All the parameters are optional.

Loadable modules

Optional modules can be imported using the load function.

# Load the "json" module as "json" which allows you to invoke its functions like `json.dumps` and so on.
load("json", "json")
print(json.dumps(x))

# Load the "hashlib" module as "hash" which allows you to invoke its functions like `hash.sha256` and so on.
load("hashlib", hash="hashlib")
print(hash.sha256(c))

The following modules are available to load.

base64

Function Description Signature

base64.decode

Decode base64 string to plain text

decode(src,encoding="standard") → string

base64.encode

Encode string to base64

encode(src,encoding="standard") → string

csv

Function Description Signature

csv.read_all

Read CSV string into list of string lists

read_all(source, comma=",", comment="", lazy_quotes=False, trim_leading_space=False, fields_per_record=0, skip=0, limit=0) → [][]string

csv.write_all

Write list of string lists to CSV string

write_all(source, comma=",") → string

csv.write_dict

Write list of dictionaries to CSV string

write_dict(data, header, comma=",") → string

hashlib

Function Description Signature

hashlib.md5

Calculate MD5 hash

md5(data) → string

hashlib.sha1

Calculate SHA-1 hash

sha1(data) → string

hashlib.sha256

Calculate SHA-256 hash

sha256(data) → string

hashlib.sha512

Calculate SHA-512 hash

sha512(data) → string

http

Function Description Signature

http.call

Perform HTTP request with specified method

call(method, url, …​) → response

http.delete

Perform HTTP DELETE request

delete(url, …​) → response

http.get

Perform HTTP GET request

get(url, …​) → response

http.get_timeout

Get current timeout setting

get_timeout() → float

http.options

Perform HTTP OPTIONS request

options(url, …​) → response

http.patch

Perform HTTP PATCH request

patch(url, …​) → response

http.post

Perform HTTP POST request

post(url, …​) → response

http.postForm

Perform HTTP POST request with form data

postForm(url, …​) → response

http.put

Perform HTTP PUT request

put(url, …​) → response

http.set_timeout

Set global timeout for requests

set_timeout(timeout)

json

Function Description Signature

json.decode

Convert JSON to value with optional default

decode(x,default?) → any

json.dumps

Convert value to JSON string with indentation

dumps(obj, indent=0) → string

json.encode

Convert value to JSON

encode(x) → string

json.eval

Evaluate JSONPath expression

eval(data, expr) → value

json.indent

Pretty-print JSON with indentation

indent(str, prefix="", indent="\t") → string

json.path

Query JSON using JSONPath

path(data, path) → list

json.try_decode

Decode JSON with error handling

try_decode(x) → tuple

json.try_dumps

Dump JSON with error handling

try_dumps(obj, indent=0) → tuple

json.try_encode

Encode JSON with error handling

try_encode(x) → tuple

json.try_eval

Evaluate JSONPath with error handling

try_eval(data, expr) → tuple

json.try_indent

Indent JSON with error handling

try_indent(str, prefix="", indent="\t") → tuple

json.try_path

Query JSONPath with error handling

try_path(data, path) → tuple

random

Function Description Signature

random.choice

Return random element from sequence

choice(seq) → any

random.choices

Return k-sized list of random elements

choices(population, weights=None, cum_weights=None, k=1) → list

random.randb32

Generate random base32 string

randb32(n, sep) → string

random.randbytes

Generate random byte string

randbytes(n) → bytes

random.randint

Return random integer in range

randint(a,b) → int

random.random

Return random float in [0.0, 1.0)

random() → float

random.randstr

Generate random string from charset

randstr(chars, n) → string

random.shuffle

Shuffle sequence in place

shuffle(x)

random.uniform

Return random float in range

uniform(a, b) → float

random.uuid

Generate random UUID

uuid() → string

re

Function Description Signature

re.compile

Compile regex pattern

compile(pattern) → Pattern

re.findall

Find all matches in string

findall(pattern, text, flags=0) → list

re.match

Match pattern at string start

match(pattern, string, flags=0) → match

re.search

Search for pattern in string

search(pattern,string,flags=0) → match

re.split

Split string by pattern

split(pattern, text, maxsplit=0, flags=0) → list

re.sub

Replace pattern matches in string

sub(pattern, repl, text, count=0, flags=0) → string

stats

Function Description Signature

stats.average

Calculate mean (alias)

average(data) → float

stats.correlation

Calculate correlation coefficient

correlation(data1, data2) → float

stats.covariance

Calculate covariance

covariance(data1, data2) → float

stats.covariance_population

Calculate population covariance

covariance_population(data1, data2) → float

stats.euclidean_distance

Calculate Euclidean distance

euclidean_distance(data1, data2) → float

stats.geometric_mean

Calculate geometric mean

geometric_mean(data) → float

stats.harmonic_mean

Calculate harmonic mean

harmonic_mean(data) → float

stats.manhattan_distance

Calculate Manhattan distance

manhattan_distance(data1, data2) → float

stats.max

Find maximum value

max(data) → float

stats.mean

Calculate arithmetic mean

mean(data) → float

stats.median

Find median value

median(data) → float

stats.midrange

Calculate midrange

midrange(data) → float

stats.min

Find minimum value

min(data) → float

stats.mode

Find most frequent values

mode(data) → list

stats.pearson

Calculate Pearson correlation

pearson(data1, data2) → float

stats.percentile

Calculate percentile

percentile(data, p) → float

stats.percentile_nearest_rank

Calculate percentile by nearest rank

percentile_nearest_rank(data, p) → float

stats.population_variance

Calculate population variance

population_variance(data) → float

stats.sample

Randomly sample elements

sample(data, take, replace=False) → list

stats.sample_variance

Calculate sample variance

sample_variance(data) → float

stats.sigmoid

Apply sigmoid function

sigmoid(data) → list

stats.softmax

Apply softmax function

softmax(data) → list

stats.standard_deviation

Calculate standard deviation

standard_deviation(data) → float

stats.stddev

Calculate standard deviation (alias)

stddev(data) → float

stats.stddev_sample

Calculate sample standard deviation

stddev_sample(data) → float

stats.sum

Calculate sum

sum(data) → float

stats.trimean

Calculate trimean

trimean(data) → float

stats.variance

Calculate variance

variance(data) → float

string

Function Description Signature

string.codepoint

Get Unicode codepoint at index

codepoint(s, index) → string

string.escape

Escape HTML entities

escape(str) → string

string.find

Find first occurrence of substring

find(s, sub) → int

string.index

Find first occurrence (raises error)

index(s, sub) → int

string.length

Get length in Unicode code points

length(obj) → int

string.quote

Shell-escape string

quote(str) → string

string.reverse

Reverse string

reverse(str) → string

string.rfind

Find last occurrence of substring

rfind(s, sub) → int

string.rindex

Find last occurrence (raises error)

rindex(s, sub) → int

string.substring

Extract substring

substring(s, start, end) → string

string.unescape

Unescape HTML entities

unescape(str) → string

string.unquote

Shell-unescape string

unquote(str) → string

Data source extension

A script that implements a data source must export a function named lookup. This function will be called by Synapse with an object in the following shape:

Input
struct(
    data_source = "myDataSource", (1)
    query = "SELECT department FROM employees WHERE id = :employee_id", (2)
    query_parameters = { (3)
        "employee_id": "simon"
    },
    cache_options = struct( (4)
        cache_key = "simon", (5)
        cache_expiry = time.minute, (6)
        if_not_exists = True (7)
    )
}
1 Name of the data source to query
2 Query to execute on the data source. This is specific to the data source. The value can be any type — including complex objects and arrays
3 Optional query parameters as a dictionary with string keys and values of any type
4 Optional cache options for caching the result of the lookup
5 Cache key for this lookup
6 Optional duration for caching the result of the lookup
7 Optional flag to prevent overwriting an existing cache key

The function must return a struct that contains a field named result with the result of the lookup.

def lookup(req):
    result = {"output": "hello from starlark lookup", "query": req.query}
    return struct(result = result)

Envoy extension

An Envoy extension must export a function named envoy_check that accepts an object in the shape of Envoy check request and returns one of the following responses.

Direct response

Return a complete Envoy check response to return back to the caller.

struct(envoy_check_response = struct(
    status = struct(code = 7),
    denied_response = struct(body = "Go away")
))

Cerbos mapping

Return a struct that contains a Cerbos CheckRequest and the Envoy check response to send based on the Cerbos response. Synapse sends the Cerbos request to the configured PDP (applying any configured proxy extensions) and responds with the appropriate Envoy response based on whether Cerbos returns ALLOW or DENY.

struct(envoy_cerbos_mapping = struct(
    check_request = struct( (1)
        principal = struct(
            id = "daffy",
            roles = ["duck"]
        ),
        resources = [
            struct(
                actions = ["GET"],
                resource = struct(
                    id = "http",
                    kind = "request",
                    attr = {
                        "path": "/foo"
                    }
                )
            )
        ]
    ),
    allow_response = struct( (2)
        status = struct(code = 0)
    ),
    deny_response = struct( (3)
        status = struct(code = 7),
        denied_response = struct(body = "Go away")
    )
))
1 Cerbos CheckResources request to send
2 Envoy response to send if Cerbos returns EFFECT_ALLOW
3 Envoy response to send if Cerbos returns EFFECT_DENY

Cerbos check request

Return a Cerbos CheckRequest to send to the PDP (applying any configured proxy extensions). In this mode, the extension must implement a second function named map_cerbos_response to map the CheckResources response to an Envoy response. Use this mode when you need to write complex mapping logic to convert a Cerbos response to an Envoy response.

struct(cerbos_check_request = struct(
    principal = struct(
        id = "daffy",
        roles = ["duck"]
    ),
    resources = [
        struct(
            actions = ["GET"],
            resource = struct(
                id = "http",
                kind = "request",
                attr = {
                    "path": "/foo"
                }
            )
        )
    ]
))

Synapse will send the above request to the PDP and invoke the extension a second time by calling the map_cerbos_response function with an object of the following shape:

{
    envoy_request: { ... }, (1)
    cerbos_ request: { ... }, (2)
    cerbos_response: { ... } (3)
}
1 Original Envoy Check request that triggered this chain
2 Cerbos CheckResources request
3 Cerbos CheckResources response

The map_cerbos_response function should take the above input and return a struct representing an Envoy check response which will be sent back to the caller.

struct(
    status = struct(code = 7),
    denied_response = struct(
        body = "Denied by policy secops-general"
    )
)

Example

The following example invokes a PDP call for any paths that are in the secure_paths configuration value.

def envoy_check(req):
    http_req = req.attributes.request.http
    path = http_req.path

    if path in context.extension_config["secure_paths"]:
        return struct(cerbos_mapping = struct(
            check_request = struct(
                principal = struct(id = "daffy", roles = ["user"]),
                resources = [struct(
                    resource = struct(id = "x", kind = "request", attr = {"path": path}),
                    actions = [http_req.method]
                )]
            ),
            allow_response = struct(
                status = struct(code = 0),
                ok_response = struct(
                    headers_to_remove = ["bar"]
                )
            ),
            deny_response = struct(
                status = struct(code = 7),
                denied_response = struct(
                    body = "no go"
                )
            )
        ))

    return struct(envoy_check_response = struct(
        status = struct(code = 0),
        ok_response = struct(
            headers_to_remove = ["foo"]
        )
    ))

Proxy extension

A proxy extension must export at least one of the following functions.

augment_check_request

Modify a CheckResources request

augment_check_response

Modify a CheckResources response

augment_plan_request

Modify a PlanResources request

augment_plan_response

Modify a PlanResources response

Each function is invoked with an object in the shape of a CheckResources/PlanResources request or response. The function can modify the object as required and return it back. Refer to https://docs.cerbos.dev/cerbos/latest/api/#_request_and_response_formats for details about the request and response types.

Example

The following example injects hard-coded attributes to a known principal.

def handle_augment(req):
    if hasattr(req, "principal"):
        if req.principal.id == "john":
            req.principal.attr = {
                "geography": "GB",
                "department": "IT"
            }

    return req

def augment_check_request(req):
    return handle_augment(req)

def augment_plan_request(req):
    return handle_augment(req)

Route extension

A route extension must export a function named handle_http_route which receives an object in the following shape as the argument.

struct(
    method = "POST", (1)
    headers = { (2)
        "x-forwarded-for": struct(values = ["127.0.0.1:2090"])
    },
    raw_url = "https://example.com/path/to/foo?a=av&b=bv1&b=bv2", (3)
    host = "example.com", (4)
    path = "/path/to/foo", (5)
    query_params = { (6)
        "a": struct(values = ["av"]),
        "b": struct(values = ["bv1", "bv2"])
    },
    body = "Hello" (7)
}
1 HTTP method
2 Header values
3 Full URL of the request
4 Host segment from the URL
5 Path segment from the URL
6 Query parameters from the URL
7 Request body as a bytes string

The return value of the function must be one of the following values.

Direct response

Return a complete HTTP response to be sent back to the caller.

struct(http_response = struct(
    status = 401 (1)
    headers = { (2)
        "content-type": struct(values = ["application/text"])
    },
    body = "Access denied" (3)
))
1 HTTP status code
2 Headers to add to the response
3 Body of the response

Cerbos mapping

Return an object that represents a Cerbos CheckResources request and the HTTP response to send based on the Cerbos response. Synapse sends the Cerbos request to the configured PDP (applying any configured proxy extensions) and responds with the appropriate HTTP response based on whether Cerbos returns ALLOW or DENY.

struct(cerbos_mapping = struct(
    check_request = struct( (1)
        principal = struct(
            id = "daffy",
            roles = ["duck"]
        ),
        resources = [
            struct(
                actions = ["GET"],
                resource = struct(
                    id = "http",
                    kind = "request",
                    attr = {
                        "path": "/foo"
                    }
                )
            )
        ]
    ),
    allow_response = struct( (2)
        status = 200,
        body = "Welcome"
    ),
    deny_response = struct( (3)
        status = 401,
        body = "Access denied"
    )
))
1 Cerbos CheckResources request
2 HTTP response to send if Cerbos returns EFFECT_ALLOW
3 HTTP response to send if Cerbos resturns EFFECT_DENY

Cerbos request

Return an object that contains a Cerbos CheckResources request. In this mode, the extension must export a second function named handle_cerbos_response that accepts the CheckResources response and returns a HTTP response. Use this mode when you want to construct a complex HTTP response using values from the Cerbos response.

struct(check_request = struct(
    principal = struct(
        id = "daffy",
        roles = ["duck"]
    ),
    resources = [
        struct(
            actions = ["GET"],
            resource = struct(
                id = "http",
                kind = "request",
                attr = {
                    "path": "/foo"
                }
            )
        )
    ]
))

Synapse sends the Check request to the PDP and then invokes the handle_cerbos_response function of the extension with the following information.

struct(
    http_request = struct( ... ), (1)
    cerbos_request =  struct( ... ), (2)
    cerbos_response = struct( ... ) (3)
}
1 Original HTTP request that triggered this route extension
2 Cerbos CheckResources request
3 Cerbos CheckResources response

handle_cerbos_response should return a HTTP response to send back to the caller.

struct(
    status = 401,
    headers = {
        "content-type": struct(values = ["application/json"])
    },
    body = "Access denied"
}

Example

def handle_http_route(req):
    return struct(cerbos_mapping = struct(
        check_request = struct(
            principal = struct(id = "daffy", roles = ["user"]),
            resources = [struct(
                resource = struct(id = "x", kind = "request", attr = {"path": req.path}),
                actions = [req.method]
            )]
        ),
        allow_response = struct(
            status = 200,
            headers = {"foo": struct(values = ["foo"])},
            body = "Welcome"
        ),
        deny_response = struct(
            status = 403,
            headers = {"foo": struct(values = ["foo"])},
            body = "No entry"
        )
    ))