Developing Starlark extensions
Starlark is a lightweight Python dialect that can be used to drive the Synapse extension logic. They are easy to write and test because they do not require any intermediate compilation stages or specialised tools.
| For an end-to-end walkthrough of building a Starlark proxy extension and running it against Synapse, see the Quick Start. This page is the reference for the Starlark runtime and the request/response shapes each extension type receives. |
To use a Starlark script as the extension implementation, set the extensionURL configuration field to a valid extension URL in one of the following forms:
-
/path/to/script.star(File name must have a.starextension) -
starlark+http://scripts.example.com/synapse/envoy?checksum=sha256:677cce2788330f16f27981130eaa50aa4189beab2121903bcea5b3c4c918dccc(Protocol of the URL must be prefixed withstarlark+)
All Synapse extension points support running Starlark scripts. Each request is served by a fresh instance of a script and there’s no shared global state. Each extension type requires a specific set of functions that must be implemented by the script. See below for details about the API contracts for each extension type.
Starlark primer
Starlark is a lightweight dialect of Python with syntax compatible with Python 3. Refer to the language specification for supported operators, keywords and types. Synapse extends the base language with extra functions and loadable modules that are useful for authoring extensions. The built-in REPL can be used to experiment with the language and debug your scripts.
Constructing input/output messages for extensions
The API contracts for the extensions require constructing specific message types whose schemas are defined as protocol buffers (protobufs). The mapping rules for constructing protobuf values in Starlark are as follows:
| Protobuf type | Starlark constructor |
|---|---|
Corresponding Starlark scalar type |
|
|
|
Maps |
|
|
|
Enums |
String value of the enum |
|
Corresponding Starlark scalar type |
For example, a Cerbos CheckResources request would be constructed as follows:
struct( (1)
request_id = "test", (2)
principal = struct(id = "john", roles = ["employee"]) <3>,
resources = [struct( (4)
actions = ["view"], (5)
resource = struct(
kind = "invoice",
id = "XX125",
attr = { (6)
"owner": "john",
"department": "IT",
"geography": "GB",
"groups": ["it_admins", "employees"]
}
)
)]
)
| 1 | CheckResources is a protobuf Message |
| 2 | request_id field is a string |
| 3 | principal is another Message type |
| 4 | resources is a repeated Message field |
| 5 | actions is a repeated string field |
| 6 | attr is a map<string,google.protobuf.Value> |
API contracts for extensions
Data source extensions
A script that implements a data source must export a function named lookup. This function will be called by Synapse with an object in the following shape:
struct(
data_source = "myDataSource", (1)
query = "SELECT department FROM employees WHERE id = :employee_id", (2)
query_parameters = { (3)
"employee_id": "simon"
},
cache_options = struct( (4)
cache_key = "simon", (5)
cache_expiry = time.minute, (6)
if_not_exists = True (7)
)
)
| 1 | Name of the data source to query |
| 2 | Query to execute on the data source. This is specific to the data source. The value can be any type — including complex objects and arrays |
| 3 | Optional query parameters as a dictionary with string keys and values of any type |
| 4 | Optional cache options for caching the result of the lookup |
| 5 | Cache key for this lookup |
| 6 | Optional duration for caching the result of the lookup |
| 7 | Optional flag to prevent overwriting an existing cache key |
The function must return a struct that contains a field named result with the result of the lookup.
def lookup(req):
result = {"output": "hello from starlark lookup", "query": req.query}
return struct(result = result)
Envoy extension
An Envoy extension must export a function named envoy_check that accepts an object in the shape of Envoy check request. The return value from the function should be one of the following:
| Direct response |
An Envoy |
| Cerbos mapping |
Construct a Cerbos |
| Cerbos request |
Construct a Cerbos |
Direct response
Return a complete Envoy check response to return back to the caller. This is useful for situations where you know the exact response to send such as a blanket deny for a hard-coded path.
def envoy_check(req):
path = req.attributes.request.http.path
# Deny the request if the path starts with /restricted
if path.startswith("/restricted/"):
return struct(envoy_check_response = struct(
status = struct(code = 7),
denied_response = struct(body = "Go away")
))
# Allow the request to go through with one of the headers removed
return struct(envoy_check_response = struct(
status = struct(code = 0),
ok_response = struct(
headers_to_remove = ["x-confidential-header"]
)
))
The following example invokes a PDP call for any paths that are in the secure_paths configuration value of the extension.
def envoy_check(req):
http_req = req.attributes.request.http
path = http_req.path
if path in context.extension_config["secure_paths"]:
return struct(cerbos_mapping = struct(
check_request = struct(
principal = struct(id = "daffy", roles = ["user"]),
resources = [struct(
resource = struct(id = "x", kind = "request", attr = {"path": path}),
actions = [http_req.method]
)]
),
allow_response = struct(
status = struct(code = 0),
ok_response = struct(
headers_to_remove = ["bar"]
)
),
deny_response = struct(
status = struct(code = 7),
denied_response = struct(
body = "no go"
)
)
))
return struct(envoy_check_response = struct(
status = struct(code = 0),
ok_response = struct(
headers_to_remove = ["foo"]
)
))
Cerbos mapping
Return a struct that contains a Cerbos CheckRequest and the Envoy check response to send based on the Cerbos response. Synapse sends the Cerbos request to the configured PDP (applying any configured proxy extensions) and responds with the appropriate Envoy response based on whether Cerbos returns ALLOW or DENY.
def envoy_check(req):
# Add code here to inspect the request and extract the information you need to pass to Cerbos
# Return the Cerbos request and the corresponding Envoy response to return based on the decision from the PDP
return struct(cerbos_mapping = struct(
check_request = struct( (1)
principal = struct(
id = "daffy",
roles = ["duck"]
),
resources = [
struct(
actions = ["GET"],
resource = struct(
id = "http",
kind = "request",
attr = {
"path": "/foo"
}
)
)
]
),
allow_response = struct( (2)
status = struct(code = 0)
),
deny_response = struct( (3)
status = struct(code = 7),
denied_response = struct(body = "Go away")
)
))
| 1 | Cerbos CheckResources request to send |
| 2 | Envoy response to send if Cerbos returns EFFECT_ALLOW |
| 3 | Envoy response to send if Cerbos returns EFFECT_DENY |
Cerbos check request
Return a Cerbos CheckRequest to send to the PDP (applying any configured proxy extensions). In this mode, the extension must implement a second function named map_cerbos_response to map the CheckResources response to an Envoy response. Use this mode when you need to write complex mapping logic to convert a Cerbos response to an Envoy response.
def envoy_check(req): (1)
# Add code here to inspect the request and extract the information you need to pass to Cerbos
# Return the Cerbos request
return struct(cerbos_check_request = struct(
principal = struct(
id = "daffy",
roles = ["duck"]
),
resources = [
struct(
actions = ["GET"],
resource = struct(
id = "http",
kind = "request",
attr = {
"path": "/foo"
}
)
)
]
))
def map_cerbos_response(resp): (2)
# Inspect the Cerbos response here and construct the appropriate Envoy response
# Return the constructed Envoy response
return struct(
status = struct(code = 0),
ok_response = struct(
headers_to_remove = ["baz"]
)
)
| 1 | First entrypoint which receives the Envoy Check request and returns a Cerbos CheckResources request |
| 2 | Second entrypoint which receives the Cerbos CheckResources response and returns the final Envoy Check response |
Synapse will send the Cerbos CheckResources request returned by the envoy_check function to the PDP and then invoke the extension a second time by calling the map_cerbos_response function with an object of the following shape:
struct(
envoy_request = struct( ... ), (1)
cerbos_request = struct( ... ), (2)
cerbos_response = struct( ... ) (3)
)
| 1 | Original Envoy Check request that triggered this chain |
| 2 | Cerbos CheckResources request |
| 3 | Cerbos CheckResources response |
The map_cerbos_response function should take the above input and return a struct representing an Envoy check response which will be sent back to the caller.
Proxy extensions
A proxy extension must export at least one of the following functions.
augment_authzen_evaluation_batch_request
|
Modify an AuthZEN AccessEvaluations request |
augment_authzen_evaluation_batch_response
|
Modify an AuthZEN AccessEvaluations response |
augment_authzen_evaluation_request
|
Modify an AuthZEN AccessEvaluation request |
augment_authzen_evaluation_response
|
Modify an AuthZEN AccessEvaluation response |
augment_check_request
|
Modify a CheckResources request |
augment_check_response
|
Modify a CheckResources response |
augment_plan_request
|
Modify a PlanResources request |
augment_plan_response
|
Modify a PlanResources response |
Each function is invoked with an object in the shape of a AccessEvaluation/CheckResources/PlanResources request or response. The function can modify the object as required and return it back.
Example: inject hard-coded attributes into Cerbos CheckResources/PlanResources calls
The following example injects hard-coded attributes to a known principal.
def handle_augment(req):
if hasattr(req, "principal"):
if req.principal.id == "john":
req.principal.attr = {
"geography": "GB",
"department": "IT"
}
return req
def augment_check_request(req):
return handle_augment(req)
def augment_plan_request(req):
return handle_augment(req)
Example: enrich principals from a data source
Consider a Cerbos policy that requires attributes about the principal.
apiVersion: api.cerbos.dev/v1
resourcePolicy:
resource: "invoice"
version: "default"
rules:
- actions: ["view"]
effect: EFFECT_ALLOW
roles: ["user"]
condition:
match:
expr: P.attr.employee.department == R.attr.department
A Synapse proxy extension can read the principal.id from each CheckResources and PlanResources request, look up the principal from a data source named employees (for example, the built-in sqldb data source backed by PostgreSQL), and inject the found attributes into principal.attr.employee for policies to use.
QUERY = "SELECT department, country, manager_email FROM employees WHERE id = :uid"
def enrich(req):
if not hasattr(req, "principal") or req.principal.id == "":
return req
resp = cerbos.data_source_lookup( (1)
datasource = "employees",
query = QUERY,
query_parameters = {"uid": req.principal.id}, (2)
cache_key = "employee:" + req.principal.id, (3)
cache_expiry = 5 * time.minute,
)
if resp == None or resp.result == None: (4)
return req
req.principal.attr["employee"] = resp.result
return req
def augment_check_request(req):
return enrich(req)
def augment_plan_request(req):
return enrich(req)
| 1 | Call into the data source by its configured name. The same cerbos.data_source_lookup function works for any configured data source — built-in or custom. |
| 2 | Bind query parameters using query_parameters rather than string-concatenating values into the SQL. Named placeholders (:uid) keep the query safe from injection. |
| 3 | cache_key and cache_expiry are optional. When set, repeated lookups for the same principal are served from the Synapse cache without hitting the database. |
| 4 | resp.result is None if the query returned no rows, a single object (column to value) for a single-row result, or a list of objects for multi-row results. |
Route extensions
A route extension must export a function named handle_http_route which receives an object describing the HTTP request received and return one of the supported result types to produce the HTTP response.
| Direct response |
A HTTP response that will be sent as-is back to the caller. |
| Cerbos mapping |
Construct a Cerbos |
| Cerbos request |
Construct a Cerbos |
Route extensions are mounted under the /ext/ path prefix at the Synapse listen address, and that prefix is not stripped before the request reaches the extension. For a route configured as /foo and a client request to POST /ext/foo?a=av&b=bv1&b=bv2, the input takes the following form.
struct(
method = "POST", (1)
headers = { (2)
"X-Forwarded-For": struct(values = ["127.0.0.1:2090"])
},
raw_url = "https://example.com/ext/path/to/foo?a=av&b=bv1&b=bv2", (3)
host = "example.com", (4)
path = "/ext/path/to/foo", (5)
query_params = { (6)
"a": struct(values = ["av"]),
"b": struct(values = ["bv1", "bv2"])
},
body = "Hello" (7)
)
| 1 | HTTP method. |
| 2 | Headers and their values. Header names are in canonical form, also known as train case (X-Forwarded-For). |
| 3 | The URL including the /ext/ prefix and any query parameters. This will be a relative URL without the protocol and host unless it’s an absolute request sent via a proxy. |
| 4 | Host segment from the URL. |
| 5 | Path segment, including the /ext/ prefix. |
| 6 | Query parameters from the URL. |
| 7 | Request body as a bytes string. |
The return value of the function must be one of the following types.
Direct response
Return the complete HTTP response that should be sent back to the caller.
def handle_http_route(req):
# Add code here to process the request and determine a response.
# Return the response
return struct(http_response = struct(
status = 401, (1)
headers = { (2)
"Content-Type": struct(values = ["application/text"])
},
body = "Access denied" (3)
))
| 1 | HTTP status code |
| 2 | Headers to add to the response |
| 3 | Body of the response |
Cerbos mapping
Return an object that contains a Cerbos CheckResources request and the HTTP response to send based on the response to that Cerbos request. Synapse sends the Cerbos request to the configured PDP (applying any configured proxy extensions) and responds with the appropriate HTTP response based on whether Cerbos returns ALLOW or DENY.
def handle_http_route(req):
# Add code here to examine the request and construct a Cerbos CheckResources request from it.
# Return the Cerbos request and the HTTP response to send based on the Cerbos response.
return struct(cerbos_mapping = struct(
check_request = struct( (1)
principal = struct(
id = "daffy",
roles = ["duck"]
),
resources = [
struct(
actions = ["GET"],
resource = struct(
id = "http",
kind = "request",
attr = {
"path": "/foo"
}
)
)
]
),
allow_response = struct( (2)
status = 200,
body = "Welcome"
),
deny_response = struct( (3)
status = 401,
body = "Access denied"
)
))
| 1 | Cerbos CheckResources request |
| 2 | HTTP response to send if Cerbos returns EFFECT_ALLOW |
| 3 | HTTP response to send if Cerbos resturns EFFECT_DENY |
Example
def handle_http_route(req):
return struct(cerbos_mapping = struct(
check_request = struct(
principal = struct(id = "daffy", roles = ["user"]),
resources = [struct(
resource = struct(id = "x", kind = "request", attr = {"path": req.path}),
actions = [req.method]
)]
),
allow_response = struct(
status = 200,
headers = {"foo": struct(values = ["foo"])},
body = "Welcome"
),
deny_response = struct(
status = 403,
headers = {"foo": struct(values = ["foo"])},
body = "No entry"
)
))
Cerbos request
Return an object that contains a Cerbos CheckResources request. In this mode, the extension must export a second function named handle_cerbos_response that accepts the CheckResources response and returns a HTTP response. Use this mode when you want to construct a complex HTTP response using values from the Cerbos response.
def handle_http_route(req):
# Add code here to inspect the request and construct a Cerbos CheckResources request from it.
# Return the Cerbos request constructed above
return struct(check_request = struct(
principal = struct(
id = "daffy",
roles = ["duck"]
),
resources = [
struct(
actions = ["GET"],
resource = struct(
id = "http",
kind = "request",
attr = {
"path": "/foo"
}
)
)
]
))
def handle_cerbos_response(resp):
# Add code here to inspect the Cerbos response and construct the appropriate HTTP response
# Return the constructed HTTP response
return struct(http_response = struct(
status = 401,
headers = {
"Content-Type": struct(values = ["application/text"])
},
body = "Access denied"
))
Synapse sends the Check request to the PDP and then invokes the handle_cerbos_response function of the extension with the following information.
struct(
http_request = struct( ... ), (1)
cerbos_request = struct( ... ), (2)
cerbos_response = struct( ... ) (3)
)
| 1 | Original HTTP request that triggered this route extension |
| 2 | Cerbos CheckResources request |
| 3 | Cerbos CheckResources response |
handle_cerbos_response should return a HTTP response to send back to the caller.
struct(
status = 401,
headers = {
"Content-Type": struct(values = ["application/json"])
},
body = "Access denied"
)
Starlark REPL
Synapse ships with a REPL (read, evaluate, print loop) to help with debugging Starlark scripts.
Synapse starlark repl [--exec] [SCRIPT_FILE]
Without any arguments, the command starts the REPL where you can write expressions and evaluate them. If started with a script file as the argument, it executes the script and populates the globals with symbols exported by the script (variables, function definitions and so on). If the --exec argument is provided, the REPL exits after executing the script and printing its outputs.
Press Ctrl+D to exit the REPL. You can load script files from disk using the load function.
>>> load("my_script.star", "my_script")
>>> my_script.foo("bar") # Executes the `foo` function exported by the script.
Starlark standard environment
In addition to the built-in language constructs, the following variables and functions are available to use by any script.
| Name | Description |
|---|---|
|
Delete the cached value from Synapse cache |
|
Return the cached value from Synapse cache |
|
Save a value (string or bytes) to shared Synapse cache with optional expiry and existence check |
|
Do a |
|
Do a lookup using one of the configured data sources. Cache options are passed as flat keyword arguments on the caller side; the inbound request struct received by a data source’s |
|
Do a |
|
The optional configuration map attached to the extension definition in the Synapse configuration file |
|
The kind of the extension. One of |
|
The name of this extension as defined in the Synapse configuration file |
|
Ceiling of x |
|
Absolute value of x as a float |
|
Floor of x |
|
Value of x modulo y |
|
Value of x raised to the power of y |
|
Remainder of x/y |
|
Round x to nearest integer |
|
Create structs. E.g. |
|
Converts the given Unix time corresponding to the number of seconds and (optionally) nanoseconds since January 1, 1970 UTC into an object of type Time |
|
A constant representing a duration of one hour |
|
Reports whether loc is a valid time zone name |
|
A constant representing a duration of one microsecond |
|
A constant representing a duration of one millisecond |
|
A constant representing a duration of one minute |
|
A constant representing a duration of one nanosecond |
|
Returns the current local time |
|
Parses the given duration string. For more details, refer to https://pkg.go.dev/time#ParseDuration |
|
Parses the given time string using a specific time format and location. The expected arguments are a time string (mandatory), a time format (optional, set to RFC3339 by default, e.g. "2021-03-22T23:20:50.52Z") and a name of location (optional, set to UTC by default). For more details, refer to https://pkg.go.dev/time#Parse and https://pkg.go.dev/time#ParseInLocation. |
|
A constant representing a duration of one second |
|
Returns the Time corresponding to yyyy-mm-dd hh:mm:ss + nsec nanoseconds in the appropriate zone for that time in the given location. All the parameters are optional. |
Loadable modules
Optional modules can be imported using the load function.
# Load the "json" module as "json" which allows you to invoke its functions like `json.dumps` and so on.
load("json", "json")
print(json.dumps(x))
# Load the "hashlib" module as "hash" which allows you to invoke its functions like `hash.sha256` and so on.
load("hashlib", hash="hashlib")
print(hash.sha256(c))
The following modules are available to load.
base64
| Function | Description | Signature |
|---|---|---|
|
Decode base64 string to plain text |
|
|
Encode string to base64 |
|
Valid values for encoding are standard, standard_raw, url and url_raw.
csv
| Function | Description | Signature |
|---|---|---|
|
Read CSV string into list of string lists |
|
|
Write list of string lists to CSV string |
|
|
Write list of dictionaries to CSV string |
|
hashlib
| Function | Description | Signature |
|---|---|---|
|
Calculate MD5 hash |
|
|
Calculate SHA-1 hash |
|
|
Calculate SHA-256 hash |
|
|
Calculate SHA-512 hash |
|
http
| Function | Description | Signature |
|---|---|---|
|
Perform HTTP request with specified method |
|
|
Perform HTTP DELETE request |
|
|
Perform HTTP GET request |
|
|
Get current timeout setting |
|
|
Perform HTTP OPTIONS request |
|
|
Perform HTTP PATCH request |
|
|
Perform HTTP POST request |
|
|
Perform HTTP POST request with form data |
|
|
Perform HTTP PUT request |
|
|
Set global timeout for requests |
|
json
| Function | Description | Signature |
|---|---|---|
|
Convert JSON to value with optional default |
|
|
Convert value to JSON string with indentation |
|
|
Convert value to JSON |
|
|
Evaluate JSONPath expression |
|
|
Pretty-print JSON with indentation |
|
|
Query JSON using JSONPath |
|
|
Decode JSON with error handling |
|
|
Dump JSON with error handling |
|
|
Encode JSON with error handling |
|
|
Evaluate JSONPath with error handling |
|
|
Indent JSON with error handling |
|
|
Query JSONPath with error handling |
|
oauth
| Function | Description | Signature |
|---|---|---|
|
Create an HTTP client that attaches a token obtained using OAuth client credentials flow to every request. The |
|
Synapse extensions are isolated instances and any objects created by a handler function are request-scoped (they survive only for the duration of the request). However, the oauth module allows callers to specify an optional persist_key parameter to help persist the authenticated client for longer and share it with multiple requests. This is to avoid constantly re-authenticating with the upstream OAuth server for every request. The runtime only considers the value of the persist_key when deciding whether to create a new instance of the client or reuse and existng one. Therefore, ensure that the persist_key is unique per combination of parameters required for performing OAuth flows.
|
load("oauth", "oauth")
def do_oauth_request():
client = oauth.client_credentials_client(token_url=TOKEN_URL, client_id=CLIENT_ID, client_secret=CLIENT_SECRET, scopes=[SCOPE], endpoint_params={"foo": "bar"}, persist_key=PERSIST_KEY)
resp = client.get(AUTHENTICATED_ENDPOINT)
return resp.status_code
random
| Function | Description | Signature |
|---|---|---|
|
Return random element from sequence |
|
|
Return k-sized list of random elements |
|
|
Generate random base32 string |
|
|
Generate random byte string |
|
|
Return random integer in range |
|
|
Return random float in [0.0, 1.0) |
|
|
Generate random string from charset |
|
|
Shuffle sequence in place |
|
|
Return random float in range |
|
|
Generate random UUID |
|
re
| Function | Description | Signature |
|---|---|---|
|
Compile regex pattern |
|
|
Find all matches in string |
|
|
Match pattern at string start |
|
|
Search for pattern in string |
|
|
Split string by pattern |
|
|
Replace pattern matches in string |
|
stats
| Function | Description | Signature |
|---|---|---|
|
Calculate mean (alias) |
|
|
Calculate correlation coefficient |
|
|
Calculate covariance |
|
|
Calculate population covariance |
|
|
Calculate Euclidean distance |
|
|
Calculate geometric mean |
|
|
Calculate harmonic mean |
|
|
Calculate Manhattan distance |
|
|
Find maximum value |
|
|
Calculate arithmetic mean |
|
|
Find median value |
|
|
Calculate midrange |
|
|
Find minimum value |
|
|
Find most frequent values |
|
|
Calculate Pearson correlation |
|
|
Calculate percentile |
|
|
Calculate percentile by nearest rank |
|
|
Calculate population variance |
|
|
Randomly sample elements |
|
|
Calculate sample variance |
|
|
Apply sigmoid function |
|
|
Apply softmax function |
|
|
Calculate standard deviation |
|
|
Calculate standard deviation (alias) |
|
|
Calculate sample standard deviation |
|
|
Calculate sum |
|
|
Calculate trimean |
|
|
Calculate variance |
|
string
| Function | Description | Signature |
|---|---|---|
|
Get Unicode codepoint at index |
|
|
Escape HTML entities |
|
|
Find first occurrence of substring |
|
|
Find first occurrence (raises error) |
|
|
Get length in Unicode code points |
|
|
Shell-escape string |
|
|
Reverse string |
|
|
Find last occurrence of substring |
|
|
Find last occurrence (raises error) |
|
|
Extract substring |
|
|
Unescape HTML entities |
|
|
Shell-unescape string |
|