Simple user defined functions in Azure confidential ledger (preview)

Simple user defined functions (UDFs) in Azure confidential ledger allow you to create custom JavaScript functions that can be executed inside the ledger trust boundary. This feature is designed to be simple and easy to use, allowing you to extend the functionality of the ledger API without the need for complex application development.

Using the built-in JavaScript API, you can run custom code to achieve various tasks, such as custom queries and computations, conditional checks, post-processing tasks, and more. This feature is suitable for scenarios where you need a direct integration with the existing ledger API or run lightweight custom logic in a confidential environment.

For a quick overview and demo of UDFs, watch the following video:

Use cases

Azure confidential ledger UDFs allow you to extend the functionality of the ledger by running custom logic. Some common use cases for UDFs include:

• Custom computations and queries: run standalone UDFs to read or write data into any ledger application table according to your business logic.

• Data validation and input checks: use UDFs as pre-hooks to run pre-processing actions before a ledger entry is written to the ledger, for example to sanitize input data or check for pre-conditions.

• Data enrichment and smart contracts: use UDFs as post-hooks to run post-processing actions after a ledger entry is written, for example to add custom metadata into the ledger or trigger post-write workflows.

Writing UDFs

An Azure confidential ledger UDF is an entity stored in the ledger with a unique ID and contains the JavaScript code that is executed when the UDF is called. This section describes how to write UDF code and use the JavaScript API for achieving different tasks.

Function structure

The code of a UDF requires an exported function that is the entry point of the script at the time of execution. A basic UDF code template looks like this:

export function main() {
    // Your JavaScript code here
}

export const main = () => { 
    // Your JavaScript code here 
};

Function arguments

You can specify any optional runtime arguments accepted by the UDF. The arguments' values can be passed at runtime when running the UDF using the arguments parameter.

The arguments are always passed as an array of strings. It's the user responsibility to ensure that the arguments specified in the UDF code match the arguments passed when running the UDF. The user should also ensure that the arguments are properly parsed to the expected data type at runtime.

export function main(arg1, arg2) {
    // Your JavaScript code here
}

JavaScript API

The JavaScript code of a UDF is executed inside a sandbox environment that provides a limited set of APIs.

All the JavaScript standard global functions, objects, and values can be used. A global object called ccf can be used to access specific functionalities and utilities provided by the Confidential Consortium Framework (CCF) (for example, cryptography helper functions, ledger tables accessors, etc.). The full API of the ccf global object is documented here.

You can also access contextual information of the current request by using the context global object. This object provides access to the request metadata that originated the function execution (context.request) and the user ID of the function caller (context.userId). For transaction hooks, the collection ID and the transaction contents associated to the write operation are also added to the context object (context.collectionId and context.contents respectively).

The following snippet shows some basic examples of the use of the JavaScript API:

export function main(args) {
    
    // Basic instructions
    const a = 1 + 1;

    // Basic statements
    if (a > 0) {
        console.log("a is positive");
    } else {
        console.log("a is negative or zero");
    }

    // Parse the string argument as a JSON object
    JSON.parse(args);

    // Logging utilities
    console.log("Hello world");
    
    // Math utilities
    Math.random();
    
    // CCF cryptography utilities
    ccf.crypto.digest("SHA-256", ccf.strToBuf("Hello world"));
    
    // Write to a custom ledger table
    ccf.kv["public:mytable"].set(ccf.strToBuf("myKey"), ccf.strToBuf("myValue"));

    // Read from a custom ledger table
    ccf.bufToStr(ccf.kv["public:mytable"].get(ccf.strToBuf("myKey")));

    // Read from the ledger entry table
    ccf.kv["public:confidentialledger.logs"].get(ccf.strToBuf("subledger:0"));

    // Get the request metadata that originated the function execution
    const requestMetadata = context.request;
    
    // Get the collection ID and transaction content (for transaction hooks only)
    const collectionId = context.collectionId;
    const contents = context.contents;

    // Throw exceptions
    throw new Error("MyCustomError");
}

Managing UDFs

Azure confidential ledger applications provide a dedicated CRUD API to create, read, update, and delete UDF entities. UDFs are securely stored in the ledger and are accessible only to the ledger application.

Create or update a UDF

PUT /app/userDefinedFunctions/myFunction
{
    "code": "export function main() { return "Hello World"; }",
}

Get a UDF

GET /app/userDefinedFunctions/myFunction

List UDFs

GET /app/userDefinedFunctions

Delete a UDF

DELETE /app/userDefinedFunctions/myFunction

Running UDFs

Once created, Azure confidential ledger users can execute a UDF either as a standalone function or as a transaction hook associated to a write operation. Each UDF execution runs on a separate runtime environment and sandbox, meaning that the UDF execution is isolated from other UDFs or other ledger operations.

The UDF execution can be controlled using optional properties that can be specified in the request body. The properties currently supported are:

• arguments: an array of strings that represent the arguments to be passed to the UDF. The arguments are passed in the same order as they're defined in the UDF code. The default value is an empty array.

• exportedFunctionName: the name of the exported function to be called during execution. If not specified, the default value is main.

• runtimeOptions: an object that specifies the runtime options for the UDF execution. The following options are available:

  • max_heap_bytes: the maximum heap size in bytes. The default value is 10,485,760 (10 MB).

  • max_stack_bytes: the maximum stack size in bytes. The default value is 1,048,576 (1 MB).

  • max_execution_time_ms: the maximum execution time in milliseconds. The default value is 1000 (1 second).

  • log_exception_details: a boolean value that specifies whether to log exception details. The default value is true.

  • return_exception_details: a boolean value that specifies whether to return exception details in the response. The default value is true.

Standalone functions

A UDF can be directly executed using the POST /app/userDefinedFunctions/{functionId}:execute API.

POST /app/userDefinedFunctions/myFunction:execute
{}

The request body can be used to specify optional execution parameters, such as function arguments and JavaScript runtime properties.

POST /app/userDefinedFunctions/myFunction:execute
{
    "arguments": ["arg1", "arg2"],
    "exportedFunctionName": "myMainFunction",
    "runtimeOptions": {
        "max_heap_bytes": 5,
        "max_stack_bytes": 1024,
        "max_execution_time_ms": 5000,
        "log_exception_details": true,
        "return_exception_details": true
    }
}

The response indicates the outcome of the UDF execution (succeeded or failed). If the UDF succeeded, the response includes the function returned value in string format (if any).

{
    "result": 
        {
            "returnValue": "MyReturnValue"
        }, 
    "status": "Succeeded"
}

If the UDF failed, the response includes the error message with the detailed stack trace.

{
    "error": {
        "message": "Error while executing function myFunction: Error: MyCustomError\n    at myMainFunction (myFunction)\n"
    }, 
    "status": "Failed"
}

Transaction hooks

A UDF can alternatively be executed as a hook before (pre-hook) or after (post-hook) an entry is written to the ledger as part of the ledger write API (POST /app/transactions). Hooks run in the same context of the write operation; this means that any data written to the ledger by the hooks is automatically included in the same write transaction.

The request body of the write request can be used to specify any UDF IDs to be executed as pre-hooks and post-hooks respectively.

POST /app/transactions?collectionId=myCollection
{
  "contents": "myValue", 
  "preHooks": [ 
    { 
        "id": "myPreHook"
    } 
  ], 
  "postHooks": [ 
    { 
        "id": "myPostHook" 
    }
  ] 
} 

For each hook, it's possible to specify any optional execution properties. For example:

POST /app/transactions?collectionId=myCollection
{
  "contents": "myValue", 
  "preHooks": [ 
    { 
        "id": "myPreHook", 
        "properties": { 
            "arguments": [ 
                "arg1",
                "arg2"
            ], 
            "exportedFunctionName": "myMainFunction", 
            "runtimeOptions": { 
                "max_heap_bytes": 5,
                "max_stack_bytes": 1024,
                "max_execution_time_ms": 5000,
                "log_exception_details": true,
                "return_exception_details": true
            } 
        } 
    } 
  ], 
  "postHooks": [ 
    { 
        "id": "myPostHook", 
        "properties": { 
            "arguments": [ 
                "arg1"
            ], 
            "exportedFunctionName": "myMainFunction", 
            "runtimeOptions": { 
                "max_heap_bytes": 5,
                "max_stack_bytes": 1024,
                "max_execution_time_ms": 5000,
                "log_exception_details": true,
                "return_exception_details": true
            } 
        } 
    }
  ] 
} 

You can specify up to 5 pre-hooks and post-hooks in the request body, with any combination. The hooks are always executed in the order they're provided in the request body.

If a pre-hook or post-hook fails, the entire transaction is aborted. In that case, the response contains the error message with the reason for the failure:

{
    "error": {
        "code": "InternalError",
        "message": "Error while executing function myPreHook: Error: MyCustomError\n    at myMainFunction (myPreHook)\n",
    }
}

Examples

This section walks through some practical examples of how to use UDFs in Azure confidential ledger. For the following example scenarios, we assume using Azure confidential ledger to store banking transactions for different bank users.

Context

To store a banking transaction for a user, the existing ledger write API can be used: the value of the transaction is the content of the ledger entry and the user ID can be the collection, or key, where the content is written under.

POST /app/transactions?collectionId=John
{
    "contents": "10"
}

HTTP/1.1 200 OK

Since there's no validation on the input contents, it's possible to write a non-numeric value as a contents. For example, this request succeeds even if the contents value isn't a number:

POST /app/transactions?collectionId=Mark
{
    "contents": "This is not a number"
}

HTTP/1.1 200 OK

Pre-hooks for data validation

To ensure the transaction contents is always a number, a UDF can be created to check the input contents. The following pre-hook check if the contents value is a number and throws an error if not.

PUT /app/userDefinedFunctions/validateTransaction
{
    "code": "export function main() { if (isNaN(context.contents)) { throw new Error('Contents is not a number'); } }"
}

HTTP/1.1 201 CREATED

Using the pre-hook in the write request, it's possible to enforce that the input data corresponds to the expected format. The previous request now fails as expected:

POST /app/transactions?collectionId=Mark
{
    "contents": "This is not a number",
    "preHooks": [
        {
            "functionId": "validateTransaction"
        }
    ]
}

HTTP/1.1 500 INTERNAL_SERVER_ERROR
{
  "error": {
    "code": "InternalError",
    "message": "Error while executing function validateTransaction: Error: Contents is not a number\n    at main (validateTransaction)\n"
  }
}

Valid requests containing numeric values would instead succeed as expected:

POST /app/transactions?collectionId=Mark
{
    "contents": "30",
    "preHooks": [
        {
            "functionId": "validateTransaction"
        }
    ]
}

HTTP/1.1 200 OK

Post-hooks for data enrichment

As users perform new banking transactions, we want to record when a transaction is higher than a certain threshold for audit reasons. A post-hook can be used to write custom metadata in a ledger after a write operation to indicate whether the transaction was higher than a certain threshold.

For example, a UDF can be created to check the transaction value and write a dummy message ("Alert" for high values, "Normal" otherwise) under the input user into a custom ledger table (payment_metadata) if the value is higher than 50.

PUT /app/userDefinedFunctions/detectHighTransaction
{
    "code": "export function main() { let value = 'Normal'; if (context.contents > 50) { value = 'Alert' } ccf.kv['public:payment_metadata'].set(ccf.strToBuf(context.collectionId), ccf.strToBuf(value)); }"
}

HTTP/1.1 201 CREATED

Once the UDF is successfully created, the post-hook can be used in new write requests:

POST /app/transactions?collectionId=Mark
{
    "contents": "100",
    "preHooks": [
        {
            "functionId": "validateTransaction"
        }
    ],
    "postHooks": [
        {
            "functionId": "detectHighTransaction"
        }
    ]
}

HTTP/1.1 200 OK

POST /app/transactions?collectionId=John
{
    "contents": "20",
    "preHooks": [
        {
            "functionId": "validateTransaction"
        }
    ],
    "postHooks": [
        {
            "functionId": "detectHighTransaction"
        }
    ]
}

HTTP/1.1 200 OK

Standalone UDFs for custom queries

To inspect the latest values written to the custom table payment_metadata using the post-hook, a UDF can be created to read the values from the table given an input user ID:

PUT /app/userDefinedFunctions/checkPaymentMetadataTable
{
    "code": "export function main(user) { const value = ccf.kv['public:payment_metadata'].get(ccf.strToBuf(user)); if (value === undefined) { throw new Error('UnknownUser'); } return ccf.bufToStr(value); }"
}

HTTP/1.1 201 CREATED

By running the UDF directly, it's possible to check the latest value recorded in the custom metadata table for a given user.

For users with a recent high transaction, the UDF returns value "Alert" as expected.

POST /app/userDefinedFunctions/checkPaymentMetadataTable:execute
{
    "arguments": [
        "Mark"
    ]
}

HTTP/1.1 200 OK
{
  "result": {
    "returnValue": "Alert"
  },
  "status": "Succeeded"
}

For users with a recent low transaction, the UDF returns value "Normal" instead.

POST /app/userDefinedFunctions/checkPaymentMetadataTable:execute
{
    "arguments": [
        "John"
    ]
}

HTTP/1.1 200 OK
{
  "result": {
    "returnValue": "Normal"
  },
  "status": "Succeeded"
}

For users that don't have any entry in the custom table, the UDF throws an error as defined in the UDF code.

POST /app/userDefinedFunctions/checkPaymentMetadataTable:execute
{
    "arguments": [
        "Jane"
    ]
}

HTTP/1.1 200 OK
{
  "error": {
    "message": "Error while executing function checkPaymentMetadataTable: Error: UnknownUser\n    at main (checkPaymentMetadataTable)\n"
  },
  "status": "Failed"
}

Considerations

• Transaction hooks are currently only supported for the POST /app/transactions API, when adding a new entry to the ledger.

• UDFs and hooks are always executed on the primary replica of the ledger, to ensure transaction ordering and strong consistency.

• UDF code execution is always wrapped in a single atomic transaction. If the JavaScript logic in a UDF completes without any exceptions, all the operations within the UDF are committed to the ledger. If any exception is thrown, the entire transactions are rolled back. Similarly, pre-hooks and post-hooks are executed in the same context of the write operation they're registered to. If a pre-hook or a post-hook fails, the entire transaction is aborted and no entry is added to the ledger.

• UDFs can only access CCF application tables and can't access the ledger's internal and governance tables or other built-in tables for security reasons. The ledger tables where entries are written to (public:confidentialledger.logs for public ledgers and private:confidentialledger.logs for private ledgers) are read-only.

• The maximum number of pre-hooks and post-hooks that can be registered for a single write transaction is 5.

• UDF and hook execution is limited to 5 seconds. If a function takes longer than 5 seconds to execute, the operation is aborted and an error is returned.

Related content

• User defined functions overview in Azure confidential ledger
• Advanced user defined functions in Azure confidential ledger
• Overview of Microsoft Azure confidential ledger