Runtime Configuration

The engine accepts functional options that control which capabilities are available to scripts, how results are encoded, how execution is logged, and how concurrency is managed.

Configuration errors

Engine and session options are applied in order. Ferret ignores nil options, applies every other option, and joins multiple failures into the returned error. Construction returns no engine or session when option application fails, and releases any engine-owned services created by those options.

Simple validation failures can be inspected with github.com/ziflex/go-options.ValidationError. Applications only need this import when they want structured error details; ordinary Ferret option calls continue to use ferret.Option, ferret.SessionOption, and the ferret.With* functions.

example.go

read-only

engine, err := ferret.New( ferret.WithMaxActiveSessions(-1), ferret.WithFSRoot(" "), ) if err != nil { var validationErr gooptions.ValidationError if errors.As(err, &validationErr) { log.Printf("invalid option %s: %s", validationErr.Field, validationErr.Reason) } return err } defer engine.Close()

errors.As traverses joined validation failures, while errors.Is continues to find sentinel errors returned by compound options and their underlying components.

Standard library

By default, the engine loads the full standard library. You can restrict it to a subset of groups or disable it entirely.

Available groups

Group	Contents
`Types`	Type checking and conversion functions
`Strings`	String manipulation
`Math`	Mathematical operations
`Collections`	Generic collection operations
`DateTime`	Date and time functions
`Arrays`	Array-specific operations
`Objects`	Object-specific operations
`IO`	Expands to `FS` + `NET`
`FS`	File system access
`NET`	Network operations
`Path`	File path manipulation
`Utils`	Utility functions
`Testing`	Test assertion functions

Selecting groups

example.go

read-only

// Full standard library (default) ferret.New( ferret.WithStdlib(stdlib.Full()), ) // Everything except file system and network access ferret.New( ferret.WithStdlib(stdlib.Safe()), ) // Only specific groups ferret.New( ferret.WithStdlib(stdlib.Only(stdlib.Strings, stdlib.Math, stdlib.Arrays)), ) // Full minus specific groups ferret.New( ferret.WithStdlib(stdlib.Full().Without(stdlib.IO, stdlib.Testing)), ) // No standard library at all ferret.New( ferret.WithoutStdlib(), )

stdlib.Safe() is equivalent to stdlib.Full().Without(stdlib.IO) — it removes file system and network access while keeping everything else.

Logging

Logging is configured at two levels: engine-wide defaults and per-session overrides.

Engine-level logging

example.go

read-only

engine, err := ferret.New( ferret.WithLog(os.Stdout), ferret.WithLogLevel(ferret.LogInfo), ferret.WithLogFields(map[string]any{ "service": "query-engine", }), )

Session-level logging

Per-session options override the engine defaults for that execution:

example.go

read-only

session, err := plan.NewSession(ctx, ferret.WithSessionLog(os.Stderr), ferret.WithSessionLogLevel(ferret.LogDebug), ferret.WithSessionLogFields(map[string]any{ "request_id": "abc-123", }), )

Log levels

Level	Constant
Trace	`ferret.LogTrace`
Debug	`ferret.LogDebug`
Info	`ferret.LogInfo`
Warn	`ferret.LogWarn`
Error	`ferret.LogError`
Fatal	`ferret.LogFatal`
Panic	`ferret.LogPanic`
None	`ferret.LogNone`
Disabled	`ferret.LogDisabled`

The constants and parsing helpers use the root ferret.LogLevel type. Parse a level supplied through configuration before constructing the engine:

example.go

read-only

level, err := ferret.ParseLogLevel(os.Getenv("FERRET_LOG_LEVEL")) if err != nil { return err } engine, err := ferret.New( ferret.WithLogLevel(level), )

ferret.MustParseLogLevel is available for trusted static configuration and panics when the input is invalid.

Reproducible randomness

In a runtime containing this API, provide a seed when reproducible execution is useful:

example.go

read-only

session, err := plan.NewSession(ctx, ferret.WithSessionRandomSeed(42)) if err != nil { return err } defer session.Close() output, err := session.Run(ctx)

The same option works with Engine.Run and Plan.NewDebugSession. Zero and negative seeds are valid; the last seed option wins. Reusing an option creates independent sources. Without a seed, the Session obtains fresh operating-system entropy once, on its first actual random draw. Construction and queries that never draw do not read entropy. Explicit seeds require no entropy read. There is no engine-wide RNG state or query-level seed mutation.

Within a Ferret version, the same seed, parameters, relevant inputs, and executed control flow produce the same random sequence. Sequential runs of an existing Session and debugger resumes continue the sequence. Create another Session with the same seed to replay it from the beginning. Exact sequences may change between Ferret versions. External I/O and timing can change control flow, including the number of jitter draws; a seed does not make those inputs deterministic.

Pseudo-random values are unsuitable for passwords, authentication tokens, secrets, keys, or other security-sensitive uses. Use cryptographic functionality for those purposes.

Output encoding

Query results are encoded before being returned as an Output. The default encoding is JSON; MessagePack is also built in. Set the content type at the session level with WithOutputContentType:

example.go

read-only

session, err := plan.NewSession(ctx, ferret.WithOutputContentType("application/vnd.msgpack"), )

You can register custom codecs on the engine with WithEncodingCodec. See Value Encoders for the codec interfaces, hooks, registry, and a complete custom codec example.

ferret.WithEnvironmentOptions is an advanced escape hatch for applications that integrate directly with VM environment options. Prefer the root session options for parameters, logging, output encoding, and other ordinary embedding configuration.

Concurrency control

Three options control how the engine manages concurrent execution:

`WithMaxActiveSessions`

Limits the total number of sessions that can be running at the same time across the entire engine. When the limit is reached, plan.NewSession blocks until another session closes or the context is cancelled.

example.go

read-only

engine, err := ferret.New( ferret.WithMaxActiveSessions(100), )

Use this to cap global resource consumption — CPU, memory, network, or downstream service pressure.

`WithMaxVMsPerPlan`

A hard cap on the total number of virtual machines a single plan can own, including both idle and active VMs. When the limit is reached and no idle VM is available, session creation fails with an error.

example.go

read-only

engine, err := ferret.New( ferret.WithMaxVMsPerPlan(16), )

Use this to bound the resource footprint of a single frequently-executed query.

`WithMaxIdleVMsPerPlan`

Controls how many idle VMs each plan keeps cached for reuse after sessions close. When the idle cache is full, additional returned VMs are discarded instead of retained. The default is 8.

example.go

read-only

engine, err := ferret.New( ferret.WithMaxIdleVMsPerPlan(4), )

This is a memory-vs-latency trade-off: more idle VMs means faster session creation but higher steady-state memory usage.

How they work together

WithMaxActiveSessions is an engine-wide concurrency gate — it controls how many sessions run at once across all plans
WithMaxVMsPerPlan is a per-plan resource cap — it bounds the VM count for one specific compiled query
WithMaxIdleVMsPerPlan is a per-plan cache policy — it decides how many idle VMs stay warm

A value of 0 disables the limit for WithMaxActiveSessions and WithMaxVMsPerPlan. For WithMaxIdleVMsPerPlan, 0 disables idle VM retention.

Sandboxed components

Sandboxed components control how scripts access host resources outside the Ferret runtime. Configure each component independently according to what the embedding application should allow. The engine adds the configured components to the context.Context passed to functions during execution.

File system

Scripts that use file system functions (from the FS stdlib group) operate within a sandboxed file system. The host can configure the root directory and restrict access to read-only.

example.go

read-only

engine, err := ferret.New( ferret.WithFSRoot("/data/extractions"), ferret.WithFSReadOnly(), )

If WithFSRoot is not set, file system access is disabled and FS functions return a root-not-configured error. WithFSReadOnly prevents scripts from writing within the configured root.

Accessing the file system from a function

Retrieve the configured file system from the function context. Prefer the narrowest helper for the operation the function performs:

example.go

read-only

package files import ( "context" ferretfs "github.com/MontFerret/ferret/v2/pkg/fs" "github.com/MontFerret/ferret/v2/pkg/runtime" ) func ReadFile(ctx context.Context, pathArg runtime.Value) (runtime.Value, error) { path, err := runtime.CastArg[runtime.String](pathArg, 0) if err != nil { return runtime.None, err } reader, err := ferretfs.ReaderFrom(ctx) if err != nil { return runtime.None, err } data, err := reader.ReadFile(path.String()) if err != nil { return runtime.None, err } return runtime.NewBinary(data), nil }

The fs package also provides WriterFrom, DirectoriesFrom, and RemoverFrom. Use FileSystemFrom only when a function needs several capabilities.

For public modules, these context helpers are the recommended way to access files. They keep the module inside the root and read-only policy selected by the embedding host. Calling os.ReadFile, os.WriteFile, or constructing a separate file system would bypass that configuration.

HTTP client

Scripts that use network functions (from the NET stdlib group) make outbound requests through the engine’s network service. Supply a policy-configured HTTP client to restrict which destinations scripts can reach and how much data they can send or receive.

example.go

read-only

httpClient, err := ferrethttp.New( ferrethttp.WithAllowedSchemes("https"), ferrethttp.WithAllowedHosts("api.example.com"), ferrethttp.WithTimeout(10*time.Second), ferrethttp.WithMaxRequestSize(1<<20), // 1 MiB ferrethttp.WithMaxResponseSize(4<<20), // 4 MiB ) if err != nil { return err } network, err := ferretnet.New( ferretnet.WithHTTPClient(httpClient), ) if err != nil { return err } engine, err := ferret.New( ferret.WithNetwork(network), )

A network passed to WithNetwork remains caller-owned: Ferret never closes it, including when engine construction fails. Ferret owns its default network and networks created through WithNetworkOptions; it releases their idle connections on construction failure or engine shutdown.

Network options apply in order. Replacing an Engine-created network releases its idle connections immediately; replacing a borrowed network leaves it untouched. The last network-setting option selects the service used by the engine. WithNetworkOptions() without arguments leaves the current selection unchanged.

The HTTP client supports these policy controls:

Control	Options
URL schemes and destinations	`WithAllowedSchemes`, `WithAllowedHosts`, `WithBlockedHosts`
Local, private, and link-local addresses	`WithAllowLocalhost`, `WithAllowPrivateNetworks`, `WithAllowLinkLocal`
Redirects	`WithFollowRedirects`, `WithMaxRedirects`
Request headers	`WithDefaultHeader`, `WithDefaultHeaders`, `WithBlockedRequestHeaders`
Time and payload limits	`WithTimeout`, `WithNoTimeout`, `WithMaxRequestSize`, `WithUnlimitedRequestSize`, `WithMaxResponseSize`, `WithUnlimitedResponseSize`, `WithMaxResponseHeaderSize`

By default, the client allows HTTP and HTTPS and follows up to 10 redirects, but denies localhost, private networks, carrier-grade NAT, and link-local destinations. Unspecified, multicast, reserved, and invalid destinations are always denied. This includes cloud metadata endpoints on link-local addresses. Requests have a 30-second timeout, request and response bodies are limited to 16 MiB, and response headers are limited to 1 MiB.

Policy construction is fallible. A zero value passed to a numeric option restores that option’s secure default, while a negative value returns ferrethttp.ErrInvalidPolicyConfiguration. Disabling the timeout or a body-size limit requires WithNoTimeout, WithUnlimitedRequestSize, or WithUnlimitedResponseSize. Response headers always retain a finite limit. The zero value of ferrethttp.Policy is deny-all; call ferrethttp.NewPolicy to obtain the standard defaults when reusing a policy with another client.

Configuration errors support both errors.Is and errors.As. Use github.com/ziflex/go-options.ValidationError to inspect the rejected field, safe value, and reason:

example.go

read-only

policy, err := ferrethttp.NewPolicy( ferrethttp.WithAllowedHosts("api.example.com"), ) if err != nil { if errors.Is(err, ferrethttp.ErrInvalidPolicyConfiguration) { var validationErr gooptions.ValidationError if errors.As(err, &validationErr) { log.Printf("invalid HTTP option %s: %s", validationErr.Field, validationErr.Reason) } } return err }

Policy.Eval and Policy.Prepare accept a standard *net/http.Request. This is an intentional source break from the earlier Policy.Eval(*ferrethttp.Request) API: construct a standard outbound request, use Prepare when policy default headers should be added, or use Eval when the request is already complete. Eval does not mutate the request. Prepare adds missing defaults to Request.Header and then evaluates it; defaults added before a later validation failure remain on the request.

Neither method takes a separate context. Put cancellation and deadlines on the standard request with net/http.NewRequestWithContext or its WithContext method before sending it. Both methods require client-side outbound request state: server-side RequestURI and explicit transport-control state are rejected, as are Host overrides that target a different authority. With a finite request-body limit, every non-empty body must have a positive, known ContentLength; unknown-length streaming bodies return RequestBodyLengthError before transport. Buffer the body so net/http can determine its length, set a trustworthy length, or explicitly choose WithUnlimitedRequestSize.

example.go

read-only

request, err := http.NewRequestWithContext(ctx, http.MethodGet, targetURL, nil) if err != nil { return err } if err := policy.Prepare(request); err != nil { return err } response, err := customHTTPClient.Do(request)

Runtime denials can be inspected without parsing their messages:

example.go

read-only

var policyErr *ferrethttp.PolicyError if errors.As(err, &policyErr) { log.Printf( "HTTP policy denied %s %q: %s", policyErr.Target, policyErr.Subject, policyErr.Reason, ) }

Request header names and values are validated before transport. Connection, Content-Length, Host, Keep-Alive, Proxy-Authenticate, Proxy-Authorization, Proxy-Connection, TE, Trailer, Transfer-Encoding, and Upgrade are transport-controlled and cannot be supplied through Request.Header or configured as defaults.

The secure destination defaults are intentionally backward-incompatible. Applications that previously used the built-in client to reach development servers, containers, cluster-local services, or private APIs must opt in to the required address classes or inject a custom HTTP client.

The built-in client resolves hostnames and checks every returned address before the initial request and each followed redirect. It also checks the concrete numeric address immediately before connecting, so a DNS change between validation and connection cannot redirect the dial to a forbidden address. Redirects are checked before their requests are sent. An allowed-host list restricts names but never overrides these address-class checks.

For production, use the narrowest practical host allowlist, as in the example above. If an application deliberately needs internal destinations, enable each class separately:

example.go

read-only

internalHTTPClient, err := ferrethttp.New( ferrethttp.WithAllowedHosts("localhost", "api.internal.example"), ferrethttp.WithAllowLocalhost(true), ferrethttp.WithAllowPrivateNetworks(true), ferrethttp.WithAllowLinkLocal(true), ) if err != nil { return err }

WithAllowLocalhost enables only localhost names and loopback addresses. WithAllowPrivateNetworks enables RFC 1918, IPv6 unique-local, and carrier-grade NAT addresses; it does not enable link-local addresses. Enable WithAllowLinkLocal only when link-local access, including access to potential cloud metadata services, is explicitly required.

The built-in client does not inherit HTTP_PROXY, HTTPS_PROXY, or NO_PROXY. A proxy may resolve or connect to a different destination than the client validated. Applications that require a proxy or an exceptional destination class can inject their own ferrethttp.Client with ferretnet.WithHTTPClient; that client is responsible for equivalent destination and redirect controls.

Policy.Eval and Policy.Prepare are initial-request preflight checks, not replacements for the built-in secure transport. When a policy is reused with net/http or another custom transport, that integration is responsible for checking every redirect, validating all DNS results and the concrete dial address, applying the configured timeout, enforcing response-header and response-body limits, and cleaning up connections. A successful preflight does not make later redirect or resolution targets safe.

HTTP policy is defense in depth, not a complete sandbox for arbitrary untrusted FQL. Combine it with production allowlists, execution limits, restricted module sets, and infrastructure-level egress controls.

Accessing HTTP from a function

Retrieve the configured HTTP client from the function context and pass the same context to the request:

example.go

read-only

package request import ( "context" ferretnet "github.com/MontFerret/ferret/v2/pkg/net" ferrethttp "github.com/MontFerret/ferret/v2/pkg/net/http" "github.com/MontFerret/ferret/v2/pkg/runtime" ) func FetchStatus(ctx context.Context, urlArg runtime.Value) (runtime.Value, error) { url, err := runtime.CastArg[runtime.String](urlArg, 0) if err != nil { return runtime.None, err } client, err := ferretnet.HTTPClientFrom(ctx) if err != nil { return runtime.None, err } response, err := client.Do(ctx, &ferrethttp.Request{ Method: "GET", URL: url.String(), }) if err != nil { return runtime.None, err } if response == nil { return runtime.None, runtime.Error(runtime.ErrUnexpected, "HTTP response is nil") } return runtime.NewInt(response.StatusCode), nil }

For public modules, use HTTPClientFrom instead of net/http or a separately constructed client. Requests then follow the host’s destination, redirect, header, timeout, and payload-size policies, while the execution context continues to carry cancellation and deadlines. Use NetworkFrom only when a function needs the complete network service.

Both examples have valid Ferret function signatures and can be registered with sdk.Func. See Develop a Ferret module for the complete registration pattern.

Compiler options

Set the optimization level for plans compiled by an engine with WithOptimizationLevel:

example.go

read-only

engine, err := ferret.New( ferret.WithOptimizationLevel(ferret.OptimizationFull), )

Level	Constant	Description
None	`ferret.OptimizationNone`	No optimizer passes; UDF calls retain caller frames
Basic	`ferret.OptimizationBasic`	Tail-call elimination for proven-safe calls, constant propagation, liveness analysis, and peephole optimization
Full	`ferret.OptimizationFull`	Basic pipeline plus register coalescing (default)

Use the native ferret.WithPlanOptimizationLevel option to override that default for one compilation:

example.go

read-only

plan, err := engine.Compile(ctx, ferret.NewSource("query.fql", `return @value + 1`), ferret.WithPlanOptimizationLevel(ferret.OptimizationNone), )

Omitting the option inherits the engine configuration. Explicit ferret.OptimizationNone, ferret.OptimizationBasic, and ferret.OptimizationFull affect only that plan, including concurrent compilations. Other levels are unsupported.

Debug compilation (engine.CompileDebug) accepts omission or explicit ferret.OptimizationNone and rejects other levels to preserve source-level debugging metadata. Compilation is synchronous and observes cancellation between phases; it does not preempt a parser already running.

Debug compilation retains UDF caller frames, including calls in return position. Tail-call elimination is optional and does not guarantee constant-space recursion: unoptimized and debug execution use stack space proportional to recursive call depth. See User-defined functions for the supported tail-call behavior.

Native ferret.PlanOption and ferret.SessionOption configure Ferret’s runtime directly and are distinct from Universal API functional options. Use the native ferret.With* factories for native engines and sessions.

Next steps

First steps Getting Started Install the library, run your first query from Go, and handle the result. Runtime modules Modules Install, configure, register, and manage modules in an embedded runtime. Encoding Value Encoders Understand how output values are encoded and implement custom codecs. Artifacts Programs Compile queries to binary artifacts, load pre-compiled programs, and work with the artifact format.