Value Encoders
When a session finishes, Ferret encodes the result value into bytes using a codec selected by content type. Run returns a consumable Output handle. Collect it to read the encoded bytes and their MIME type:
These examples use the documented Core release. For an existing byte-slice codec, follow the Encoding API migration.
Built-in codecs
Two codecs are registered by default:
| Content type | Format | Package |
|---|---|---|
application/json |
JSON (default) | pkg/encoding/json |
application/vnd.msgpack |
MessagePack | pkg/encoding/msgpack |
MessagePack decodes signed integers and unsigned integers up to math.MaxInt64
as runtime.Int, preserving their exact values on both 32-bit and 64-bit
systems. Unsigned integers above that limit return an overflow error. Values
encoded as MessagePack floats decode as runtime.Float, even when they have
no fractional part.
Selecting the output format
Set the content type when creating a session:
If no content type is set, the session defaults to application/json.
The Codec interface
A codec combines encoding and decoding behind a content type:
Writing and reading values
The following snippets use encoding from pkg/encoding and json from pkg/encoding/json. Pass your context and destination directly:
The codec borrows I/O synchronously until the method returns. It must not retain,
close, or use the supplied reader or writer afterward. Scratch buffers may be
reused after each Write returns. Success completes all codec-owned writes and
flushing; callers flush, finalize, and close their own wrappers.
Failure may leave a written prefix or consume part or all of the input.
Non-nil I/O errors retain their identity. An incomplete write without an error
returns io.ErrShortWrite and stops without retrying. Decoding returns a fully
materialized value, may read ahead, and rejects malformed, truncated, or multiple
documents.
Use the byte helpers when your caller needs bytes:
EncodeBytes collects into a bytes.Buffer and returns the written prefix even
on error. DecodeBytes uses bytes.NewReader. Check encoding success before
publishing buffered bytes when publication must be atomic.
Native encoding remains eager and byte-backed. Successful zero-byte encoding
produces present empty Content; a failed zero-byte encoding leaves content
absent. A nonempty prefix remains available with LengthKnown=false.
Collected bytes survive encoder scratch reuse and result/session cleanup.
See Output.
Context and buffering
Caller contexts reach hooks and runtime capabilities. Built-in codecs first
reject nil contexts and nil or typed-nil readers/writers with errors wrapping
runtime.ErrInvalidArgument, before resource discovery, hooks, or I/O.
After validation, encoding completes stored-root resource discovery before checking cancellation. An already canceled context, or cancellation during discovery, returns before hooks, serialization, or writes. Decoding checks cancellation immediately after validation, before hooks or reads. During active work, both codecs check cancellation at bounded I/O boundaries, every 256 traversal values/tokens, and between large payload chunks. Arbitrary blocking I/O and context-free custom marshalers cannot be forcibly interrupted.
JSON writes progressively, including escaped strings and Base64 payloads in bounded chunks. Custom marshalers and generic Go-value fallbacks may buffer. MessagePack uses the supplied I/O directly, but generic iterables still materialize to determine container lengths. Both decoders materialize their result; this interface makes no constant-memory guarantee. Query execution, record iteration, and output lifecycle behavior are unchanged.
Encoding borrows the source value. A distinct acquired iterator closes once after
traversal and child encoding, including on failure or cancellation. An iterator
that is the source itself, or shares its runtime.Resource identity, stays with
the source’s owner. Non-comparable resource aliases need runtime.Resource for
stable identity. Direct codec callers own cleanup of their source and child
resources. Native materialization adopts resources into its VM result.
Both built-in JSON and MessagePack adopt successful iterator yields, host
List.At results, and map callback values and keys before cancellation checks,
key/separator writes, or user hooks can abandon them. They also discover resources
already stored in concrete built-in arrays and objects at the root and these
acquisition boundaries, and around hooks or container failures. This iterative,
cycle-safe discovery completes even when the context is canceled. It never
reads lazy iterables, invokes host collection capabilities, marshals values, or
runs user encoding hooks. Repeated containers are rescanned so later mutations
do not hide new resources.
The codecs report discoveries to the VM result without closing source or child
resources. Comparable identities and runtime.Resource aliases use the result’s
existing deduplication; dynamically non-comparable aliases need
runtime.Resource for a safe identity. Configured and pointer instances of the
built-in codecs share this behavior. Custom codecs and decorators retain the
existing ownership pre-hook; the built-in path is selected by Go type, not
content type. The original contexts still reach hooks and host capabilities.
MessagePack encodes its collected items only after traversal succeeds, then
closes a distinct iterator. A traversal failure leaves yielded resources with
the VM result for cleanup even though child encoding and its hooks never run.
Traversal errors joined with io.EOF retain their other causes. Iterator
cleanup errors join traversal or encoding errors before the parent’s
post-hooks. Yielded-resource cleanup errors belong to result cleanup and remain
separate from encoding errors.
Encoder and decoder hooks
Configurers build encoders and decoders with hooks attached:
| Hook | Signature | When it runs |
|---|---|---|
PreEncoderHook |
func(context.Context, runtime.Value) error |
Before each visited value |
PostEncoderHook |
func(context.Context, runtime.Value, error) error |
After each value, including on encoding failure |
PreDecoderHook |
func(context.Context) error |
Once before decoding, without consuming input |
PostDecoderHook |
func(context.Context, runtime.Value, error) error |
Once after decoding |
Encoder hooks run recursively in depth-first order. Decoder post-hooks receive
the decoded value on success and runtime.None on decoding failure. Hooks of the
same kind run in registration order and stop at the first hook error. A pre-hook
error skips that operation and its post-hooks; ancestor encoder post-hooks still
receive descendant errors. Operation and hook errors preserve their causes.
Decoder hooks work with the operation and decoded value. For example, reject a
successfully decoded none without hiding a decoding failure:
Built-in JSON and MessagePack configurers publish independent snapshots. Adding hooks to a configurer or deriving another encoder does not change an encoder or decoder already returned by it. Copies of a built-in codec retain their published configuration. Configure before sharing; configurer mutation and any mutable state captured by hooks require caller synchronization.
The Registry
The encoding.Registry stores codecs by MIME-normalized content type:
Registry methods
| Method | Returns | Purpose |
|---|---|---|
Register(codec) |
error |
Store a codec by its content type |
Codec(contentType) |
Codec, error |
Look up a full codec |
Encoder(contentType) |
Encoder, error |
Look up an encoder |
Decoder(contentType) |
Decoder, error |
Look up a decoder |
Clone() |
*Registry |
Copy registrations while retaining the codec values |
Content types are normalized using MIME media type parsing, so application/json and application/json; charset=utf-8 resolve to the same codec. Registering the same normalized type replaces the earlier entry.
NewRegistry ignores nil seed codecs and panics if a seed has an invalid content type. Use NewEmptyRegistry and check each Register error for runtime-supplied configuration. A clone has its own registration map; it does not deep-copy custom codec state.
Registering codecs on the engine
Add or override a single codec:
Replace the entire registry:
When you use WithEncodingRegistry, only the codecs in the provided registry are available. The default JSON and MessagePack codecs are not included unless you add them yourself.
Complete example
This plain-text codec writes directly, preserves short-write errors, and runs configured hooks. Its decoder buffers the input until EOF because plain text has no document delimiter. Applications needing interruption during large custom operations should add bounded cancellation checks between chunks.