Work with APIs

Ferret is not limited to HTML. It can fetch data from REST APIs, parse JSON responses, and combine API data with HTML extraction in a single script.

Make a GET request

Use IO::NET::HTTP::GET to fetch data from an API:

ferret run -e '
LET response = IO::NET::HTTP::GET("https://jsonplaceholder.typicode.com/posts/1")
LET data = JSON_PARSE(TO_STRING(response))
RETURN data
'

terminal

$ferret run -e '$LET response = IO::NET::HTTP::GET("https://jsonplaceholder.typicode.com/posts/1")$LET data = JSON_PARSE(TO_STRING(response))$RETURN data$'

IO::NET::HTTP::GET returns raw bytes. Use TO_STRING to convert to a string, then JSON_PARSE to decode JSON.

Make a POST request

Use IO::NET::HTTP::POST with a body and headers:

example.fql

read-only

LET response = IO::NET::HTTP::POST({ url: "https://jsonplaceholder.typicode.com/posts", body: TO_BINARY(JSON_STRINGIFY({ title: "Ferret", body: "Data extraction", userId: 1 })), headers: { "Content-Type": "application/json" } }) RETURN JSON_PARSE(TO_STRING(response))

Iterate over API results

Fetch a list and process it with FOR:

ferret run -e '
LET response = IO::NET::HTTP::GET("https://jsonplaceholder.typicode.com/posts")
LET posts = JSON_PARSE(TO_STRING(response))

FOR post IN posts
    LIMIT 5
    RETURN {
        id: post.id,
        title: post.title
    }
'

terminal

$ferret run -e '$LET response = IO::NET::HTTP::GET("https://jsonplaceholder.typicode.com/posts")$LET posts = JSON_PARSE(TO_STRING(response))$$FOR post IN posts$    LIMIT 5$    RETURN {$        id: post.id,$        title: post.title$    }$'

Paginate an API

Many APIs use offset or page-based pagination:

example.fql

read-only

LET baseURL = "https://jsonplaceholder.typicode.com/posts?_start=" LET pageSize = 10 LET result = ( FOR pageNum IN 0..2 LET offset = pageNum * pageSize LET url = baseURL + TO_STRING(offset) + "&_limit=" + TO_STRING(pageSize) LET response = IO::NET::HTTP::GET(url) LET posts = JSON_PARSE(TO_STRING(response)) FOR post IN posts RETURN { id: post.id, title: post.title } ) RETURN result

Add headers and authentication

Pass custom headers for APIs that require authentication:

example.fql

read-only

LET response = IO::NET::HTTP::GET({ url: "https://api.example.com/data", headers: { "Authorization": "Bearer " + @token, "Accept": "application/json" } }) RETURN JSON_PARSE(TO_STRING(response))

Use a bind parameter (@token) so the secret is not hardcoded in the script:

ferret run script.fql --param token=your-api-key

terminal

$ferret run script.fql --param token=your-api-key

Combine API and HTML data

A powerful pattern: fetch structured data from an API and enrich it with data from HTML pages:

example.fql

read-only

LET response = IO::NET::HTTP::GET("https://jsonplaceholder.typicode.com/posts") LET posts = JSON_PARSE(TO_STRING(response)) FOR post IN posts LIMIT 3 LET page = WEB::HTML::OPEN("https://mockery.ferretlang.org") ON ERROR RETURN NONE RETURN { id: post.id, title: post.title, pageTitle: page?.title }

Handle API errors

Use ON ERROR RETURN and ON ERROR RETRY for network failures:

example.fql

read-only

LET response = IO::NET::HTTP::GET("https://api.example.com/data") ON ERROR RETRY 3 DELAY 1s BACKOFF EXPONENTIAL OR RETURN NONE RETURN response != NONE ? JSON_PARSE(TO_STRING(response)) : { error: "API unavailable" }

Next steps

Guide Extract from static pages Load HTML, query with CSS selectors, and shape the output. Guide Error handling Handle missing elements, network failures, and timeouts. Learn Standard library Functions and types available to every Ferret script. Script inputs Parameters Use parameters to provide runtime input without hardcoding values in FQL scripts.