Skip to content

Persistent Storage

SpyWeb supports state management through both Runtime Memory (transient) and an Embedded Database (persistent). Memory is fast but resets on reload; the database survives restarts.

Job-Scoped Storage (Persistent)

Section titled “Job-Scoped Storage (Persistent)”

store_set

Section titled “store_set”
store_set(key, value)
async

Saves a string (prefixed with job name).

store_get

Section titled “store_get”
store_get(key)
async

Retrieves a string or nil.

store_delete

Section titled “store_delete”
store_delete(key)
async

Removes a key.

Example

Section titled “Example”
-- Circuit breaker: stop after 5 consecutive failures, reset after 300s cooldown
local max_failures = 5
local cooldown = 300
local last_reset = tonumber(store_get("breaker_last_reset") or "0")


if tonumber(store_get("breaker_failures") or "0") >= max_failures then
    if os.time() - last_reset < cooldown then
        log("circuit breaker open - skipping request")
        return
    end
    store_set("breaker_failures", "0")
    store_set("breaker_last_reset", tostring(os.time()))
end


local ok, err = http_get("https://api.example.com/status")
if not ok then
    local count = tonumber(store_get("breaker_failures") or "0") + 1
    store_set("breaker_failures", tostring(count))
    log("failure " .. count .. "/" .. max_failures .. ": " .. tostring(err.error))
end

Global Shared Storage (Persistent)

Section titled “Global Shared Storage (Persistent)”

global_store_set

Section titled “global_store_set”
global_store_set(key, value)
async

Saves a string shared across all jobs.

global_store_get

Section titled “global_store_get”
global_store_get(key)
async

Retrieves a shared string or nil.

global_store_delete

Section titled “global_store_delete”
global_store_delete(key)
async

Removes a shared key.

global_store_incr

Section titled “global_store_incr”
global_store_incr(key, default, delta)
async

Atomic shared increment across all jobs. Returns the new count. Errors are logged internally and the default value is returned as a fallback.

Example

Section titled “Example”
-- WRONG (race condition prone across concurrent jobs)
local v = tonumber(global_store_get("global_fail_count") or "0")
global_store_set("global_fail_count", tostring(v + 1))


-- CORRECT (race condition safe atomic increment)
local new_count = global_store_incr("global_fail_count", 0, 1)

Runtime Memory (Transient)

Section titled “Runtime Memory (Transient)”

Standard Lua variables persist in memory as long as the job is active in its isolated VM. Resets on hot-reload or restart.

visit_count = (visit_count or 0) + 1
log("Session visit: " .. visit_count)