# lua_shape

A comprehensive schema validation and transformation library for Rspamd, designed to replace tableshape with improved error reporting, documentation generation, and export capabilities.

## Features

1. **Better Error Reporting**: Structured error trees with intersection analysis for `one_of` types
2. **Documentation Generation**: Extract structured documentation from schemas
3. **Type Constraints**: Numeric ranges, string lengths, patterns, and more
4. **First-class Mixins**: Field composition with origin tracking
5. **JSON Schema Export**: Export schemas for UCL validation
6. **Transform Support**: Immutable transformations with validation
7. **Pure Lua**: No dependencies on external modules (except optional lpeg for patterns)

## Quick Start

```lua
local T = require "lua_shape.core"

-- Define a schema
local config_schema = T.table({
  host = T.string({ min_len = 1 }),
  port = T.integer({ min = 1, max = 65535 }):with_default(8080),
  timeout = T.number({ min = 0 }):optional(),
  ssl = T.boolean():with_default(false)
})

-- Validate configuration
local ok, result = config_schema:check({
  host = "localhost",
  port = 3000
})

if not ok then
  print("Validation error:")
  print(T.format_error(result))
end

-- Transform with defaults applied
local ok, config = config_schema:transform({
  host = "example.com"
})
-- config.port == 8080 (default applied)
-- config.ssl == false (default applied)
```

## Core Types

### Scalars

- `T.string(opts)` - String with optional constraints
  - `min_len`, `max_len` - Length constraints
  - `pattern` - Lua pattern for validation (e.g., `"^%d+$"` for digits only)
  - `lpeg` - Optional lpeg pattern for complex parsing
- `T.number(opts)` - Number with optional range constraints (min, max)
  - Accepts both numbers and values convertible via `tonumber`
- `T.integer(opts)` - Integer (number with integer constraint)
- `T.boolean()` - Boolean value
- `T.callable()` - Function/callable value
- `T.enum(values)` - One of a fixed set of values
- `T.literal(value)` - Exact value match

### Structured Types

- `T.array(item_schema, opts)` - Array with item validation
  - Underlying table must be a dense, 1-indexed array (no sparse or string keys)
  - `min_items`, `max_items` - Size constraints
- `T.table(fields, opts)` - Table/object with field schemas
  - `open = true` - Allow additional fields not defined in schema
  - `open = false` (default) - Reject unknown fields
  - `extra = schema` - Schema for validating extra fields
  - `mixins` - Array of mixin schemas for composition (applied when the schema is resolved via the registry)
- `T.one_of(variants)` - Sum type (match exactly one alternative)

### Composition

- `schema:optional()` - Make schema optional
- `schema:with_default(value)` - Add default value (can be a function for dynamic defaults)
- `schema:doc(doc_table)` - Add documentation
- `schema:transform_with(fn)` - Apply transformation
- `T.transform(schema, fn)` - Transform wrapper
- `T.ref(id)` - Reference to registered schema
- `T.mixin(schema, opts)` - Mixin for table composition

## Examples

### Basic Types with Constraints

```lua
-- String with length constraint
local name_schema = T.string({ min_len = 3, max_len = 50 })

-- String with Lua pattern (validates format)
local email_schema = T.string({ pattern = "^[%w%.]+@[%w%.]+$" })
local ipv4_schema = T.string({ pattern = "^%d+%.%d+%.%d+%.%d+$" })

-- Integer with range
local age_schema = T.integer({ min = 0, max = 150 })

-- Enum
local level_schema = T.enum({"debug", "info", "warning", "error"})
```

### Arrays and Tables

```lua
-- Array of strings
local tags_schema = T.array(T.string())

-- Table with required and optional fields
local user_schema = T.table({
  name = T.string(),
  email = T.string(),
  age = T.integer():optional(),
  role = T.enum({"admin", "user"}):with_default("user")
})

-- Closed table (default): rejects unknown fields
local strict_config = T.table({
  host = T.string(),
  port = T.integer()
}, { open = false })

-- Open table: allows additional fields not in schema
local flexible_config = T.table({
  host = T.string(),
  port = T.integer()
}, { open = true })
-- Accepts: { host = "localhost", port = 8080, custom_field = "value" }
```

### one_of with Intersection

```lua
-- Multiple config variants
local config_schema = T.one_of({
  {
    name = "file_config",
    schema = T.table({
      type = T.literal("file"),
      path = T.string()
    })
  },
  {
    name = "redis_config",
    schema = T.table({
      type = T.literal("redis"),
      host = T.string(),
      port = T.integer():with_default(6379)
    })
  }
})

-- Error messages show intersection:
-- "all alternatives require: type (string)"
```

### Transforms

`T.transform(accepted_type, transformer)` validates input against `accepted_type`, then applies `transformer` function.

```lua
-- Accept string, convert to number
local num_from_string = T.transform(T.string(), tonumber)

-- Accept number or string, convert both to number
local flexible_number = T.one_of({
  T.number(),
  T.transform(T.string(), tonumber)
})

-- Accept string, parse time interval to number
local timeout_schema = T.one_of({
  T.number({ min = 0 }),
  T.transform(T.string(), parse_time_interval)  -- "5s" -> 5.0
})
```

**Semantics:**
1. Input is validated against accepted type (first argument)
2. If valid, transformer function is called with pcall
3. If transformer returns `nil` or errors, validation fails
4. Otherwise, result is accepted without type checking

> **Note:** Transform functions run only in `schema:transform(...)` mode. In `schema:check(...)` mode, only the input type is validated.

### Callable Defaults

Defaults can be functions that are called each time a default is needed:

```lua
local function get_current_timestamp()
  return os.time()
end

local event_schema = T.table({
  name = T.string(),
  timestamp = T.number():with_default(get_current_timestamp),  -- Function called each time
  priority = T.integer():with_default(0)  -- Static default
})

-- Each transform gets a fresh timestamp
local ok, event1 = event_schema:transform({ name = "login" })
-- event1.timestamp will be the current time when transform was called
```

### Schema Registry

```lua
local Registry = require "lua_shape.registry"
local reg = Registry.global()

-- Define reusable schemas
local redis_schema = reg:define("redis.options", T.table({
  servers = T.array(T.string()),
  db = T.integer({ min = 0, max = 15 }):with_default(0)
}))

-- Reference in other schemas
local app_schema = T.table({
  cache = T.ref("redis.options")
})

-- Resolve references
local resolved = reg:resolve_schema(app_schema)
-- Validate/transforms should use the resolved schema so mixins/references are applied
local ok, cfg_or_err = resolved:transform({
  cache = {
    servers = {"redis:6379"}
  }
})
```

### Mixins with Origin Tracking

```lua
-- Base mixin
local redis_mixin = T.table({
  redis_host = T.string(),
  redis_port = T.integer():with_default(6379)
})

-- Use mixin in another schema
local plugin_schema = T.table({
  enabled = T.boolean(),
  plugin_option = T.string()
}, {
  mixins = {
    T.mixin(redis_mixin, { as = "redis" })
  }
})

-- Documentation will show:
-- Direct fields: enabled, plugin_option
-- Mixin "redis": redis_host, redis_port
```

Mixins are merged into the resulting table schema by `Registry:resolve_schema` (or `Registry:define`). Always validate against the resolved schema so that mixin fields participate in `:check` / `:transform` and emit proper documentation/JSON Schema output.

### JSON Schema Export

```lua
local jsonschema = require "lua_shape.jsonschema"

-- Export single schema
local json = jsonschema.from_schema(config_schema, {
  id = "https://rspamd.com/schema/config",
  title = "Application Config"
})

-- Export all schemas from registry
local all_schemas = jsonschema.export_registry(Registry.global())
```

### Documentation Generation

```lua
local docs = require "lua_shape.docs"

-- Generate documentation IR
local doc_tree = docs.for_schema(config_schema)

-- Render as markdown
local markdown_lines = docs.render_markdown(doc_tree.schema_doc)
for _, line in ipairs(markdown_lines) do
  print(line)
end
```

## Error Reporting

### Structured Errors

Errors are represented as trees:

```lua
{
  kind = "table_invalid",
  path = "config",
  details = {
    errors = {
      port = {
        kind = "constraint_violation",
        path = "config.port",
        details = { constraint = "max", expected = 65535, got = 99999 }
      }
    }
  }
}
```

### Human-Readable Formatting

```lua
local T = require "lua_shape.core"
print(T.format_error(error_tree))
```

Output:
```
table validation failed at config:
  constraint violation at config.port: max (expected: 65535, got: 99999)
```

### one_of Intersection Errors

When all variants of a one_of fail, the error shows common requirements:

```
value does not match any alternative at :
  all alternatives require:
    - name: string
    - type: string
  some alternatives also expect:
    - path: string (in file_config variant)
    - host: string (in redis_config variant)
  tried alternatives:
    - file_config: ...
    - redis_config: ...
```

## API Reference

### Core Module (`lua_shape.core`)

#### Type Constructors

- `T.string(opts?)` - String type
  - opts: `min_len`, `max_len`, `pattern`, `lpeg`, `doc`
- `T.number(opts?)` - Number type
  - opts: `min`, `max`, `doc`
- `T.integer(opts?)` - Integer type (number with integer=true)
  - opts: `min`, `max`, `doc`
- `T.boolean(opts?)` - Boolean type
- `T.enum(values, opts?)` - Enum type
- `T.literal(value, opts?)` - Literal value type
- `T.array(item_schema, opts?)` - Array type
  - opts: `min_items`, `max_items`, `doc`
- `T.table(fields, opts?)` - Table type
  - opts: `open`, `extra`, `mixins`, `doc`
- `T.one_of(variants, opts?)` - Sum type
- `T.optional(schema, opts?)` - Optional wrapper
- `T.default(schema, value)` - Default value wrapper
- `T.transform(accepted_type, transformer, opts?)` - Transform wrapper (validates input against accepted_type, then applies transformer)
- `T.ref(id, opts?)` - Schema reference placeholder (must be resolved via the registry before validation)
- `T.mixin(schema, opts?)` - Mixin definition

#### Schema Methods

- `schema:check(value, ctx?)` - Validate value
- `schema:transform(value, ctx?)` - Transform and validate (tableshape-compatible `(result)` / `(nil, err)` contract)
- `schema:optional(opts?)` - Make optional
- `schema:with_default(value)` - Add default
- `schema:doc(doc_table)` - Add documentation
- `schema:transform_with(fn, opts?)` - Add transformation

### Registry Module (`lua_shape.registry`)

- `Registry.global()` - Get/create global registry
- `registry:define(id, schema)` - Register schema with ID (returns the resolved version with mixins/reference chains applied)
- `registry:get(id)` - Get schema by ID
- `registry:resolve_schema(schema)` - Resolve references and mixins (recurses into nested arrays/one_of/options and caches the result)
- `registry:list()` - List all schema IDs
- `registry:export_all()` - Export all schemas

### Core Utilities

The core module also includes utility functions:

- `T.format_error(err)` - Format error tree as human-readable string
- `T.deep_clone(value)` - Deep clone value for immutable transformations

### JSON Schema Module (`lua_shape.jsonschema`)

- `jsonschema.from_schema(schema, opts?)` - Convert to JSON Schema
- `jsonschema.export_registry(registry, opts?)` - Export all schemas

### Docs Module (`lua_shape.docs`)

- `docs.for_schema(schema, opts?)` - Generate documentation IR
- `docs.for_registry(registry, opts?)` - Generate docs for all schemas
- `docs.render_markdown(doc_tree, indent?)` - Render as markdown

## Migration from tableshape

See [MIGRATION.md](MIGRATION.md) for detailed migration guide.

Quick reference:

| tableshape | lua_shape |
|------------|---------------|
| `ts.string` | `T.string()` |
| `ts.number` | `T.number()` |
| `ts.array_of(ts.string)` | `T.array(T.string())` |
| `ts.shape({...})` | `T.table({...})` |
| `field:is_optional()` | `field:optional()` or `{ schema = ..., optional = true }` |
| `ts.string + ts.number` | `T.one_of({ T.string(), T.number() })` |
| `ts.string / fn` | `T.string():transform_with(fn)` or `T.transform(T.string(), fn)` |
| `field:describe("...")` | `field:doc({ summary = "..." })` |

## Files

- `core.lua` - Core type system, validation, and utilities
- `registry.lua` - Schema registration and reference resolution
- `jsonschema.lua` - JSON Schema export
- `docs.lua` - Documentation generation
- `MIGRATION.md` - Migration guide from tableshape
- `README.md` - This file

## License

Apache License 2.0 - Same as Rspamd