Architecture

TinyLang is a complete language implementation featuring both a tree-walk interpreter and a bytecode compiler with virtual machine. This guide explains how each component works together.

Compilation Pipeline

TinyLang supports two execution paths:

1. Interpreter path: Source → Lexer → Parser → AST → Tree-walk Interpreter → Output
2. Compiler path: Source → Lexer → Parser → AST → Compiler → Optimizer → Bytecode → VM → Output

Lexer (Tokenizer)

The lexer (src/lexer/) converts raw source text into a stream of tokens. It handles:

• Keywords: let, const, fn, class, if, else, while, for, in, return, match, when, import, from, new, this, extends, test, break, continue, and, or, not, true, false, null
• Literals: Numbers (integers, floats), strings (single/double quoted)
• Operators: +, -, *, /, %, **, ==, !=, <, >, <=, >=, =, +=, -=, *=, /=, =>, ..
• Delimiters: (, ), {, }, [, ], ,, ., :, ;
• Comments: Single-line (//)

Each token carries its source position (line and column), enabling precise error messages:

// Token structure
{
  type: TokenType,      // e.g. "NUMBER", "IDENTIFIER", "KEYWORD"
  value: string,       // raw text: "42", "let", "hello"
  line: number,         // 1-based line number
  column: number       // 1-based column number
}

Parser (Pratt Parsing)

The parser (src/parser/) uses a Pratt parser (top-down operator precedence) to convert tokens into an Abstract Syntax Tree (AST). This approach elegantly handles:

• Operator precedence: 2 + 3 * 4 correctly parses as 2 + (3 * 4)
• Associativity: a = b = c associates right-to-left
• Prefix operators: -x, not y
• Postfix/infix: Function calls, index access, property access
• Mixed expressions: obj.method(args)[0]

Operator Precedence (low to high)

AST Node Types

The AST (src/types/ast.ts) represents every syntactic construct:

// Example AST for: let x = 2 + 3
{
  type: "VariableDeclaration",
  name: "x",
  mutable: true,
  initializer: {
    type: "BinaryExpression",
    operator: "+",
    left: { type: "NumberLiteral", value: 2 },
    right: { type: "NumberLiteral", value: 3 }
  }
}

Tree-Walk Interpreter

The interpreter (src/interpreter/) recursively visits each AST node and evaluates it directly. It uses an Environment class with parent chain for lexical scoping:

// Environment chain for closure support
class Environment {
  values: Map<string, RuntimeValue>
  parent: Environment | null

  get(name)    // Look up variable, walk parent chain
  set(name, v) // Set variable in current scope
  define(name) // Create new binding in current scope
}

Runtime Values

Values in the interpreter are represented as discriminated unions (src/types/values.ts):

Bytecode Compiler

The compiler (src/compiler/compiler.ts) translates AST nodes into a linear sequence of bytecode instructions stored in a Chunk.

Chunk Structure

// A compiled bytecode chunk
{
  code: Uint8Array,         // Bytecode instructions
  constants: RuntimeValue[], // Constant pool
  lines: number[]           // Source line mapping
}

Opcode Reference

TinyLang uses 63 bytecode instructions organized into categories:

Stack Operations

Arithmetic Operations

Comparison Operations

Logical Operations

Control Flow

Variable Access

Functions

Data Structures

Object-Oriented Programming

Error Handling

I/O and Control

Optimizer

The optimizer runs peephole optimizations on the bytecode before execution:

• Constant folding: CONST 2; CONST 3; ADD → CONST 5
• Dead code elimination: Removes unreachable instructions after unconditional jumps
• Redundant pop removal: Eliminates push-then-pop sequences
• Jump threading: Collapses chains of jumps to their final target

Virtual Machine (VM)

The VM (src/vm/vm.ts) is a stack-based virtual machine that executes compiled bytecode:

VM Components

• Value Stack: Holds operands and intermediate results
• Call Frame Stack: Tracks function calls (return address, local variables offset)
• Instruction Pointer (IP): Points to the next instruction to execute
• Globals Table: Stores global variables (including stdlib functions)

Execution Example

// Source: print(2 + 3 * 4)
// Compiles to:
CONST 2         // Push 2        Stack: [2]
CONST 3         // Push 3        Stack: [2, 3]
CONST 4         // Push 4        Stack: [2, 3, 4]
MUL             // 3 * 4 = 12    Stack: [2, 12]
ADD             // 2 + 12 = 14   Stack: [14]
PRINT           // Output: 14    Stack: []
HALT            // Stop

Closure Implementation

Closures are implemented using upvalues. When a function captures a variable from an outer scope, the compiler generates LOAD_UPVALUE/STORE_UPVALUE instructions. The VM maintains upvalue references that can outlive their original stack frame.

Debugger

The debugger (src/debugger/) provides step-through execution with:

• Breakpoints: Set at specific source lines, with optional conditions
• Step/Step-into/Step-out: Fine-grained execution control
• Variable inspection: View local variables at any point
• Expression evaluation: Evaluate expressions in the current context
• Source display: Shows current position in source code

Supporting Tools

Formatter

The formatter (src/formatter/) parses source code and re-emits it with consistent style. It handles indentation, spacing, line breaks, and trailing newlines based on configurable rules.

Linter

The linter (src/linter/) performs static analysis by walking the AST and checking for common issues:

• prefer-const - Variables that could be const
• no-unused-variables - Declared but never referenced variables
• no-empty-blocks - Empty function/loop bodies
• unreachable-code - Code after return/break
• no-shadow - Variable shadowing in nested scopes

Test Runner

The test runner (src/testing/) executes test blocks and reports results. It provides assertion functions like expectToBe and tracks pass/fail status.

Module Loader

The module loader (src/modules/) resolves import paths, reads module files, parses them, and makes exported bindings available to the importing module.

Project Structure

src/
  lexer/        - Tokenization
  parser/       - Pratt parser, AST construction
  interpreter/  - Tree-walk evaluation
  compiler/     - Bytecode compilation & optimization
  vm/           - Stack-based virtual machine
  debugger/     - Interactive debugger
  formatter/    - Code formatter
  linter/       - Static analysis
  testing/      - Test runner
  modules/      - Module system & loader
  stdlib/       - Standard library (60+ functions)
  repl/         - Interactive REPL
  cli/          - Command-line interface
  types/        - AST nodes, tokens, runtime values
tests/          - Mirrors src/ structure
examples/       - 18 example programs
playground/     - Web IDE (CodeMirror + bundled TinyLang)
docs/           - Documentation site