Wyrm Language Grammar #

This grammar file is generated directly from the packrate/PEG style parser and lexxer included in the wyrm source tree. As wyrm is still at prototype stage, the grammar is expected to drift. However, at this point, the majority of primitives with basic execution should be considered stable.

Lexical Conventions #

Basics #

Wyrm files shall be encoded in UTF-8.

Keywords #

Keywords are considered own tokens: and, break, catch, class, co, continue, defer, do, elif, else, emit, false, fn, for, from, if, import, in, is, nil, not, or, pass, return, signal, slot static, task, thread, true, try, var, while, with, yield

Newlines #

Newlines are substantial. Supported line termination is CR, CRLF, LF. Whitespace and comments may appear between any two tokens and are ignored with the exception of leading whitespace for the offside rule.

Comments #

Example: # This is a comment

comment ::= "#" , { any_char - newline } , newline ;

Line Continuations #

A line ending with a \ outside of a is a continuation. A continuation allows the string to span multiple lines, and is treated as an escape that is deleted from the string.

Offside rule tokens #

The offside rule is applied whenver no open grouping or string operator is active at the beginning of a line. Whitespace at the beginning of a line must match.

An ‘indent’ is determined by a line including extra whitespace prior to active code. An indent level is defined by the the whitespace added.

A ‘dedent’ is determined by return to a previous indentation level. A DEDENT token is emitted for each INDENT until the whitespace matches.

NEWLINE         ::= (* virtual statement-separator token, emitted by lexer *) ;
INDENT          ::= (* virtual token: rise in layout column *) ;
DEDENT          ::= (* virtual token: fall in layout column *) ;

Numbers #

Example:

(* Numbers are unsigned at the lexical level; a leading “-” or “+” is unary_expr’s job, not the lexer’s. *)

 int_literal ::= digit , { digit | "_" }
                 | "0x" , hex_digit , { hex_digit | "_" }
                 | "0b" , bin_digit , { bin_digit | "_" } ;
 float_literal ::= digit , { digit | "_" } , "." , [ digit , { digit | "_" } ] ,
                   [ ( "e" | "E" ) , [ "+" | "-" ] , digit , { digit } ]
                   | digit , { digit | "_" } ,
                   ( "e" | "E" ) , [ "+" | "-" ] , digit , { digit } ;

Digits #

digits ::= digit , { digit | "_" } ;
digit ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;

Hexadecimal Digits #


hexdigits     ::= hexdigit , { hexdigit | "_" } ;
hex_digit     ::= digit | "a" | "b" | "c" | "d" | "e" | "f"
                        | "A" | "B" | "C" | "D" | "E" | "F" ;

Binary Digits #

    bindigits     ::= bindigit , { bindigit | "_" } ;

Exponent_ #

exponent      ::= ( "e" | "E" ) , [ "+" | "-" ] , digits ;

Identifiers and Symbols #

Identifiers follow normal Unicode ‘xid’ and ‘xid-continuation’ character rules with the additional allowance of ‘$’ - ‘$’ may appear anywhere in the identifier, including as the entire identifier by itself; it is not required to be followed by a letter.

A symbol literal is either an xid-shaped name (note: NOT the wyrm-widened identifier rule above - ‘$’ is not part of a symbol name) or one of a fixed whitelist of operator spellings (see _OPERATOR_SYMBOLS below). Nothing else is a legal symbol; e.g. '@ is a lex error, not a symbol.

  identifier      ::= ( xid_start | "$" ) , { xid_continue | "$" } ;
  symbol_literal  ::= "'" , ( identifier | operator_symbol ) ;
                 (* At least 31 significant characters recognized.
                    A symbol's name may be a reserved word ('fn, 'return,
                    'if): they are node kinds in the canonical s-expression
                    format, so the literal is lexed whole rather than as
                    "'" followed by a name. *)

  bool_literal    ::= "true" | "false" ;

Example: true, false

  char_literal    ::= "\" , ( ( any_char - ( control_char | " " ) ) | identifier ) ;

Example: \a, \newline

Program Structure #

program ::= { [stmt_sep] , top_level_stmt } ;
top_level_stmt ::= import_stmt | stmt_line ;

Blocks and statements #

block           ::= ":" , NEWLINE , INDENT , stmt_list , DEDENT
                   | ":" , stmt_line, [stmt_sep]
                   | "{" , stmt_list , "}" ;

Statements #

Statements include all expressions with the addition of statements that introduce name definitions or scope operations.

  stmt_line       ::= { decorator } ,
                       ( var_stmt
                       | assignment_stmt
                       | expression
                       | static_stmt
                       | import_stmt
                       | defer_stmt
                       | fn_def
                       | co_def
                       | class_def
                       | slot_def ) ;
  stmt_list ::= stmt_line , { [stmt_sep] , stmt_line } , [stmt_sep] ;
  stmt_sep ::= NEWLINE | ";" | EOF ;

Decorators #

Decorators provide macros to a statement line. Example:

  decorator       ::= "@" , identifier , [ "(" , [ arg_list ] , ")" ] ;

  defer_stmt      ::= "defer" , [ "on" , type_constraint ] , block ;

Example:

v := resource()
defer:
    v ! release()

v := resource()
defer on error:        # runs only if the calling block force-returned an error
    v ! release()

  if_stmt         ::= "if" , expression , block ,
                       { "elif" , expression , block } ,
                       [ "else" , block ] ;

Example:

if condition:
    statements
elif condition_2:
    statements
else:
    statements

  while_stmt      ::= "while" , expression , block ;

Example:

while condition:
    statement
    if condition_2:
        break

  for_stmt        ::= "for" , identifier , "in" , expression , block ,
                       [ "else" , block ] ;

Example:

for name in iterable:
    statement
    if condition:
        break
else:
    statement_if_no_break

  return_stmt     ::= "return" , [ expression ] ;

  yield_stmt      ::= "yield" , "from" , expression
                     | "yield" , [ expression ] ;

  break_stmt      ::= "break" ;

  continue_stmt   ::= "continue" ;

  pass_stmt       ::= "pass" ;

  var_stmt        ::= "var" , var_target , { "," , var_target } , [ "=" , tuple_expression ] ;
  var_target      ::= identifier , [ ":" , type_expression ] ;

Example:

var foo: int = 5           # canonical declaration with type constraint
var foo = 5                # declaration with inferred type
var x: int, y: int = 6, 5  # multiple names, bound from a tuple init
foo := 5                   # shorthand for the line above

S-Expression

(define var_target type_expression expression) ; for 1 binding
(define_values ((var_0 type_0)(var_1 type_1)) expression) ; for n binding

  assignment_stmt ::= target , { "," , target } , "=" , tuple_expression
                     | target , "?=" , tuple_expression ;

Example: a, b = b, a (swap; both names must already be declared)

Example: k ?= 4

(set var value) ; set var to value
(set_values (var1 var2) value) ; set multiple values

  target          ::= ( identifier | postfix_expr , "." , identifier ) , { "[" , expression , "]" } ;

  assign_op       ::= "=" | "?=" | ":=" ;

  static_stmt     ::= "static" , identifier , [ ":" , type_constraint ] , [ "=" , expression ] ;

Example:

fn call_count():
    static foo: int = 0
    foo = foo + 1
    return foo

Type Expression #

  type_atom       ::= qualified_name | "nil" ;
  type_constraint ::= type_atom , { "|" , type_atom } ;

Modules and Imports #

    import_stmt     ::= "import" , [ "static" ] , import_body ;

Example:

import mod::baz::bar
import mod::baz as bt
import std::io::(File, StreamReader, StreamWriter)
import std::io::* except (File, StreamReader)
import static std::io

    import_body     ::= qualified_name , "::" , "*" , "except" , except_names
                     | qualified_name , "::" , "*"
                     | qualified_name , "::" , "(" , import_item , { "," , import_item } , ")"
                     | qualified_name , [ "as" , identifier ] ;

  import_item     ::= identifier , [ "as" , identifier ] ;

  except_names    ::= identifier
                     | "(" , identifier , { "," , identifier } , ")" ;

  fn_def          ::= "fn" , [ class_target ] , identifier , "(" , [ param_list ] , ")" ,
                       [ "->" , type_constraint ] , block ;

Example:

fn message(greeting: str, name: str) -> str:
    return greeting + name

fn [int, str] message(what: str) -> str:   # multi-dispatch on (int, str)
    ...

  class_target    ::= "[" , identifier , { "," , identifier } , "]" ;   (* multi-dispatch receivers *)

  param_list      ::= param , { "," , param } ;

  param           ::= var_positional | var_keyword | plain_param ;

  plain_param     ::= identifier , [ ":" , type_constraint ] , [ "=" , expression ] ;

  var_positional  ::= "*" , identifier ;

  var_keyword     ::= "**" , identifier ;

  co_def          ::= "co" , [ class_target ] , identifier , "(" , [ co_param_list ] , ")" ,
                       [ "->" , type_constraint ] , block ;

  co_param_list   ::= "<-" , type_constraint , [ "," , param_list ]   (* send-type, not a param *)
                     | param_list ;

Classes #

  class_def       ::= "class" , identifier , [ "(" , [ arg_list ] , ")" ] , block ;

Example:

class person:
    slot name: str

class family_member(person):
    slot relation: str

  slot_def        ::= "slot" , identifier , [ ":" , type_constraint ] ,
                       [ "=" , expression ] , [ "with" , slot_options ] ;

Example:

slot name: str = "John Doe"

slot name: str = "John Doe" with:
    setter = fn (value) { this.name = value; }
    getter = undefined;

  slot_options    ::= ":" , NEWLINE , INDENT , { slot_option , stmt_sep } , DEDENT
                     | "{" , { slot_option , [ ";" ] } , "}" ;

  slot_option     ::= ( "setter" | "getter" ) , "=" , ( expression | "undefined" ) ;

Expressions #

  tuple_expression ::= list_expression ;        (* comma = tuple *)

  try_expr ::= ( "try", catch_expr ) | catch_expr;

Example: file := try open('badfile.txt') (an error return propagates immediately)

  catch_expr ::= logical_expr , [ "catch" , expression ] ;

Example: file := open('badfile.txt') catch open('goodfile.txt')

  or_expr ::= logical_expr , { "or", logical_expr }

  and_expr ::= or_expr , { "and" , or_expr }

  not_expr ::= "not" , not_expr | comparison

  comparison      ::= binary_expr , "is" , [ "not" ] , type_constraint
                     | binary_expr , [ "not" ] , "in" , binary_expr
                     | binary_expr , [ comp_op , binary_expr ] ;
  comp_op         ::= "<=" | ">=" | "<=>" | "<" | ">" | "==" | "!=" ;

  binary_expr     ::= unary_expr , { binary_op , unary_expr } ;
  binary_op       ::= "|" | "^" | "&" | "<<" | ">>" | "+" | "-" | "*" | "/" | "%" ;

  unary_expr      ::=
                     | ( "-" | "+" | "~" | "not" ) , unary_expr
                     | power_expr ;

  power_expr      ::= postfix_expr , [ "**" , unary_expr ] ;       (* right-assoc *)

  postfix_expr    ::= primary , { postfix_op } ;
  postfix_op      ::= call_op | index_op | attr_op | message_op ;

  call_op         ::= "(" , [ arg_list ] , ")" ;
  index_op        ::= "[" , expression , "]" ;
  attr_op         ::= "." , identifier ;
  message_op      ::= "!" , identifier , [ "(" , [ arg_list ] , ")" ] ;

  arg_list        ::= argument , { "," , argument } ;
  argument        ::= expression
                     | identifier , "=" , expression          (* keyword argument *)
                     | "*" , expression                       (* spread positional *)
                     | "**" , expression ;                    (* spread keyword *)

  primary         ::= literal
                     | qualified_name
                     | "(" , tuple_expression , ")"
                     | "(" , ")"                              (* empty tuple *)
                     | array_literal
                     | pair_literal
                     | dict_literal
                     | "return" , [ expression ]
                     | "yield" , "from" , expression
                     | "yield" , [ expression ]
                     | "break"
                     | "continue"
                     | "pass"
                     | if_stmt
                     | while_stmt
                     | for_stmt
                     | lambda_expr
                     | co_lambda_expr
                     | anonymous_class
                     | do_expr ;

  decorated_expr  ::= decorator , { decorator } , expression ;

Example: v := @accept 'TOK_XID (see decorator in parser.wy)

  literal         ::= nil_literal
                     | number_literal
                     | bool_literal
                     | string_literal
                     | symbol_literal
                     | char_literal
                     | "..." ;

  array_literal   ::= "[" , [ list_expression ] , "]" ;

Example: [1, 2, 3, 4]

  pair_literal    ::= "$[" , [ list_expression ] , "]" ;             (* cons-list primitive *)

Example: $[] (empty), $['a] (single element), $[1, 2, 3] (proper pair list)

  list_expression ::= expression , { "," , expression } , [ "," ] ;

  dict_literal    ::= "{" , [ dict_pair , { "," , dict_pair } ] , "}" ;
  dict_pair       ::= expression , ":" , expression ;

Example: { "Name": 15 }

Anonymous Closures and Types #

These provide a way to create an anonymous class or function. Note that anonymous classes must provide their messages internally as message definitions require the lexical class name.

  lambda_expr     ::= "fn" , "(" , [ param_list ] , ")" , [ "->" , type_constraint ] , block ;

  co_lambda_expr  ::= "co" , "(" , [ co_param_list ] , ")" , [ "->" , type_constraint ] , block ;

anonymous_class ::= "class" , [ "(" , [ qualified_name ] , ")" ] , block ;

Example:

AnonPerson := class() { slot name: str; slot age: int; }

# Equivalent
class NamedPerson:
    slot name: str
    slot age: int

  do_expr         ::= "do" , block ;

Example:

complex_answer := do:
    step_1()
    step_2()
    10
# complex_answer == 10