Cone Language Reference

Lexical Elements

On this page

A source program is a collection of UTF-8 encoded characters. The job of the lexer is to group adjacent characters together into distinct tokens, separated by white-space and comments. The compiler's grammar-driven parser only uses the tokens, and ignores all human-friendly white-space. The one place white-space carries meaning is inside a multi-line string literal, whose content is what its lines hold.

Tokens#

The type of every token is determined by its first characters:

Numeric Literal#

A numeric digit (from '0' to '9') starts an integer or float literal. Although a negative sign ('-') preceding a numeric literal is not considered part of the token (as that dash might be a minus sign), it still has the desired effect of negating the number that follows. Using a negative sign on an unsigned integer literal does not make it a signed one.

Underscores may be used within a numeric literal to improve readability.

Integer Literal#

An Integer literal may be:

  • Decimal integer. A simple sequence of numeric digits which stops at the first non-digit character.
  • Hexadecimal integer. Starts with '0x' followed by either numeric digits or 'a'-'f' (or 'A'-'F') to represent the hexadecimal digit values ten through fifteen.

By default, an integer literal is a signed 32-bit number. To change this, specify one of the following suffixes:

  • i8 signed, 8-bit.
  • i16 signed, 16-bit.
  • i64 signed, 64-bit.
  • u8 unsigned, 8-bit.
  • u16 unsigned, 16-bit.
  • u32 or u unsigned, 32-bit.
  • u64 unsigned, 64-bit.
  • usize unsigned, 32- or 64-bit.
  • isize signed, 32- or 64-bit.

An unsuffixed integer literal takes the number type its context wants, such as a variable's declared type or a parameter's. Whatever type an integer literal ends up with, from its suffix, its context or the default, its value must fit that type, or it is an error: 300u8 and imm b u8 = 300 are both refused. A negative sign counts: -128i8 fits and 128i8 does not. One whose context gives it no other type keeps the default, and one whose value does not fit a signed 32-bit number is then an error: give it a suffix, or a declared type.

Float Literal#

A float literal also starts with a digit from '0' to '9'. To distinguish it from an integer literal, it must contain a decimal point, exponent ('E' or 'e'), or a type suffix that explicitly declares it to be a float ('f', 'd', 'f32' or 'f64'). A period is considered to be a decimal point if it is unambiguously not being used as part of a range operator; which is to say that the period is not immediately followed by another period.

The Float token may specify an exponent, which is indicated by an 'e' or 'E' followed by an optional minus sign and additional numeric digits.

By default, an float literal is a 32-bit number. To change this, specify one of the following suffixes:

  • f or f32 32-bit.
  • d or f64 64-bit.

A float literal whose value is too large for its type is an error, wherever it is used: 1e39 is a 32-bit number even where a 64-bit one is wanted, and is written 1e39d.

Character Literal#

A character literal begins and ends with a single quote (') within which must be found a single UTF-8 unicode character. Its type is char, a Unicode code point held in 32 bits, whatever its value: 'a' is U+0061 and 'é' U+00E9, written as itself or with an escape. A char is a Unicode scalar value, so a surrogate (U+D800 to U+DFFF) and anything past U+10FFFF is an error. A char is no number: it is compared (==, < and the rest, by code point) and hashed, has no arithmetic, and converts to a number or from one only explicitly, u32.from(c) and char.from(n) (Explicit Conversion). The characters of a string are its code points, s.chars().

One literal reaches a number implicitly, as an integer literal takes the number type its context wants: a character literal written as it is, whose value is ASCII (below 128), is a u8 where a u8 is wanted, so a byte is compared with, set to and computed against 'a' directly:

imm b u8 = 'a';              // the byte 97
if b >= '0' and b <= '9' { .. }
imm digit = b - '0';         // a u8
imm c char = 'é';            // U+00E9
imm bad u8 = 'é';            // **ERROR** a code point beyond ASCII is no byte: U+00E9 is not 0xE9 in UTF-8
imm n u32 = c;               // **ERROR** a char is not a number
imm n u32 = u32.from(c);     // 233

Only the literal as written does this: a name holding one, const A = 'a', is a char. (A 'u' suffix, which once made the literal a u32, is an error.)

Any character whose Unicode value is 0x0020 or higher can be specified explicitly, other than delete (U+007F), and so can a tab. A line's end cannot: a character literal ends with its line, so a new-line is written '\n' and a carriage return '\r'. Nor can any other control character (U+0001 to U+001F, and U+007F): it cannot be seen in an editor, so writing one as itself is an error, and it is written with an escape sequence, such as '\x1B'. Alternatively, one of the following escape sequences that begin with '\' may be used:

\a
Alarm (U+0007)
\b
Backspace (U+0008)
\f
Form feed (U+000C)
\n
New-line (U+000A)
\r
Return (U+000D)
\t
Tab (U+0009)
\v
Vertical tab (U+000B)
\\
\
\'
'
\"
"
\0
Null character (U+0000). It may not be followed by a digit: Cone has no octal escapes, so "\012" is an error rather than a null followed by '1' and '2'. A null followed by a digit is written with \x00, as in "\x001".
\xnn
hexadecimal code value for a byte, in exactly two hexadecimal digits. In a character literal it is the code point of that value, so '\xFF' is U+00FF.
\unnnn
unicode character which matches the specified hexadecimal code point.
\Unnnnnnnn
unicode character which matches the specified hexadecimal code point.

String Literal#

With string literals, multiple techniques are offered for specifying text content: All allow specification of multiple unicode characters, but they vary in the handling of a few special characters as needed for different circumstances.

A null character (U+0000) is always appended to the end of a string literal, for C compability. All string literals are treated as immutable. The count of its bytes leaves out that null, and a string literal's type is &str, a borrow of its text that lasts as long as the program (managed references). Where a C string is wanted, a cstr, a literal converts to one.

A tab written as itself inside a string literal is part of its content. Any other control character (U+0001 to U+001F, and U+007F) written as itself is an error, since it cannot be seen in an editor: it is written with an escape sequence, such as \x1B. Line ends are the exception, as described for multi-line string literals below.

Escaped vs. Raw Text#

Many times it is convenient to use escape sequences in order to visibly include control and unicode characters. Other times, such as with regular expressions or XML text, it is less error-prone and more readable to be able to specify backslashes or double quotes without having to escape them with backslashes.

The first characters of the string literal establishes how escape sequences are handled:

  • Just a double quote (") begins a string literal that uses escape sequences. If a double quote or backslash is needed as part of the content, they must be escaped with a backslash.
  • 'r' followed by a double quote (r") begins a raw string literal which does not use escape sequences. Backslashes need not be escaped. Double quotes cannot appear in the string literal.
  • 'r' followed by a backtick (r`) begins a raw string literal which does not use escape sequences. Backslashes and double-quotes need not be escaped. Backticks cannot appear in the string literal.
  • Three double-quotes (""") begins a string literal that uses escape sequences. Backslashes must be escaped, but double quotes need not be.
  • 'r' followed by three double quotes (r""") begins a raw string literal which does not use escape sequences. Neither backslashes nor double quotes need to be escaped to appear in the text content.

If a string literal begins with a single double-quote (or backtick), it ends with the next double quote (or backtick). If the string literal begins with a triple double-quote, it ends with a triple double-quote. If there are more than three double-quotes at the end, the terminator is the last three:

""""Happy Birthday!""""   // yields the string literal: "Happy Birthday!"

Multi-line String Literals#

Some string literals are long enough to require multiple lines of code to specify. It would be convenient to be able to format such content properly using indentation and readable margins, without that formatting necessarily carrying over into the text of the literal. A multi-line literal can sit indented with the code around it and still come out flush: a shader embedded in a program, say, is indented as the code is, but the shader's own text needs none of that indentation, and would look out of place written hard against the left edge.

These are the rules that make that possible:

  • Multi-line string literals must end the line right after the last of the opening double-quotes. The end-of-line character(s) that follow are not included in the string literal.
  • The double quotes that terminate the literal must appear at the start of a subsequent line, although they may be indented by any number of spaces or tabs. That indentation is the literal's margin.
  • The lines in-between contain the string literal's content. A line that is blank, or holds nothing but spaces and tabs, is an empty line. Every other line must begin with exactly the margin: the same spaces and tabs in the same order, not just as many of them. The margin is stripped, and any further indentation is content. A line that does not begin with the margin is an error. A new-line character is appended to the line's content, unless the end-of-line is preceded by a backslash.

For example:

// Equivalent to "a\nb"
"
   a
   b\
   "

Lifetime Annotation#

A lifetime annotation looks similar to a character literal, in that it begins with a single quote ('). It is followed by a letter and then any number of alphanumeric characters. The absence of a closing single-quote distinguishes it from a character literal.

'a
'static

Identifier#

Other than the reserved keywords, a program may define and use any identifier as a variable, member, function, method, type, etc.

An identifier begins with a letter or '_'. A letter may be 'a'-'z', 'A'-'Z', or any unicode-defined universal letter as defined by C99 in ISO/IEC 9899:1999(E) Appendix D. Identifiers are case-sensitive; 'abc' is different from 'ABC'.

Subsequent characters may be a letters, digits, '$', or '_'.

To be able to include other characters, such as punctuation, as part of an identifier enclose the entire identifier in back-ticks.

The following are all valid identifiers:

balance toReturn True temp_ $ π `*`

Note: '_' by itself (not followed by a letter or a number) is not an identifier, but a special punctuation token.

Keywords#

These keywords are reserved and may not be used as identifiers:

Some are reserved ahead of the feature that will use them, so that a program written today does not have to be rewritten when that feature arrives. The compiler reports the use of any reserved word as an identifier. The six static permissions, imm, mut, mut1, opaq, ro and uni, are reserved words too: the compiler reads each as a permission wherever it is written, so none can name a variable, a function, a type or a module. panic and assert are not keywords but the core library's functions (panics), names every module has, as it has core's others. Nor are of and fill: each is a word only in the list after the append operator, xs <- 3 of 0 and xs <- fill 0, and an ordinary name everywhere else. So are yields, a word only after a function's parameters (generators), and parallel, a word only directly before each (parallel each). yield, by contrast, is a keyword.

actor
declares an actor, an independent unit of concurrent execution that communicates by message (actors). actor trait is the abstraction of one, which is not built yet: the compiler reports it where it is written
alias
declares a second name for something that already exists: alias Name = target; (alias)
and
logical 'and' operator used in boolean expressions
async
the first word of async do, which declares an actor's behaviour: a method sent as a message, run when the actor is scheduled. Alone it means nothing, and is refused
await
in an actor's behaviour, await x waits for x: the behaviour returns to its actor's dispatcher there and resumes when it is answered. Built on a behaviour of an actor that returns a value; refused on one that returns nothing; on anything else the compiler checks where it stands and what may cross it, and reports one it accepts where it is written (but for testing, under --await-direct)
baseurl
url of the program's source code
break
terminate a loop block, giving the loop's value when the loop is an expression (break)
but
leaves named members out of a fold of every member: a field's, a global's or an import's use *, or a use that names a sibling, an enum or a submodule
by
the step of a number range in an each loop: each x in 0 .. 10 by 2
catch
handle an error thrown by the block it follows
context
The value of the currently executing execution state
continue
Re-iterate a 'while' or 'each' block
do
the second word of async do, which declares an actor's behaviour. Alone it means nothing, and is refused
each
The block for iterating over a sequence of values; also an entry in the list after the append operator, and, after yield, the values of another generator
else
A clause within an 'if' statement, and the value of an 'each' or 'while' loop that runs out of items (each, while)
elif
A clause within an 'if' statement
false
The value of 'false'
if
A conditional statement; also the filter after an 'each' source and the condition that may end a 'break', 'continue' or 'return' (each, trailing if)
imm
the immutable static permission
in
A clause within a 'each' block
include
retired: a module's files are the files of its folder, so no statement brings one in. The compiler reports an include statement where it is written
into
retired: a value converts with its type's method, f32.from(x) (conversion), and a reference narrows to a variant with a check, by a match or if imm c Circle = &s. The compiler reports an into where it is written
is
asks whether one thing is another. Of a value it is the runtime test p is Mobile, listed again below among the evaluation operators. Of a type, in its own declaration, it asserts compliance with one or more traits: struct Gauge is Meter. The question is the same in both; only the operand differs, and a type's answer is settled at its declaration where a value's is not
local
ensures variable(s) are treated as locally scoped
match
Matches a calculated value to several possible values
mixin
retired: a type takes a trait in by naming it with is on its declaration line — the struct's or trait's, the enum's or the variant's. The compiler reports a mixin statement where it is written
mod
declares a module, as mod name; at the head of the file that declares it, and pub mod name; opens a submodule to its parent's neighbours. mod name extends base; makes it a module that extends another, reusing what it declares and folds in. mod name use Thing; names the module's default fold, what a bare import name; folds, and comes last on the line, after any extends and is. mod trait Name { ... } declares a module trait, the interface a module conforms to, and mod name is Name; says the module conforms to it. Source has no mod name { ... }: a module is never declared inside a source file, and a nested module is a file of its own or a subfolder with its own designated file. Only a package's generated include file writes one
mut
the mutable static permission
mut1
the sixth static permission: it may read, change and share its value, and a reference into an enum variant's payload may be taken through it
new
constructs a new value by one of its type's initializers, new Point(1, 2), or allocates one in a region, new Rc[mut, Node](1); and, written &new, the permission of an initializer's self, a reference to memory not yet filled
not
logical 'not' operator used in boolean expressions
null
The null raw pointer, of whichever pointer type it is wanted as
opaq
the opaque static permission, which neither reads nor changes its value
or
logical 'or' operator used in boolean expressions
pub
declares a name visible outside the module or type that owns it; a name not so declared is private. Before import it makes every name the import binds public; before a fold clause's use, as pub use, only the names the clause folds in. It always comes first, before the keyword of what it makes public
return
terminate execution of a method with a return value
ro
the read-only static permission
self
references the method's self parameter value
selfactor
in an actor's method, the actor's own handle, another owner of it; self is its state, which never leaves it
selfmethod
the currently executing method or closure
spawn
begin a new independent execution context
this
The value of the most inclusive 'with' block
throw
abandon the current block with an error value
trait
modifies a kind to declare its abstraction, as in struct trait. Written by itself it means struct trait
true
The value of 'true'
trynew
an allocation that may fail, trynew Rc[mut, Node](1): an Option of the owning reference, None when memory runs out
typedef
retired: an alias is declared alias Name = type;. The compiler reports a typedef statement where it is written
uni
the unique static permission
use
folds members of a field's type, of a sibling enrichment, or of a module's global in as names of the type or module holding it (delegated inheritance); on an import, it folds the imported module's public names in as names of the importing module (Namespace folding); written as a statement of a module, it folds an enum's variants (Naming a variant) or a submodule's public names (Submodules) in as names of the module; on a mod line it runs the other way, naming what a bare import of the module folds into the importer (A module's default fold)
using
clause on this block
wait
A block that waits until all its execution contexts are done.
where
a generic's constraints, just before its block: where T is Integer and T is Copy (Generic constraints)
while
A repetitive block
with
A clause in a 'match' block
yield
in a generator, a function declared with yields after its parameters, hand the caller a value and wait to be resumed; yield each hands on the values of another generator. yields is a word only there, a name everywhere else

Attributes#

An attribute is a word beginning with '@': an instruction to the compiler about a declaration's representation or linkage, which no code asks about. A type declaration takes two, written after the keyword that declares it: @opaque and @unsized. (A function or a module takes its own: @c, @initpure, @intrinsic and @compute; a global variable one of two, written after its permission: @threadlocal or @workgroup.) The compiler reports any other word beginning with '@' and reads the declaration without it.

A fact about a type that other code may ask about or constrain on is not an attribute but a trait, declared with is. Whether a type moves is such a fact, so a type that moves declares is Move; the old @move attribute is reported with that advice.

Held for unimplemented features#

Two spellings besides the reserved words are held for features that are not implemented yet. The compiler reports each wherever it is written:

#
begins a word of metaprogramming, such as #if. The compiler reports the word and ignores the rest of its line
?.
None propagation on a method call or field access. The compiler reports it and reads it as '.'

Operator and Precedence#

An operator is a sequence of one or more punctuation characters with no intervening white space. The compiler is greedy, and will look for the longest character sequence that matches one of these operators.

The operators are sequenced from highest to lowest evaluation priority. Operators grouped together have the same priority. In parenthesis is shown whether an operator appears in front of a value (prefix) or between values (infix). Some operators can be used in both contexts. It also specifies the symbolic name of any method associated with that operator.

Value operators#

( )
(p) prioritizes expression to be evaluated as a group
[ ]
(p) array
{ }
(p) code block

Term operators#

.
(i) method call/property access, and a path through a module or a type
( )
(i) method call parameters
[ ]
(i '[]') indexed access

Prefix operators#

-
(p '@-') negate
@
(p) Internet resource load
<<
(p '<<') append to 'this'
>>
(p '>>') prepend to 'this'

Arithmetic/Collection operators#

**
(i '**') exponent
*
(i '*') multiply
/
(i '/') divide/split
%
(i '%') remainder
+
(i '+') add
-
(i '-') subtract

Range operator#

..
between the bounds of a range, which runs through the last: of an index, &s[a..b], the elements from a through b, either bound optional (slices); of a pattern, case 3 .. 6; of an each loop, each x in 0 .. 10 by 2. It makes no value on its own
..<
the same range operator, stopping before the last: &s[a..<b], case 0 ..< 3, each x in 0 ..< 10. There is no ...

Evaluation operators#

==
(i '<=>') equal
!=
(i '<=>') not equal
===
(i) identical: two references (or pointers) point to the same place
!==
(i) not identical
is
(i 'is') match
<=>
(i '<=>') compare (rocketship)
<
(i '<=>') less than
<=
(i '<=>') less than or equal
>
(i '<=>') greater than
>=
(i '<=>') greater than or equal

Evaluation operators#

! not
(p) logical not
and
(i) logical and (there is no &&)
or
(i) logical or. (|| is not logical or: it is the "or else" of a throwing call, designed in Exceptions and not built.)

Append/Prepend operators#

<<
(i '<<') append
>>
(i '>>') prepend

Assignment operators#

Note: Within variable declarations, the evaluation priority of ',' and '=' are reversed.

,
(i) value separation
=
(i) assign
+=
(i '+') add in place
-=
(i '-') subtract in place
*=
(i '*') multiply in place
/=
(i '/') divide in place

Statement terminator#

The semicolon ; ends every statement that does not end in a block. See lines, statements and blocks.

End-of-File#

The program code ends when it reaches a null character (U+0000) or has no more characters. The lexer will tidy up whatever is unfinished (e.g., still open blocks, literals, or comments) and then will generate the end-of-file token for the parser.

Whitespace#

Whitespace consists of the source characters which are largely ignored by the parser. Whitespace comes in two forms:

  • Whitespace characters. These are all the ASCII control codes (1 - 32), including space, tab, new-line and carriage return. These are useful for fashioning a human-digestible layout to the program and for separating two tokens, and that is all they do: the indentation of a line means nothing to the compiler.

    Note: If the program source begins with the UTF-8 byte-order mark (U+FEFF), it is ignored.

  • Comments. These are used to provide helpful documentation that improve a human reader's understanding of the source program. Cone supports both line and block comments.

Lines, statements and blocks#

The newline character separates one line from the next (carriage returns are ignored). Lines are counted so that a diagnostic can name the one it is about, and that is the only use the compiler has for them. A line's indentation is formatting for the reader; it never decides where a statement or a block ends.

Some statements (e.g., if) finish with a block. Every other statement is terminated by a semicolon, and the semicolon is never optional: not when the statement finishes at the end of a line, and not on the last statement before a block's closing brace. Several statements may share a line, each ended by its own semicolon:

isEmpty = true; count = repl * elements;

Since nothing but the semicolon ends a statement, a statement may run on across as many lines as it needs, whatever the follow-on line begins with. An operator that could serve as either a prefix or an infix operator (such as '*', '-', '.', '<-', '(', or '[') at the start of a line is always read as continuing the statement in progress:

mut a = b
  * c;       // One statement: mut a = b * c;

To write two statements instead, end the first one:

mut a = b;
*c;

A forgotten semicolon is reported rather than inferred. The diagnostic is placed at the end of the statement the semicolon belongs to, not on the line that follows it.

Blocks#

A block is delimited by curly braces, and by nothing else:

if a == 0 {
  break;
}

A small block may sit on one line, its statements each ended by a semicolon:

if a == 0 { break; }
if onSameLine { doSomething(); doSomethingElse(); }

Earlier versions of Cone also accepted a colon in place of the opening brace, with the block's extent given by the indentation of the lines that followed. That form is gone: a colon where a block should start is reported as such, with the braces to write instead.

Comments#

Comments make the code's logic easier to understand and maintain. They are for people. They have no impact on program execution.

Comments may be placed between any two tokens (and certainly not within a text literal).

There are two types of comments:

  • A line comment. begins with '//' and continues to the end of line.
  • A block comment starts with '/*'. It ends with its corresponding '*/', extending across multiple lines, if desired. Block comments may be nested inside each other. Line comments may be nested inside a block comment. '/*' or '*/' embedded in text literals or line comments are ignored.