Calling C: extern and @c

Much of the world's useful code is written in C: operating-system services, graphics libraries, compression, the C library itself. A Cone program calls it directly, at the cost of a C call, and C can call back into Cone the same way. This page covers declaring what C defines, C names and the bindings built from them, publishing Cone functions to C, and how structs cross between the two. Afterwards you can write a binding to a C library. Here a Cone program uses three C functions, one of which calls Cone back:

mod csort;

import stdio use *;

// C's div_t, laid out as C lays it out: two ints
struct DivT { pub quot i32; pub rem i32; }

// Three functions of the C library, declared with their C names
extern {
  fn @c div(numer i32, denom i32) DivT;
  fn @c strlen(s cstr) usize;
  fn @c qsort(base *u8, count usize, size usize, compare &fn @c (a *u8, b *u8) i32);
}

// A Cone function C can call: qsort hands it two pointers into the array
fn @c byValue(a *u8, b *u8) i32 {
  let x = *(a as *i32);
  let y = *(b as *i32);
  if x < y { -1; } elif x > y { 1; } else { 0; }
}

fn main() {
  let d = div(17, 5);
  print <- "17 / 5 is ", d.quot, " remainder ", d.rem, "\n";

  print <- "\"Cone\" is ", strlen("Cone"), " bytes long\n";

  var loaves = [42, 7, 19, 3, 25];
  qsort(&mut loaves as *u8, 5usize, mem.sizeof[i32](), &byValue);
  each n in loaves {
    print <- n, " ";
  }
  print <- "\n";
}

It prints:

17 / 5 is 3 remainder 2
"Cone" is 4 bytes long
3 7 19 25 42

Try this: make byValue sort the loaves from most to fewest.

C is not held to Cone's rules, and Cone cannot check that C code is as careful as the compiler is with Cone's. So the boundary is plain to see: C's types are spelled out (*u8 is a raw pointer, cstr a C string), and a call to an extern function is one of the things trust exists for, once it is built. Reaching another Cone module is a different matter, the business of import (Namespaces).

extern: defined elsewhere#

extern says one thing: this function, method or global variable is defined elsewhere, in another compiled unit, a C library or a separately built Cone package. So it is declared here without a body or a value, and the linker finds its definition. It says nothing about how the symbol is spelled; that is for @c and the module to say. extern is written in front of a declaration, or of a block of them:

pub extern fn addOne(n i64) i64;
extern let hits Atomic[u64];

pub extern {
  fn printStr(str &str);
  fn printInt(nbr i64);
}

A pub before a block reaches every declaration in it. A declaration may be private: an include file declares the private names a public inline body reaches, since that body is compiled into the importer, which then needs them. A function with a body, or a global with a value, is an error in an extern declaration. A thread-local global is declared with its attribute, extern var @threadlocal depth i64;, and may name a C thread-local in a @c module as well as a Cone one.

Methods and operators#

Inside a type, extern fn declares a method, a static function or an operator defined elsewhere:

pub struct Acc {
  pub total i64;

  pub extern fn plus(self, n i64) i64;
  pub extern fn `+`(self, other Acc) Acc;
  pub extern fn zero() Acc;
}

Such a method meets a trait's requirement like any other. What a user needs the body of cannot be extern, and is an error: an inline or generic function, a trait's methods, a generic type's methods. They are written whole, and compiled where they are used. A type's fields are part of its value, and are declared as they are.

The types a C library uses#

A C library's functions and globals usually come with types and constants of its own. These are declared beside the extern declarations, in the same module, since an extern block holds functions and variables only. Marking an external function trustable, so that a call to it that passes a pointer needs no trust block, is not built yet.

A struct C shares is declared with the same fields in the same order, as DivT is above. Where C's layout has a part Cone has no use for, padding or a reserved slot, the field may be named _: it keeps its place in the layout, and nothing can read it by name. Since a value must still be built with every field, give such a field a default, or a value in its place when the fields are listed in order:

struct Header {
  pub kind u16;
  _ u16 = 0u16;      // C's reserved half-word
  pub size u32;
}

let h = new Header(kind: 1u16, size: 64u32);     // 8 bytes, as C lays it out

C's constants as an enum: tag values#

A C library's constants are often a list of named numbers, and an enum whose tag values match them lines up with it. Enums shows the basics: a variant may write its own tag value, counting resumes from the last value written, and the enum may name the integer type its tags are laid out in. Naming that type fixes the tag's width, so a value too large for it is an error rather than a silent widening away from the layout you asked for:

enum TooBig u8 {
  Fine = 1,
  Large = 300;      // **ERROR** 1092: Tag value 300 of Large does not fit u8, this enum's integer type,
}                   //   whose values run from 0 to 255.

Without a named type, the tag is as wide as the largest value needs. Note that this follows the largest value, not the number of variants: an enum of two variants, one of them 300, has a 2-byte tag.

A tag value may be negative, as some libraries' constants are. Each value must fit the named integer type the way any integer literal must fit its type, and a negative sign counts: -4 fits i32, while its bit pattern 0xFFFFFFFC does not (1092 again), and a negative value fits no unsigned type. Counting resumes from a negative value too, counting up. An enum that names no type and holds a negative value gets the narrowest signed tag that holds every value: one holding -3 and 100 takes a single byte.

enum VkResult i32 {
  Success = 0,
  NotReady,                  // 1
  ErrorOutOfHostMemory = -1,
  ErrorOutOfDeviceMemory = -2,
  ErrorDeviceLost = -4;
}

For alignment, the tag may be wanted somewhere other than first. Write an anonymous field of type tag where it should go. tag means this only where a field's type is written, so it is not a reserved word, and a field may still be named tag:

enum Aligned {
  pub wide i64;
  _ tag;

  pub struct Small {pub n i32;}
  pub struct Large {pub a i64; pub b i64;}
}

Opaque types#

Some types only the C library knows the size and layout of: it hands out pointers to them and takes them back, and is reluctant to say more, or the program has no need to know. Such a type is opaque, since its contents cannot be seen, and it is declared as a struct with no fields:

struct @opaque Handle;

extern fn @c fopen(path cstr, mode cstr) *Handle;
extern fn @c fclose(f *Handle) i32;

No variable or field may hold an opaque value, so none can be created or passed around, its fields cannot be reached, and even its size is unknown (mem.sizeof refuses it). It is used through references and pointers, which can be obtained, passed around and stored like any other. Methods may be declared on it, typically implemented by the library that knows its layout, but a pointer to it may not be dereferenced to read or change the value:

let h = *f;    // **ERROR** 1069: Variable h cannot be held by value: Handle is declared @opaque,
               //   so this program is not told how large it is. Hold it through a reference.

Text and buffers#

C passes text as a pointer to bytes ending in a zero. Cone's name for that is cstr, and a binding declares C's strings with it, as strlen does above. A string literal is a cstr wherever one is wanted, and so is a String's cstr(), with no copy; a plain &str is not, since it promises no zero after its bytes. Everything about C strings, including how text C hands back is read and kept, is in C strings: cstr. A C buffer is a pointer and a count, two parameters: a slice is passed as xs as *T and its length (A slice is no pointer).

A list that a C function fills#

Many C functions fill a buffer the caller provides, after the caller asks how many values there will be. A List handles that shape with no unsafe code: list.resize(n, zero) makes it n copies of a value long (or drops the values past n), the C function is handed list.viewMut(), as a pointer, list.viewMut() as *T, with the count, and list.truncate(written) cuts it to the count C wrote, finalizing the values it drops (it does nothing when written is not less than the length). resize exists only where T copies; a list of values with finalizers is shortened with truncate. Here a Cone function stands in for the C one:

mod cfill;

import stdio use *;
import collections use *;

// Stands in for a C function that fills at most out.len values and says how many it wrote
fn squares(out &mut Array[i32]) usize {
  var n = 0usize;
  while n < out.len and n < 5usize {
    out[n] = i32.from(n) * i32.from(n);
    n = n + 1usize;
  }
  n;
}

fn main() i32 {
  var xs = List[i32].empty();
  xs.resize(8usize, 0);                 // room to fill: eight zeros
  let written = squares(xs.viewMut());
  xs.truncate(written);                 // keep only what was written
  print <- xs.len();
  print <- ": ";
  each x in xs {
    print <- *x;
    print <- " ";
  }
  print <- "\n";
  0i32;
}

It prints 5: 0 1 4 9 16. The list's other methods are in the Collections reference.

extern in a file of its own#

It can help to put the extern declarations for a C library (libcurl, say, or SDL) in a file of their own. Placed in a module's folder, that file is one of the module's files like any other, so every file of the module can use its declarations. A C-named binding is a module of its own, since @c is on its mod line.

C names: @c#

Cone spells the names the linker sees its own way, with the module and type a declaration belongs to. C spells them bare. @c says a name is spelled as C spells it, and is written after the keyword of a mod line or a fn, in one of four forms: @c, @c("text"), @c(system) and @c("text", system).

On a fn it gives that one function a C name: bare, its own name; with a string, the string is its whole symbol. That is how the program above declares div, and how a C function is declared in an ordinary module:

pub extern fn @c abs(n i32) i32;

system selects the platform's system calling convention (stdcall on 32-bit Windows), for the function or for every function of the module, and an extern one is also imported from a DLL:

extern fn @c(system) GetTickCount() u32;

@c goes only where there is one symbol to name. It is an error on a type, on an anonymous, generic or inline function, and on a trait's or a generic type's method. extern system, the older spelling of the system convention, is an error that names @c(system).

C-named modules: a binding to a C library#

A whole module is C-named by writing @c after mod. Its functions and global variables then take their names unchanged, as C spells them, after the prefix a string gives, and @c(system) gives its functions the platform's system calling convention. That is what a binding to a C library is: a module whose names bind to the library's symbols, imported like any other, and seen from outside as Cone:

mod @c("SDL_") sdl;

pub extern {
  fn Init(flags u32) i32;                    // the C function SDL_Init
  fn Quit();                                 // SDL_Quit
  fn @c("IMG_Load") LoadImage(file cstr) *u8; // a name outside the prefix
}

Everyone else imports it and calls sdl.Init. The C library itself is bound this way: the libc package is the module mod @c libc;, and a program that imports it calls libc.fwrite, libc.remove and the rest by their C names.

Inside a C-named module, a @c with a string on a function names a symbol outside the prefix, as LoadImage does; a bare @c on a function the module already gives a C name says nothing, and is an error. Naming, and the C calling convention that comes with a C name, are the only things the module changes about its declarations (apart from its prelude, below). Whether each declaration is defined elsewhere is its own extern, and a C-named module may hold bodies too, which it compiles and publishes under C names when they are pub. A type's methods and a generic function keep Cone names, since a C name has no room for an owner or type arguments. Such a module, alone in its package, is a C package, whose manifest names the C library to link.

A C-named module does not import core automatically, as every other module does: it declares what a C library defines, in C's types, and core itself imports one, libc. So core's names, such as Option and malloc, are not in reach in it unless it imports core itself, and declaring one of them there is no clash.

Exporting to C#

The other direction works the same way. A Cone function with a body is published to C under a C name by writing pub and @c: the symbol is then external, and C code links against it. Without pub it is still spelled as C spells it, but stays private to its object, which is all a callback like byValue needs. The same holds for the bodies of a C-named module.

pub fn @c("cone_square") square(x i64) i64 { x * x; }

fn @c("main") start() i32 {    // the program's entry, under another Cone name
  0i32;
}

A callback parameter that C calls is typed &fn @c (sig), as qsort's compare is, and the Cone function handed to it is declared fn @c. That reference type is never taken for a Cone function's &fn(sig), since the two are called differently (A reference to a C-named function). A pointer C hands Cone to a function is called as C calls it, too. A type's method named with @c keeps Cone's convention, since a vtable calls it by its Cone signature.

Structs and bools by value#

Note: a C-named function passes a struct by value as C does on x86-64 (Windows, Linux and macOS) only, not yet on WebAssembly.

A function with a C name is called as C calls it, and calls C as C expects. Where it takes or returns a struct by value, the struct is passed the way the platform's C compiler passes the same C struct. So a Cone struct laid out as C lays out the struct crosses either way unchanged: to an extern function C defines, and into a Cone function C calls, directly or through a pointer that Cone handed C.

struct Vec4 { pub x f32; pub y f32; pub z f32; pub w f32; }

extern fn @c v4_sum(v Vec4) f32;       // float v4_sum(Vec4 v);
pub fn @c scaled(v Vec4, k f32) Vec4 { // C calls it: Vec4 scaled(Vec4 v, float k);
  new Vec4(v.x * k, v.y * k, v.z * k, v.w * k);
}

How a struct crosses is the platform's C ABI, and the compiler follows it:

• On 64-bit Windows a struct of 1, 2, 4 or 8 bytes crosses as one integer of its size, whatever its fields (floats too). Any other size is passed as a pointer to a copy the caller made, and is returned through a pointer to the caller's result slot, which C passes as a hidden first parameter. The Vec4 above is 16 bytes, so it crosses by pointer both ways; a struct of two i32s is one 64-bit integer.
• On x86-64 Linux and macOS (System V) a struct of up to 16 bytes is passed in registers, each eightbyte of it in an integer register if any integer or pointer is in it, and in a floating-point register if it holds only floats: the Vec4 above travels in two floating-point registers, a struct of an i32 and an f64 in one of each, and a result likewise comes back in registers. A larger struct is copied to the stack, as is one that finds too few registers left, and a larger result comes back through the caller's result slot.

The C library's own div and lldiv return a struct each way, on 64-bit Windows: div's 8 bytes as one integer, and lldiv's 16 through a result pointer:

struct DivT { pub quot i32; pub rem i32; }      // C's div_t
struct LldivT { pub quot i64; pub rem i64; }    // C's lldiv_t

extern fn @c div(numer i32, denom i32) DivT;
extern fn @c lldiv(numer i64, denom i64) LldivT;

let d = div(17, 5);          // d.quot is 3 and d.rem is 2
let l = lldiv(17i64, 5i64);  // so are l.quot and l.rem

A bool is C's bool: an argument or a result crosses as one byte holding 0 or 1, widened in its register as the platform's C compiler widens it, and a bool field of a struct is one byte holding 0 or 1, as a C bool field is. An integer narrower than C's int is widened by its sign as C widens it on that platform. A bool, and an 8- or 16-bit integer, cross as C's do on 64-bit Windows, x86-64 Linux and macOS, and WebAssembly.

extern fn @c SetWindowFullscreen(window *Window, fullscreen bool) bool;  // bool f(Window *, bool);

A slice, a tuple or an array is not a C type, and passes as Cone passes it: a slice's pointer and length arrive as two arguments, as a C function's (char *, size_t) takes them. Platforms and targets gathers what differs by target.

One C name, one symbol#

C has one namespace for its symbols, so a C name means the same thing wherever it is declared. Several modules of one program may each declare the same C function, and they all call the one function:

mod geometry;

extern {
  fn @c abs(n i32) i32;     // also declared by the module that contains this one
}

Every declaration of one C name must agree: a function in its parameter and return types and its calling convention, a global in its type and its permission. Two that do not are an error, as C's "conflicting types" is, and the error names both. One of them may have a body, which is then the C name's definition, and the others are declarations of it; two bodies under one C name are an error too. A private C-named function is not among them: it stays private to its module, and a C function of the same name declared elsewhere is still reached. The compiler checks the modules it compiles together; between packages compiled separately, the linker does.

Linking against a Cone package compiled on its own#

Needed by those who publish a Cone package for others to link against, and useful to anyone reading the files Congo generates.

extern is also how one Cone package reaches another that was compiled separately. The package is reached through an include file: the package's root module, with everything its object defines declared extern, and its types, macros, inline bodies and generics written in full. The include file's module is an ordinary, Cone-named module of the package's name, so each extern declaration is spelled exactly as the package defines it, and the two objects link. It is Cone source, named for the package: stdio's is build/debug/stdio.cone in a program Congo builds, generated by compiling stdio, and it declares the globals print and eprint and the methods of their type IOStream, all defined in stdio's own object.

Compiling a package as a library writes its include file, name.cone, beside the object. It is the package's own source with what a program does not need taken out: each body the package's object holds is left out and its declaration marked extern, each private declaration nothing a program expands reaches is left out, and everything else, comments included, is kept as written, under a comment saying the file is generated and is not to be edited. A global whose type the source leaves to be inferred has its type written in. Where the include file's lines stop being the source's, a comment such as //#line 24 "loc.cone" says the line after it is line 24 of the package's loc.cone, so an error or a panic in code a program compiles from the include file names the package's source file and line. Congo compiles each package before what imports it, and every importer against the include file that compile wrote, the packages that come with Cone included; an include file may still be written by hand where a build description names one.

What the package's root reaches in one of its submodules (a type a public function names, a function an inline body calls, the names a pub use re-exports) is declared in a nested module block inside the include file, holding only what is reached, edited the same way:

mod coll;

mod vec {
pub struct Stack {
  pub len i64;
  a i64;

  pub extern fn push(self, v i64) Stack;
}

pub extern fn stackOf(x i64, y i64) Stack;
}

pub use vec Stack, stackOf;

pub extern fn countOf(s &vec.Stack) i64;

Each name in a block is spelled after the submodule that declares it, as the package's object spells it. A block is private to the package: a program reaches Stack as coll.Stack, through the root's pub use, and cannot name coll.vec. A submodule nothing reaches has no block, and a generic submodule's block holds its whole text. Only a generated include file (one an import line names that opens with the generator's comment) may hold a module block; in source, and in an include file written by hand, a module is never declared inside a file, and the compiler says so (1100: A module cannot be declared inside a file).

An include file opens with its mod line and its imports follow, as in any module file. A package whose public functions take or return another package's types imports that package in its include file, as it does in its source. A program importing the first package can then use those values, their fields and their methods; it names the second package only if it imports it too, or if the include file's import says pub.

A global the package folds through is declared extern with the source's fold clause, as in pub extern var config Config pub use *;, so the names the fold puts into the package are names of the include file too.

A generic function or type is written whole, because each program makes the instances it uses from that body: the package cannot know which type arguments its users will give. The package's object holds the instances the package itself uses, and the program's object holds the ones it uses; where both use the same one, say larger[i64], both objects define it, and the linker keeps one copy. Any private function the generic's body calls is declared extern in the include file, as for an inline body.

pub fn larger[T](a T, b T) T {
  if a > b {a;} else {b;}
}

pub struct Pair[T] {
  pub a T;
  pub b T;

  pub fn sum(self) T {a + b;}
}

A package whose root module is a generic module, mod g[T];, is reached the same way: its include file is the generic module's whole source, and each package or program that uses an instance, say g[i64], compiles it. The copies merge at link, globals included, so the instance has one set of globals however many objects use it. A package's include file shows its root module alone, so a generic module inside a package is not yet reachable from outside it. Packages and the build covers the build description and the rest of what a package is.

You can now call C, be called by it, and write the binding a C library needs. What C hands back is often a raw pointer, which is next.