
        Precompile (a.k.a. ahead-of-time compile) Scheme code


*** Note: This is a feature still in development and very likely
    change over time.
    Especially, we usually keep ABI compatibility across micro versions
    so that you don't need to recompile extensions for every new
    release, but we don't guarantee that for precompiled binary yet---
    if you precompile your library to distribute, let users know
    that they need to recompile it for new Gauche releases, even if
    it's just a micro version update.
        
As a script engine, the default operating mode of Gauche is to read
Scheme source code at run-time, compile it on-the-fly and execute.
However, that incurs some overhead of loading and compiling, and
you might want to cut the time if you have tons of Scheme code.

Gauche has a feature to compile Scheme code and dump the result
as C code, which can be compiled by a C compiler into DSO and dlopen-ed
at runtime.  At this moment, we don't really "compile Scheme to C"---
what we do is to compile Scheme into Gauche's VM instruction sequence,
as we do at run-time, and to dump the result as a C static data structure.
Thus run-time performance won't improve by precompilation, but you do
cut the compile time, including macro expansion, and the parsing time.
In future, we may improve performance of precompiled code, for
we can spend more time on optimization.

Things to consider before precompiling
======================================

 1. The precompiler needs to make sense of source statically (i.e. without
    executing it), so we recommend the source to follow certain constraints:

    - Each scm source begins with a define-module form.
    - One scm source defines one module.

 2. In the normal operation, Gauche compiles and executes each
    toplevel form.  It allows you to use a value computed at
    run-time to be used in the macro expansion of the subsequent
    toplevel forms in the same file.  It doesn't work in precompilation,
    since macro expansion precedes execution.  This limitation
    includes procedure definitions.   For example:

      (define (quote-it x) `(quote ,x))

      (define-syntax quote-self
        (er-macro-transformer
          (^[f r c] (quote-it f))))

      (define (test)
        (quote-self a b c))

    This works by loading the source, but won't work in precompilation,
    since the expansion of quote-self requires to call quote-it,
    which is only available at run-time.

    You need to split the file, one contains the values the macro
    expander depends, and another contains the forms that uses the
    macro.  The macro definition itself can be in either file.
    (If the macro expander does not depend on the runtime value
    computed in the same file, you can put the macro definition
    and macro use in the same file.)

 3. The cond-expand macro is handy to switch architecture-dependent
    code.  Since it is a macro, it's expanded at the precompile time,
    so the features are selected based on the platform you're
    precompiling, not the platform you actually run the program on.
    That means you can't simply precompile a Scheme file and carry over
    the generated C file to other platform, if the Scheme file needs
    to select architecture-dependent part with cond-expand.  C code
    itself is portable, but the unportable selection is to be done
    before C file is created.

    If you need to distribute generated C code and you need to use
    cond-expand, just extract the part that uses cond-expand into
    a Scheme file and keep it from being precompiled.


The first step
==============

Let's begin with the simplest case.  Precompiling a single Scheme file.

Suppose we have greet.scm:
---------------------------------
(define-module greet
  (export greeting))
(select-module greet)

(define *message* '("Hello, " ".  How are you?"))

(define (greeting who)
  #"~(car *message*)~|who|~(cadr *message*)")
---------------------------------

Run precomp subcommand as follows.  The -e option is important---that
tells precomp to create a C file suitable for Gauche extension module
on its own.

  $ gosh precomp -e greet.scm

Now you see two files are generated: greet.c and greet.sci

  $ ls
  greet.c  greet.sci  greet.scm

The ".sci" suffix stands for SCheme Interface, which contains a small
Scheme code fragment for module definitions, plus forms that can't be
compiled to C, and the call to dynamic-load that loads DSO file:

  $ cat greet.sci
  ;; generated automatically.  DO NOT EDIT
  #!no-fold-case
  (define-module greet (export greeting))
  (select-module greet)
  (dynamic-load "greet")

Precompiled DSOs are always loaded via its interface file---it resolves
module dependency.  That means you have to distribute the interface
file along the DSO.  You don't usually see *.sci, though, since
gauche-install command can rename *.sci to *.scm while it copies
the file (with -C option).  We use *.sci suffix only because we
want to be able to generate the file in the same directory as
the source *.scm file.

Next, you can compile greet.c to generate greet.so (or greet.dylib
or greet.dll, depending on your platform):

  $ gauche-package compile greet greet.c
  $ ls
  greet.c  greet.o  greet.sci  greet.scm  greet.so

`gauche-package compile' command knows which C compiler and flags
are used when Gauche is compiled, where the Gauche header files are,
etc.  You have to compile the *.c file with the same compiler and
same CFLAGS, otherwise Gauche can't safely dlopen the resulting DSO.
You can give extra flags to the `gauche-package compile'; just type
this command with no arguments and it tells the usage.

Now, let's try the precompiled module.  We give -I. option to gosh
so that it can find greet.sci and greet.so.  We also give -fload-verbose
option to check if it really loads the precompiled file, not the
source file:

  $ gosh -I. -fload-verbose
  ;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm...
  ;;  Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm...
  ;;  Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm...
  gosh> (use greet)
  ;;Loading ./greet.sci...
  ;;  Dynamically Loading ./greet.so...
  gosh> (greeting "John")
  "Hello, John.  How are you?"  

Woohoo!  Note that *.sci takes precedence over *.scm, so even we have
both greet.sci and greet.scm, gosh loads greet.sci (and subsequently,
greet.so).

In practice, you have to install *.sci file and *.so file to
appropriate places so that other's code can find them.  The standard
places can be queried to with gauche-config command:

  $ gauche-config --sitelibdir
  /usr/share/gauche-0.9/site/lib
  $ gauche-config --sitearchdir
  /usr/lib/gauche-0.9/site/x86_64-pc-linux-gnu

You can use gauche-install to copy the files:

  $ gauche-install -T `gauche-config --sitelibdir` -C greet.sci
  $ gauche-install -T `gauche-config --sitearchdir` greet.so

The -C option renames greet.sci to greet.scm in the target directory.

(If you use the template makefile generated by 'gauche-package generate',
it has the skeleton to set up the installation destination.)


Multiple source files
=====================

The previous section shows converting one Scheme source into one DSO.
However, usually you want to make one DSO per library, which may
contain lots of source files.

Suppose you have the following source code structure, and foo.scm
uses sub files as foo.util1 and foo.util2.

   foo.scm
   foo/boo.scm
   foo/boo.scm

You have to generate C file for each Scheme source, then compile and
link all together.

   $ gosh -I. precomp -e foo.scm foo/util1.scm foo/util2.scm
   $ ls
   foo/  foo--boo.c  foo--woo.c  foo.c  foo.sci  foo.scm
   $ ls foo
   boo.sci  boo.scm  woo.sci  woo.scm

Note the -I. option we give to gosh, which allows precomp to find
foo/util1.scm and foo/util2.scm referred from foo.scm.
(This is actually a design flaw, since if the current directory contains
some files that conflicts the name with modules precomp depends, we
load the files under this directory instead of the standard one.
In future, we make precomp to take -I. option by itself, instead of gosh.)

You see C files are generated in the current directory, while 
SCI files are on the side of each file under the subdirectory.

Now you can compile those C files into a single DSO:

   $ gauche-package compile foo foo.c foo--boo.c foo--woo.c
   $ ls
   foo/        foo--boo.o  foo--woo.o  foo.o    foo.scm
   foo--boo.c  foo--woo.c  foo.c       foo.sci  foo.so

You still have an SCI file per each Scheme source, but there's only
one DSO and that can be loaded at once.


But I don't want a bunch of SCI files!
======================================

The reason we have one *.sci file per one *.scm file is to guarantee
the consitent behavior between source form and precompiled form.
With keeping the interface file in the same relative path as the source,
we can guarantee the library user can say not only (use foo),
but also (use foo.boo) and (use foo.woo).  The latter two still
work, since we have foo/boo.sci and foo/woo.sci, with appropriate
initialization in it.

However, if you have hundreds of source files and you *know*
the external user will only use the toplevel module and not
directly 'use' the submodules, then you can consolidate those
SCI files into single one.  Just give --single-interface
option to precomp.

   $ gosh -I. precomp -e --single-interface foo.scm foo/util1.scm foo/util2.scm
   $ ls
   foo/  foo--boo.c  foo--woo.c  foo.c  foo.sci  foo.scm
   $ ls foo
   boo.scm  woo.scm

Having this in foo.sci, you only need it and foo.so for the library
to work:

   $ gosh -I. -fload-verbose
   ;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm...
   ;;  Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm...
   ;;  Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm...
   gosh> (use foo)
   ;;Loading ./foo.sci...
   ;;  Dynamically Loading ./foo.so...
