and defining new words. The \texttt{scratchpad} vocabulary is the default vocabulary for new
word definitions.
-At the interactive listener, the default search path contains many more vocabularies. Details on the default search path and parser invocation are found in \ref{parsing-quotations}.
+At the interactive listener, the default search path contains many more vocabularies. Details on the default search path and parser invocation are found in \ref{parser-chapter}.
\wordtable{
\vocabulary{syntax}
\parsingword{USE:}{USE: \emph{vocabulary}}
}
Sort a list of words by name.
-\section{Vocabularies}
+\section{Vocabularies}\label{vocabularies}
\wordtable{
\vocabulary{words}
\symbolword{vocabularies}
\item A ratio; see \ref{ratio-literals}
\item A float; see \ref{float-literals}
\end{itemize}
-In particular, complex numbers are parsed by the \verb|#{| and \verb|}#| parsing words, not by the number parser. To parse complex number literals, use the \texttt{parse} word (\ref{parsing-quotations}).
+In particular, complex numbers are parsed by the \verb|#{| and \verb|}#| parsing words, not by the number parser. To parse complex number literals, use the \texttt{parse} word (\ref{parser-chapter}).
\wordtable{
\vocabulary{math}
\genericword{base>}{base>~( string base -- integer )}
}
If set to \verb|t|, strings longer than the margin are truncated. Otherwise, strings are printed fully, regardless of length. The default is \verb|f|.
-\chapter{The parser}
+\chapter{The parser}\label{parser-chapter}
This section concerns itself with reflective access and extension of the parser. The parser algorithm and standard syntax is described in \ref{syntax}. Before the parser proper is documented, we draw attention to a set of words for parsing numbers. They are called by the parser, and are useful in their own right.
\verb|"| &\verb|"|
\end{tabular}
-\chapter{Alien interface}
+\part{The implementation}
-Factor's alien inteface provides a means of directly calling native libraries written in C and other languages. There are no
-wrappers to write, other than having to specify the return type and parameter types for
-the functions you wish to call.
+\chapter{Development tools}
-\section{Loading native libraries}
+\section{Command line usage}
-A native library must be made available to Factor under a logical name before use. This is done via command line parameters, or the \verb|add-library| word.
+On startup, Factor reads the \texttt{.factor-rc} file from your home directory. You can put
+any quick definitions you want available at the listener there. To avoid loading this
+file, pass the \texttt{-no-user-init} command line switch. Another way to have a set of definitions available at all times is to save a custom image (see \ref{images}).
-The following two command line parameters can be specified for each library to load; the second parameter is optional.
-\begin{description}
-\item[\texttt{-libraries:\emph{logical}:name=\emph{name}}] associates a logical name with a system-specific native library name,
-\item[\texttt{-libraries:\emph{logical}:abi=\emph{type}}] specifies the calling convention to use; \verb|type| is either \verb|cdecl| or \verb|stdcall|. If not specified, the default is \verb|cdecl|. On Unix, all libraries follow the \verb|cdecl| convention. On Windows, most libraries (but not all) follow \verb|stdcall|.
-\end{description}
+\section{The listener}\label{listener}
-For example:
+The listener reads Factor code from the terminal and executes it. The listener is a piece of Factor code, like any other; however, it helps to think of it as the primary interface to the Factor system. You can try the classical first program:
\begin{alltt}
-\textbf{\$} ./f factor.image -libraries:sdl:name=libSDL-1.2.so
+ "Hello, world." print
+\textbf{Hello, world.}
+\end{alltt}
+Multi-line phrases are supported:
+\begin{alltt}
+ [ 1 2 3 ] [
+.
+] each
+\textbf{1
+2
+3}
\end{alltt}
+The listener knows when to expect more input by looking at the height of the
+stack. Parsing words such as \texttt{[} and \texttt{:} leave elements on the parser
+stack; these elements are popped by \texttt{]} and \texttt{;}.
+
+Often it is useful to look at the state of the stacks in the listener.
-Another option is to add libraries while Factor is running.
\wordtable{
-\vocabulary{alien}
-\ordinaryword{add-library}{add-library ( library name abi -- )}
+\vocabulary{prettyprint}
+\ordinaryword{.s}{.s ( -- )}
+\ordinaryword{.r}{.r ( -- )}
}
-Adds a logical library named \verb|library|. The underlying shared library name is \verb|name|, and the calling convention is \verb|abi| and must be either \verb|"cdecl"| or \verb|"stdcall"|.
+To see the contents of the data or return stack, use the \texttt{.s} and \texttt{.r} words.
+Each stack is printed with each element on its own line; the top of the stack is the first element printed.
-For example:
+\wordtable{
+\vocabulary{words}
+\ordinaryword{watch}{watch ( word -- )}
+}
+To print the stack automatically when a word is entered and exited, use \verb|watch|. It modifies the word definition to act accordingly:
\begin{alltt}
- "kernel32" "kernel32.dll" "stdcall" add-library
+ : squared dup * ;
+ \bs squared watch
+ 5 squared
+\textbf{===> Entering: squared
+5
+===> Leaving: squared
+25}
\end{alltt}
-The next word is used in the implementation of the alien interface, and it can also be used
-interactively to test if a library can be loaded.
+This modifies the word definition (as you can verify using \verb|see|); to restore the original definition, reload the word from its source file.
\wordtable{
-\vocabulary{alien}
-\ordinaryword{load-library}{load-library ( library -- dll )}
+\vocabulary{words}
+\ordinaryword{reload}{reload ( word -- )}
}
-Attempts to load the library with the given logical name, and outputs a DLL handle. If the library is already loaded, the existing DLL is output.
-More will be said about DLL handles in \ref{alien-internals}.
-
-\section{Calling native functions}
+Reload the source file the word was originally loaded from. If the word was automatically-generated, or defined in the listener, an exception is thrown.
-Native function bindings are established using a pair of parsing words.
+\section{Looking at word definitions}
\wordtable{
-\vocabulary{alien}
-\parsingword{LIBARARY:}{LIBARARY:~\emph{name}}
+\vocabulary{words}
+\ordinaryword{vocabs}{vocabs ( -- list )}
}
-Specifies the logical name of the C library in which the following functions are found.
+Entering \texttt{vocabs .}~in the listener produces a list of all existing vocabularies:
+
+\begin{alltt}
+ vocabs .
+\textbf{[ "alien" "ansi" "assembler" "browser-responder"
+"command-line" "compiler" "cont-responder" "errors"
+"file-responder" "files" "gadgets" "generic"
+"hashtables" "html" "httpd" "httpd-responder" "image"
+"inference" "interpreter" "io-internals" "jedit"
+"kernel" "kernel-internals" "line-editor" "listener"
+"lists" "logging" "math" "math-internals" "memory"
+"namespaces" "parser" "prettyprint" "profiler"
+"quit-responder" "random" "resource-responder"
+"scratchpad" "sdl" "shells" "stdio" "streams"
+"strings" "syntax" "telnetd" "test" "test-responder"
+"threads" "unparser" "url-encoding" "vectors" "words" ]}
+\end{alltt}
\wordtable{
-\vocabulary{alien}
-\parsingword{FUNCTION:}{FUNCTION:~\emph{returns} \emph{name} ( \emph{type} \emph{name}, \emph{...} )}
+\vocabulary{words}
+\ordinaryword{words}{words ( vocabulary -- list )}
}
-Defines a new word \verb|name| that calls the C function with the same name, found in the library given by the most recent \verb|LIBRARY:| declaration.
-
-The \verb|return| value names a C type, or \verb|void| if no return value is expected.
-Parameters are given by consecutive type/name pairs, where the type is again a C type, and the name is for documentation purposes. C types are documented in \ref{aliens}.
+You can use \texttt{words .}~to list the words inside a given vocabulary:
-The word generated by \verb|FUNCTION:| must be compiled before use \ref{compiler}). Executing it without compiling will throw an exception.
+\begin{alltt}
+ "namespaces" words .
+\textbf{[ (get) , <namespace> >n append, bind change cons@
+dec extend get global inc init-namespaces list-buffer
+literal, make-list make-rlist make-rstring make-string
+make-vector n> namespace namestack nest off on put set
+set-global set-namestack unique, unique@ with-scope ]}
+\end{alltt}
-For example, suppose you have a \verb|foo| library exporting the following function:
-\begin{verbatim}
-void the_answer(char* question, int value) {
- printf("The answer to %s is %d.\n",question,value);
+\wordtable{
+\vocabulary{generic}
+\ordinaryword{classes}{classes ( -- list )}
}
-\end{verbatim}
-You can define a word for invoking it:
-\begin{verbatim}
-LIBRARY: foo
-FUNCTION: the_answer ( char* question, int value ) ;
-\end{verbatim}
-Now, after being compiled, the word can be executed with two parameters on the stack:
+You can use \texttt{classes .}~to output a list of classes in the system:
+
\begin{alltt}
- \bs the-answer compile
-\textbf{Compiling the-answer}
- "the question" 42 the-answer
-\textbf{The answer to the question is 42.}
+ classes .
+\textbf{[ alien alien-error byte-array displaced-alien
+dll ansi-stream disp-only displaced indirect operand
+register absolute absolute-16/16 relative relative-bitfld
+item kernel-error no-method border checkbox dialog editor
+ellipse etched-rect frame gadget hand hollow-ellipse
+hollow-rect label line menu pane pile plain-ellipse
+plain-rect rectangle roll-rect scroller shelf slider
+stack tile viewport world 2generic arrayed builtin
+complement generic null object predicate tuple
+tuple-class union hashtable html-stream class-tie
+computed inference-error inference-warning literal
+literal-tie value buffer port jedit-stream boolean
+general-t array cons general-list list bignum complex
+fixnum float integer number ratio rational real
+parse-error potential-float potential-ratio
+button-down-event button-up-event joy-axis-event
+joy-ball-event joy-button-down-event joy-button-up-event
+joy-hat-event key-down-event key-up-event motion-event
+quit-event resize-event user-event sequence stdio-stream
+client-stream fd-stream null-stream server string-output
+wrapper-stream LETTER blank digit letter printable sbuf
+string text POSTPONE: f POSTPONE: t vector compound
+primitive symbol undefined word ]}
\end{alltt}
-Note that the parentheses and commas are only syntax sugar; the following two definitions are equivalent, although the former is slightly more readable:
-\begin{verbatim}
-FUNCTION: void glHint ( GLenum target, GLenum mode ) ;
-FUNCTION: void glHint GLenum target GLenum mode ;
-\end{verbatim}
-
-\section{Alien objects}\label{aliens}
+\wordtable{
+\vocabulary{prettyprint}
+\ordinaryword{see}{see ( word -- )}
+}
+You can look at the definition of any word, including library words, using \texttt{see}. Keep in mind you might have to \texttt{USE:} the vocabulary first.
-\glossary{
-name=alien,
-description={an instance of the \verb|alien| class, holding a pointer to native memory outside the Factor heap}}
-The alien interface can work with an assortment of native data types:
-\begin{itemize}
-\item integer and floating point values
-\item null-terminated strings
-\item structures (\ref{alien-structs})
-\item unions (\ref{alien-unions})
-\end{itemize}
-Table \ref{c-types} lists the built-in return value and parameter types. The sizes are given for a 32-bit system. Native numbers and strings are handled in a straight-forward way. Pointers are a bit more complicated, and are wrapped inside alien objects on the Factor side.
+\begin{alltt}
+ USE: httpd
+ \bs httpd-connection see
+\textbf{IN: httpd
+: httpd-connection ( socket -- )
+ "http-server" get accept [
+ httpd-client
+ ] in-thread drop ;}
+\end{alltt}
-\begin{table}
-\caption{\label{c-types}Supported native types}
-\begin{tabular}{l|l|l}
-Name&Size&Representation\\
-\hline
-\texttt{char} &1& Signed integer\\
-\texttt{uchar} &1& Unsigned integer\\
-\texttt{short} &2& Signed integer\\
-\texttt{ushort} &2& Unsigned integer\\
-\texttt{int} &4& Signed integer\\
-\texttt{uint} &4& Unsigned integer\\
-\texttt{long} &4& Signed integer\\
-\texttt{ulong} &4& Unsigned integer\\
-\texttt{longlong} &8& Signed integer\\
-\texttt{ulonglong} &8& Unsigned integer\\
-\texttt{float} &4& Single-precision float\\
-\texttt{double} &8& Double-precision float\\
-\texttt{char*} &4& Pointer to null-terminated byte string\\
-\texttt{ushort*} &4& Pointer to null-terminated UTF16 string\\
-\texttt{void*} &4& Generic pointer
-\end{tabular}
-\end{table}
+If you \texttt{see} a generic word, all methods defined on the generic word are shown. If you see a class word, all methods specializing on the class are shown:
+\begin{alltt}
+ \bs list see
+\textbf{PREDICATE: general-list list
+ dup [
+ last* cdr
+ ] when not ;
+IN: gadgets
+M: list custom-sheet
+ [
+ length count
+ ] keep zip alist>sheet "Elements:" <titled> ;
+IN: prettyprint
+M: list prettyprint*
+ [
+ [
+ POSTPONE: [
+ ] car swap [
+ POSTPONE: ]
+ ] car prettyprint-sequence
+ ] check-recursion ;}
+\end{alltt}
-A facility similar to C's \verb|typedef| type aliasing is provided. It can help with readability, as well as ease of development of library bindings.
+The \texttt{see} word shows a reconstruction of the source code, not the original source code. So in particular, formatting and some comments are lost.
\wordtable{
-\vocabulary{alien}
-\parsingword{TYPEDEF:}{TYPEDEF:~\emph{old} \emph{new}}
+\vocabulary{prettyprint}
+\ordinaryword{apropos}{apropos ( string -- )}
}
-Defines a C type named \verb|new| that is identical to \verb|old|, along with a pointer type \verb|new*| that is identical to \verb|old*|.
+The \texttt{apropos} word is handy when searching for related words. It lists all words
+whose names contain a given string. The \texttt{apropos} word is also useful when you know the exact name of a word, but are unsure what vocabulary it is in. For example, if you're looking for ways to iterate over various collections, you can do an apropos search for \texttt{each}:
-\wordtable{
-\vocabulary{alien}
-\ordinaryword{c-size}{c-size ( type -- n )}
-}
-Outputs the size of the given C type. This is just like the \verb|sizeof| operator in C.
-Many native functions expect you to specify sizes for input and output parameters, and
-this word can be used for that purpose.
+\begin{alltt}
+ "each" apropos
+\textbf{IN: gadgets each-gesture
+IN: gadgets each-parent
+IN: hashtables hash-each
+IN: hashtables hash-each-with
+IN: inference each-node
+IN: inference each-node-with
+IN: kernel-internals each-bucket
+IN: math each-bit
+IN: memory (each-object)
+IN: memory each-object
+IN: memory each-slot
+IN: sequences 2each
+IN: sequences each
+IN: sequences each-with
+IN: sequences tree-each
+IN: sequences tree-each-with
+IN: words each-word}
+\end{alltt}
\wordtable{
-\vocabulary{alien}
-\classword{alien}
+\vocabulary{words}
+\ordinaryword{usage}{usage ( word -- seq )}
}
-Pointers to native memory, including \verb|void*| and other types, are represented as objects of the \verb|alien| class.
+The \texttt{usage} word finds all words that refer to a given word.
+This word is helpful in two situations; the first is for learning -- a good way to learn a word is to see it used in context. The second is during refactoring -- if you change a word's stack effect, you must also update all words that call it. Usually you print the
+return value of \texttt{usage} using \texttt{.}:
+\begin{alltt}
+ \bs each-parent usage .
+\textbf{[ handle-gesture user-input ]}
+\end{alltt}
-\wordtable{
-\vocabulary{alien}
-\predword{alien?}
-}
-Tests if the object at the top of the stack is an alien pointer.
+\section{The walker}
-\subsection{Structures}\label{alien-structs}
+The walker lets you step through the execution of a qotation. When a compound definition is reached, you can either keep walking inside the definition, or execute it in one step. The stacks can be inspected at each stage.
-One way to think of a C-style \verb|struct| is that it abstracts reading and writing field values stored at a range of memory given a pointer, by associating a type and offset with each field. This is the view taken by the alien interface, where defining a C structure creates a set of words for reading and writing fields of various types, offset from a base pointer given by an alien object.
+There are two ways to use the walker. First of all, you can call the \texttt{walk} word explicitly, giving it a quotation:
-\wordtable{
-\vocabulary{alien}
-\parsingword{BEGIN-STRUCT:}{BEGIN-STRUCT: \emph{name}}
-}
-Begins reading a C structure definition.
-\wordtable{
-\vocabulary{alien}
-\parsingword{FIELD:}{FIELD: \emph{type} \emph{name}}
-}
-Adds a field to the structure. The \verb|type| token identifies a C type, and \verb|name| gives a name to the field. A pair of words is defined, where \verb|structure| and \verb|field| are names, respectively:
\begin{alltt}
-\emph{structure}-\emph{field} ( alien -- value )
-set-\emph{structure}-\emph{field} ( value alien -- )
+ [ [ 10 [ dup , ] repeat ] [ ] make ] walk
+\textbf{\&s \&r show stepper stacks.
+\&get ( var -- value ) inspects the stepper namestack.
+step -- single step over
+into -- single step into
+continue -- continue execution
+bye -- exit single-stepper
+[ [ 10 [ dup , ] repeat ] make-list ]
+walk}
\end{alltt}
-\wordtable{
-\vocabulary{alien}
-\parsingword{END-STRUCT}{END-STRUCT}
-}
-Ends a structure definition.
+As you can see, the walker prints a brief help message, then the currently executing quotation. It changes the listener prompt to \texttt{walk}, to remind you that there is a suspended continuation.
-Defining a structure adds two new C types, where \verb|name| is the name of the structure:
-\begin{description}
-\item[\texttt{\emph{name}}] the type of the structure itself; structure and union definitions can define members to be of this type.
-\item[\texttt{\emph{name}*}] the type of a pointer to the structure; this type can be used with return values and parameters, and in fact it is an alias for \texttt{void*}.
-\end{description}
-Additionally, the following two words are defined:
-\begin{description}
-\item[\texttt{<\emph{name}> ( -- byte-array )}] allocates a byte array large enough to hold the structure in the Factor heap. The field accessor words can then be used to work with this byte array. This feature allows calling native functions that expect pointers to caller-allocated structures\footnote{
-There is an important restriction, however; the function must not retain the pointer in a global variable after it returns. Since the structure is allocated in the Factor heap, the garbage collector is free to move it between native function calls. If this behavior is undesirable, memory can be managed manually instead (\ref{malloc}).}.
-\item[\texttt{\emph{name}-nth ( n alien -- alien )}] given a pointer and index into an array of structures, returns a pointer to the structure at that index.
-\end{description}
-
-Here is an example of a structure with various fields:
-\begin{verbatim}
-BEGIN-STRUCT: surface
- FIELD: uint flags
- FIELD: format* format
- FIELD: int w
- FIELD: int h
- FIELD: ushort pitch
- FIELD: void* pixels
- FIELD: int offset
- FIELD: void* hwdata
- FIELD: short clip-x
- FIELD: short clip-y
- FIELD: ushort clip-w
- FIELD: ushort clip-h
- FIELD: uint unused1
- FIELD: uint locked
- FIELD: int map
- FIELD: uint format_version
- FIELD: int refcount
-END-STRUCT
-\end{verbatim}
-
-\subsection{Unions}\label{alien-unions}
-
-A C-style \verb|union| type allocates enough space for its largest member. In the alien interface, unions are used to allocate byte arrays in the Factor heap that may hold any one of the union's members.
+The first element of the quotation shown is the next object to be evaluated. If it is a literal, both \texttt{step} and \texttt{into} have the effect of pushing it on the walker data stack. If it is a compound definition, then \texttt{into} will recurse the walker into the compound definition; otherwise, the word executes in one step.
-\wordtable{
-\vocabulary{alien}
-\parsingword{BEGIN-STRUCT:}{BEGIN-STRUCT: \emph{name}}
-}
-Begins reading a C union definition.
+The \texttt{\&r} word shows the walker return stack, which is laid out just like the primary interpreter's return stack. In fact, a good way to understand how Factor's return stack works is to play with the walker.
-\wordtable{
-\vocabulary{alien}
-\parsingword{MEMBER:}{MEMBER: \emph{type}}
-}
-Adds a member type to the union.
+Note that the walker does not automatically stop when the quotation originally given finishes executing; it just keeps on walking up the return stack, and even lets you step through the listener's code. You can invoke \texttt{continue} or \texttt{bye} to terminate the walker.
-\wordtable{
-\vocabulary{alien}
-\parsingword{END-UNION}{END-UNION}
-}
-Ends a union definition.
+While the walker can be invoked explicitly using the \texttt{walk} word, sometimes it is more convenient to \emph{annotate} a word such that the walker is invoked automatically when the word is called. This can be done using the \texttt{break} word:
-Unions define C types and words analogous to those for structures; see \ref{alien-structs}.
+\begin{alltt}
+ \bs layout* break
+\end{alltt}
-Here is an example:
-\begin{verbatim}
-BEGIN-UNION: event
- MEMBER: event
- MEMBER: active-event
- MEMBER: keyboard-event
- MEMBER: motion-event
- MEMBER: button-event
-END-UNION
-\end{verbatim}
+Now, when some piece of code calls \texttt{layout*}, the walker will open, and you will be able to step through execution and see exactly what's going on. An important point to keep in mind is that when the walker is invoked in this manner, \texttt{bye} will not have the desired effect; execution will continue, but the data stack will be inconsistent, and an error will most likely be raised a short time later. Always use \texttt{continue} to resume execution after a break.
-\subsection{Enumerations}
+You can undo the effect of \texttt{break} by reloading the original source file containing the word definition in question.
-A C-style \verb|enum| type defines a set of integer constants. The alien interface lets you define a set of words that push integers on the stack in much the same way as you would in C. While these words can be used for any purpose, using them outside of interfacing with C is discouraged.
+\section{Unit testing}
+Unit tests are very easy to write, and are a great debugging aid in any dynamic language.
+It is highly recommended that you write tests before or during development of any piece of Factor code. This section will not describe unit testing methodology; it only introduces Factor's unit test framework.
\wordtable{
-\vocabulary{alien}
-\parsingword{BEGIN-ENUM:}{BEGIN-ENUM \emph{start}}
+\vocabulary{test}
+\ordinaryword{unit-test}{unit-test ( expect quot -- )}
}
-Begins an enumeration that numbers constants starting from \verb|start|.
+Asserts that executing the quotation with an empty stack produces the expected stack. If the assertion fails, an exception is thrown. Here is an example unit test:
-\wordtable{
-\vocabulary{alien}
-\parsingword{ENUM:}{ENUM: \emph{name}}
-}
-Defines a compound word \verb|name| that pushes a integer. The integer's value is incremented each time \verb|ENUM:| defines a new word.
+\begin{verbatim}
+[ "Hello, crazy world" ]
+[ "Hello, " "crazy " "world" append3 ] unit-test
+\end{verbatim}
\wordtable{
-\vocabulary{alien}
-\parsingword{END-ENUM}{END-ENUM}
+\vocabulary{test}
+\ordinaryword{unit-test-fails}{unit-test-fails ( quot -- )}
}
-Ends an enumeration.
+Asserts that executing the quotation throws an exception. If the quotation returns normally, an exception is thrown. For example, the following test should pass, since calling \verb|nth| with a negative index should fail:
-Here is an example:
-\begin{verbatim}
-BEGIN-ENUM: 0
- ENUM: monday
- ENUM: tuesday
- ENUM: wednesday
- ENUM: thursday
- ENUM: friday
- ENUM: saturday
- ENUM: sunday
-END-ENUM
-\end{verbatim}
-This is in fact functionally equivalent to the following code:
\begin{verbatim}
-: monday 0 ;
-: tuesday 1 ;
-: wednesday 2 ;
-: thursday 3 ;
-: friday 4 ;
-: saturday 5 ;
-: sunday 6 ;
+[ -3 { } nth ] unit-test-fails
\end{verbatim}
-\section{Low-level interface}\label{alien-internals}
-
-The alien interface is built on top of a handful of primitives. Sometimes, it is
-useful to call these primitives directly for debugging purposes.
+\section{Timing code}
+Factor provides two facilities for timing code; a way to time the execution of a particular quotation, and a rudimentary profiler.
\wordtable{
-\vocabulary{alien}
-\classword{dll}
-}
-Instances of this class are handles to native libraries.
-\wordtable{
-\vocabulary{alien}
-\ordinaryword{dlopen}{dlopen ( path -- dll )}
-}
-Opens the specified native library and returns a DLL object. The input parameter is the
-name of a native library file,
-\emph{not} a logical library name.
-\wordtable{
-\vocabulary{alien}
-\ordinaryword{dlsym}{dlsym ( symbol dll -- address )}
-}
-Looks up a named symbol in a native library, and outputs it address. If the \verb|dll| is \verb|f|, the lookup is performed in the runtime executable itself.
-\wordtable{
-\vocabulary{alien}
-\ordinaryword{dlclose}{dlclose ( dll -- )}
+\vocabulary{test}
+\ordinaryword{time}{time ( quot -- )}
}
-Closes a native library and frees associated native resources.
+Calls the quotation, and reports the time taken. For example:
-\wordtable{
-\vocabulary{alien}
-\ordinaryword{alien-address}{alien-address ( alien -- address )}
-}
-Outputs the address of an alien, as an integer.
+\begin{alltt}
+ [ 1000000 [ f f cons drop ] repeat ] time
+\textbf{515 ms run / 11 ms GC time}
+\end{alltt}
\wordtable{
-\vocabulary{alien}
-\ordinaryword{<alien>}{<alien> ( address -- alien )}
+\vocabulary{words}
+\ordinaryword{profile}{profile ( word -- )}
}
-Creates an alien pointing to the specified address.
+Marks the word for profiling. This modifies the word definition so that when the word executes, the execution time of the word is added to a global variable with the word's name. For example:
+\begin{alltt}
+ : foo 100 sleep ;
+ \bs foo profile
+ 10 [ drop foo ] each
+ \bs foo get .
+\textbf{1000}
+\end{alltt}
-\wordtable{
-\vocabulary{alien}
-\ordinaryword{<displaced-alien>}{<displaced-alien> ( offset alien -- alien )}
-}
-Outputs an alien pointing at an offset from the base pointer of the input alien. Displaced aliens are used to access nested structures and native arrays.
+\section{Exploring memory usage}
-\wordtable{
-\vocabulary{alien}
-\ordinaryword{alien-signed-cell}{alien-signed-cell ( alien offset -- n )}
-\ordinaryword{set-alien-signed-cell}{set-alien-signed-cell ( n alien offset -- )}
-\ordinaryword{alien-unsigned-cell}{alien-unsigned-cell ( alien offset -- n )}
-\ordinaryword{set-alien-unsigned-cell}{set-alien-unsigned-cell( n alien offset -- )}
-\ordinaryword{alien-signed-8}{alien-signed-8 ( alien offset -- n )}
-\ordinaryword{set-alien-signed-8}{set-alien-signed-8 ( n alien offset -- )}
-\ordinaryword{alien-unsigned-8}{alien-unsigned-8 ( alien offset -- n )}
-\ordinaryword{set-alien-unsigned-8}{set-alien-unsigned-8 ( n alien offset -- )}
-\ordinaryword{alien-signed-4}{alien-signed-4 ( alien offset -- n )}
-\ordinaryword{set-alien-signed-4}{set-alien-signed-4 ( n alien offset -- )}
-\ordinaryword{alien-unsigned-4}{alien-unsigned-4 ( alien offset -- n )}
-\ordinaryword{set-alien-unsigned-4}{set-alien-unsigned-4 ( n alien offset -- )}
-\ordinaryword{alien-signed-2}{alien-signed-2 ( alien offset -- n )}
-\ordinaryword{set-alien-signed-2}{set-alien-signed-2 ( n alien offset -- )}
-\ordinaryword{alien-unsigned-2}{alien-unsigned-2 ( alien offset -- n )}
-\ordinaryword{set-alien-unsigned-2}{set-alien-unsigned-2 ( n alien offset -- )}
-\ordinaryword{alien-signed-1}{alien-signed-1 ( alien offset -- n )}
-\ordinaryword{set-alien-signed-1}{set-alien-signed-1 ( n alien offset -- )}
-\ordinaryword{alien-unsigned-1}{alien-unsigned-1 ( alien offset -- n )}
-\ordinaryword{set-alien-unsigned-1}{set-alien-unsigned-1 ( n alien offset -- )}
-\ordinaryword{alien-value-string}{alien-value-string ( alien offset -- string )}
-}
-These primitives read and write native memory. They can be given an alien, displaced alien, or byte array. No bounds checking of any kind is performed.
+Factor supports heap introspection. You can find all objects in the heap that match a certain predicate using the \texttt{instances} word. For example, if you suspect a resource leak, you can find all I/O ports as follows:
-\section{Manual memory management}\label{malloc}
+\begin{alltt}
+ USE: io-internals
+ [ port? ] instances .
+\textbf{[ \#<port @ 805466443> \#<port @ 805466499> ]}
+\end{alltt}
-If for whatever reason Factor's memory management is unsuitable for a certain task, you can
-directly call the standard C memory management routines. These words are very raw and deal with addresses directly, and of course it is easy to corrupt memory or crash the runtime
-this way.
-\wordtable{
-\vocabulary{kernel-internals}
-\ordinaryword{malloc}{malloc ( size -- address )}
-}
-Allocate a block of size \verb|size| and output a pointer to it.
-\wordtable{
-\vocabulary{kernel-internals}
-\ordinaryword{realloc}{realloc ( address size -- address )}
-}
-Resize a block previously allocated with \verb|malloc|.
-\wordtable{
-\vocabulary{kernel-internals}
-\ordinaryword{free}{free ( address -- )}
-}
-Deallocate a block previously allocated with \verb|malloc|.
+The \texttt{references} word finds all objects that refer to a given object:
-\part{Development tools}
+\begin{alltt}
+ [ float? ] instances car references .
+\textbf{[ \#<array @ 805542171> [ -1.0 0.0 / ] ]}
+\end{alltt}
-Factor supports interactive development in a live environment. Instead of working with
-static executable files and restarting your application after each change, you can
-incrementally make changes to your application and test them immediately. If you
-notice an undesirable behavior, Factor's powerful reflection features will aid in
-pinpointing the error.
+You can print a memory usage summary with \texttt{room.}:
-If you are used to a statically typed language, you might find Factor's tendency to only fail at runtime hard to work with at first. However, the interactive development tools outlined in this part allow a much quicker turn-around time for testing changes. Also, write unit tests -- unit testing is a great way to ensure that old bugs do not re-appear once they've been fixed.
+\begin{alltt}
+ room.
+\textbf{Data space: 16384 KB total 2530 KB used 13853 KB free
+Code space: 16384 KB total 490 KB used 15893 KB free}
+\end{alltt}
-\chapter{System organization}
+And finally, a detailed memory allocation breakdown by type with \texttt{heap-stats.}:
-\section{The listener}\label{listener}
+\begin{alltt}
+ heap-stats.
+\textbf{bignum: 312 bytes, 17 instances
+cons: 850376 bytes, 106297 instances
+float: 112 bytes, 7 instances
+t: 8 bytes, 1 instances
+array: 202064 bytes, 3756 instances
+hashtable: 54912 bytes, 3432 instances
+vector: 5184 bytes, 324 instances
+string: 391024 bytes, 7056 instances
+sbuf: 64 bytes, 4 instances
+port: 112 bytes, 2 instances
+word: 96960 bytes, 3030 instances
+tuple: 688 bytes, 22 instances}
+\end{alltt}
+
+\chapter{Images}\label{images}
-Factor is an \emph{image-based environment}. When you compiled Factor, you also generated a file named \texttt{factor.image}. I will have more to say about images later, but for now it suffices to understand that to start Factor, you must pass the image file name on the command line:
+\glossary{name=image,
+description={the complete persistent state of a Factor system, dumped to a file}}
+
+An image is basically a dump of all objects in the heap. When Factor is started, an image name must be specified on the command line:
\begin{alltt}
-./f factor.image
+\textbf{\$} ./f factor.image
\textbf{Loading factor.image... relocating... done
-Factor 0.73 :: http://factor.sourceforge.net :: unix/x86
+Factor 0.77 :: http://factor.sourceforge.net :: linux/x86
(C) 2003, 2005 Slava Pestov, Chris Double,
-Mackenzie Straight
-ok}
+Mackenzie Straight}
\end{alltt}
-An \texttt{\textbf{ok}} prompt is printed after the initial banner, indicating the listener is ready to execute Factor phrases. The listener is a piece of Factor code, like any other; however, it helps to think of it as the primary interface to the Factor system. The listener reads Factor code and executes it. You can try the classical first program:
+The most-recently loaded image is stored in the \verb|"image"| global variable.
+The current image can be saved back to disk using one of two words.
+\wordtable{
+\vocabulary{memory}
+\ordinaryword{save}{save ( -- )}
+}
+Save the current image to disk. Defined as:
+\begin{verbatim}
+: save "image" get save-image ;
+\end{verbatim}
+\wordtable{
+\vocabulary{memory}
+\ordinaryword{save-image}{save-image ( name -- )}
+}
+Save the current image to a file with the specified name.
\begin{alltt}
- "Hello, world." print
-\textbf{Hello, world.}
+ "work.image" save-image
+\textbf{Saving work.image...}
\end{alltt}
-
-Multi-line phrases are supported; if there are unclosed brackets, the listener outputs \texttt{...} instead of the \texttt{ok} prompt, and the entire phrase is executed once all brackets are closed:
+When you save an image before exiting Factor, then start Factor again, everything will be almost as you left it. Try the following:
\begin{alltt}
- [ 1 2 3 ] [
-\textbf{...} .
-\textbf{...} ] each
-\textbf{1
-2
-3}
+\textbf{\$} ./f factor.image
+ "Learn Factor" "reminder" set
+ "factor.image" save-image
+\textbf{Saving factor.image...}
+ bye
+\textbf{\$} ./f factor.image
+ "reminder" get .
+\textbf{"Learn Factor"}
\end{alltt}
-The listener knows when to print a continuation prompt by looking at the height of the
-stack. Parsing words such as \texttt{[} and \texttt{:} leave elements on the parser
-stack; these elements are popped by \texttt{]} and \texttt{;}.
+This is what is meant by the image being an \emph{infinite session}. When you shut down and restart Factor, what happends is much closer to a Laptop's ``suspend'' mode, than a desktop computer being fully shut down.
-On startup, Factor reads the \texttt{.factor-rc} file from your home directory. You can put
-any quick definitions you want available at the listener there. To avoid loading this
-file, pass the \texttt{-no-user-init} command line switch. Another way to have a set of definitions available at all times is to save a custom image, as described in the next section.
+\section{The bootstrap process}
-\section{Source files}
+The Factor library source code cannot produce a new image from scratch, because
+of inherent circularity; critical components such as the parser and the generic
+word system are written in Factor itself.
+The Factor runtime, coded in C, knows nothing of the syntax of Factor
+source files, or even the organization of words into vocabularies.
-While it is possible to do all development in the listener and save your work in images, it is far more convenient to work with source files, at least until an in-image structure editor is developed.
+However, if \verb|save-image| was the only way one could save a new image, things
+could get out of hand; since \verb|save-image| performs a memory dump, there would be
+no way to change object formats, or perform major re-organizations of the system, and
+so on.
-By convention, Factor source files are saved with the \texttt{.factor} filename extension. They can be loaded into the image as follows:
+The bootstrap process builds a new image, from source, from inside another Factor
+image, in a reasonably clean and controlled manner. Bootstrap proceeds in two stages:
-\begin{alltt}
- "examples/numbers-game.factor" run-file
-\end{alltt}
+\begin{itemize}
+\item In the first stage, the currently-running Factor image (the \emph{host image}) runs the image generator to produce a bootstrap image (the \emph{target image}).
+\item In the second stage, the target image is run, and any final loading and initialization is performed.
+\end{itemize}
-In Factor, loading a source file replaces any existing definitions\footnote{But see \ref{compiler} for this is not true of compiled code.}. Each word definition remembers what source file it was loaded from (if any). To reload the source file associated with a definition, use the \texttt{reload} word:
+The first stage is performed by the following word.
-\begin{alltt}
- \bs draw reload
-\end{alltt}
+\wordtable{
+\vocabulary{image}
+\ordinaryword{make-image}{make-image ( name -- )}
+}
+Generates a new bootstrap image, and writes it to the file named \verb|name|.
-Word definitions also retain the line number where they are located in their original source file. This allows you to open a word definition in jEdit\footnote{\texttt{http://www.jedit.org}} for editing using the
-\texttt{jedit} word:
+The output format is determined by a pair of variables:
-\begin{alltt}
- \bs compile jedit
-\end{alltt}
+\wordtable{
+\vocabulary{image}
+\symbolword{big-endian}
+\symbolword{64-bits}
+}
+The combination of the four flags allows you to switch between the four image formats supported by Factor:
+\begin{itemize}
+\item Little endian, 32 bit (x86)
+\item Little endian, 64 bit (x86-64, Alpha)
+\item Big endian, 32 bit (PowerPC, ARM, Sparc)
+\item Big endian, 64 bit (PowerPC-64, UltraSparc)
+\end{itemize}
+
+Now we will look at the bootstrap process in more detail.
+
+First note that the \verb|vocabularies| variable can be shadowed in a new dynamic scope, forcing words like \verb|search| and \verb|create| to operate on the new set of vocabularies (see \ref{vocabularies}). Bootstrapping uses this to build the vocabularies for the target image.
-This word requires that a jEdit instance is already running.
+After initializing some internal objects, \verb|make-image| runs the file
+\verb|boot-stage1.factor|. This file then runs \verb|primitives.factor|, which sets up the initial target image vocabularies, consisting of a copy of the \verb|syntax| vocabulary from the host image, along with all primitive words defined in the runtime. The \verb|syntax| vocabulary is copied so that further source files can be parsed in.
-The \texttt{jedit} word will open word definitions from the Factor library once the full path of the Factor source tree is entered into the \texttt{"resource-path"} variable. One way to do this is to add a phrase like the following to your \texttt{.factor-rc}:
+Note that apart from words in the copied \verb|syntax| vocabulary, no words in the target image may be executed in the host image; doing so can lead to all kinds of problems, since the target image may be linked with a different set of primitives and an incompatible runtime structure than the host image.
+
+Bootstrapping proceeds to load various source files into the
+target vocabulary hash. Each file loaded must only refer to primitive
+words, and words loaded from previous files. So by reading through each
+file referenced by \verb|boot-stage1.factor|, you can see the entire construction of the Factor system from the bottom up.
+
+After the library has been loaded into the target image, there is still a problem; the \verb|syntax| vocabulary in the target image was copied from the host
+image, and not loaded from source. Loading the \verb|syntax| vocabulary for the target image is the very last task performed during bootstrap, since after this point, no more code may be loaded in (since it would invoke parsing words intended for the target image in the host image).
+
+Take a look at the start of any source file in \verb|library/syntax/|:
\begin{verbatim}
-"/home/slava/Factor/" "resource-path" set
+IN: !syntax
+USE: syntax
\end{verbatim}
-\section{Images}
+The source files defining parsing words are themselves parsed using the \verb|syntax| vocabulary, \emph{but add new definitions} to the \verb|!syntax| vocabulary. This allows bootstrap to replace the \verb|syntax| vocabulary with \verb|!syntax| as the very last step.
-The \texttt{factor.image} file is basically a dump of all objects in the heap. A new image can be saved as follows:
+Once all source files have been loaded into the target image, the target image is written to disk. This is done by tracing all objects and outputting them manually, without a dependency on their actual memory layout in the host image; this allows object formats to be changed easily.
-\begin{alltt}
- "work.image" save-image
-\textbf{Saving work.image...}
-\end{alltt}
+\chapter{Alien interface}
-When you save an image before exiting Factor, then start Factor again, everything will be almost as you left it. Try the following:
+Factor's alien inteface provides a means of directly calling native libraries written in C and other languages. There are no
+wrappers to write, other than having to specify the return type and parameter types for
+the functions you wish to call.
+
+\section{Loading native libraries}
+
+A native library must be made available to Factor under a logical name before use. This is done via command line parameters, or the \verb|add-library| word.
+The following two command line parameters can be specified for each library to load; the second parameter is optional.
+\begin{description}
+\item[\texttt{-libraries:\emph{logical}:name=\emph{name}}] associates a logical name with a system-specific native library name,
+\item[\texttt{-libraries:\emph{logical}:abi=\emph{type}}] specifies the calling convention to use; \verb|type| is either \verb|cdecl| or \verb|stdcall|. If not specified, the default is \verb|cdecl|. On Unix, all libraries follow the \verb|cdecl| convention. On Windows, most libraries (but not all) follow \verb|stdcall|.
+\end{description}
+
+For example:
\begin{alltt}
-./f factor.image
- "Learn Factor" "reminder" set
- "factor.image" save-image bye
-\textbf{Saving factor.image...}
+\textbf{\$} ./f factor.image -libraries:sdl:name=libSDL-1.2.so
\end{alltt}
-Factor will save the image and exit. Now start it again and see that the reminder is still there:
+Another option is to add libraries while Factor is running.
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{add-library}{add-library ( library name abi -- )}
+}
+Adds a logical library named \verb|library|. The underlying shared library name is \verb|name|, and the calling convention is \verb|abi| and must be either \verb|"cdecl"| or \verb|"stdcall"|.
+For example:
\begin{alltt}
-./f factor.image
- "reminder" get .
-\textbf{"Learn Factor"}
+ "kernel32" "kernel32.dll" "stdcall" add-library
\end{alltt}
+The next word is used in the implementation of the alien interface, and it can also be used
+interactively to test if a library can be loaded.
-This is what is meant by the image being an \emph{infinite session}. When you shut down and restart Factor, what happends is much closer to a Laptop's ``suspend'' mode, than a desktop computer being fully shut down.
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{load-library}{load-library ( library -- dll )}
+}
+Attempts to load the library with the given logical name, and outputs a DLL handle. If the library is already loaded, the existing DLL is output.
+More will be said about DLL handles in \ref{alien-internals}.
-\section{Looking at objects}
+\section{Calling native functions}
-Probably the most important debugging tool of them all is the \texttt{.} word. It prints the object at the top of the stack in a form that can be parsed by the Factor parser. A related word is \texttt{prettyprint}. It is identical to \texttt{.} except the output is more verbose; lists, vectors and hashtables are broken up into multiple lines and indented.
+Native function bindings are established using a pair of parsing words.
-\begin{alltt}
- [ [ \tto 1 \ttc \tto 2 \ttc ] dup car swap cdr ] .
-\textbf{[ [ \tto 1 \ttc \tto 2 \ttc ] dup car swap cdr ]}
-\end{alltt}
+\wordtable{
+\vocabulary{alien}
+\parsingword{LIBARARY:}{LIBARARY:~\emph{name}}
+}
+Specifies the logical name of the C library in which the following functions are found.
-Most objects print in a parsable form, but not all. One exceptions to this rule is objects with external state, such as I/O ports or aliens (pointers to native structures). Also, objects with circular or very deeply nested structure will not print in a fully parsable form, since the prettyprinter has a limit on maximum nesting. Here is an example -- a vector is created, that holds a list whose first element is the vector itself:
+\wordtable{
+\vocabulary{alien}
+\parsingword{FUNCTION:}{FUNCTION:~\emph{returns} \emph{name} ( \emph{type} \emph{name}, \emph{...} )}
+}
+Defines a new word \verb|name| that calls the C function with the same name, found in the library given by the most recent \verb|LIBRARY:| declaration.
-\begin{alltt}
- \tto \ttc [ unit 0 ] keep [ set-vector-nth ] keep .
-\textbf{\tto [ ... ] \ttc}
-\end{alltt}
+The \verb|return| value names a C type, or \verb|void| if no return value is expected.
+Parameters are given by consecutive type/name pairs, where the type is again a C type, and the name is for documentation purposes. C types are documented in \ref{aliens}.
-The prettyprinted form of a vector or list with many elements is not always readable. The \texttt{[.]} and \texttt{\tto.\ttc} words output a list or a vector, respectively, with each element on its own line. In fact, the stack printing words are defined in terms of \texttt{[.]} and \texttt{\tto.\ttc}:
+The word generated by \verb|FUNCTION:| must be compiled before use \ref{compiler}). Executing it without compiling will throw an exception.
+For example, suppose you have a \verb|foo| library exporting the following function:
\begin{verbatim}
-: .s datastack {.} ;
-: .r callstack {.} ;
+void the_answer(char* question, int value) {
+ printf("The answer to %s is %d.\n",question,value);
+}
\end{verbatim}
-
-Before we move on, one final set of output words comes is used to output integers in
-different numeric bases. The \texttt{.b} word prints an integer in binary, \texttt{.o} in octal, and \texttt{.h} in hexadecimal.
-
+You can define a word for invoking it:
+\begin{verbatim}
+LIBRARY: foo
+FUNCTION: the_answer ( char* question, int value ) ;
+\end{verbatim}
+Now, after being compiled, the word can be executed with two parameters on the stack:
\begin{alltt}
- 31337 .b
-\textbf{111101001101001}
- 31337 .o
-\textbf{75151}
- 31337 .h
-\textbf{7a69}
+ \bs the-answer compile
+\textbf{Compiling the-answer}
+ "the question" 42 the-answer
+\textbf{The answer to the question is 42.}
\end{alltt}
-\chapter{Word tools}
-
-\section{Exploring vocabularies}\label{exploring-vocabs}
+Note that the parentheses and commas are only syntax sugar; the following two definitions are equivalent, although the former is slightly more readable:
+\begin{verbatim}
+FUNCTION: void glHint ( GLenum target, GLenum mode ) ;
+FUNCTION: void glHint GLenum target GLenum mode ;
+\end{verbatim}
-Factor organizes code in a two-tier structure of vocabularies and words. A word is the smallest unit of code; it corresponds to a function or method in other languages. Vocabularies group related words together for easy browsing and tracking of source dependencies.
+\section{Alien objects}\label{aliens}
-Entering \texttt{vocabs .}~in the listener produces a list of all existing vocabularies:
+\glossary{
+name=alien,
+description={an instance of the \verb|alien| class, holding a pointer to native memory outside the Factor heap}}
+The alien interface can work with an assortment of native data types:
+\begin{itemize}
+\item integer and floating point values
+\item null-terminated strings
+\item structures (\ref{alien-structs})
+\item unions (\ref{alien-unions})
+\end{itemize}
+Table \ref{c-types} lists the built-in return value and parameter types. The sizes are given for a 32-bit system. Native numbers and strings are handled in a straight-forward way. Pointers are a bit more complicated, and are wrapped inside alien objects on the Factor side.
-\begin{alltt}
- vocabs .
-\textbf{[ "alien" "ansi" "assembler" "browser-responder"
-"command-line" "compiler" "cont-responder" "errors"
-"file-responder" "files" "gadgets" "generic"
-"hashtables" "html" "httpd" "httpd-responder" "image"
-"inference" "interpreter" "io-internals" "jedit"
-"kernel" "kernel-internals" "line-editor" "listener"
-"lists" "logging" "math" "math-internals" "memory"
-"namespaces" "parser" "prettyprint" "profiler"
-"quit-responder" "random" "resource-responder"
-"scratchpad" "sdl" "shells" "stdio" "streams"
-"strings" "syntax" "telnetd" "test" "test-responder"
-"threads" "unparser" "url-encoding" "vectors" "words" ]}
-\end{alltt}
+\begin{table}
+\caption{\label{c-types}Supported native types}
+\begin{tabular}{l|l|l}
+Name&Size&Representation\\
+\hline
+\texttt{char} &1& Signed integer\\
+\texttt{uchar} &1& Unsigned integer\\
+\texttt{short} &2& Signed integer\\
+\texttt{ushort} &2& Unsigned integer\\
+\texttt{int} &4& Signed integer\\
+\texttt{uint} &4& Unsigned integer\\
+\texttt{long} &4& Signed integer\\
+\texttt{ulong} &4& Unsigned integer\\
+\texttt{longlong} &8& Signed integer\\
+\texttt{ulonglong} &8& Unsigned integer\\
+\texttt{float} &4& Single-precision float\\
+\texttt{double} &8& Double-precision float\\
+\texttt{char*} &4& Pointer to null-terminated byte string\\
+\texttt{ushort*} &4& Pointer to null-terminated UTF16 string\\
+\texttt{void*} &4& Generic pointer
+\end{tabular}
+\end{table}
-As you can see, there are a lot of vocabularies! Now, you can use \texttt{words .}~to list the words inside a given vocabulary:
+A facility similar to C's \verb|typedef| type aliasing is provided. It can help with readability, as well as ease of development of library bindings.
-\begin{alltt}
- "namespaces" words .
-\textbf{[ (get) , <namespace> >n append, bind change cons@
-dec extend get global inc init-namespaces list-buffer
-literal, make-list make-rlist make-rstring make-string
-make-vector n> namespace namestack nest off on put set
-set-global set-namestack unique, unique@ with-scope ]}
-\end{alltt}
+\wordtable{
+\vocabulary{alien}
+\parsingword{TYPEDEF:}{TYPEDEF:~\emph{old} \emph{new}}
+}
+Defines a C type named \verb|new| that is identical to \verb|old|, along with a pointer type \verb|new*| that is identical to \verb|old*|.
-You can look at the definition of any word, including library words, using \texttt{see}. Keep in mind you might have to \texttt{USE:} the vocabulary first.
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{c-size}{c-size ( type -- n )}
+}
+Outputs the size of the given C type. This is just like the \verb|sizeof| operator in C.
+Many native functions expect you to specify sizes for input and output parameters, and
+this word can be used for that purpose.
-\begin{alltt}
- USE: httpd
- \bs httpd-connection see
-\textbf{IN: httpd : httpd-connection ( socket -- )
- "http-server" get accept [
- httpd-client
- ] in-thread drop ;}
-\end{alltt}
+\wordtable{
+\vocabulary{alien}
+\classword{alien}
+}
+Pointers to native memory, including \verb|void*| and other types, are represented as objects of the \verb|alien| class.
-The \texttt{see} word shows a reconstruction of the source code, not the original source code. So in particular, formatting and some comments are lost.
+\wordtable{
+\vocabulary{alien}
+\predword{alien?}
+}
+Tests if the object at the top of the stack is an alien pointer.
-\section{Cross-referencing words}
+\subsection{Structures}\label{alien-structs}
-The \texttt{apropos.} word is handy when searching for related words. It lists all words
-whose names contain a given string. The \texttt{apropos.} word is also useful when you know the exact name of a word, but are unsure what vocabulary it is in. For example, if you're looking for ways to iterate over various collections, you can do an apropos search for \texttt{map}:
+One way to think of a C-style \verb|struct| is that it abstracts reading and writing field values stored at a range of memory given a pointer, by associating a type and offset with each field. This is the view taken by the alien interface, where defining a C structure creates a set of words for reading and writing fields of various types, offset from a base pointer given by an alien object.
+\wordtable{
+\vocabulary{alien}
+\parsingword{BEGIN-STRUCT:}{BEGIN-STRUCT: \emph{name}}
+}
+Begins reading a C structure definition.
+\wordtable{
+\vocabulary{alien}
+\parsingword{FIELD:}{FIELD: \emph{type} \emph{name}}
+}
+Adds a field to the structure. The \verb|type| token identifies a C type, and \verb|name| gives a name to the field. A pair of words is defined, where \verb|structure| and \verb|field| are names, respectively:
\begin{alltt}
- "map" apropos.
-\textbf{IN: generic
-typemap
-IN: hashtables
-map>hash
-IN: sdl
-set-surface-map
-surface-map
-IN: sequences
-2map
-map
-map-with
-nmap}
+\emph{structure}-\emph{field} ( alien -- value )
+set-\emph{structure}-\emph{field} ( value alien -- )
\end{alltt}
-From the above output, you can see a few words to explore, such as \verb|map|, \verb|map-with|, and \verb|2map|.
+\wordtable{
+\vocabulary{alien}
+\parsingword{END-STRUCT}{END-STRUCT}
+}
+Ends a structure definition.
-The \texttt{usage} word finds all words that refer to a given word and pushes a list on the stack. This word is helpful in two situations; the first is for learning -- a good way to learn a word is to see it used in context. The second is during refactoring -- if you change a word's stack effect, you must also update all words that call it. Usually you print the
-return value of \texttt{usage} using \texttt{.}:
+Defining a structure adds two new C types, where \verb|name| is the name of the structure:
+\begin{description}
+\item[\texttt{\emph{name}}] the type of the structure itself; structure and union definitions can define members to be of this type.
+\item[\texttt{\emph{name}*}] the type of a pointer to the structure; this type can be used with return values and parameters, and in fact it is an alias for \texttt{void*}.
+\end{description}
+Additionally, the following two words are defined:
+\begin{description}
+\item[\texttt{<\emph{name}> ( -- byte-array )}] allocates a byte array large enough to hold the structure in the Factor heap. The field accessor words can then be used to work with this byte array. This feature allows calling native functions that expect pointers to caller-allocated structures\footnote{
+There is an important restriction, however; the function must not retain the pointer in a global variable after it returns. Since the structure is allocated in the Factor heap, the garbage collector is free to move it between native function calls. If this behavior is undesirable, memory can be managed manually instead (\ref{malloc}).}.
+\item[\texttt{\emph{name}-nth ( n alien -- alien )}] given a pointer and index into an array of structures, returns a pointer to the structure at that index.
+\end{description}
-\begin{alltt}
- \bs string-map usage .
-\textbf{schars>entities
-filter-null
-url-encode}
-\end{alltt}
+Here is an example of a structure with various fields:
+\begin{verbatim}
+BEGIN-STRUCT: surface
+ FIELD: uint flags
+ FIELD: format* format
+ FIELD: int w
+ FIELD: int h
+ FIELD: ushort pitch
+ FIELD: void* pixels
+ FIELD: int offset
+ FIELD: void* hwdata
+ FIELD: short clip-x
+ FIELD: short clip-y
+ FIELD: ushort clip-w
+ FIELD: ushort clip-h
+ FIELD: uint unused1
+ FIELD: uint locked
+ FIELD: int map
+ FIELD: uint format_version
+ FIELD: int refcount
+END-STRUCT
+\end{verbatim}
-Another useful word is \texttt{usages}. Unlike \texttt{usage}, it finds all usages, even
-indirect ones -- so if a word refers to another word that refers to the given word,
-both words will be in the output list.
+\subsection{Unions}\label{alien-unions}
-\section{Exploring classes}
+A C-style \verb|union| type allocates enough space for its largest member. In the alien interface, unions are used to allocate byte arrays in the Factor heap that may hold any one of the union's members.
-Factor supports object-oriented programming via generic words (\ref{generic}). Generic words are called
-like ordinary words, however they can have multiple definitions, one per class, and
-these definitions do not have to appear in the same source file. Such a definition is
-termed a \emph{method}, and the method is said to \emph{specialize} on a certain
-class. A class in the most
-general sense is just a set of objects. You can output a list of classes in the system
-with \texttt{classes .}:
+\wordtable{
+\vocabulary{alien}
+\parsingword{BEGIN-STRUCT:}{BEGIN-STRUCT: \emph{name}}
+}
+Begins reading a C union definition.
-\begin{alltt}
- classes.
-\textbf{[ alien alien-error byte-array displaced-alien
-dll ansi-stream disp-only displaced indirect operand
-register absolute absolute-16/16 relative relative-bitfld
-item kernel-error no-method border checkbox dialog editor
-ellipse etched-rect frame gadget hand hollow-ellipse
-hollow-rect label line menu pane pile plain-ellipse
-plain-rect rectangle roll-rect scroller shelf slider
-stack tile viewport world 2generic arrayed builtin
-complement generic null object predicate tuple
-tuple-class union hashtable html-stream class-tie
-computed inference-error inference-warning literal
-literal-tie value buffer port jedit-stream boolean
-general-t array cons general-list list bignum complex
-fixnum float integer number ratio rational real
-parse-error potential-float potential-ratio
-button-down-event button-up-event joy-axis-event
-joy-ball-event joy-button-down-event joy-button-up-event
-joy-hat-event key-down-event key-up-event motion-event
-quit-event resize-event user-event sequence stdio-stream
-client-stream fd-stream null-stream server string-output
-wrapper-stream LETTER blank digit letter printable sbuf
-string text POSTPONE: f POSTPONE: t vector compound
-primitive symbol undefined word ]}
-\end{alltt}
-
-If you \texttt{see} a generic word, all methods defined on the generic word are shown.
-Alternatively, you can use \texttt{methods.} to print all methods specializing on a
-given class:
-
-\begin{alltt}
- \bs list methods.
-\textbf{PREDICATE: general-list list
- dup [
- last* cdr
- ] when not ;
-IN: gadgets
-M: list custom-sheet
- [
- length count
- ] keep zip alist>sheet "Elements:" <titled> ;
-IN: prettyprint
-M: list prettyprint*
- [
- [
- POSTPONE: [
- ] car swap [
- POSTPONE: ]
- ] car prettyprint-sequence
- ] check-recursion ;}
-\end{alltt}
-
-\chapter{Debugging and optimizing}
-
-\section{Looking at stacks}
-
-To see the contents of the data or return stack, use the \texttt{.s} and \texttt{.r} words.
-Each stack is printed with each element on its own line; the top of the stack is the first element printed.
-
-\section{The debugger}
-
-If the execution of a phrase in the listener causes an error to be thrown, the error
-is printed and the stacks at the time of the error are saved. If you're spent any
-time with Factor at all, you are probably familiar with this type of message:
-
-\begin{alltt}
- [ 1 2 3 ] 4 append reverse
-\textbf{The generic word car does not have a suitable method for 4
-:s :r show stacks at time of error.
-:get ( var -- value ) inspects the error namestack.}
-\end{alltt}
-
-The words \texttt{:s} and \texttt{:r} behave like their counterparts that are prefixed with \texttt{.}, except they show the stacks as they were when the error was thrown.
-
-The return stack warrants some special attention. To successfully develop Factor, you will need to learn to understand how it works. Lets look at the first few lines of the return stack at the time of the above error:
-
-\begin{verbatim}
-[ swap cdr ]
-uncons
-[ r> tuck 2slip ]
-(each)
-[ swons ]
-[ each ]
-each
-\end{verbatim}
-
-You can see the sequence of calls leading up to the error was \texttt{each} calling \texttt{(each)} calling \texttt{uncons}. The error tells us that the \texttt{car} word is the one that failed. Now, you can stare at the stack dump, at notice that if the call to \texttt{car} was successful and execution returned to \texttt{(each)}, the quotation \texttt{[ r> tuck 2slip ]} would resume executing. The first word there, \texttt{r>}, would take the quotation \texttt{[ swons ]} and put it back on the data stack. After \texttt{(each)} returned, it would then continue executing the quotation \texttt{[ each ]}. So what is going on here is a recursive loop, \texttt{[ swons ] each}. If you look at the definition of \texttt{reverse}, you will see that this is exactly what is being done:
+\wordtable{
+\vocabulary{alien}
+\parsingword{MEMBER:}{MEMBER: \emph{type}}
+}
+Adds a member type to the union.
-\begin{verbatim}
-: reverse ( list -- list ) [ ] swap [ swons ] each ;
-\end{verbatim}
+\wordtable{
+\vocabulary{alien}
+\parsingword{END-UNION}{END-UNION}
+}
+Ends a union definition.
-So a list is being reversed, but at some stage, the \texttt{car} is taken of something that is not a number. Now, you can look at the data stack with \texttt{:s}:
+Unions define C types and words analogous to those for structures; see \ref{alien-structs}.
+Here is an example:
\begin{verbatim}
-<< no-method [ ] 4 car >>
-car
-4
-4
-[ 3 2 1 ]
+BEGIN-UNION: event
+ MEMBER: event
+ MEMBER: active-event
+ MEMBER: keyboard-event
+ MEMBER: motion-event
+ MEMBER: button-event
+END-UNION
\end{verbatim}
-So now, the mystery has been solved: as \texttt{reverse} iterates down the input value, it hits a cons cells whose \texttt{cdr} is not a list. Indeed, if you look at the value we are passing to \texttt{reverse}, you will see why:
-
-\begin{alltt}
- [ 1 2 3 ] 4 append .
-[[ 1 [[ 2 [[ 3 4 ]] ]] ]]
-\end{alltt}
-
-In the future, the debugger will be linked with the walker, documented below. Right now, the walker is a separate tool. Another caveat is that in compiled code, the return stack is not reconstructed if there is an error. Until this is fixed, you should only compile code once it is debugged. For more potential compiler pitfalls, see \ref{compiler}.
-
-\section{The walker}
-
-The walker lets you step through the execution of a qotation. When a compound definition is reached, you can either keep walking inside the definition, or execute it in one step. The stacks can be inspected at each stage.
-
-There are two ways to use the walker. First of all, you can call the \texttt{walk} word explicitly, giving it a quotation:
-
-\begin{alltt}
- [ [ 10 [ dup , ] repeat ] [ ] make ] walk
-\textbf{\&s \&r show stepper stacks.
-\&get ( var -- value ) inspects the stepper namestack.
-step -- single step over
-into -- single step into
-continue -- continue execution
-bye -- exit single-stepper
-[ [ 10 [ dup , ] repeat ] make-list ]
-walk}
-\end{alltt}
-
-As you can see, the walker prints a brief help message, then the currently executing quotation. It changes the listener prompt from \texttt{ok} to \texttt{walk}, to remind you that there is a suspended continuation.
-
-The first element of the quotation shown is the next object to be evaluated. If it is a literal, both \texttt{step} and \texttt{into} have the effect of pushing it on the walker data stack. If it is a compound definition, then \texttt{into} will recurse the walker into the compound definition; otherwise, the word executes in one step.
-
-The \texttt{\&r} word shows the walker return stack, which is laid out just like the primary interpreter's return stack. In fact, a good way to understand how Factor's return stack works is to play with the walker.
-
-Note that the walker does not automatically stop when the quotation originally given finishes executing; it just keeps on walking up the return stack, and even lets you step through the listener's code. You can invoke \texttt{continue} or \texttt{exit} to terminate the walker.
-
-While the walker can be invoked explicitly using the \texttt{walk} word, sometimes it is more convenient to \emph{annotate} a word such that the walker is invoked automatically when the word is called. This can be done using the \texttt{break} word:
-
-\begin{alltt}
- \bs layout* break
-\end{alltt}
-
-Now, when some piece of code calls \texttt{layout*}, the walker will open, and you will be able to step through execution and see exactly what's going on. An important point to keep in mind is that when the walker is invoked in this manner, \texttt{exit} will not have the desired effect; execution will continue, but the data stack will be inconsistent, and an error will most likely be raised a short time later. Always use \texttt{continue} to resume execution after a break.
-
-The walker is very handy, but sometimes you just want to see if a word is being called at all and when, and you don't care to single-step it. In that case, you can use the \texttt{watch} word:
-
-\begin{alltt}
- \bs draw-shape break
-\end{alltt}
-
-Now when \texttt{draw-shape} is called, a message will be printed to that effect.
-
-You can undo the effect of \texttt{break} or \texttt{watch} by reloading the original source file containing the word definition in question:
-
-\begin{alltt}
- \bs layout* reload
- \bs draw-shape reload
-\end{alltt}
+\subsection{Enumerations}
-\section{Dealing with hangs}
+A C-style \verb|enum| type defines a set of integer constants. The alien interface lets you define a set of words that push integers on the stack in much the same way as you would in C. While these words can be used for any purpose, using them outside of interfacing with C is discouraged.
-If you accidentally start an infinite loop, you can send the Factor runtime a \texttt{QUIT} signal. On Unix, this is done by pressing \texttt{Control-\bs} in the controlling terminal. This will cause the runtime to dump the data and return stacks in a semi-readable form. Note that this will help you find the root cause of the hang, but it will not let you interrupt the infinite loop.
+\wordtable{
+\vocabulary{alien}
+\parsingword{BEGIN-ENUM:}{BEGIN-ENUM \emph{start}}
+}
+Begins an enumeration that numbers constants starting from \verb|start|.
-\section{Unit testing}
+\wordtable{
+\vocabulary{alien}
+\parsingword{ENUM:}{ENUM: \emph{name}}
+}
+Defines a compound word \verb|name| that pushes a integer. The integer's value is incremented each time \verb|ENUM:| defines a new word.
-Unit tests are very easy to write. They are usually placed in source files. A unit test can be executed with the \texttt{unit-test} word in the \texttt{test} vocabulary. This word takes a list and a quotation; the quotation is executed, and the resulting data stack is compared against the list. If they do not equal, the unit test has failed. Here is an example of a unit test:
+\wordtable{
+\vocabulary{alien}
+\parsingword{END-ENUM}{END-ENUM}
+}
+Ends an enumeration.
+Here is an example:
\begin{verbatim}
-[ "Hello, crazy world" ] [
- "editor" get [ 0 caret set ] bind
- ", crazy" 5 "editor" get [ line-insert ] bind
- "editor" get [ line-text get ] bind
-] unit-test
+BEGIN-ENUM: 0
+ ENUM: monday
+ ENUM: tuesday
+ ENUM: wednesday
+ ENUM: thursday
+ ENUM: friday
+ ENUM: saturday
+ ENUM: sunday
+END-ENUM
\end{verbatim}
-
-To have a unit test assert that a piece of code does not execute successfully, but rather throws an exception, use the \texttt{unit-test-fails} word. It takes only one quotation; if the quotation does \emph{not} throw an exception, the unit test has failed.
-
+This is in fact functionally equivalent to the following code:
\begin{verbatim}
-[ -3 { } vector-nth ] unit-test-fails
+: monday 0 ;
+: tuesday 1 ;
+: wednesday 2 ;
+: thursday 3 ;
+: friday 4 ;
+: saturday 5 ;
+: sunday 6 ;
\end{verbatim}
-Unit testing is a good habit to get into. Sometimes, writing tests first, before any code, can speed the development process too; by running your unit test script, you can gauge progress.
-
-\section{Timing code}
-
-The \texttt{time} word reports the time taken to execute a quotation, in milliseconds. The portion of time spent in garbage collection is also shown:
-
-\begin{alltt}
- [ 1000000 [ f f cons drop ] repeat ] time
-\textbf{515 milliseconds run time
-11 milliseconds GC time}
-\end{alltt}
-
-\section{Exploring memory usage}
+\section{Low-level interface}\label{alien-internals}
-Factor supports heap introspection. You can find all objects in the heap that match a certain predicate using the \texttt{instances} word. For example, if you suspect a resource leak, you can find all I/O ports as follows:
+The alien interface is built on top of a handful of primitives. Sometimes, it is
+useful to call these primitives directly for debugging purposes.
-\begin{alltt}
- USE: io-internals
- [ port? ] instances .
-\textbf{[ \#<port @ 805466443> \#<port @ 805466499> ]}
-\end{alltt}
+\wordtable{
+\vocabulary{alien}
+\classword{dll}
+}
+Instances of this class are handles to native libraries.
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{dlopen}{dlopen ( path -- dll )}
+}
+Opens the specified native library and returns a DLL object. The input parameter is the
+name of a native library file,
+\emph{not} a logical library name.
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{dlsym}{dlsym ( symbol dll -- address )}
+}
+Looks up a named symbol in a native library, and outputs it address. If the \verb|dll| is \verb|f|, the lookup is performed in the runtime executable itself.
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{dlclose}{dlclose ( dll -- )}
+}
+Closes a native library and frees associated native resources.
-The \texttt{references} word finds all objects that refer to a given object:
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{alien-address}{alien-address ( alien -- address )}
+}
+Outputs the address of an alien, as an integer.
-\begin{alltt}
- [ float? ] instances car references .
-\textbf{[ \#<array @ 805542171> [ -1.0 0.0 / ] ]}
-\end{alltt}
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{<alien>}{<alien> ( address -- alien )}
+}
+Creates an alien pointing to the specified address.
-You can print a memory usage summary with \texttt{room.}:
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{<displaced-alien>}{<displaced-alien> ( offset alien -- alien )}
+}
+Outputs an alien pointing at an offset from the base pointer of the input alien. Displaced aliens are used to access nested structures and native arrays.
-\begin{alltt}
- room.
-\textbf{Data space: 16384 KB total 2530 KB used 13853 KB free
-Code space: 16384 KB total 490 KB used 15893 KB free}
-\end{alltt}
+\wordtable{
+\vocabulary{alien}
+\ordinaryword{alien-signed-cell}{alien-signed-cell ( alien offset -- n )}
+\ordinaryword{set-alien-signed-cell}{set-alien-signed-cell ( n alien offset -- )}
+\ordinaryword{alien-unsigned-cell}{alien-unsigned-cell ( alien offset -- n )}
+\ordinaryword{set-alien-unsigned-cell}{set-alien-unsigned-cell( n alien offset -- )}
+\ordinaryword{alien-signed-8}{alien-signed-8 ( alien offset -- n )}
+\ordinaryword{set-alien-signed-8}{set-alien-signed-8 ( n alien offset -- )}
+\ordinaryword{alien-unsigned-8}{alien-unsigned-8 ( alien offset -- n )}
+\ordinaryword{set-alien-unsigned-8}{set-alien-unsigned-8 ( n alien offset -- )}
+\ordinaryword{alien-signed-4}{alien-signed-4 ( alien offset -- n )}
+\ordinaryword{set-alien-signed-4}{set-alien-signed-4 ( n alien offset -- )}
+\ordinaryword{alien-unsigned-4}{alien-unsigned-4 ( alien offset -- n )}
+\ordinaryword{set-alien-unsigned-4}{set-alien-unsigned-4 ( n alien offset -- )}
+\ordinaryword{alien-signed-2}{alien-signed-2 ( alien offset -- n )}
+\ordinaryword{set-alien-signed-2}{set-alien-signed-2 ( n alien offset -- )}
+\ordinaryword{alien-unsigned-2}{alien-unsigned-2 ( alien offset -- n )}
+\ordinaryword{set-alien-unsigned-2}{set-alien-unsigned-2 ( n alien offset -- )}
+\ordinaryword{alien-signed-1}{alien-signed-1 ( alien offset -- n )}
+\ordinaryword{set-alien-signed-1}{set-alien-signed-1 ( n alien offset -- )}
+\ordinaryword{alien-unsigned-1}{alien-unsigned-1 ( alien offset -- n )}
+\ordinaryword{set-alien-unsigned-1}{set-alien-unsigned-1 ( n alien offset -- )}
+\ordinaryword{alien-value-string}{alien-value-string ( alien offset -- string )}
+}
+These primitives read and write native memory. They can be given an alien, displaced alien, or byte array. No bounds checking of any kind is performed.
-And finally, a detailed memory allocation breakdown by type with \texttt{heap-stats.}:
+\section{Manual memory management}\label{malloc}
-\begin{alltt}
- heap-stats.
-\textbf{bignum: 312 bytes, 17 instances
-cons: 850376 bytes, 106297 instances
-float: 112 bytes, 7 instances
-t: 8 bytes, 1 instances
-array: 202064 bytes, 3756 instances
-hashtable: 54912 bytes, 3432 instances
-vector: 5184 bytes, 324 instances
-string: 391024 bytes, 7056 instances
-sbuf: 64 bytes, 4 instances
-port: 112 bytes, 2 instances
-word: 96960 bytes, 3030 instances
-tuple: 688 bytes, 22 instances}
-\end{alltt}
+If for whatever reason Factor's memory management is unsuitable for a certain task, you can
+directly call the standard C memory management routines. These words are very raw and deal with addresses directly, and of course it is easy to corrupt memory or crash the runtime
+this way.
+\wordtable{
+\vocabulary{kernel-internals}
+\ordinaryword{malloc}{malloc ( size -- address )}
+}
+Allocate a block of size \verb|size| and output a pointer to it.
+\wordtable{
+\vocabulary{kernel-internals}
+\ordinaryword{realloc}{realloc ( address size -- address )}
+}
+Resize a block previously allocated with \verb|malloc|.
+\wordtable{
+\vocabulary{kernel-internals}
+\ordinaryword{free}{free ( address -- )}
+}
+Deallocate a block previously allocated with \verb|malloc|.
\chapter{Stack effect inference}
projects / factor.git / commitdiff