Joe Groff [Tue, 18 Oct 2011 22:13:02 +0000 (15:13 -0700)]
new vocab tools.annotations.assertions
Annotates unsafe words with assertions that their inputs and outputs are valid. Provide annotations for stream-read(-partial)-unsafe and (set-)nth-unsafe to start with.
Joe Groff [Tue, 18 Oct 2011 21:28:41 +0000 (14:28 -0700)]
io: error stream combinators
* with-error-stream(*)
* with-output+error-stream(*) -- sets output and error stream to same stream
* with-output>error -- sets output stream to error stream for scope
* with-input-output+error-streams(*) -- like with-streams, but sets output and error to same stream
Revert "Optimize byte-reverse macro in alien.endian -- it was shifting each byte to the 0th bit place, bitand-ing it with HEX: ff, then shifting it to its final destination -- now we immediately shift to the proper location and bitand with the proper bitmask."
Joe Groff [Mon, 17 Oct 2011 17:41:03 +0000 (10:41 -0700)]
io.sockets.windows: recv to byte-array via malloc
WSARecvFrom on Windows does its work asynchronously, so the buffer cannot be in the managed heap during the extent of the operation.
Joe Groff [Mon, 17 Oct 2011 02:37:21 +0000 (19:37 -0700)]
io.sockets: receive-unsafe and receive-into
Analogous to read-unsafe/read-into for streams (and thereby TCP sockets), provide receive-unsafe and receive-into words for datagram sockets that receive into a caller-supplied buffer.
Joe Groff [Sat, 15 Oct 2011 21:50:53 +0000 (14:50 -0700)]
io.encodings.utf16: fast path & slow path speedups
Implement M: utf16[bl]e encode-string to use a fast implementation if a string is ASCII only, and do some inlining so the slow path optimizes a bit better.
Joe Groff [Sat, 15 Oct 2011 06:24:36 +0000 (23:24 -0700)]
compiler.tree.dead-code: fix test
It was using "read" in an optimization test, but since "read" is now inline, it wasn't getting the expected optimized quotation back.
Joe Groff [Sat, 15 Oct 2011 01:08:27 +0000 (18:08 -0700)]
io.encodings: stream-contents method for decoder
stream-contents on decoder is much faster implemented using a read1 loop than with the default implementation.
Joe Groff [Sat, 15 Oct 2011 01:07:38 +0000 (18:07 -0700)]
io: stream-contents, -seekable?, -length generics
stream-contents is extremely slow on decoder streams when implemented with each-block, and it can be implemented very quickly for streams of known length using stream-read-unsafe. Make it generic and provide some off-the-shelf implementations using each-block, a read1 loop, or length + read-unsafe. Provide new stream-seekable? and stream-length generics that can be implemented by stream implementations that know their size.
Joe Groff [Fri, 14 Oct 2011 02:03:15 +0000 (19:03 -0700)]
clean up forwarding stream-read-partial-unsafe
Now that "io" provides a method on object for stream-read-partial-unsafe that forwards to stream-read-unsafe, individual stream class that don't implement partial reads don't need to provide such a method themselves.
Joe Groff [Fri, 14 Oct 2011 01:29:34 +0000 (18:29 -0700)]
io: default methods for extended stream protocols
Provide default implementations of all the input-stream methods in terms of stream-read1 and of all the output-stream methods in terms of stream-write1.
Joe Groff [Thu, 13 Oct 2011 22:24:28 +0000 (15:24 -0700)]
io.streams.peek: test with memory-stream
Some binary stream types use memcpy to implement read-unsafe, while element-agnostic wrapper streams have to use copy. Make sure peek-stream works with both when it has to divide a read between the peek buffer and the underlying buffer (it does).
Joe Groff [Thu, 13 Oct 2011 22:11:32 +0000 (15:11 -0700)]
specialized-arrays: direct-slice for byte-arrays
Implement nth-c-ptr and direct-like for byte-arrays so that direct-slice over a byte-array makes a uchar-array{ } over a part of the byte-array
Joe Groff [Thu, 13 Oct 2011 21:53:13 +0000 (14:53 -0700)]
io: add read-unsafe and read-partial-unsafe words
Add shortcut words to operate on the input-stream like the other stream methods have. Make all those words inline too just for fun.
Joe Groff [Thu, 13 Oct 2011 21:30:31 +0000 (14:30 -0700)]
kill noncopying-reader mixin, stream-read generic
Now that all streams have been updated to implement the stream-read-unsafe protocol, take out the noncopying-reader shim. Turn stream-read and stream-read-partial into plain functions over the -unsafe generics.
Joe Groff [Thu, 13 Oct 2011 04:12:57 +0000 (21:12 -0700)]
io.encodings.string: faster decode and encode
stream-contents is apparently way slow for decoders. Write decode out more directly as a read1/push loop so it's faster. encode isn't quite as bad, but we can still get a 25% speed improvement by writing to an appropriately sized byte-vector.
Joe Groff [Thu, 13 Oct 2011 04:08:58 +0000 (21:08 -0700)]
io.encodings: encoded/decoded length hints
Add guess-encoded-length and guess-decoded-length generics that encodings can implement to provide hints as to how large the translation product will be, for sizing vectors and things like that.
Joe Groff [Wed, 12 Oct 2011 23:08:57 +0000 (16:08 -0700)]
mongodb.gridfs: move to unmaintained
It has no tests and nothing appears to use it, so I don't want to mess with it to make it implement the stream-read-unsafe interface.
Joe Groff [Wed, 12 Oct 2011 18:59:10 +0000 (11:59 -0700)]
io: non-copying each-block and contents variants
each-(stream-)block* is like each-block but takes a buffer object and reads into it repeatedly. (stream-)contents* determines the stream length then does a single stream-read-unsafe into a preallocated buffers. Both functions currently only work for byte-arrays (and contents* only for seekable streams), so they can't replace the non-starred versions completely just yet.
Joe Groff [Tue, 11 Oct 2011 23:00:22 +0000 (16:00 -0700)]
io.streams.sequence: change to provide read-unsafe
Change io.streams.sequence and its children io.streams.byte-array and io.streams.string to implement the -unsafe protocol and mixin noncopying-reader.
Joe Groff [Tue, 11 Oct 2011 22:58:45 +0000 (15:58 -0700)]
io, io.streams.c: factor noncopying-reader mixin
Instances of the mixin implement stream-read and stream-read-partial in terms of stream-read-unsafe and stream-read-partial-unsafe, respectively.
Joe Groff [Tue, 11 Oct 2011 06:04:55 +0000 (23:04 -0700)]
bootstrap, io.streams.c: use new fread primitive
Change the fread primitive to fread-unsafe, matching the new primitive in the VM, and update the implementation of c-reader to implement stream-read-unsafe and stream-read in terms of fread-unsafe
Joe Groff [Tue, 11 Oct 2011 06:02:27 +0000 (23:02 -0700)]
io: non-copying binary stream read generics
Add generics stream-read-unsafe and stream-read-partial-unsafe, which take a buffer pointer and return a count of bytes read instead of returning a freshly allocated byte array.
Joe Groff [Tue, 11 Oct 2011 06:00:03 +0000 (23:00 -0700)]
vm: non-copying primitive_fread
primitive_fread now takes a buffer pointer argument and returns a count of bytes read instead of allocating a byte array.