Bivalent Streams
An overview of support for this concept at the implementation level. In simple terms, "bivalent" means that both read-char and read-byte work for the same stream, along with the corresponding output functions. The quasi-standard option would be flexi-streams, but that in turn requires Gray Streams. The need for bivalent streams comes from things like HTTP and PDF, which tend to mix binary data with text.

Put another way: What do we mean when we say "text file," exactly? After all, any 8-bit code can be interpreted as a character (printable or otherwise). This has been conceptualized as Faithful Output with Character Streams, which means that an (8-bit) char-code is not altered in transit. But a text file is a human-readable stream (weird delimiters aside). Many binary formats are container formats, and may contain text, either as metadata or as actual content. Thus a document will likely contain text, but isn't necessarily a "text file."

Clozure CL

In CCL, both socket streams and vector streams are bivalent, whereas file streams are not.

Allegro CL

In ACL, not specifying an element type gives you a simple stream, which is bivalent:

CG-USER(25): (with-open-file (stream "util.lisp") (format t "Reading one character: ~S.~%" (read-char stream)) (format t "Reading one 8-bit byte: ~S.~%" (read-byte stream)) (format t "Element type: ~S.~%" (stream-element-type stream))) Reading one character: #\;. Reading one 8-bit byte: 59. Element type: (UNSIGNED-BYTE 8).

Specifying an element type, on the other hand, gives you a Gray Stream:

CG-USER(26): (with-open-file (stream "util.lisp" :element-type 'character) (read-byte stream)) Error: No methods applicable for generic function #<STANDARD-GENERIC-FUNCTION STREAM-READ-BYTE> with args (#<EXCL::CHARACTER-INPUT-FILE-STREAM #P"util.lisp" pos 0 @ #x21868afa>) of classes (EXCL::CHARACTER-INPUT-FILE-STREAM) [condition type: PROGRAM-ERROR]

LispWorks

In LispWorks, binary streams are bivalent. Here's a somewhat unusual example:

CL-USER 24 > (with-open-file (stream "util.lisp" :element-type '(unsigned-byte 32)) (format t "Reading one character: ~S.~%" (read-char stream)) (format t "Reading one 32-bit byte: ~S.~%" (read-byte stream)) (format t "Element type: ~S.~%" (stream-element-type stream)) (format t "Stream type: ~S.~%" (type-of stream))) Reading one character: #\;. Reading one 32-bit byte: 993737531. Element type: (UNSIGNED-BYTE 32). Stream type: STREAM::LATIN-1-FILE-STREAM.

Take careful note of what's happening here:

  • Reading one character didn't actually consume a 32-bit "byte."
  • This is because the stream is using an 8-bit character encoding.
  • So after calling read-char, those 8 bits are still in the buffer.
  • Then read-byte will consume this octet—plus three more.

SBCL

In SBCL, one obtains a bivalent stream by specifying :default as the element type. Bear in mind that :default is not the actual default.

ECL

In ECL, character streams are bivalent:

> (with-open-file (stream "test.txt") (values (read-char stream) (read-byte stream))) #\; 59

Clasp

In Clasp, binary streams are bivalent.

Programming Tips