WITH-ARRAY-AS-FOREIGN-POINTER - Allows Lisp arrays to be passed to foreign functions by copying their contents to malloc'ed foreign space, using only CFFI.

Copying is done with a set of optimized type-specific copying functions to avoid consing on fast lisp implementations like SBCL.

Downloadable from the WAAF-CFFI github page. Includes a txt documentation file, and a test package. Uses ASDF.


           (U PU :FLOAT                    ;; ARRAY  POINTER-VAR  CFFI-TYPE
           :LISP-TYPE SINGLE-FLOAT         ;; promise U is a single-float array
           :START 1                        ;; begin at index 1 of of Lisp array
           :END   7                        ;; last index used is 6
           :COPY-TO-FOREIGN   T            ;; put contents of Lisp into foreign memory
           :COPY-FROM-FOREIGN T)           ;; copy back from FFI space 
         ;; at this point, PU is a foreign pointer containing indices 1..6  of U copied
         ;; from the array, of type :FLOAT.

     ;; at end, all foreign memory is deallocated, and U has been copied
     ;; back from foreign space, but the 0th (and >6)  element of U 
     ;; is untouched because START=1, END=7
     ;; The options :START :END :COPY-TO-FOREIGN :COPY-FROM-FOREIGN are optional.  
     ;; By default, :LISP-TYPE is T and any array can be given.


There exist differing opinions on how to pass Lisp arrays to foreign functions. Some Lisp implementation allow arrays to be pinned against GC, or allow GC to be suspended, so that a pointer to the actual lisp data rather than a malloc'ed copy can be passed. The author of WAAF-CFFI feels that, although passing a pointer to the Lisp data may be the elegant approach in terms of conserving memory and avoiding copying, it requires knowledge of each implementation's internals, is harder to maintain, and can break in more places.. For a fast Lisp implementation like SBCL, copying overheads are small compared to the work done on the array, and running out of malloc space should not be an issue on a 64 bit machine. Hence the copying approach works very well in practice, and will not break unless CFFI is broken.

Quick assessment (2024-03-09)

COMMON-LISP-USER> (coerce 2 'float) 2.0 COMMON-LISP-USER> (coerce 2 'integer) ;;; Warning: Cannot coerce to type INTEGER: unknown or not defined for coerce 2 COMMON-LISP-USER> (coerce 2 '(unsigned-byte 16)) ;;; Warning: Cannot coerce to type (UNSIGNED-BYTE 16): unknown or not defined for coerce 2