This is the Array Operations Reference Manual, version 1.0.0, generated automatically by Declt version 4.0b2.
Copyright © 2019-2022 Steve Nunez Copyright © 2019-2022 Tamas K. Papp
Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.
Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the section entitled “Copying” is included exactly as in the original.
Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be translated as well.
This program is distributed under the terms of the Microsoft Public License.
The main system appears first, followed by any subsystem dependency.
Array operations library for Common Lisp
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
The array-operations system is a collection of functions and macros for manipulating Common Lisp arrays and performing numerical calculations with them.
Array-operations is a ’generic’ way of operating on array like data structures using a syntax that is natural for Common Lisp. Several aops functions have been implemented for data-frame. For those that haven’t, you can transform arrays to data frames using the df:matrix-df function, and a data-frame to an array using df:as-array. This make it convenient to work with the data sets using either system.
1.0.0
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
let-plus (system).
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Array operations for array-like data structures
Steve Nunez
Tamas K. Papp <tkpapp@gmail.com>
MS-PL
Files are sorted by type and then listed depth-first from the systems components trees.
array-operations (system).
array-operations/all (system).
array-operations/generic (system).
array-operations/reducing (system).
array-operations/matrices (system).
array-operations/creating (system).
array-operations/utilities (system).
array-operations/indexing (system).
array-operations/displacing (system).
sub-location% (function).
array-operations/transforming (system).
array-operations/stacking (system).
stack*0 (function).
Packages are listed by definition order.
Macros for operating over indexes of arrays.
Generic functions for elementary array operations, with methods on ’array. Enables new methods to be defined to enable treating other data structures as arrays.
This top level package re-exports the individual packages: generic, reducing, matrices, creating, indexing, displacing, transforming and stacking. It does not export utilities. The reason for this structure is the use of the ASDF package-inferred-system, where each file is its own package. None of Papp’s other libraries use this, and it seems to have been added after he abandoned the library.
lisp-stat.
Functions for creating arrays or data frames filled with various values.
Functions that return arrays displaced in various ways from another array.
sub-location% (function).
Functions for reducing arrays, or performing reducing like operations over the elements of an array.
common-lisp.
Functions for representing matrices as 2D arrays. A matrix is a two-dimensional array often used for linear algebra. See also the matrix functions in NUM-UTILS, which should be migrated to AOPS.
Functions for transforming arrays in various ways.
Functions for composing arrays into new arrays, by stacking.
One may think of stacking blocks as the guiding metaphor.
For example, stack two row vectors to yield a 2x2 matrix:
(stack-rows #(1 2) #(3 4)) -> #2A((1 2)
(3 4))
stack*0 (function).
Definitions are sorted by export status, category, package, and then by lexicographic order.
Dimensions of array-like object.
Given one or more symbols INDEX, walks the BODY expression to determine the index ranges by looking for AREF and ROW-MAJOR-AREF calls.
Transpose of 2D array A
(each-index (i j)
(aref A j i))
Diagonal of a square 2D array
(each-index i (aref A i i))
Turn a 2D array into an array of arrays
(each-index i
(each-index j
(aref A i j)))
Matrix-vector product:
(each-index i
(sum-index j
(* (aref A i j) (aref x j))))
Sets elements of the given ARRAY to values of the BODY, evaluated at array indices INDEX
Note: This has the same semantics as each-index and each-index*,
but the INDEX ranges are taken from the ARRAY dimensions, not
a code walker.
Given one or more symbols INDEX, creates an array with ELEMENT-TYPE, then iterates over the index ranges with the innermost loop using the last index.
Each iteration evaluates BODY, and sets the array element.
To find the range of the indices, walks the BODY expression
to determine the index ranges by looking for
AREF and ROW-MAJOR-AREF calls.
Transpose of 2D array A
(each-index* t (i j)
(aref A j i))
Diagonal of a square 2D array
(each-index* t i (aref A i i))
Turn a 2D array into an array of arrays
(each-index* t i
(each-index* t j
(aref A i j)))
Outer product of two 1D arrays to create a 2D array
(each-index* t (i j)
(* (aref x i) (aref y j)))
Matrix-vector product:
(each-index* t i
(sum-index j
(* (aref A i j) (aref x j))))
Multiply by the arguments
Iterates over a multidimensional range of indices.
SYMS must be a list of symbols, with the first symbol
corresponding to the outermost loop.
DIMENSIONS will be evaluated, and must be a list of
dimension sizes, of the same length as SYMS.
Example:
(nested-loop (i j) ’(10 20) (format t ’~a ~a~%’ i j))
expands to:
; Check dimensions
(destructuring-bind (g1 g2) ’(10 20)
(loop for i from 0 below g1 do
(loop for j from 0 below g2 do
(format t ’~a ~a~%’ i j))))
with some additional type and dimension checks.
Reduction over one or more INDEX symbols in an array expression.
The range of these symbols is determined by walking the tree
for AREF and ROW-MAJOR-AREF calls.
Example:
(defparameter A #2A((1 2) (3 4)))
(reduce-index #’+ i (row-major-aref A i)) ; Sum all elements (sum-index)
=> 10
(reduce-index #’* (i j) (aref A i j)) ; Multiply all elements
=> 24
(reduce-index #’max i (row-major-aref A i)) ; Maxmum value
=> 4
Sums over one or more INDEX symbols in an array expression.
The range of these symbols is determined by walking the tree
for AREF and ROW-MAJOR-AREF calls.
Example:
(defparameter A #2A((1 2) (3 4)))
(sum-index i (row-major-aref A i)) ; Sum all elements
=> 10
(sum-index (i j) (aref A i j)) ; Sum all elements
=> 10
(sum-index i (aref A i i)) ; Trace of array
=> 5
Makes a new array of type ELEMENT-TYPE, containing the result of an array expression. All input and outputs have the same shape, and BODY is evaluated for each index
VARIABLES must be a list of symbols bound to arrays.
Each array must have the same dimensions. These are
checked at compile and run-time respectively.
(let ((a #2A((1 2) (3 4))))
(vectorize (a) (+ a 1)))
-> #2A((2 3) (4 5))
(let ((a #(1 2 3))
(b #(4 5 6)))
(vectorize (a b) (+ a (* b 2))))
-> #(9 12 15)
Fills an array RESULT with the result of an array expression. All input and outputs have the same shape, and BODY is evaluated for each index
VARIABLES must be a list of symbols bound to arrays. Each array must have the same dimensions. These are checked at compile and run-time respectively.
(let ((a #2A((1 2) (3 4)))
(b (make-array ’(2 2))))
(vectorize! b (a) (+ a 1)))
-> #2A((2 3) (4 5))
(let ((a #(1 2 3))
(b #(4 5 6)))
(vectorize! b (a b) (+ a (* b 2))))
-> #(9 12 15)
Makes a new array of type ELEMENT-TYPE, containing the result of an array expression. All input and outputs have the same shape, and BODY is evaluated for each index
VARIABLES must be a list of symbols bound to arrays.
Each array must have the same dimensions. These are
checked at compile and run-time respectively.
(let ((a #2A((1 2) (3 4))))
(vectorize* t (a) (+ a 1)))
-> #2A((2 3) (4 5))
(let ((a #(1 2 3))
(b #(4 5 6)))
(vectorize* t (a b) (+ a (* b 2))))
-> #(9 12 15)
Performs a reduction using FN over all elements in a vectorized expression
on array VARIABLES.
VARIABLES must be a list of symbols bound to arrays.
Each array must have the same dimensions. These are
checked at compile and run-time respectively.
Example: Maximum value in an array A
(vectorize-reduce #’max (a) a)
Example: Maximum absolute difference between two arrays A and B
(vectorize-reduce #’max (a b) (abs (- a b)))
Iterate over the subscripts of an array with given DIMENSIONS. SUBSCRIPTS contains the current subscripts as a vector of fixnums, POSITION has the row-major index. Consequences are undefined if either POSITION or SUBSCRIPTS is modified.
Like WALK-SUBSCRIPTS, but SUBSCRIPTS is a newly created list for each position that does not share structure and can be freely used/modified/kept etc.
Find the row-major-aref in ARRAY with the maximum value Returns both the index and the value of ARRAY at that index
Find the row-major-aref in ARRAY with the minimum value Returns both the index and the value of ARRAY at that index
FN must accept two inputs and return true/false. This function is applied
to elements of ARRAY, to find the ’best’. The row-major-aref index is returned.
Example: The index of the maximum is
* (best #’> #(1 2 3 4))
3 ; row-major index
4 ; value
This function was adapted from Paul Graham’s On Lisp
Check if PERMUTATION is a valid permutation (of the given RANK), and signal an error if necessary.
Return a function composed of a univariate function that coerces to ELEMENT-TYPE and function. When FUNCTION is not given, return a closure that coerces to ELEMENT-TYPE.
The opposite of SUBARRAYS.
If ELEMENT-TYPE is not given, it is inferred from the first element of array, which also determines the dimensions. If that element is not an array, the original ARRAY is returned as it is.
Return a list of increasing indices that complement PERMUTATION, i.e. form a permutation when appended. Atoms are accepted and treated as lists of a single element.
Return a completed version of permutation, appending it to its complement.
Copy SOURCE into TARGET, for array arguments of compatible dimensions (checked). Return TARGET, making the implementation of the semantics of SETF easy.
Copy elements with row major indexes between the given start and end from SOURCE to DESTINATION, respectively. Elements are coerced to ELEMENT-TYPE when necessary. Return no values.
This function should be used to implement copying of contiguous row-major blocks of elements, most optimizations should happen here.
Shorthand function for displacing an array.
Like EACH*, with ELEMENT-TYPE T.
Apply function to the array arguments elementwise, and return the result as an array with the given ELEMENT-TYPE. Arguments are checked for dimension compatibility.
Return a list of dimensions corresponding to OBJECT. Positive integers are
treated as dimensions of rank 1, lists are returned as they are, and arrays
are queried for their dimensions.
OBJECTS accepted by this function as valid dimensions are called ‘dimension specifications’ in this library.
Fills a given ARRAY with VALUE, coerced to the same element type as ARRAY
If one of the dimensions is missing (indicated with T), replace it with a dimension so that the total product equals SIZE. If that’s not possible, signal an error. If there are no missing dimensions, just check that the product equals size. Also accepts other dimension specifications (integer, array).
Return ARRAY flattened to a vector. Will share structure.
Like GENERATE*, with ELEMENT-TYPE T.
Return an array with given DIMENSIONS and ELEMENT-TYPE, with elements generated by calling FUNCTION.
Function is called with:
- no arguments, when ARGUMENTS is nil
- the position (= row major index), when ARGUMENTS is :POSITION
- a list of subscripts, when ARGUMENTS is :SUBSCRIPTS
- both when ARGUMENTS is :POSITION-AND-SUBSCRIPTS
The traversal order is unspecified and may be nonlinear.
Test if PERMUTATION is the identity permutation, i.e. a sequence of consecutive integers starting at 0. Note that permutation is otherwise not checked, i.e. it may not be a permutation.
Test if PERMUTATION is the identity permutation, i.e. a sequence of consecutive integers starting at 0. Note that permutation is otherwise not checked, i.e. it may not be a permutation.
Invert a permutation.
Make a vector of N elements and type T, containing evenly spaced numbers over an interval.
The first element is equal to START and last element STOP,
with constant difference between consecutive elements.
(linspace 0 4 5) -> #(0 1 2 3 4)
(linspace 1 3 5) -> #(0 1/2 1 3/2 2)
(linspace 0 4d0 3) -> #(0.0d0 2.0d0 4.0d0)
Fill an array with evenly spaced numbers over an interval.
The first element is equal to START and last element STOP,
with constant difference between consecutive elements in ROW-MAJOR-INDEX.
Make a vector of N elements and type ELEMENT-TYPE, containing evenly spaced numbers over an interval.
The first element is equal to START and last element STOP,
with constant difference between consecutive elements.
Returns new array with the same properties as ARRAY.
Keyword arguments will override properties of ARRAY.
If INITIAL-ELEMENT is specified, it is coerced to ELEMENT-TYPE.
Like MARGIN*, with ELEMENT-TYPE T.
PERMUTE ARRAY with ‘(,@OUTER ,@INNER), split the inner subarrays, apply FUNCTION to each, return the results in an array of dimensions OUTER, with the given ELEMENT-TYPE.
Return NIL if MATRIX does not have rank 2.
Return NIL if MATRIX does not have rank 2.
Finds the element of ARRAY that returns the value closest to positive infinity when FN is applied to the array value.
Returns the row-major-aref index, and the winning value.
Example: The maximum of an array is
(most #’identity #(1 2 3))
-> 2 (row-major index)
3 (value)
Minimum of an array is
(most #’- #(1 2 3))
0
-1
This function was adapted from Paul Graham’s On Lisp
Makes an array of shape DIMENSIONS and type T, filled with ones
Fills the given ARRAY with 1’s, coerced to the element type. Returns ARRAY.
Makes an array of shape DIMENSIONS and type ELEMENT-TYPE, filled with ones coerced to the specified type ELEMENT-TYPE.
Like OUTER, with ELEMENT-TYPE t.
Generalized outer product of ARRAYS with FUNCTION. The resulting array has the concatenated dimensions of ARRAYS, and the given ELEMENT-TYPE.
Return a subset of the array, on the first indexes between START and END.
Return ARRAY with the axes permuted by PERMUTATION, which is a sequence of indexes. Specifically, an array A is transformed to B, where
B[b_1,...,b_n] = A[a_1,...,a_n] with b_i=a_{P[i]}
P is the permutation.
Array element type is preserved.
Product of elements in the argument. NOT EXPORTED.
Makes an array of shape DIMENSIONS and type T, filled with random numbers uniformly distributed between 0 and 1.
Uses the built-in RANDOM function.
(rand 3) -> #(0.39319038 0.69693553 0.5021677)
(rand ’(2 2)) -> #2A((0.91003513 0.23208928) (0.5577954 0.94657767))
NOTE: If it’s important that these numbers are really random
(e.g. cryptographic applications), then you should probably
not use this function.
Fills a given ARRAY with random numbers, uniformly distributed between 0 and 1. Uses the built-in RANDOM function. Returns ARRAY.
Makes an array of shape DIMENSIONS and type ELEMENT-TYPE, filled with random numbers uniformly distributed between 0 and 1.
Uses the built-in RANDOM function.
(rand 3) -> #(0.39319038 0.69693553 0.5021677)
(rand ’(2 2)) -> #2A((0.91003513 0.23208928) (0.5577954 0.94657767))
NOTE: If it’s important that these numbers are really random
(e.g. cryptographic applications), then you should probably
not use this function.
Creates an array of shape DIMENSIONS and type T, and fills with normally distributed numbers with a mean of zero and standard deviation of 1
Uses the Box-Muller algorithm and built-in random number generator.
(rand 3) -> #(-0.82067037 -0.60068226 -0.21494178)
(randn ’(2 2)) -> #2A((1.6905352 -2.5379088) (0.8461403 -1.505984))
NOTE: If it’s important that these numbers are really random
(e.g. cryptographic applications), then you should probably
not use this function.
Fills ARRAY with normally distributed numbers with a mean of zero and standard deviation of 1
Uses the Box-Muller algorithm and built-in random number generator.
NOTE: If it’s important that these numbers are really random
(e.g. cryptographic applications), then you should probably
not use this function.
Creates an array of shape DIMENSIONS and type ELEMENT-TYPE, and fills with normally distributed numbers with a mean of zero and standard deviation of 1
Uses the Box-Muller algorithm and built-in random number generator.
(rand 3) -> #(-0.82067037 -0.60068226 -0.21494178)
(randn ’(2 2)) -> #2A((1.6905352 -2.5379088) (0.8461403 -1.505984))
NOTE: If it’s important that these numbers are really random
(e.g. cryptographic applications), then you should probably
not use this function.
Recycle elements of OBJECT, extending the dimensions by outer (repeating OBJECT) and inner (repeating each element of OBJECT). When both INNER and OUTER are nil, the OBJECT is returned as is. Non-array OBJECTs are interpreted as rank 0 arrays, following the usual semantics.
Reshape ARRAY using DIMENSIONS (which can also be dimension specifications).
If DIMENSIONS is a list, it may contain a single element T which will be calculated to match the total size of the resulting array.
Array reshaped as an Nx1 matrix.
Array reshaped as an 1xN matrix.
Test if arguments have the same dimensions. NOT EXPORTED.
Returns new array with the same properties as ARRAY.
Keyword arguments will override properties of ARRAY.
If INITIAL-ELEMENT is specified, it is coerced to ELEMENT-TYPE.
Return an array of subarrays, split off at RANK. All subarrays are displaced and share structure.
Test if MATRIX has two dimensions and that they are equal.
Test if MATRIX has two dimensions and that they are equal.
Like STACK*, with element-type T.
Stack array arguments along AXIS. ELEMENT-TYPE determines the element-type of the result.
Like STACK-COLS*, with ELEMENT-TYPE T.
Stack OBJECTS column-wise into an array of the given ELEMENT-TYPE, coercing if necessary. Always return a simple array of rank 2.
How objects are used depends on their dimensions, queried by DIMS:
- when the object has 0 dimensions, fill a column with the element.
- when the object has 1 dimension, use it as a column.
- when the object has 2 dimensions, use it as a matrix.
When applicable, compatibility of dimensions is checked, and the result is used to determine the number of rows. When all objects have 0 dimensions, the result has one row.
Like STACK-ROWS*, with ELEMENT-TYPE T.
Stack OBJECTS row-wise into an array of the given ELEMENT-TYPE, coercing if necessary. Always return a simple array of rank 2.
How objects are used depends on their dimensions, queried by DIMS:
- when the object has 0 dimensions, fill a row with the element.
- when the object has 1 dimension, use it as a row.
- when the object has 2 dimensions, use it as a matrix.
When applicable, compatibility of dimensions is checked, and the result is used to determine the number of columns. When all objects have 0 dimensions, the result has one column.
Given a partial list of subscripts, return the subarray that starts there, with all the other subscripts set to 0, dimensions inferred from the original.
If no subscripts are given, the original array is returned. Implemented by displacing, may share structure.
Displaced vector between START and END.
Turns an array by a specified number of clockwise 90° rotations. The axis of rotation is specified by RANK-1 (defaulting to 0) and RANK-2 (defaulting to 1).
Makes an array of shape DIMENSIONS and type T, filled with zeros
Fills the given ARRAY with zero values, coerced to the element type. Returns ARRAY.
Makes an array of shape DIMENSIONS and type ELEMENT-TYPE, filled with zeros coerced to the specified type ELEMENT-TYPE.
Return the contents of OBJECT as an array. Exact semantics depends on OBJECT, but generally objects which contain elements in a rectilinear coordinate system should have a natural mapping to arrays.
When the second value is T, the array itself does not share structure with OBJECT, but its elements may. Otherwise, it is indeterminate whether the two objects share structure, and consequences of modifying the result are not defined. Methods are encouraged but not required to return a second value.
Return specificed dimension of ARRAY.
Return a list of dimensions.
For non-array objects, SIZE, DIM, NROW and NCOL use this method by default, so it is enough to define it (unless efficiency is a concern).
When DIMS is not defined for an object, it falls back to as-array, which may be very inefficient for objects which need to be consed. It is always advisable to define DIMS.
Return TYPE such that
1. all elements of ARRAY are guaranteed to be a subtype of TYPE,
2. if applicable, elements of ARRAY can be set to values which are of a type that is a subtype of TYPE.
Number of columns. Will signal an error if ARRAY is not a matrix.
Number of rows. Will signal an error if ARRAY is not a matrix.
Return the rank of ARRAY.
Return the total number of elements in array.
Method used to implement the copying of objects in STACK-COL*, by copying the elements of SOURCE to DESTINATION, starting with the column index START-COL in the latter. Elements are coerced to ELEMENT-TYPE.
This method is only called when (DIMS SOURCE) was non-nil. It is assumed that it only changes elements in DESTINATION which are supposed to be copies of SOURCE. DESTINATION is always a matrix with element-type upgraded from ELEMENT-TYPE, and its NROW should match the relevant dimension of SOURCE.
All objects have a fallback method, defined using AS-ARRAY. The only reason for defining a method is efficiency.
Method used to implement the copying of objects in STACK-ROW*, by copying the elements of SOURCE to DESTINATION, starting with the row index START-ROW in the latter. Elements are coerced to ELEMENT-TYPE.
This method is only called when (DIMS SOURCE) was non-nil. It is assumed that it only changes elements in DESTINATION which are supposed to be copies of SOURCE. DESTINATION is always a matrix with element-type upgraded from ELEMENT-TYPE, and its NCOL should match the relevant dimension of SOURCE.
All objects have a fallback method, defined using AS-ARRAY. The only reason for defining a method is efficiency.
error.
error.
:index
error.
:index
Examples:
Matrix-matrix multiply
(foreach :index (i j) :sum k
:value (* (aref A i k) (aref B k j)))
Sum over vector
(foreach :sum i :value (aref A i))
Walks an expression tree EXPR, finds AREF and ROW-MAJOR-AREF, SVREF or ELT calls.
Returns a list of (symbol, expr) where EXPR is an expression which
evaluates to the array dimension size for SYMBOL.
Example:
(find-array-dimensions ’(+ (aref a i) (* 2 (aref b j k))))
-> ((I ARRAY-DIMENSION A 0) (K ARRAY-DIMENSION B 1) (J ARRAY-DIMENSION B 0))
Make a bit vector of flags with indexes from PERMUTATION, signaling errors for invalid and repeated indices. NOT EXPORTED.
Stack arrays along the 0 axis, returning an array with given ELEMENT-TYPE.
Return (values OFFSET REMAINING-DIMENSIONS) that can be used to displace a row-major subarray starting at SUBSCRIPTS in an array with the given DIMENSIONS. NOT EXPORTED.
| Jump to: | &
(
A B C D E F G I L M N O P R S T V W Z |
|---|
| Jump to: | &
(
A B C D E F G I L M N O P R S T V W Z |
|---|
| Jump to: | I S |
|---|
| Index Entry | Section | ||
|---|---|---|---|
| | |||
| I | |||
index: | Public conditions | ||
index: | Public conditions | ||
| | |||
| S | |||
Slot, index: | Public conditions | ||
Slot, index: | Public conditions | ||
| | |||
| Jump to: | I S |
|---|
| Jump to: | A C F P S T |
|---|
| Jump to: | A C F P S T |
|---|