The Select Reference Manual

The Select Reference Manual

This is the Select Reference Manual, version 1.0.0, generated automatically by Declt version 4.0b2.

Copyright © 2019-2022 Steve Nunez

Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the section entitled “Copying” is included exactly as in the original.

Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be translated as well.

Table of Contents


Copying

This program is distributed under the terms of the Microsoft Public License.


1 Introduction

Select is a library for taking slices from array-like objects. The most frequently used form is:

(select object selection1 selection2 ...)

where each selection specifies a set of subscripts along the corresponding axis. The selection specifications are found below.

1.1 Selection Specifiers

Selecting Single Values

A non-negative integer selects the corresponding index, while a negative integer selects an index counting backwards from the last index. For example:

(select #(0 1 2 3) 1)                  ; => 1
(select #(0 1 2 3) -2)                 ; => 2

These are called singleton slices. Each singleton slice drops the dimension: vectors become atoms, matrices become vectors, etc..

Selecting Ranges

(range start end) selects subscripts i where start <= i < end. When end is nil, the last index is included (cf. subseq). Each boundary is resolved according to the other rules if applicable, so you can use negative integers:

(select #(0 1 2 3) (range 1 3))         ; => #(1 2)
(select #(0 1 2 3) (range 1 -1))        ; => #(1 2)

Selecting All Subscripts of a Dimension

t selects all subscripts:

(select #2A((0 1 2)
            (3 4 5))
         t 1)                           ; => #(1 4)

Selecting with a Sequence

Sequences can be used to make specific selections from the object. For example:

(select #(0 1 2 3 4 5 6 7 8 9)
	(vector (range 1 3) 6 (range -2 -1))) ; => #(1 2 3 6 8 9)
(select #(0 1 2) '(2 2 1 0 0))                ; => #(2 2 1 0 0)

Using Bit Vectors as a Mask

Bit vectors can be used to select elements of arrays and sequences as well:

(select #(0 1 2 3 4) #*00110)          ; => #(2 3)

1.1.1 Extensions

Section 1.1 describes the core functionality. The semantics can be extended, as you will see in the next section. The extensions in this section are provided by the library and prove useful in practice. Their implementation provides good examples of extending the library.

including is convenient if you want the selection to include the end of the range:

(select #(0 1 2 3) (including 1 2))
                          ; => #(1 2), cf. (select ... (range 1 3))

nodrop is useful if you do not want to drop dimensions:

(select #(0 1 2 3) (nodrop 2))
                          ; => #(2), cf. (select ... (range 2 3))

head and tail do the obvious:

(select #(0 1 2 3) (head 2))            ; => #(0 1)
(select #(0 1 2 3) (tail 2))            ; => #(2 3)

All of these are trivial to implement. If there is something you are missing, you can easily extend select. Pull request are welcome.

ref is a version of select that always returns a single element, so it can only be used with singleton slices.

1.2 Select Semantics

Arguments of select, except the first one, are meant to be resolved using canonical-representation, in the select-dev package. If you want to extend select, you should define methods for canonical-representation. See the source code for the best examples. Below is a simple example that extends the semantics with ordinal numbers.

(defmacro define-ordinal-selection (number)
  (check-type number (integer 0))
  `(defmethod select-dev:canonical-representation
       ((axis integer)
	(slice (eql ',(intern (format nil \"~:@(~:r~)\" number)))))
     (assert (< ,number axis))
     (select-dev:canonical-singleton ,number)))

(define-ordinal-selection 1)
(define-ordinal-selection 2)
(define-ordinal-selection 3)

(select #(0 1 2 3 4 5) (range 'first 'third)) ; => #(1 2)

Note the following:

  • The value returned by canonical-representation needs to be constructed using canonical-singleton, canonical-range, or canonical-sequence. You should not use the internal representation directly as it is subject to change.
  • You can assume that axis is an integer: this is the default. An object may define a more complex mapping (such as, for example, named rows & columns), but unless a method specialized to that is found, canonical-representation will just query its dimension (with axis-dimension) and try to find a method that works on integers.
  • You need to make sure that the subscript is valid, hence the assertion.

2 Systems

The main system appears first, followed by any subsystem dependency.


2.1 select

DSL for array and data-frame slices

Long Name

Slicing for Data Frames

Author

Steve Nunez <steve@symbolics.tech>

Home Page

https://lisp-stat.dev/docs/manuals/select

Source Control

(GIT git://github.com/Lisp-Stat/select)

Bug Tracker

https://github.com/Lisp-Stat/select/issues/

License

MS-PL

Long Description

Select is a facility for selecting portions of sequences, arrays or data-frames. It provides:

An API for taking slices (elements selected by the Cartesian product of vectors of subscripts for each axis) of array-like objects. The most important function is ‘select‘. Unless you want to define additional methods for ‘select‘, this is pretty much all you need from this library. See the documentation at https://lisp-stat.github.io/select/ for a tutorial.

An extensible DSL for selecting a subset of valid subscripts. This is useful if, for example, you want to resolve column names in a data frame in your implementation of slice.

A set of utility functions for traversing slices in array-like objects.

Version

1.0.0

Dependencies
  • alexandria (system).
  • anaphora (system).
  • let-plus (system).
Source

select.asd.

Child Components

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


3.1 Lisp


3.1.1 select/select.asd

Source

select.asd.

Parent Component

select (system).

ASDF Systems

select.


3.1.2 select/package.lisp

Source

select.asd.

Parent Component

select (system).

Packages

3.1.3 select/select-dev.lisp

Dependency

package.lisp (file).

Source

select.asd.

Parent Component

select (system).

Public Interface
Internals

3.1.4 select/select.lisp

Dependency

select-dev.lisp (file).

Source

select.asd.

Parent Component

select (system).

Public Interface
Internals

4 Packages

Packages are listed by definition order.


4.1 select

SELECT is a facility for selecting portions of sequences or arrays.

Source

package.lisp.

Nickname

slct

Use List
  • alexandria.
  • anaphora.
  • common-lisp.
  • let-plus.
  • select-dev.
Used By List
  • data-frame.
  • lisp-stat.
  • num-utils.matrix.
Public Interface
Internals

4.2 select-dev

SELECT-DEV is used to implement SELECT operations on data structures other than arrays.

Source

package.lisp.

Use List
  • alexandria.
  • anaphora.
  • common-lisp.
  • let-plus.
Used By List
Public Interface
Internals

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


5.1 Public Interface


5.1.1 Macros

Macro: traverse-representations ((subscripts representations &key index setup) &body body)

Loops over all possible subscripts in REPRESENTAITONS, making them available in SUBSCRIPTS during the execution of BODY. The iterator is constructed using the function SETUP (see for example ROW-MAJOR-SETUP). When INDEX is given, a variable with that name is provided, containing an index that counts iterations.

Package

select-dev.

Source

select-dev.lisp.


5.1.2 Ordinary functions

Function: all-singleton-representations? (representations)

Test if all canonical representations are singletons.

Package

select-dev.

Source

select-dev.lisp.

Function: canonical-range (start end)

Canonical representation of a contiguous set of array indices from START (inclusive) to END (exclusive).

Package

select-dev.

Source

select-dev.lisp.

Function: canonical-representations (axes selections)

Return the canonical representations of SELECTIONS given the corresponding AXES, checking for matching length.

Package

select-dev.

Source

select-dev.lisp.

Function: canonical-sequence (sequence)

Canonical representation of array indexes from canonical-sequence SEQUENCE.

May share structure. Vectors of the upgraded type of (SIMPLE-ARRAY ARRAY-INDEX (*)) are preferred for efficiency, otherwise they are coerced.

Package

select-dev.

Source

select-dev.lisp.

Function: canonical-singleton (index)

Canonical representation of a singleton index (a nonnegative integer, which is a valid array index).

Package

select-dev.

Source

select-dev.lisp.

Function: column-major-setup (representations terminator)

Return SUBSCRIPTS (a list) and ITERATOR (a closure, no arguments) that increments the contents of SUBSCRIPTS in column-major order. TERMINATOR is called when all subscripts have been visited.

Package

select-dev.

Source

select-dev.lisp.

Function: including (start end)

Range, including both ends.

Package

select.

Source

select.lisp.

Function: nodrop (index)

Select a single index, but do not drop a dimension.

Package

select.

Source

select.lisp.

Function: range (start end)

Range, including START, excluding END.

Package

select.

Source

select.lisp.

Function: representation-dimensions (representations)

Return a list for the dimensions of canonical representations, dropping singletons.

Package

select-dev.

Source

select-dev.lisp.

Function: row-major-setup (representations terminator)

Return SUBSCRIPTS (a list) and ITERATOR (a closure, no arguments) that increments the contents of SUBSCRIPTS in row-major order. TERMINATOR is called when all subscripts have been visited.

Package

select-dev.

Source

select-dev.lisp.

Function: select-reserved-symbol? (symbol)

Test if SYMBOL has special semantics for SELECTION.

Package

select-dev.

Source

select-dev.lisp.

Function: singleton-representation? (representation)

Test if a canonical REPRESENTATION is a singleton.

Package

select-dev.

Source

select-dev.lisp.


5.1.3 Generic functions

Generic Function: axis-dimension (axis)

Return the dimension of axis. Needs to be defined for non-integer axes.

Package

select-dev.

Source

select-dev.lisp.

Generic Function: canonical-representation (axis selection)

Canonical representation of SELECTION, given information in AXIS. The default methods use dimensions as AXIS.

Each selection needs to be resolved into a canonical representation, which is either a singleton, a range, or a sequence of subscripts. They should only be constructed with the corresponding CANONICAL-SINGLETION, CANONICAL-RANGE and CANONICAL-SEQUENCE functions.

@c(CANONICAL-REPRESENTATION) needs to ensure that the represented subscripts are valid for the axis.

Unless a specialized method is found, the dimension of the axis is queried with AXIS-DIMENSION and resolution is attempted using the latter. Methods that resolve symbols should test them with SELECT-RESERVED-SYMBOL? and use CALL-NEXT-METHOD.

Package

select-dev.

Source

select-dev.lisp.

Methods
Method: canonical-representation (axis (selection nodrop))

The canonical representation for NODROP.

Source

select.lisp.

Method: canonical-representation (axis (selection range))

The canonical representation for RANGE.

Source

select.lisp.

Method: canonical-representation (axis (selection including))

The canonical representation for INCLUDING.

Source

select.lisp.

Method: canonical-representation (axis selection)
Method: canonical-representation (axis (canonical-range canonical-range))
Method: canonical-representation (axis (canonical-sequence canonical-sequence))
Method: canonical-representation ((axis integer) (slice null))
Method: canonical-representation ((axis integer) (selection integer))
Method: canonical-representation (axis (selection sequence))
Method: canonical-representation ((axis integer) (selection (eql t)))
Method: canonical-representation (axis (selection bit-vector))
Generic Function: mask (sequence predicate)

Map sequence into a simple-bit-vector, using 1 when PREDICATE yields true, 0 otherwise.

Package

select.

Source

select.lisp.

Methods
Method: mask ((sequence sequence) predicate)
Generic Function: ref (object &rest subscripts)

Return the element of OBJECT specified by SUBSCRIPTS.

Package

select.

Source

select.lisp.

Methods
Method: ref ((array array) &rest subscripts)
Generic Function: (setf ref) (object &rest subscripts)

Stores VALUE into the place specified by SUBSCRIPTS.

Package

select.

Source

select.lisp.

Methods
Method: (setf ref) ((array array) &rest subscripts)
Generic Function: select (object &rest selections)

Return the slices of OBJECT specified by SELECTIONS.

Package

select.

Source

select.lisp.

Methods
Method: select ((lst list) &rest selections)

Select from LST the subscripts or range specified in SELECTIONS. SELECTIONS must be a VECTOR, LIST or RANGE.

Method: select ((array array) &rest selections)

Return the SELECTIONS in the given ARRAY.

Generic Function: (setf select) (object &rest selections)

Stores VALUES into the locations given by SELECTIONS.

Package

select.

Source

select.lisp.

Methods
Method: (setf select) ((array array) &rest selections)
Method: (setf select) ((array array) &rest selections)
Generic Function: which (sequence &key predicate)

Return an index of the positions in SEQUENCE which satisfy PREDICATE. Defaults to return non-NIL indices.

Package

select.

Source

select.lisp.

Methods
Method: which ((sequence sequence) &key predicate)

5.1.4 Structures

Structure: canonical-range

Canonical representation of a contiguous set of array indices from START (inclusive) to END (exclusive).

Package

select-dev.

Source

select-dev.lisp.

Direct superclasses

structure-object.

Direct methods

canonical-representation.

Direct slots
Slot: start
Type

alexandria:array-index

Readers

canonical-range-start.

Writers

(setf canonical-range-start).

Slot: end
Type

alexandria:array-index

Readers

canonical-range-end.

Writers

(setf canonical-range-end).

Structure: canonical-sequence

Canonical representation of a sequence of array indexes.

Package

select-dev.

Source

select-dev.lisp.

Direct superclasses

structure-object.

Direct methods

canonical-representation.

Direct slots
Slot: vector
Package

common-lisp.

Type

(simple-array alexandria:array-index (*))

Readers

canonical-sequence-vector.

Writers

(setf canonical-sequence-vector).

Structure: including

Range, including both ends.

Package

select.

Source

select.lisp.

Direct superclasses

structure-object.

Direct methods

canonical-representation.

Direct slots
Slot: start
Readers

including-start.

Writers

(setf including-start).

Slot: end
Readers

including-end.

Writers

(setf including-end).

Structure: nodrop

Select a single index, but don’t drop a dimension.

Package

select.

Source

select.lisp.

Direct superclasses

structure-object.

Direct methods

canonical-representation.

Direct slots
Slot: index
Readers

nodrop-index.

Writers

(setf nodrop-index).

Structure: range

Range, including start, excluding end.

Package

select.

Source

select.lisp.

Direct superclasses

structure-object.

Direct methods

canonical-representation.

Direct slots
Slot: start
Readers

range-start.

Writers

(setf range-start).

Slot: end
Readers

range-end.

Writers

(setf range-end).


5.2 Internals


5.2.1 Ordinary functions

Reader: canonical-range-end (instance)
Writer: (setf canonical-range-end) (instance)
Package

select-dev.

Source

select-dev.lisp.

Target Slot

end.

Function: canonical-range-p (object)
Package

select-dev.

Source

select-dev.lisp.

Reader: canonical-range-start (instance)
Writer: (setf canonical-range-start) (instance)
Package

select-dev.

Source

select-dev.lisp.

Target Slot

start.

Function: canonical-sequence-p (object)
Package

select-dev.

Source

select-dev.lisp.

Reader: canonical-sequence-vector (instance)
Writer: (setf canonical-sequence-vector) (instance)
Package

select-dev.

Source

select-dev.lisp.

Target Slot

vector.

Function: copy-canonical-range (instance)
Package

select-dev.

Source

select-dev.lisp.

Function: copy-canonical-sequence (instance)
Package

select-dev.

Source

select-dev.lisp.

Function: copy-including (instance)
Package

select.

Source

select.lisp.

Function: copy-nodrop (instance)
Package

select.

Source

select.lisp.

Function: copy-range (instance)
Package

select.

Source

select.lisp.

Reader: including-end (instance)
Writer: (setf including-end) (instance)
Package

select.

Source

select.lisp.

Target Slot

end.

Function: including-p (object)
Package

select.

Source

select.lisp.

Reader: including-start (instance)
Writer: (setf including-start) (instance)
Package

select.

Source

select.lisp.

Target Slot

start.

Function: make-canonical-range (&key start end)
Package

select-dev.

Source

select-dev.lisp.

Function: make-canonical-sequence (&key vector)
Package

select-dev.

Source

select-dev.lisp.

Function: make-including (&key start end)
Package

select.

Source

select.lisp.

Function: make-nodrop (&key index)
Package

select.

Source

select.lisp.

Function: make-range (&key start end)
Package

select.

Source

select.lisp.

Reader: nodrop-index (instance)
Writer: (setf nodrop-index) (instance)
Package

select.

Source

select.lisp.

Target Slot

index.

Function: nodrop-p (object)
Package

select.

Source

select.lisp.

Reader: range-end (instance)
Writer: (setf range-end) (instance)
Package

select.

Source

select.lisp.

Target Slot

end.

Function: range-p (object)
Package

select.

Source

select.lisp.

Reader: range-start (instance)
Writer: (setf range-start) (instance)
Package

select.

Source

select.lisp.

Target Slot

start.

Function: representation-dimension (representation)

Return the dimension of a canonical-representation, or NIL for singleton selections (they are dropped).

Package

select-dev.

Source

select-dev.lisp.

Function: representation-initial-value (representation)

Initial value for iteration.

Package

select-dev.

Source

select-dev.lisp.

Function: representation-iterator (representation carry cons)

Return a closure that sets the car of CONS to the next value each time it is called, resetting and calling CARRY when it reaches the end of its range.

Package

select-dev.

Source

select-dev.lisp.


6 Conclusion

select was originally called slice and written by Tamas K. Papp. Since it was abandoned in 2017, I have taken it over to be part of a rebooted Common Lisp statistics library. Changes in this version include:

Documentation Improvements

Test Improvements

Enhancements

Bug Fixes


Appendix A Indexes


A.1 Concepts


A.2 Functions

Jump to:   (  
A   C   F   G   I   M   N   R   S   T   W  
Index Entry  Section

(
(setf canonical-range-end): Private ordinary functions
(setf canonical-range-start): Private ordinary functions
(setf canonical-sequence-vector): Private ordinary functions
(setf including-end): Private ordinary functions
(setf including-start): Private ordinary functions
(setf nodrop-index): Private ordinary functions
(setf range-end): Private ordinary functions
(setf range-start): Private ordinary functions
(setf ref): Public generic functions
(setf ref): Public generic functions
(setf select): Public generic functions
(setf select): Public generic functions
(setf select): Public generic functions

A
all-singleton-representations?: Public ordinary functions
axis-dimension: Public generic functions

C
canonical-range: Public ordinary functions
canonical-range-end: Private ordinary functions
canonical-range-p: Private ordinary functions
canonical-range-start: Private ordinary functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representation: Public generic functions
canonical-representations: Public ordinary functions
canonical-sequence: Public ordinary functions
canonical-sequence-p: Private ordinary functions
canonical-sequence-vector: Private ordinary functions
canonical-singleton: Public ordinary functions
column-major-setup: Public ordinary functions
copy-canonical-range: Private ordinary functions
copy-canonical-sequence: Private ordinary functions
copy-including: Private ordinary functions
copy-nodrop: Private ordinary functions
copy-range: Private ordinary functions

F
Function, (setf canonical-range-end): Private ordinary functions
Function, (setf canonical-range-start): Private ordinary functions
Function, (setf canonical-sequence-vector): Private ordinary functions
Function, (setf including-end): Private ordinary functions
Function, (setf including-start): Private ordinary functions
Function, (setf nodrop-index): Private ordinary functions
Function, (setf range-end): Private ordinary functions
Function, (setf range-start): Private ordinary functions
Function, all-singleton-representations?: Public ordinary functions
Function, canonical-range: Public ordinary functions
Function, canonical-range-end: Private ordinary functions
Function, canonical-range-p: Private ordinary functions
Function, canonical-range-start: Private ordinary functions
Function, canonical-representations: Public ordinary functions
Function, canonical-sequence: Public ordinary functions
Function, canonical-sequence-p: Private ordinary functions
Function, canonical-sequence-vector: Private ordinary functions
Function, canonical-singleton: Public ordinary functions
Function, column-major-setup: Public ordinary functions
Function, copy-canonical-range: Private ordinary functions
Function, copy-canonical-sequence: Private ordinary functions
Function, copy-including: Private ordinary functions
Function, copy-nodrop: Private ordinary functions
Function, copy-range: Private ordinary functions
Function, including: Public ordinary functions
Function, including-end: Private ordinary functions
Function, including-p: Private ordinary functions
Function, including-start: Private ordinary functions
Function, make-canonical-range: Private ordinary functions
Function, make-canonical-sequence: Private ordinary functions
Function, make-including: Private ordinary functions
Function, make-nodrop: Private ordinary functions
Function, make-range: Private ordinary functions
Function, nodrop: Public ordinary functions
Function, nodrop-index: Private ordinary functions
Function, nodrop-p: Private ordinary functions
Function, range: Public ordinary functions
Function, range-end: Private ordinary functions
Function, range-p: Private ordinary functions
Function, range-start: Private ordinary functions
Function, representation-dimension: Private ordinary functions
Function, representation-dimensions: Public ordinary functions
Function, representation-initial-value: Private ordinary functions
Function, representation-iterator: Private ordinary functions
Function, row-major-setup: Public ordinary functions
Function, select-reserved-symbol?: Public ordinary functions
Function, singleton-representation?: Public ordinary functions

G
Generic Function, (setf ref): Public generic functions
Generic Function, (setf select): Public generic functions
Generic Function, axis-dimension: Public generic functions
Generic Function, canonical-representation: Public generic functions
Generic Function, mask: Public generic functions
Generic Function, ref: Public generic functions
Generic Function, select: Public generic functions
Generic Function, which: Public generic functions

I
including: Public ordinary functions
including-end: Private ordinary functions
including-p: Private ordinary functions
including-start: Private ordinary functions

M
Macro, traverse-representations: Public macros
make-canonical-range: Private ordinary functions
make-canonical-sequence: Private ordinary functions
make-including: Private ordinary functions
make-nodrop: Private ordinary functions
make-range: Private ordinary functions
mask: Public generic functions
mask: Public generic functions
Method, (setf ref): Public generic functions
Method, (setf select): Public generic functions
Method, (setf select): Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, canonical-representation: Public generic functions
Method, mask: Public generic functions
Method, ref: Public generic functions
Method, select: Public generic functions
Method, select: Public generic functions
Method, which: Public generic functions

N
nodrop: Public ordinary functions
nodrop-index: Private ordinary functions
nodrop-p: Private ordinary functions

R
range: Public ordinary functions
range-end: Private ordinary functions
range-p: Private ordinary functions
range-start: Private ordinary functions
ref: Public generic functions
ref: Public generic functions
representation-dimension: Private ordinary functions
representation-dimensions: Public ordinary functions
representation-initial-value: Private ordinary functions
representation-iterator: Private ordinary functions
row-major-setup: Public ordinary functions

S
select: Public generic functions
select: Public generic functions
select: Public generic functions
select-reserved-symbol?: Public ordinary functions
singleton-representation?: Public ordinary functions

T
traverse-representations: Public macros

W
which: Public generic functions
which: Public generic functions

Jump to:   (  
A   C   F   G   I   M   N   R   S   T   W