The Select Reference Manual

Table of Contents

The Select Reference Manual

This is the Select Reference Manual, version 1.0.

1 Introduction

Select is a library for taking slices from array-like objects. The most frequently used form is:

(select object selection1 selection2 ...)

where each selection specifies a set of subscripts along the corresponding axis. The selection specifications are found below.

1.1 Selection Specifiers

Selecting Single Values

A non-negative integer selects the corresponding index, while a negative integer selects an index counting backwards from the last index. For example:

(select #(0 1 2 3) 1)                  ; => 1
(select #(0 1 2 3) -2)                 ; => 2

These are called singleton slices. Each singleton slice drops the dimension: vectors become atoms, matrices become vectors, etc..

Selecting Ranges

(range start end) selects subscripts i where start <= i < end. When end is nil, the last index is included (cf. subseq). Each boundary is resolved according to the other rules if applicable, so you can use negative integers:

(select #(0 1 2 3) (range 1 3))         ; => #(1 2)
(select #(0 1 2 3) (range 1 -1))        ; => #(1 2)

Selecting All Subscripts of a Dimension

t selects all subscripts:

(select #2A((0 1 2)
            (3 4 5))
         t 1)                           ; => #(1 4)

Selecting with a Sequence

Sequences can be used to make specific selections from the object. For example:

(select #(0 1 2 3 4 5 6 7 8 9)
	(vector (range 1 3) 6 (range -2 -1))) ; => #(1 2 3 6 8 9)
(select #(0 1 2) '(2 2 1 0 0))                ; => #(2 2 1 0 0)

Using Bit Vectors as a Mask

Bit vectors can be used to select elements of arrays and sequences as well:

(select #(0 1 2 3 4) #*00110)          ; => #(2 3)

1.1.1 Extensions

Section 1.1 describes the core functionality. The semantics can be extended, as you will see in the next section. The extensions in this section are provided by the library and prove useful in practice. Their implementation provides good examples of extending the library.

including is convenient if you want the selection to include the end of the range:

(select #(0 1 2 3) (including 1 2))
                          ; => #(1 2), cf. (select ... (range 1 3))

nodrop is useful if you do not want to drop dimensions:

(select #(0 1 2 3) (nodrop 2))
                          ; => #(2), cf. (select ... (range 2 3))

head and tail do the obvious:

(select #(0 1 2 3) (head 2))            ; => #(0 1)
(select #(0 1 2 3) (tail 2))            ; => #(2 3)

All of these are trivial to implement. If there is something you are missing, you can easily extend select. Pull request are welcome.

ref is a version of select that always returns a single element, so it can only be used with singleton slices.

1.2 Select Semantics

Arguments of select, except the first one, are meant to be resolved using canonical-representation, in the select-dev package. If you want to extend select, you should define methods for canonical-representation. See the source code for the best examples. Below is a simple example that extends the semantics with ordinal numbers.

(defmacro define-ordinal-selection (number)
  (check-type number (integer 0))
  `(defmethod select-dev:canonical-representation
       ((axis integer)
	(slice (eql ',(intern (format nil "~:@(~:r~)" number)))))
     (assert (< ,number axis))
     (select-dev:canonical-singleton ,number)))

(define-ordinal-selection 1)
(define-ordinal-selection 2)
(define-ordinal-selection 3)

(select #(0 1 2 3 4 5) (range 'first 'third)) ; => #(1 2)

Note the following:

• The value returned by canonical-representation needs to be constructed using canonical-singleton, canonical-range, or canonical-sequence. You should not use the internal representation directly as it is subject to change.
• You can assume that axis is an integer: this is the default. An object may define a more complex mapping (such as, for example, named rows & columns), but unless a method specialized to that is found, canonical-representation will just query its dimension (with axis-dimension) and try to find a method that works on integers.
• You need to make sure that the subscript is valid, hence the assertion.

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 select

Author

Steve Nunez

Home Page

https://symbolics.github.io/select/

Source Control

(:git "git://github.com/symbolics/select")

Bug Tracker

https://github.com/Symbolics/select/issues/

License

MS-PL

Description

DSL for array slices.

Long Description

Select is a facility for selecting portions of sequences or arrays. It
provides:

An API for taking slices (elements selected by the Cartesian product of vectors of subscripts for each axis) of array-like objects. The most important function is ‘select‘. Unless you want to define additional methods for ‘select‘, this is pretty much all you need from this library. See the documentation at https://symbolics.github.io/select/ for a tutorial.

An extensible DSL for selecting a subset of valid subscripts. This is useful if, for example, you want to resolve column names in a data frame in your implementation of slice.

A set of utility functions for traversing slices in array-like objects.

Version

1.0

Dependencies

• alexandria
• anaphora
• let-plus

Source

select.asd (file)

Directory

s:/src/select/

Components

• package.lisp (file)
• select-dev.lisp (file)
• select.lisp (file)

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

3.1 Lisp

3.1.1 select.asd

Location

select.asd

Systems

select (system)

3.1.2 select/package.lisp

Parent

select (system)

Location

package.lisp

Packages

slct

3.1.3 select/select-dev.lisp

Dependency

package.lisp (file)

Parent

select (system)

Location

select-dev.lisp

Exported Definitions

• select-reserved-symbol? (function)
• singleton-representation? (function)

Internal Definitions

• all-singleton-representations? (function)
• axis-dimension (generic function)
• canonical-range (function)
• canonical-range (structure)
• canonical-range-end (function)
• (setf canonical-range-end) (function)
• canonical-range-p (function)
• canonical-range-start (function)
• (setf canonical-range-start) (function)
• canonical-representation (generic function)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representations (function)
• canonical-sequence (function)
• canonical-sequence (structure)
• canonical-sequence-p (function)
• canonical-sequence-vector (function)
• (setf canonical-sequence-vector) (function)
• canonical-singleton (function)
• column-major-setup (function)
• copy-canonical-range (function)
• copy-canonical-sequence (function)
• make-canonical-range (function)
• make-canonical-sequence (function)
• representation-dimension (function)
• representation-dimensions (function)
• representation-initial-value (function)
• representation-iterator (function)
• row-major-setup (function)
• traverse-representations (macro)

3.1.4 select/select.lisp

Dependency

select-dev.lisp (file)

Parent

select (system)

Location

select.lisp

Exported Definitions

• head (function)
• including (function)
• including (structure)
• mask (generic function)
• mask (method)
• nodrop (function)
• nodrop (structure)
• range (function)
• range (structure)
• ref (generic function)
• ref (method)
• (setf ref) (method)
• (setf ref) (generic function)
• select (generic function)
• select (method)
• select (method)
• (setf select) (method)
• (setf select) (generic function)
• tail (function)
• which (generic function)
• which (method)

Internal Definitions

• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• copy-including (function)
• copy-nodrop (function)
• copy-range (function)
• including-end (function)
• (setf including-end) (function)
• including-p (function)
• including-start (function)
• (setf including-start) (function)
• make-including (function)
• make-nodrop (function)
• make-range (function)
• nodrop-index (function)
• (setf nodrop-index) (function)
• nodrop-p (function)
• range-end (function)
• (setf range-end) (function)
• range-p (function)
• range-start (function)
• (setf range-start) (function)

4 Packages

Packages are listed by definition order.

4.1 slct

SELECT is a facility for selecting portions of sequences or arrays.

Source

package.lisp (file)

Nickname

select

Use List

• let-plus
• anaphora
• alexandria.1.0.0
• common-lisp

Exported Definitions

• head (function)
• including (function)
• including (structure)
• mask (generic function)
• mask (method)
• nodrop (function)
• nodrop (structure)
• range (function)
• range (structure)
• ref (generic function)
• ref (method)
• (setf ref) (method)
• (setf ref) (generic function)
• select (generic function)
• select (method)
• select (method)
• (setf select) (method)
• (setf select) (generic function)
• select-reserved-symbol? (function)
• singleton-representation? (function)
• tail (function)
• which (generic function)
• which (method)

Internal Definitions

• all-singleton-representations? (function)
• axis-dimension (generic function)
• canonical-range (function)
• canonical-range (structure)
• canonical-range-end (function)
• (setf canonical-range-end) (function)
• canonical-range-p (function)
• canonical-range-start (function)
• (setf canonical-range-start) (function)
• canonical-representation (generic function)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representation (method)
• canonical-representations (function)
• canonical-sequence (function)
• canonical-sequence (structure)
• canonical-sequence-p (function)
• canonical-sequence-vector (function)
• (setf canonical-sequence-vector) (function)
• canonical-singleton (function)
• column-major-setup (function)
• copy-canonical-range (function)
• copy-canonical-sequence (function)
• copy-including (function)
• copy-nodrop (function)
• copy-range (function)
• including-end (function)
• (setf including-end) (function)
• including-p (function)
• including-start (function)
• (setf including-start) (function)
• make-canonical-range (function)
• make-canonical-sequence (function)
• make-including (function)
• make-nodrop (function)
• make-range (function)
• nodrop-index (function)
• (setf nodrop-index) (function)
• nodrop-p (function)
• range-end (function)
• (setf range-end) (function)
• range-p (function)
• range-start (function)
• (setf range-start) (function)
• representation-dimension (function)
• representation-dimensions (function)
• representation-initial-value (function)
• representation-iterator (function)
• row-major-setup (function)
• traverse-representations (macro)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Functions

Function: head COUNT

First COUNT indexes.

Package

slct

Source

select.lisp (file)

Function: including START END

Range, including both ends.

Package

slct

Source

select.lisp (file)

Function: nodrop INDEX

Select a single index, but do not drop a dimension.

Package

slct

Source

select.lisp (file)

Function: range START END

Range, including START, excluding END.

Package

slct

Source

select.lisp (file)

Function: select-reserved-symbol? SYMBOL

Test if SYMBOL has special semantics for SELECTION.

Package

slct

Source

select-dev.lisp (file)

Function: singleton-representation? REPRESENTATION

Test if a canonical REPRESENTATION is a singleton.

Package

slct

Source

select-dev.lisp (file)

Function: tail COUNT

Last COUNT indexes.

Package

slct

Source

select.lisp (file)

5.1.2 Generic functions

Generic Function: mask PREDICATE SEQUENCE

Map sequence into a simple-bit-vector, using 1 when PREDICATE yields true, 0 otherwise.

Package

slct

Source

select.lisp (file)

Methods

Method: mask PREDICATE (SEQUENCE sequence)

Generic Function: ref OBJECT &rest SUBSCRIPTS

Return the element of OBJECT specified by SUBSCRIPTS.

Package

slct

Source

select.lisp (file)

Writer

(setf ref) (generic function)

Methods

Method: ref (ARRAY array) &rest SUBSCRIPTS

Generic Function: (setf ref) VALUE OBJECT &rest SUBSCRIPTS

Stores VALUE into the place specified by SUBSCRIPTS.

Package

slct

Source

select.lisp (file)

Reader

ref (generic function)

Methods

Method: (setf ref) VALUE (ARRAY array) &rest SUBSCRIPTS

Generic Function: select OBJECT &rest SELECTIONS

Return the slices of OBJECT specified by SELECTIONS.

Package

slct

Source

select.lisp (file)

Writer

(setf select) (generic function)

Methods

Method: select (LST list) &rest SELECTIONS

Select from LST the subscripts or range specified in SELECTIONS. SELECTIONS must be a VECTOR, LIST or RANGE.

Method: select (ARRAY array) &rest SELECTIONS

Return the SELECTIONS in the given ARRAY.

Generic Function: (setf select) VALUE OBJECT &rest SELECTIONS

Stores VALUES into the locations given by SELECTIONS.

Package

slct

Source

select.lisp (file)

Reader

select (generic function)

Methods

Method: (setf select) VALUE (ARRAY array) &rest SELECTIONS

Generic Function: which PREDICATE SEQUENCE

Return an index of the positions in SEQUENCE which satisfy PREDICATE.

Package

slct

Source

select.lisp (file)

Methods

Method: which PREDICATE (SEQUENCE sequence)

5.1.3 Structures

Structure: including ()

Range, including both ends.

Package

slct

Source

select.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

canonical-representation (method)

Direct slots

Slot: start

Readers

including-start (function)

Writers

(setf including-start) (function)

Slot: end

Readers

including-end (function)

Writers

(setf including-end) (function)

Structure: nodrop ()

Select a single index, but don’t drop a dimension.

Package

slct

Source

select.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

canonical-representation (method)

Direct slots

Slot: index

Readers

nodrop-index (function)

Writers

(setf nodrop-index) (function)

Structure: range ()

Range, including start, excluding end.

Package

slct

Source

select.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

canonical-representation (method)

Direct slots

Slot: start

Readers

range-start (function)

Writers

(setf range-start) (function)

Slot: end

Readers

range-end (function)

Writers

(setf range-end) (function)

5.2 Internal definitions

5.2.1 Macros

Macro: traverse-representations (SUBSCRIPTS REPRESENTATIONS &key INDEX SETUP) &body BODY

Loops over all possible subscripts in REPRESENTAITONS, making them available in SUBSCRIPTS during the execution of BODY. The iterator is constructed using the function SETUP (see for example ROW-MAJOR-SETUP). When INDEX is given, a variable with that name is provided, containing an index that counts iterations.

Package

slct

Source

select-dev.lisp (file)

5.2.2 Functions

Function: all-singleton-representations? REPRESENTATIONS

Test if all canonical representations are singletons.

Package

slct

Source

select-dev.lisp (file)

Function: canonical-range START END

Canonical representation of a contiguous set of array indices from START (inclusive) to END (exclusive).

Package

slct

Source

select-dev.lisp (file)

Function: canonical-range-end INSTANCE

Function: (setf canonical-range-end) VALUE INSTANCE

Package

slct

Source

select-dev.lisp (file)

Function: canonical-range-p OBJECT

Package

slct

Source

select-dev.lisp (file)

Function: canonical-range-start INSTANCE

Function: (setf canonical-range-start) VALUE INSTANCE

Package

slct

Source

select-dev.lisp (file)

Function: canonical-representations AXES SELECTIONS

Return the canonical representations of SELECTIONS given the corresponding AXES, checking for matching length.

Package

slct

Source

select-dev.lisp (file)

Function: canonical-sequence SEQUENCE

Canonical representation of array indexes from canonical-sequence SEQUENCE.

May share structure. Vectors of the upgraded type of (SIMPLE-ARRAY ARRAY-INDEX (*)) are preferred for efficiency, otherwise they are coerced.

Package

slct

Source

select-dev.lisp (file)

Function: canonical-sequence-p OBJECT

Package

slct

Source

select-dev.lisp (file)

Function: canonical-sequence-vector INSTANCE

Function: (setf canonical-sequence-vector) VALUE INSTANCE

Package

slct

Source

select-dev.lisp (file)

Function: canonical-singleton INDEX

Canonical representation of a singleton index (a nonnegative integer, which is a valid array index).

Package

slct

Source

select-dev.lisp (file)

Function: column-major-setup REPRESENTATIONS TERMINATOR

Return SUBSCRIPTS (a list) and ITERATOR (a closure, no arguments) that increments the contents of SUBSCRIPTS in column-major order. TERMINATOR is called when all subscripts have been visited.

Package

slct

Source

select-dev.lisp (file)

Function: copy-canonical-range INSTANCE

Package

slct

Source

select-dev.lisp (file)

Function: copy-canonical-sequence INSTANCE

Package

slct

Source

select-dev.lisp (file)

Function: copy-including INSTANCE

Package

slct

Source

select.lisp (file)

Function: copy-nodrop INSTANCE

Package

slct

Source

select.lisp (file)

Function: copy-range INSTANCE

Package

slct

Source

select.lisp (file)

Function: including-end INSTANCE

Function: (setf including-end) VALUE INSTANCE

Package

slct

Source

select.lisp (file)

Function: including-p OBJECT

Package

slct

Source

select.lisp (file)

Function: including-start INSTANCE

Function: (setf including-start) VALUE INSTANCE

Package

slct

Source

select.lisp (file)

Function: make-canonical-range &key (START START) (END END)

Package

slct

Source

select-dev.lisp (file)

Function: make-canonical-sequence &key (VECTOR VECTOR)

Package

slct

Source

select-dev.lisp (file)

Function: make-including &key (START START) (END END)

Package

slct

Source

select.lisp (file)

Function: make-nodrop &key (INDEX INDEX)

Package

slct

Source

select.lisp (file)

Function: make-range &key (START START) (END END)

Package

slct

Source

select.lisp (file)

Function: nodrop-index INSTANCE

Function: (setf nodrop-index) VALUE INSTANCE

Package

slct

Source

select.lisp (file)

Function: nodrop-p OBJECT

Package

slct

Source

select.lisp (file)

Function: range-end INSTANCE

Function: (setf range-end) VALUE INSTANCE

Package

slct

Source

select.lisp (file)

Function: range-p OBJECT

Package

slct

Source

select.lisp (file)

Function: range-start INSTANCE

Function: (setf range-start) VALUE INSTANCE

Package

slct

Source

select.lisp (file)

Function: representation-dimension REPRESENTATION

Return the dimension of a canonical-representation, or NIL for singleton selections (they are dropped).

Package

slct

Source

select-dev.lisp (file)

Function: representation-dimensions REPRESENTATIONS

Return a list for the dimensions of canonical representations, dropping singletons.

Package

slct

Source

select-dev.lisp (file)

Function: representation-initial-value REPRESENTATION

Initial value for iteration.

Package

slct

Source

select-dev.lisp (file)

Function: representation-iterator REPRESENTATION CARRY CONS

Return a closure that sets the car of CONS to the next value each time it is called, resetting and calling CARRY when it reaches the end of its range.

Package

slct

Source

select-dev.lisp (file)

Function: row-major-setup REPRESENTATIONS TERMINATOR

Return SUBSCRIPTS (a list) and ITERATOR (a closure, no arguments) that increments the contents of SUBSCRIPTS in row-major order. TERMINATOR is called when all subscripts have been visited.

Package

slct

Source

select-dev.lisp (file)

5.2.3 Generic functions

Generic Function: axis-dimension AXIS

Return the dimension of axis. Needs to be defined for non-integer axes.

Package

slct

Source

select-dev.lisp (file)

Generic Function: canonical-representation AXIS SELECTION

Canonical representation of SELECTION, given information in AXIS. The default methods use dimensions as AXIS.

Each selection needs to be resolved into a canonical representation, which is either a singleton, a range, or a sequence of subscripts. They should only be constructed with the corresponding CANONICAL-SINGLETION, CANONICAL-RANGE and CANONICAL-SEQUENCE functions.

@c(CANONICAL-REPRESENTATION) needs to ensure that the represented subscripts are valid for the axis.

Unless a specialized method is found, the dimension of the axis is queried with AXIS-DIMENSION and resolution is attempted using the latter. Methods that resolve symbols should test them with SELECT-RESERVED-SYMBOL? and use CALL-NEXT-METHOD.

Package

slct

Source

select-dev.lisp (file)

Methods

Method: canonical-representation AXIS (SELECTION nodrop)

The canonical representation for NODROP.

Source

select.lisp (file)

Method: canonical-representation AXIS (SELECTION range)

The canonical representation for RANGE.

Source

select.lisp (file)

Method: canonical-representation AXIS (SELECTION including)

The canonical representation for INCLUDING.

Source

select.lisp (file)

Method: canonical-representation AXIS SELECTION

Method: canonical-representation AXIS (CANONICAL-RANGE canonical-range)

Method: canonical-representation AXIS (CANONICAL-SEQUENCE canonical-sequence)

Method: canonical-representation (AXIS integer) (SLICE null)

Method: canonical-representation (AXIS integer) (SELECTION integer)

Method: canonical-representation AXIS (SELECTION sequence)

Method: canonical-representation (AXIS integer) (SELECTION (eql t))

Method: canonical-representation AXIS (SELECTION bit-vector)

5.2.4 Structures

Structure: canonical-range ()

Canonical representation of a contiguous set of array indices from START (inclusive) to END (exclusive).

Package

slct

Source

select-dev.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

canonical-representation (method)

Direct slots

Slot: start

Type

alexandria.1.0.0:array-index

Readers

canonical-range-start (function)

Writers

(setf canonical-range-start) (function)

Slot: end

Type

alexandria.1.0.0:array-index

Readers

canonical-range-end (function)

Writers

(setf canonical-range-end) (function)

Structure: canonical-sequence ()

Canonical representation of a sequence of array indexes.

Package

slct

Source

select-dev.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

canonical-representation (method)

Direct slots

Slot: vector

Type

(simple-array alexandria.1.0.0:array-index (*))

Readers

canonical-sequence-vector (function)

Writers

(setf canonical-sequence-vector) (function)

6 Conclusion

Select was originally called slice and written by Tamas K. Papp. Since it was abandoned in 2017, I have taken it over to be part of a rebooted Common Lisp statistics library. Changes in this version include:

Documentation Improvements

• Move to HTML based documentation system
• Docs now on github.io

Test Improvements

• Ported to FiveAM and refactored
• Improved test coverage
• Added failure messages to aid debugging
• Added tests for selection iteration

Enhancements

• Renamed to ’cons’ to ’range’
• Range now handles (range x x) => nil
• Selections work identically on sequences; previously differed between lists and vectors
• Selections may be specified using a list; previously could only be a vector
• Sequence selections now honor fill-pointer, if any

Bug Fixes

• Range now handles END = (length <sequence>)
• Selecting from a list no longer drops dimension

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables

A.4 Data types