This is the Data Frame I/O Reference Manual, version 1.0.0, generated automatically by Declt version 4.0b2.
Copyright © 2019-2022 Steve Nunez
Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.
Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the section entitled “Copying” is included exactly as in the original.
Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be translated as well.
This program is distributed under the terms of the Microsoft Public License.
The main system appears first, followed by any subsystem dependency.
Common Lisp library for reading and writing data-frames
Data frame I/O
Steve Nunez <steve@symbolics.tech>
(GIT https://github.com/Lisp-Stat/dfio.git)
MS-PL
DFIO is used to read and write data frames from string, streams or disk. When building a library like this, one question that needs to be answered is "where do you draw the line on formats?". For example, should DFIO read/write JSON? The answer to that question is ’no’, and the reason is that there are many JSON formats – they are applicaton specific. Application specific format should be read in an application specific package, and this is the case, for example, with Vega. Formats like CSV, HD5 or Feather are application agnostic and should be included in DFIO.
1.0.0
Files are sorted by type and then listed depth-first from the systems components trees.
pkgdcl.lisp (file).
dfio (system).
decimal.lisp (file).
dfio (system).
string-table.lisp (file).
dfio (system).
non-negative-integer (type).
data-column.lisp (file).
dfio (system).
utils.lisp (file).
dfio (system).
write.lisp (file).
dfio (system).
csv-to-data-columns (function).
Packages are listed by definition order.
lisp-stat.
dfio.
non-negative-integer (type).
Definitions are sorted by export status, category, package, and then by lexicographic order.
Return the counts.
Return the collected elements as a vector.
Parse a decimal rational in (subseq string start end) of the form [sign][whole][.[fraction]][exponent] where
sign ::= + | - | empty
whole ::= digit*
fraction ::= digit*
exponent ::= exponent-char[sign]digit+
with the restriction that WHOLE and FRACTION cannot be empty at the same time. EXPONENT-CHAR is a string and contains the valid exponent chars.
Whitespace is NOT trimmed, and leads to an error. In case of a parsing failure, PARSE-RATIONAL-ERROR is used.
Return (values NUMBER DECIMAL-DOT? EXPONENT-CHAR). NUMBER is a RATIONAL, DECIMAL-DOT? is T when a decimal dot is present, otherwise NIL, EXPONENT-CHAR contains the exponent character, NIL if not present.
Numbers of the form .112 and 112. are valid syntax, representing 0.112 and 112.0, respectively.
Examples:
(parse-rational "7") => (values 7 NIL NIL)
(parse-rational "7.") => (values 7 T NIL)
(parse-rational "0.7") => (values 7/10 T NIL)
(parse-rational ".7") => (values 7/10 T NIL)
(parse-rational "7.e2") => (values 700 T #e)
(parse-rational ".7d1") => (values 7 T #d)
Wrapper for PARSE-RATIONAL, converting non-integers to floats. The float type is determined by the -float arguments for each exponent character. Integers are not converted to floats. Return a single value, type of (or integer float).
See PARSE-RATIONAL for accepted formats, errors, etc.
Read a CSV file, stream, string or URL into a DATA-FRAME, which is returned.
When SKIP-FIRST-ROW?, the first row is read separately and COLUMN-KEYS-OR-FUNCTION is used to form column keys.
When COLUMN-KEYS-OR-FUNCTION is a sequence, it is used for column keys, regardless of the value of SKIP-FIRST-ROW?.
PACKAGE indicates the package to intern column names into.
MAP-ALIST maps values during the import. This is useful if you want special mappings for missing, though the mechanism is general. Returns two values, the data-frame and the source
dfio.
Save DF in the file named by PATHSPEC. By default, a suffix of .lisp is added.
dfio.
Add STRING mapped to VALUE to STRING-TABLE, raising STRING-TABLE-DUPLICATE if STRING is already in the table. Return VALUE.
Number of distinct strings in the table.
If STRING is already in STRING-TABLE, return its value, otherwise add it and return NEW-VALUE. When used with the default argument for NEW-VALUE, EQUAL strings are always mapped to values that are EQ.
Return the value corresponding to STRING in STRING-TABLE, or raise the STRING-TABLE-NOT-FOUND error.
List of strings in STRING-TABLE.
Map string to a keyword.
The current implementation replaces #. and #space with a #-, and upcases all other characters.
dfio.
Map STRING to a symbol in PACKAGE, replacing #., #_ and #space with a #-, and upcasing all other characters. Exports symbol.
dfio.
Map the symbol-name of S to something that can be part of a logical-pathname
dfio.
Write DF to STRING-OR-STREAM in CSV format. STRING-OR-STREAM can be a STREAM, a STRING or a file PATHSPEC.
Keywords:
string-or-stream: stream to write to. Default: nil, returning a string
add-first-row: add column names as the first row
separator: separator to use when reading or writing CSV files. A character. By default, a comma: #,
quote: quote character to use when reading or writing CSV files. A character. By default, a double-quote: #"
eol: line ending to use when writing CSV files. A string. By default, +CRLF+ as specified by creativyst.
Notes:
The :newline keyword requires a sequence, so use :newline ’(#newline)
dfio.
Write DATA-SYMBOL to STREAM in a format suitable for reading back in with the Lisp reader
dfio.
Write the variable PROPERTY strings to stream so they can be read back in when LOADed from a lisp file. By convention, the name of the function that sets the property is the same as the property. Example (write-property mtcars :label)
dfio.
Error used by parse-rational and parse-real.
String is already in the table.
error.
String not found in table.
error.
A table of distinct strings, optionally mapping each one to a value.
structure-object.
hash-table
(make-hash-table :test (function equalp))
list
symbol
:default-float-format
dfio.data-column::non-negative-integer
0
dfio.data-column::non-negative-integer
0
integer
0
integer
0
dfio.data-column::non-negative-integer
0
dfio.string-table:string-table
:map-table
dfio.data-column::non-negative-integer
0
dfio.string-table:string-table
(dfio.string-table:string-table)
External format used for opening files
dfio.
Default exponent characters.
creates a stream from the given thing, trying to DWIM
dfio.
Read a CSV file (or stream, or string), accumulate the values in DATA-COLUMNs, return a list of these. Rows are checked to have the same number of elements.
When SKIP-FIRST-ROW?, the first row is read separately and returned as the second value (list of strings), otherwise it is considered data like all other rows.
dfio.
If (SUBSEQ STRING START END) starts with a nonnegative integer (ie a sequence of digits 0-9), return the integer and position at which it ends as two values.
Otherwise, return NIL and 0.
START < END has to hold, END cannot be NIL. Consequences are undefined when START >= END.
Return (values SIGNUM INDEX), where SIGNUM is -1 or 1 depending on whether (CHAR STRING START) was a sign, and INDEX is the index of the subsequent character (START or START+1).
Synonym for GETHASH, used internally.
Synonym for (SETF GETHASH), used internally, checks that STRING is a string.
| Jump to: | %
(
C D F G M P R S W |
|---|
| Jump to: | %
(
C D F G M P R S W |
|---|
| Jump to: | *
+
D F I M R S T |
|---|
| Jump to: | *
+
D F I M R S T |
|---|
| Jump to: | C D F N P S T U W |
|---|
| Jump to: | C D F N P S T U W |
|---|