vlin
Dense matrices and vectors: construction, arithmetic, products, reductions, and in-place training kernels.
| Quick fact | Value |
|---|---|
| Module | vlin |
| Import | use external "vlin/vlin.vy"; |
| Depends on | vmath, vcolors |
| Status | Unstable — the public surface may change between releases. |
This page is generated from the library's in-repo manual (vyne/modules/external/vlin/README.md) so signatures and behaviour stay aligned with the source tree. Examples assume a desktop target unless the text says otherwise.
Matrix and vector linear algebra for Vyne.
On the name. Earlier drafts of this document referred to the library as vlinalg. The source has always used vlin — in the module vlin; declaration, in the mangled names, and in the use external path. This manual uses vlin exclusively. If you have call sites that write vlinalg.foo(...), they will not resolve.
Contents
- Overview
- Quick start
- Importing
- Module layout
- The
Matrixtype - The
Vectortype - Reference semantics and the memory model
- API reference
- Worked example
- Limitations
- Design notes
- Roadmap
- Version history
Overview
vlin is a pure-Vyne linear algebra library. It provides a small, predictable set of matrix and vector operations sufficient to build and prototype dense neural networks, solve small systems of linear equations, and drive numerical examples directly from Vyne source.
The library deliberately omits the features that make NumPy and BLAS fast — broadcasting, strided views, SIMD kernels, integer matrices. It does one thing well: it represents a matrix as a flat Array<Float64> and provides the primitive operations over that representation that machine-learning workloads actually call.
That flat representation is the same one the Vyne C backend unboxes to native C when the element type is statically known. A well-typed vlin program compiles to code where A.data[i] is a direct double load rather than a boxed vyne_index_get dispatch, so the C compiler can optimize the inner loops.
The library is intentionally small. Everything it exposes is meant to be read, not just called. The source files are the specification.
Quick start
Two calling conventions coexist and are equivalent:
- Function form —
vlin.multiply(a, b)— reads as "apply the multiply operation toaandb". - Method form —
a.transposed(),a.sum()— reads as "ask the matrix to do X".
For operations that exist in both forms, use whichever reads better at the call site. They dispatch to identical kernels.
Importing
vlin.vy is a facade. It pulls in the leaf modules:
| Module | Contents |
|---|---|
Types.vy | Matrix and Vector interfaces, group Types :: vlin |
Kernels.vy | Hot loops over Array<Float64>; the native-ABI surface |
Constructors.vy | Factories and random initializers |
Ops.vy | Matrix arithmetic, matrix products, stacking, in-place variants |
Reductions.vy | Scalar reductions: sum, mean, min, max, trace, norm |
You may import individual leaf modules if you want to avoid pulling in the full surface, but the facade is the recommended entry point.
Module layout
Two-layer design
Every user-visible operation is a two-layer composition:
- Kernels (
k_*.vy): loops over rawArray<Float64>. Every function takes typed-array parameters and returns eitherInt64(a status code, always 0) orFloat64. This shape makes the call site eligible for the native array ABI — the emitter passes.datadirectly and the body compiles to a bare C loop overdouble*.
- Ops / Reductions / Constructors: matrix-level wrappers that validate shapes, allocate the output buffer, and dispatch to the corresponding kernel.
Splitting the two layers this way is what lets vlin pay for shape checks exactly once, at the boundary, and then run unguarded loops in the hot path.
The Matrix type
A Matrix is a triple: two Int64 dimensions and a flat Array<Float64>. There is no stride field, no offset, no view mechanism. Row-major layout is the only layout.
Storage layout
Element (r, c) lives at index r * col + c in the flat data buffer.
This is the layout C uses for nested arrays, the layout BLAS uses for row-major matrices, and the layout most ML frameworks use internally. It matters if you plan to interoperate with vlin from native code, because the flat data buffer is the interop surface.
The Array<Float64> annotation is load-bearing
The data field is annotated Array<Float64>, not Array. The annotation is read by the Vyne C backend: when a struct field is annotated with a typed-array type, the backend unboxes that field into a native VyneArray_f64 on every read, so A.data[i] compiles to a direct double load rather than a boxed vyne_index_get dispatch.
Changing the annotation to plain Array still type-checks in Vyne source, but makes every element access in the library roughly 8× slower. Do not change it.
The Vector type
Vector is a 2-D geometric vector — two Int64 components, not an n-dimensional container. It exists for geometry problems and is not used by the matrix code paths. Neural-network code represents vectors as Matrix instances with one dimension equal to 1.
If you need a general n-dimensional vector, use Matrix with shape (N, 1) or (1, N).
Reference semantics and the memory model
Arrays and matrices are passed by reference
Vyne arrays are passed by reference, not by value. A Matrix parameter aliases the caller's data buffer:
Same semantics as Python lists, JavaScript arrays, and Lua tables.
If you want a defensive copy, call m.copy() at the top of the function. That is the only way to obtain value semantics in Vyne.
Every operation allocates, except m.flatten() and in-place variants
Each function in Ops.vy and Reductions.vy that returns a Matrix allocates a fresh backing buffer from the arena. Two exceptions exist:
m.flatten()returns the internaldatabuffer itself, not a copy. Mutating it mutates the matrix.- The
*_intofunctions (add_into,multiply_into,add_bias_into) write into a caller-provided destination instead of allocating a new result.
Arena lifetime and vmem
Under the default memory model, arena allocations live until process exit. A training loop that allocates a fresh matrix per iteration grows memory monotonically unless you checkpoint with vmem:
For a 64 → 16 → 12 → 1 MLP at batch size 240, that is roughly 1.5 MB of intermediates per forward-plus-backward pass. Across 500 epochs, 750 MB without a checkpoint; a few hundred kB with one.
API reference
Constructors
Fill matrices. identity produces an n × n diagonal matrix of ones with zeros elsewhere.
Build a matrix from a nested array. data[r][c] becomes element (r, c). Column count is inferred from data[0]. The function does not validate that inner arrays have consistent lengths.
Build a matrix from a flat array. data[r * cols + c] becomes element (r, c). Same absence of length validation.
Weight initializers.
random_uniformdraws every element uniformly from[lo, hi).xavier_initdraws uniformly from[-L, L]withL = sqrt(6 / (rows + cols))(Glorot uniform, appropriate fortanh/sigmoid).he_initdraws uniformly from[-L, L]withL = sqrt(6 / cols)(He uniform, appropriate for ReLU networks when the matrix is applied asY = X · W).random_initdraws uniformly from[-0.5, 0.5). Provided for symmetry; rarely what you want.
All initializers call vmath.random_float. See Limitations for the RNG's quality.
Low-level typed-array allocators
These are part of the public surface because they are the ABI boundary between kernel and wrapper. Most users should not call them directly.
Element-wise ops
All three require a.row == b.row and a.col == b.col. On mismatch they print a red diagnostic via vcolors and return a 0 × 0 matrix. They do not raise.
addcomputesa[i] + b[i].subtractcomputesa[i] - b[i].hadamardcomputesa[i] * b[i](Schur product).
Matrix products
multiply is the ordinary matrix product. Requires a.col == b.row, returns shape (a.row, b.col). On mismatch, red diagnostic and 0 × 0.
multiply_trans_b computes a @ b.T. Requires a.col == b.col and returns shape (a.row, b.row). Both inner reads are contiguous in the reduction dimension, which is the form you want for the backward pass of a dense layer.
transpose delegates to Matrix.transposed(). Same semantics, same allocation profile.
The implementation is a straightforward triple loop with no blocking, no cache-aware traversal, and no SIMD. For matrices larger than a few hundred elements per side, the constant factor is significantly worse than a tuned BLAS. Acceptable for the library's target use cases (small dense networks, small linear systems); not for large-scale numerical work.
Scalar ops
add_scalar and multiply_scalar apply an affine transformation element-wise. clip applies vmath.clamp(x, lo, hi) — which silently swaps the bounds if lo > hi.
Bias, stacking
add_bias takes a plain Array<Float64>, not a Matrix. The bias vector broadcasts row-wise: out[r][c] = m[r][c] + b[c]. b must have length m.col.
vstack requires matching column counts and produces a matrix with a.row + b.row rows. hstack requires matching row counts and produces a matrix with a.col + b.col columns. Both return 0 × 0 on mismatch with a red diagnostic.
In-place variants
Write into a caller-provided output matrix instead of allocating a fresh result. Return 0 on success, -1 on shape mismatch (also printing a red diagnostic).
These exist because the arena is not free. In a training loop, the functional form of add allocates a new backing buffer per call. The in-place variant lets you reuse one buffer across iterations.
These functions mutate their destination. If output aliases any of the inputs (e.g. add_into(a, a, b)), behavior is undefined — the kernel writes one element at a time and reads both inputs at each step.
Reductions
sum— sum of all elements.mean—sum / (row * col). Returns 0 for an empty matrix.minimum,maximum— scalar extremum over all elements. Return 0 for an empty matrix.norm_fro— Frobenius norm,sqrt(sum of squares). Never negative.trace— sum of the main diagonal. For a non-square matrix, the diagonal length ismin(row, col).argmax,argmin— flat index of the extremum. Return 0 for an empty matrix.
Matrix methods
The Matrix interface exposes the following methods. All are called m.method(args).
Shape and structure
| Method | Returns | Notes |
|---|---|---|
m.shape() | Array | [row, col] |
m.size() | Int64 | row * col |
m.is_square() | Bool | row == col |
m.is_vector() | Bool | row == 1 or col == 1 |
m.get(r, c) | Float64 | Element at (r, c). No bounds check. |
m.copy() | Matrix | Deep copy of the data buffer |
m.flatten() | Array<Float64> | Returns m.data (shared, not copied) |
m.row_at(r) | Array<Float64> | Row r as a fresh flat array |
m.col_at(c) | Array<Float64> | Column c as a fresh flat array |
Reductions (delegating to vlin.*)
| Method | Returns |
|---|---|
m.sum() | Float64 |
m.mean() | Float64 |
m.minimum() | Float64 |
m.maximum() | Float64 |
m.trace() | Float64 |
m.norm_fro() | Float64 |
m.argmax() | Int64 |
m.argmin() | Int64 |
Element-wise (returns fresh Matrix)
| Method | Returns | Notes |
|---|---|---|
m.negate() | Matrix | Element-wise -x |
m.add_scalar(s) | Matrix | Element-wise x + s |
m.mul_scalar(s) | Matrix | Element-wise x * s |
Shape manipulation
| Method | Returns | Notes |
|---|---|---|
m.reshape(rows, cols) | Matrix | Requires rows*cols == row*col; else prints diagnostic and returns m.copy() |
m.transposed() | Matrix | Returns a fresh (col, row) matrix |
Vector methods
| Method | Returns | Notes |
|---|---|---|
v.magnitude() | Float64 | Euclidean length: sqrt(x² + y²) |
v.slope() | Float64 | y / x; undefined for x == 0 |
v.dot(other) | Int64 | x · o.x + y · o.y |
v.cross_product(other) | Int64 | x · o.y − y · o.x |
Internal kernels
These are the k_* functions in Kernels.vy. They are public in the sense that the parser will resolve vlin.k_add(...), but they are not part of the stable interface — the parameter order, return type, and even the function list may change between versions. Use them only if you are building a new wrapper layer and want to reuse an existing loop.
Worked example
A minimal forward pass through a 2-layer MLP, using only the current surface.
The example above is intentionally incomplete — it exercises exactly the operations that exist in 0.1.0 and marks the one that does not (apply_tanh) so you can see the shape of the missing surface. For a forward pass today, replace vlin.apply_tanh with a hand-written loop using m.data[i] = vmath.tanh(m.data[i]) and construct the result matrix manually. See Roadmap for the plan.
For a fully working example that only uses present functions, see the vlin_test.vy file in examples/.
Limitations
No broadcasting
Binary operations require matching shapes. There is no NumPy-style implicit expansion. vlin.add of a (N, C) matrix and a (1, C) row vector will fail; use vlin.add_bias, which takes a plain Array<Float64> of length C and broadcasts row-wise.
No views or slicing at the library level
There is no vlin.slice. If you need a submatrix, copy the elements into a fresh matrix yourself. m.data[lo:hi] returns a flat array, not a matrix; reconstructing a matrix from a flat slice requires vlin.from_flat.
No shape validation in constructors
vlin.from_flat and vlin.from_array trust that the shape you supply matches the data. Passing rows * cols != data.size() produces a matrix that will misbehave on any subsequent operation. The constructor vlin.Types.Matrix(rows, cols, data) is even more permissive — it does no checking at all.
The random number generator is not strong
vlin's weight initializers call vmath.random_float, which uses a small linear congruential generator. The low-order bits have short periods and consecutive outputs show visible correlation. Adequate for weight initialization (any reasonably uniform spread of small values works); not adequate for cryptographic or statistical use. A stronger generator is a language-level change, not a library change.
Performance
Every operation is a scalar loop with no blocking, no vectorization, and no tuned kernels. The library compiles to code the C compiler can optimize, but not to code that uses SIMD instructions or multithreaded BLAS. For small matrices (up to a few thousand elements) the constant factor is negligible. For larger matrices, runtime is dominated by memory traffic and will be significantly slower than a tuned library.
If you need fast dense linear algebra, use a system BLAS through the extern module system. vlin is not intended to replace one.
No integer matrices
Every element is a Float64. If you need integer arithmetic, work directly with Array<Int64> or accept the floating-point rounding that comes with representing integers as doubles.
Design notes
Why a flat buffer instead of nested arrays
A nested Array<Array<Float64>> would be more natural to write and would allow each row to be independently allocated, which is sometimes useful. The flat buffer is faster because the C backend unboxes the entire Array<Float64> into a contiguous double*, and element access becomes a single index computation. A nested array would have each row as a separate boxed VyneArray_f64, doubling the number of unboxing boundaries in the inner loops.
The flat representation is also the standard for interop. BLAS, LAPACK, and most GPU libraries assume row-major or column-major flat buffers, not nested arrays.
Why both method and function forms
The method form on Matrix is the interface's natural expression: a matrix "has" a shape, "can" be transposed. The function form in Ops and Reductions is what you get when you want to treat operations as first-class values, compose them, or pass them to higher-order code.
Maintaining both costs one line per operation (the function simply calls the method, or vice versa) and provides a significantly more flexible surface. Same convention as Kotlin's collections library and Swift's standard library.
Why the k_ kernels are separate from the ops
The split exists because of the native array ABI. A function whose parameters are all Array<Float64> and whose return type is Int64 or Float64 is emitted as a native C function taking double* arguments and returning a scalar. There is no boxing, no per-element tag check, no arena traffic inside the loop.
If the shape check lived inside the kernel, the kernel would need to take two Matrix values instead of two Array<Float64> values, and would lose the ABI. Keeping validation in the wrapper and iteration in the kernel is what makes the fast path available.
Why no broadcasting
Broadcasting is subtle to specify, subtle to implement, and subtle to reason about when shapes interact across multiple operations. For a library whose primary use case is small dense neural networks, the explicit shapes are clearer and the additional code is negligible. add_bias exists specifically because the bias-broadcast pattern is common enough to justify its own function, but there is no general mechanism.
If you want broadcasting, implement it in user code by replicating the smaller operand into a shape-matched copy and calling the binary operation. That gives you full control over which broadcast rule applies.
Why the group is Types :: vlin
The Types group isolates the interface definitions from the free-function namespace. Users write vlin.Types.Matrix, not vlin.Matrix, which mirrors how the constructors and ops are namespaced and makes it obvious at a call site that a name refers to a type, not an operation.
Roadmap
The following are planned or in progress. They are listed here so that the manual is honest about the gap between the intended surface and the current one.
Activations module (planned: Activations.vy)
Element-wise non-linearities: apply_sigmoid, apply_tanh, apply_relu, apply_exp, apply_log, plus their derivatives sigmoid_prime, tanh_prime, relu_prime, and the normalization helpers softmax and normalize.
relu_prime is blocked on a codegen issue: the current C backend emits both branches of an if/else inside a collect block and pushes vyne_null(). Until ForNode::getCExpr is fixed, ReLU's derivative produces a matrix of nulls. Until then, use tanh or sigmoid for hidden-layer activations.
Losses module (planned: Losses.vy)
Mean squared error and binary cross-entropy, plus their derivatives:
In-place SGD update (planned)
Applies W[i] = W[i] - lr * grad[i] directly to W's backing buffer. Returns null.
This is the one mutating operation planned for the library. It exists for two reasons:
- It eliminates the two allocations (
multiply_scalarandsubtract) that the functional equivalent produces. For a training loop, that is a real win. - It keeps the weight matrix at a fixed address across training iterations — required for the weight matrix to remain valid across a
vmem.rewindboundary.
The functional equivalent without the in-place semantics is vlin.subtract(W, vlin.multiply_scalar(grad, lr)). That form is correct and unambiguous but allocates two intermediate matrices.
insert_row method (planned)
Appends a row to a matrix. new_row.size() must equal m.col.
Dot and outer for matrix-vector (planned)
dot handles the four shapes 1×n · 1×n, m×1 · m×1, 1×n · m×1, and m×1 · 1×n. outer computes out[i][j] = a[i] * b[j].
More scalar methods (planned)
m.sub_scalar(s) and m.div_scalar(s), for symmetry with the current add_scalar / mul_scalar.
Version history
0.1.0 — initial release. Types, Kernels, Constructors, Ops, Reductions. Method and function forms for the reductions. In-place *_into variants for element-wise add, matrix product, and bias addition.