•explain — Explain
•explain text runs text in the caller’s scope, as ⍎ does, and returns a record. value is the result of text. Each other field is a table of events that were recorded while text ran. A table is a record of columns, as •csv returns, with one row for each event.
Events are recorded only while •explain runs. Events from code that another thread runs at the same time are recorded in the same tables. •explain computes a pending result of text before it returns, and adds no forced row for that.
Performance explains the behaviour that these tables report. •memory counts the live pending arrays, and reports what a value keeps alive.
Fused groups
fused
One row for each fused group that ran. Calls that wait lists the calls that can join a fused group.
| Column | Holds |
|---|---|
glyphs |
the glyphs of the fused group’s calls, in the order they ran. A fold that read the fused group’s result without storing it comes last |
items |
the number of items that the fused group processed |
blocks |
the number of blocks of items that the fused group processed, one after another |
runs |
the number of pieces that the blocks were split into |
copied |
the number of items copied out of inputs before the calls read them, as a reversed input needs |
A block is split where it crosses the end of a long repeat or cycle in an input, as when a row of 5,000 items is added to every row of a matrix. When runs is greater than blocks, the fused group ran in smaller pieces. When copied is 0, the fused group read every input in place.
A fused group that ran a compiled loop also adds a "compiled" row to routes. Compiled loops gives the sizes at which BPL compiles a fused group.
y←¿1000000ρ0
(•explain "+/ 2×y+1.5 - 1").fused ⍝ ["glyphs":["+×-+/"] "items":[1000000]ₓ "blocks":[245]ₓ "runs":[245]ₓ "copied":[0]ₓ]
(•explain "2×⌽y+1").fused.copied ⍝ [1000000]ₓ
t←y-0.25 ◇ (•explain "t×1-t").fused.glyphs ⍝ ["--×"]compiles
One row for each piece of a fused group’s code that BPL compiled into machine code.
| Column | Holds |
|---|---|
glyphs |
the glyphs of the fused group’s calls |
microseconds |
the time that the compile took, which is about 250 for a fused group of a few calls |
stored
One row for each pending array that a fused group read as an input after the array was computed separately.
| Column | Holds |
|---|---|
glyphs |
the glyph of the call that computed the array, or "" for a pending copy such as a reversal |
items |
the number of items in the array |
reason |
why the fused group did not compute the array as one of its own calls |
partway |
always $f |
reason |
Meaning |
|---|---|
"held outside the group" |
The array is a pending copy that a name or another array also holds |
"kept by a fold" |
An earlier fold read the array without storing its items, which left the array pending |
"a kind that the group doesn't hold" |
The array’s items have a type that a fused group cannot process, such as complex numbers |
A pending array that is not a copy, and that a name or another array holds, adds no row. The fused group computes it as one of its outputs and stores its items in the same pass. Names and writes says when a dfn’s local name stops holding its array.
y←¿1000000ρ0
q←1000005↑1⌽y ◇ (•explain "+/q+1").stored ⍝ ["glyphs":[""] "items":[1000000]ₓ "reason":["held outside the group"] "partway":[$f]]forced
One row for each call or assignment that computed items of pending arrays.
| Column | Holds |
|---|---|
function |
the call’s function, or ← for an assignment |
line, column |
its position in text |
items |
the number of items computed |
reason, site |
always empty |
An assignment computes a pending array when it writes into the array, displays it with ⎕← or destructures it. A call that reads only a few items computes only those items, as Calls that wait describes. •prefs ["fusion":$t] prints a line for each of these calls and assignments as it happens.
y←¿1000000ρ0
e←•explain "≢⍕ 2×y+1.5 - 1"
e.forced.function ⍝ ["⍕"]
e.fused.glyphs ⍝ ["+×-"]
(•explain "5↑2×y+1").forced.items ⍝ [5]ₓ
s←2×y+1 ◇ (•explain "5↑s").forced.items ⍝ [5]ₓ
(•explain "5↑s").forced.items ⍝ [1000000]ₓ
(•explain "5↑s").forced.items ⍝ ⍬ₓ
r←⌽y ◇ (•explain "r₀←0").forced.function ⍝ ["←"]Maps and items
fallbacks
One row for each map that called its operand once for each cell. The map did so because its batched call stopped, or because an earlier batched call of the same dfn stopped and the map did not try one. Evaluation describes batched calls.
| Column | Holds |
|---|---|
reason |
why the batched call stopped |
items |
the number of cells |
reason |
Meaning |
|---|---|
"an effect" |
The operand writes output, reads input, formats a cell’s value with ⍕, or calls a system function that reads or writes outside BPL or makes something new on each call, such as •nget or •time. A batched call would do each of these once for all the cells |
"a write" |
The operand writes to a name outside the map, appends to a value that differs from cell to cell, or writes into such a value in a way that would leave it in mixed storage |
"a cell's value" |
The operand reads a cell’s value, for example as a count, a position or a dfn predicate. Each cell then runs the body that its predicate selects |
"no batch rule" |
The operand calls a function that has no rule for batches of these arguments, such as ∪. When a primitive stopped the call, its glyph follows, as in "no batch rule for ∪" |
"too many items" |
A call would repeat an array that every cell shares into far more items than its arguments hold, and the array is too large for the map to run on chunks of cells |
"a caught error" |
An error guard would catch an error |
"an earlier stop: " and one of the reasons above |
The operand is a dfn whose batched call stopped on an earlier call, for a cause that would recur. The map did not try a batched call |
"an error" |
The batched call failed. If the call on a cell also fails, that call reports the error |
"the result" |
The operand’s result was not a batch of the cells’ results. A function is one such result. A strand is a batch only when each of its items holds one atom for each cell and the items’ kinds can be combined. A strand that holds a vector for each cell is not a batch, and neither is one that mixes numbers and characters |
(•explain "{2(×⍣ω)1}¨ι5ₓ").fallbacks ⍝ ["reason":["a cell's value"] "items":[5]ₓ]itemwise
One row for each route that handled an array’s items one at a time. Most calls run code that handles a whole array.
| Column | Holds |
|---|---|
function |
the innermost call that ran the route, the target of a selective assignment, or •explain for a route outside both |
line, column |
its position in text |
route |
the name of the route |
items |
the number of items that the route handled |
route |
Meaning |
|---|---|
"item pairs" |
A pervasive function was called on each pair of items of mixed storage or of a nested array |
"batched items" |
Code with no rule for batches read the items of a batch one at a time. The batched call can still succeed |
"∂ items" |
Code with no rule for ∂ read, one at a time, the items of a value that ∂ is differentiating |
"traced items" |
Code that was building a program read traced items one at a time |
"accumulation" |
A sum added items one at a time, as the accumulations path "item by item" does |
"item lanes" |
A fold or scan called its function on each pair of items in turn, because the function has no kernel for its arguments. A primitive on mixed storage is one case, and a dfn that no batched call covered is another |
"number parts" |
Monadic ∨ or ∧ took the parts of integers or exact numbers one at a time. Floats and complex numbers take one pass |
"nested labels", "label ids" |
A selection from a nested array labelled or placed its items one at a time |
"key groups" |
Key called its operand once for each group, because no fast combination covered the call |
"batch cells" |
A batched call ran a system function, or a deal by ¿, once for each cell in turn |
"inverse scan" |
The inverse of a scan called the function’s inverse on each pair of neighbouring items in turn, because the function has no kernel |
m←1ₓ 2.5 ◇ (•explain "m+1").itemwise ⍝ ["function":["+"] "line":[1]ₓ "column":[2]ₓ "route":["item pairs"] "items":[2]ₓ]mixed
One row for each write that left a compact array in mixed storage. Under dynamic kinds, assigning an exact integer into an array of floats does so, by the write rule.
| Column | Holds |
|---|---|
code |
the target or call that wrote |
line, column |
its position in text |
target |
the array’s storage before the write, as •storage names it |
written |
the storage of the values written |
items |
the number of items in the array |
w←4ρ0.5 ◇ (•explain "(ι2ₓ)@[1 3] w").mixed ⍝ ["code":["(ι2ₓ)@[1 3]"] "line":[1]ₓ "column":[1]ₓ "target":["float64"] "written":["integer"] "items":[4]ₓ]accumulations
One row for each sum that adds items at indexed positions. These sums include the gradients of selections under ∂, and modified assignments such as z.[p]+←1.
| Column | Holds |
|---|---|
items |
the number of items added |
size |
the number of items in the result |
path |
how the items were added |
path |
Meaning |
|---|---|
"in place" |
The items are added into the array’s buffer. Only that array holds the buffer, and the sum cannot fail for any item, as with floats |
"on a copy" |
The items are added into a copy of the array, because the sum can fail or because another array shares the buffer. A sum of ordinary integers can fail by overflow under dynamic kinds |
"rounds" |
Some positions repeat. Each call adds one item at each position that still has items to add |
"one call" |
The positions are distinct, and one call adds every item |
"item by item" |
The function is called once for each target, as a dfn is |
These sums add no row:
- a gradient that BPL computes as direct reads and sums, including one of windows with a step of 1
- a sum whose positions are one ascending run, such as the gradient of Take or Drop, which is added as a block
- the gradient of a
⌈or⌊fold
x←¿1000ρ0 ◇ p←¿1000ρ1000
(•explain "{+/, 3↕ω (×¹) 1 2 1}∂ x").accumulations.items ⍝ ⍬ₓ
(•explain "{+/ω.[p]}∂ x").accumulations ⍝ ["items":[1000]ₓ "size":[1000]ₓ "path":["in place"]]
z←1000ρ0ₓ ◇ (•explain "z.[p]+←1ₓ").accumulations.path ⍝ ["on a copy"]Shortcuts
fast
One row for each fast combination that BPL used, and for each fold of products that ran as a matrix product.
| Column | Holds |
|---|---|
rule |
the name of the combination: "partial grade", "partial sort", "ranks", "group counts", "group folds", "determinant", "permanent", "masked select" or "contraction" |
function |
the function that used it |
When BPL recognises a combination from the form of the code, function is the same text for every combination of that kind: "↑⍋" for every selection, and "f/⊢⌸" for every group fold.
y←¿1000000ρ0
(•explain "(10↑)∘⍋ y").fast.rule ⍝ ["partial grade"]routes
One row for each fast path that succeeded, with totals for all its calls.
| Column | Holds |
|---|---|
route |
the name of the fast path |
calls |
the number of calls that took it |
items |
the number of items that those calls handled, or 0 for a route that counts no items |
The routes include tracing ("trace"), fused groups ("fusion"), batched calls ("batching"), compiled loops ("compiled") and the fast combinations. They also include these:
route |
Meaning |
|---|---|
"vector kernel" |
a pervasive call computed by its kernel on stored arrays |
"mapped kernel" |
a "vector kernel" call that read an argument through a mapping, not from consecutive items: an extended scalar, a row repeated down a matrix, or a pending copy such as ⌽y |
"bulk catenate" |
a , or ⍪ fold over arrays of one rank and type, joined in one copy |
"alternated fold" |
a fold by - or ÷ |
"sliding fold" |
a fold of windows |
"mapped fold" |
a fold that reads a pending copy without building the copy |
"in place" |
a mapped fold that reads its items where they are stored |
"mapped product" |
a matrix product that reads a pending copy larger than its result without building the copy |
(•explain t).routes.route lists the routes that t took.
declined
One row for each optional shortcut that BPL considered and did not take, for each reason. Another fast path can still handle a call that declined one.
| Column | Holds |
|---|---|
route |
the shortcut |
reason |
why BPL did not take it |
calls |
the number of calls that declined |
items |
the number of items in those calls |
route |
reason |
Meaning |
|---|---|---|
"fusion" |
"a result kind that depends on values" |
A call ran separately, because the type of its result depends on its items, as with √ of floats under dynamic kinds |
"fusion" |
"an argument of another kind" |
A call ran separately, because an argument is a stored array of Booleans or integers whose type is not the type that the call computes at, as in integers times a float. A fused group large enough to compile on its first run takes such an argument |
"fused fold" |
"a copy that no call moves under" |
A fold computed its argument, a pending copy, because the fold could not read through the copy to its source |
"fused fold" |
"a fold that its group can't run" |
A fold computed its argument, because the fused group cannot run that fold |
"compiled" |
the name of a call, or "a kernel that doesn't compile" |
A fused group that would have run a compiled loop ran its usual kernels, because that kernel has no machine code |
the name of a fast combination, such as "partial grade" |
"arguments that don't suit it" |
The general code ran |
"batching" |
"the batch pref is off" |
A map did not try a batched call |
"batching" |
"a label level, or a ∂ call inside a batched run" |
A map did not try a batched call, because it ran inside the selection of an Under or a selective assignment, or inside a ∂ call that a batched call was running |
"batching" |
"an argument too large to lay over the frame", "an argument that isn't compact" |
A map did not try a batched call, because of that argument |
"program" |
"a call under another level" |
A call did not use a program, because it ran inside a batched call or another such transformation |
"program" |
"an argument that programs can't hold" |
A call did not use a program, because a program cannot take one of its arguments |
"program" |
"an effect", "a write", "a random draw", "a free value that changes on every call" |
An ordinary call’s trace stopped, for that reason |
"program" |
"a body outside dense elementwise reuse" |
An ordinary call got no program, because its body does not give one array, or has a step that cannot join a fused group |
"in place" |
"a repeated item" |
A kernel copied an argument’s items into a buffer, because the argument is one item used at every position |
"in place" |
"repeats of each item" |
A kernel copied an argument’s items into a buffer, because each item is used several times in a row, as a vector’s items are when the vector is added to the rows of a matrix along the first axis |
"in place" |
"a cycle that wraps" |
A kernel copied an argument’s items into a buffer, because the argument’s items repeat as a cycle and the block being read passed the end of the cycle |
"in place" |
"steps other than a forward run" |
A kernel copied an argument’s items into a buffer, because the argument is a pending copy whose items are not read in stored order, as with a reversal |
"contraction" |
a reason such as "an operand that doesn't step along the fold" |
A fold of elementwise products could not run as a matrix product |
A fold of a pending array that a name also holds stores the array’s items in the same pass, and adds no row.
y←¿1000000ρ0
(•explain "+/⌽y").declined ⍝ ["route":["fused fold"] "reason":["a copy that no call moves under"] "calls":[1]ₓ "items":[1000000]ₓ]Programs
programs
One row for each call that built or replayed a program. Programs says which calls use programs, and when BPL keeps one.
| Column | Holds |
|---|---|
event |
"built" for a program that this call recorded, or "replayed" for one that BPL had kept |
steps |
the number of steps in the program |
listing |
the program as text, with one line for each value. A width suffix on a value is listed as a convert step |
A program for a ∂ call holds the work of the function and of its derivative. A program for an ordinary call holds the work of that call’s body. The work that a program does when it runs is reported in fused, compiles and routes.
When the trace of a ∂ call stops, stopped has a row in place of this one. When the trace of an ordinary call stops, declined has a "program" row.
stopped
One row for each reason that a trace of a ∂ call stopped. The interpreter then ran the call without a program.
| Column | Holds |
|---|---|
reason |
why the trace stopped |
calls |
the number of calls, including calls that reused an earlier stopped trace |
items |
the total number of items in the calls’ right arguments |
site |
always empty |
reason |
Meaning |
|---|---|
"an effect" |
The function writes output or has another effect |
"a result whose type depends on values" |
The function calls a function such as √, whose result can be complex under dynamic kinds |
"a free value that changes on every call" |
The function uses an outside name or a bound array as positions, as a count or in a predicate, and its value changed from call to call |
"a function called on each item" |
The function calls a function once for each item of a traced array. A fold by a dfn does so, and so does an Each whose dfn has a predicate |
What stops a trace lists every cause.