Skip to main content

•explain — Explain

•explain text runs text in the caller’s scope, as ⍎ does, and returns a record. value is the result of text. Each other field is a table of events that were recorded while text ran. A table is a record of columns, as •csv returns, with one row for each event.

Events are recorded only while •explain runs. Events from code that another thread runs at the same time are recorded in the same tables. •explain computes a pending result of text before it returns, and adds no forced row for that.

Performance explains the behaviour that these tables report. •memory counts the live pending arrays, and reports what a value keeps alive.

Fused groups

fused

One row for each fused group that ran. Calls that wait lists the calls that can join a fused group.

Column Holds
glyphs the glyphs of the fused group’s calls, in the order they ran. A fold that read the fused group’s result without storing it comes last
items the number of items that the fused group processed
blocks the number of blocks of items that the fused group processed, one after another
runs the number of pieces that the blocks were split into
copied the number of items copied out of inputs before the calls read them, as a reversed input needs

A block is split where it crosses the end of a long repeat or cycle in an input, as when a row of 5,000 items is added to every row of a matrix. When runs is greater than blocks, the fused group ran in smaller pieces. When copied is 0, the fused group read every input in place.

A fused group that ran a compiled loop also adds a "compiled" row to routes. Compiled loops gives the sizes at which BPL compiles a fused group.

y←¿1000000ρ0
(•explain "+/ 2×y+1.5 - 1").fused            ⍝ ["glyphs":["+×-+/"] "items":[1000000]ₓ "blocks":[245]ₓ "runs":[245]ₓ "copied":[0]ₓ]
(•explain "2×⌽y+1").fused.copied             ⍝ [1000000]ₓ
t←y-0.25 ◇ (•explain "t×1-t").fused.glyphs    ⍝ ["--×"]

compiles

One row for each piece of a fused group’s code that BPL compiled into machine code.

Column Holds
glyphs the glyphs of the fused group’s calls
microseconds the time that the compile took, which is about 250 for a fused group of a few calls

stored

One row for each pending array that a fused group read as an input after the array was computed separately.

Column Holds
glyphs the glyph of the call that computed the array, or "" for a pending copy such as a reversal
items the number of items in the array
reason why the fused group did not compute the array as one of its own calls
partway always $f
reason Meaning
"held outside the group" The array is a pending copy that a name or another array also holds
"kept by a fold" An earlier fold read the array without storing its items, which left the array pending
"a kind that the group doesn't hold" The array’s items have a type that a fused group cannot process, such as complex numbers

A pending array that is not a copy, and that a name or another array holds, adds no row. The fused group computes it as one of its outputs and stores its items in the same pass. Names and writes says when a dfn’s local name stops holding its array.

y←¿1000000ρ0
q←1000005↑1⌽y ◇ (•explain "+/q+1").stored    ⍝ ["glyphs":[""] "items":[1000000]ₓ "reason":["held outside the group"] "partway":[$f]]

forced

One row for each call or assignment that computed items of pending arrays.

Column Holds
function the call’s function, or ← for an assignment
line, column its position in text
items the number of items computed
reason, site always empty

An assignment computes a pending array when it writes into the array, displays it with ⎕← or destructures it. A call that reads only a few items computes only those items, as Calls that wait describes. •prefs ["fusion":$t] prints a line for each of these calls and assignments as it happens.

y←¿1000000ρ0
e←•explain "≢⍕ 2×y+1.5 - 1"
e.forced.function                            ⍝ ["⍕"]
e.fused.glyphs                               ⍝ ["+×-"]
(•explain "5↑2×y+1").forced.items            ⍝ [5]ₓ
s←2×y+1 ◇ (•explain "5↑s").forced.items      ⍝ [5]ₓ
(•explain "5↑s").forced.items                ⍝ [1000000]ₓ
(•explain "5↑s").forced.items                ⍝ ⍬ₓ
r←⌽y ◇ (•explain "r₀←0").forced.function   ⍝ ["←"]

Maps and items

fallbacks

One row for each map that called its operand once for each cell. The map did so because its batched call stopped, or because an earlier batched call of the same dfn stopped and the map did not try one. Evaluation describes batched calls.

Column Holds
reason why the batched call stopped
items the number of cells
reason Meaning
"an effect" The operand writes output, reads input, formats a cell’s value with ⍕, or calls a system function that reads or writes outside BPL or makes something new on each call, such as •nget or •time. A batched call would do each of these once for all the cells
"a write" The operand writes to a name outside the map, appends to a value that differs from cell to cell, or writes into such a value in a way that would leave it in mixed storage
"a cell's value" The operand reads a cell’s value, for example as a count, a position or a dfn predicate. Each cell then runs the body that its predicate selects
"no batch rule" The operand calls a function that has no rule for batches of these arguments, such as ∪. When a primitive stopped the call, its glyph follows, as in "no batch rule for ∪"
"too many items" A call would repeat an array that every cell shares into far more items than its arguments hold, and the array is too large for the map to run on chunks of cells
"a caught error" An error guard would catch an error
"an earlier stop: " and one of the reasons above The operand is a dfn whose batched call stopped on an earlier call, for a cause that would recur. The map did not try a batched call
"an error" The batched call failed. If the call on a cell also fails, that call reports the error
"the result" The operand’s result was not a batch of the cells’ results. A function is one such result. A strand is a batch only when each of its items holds one atom for each cell and the items’ kinds can be combined. A strand that holds a vector for each cell is not a batch, and neither is one that mixes numbers and characters
(•explain "{2(×⍣ω)1}¨ι5ₓ").fallbacks  ⍝ ["reason":["a cell's value"] "items":[5]ₓ]

itemwise

One row for each route that handled an array’s items one at a time. Most calls run code that handles a whole array.

Column Holds
function the innermost call that ran the route, the target of a selective assignment, or •explain for a route outside both
line, column its position in text
route the name of the route
items the number of items that the route handled
route Meaning
"item pairs" A pervasive function was called on each pair of items of mixed storage or of a nested array
"batched items" Code with no rule for batches read the items of a batch one at a time. The batched call can still succeed
"∂ items" Code with no rule for ∂ read, one at a time, the items of a value that ∂ is differentiating
"traced items" Code that was building a program read traced items one at a time
"accumulation" A sum added items one at a time, as the accumulations path "item by item" does
"item lanes" A fold or scan called its function on each pair of items in turn, because the function has no kernel for its arguments. A primitive on mixed storage is one case, and a dfn that no batched call covered is another
"number parts" Monadic ∨ or ∧ took the parts of integers or exact numbers one at a time. Floats and complex numbers take one pass
"nested labels", "label ids" A selection from a nested array labelled or placed its items one at a time
"key groups" Key called its operand once for each group, because no fast combination covered the call
"batch cells" A batched call ran a system function, or a deal by ¿, once for each cell in turn
"inverse scan" The inverse of a scan called the function’s inverse on each pair of neighbouring items in turn, because the function has no kernel
m←1ₓ 2.5 ◇ (•explain "m+1").itemwise  ⍝ ["function":["+"] "line":[1]ₓ "column":[2]ₓ "route":["item pairs"] "items":[2]ₓ]

mixed

One row for each write that left a compact array in mixed storage. Under dynamic kinds, assigning an exact integer into an array of floats does so, by the write rule.

Column Holds
code the target or call that wrote
line, column its position in text
target the array’s storage before the write, as •storage names it
written the storage of the values written
items the number of items in the array
w←4ρ0.5 ◇ (•explain "(ι2ₓ)@[1 3] w").mixed  ⍝ ["code":["(ι2ₓ)@[1 3]"] "line":[1]ₓ "column":[1]ₓ "target":["float64"] "written":["integer"] "items":[4]ₓ]

accumulations

One row for each sum that adds items at indexed positions. These sums include the gradients of selections under ∂, and modified assignments such as z.[p]+←1.

Column Holds
items the number of items added
size the number of items in the result
path how the items were added
path Meaning
"in place" The items are added into the array’s buffer. Only that array holds the buffer, and the sum cannot fail for any item, as with floats
"on a copy" The items are added into a copy of the array, because the sum can fail or because another array shares the buffer. A sum of ordinary integers can fail by overflow under dynamic kinds
"rounds" Some positions repeat. Each call adds one item at each position that still has items to add
"one call" The positions are distinct, and one call adds every item
"item by item" The function is called once for each target, as a dfn is

These sums add no row:

  • a gradient that BPL computes as direct reads and sums, including one of windows with a step of 1
  • a sum whose positions are one ascending run, such as the gradient of Take or Drop, which is added as a block
  • the gradient of a ⌈ or ⌊ fold
x←¿1000ρ0 ◇ p←¿1000ρ1000
(•explain "{+/, 3↕ω (×¹) 1 2 1}∂ x").accumulations.items  ⍝ ⍬ₓ
(•explain "{+/ω.[p]}∂ x").accumulations            ⍝ ["items":[1000]ₓ "size":[1000]ₓ "path":["in place"]]
z←1000ρ0ₓ ◇ (•explain "z.[p]+←1ₓ").accumulations.path  ⍝ ["on a copy"]

Shortcuts

fast

One row for each fast combination that BPL used, and for each fold of products that ran as a matrix product.

Column Holds
rule the name of the combination: "partial grade", "partial sort", "ranks", "group counts", "group folds", "determinant", "permanent", "masked select" or "contraction"
function the function that used it

When BPL recognises a combination from the form of the code, function is the same text for every combination of that kind: "↑⍋" for every selection, and "f/⊢⌸" for every group fold.

y←¿1000000ρ0
(•explain "(10↑)∘⍋ y").fast.rule             ⍝ ["partial grade"]

routes

One row for each fast path that succeeded, with totals for all its calls.

Column Holds
route the name of the fast path
calls the number of calls that took it
items the number of items that those calls handled, or 0 for a route that counts no items

The routes include tracing ("trace"), fused groups ("fusion"), batched calls ("batching"), compiled loops ("compiled") and the fast combinations. They also include these:

route Meaning
"vector kernel" a pervasive call computed by its kernel on stored arrays
"mapped kernel" a "vector kernel" call that read an argument through a mapping, not from consecutive items: an extended scalar, a row repeated down a matrix, or a pending copy such as ⌽y
"bulk catenate" a , or ⍪ fold over arrays of one rank and type, joined in one copy
"alternated fold" a fold by - or ÷
"sliding fold" a fold of windows
"mapped fold" a fold that reads a pending copy without building the copy
"in place" a mapped fold that reads its items where they are stored
"mapped product" a matrix product that reads a pending copy larger than its result without building the copy

(•explain t).routes.route lists the routes that t took.

declined

One row for each optional shortcut that BPL considered and did not take, for each reason. Another fast path can still handle a call that declined one.

Column Holds
route the shortcut
reason why BPL did not take it
calls the number of calls that declined
items the number of items in those calls
route reason Meaning
"fusion" "a result kind that depends on values" A call ran separately, because the type of its result depends on its items, as with √ of floats under dynamic kinds
"fusion" "an argument of another kind" A call ran separately, because an argument is a stored array of Booleans or integers whose type is not the type that the call computes at, as in integers times a float. A fused group large enough to compile on its first run takes such an argument
"fused fold" "a copy that no call moves under" A fold computed its argument, a pending copy, because the fold could not read through the copy to its source
"fused fold" "a fold that its group can't run" A fold computed its argument, because the fused group cannot run that fold
"compiled" the name of a call, or "a kernel that doesn't compile" A fused group that would have run a compiled loop ran its usual kernels, because that kernel has no machine code
the name of a fast combination, such as "partial grade" "arguments that don't suit it" The general code ran
"batching" "the batch pref is off" A map did not try a batched call
"batching" "a label level, or a ∂ call inside a batched run" A map did not try a batched call, because it ran inside the selection of an Under or a selective assignment, or inside a ∂ call that a batched call was running
"batching" "an argument too large to lay over the frame", "an argument that isn't compact" A map did not try a batched call, because of that argument
"program" "a call under another level" A call did not use a program, because it ran inside a batched call or another such transformation
"program" "an argument that programs can't hold" A call did not use a program, because a program cannot take one of its arguments
"program" "an effect", "a write", "a random draw", "a free value that changes on every call" An ordinary call’s trace stopped, for that reason
"program" "a body outside dense elementwise reuse" An ordinary call got no program, because its body does not give one array, or has a step that cannot join a fused group
"in place" "a repeated item" A kernel copied an argument’s items into a buffer, because the argument is one item used at every position
"in place" "repeats of each item" A kernel copied an argument’s items into a buffer, because each item is used several times in a row, as a vector’s items are when the vector is added to the rows of a matrix along the first axis
"in place" "a cycle that wraps" A kernel copied an argument’s items into a buffer, because the argument’s items repeat as a cycle and the block being read passed the end of the cycle
"in place" "steps other than a forward run" A kernel copied an argument’s items into a buffer, because the argument is a pending copy whose items are not read in stored order, as with a reversal
"contraction" a reason such as "an operand that doesn't step along the fold" A fold of elementwise products could not run as a matrix product

A fold of a pending array that a name also holds stores the array’s items in the same pass, and adds no row.

y←¿1000000ρ0
(•explain "+/⌽y").declined  ⍝ ["route":["fused fold"] "reason":["a copy that no call moves under"] "calls":[1]ₓ "items":[1000000]ₓ]

Programs

programs

One row for each call that built or replayed a program. Programs says which calls use programs, and when BPL keeps one.

Column Holds
event "built" for a program that this call recorded, or "replayed" for one that BPL had kept
steps the number of steps in the program
listing the program as text, with one line for each value. A width suffix on a value is listed as a convert step

A program for a ∂ call holds the work of the function and of its derivative. A program for an ordinary call holds the work of that call’s body. The work that a program does when it runs is reported in fused, compiles and routes.

When the trace of a ∂ call stops, stopped has a row in place of this one. When the trace of an ordinary call stops, declined has a "program" row.

stopped

One row for each reason that a trace of a ∂ call stopped. The interpreter then ran the call without a program.

Column Holds
reason why the trace stopped
calls the number of calls, including calls that reused an earlier stopped trace
items the total number of items in the calls’ right arguments
site always empty
reason Meaning
"an effect" The function writes output or has another effect
"a result whose type depends on values" The function calls a function such as √, whose result can be complex under dynamic kinds
"a free value that changes on every call" The function uses an outside name or a bound array as positions, as a count or in a predicate, and its value changed from call to call
"a function called on each item" The function calls a function once for each item of a traced array. A fold by a dfn does so, and so does an Each whose dfn has a predicate

What stops a trace lists every cause.