# compact
``` python
from fastcore.test import *
from fastllm.chat import mk_msg
```
`llmsurgery.compact` defines a concise format for compacted LLM
conversations. Long conversations need to preserve the theory,
decisions, and evidence needed for useful continuation while using far
fewer tokens. The format provides readable canonical rendering, bounded
token budgets, tool calls and results, truncation, and repeated
compaction.
## Compact DSL
We’ll start with the compact DSL, which takes fastllm’s canonical
conversation data structures and renders them into this concise format.
First we will define and inspect the sample conversation using fastllm’s
canonical Msg and Part data structures as our basis.
------------------------------------------------------------------------
source
### tool_result
``` python
def tool_result(
text, tool_name:NoneType=None
):
```
*Create a canonical tool-result message; pass `tool_name` to pair it
with its
[`tool_part`](https://AnswerDotAI.github.io/llmsurgery/compact.html#tool_part)
call*
------------------------------------------------------------------------
source
### tool_part
``` python
def tool_part(
tool_name, **arguments
):
```
*Create a canonical tool-use part*
``` python
python_str = 'files = ["a.py", "b.py"]'
bash_str = 'printf "ready\\n"\nls -la'
generic_args = {'path':'fixture.json', 'lines':[1, 2]}
res_str = 'first line\n\nsecond line\nfinal line'
python_part = tool_part('mcp__clikernel__execute', code=python_str)
bash_part = tool_part('Bash', command=bash_str)
generic_part = tool_part('inspect', **generic_args)
```
``` python
turn1 = [mk_msg('List the project files, use the project skill, then run a shell check.'),
mk_msg(['I will inspect the project.', python_part], role='assistant'),
tool_result('files = ["a.py", "b.py"]', 'mcp__clikernel__execute'),
mk_msg([tool_part('Skill', skill='project-files')], role='assistant'),
tool_result('Skill loaded: project-files', 'Skill'),
mk_msg('Base directory for this skill: /skills/project-files\n\nInspect project files before editing.'),
mk_msg([bash_part], role='assistant'),
tool_result('ready', 'Bash')]
```
``` python
turn2 = [mk_msg('Keep the agreed names; do not rename the fixture.'), # chkstyle: ignore-node
mk_msg(['Understood. I will verify the final state.', generic_part], role='assistant'),
tool_result(res_str, 'inspect'),
mk_msg('The fixture is short, readable, and ready for the renderer.', role='assistant')]
```
``` python
conversation = turn1 + turn2
len(conversation)
```
12
``` python
turn1
```
[Msg(role='user', content=[Part(type=, text='List the project files, use the project skill, then run a shell check.', data=None)]),
Msg(role='assistant', content=[Part(type=, text='I will inspect the project.', data=None), Part(type=, text=None, data={'name': 'mcp__clikernel__execute', 'arguments': {'code': 'files = ["a.py", "b.py"]'}})]),
Msg(role='tool', content=[Part(type=, text='files = ["a.py", "b.py"]', data=None)]),
Msg(role='assistant', content=[Part(type=, text=None, data={'name': 'Skill', 'arguments': {'skill': 'project-files'}})]),
Msg(role='tool', content=[Part(type=, text='Skill loaded: project-files', data=None)]),
Msg(role='user', content=[Part(type=, text='Base directory for this skill: /skills/project-files\n\nInspect project files before editing.', data=None)]),
Msg(role='assistant', content=[Part(type=, text=None, data={'name': 'Bash', 'arguments': {'command': 'printf "ready\\n"\nls -la'}})]),
Msg(role='tool', content=[Part(type=, text='ready', data=None)])]
``` python
turn2
```
[Msg(role='user', content=[Part(type=, text='Keep the agreed names; do not rename the fixture.', data=None)]),
Msg(role='assistant', content=[Part(type=, text='Understood. I will verify the final state.', data=None), Part(type=, text=None, data={'name': 'inspect', 'arguments': {'path': 'fixture.json', 'lines': [1, 2]}})]),
Msg(role='tool', content=[Part(type=, text='first line\n\nsecond line\nfinal line', data=None)]),
Msg(role='assistant', content=[Part(type=, text='The fixture is short, readable, and ready for the renderer.', data=None)])]
We use a *token* budget (based on OpenAI’s public tokenizer, which
should be at least indicative of other LLM’s too), and truncate the
*middle*, since the start and end are often the interesting bits.
------------------------------------------------------------------------
source
### len_toks
``` python
def len_toks(
s, enc:NoneType=None
):
```
*Number of tokens in `s`.*
------------------------------------------------------------------------
source
### enc_toks
``` python
def enc_toks(
s, enc:NoneType=None
):
```
*Encode `s` into tokens.*
------------------------------------------------------------------------
source
### compact_enc
``` python
def compact_enc():
```
*Call self as a function.*
``` python
enc = compact_enc()
```
------------------------------------------------------------------------
source
### trunctoks_mid
``` python
def trunctoks_mid(
s, max_toks, enc:NoneType=None, mark:str=' … '
):
```
*Truncate the middle of `s` to at most `max_toks` tokens.*
[`compact_text`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_text)
renders ordinary user or assistant prose with a marker and a token
limit. It is the simplest DSL conversion: delimit the text, then use
middle truncation so both the beginning and ending remain available when
the budget is exceeded.
------------------------------------------------------------------------
source
### compact_text
``` python
def compact_text(
s, mark, max_toks, enc:NoneType=None
):
```
*Render delimited prose within `max_toks`.*
``` python
text = compact_text('beginning ' + 'middle ' * 100 + ' ending', '§', 20)
test(text, '§ beginning', str.startswith)
test(text, 'ending §', str.endswith)
assert len_toks(text) <= 20, text
print(text)
```
§ beginning middle middle middle middle middle middle middle … middle middle middle middle middle middle ending §
[`compact_result`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_result)
renders tool output compactly. It replaces internal newlines with `¶`,
prefixes the result with `>`, and applies the same middle-truncation
policy so the output remains useful within a token budget. An empty
result is represented by the `>` marker alone. ``/``
doc-state blocks are dropped first: they re-fire mechanically after a
restart, so preserved copies carry no forward information — and worse,
demonstrate the very call-before-doc pattern they exist to correct. A
result wrapped in `𝍁…𝍁` is unwrapped and kept whole, mirroring fastllm’s
truncation contract (`FullResponse` and friends skip truncation, and the
`𝍁` marker is the form that survives serialization): a tool marks its
output this way exactly when it is expensive or impossible to
regenerate.
------------------------------------------------------------------------
source
### compact_result
``` python
def compact_result(
s, max_toks, enc:NoneType=None
):
```
*Render a tool result within `max_toks`; ``/`` doc-state
blocks are dropped, and a `𝍁`-wrapped result is kept whole.*
``` python
result = compact_result(res_str, 20)
empty = compact_result('', 20)
test_eq(result, '> first line ¶ second line ¶ final line')
test_eq(empty, '>')
warned = compact_result('\nRule violation: docs not read.\n\n' + res_str, 20)
test_eq(warned, result)
test_eq(compact_result('\nA doc-state nudge.\n', 20), '>')
full = compact_result('𝍁' + 'keep me whole ' * 30 + '𝍁', 20)
test_eq(full, '> ' + ('keep me whole ' * 30).strip())
print(result)
print(empty)
```
> first line ¶ second line ¶ final line
>
[`fenced_code`](https://AnswerDotAI.github.io/llmsurgery/compact.html#fenced_code)
renders Python tool-call code as Markdown without allowing backticks
inside the code to close the fence. It chooses a fence longer than any
run of backticks in the source, then applies the shared token budget.
------------------------------------------------------------------------
source
### fenced_code
``` python
def fenced_code(
code, max_toks, enc:NoneType=None
):
```
*Render Python in a safe Markdown fence within `max_toks`.*
``` python
simple = fenced_code(python_str, 40)
with_inline_backticks = fenced_code("s = '```text```'", 40)
with_fence_line = fenced_code("before\n```\nafter", 40)
truncated = fenced_code(python_str * 20, 20)
test_eq(simple, f'```python\n{python_str}\n```')
test_eq(with_inline_backticks, "```python\ns = '```text```'\n```")
test_eq(with_fence_line, "````python\nbefore\n```\nafter\n````")
test(truncated, '```python\n', str.startswith)
test(truncated, '\n```', str.endswith)
assert len_toks(truncated) <= 20, truncated
print(simple)
```
```python
files = ["a.py", "b.py"]
```
``` python
print(with_fence_line)
```
````python
before
```
after
````
[`bash_call`](https://AnswerDotAI.github.io/llmsurgery/compact.html#bash_call)
renders a Bash tool call as a compact shell command. It uses `!` as the
marker, collapses non-empty command lines with `¶`, and applies the
shared token budget.
------------------------------------------------------------------------
source
### bash_call
``` python
def bash_call(
command, max_toks, enc:NoneType=None
):
```
*Render a Bash command within `max_toks`.*
``` python
bash = bash_call(bash_str, 30)
test_eq(bash, '! printf "ready\\n" ¶ ls -la')
print(bash)
```
! printf "ready\n" ¶ ls -la
[`generic_call`](https://AnswerDotAI.github.io/llmsurgery/compact.html#generic_call)
renders non-Bash tool calls as `▶ name(args)`. It formats named
arguments compactly, replaces embedded newlines with `¶`, and applies
the shared token budget.
------------------------------------------------------------------------
source
### generic_call
``` python
def generic_call(
name, args, max_toks, enc:NoneType=None
):
```
*Render a generic tool call within `max_toks`.*
``` python
call = generic_call(generic_part.name, generic_part.arguments, 40)
test_eq(call, "▶ inspect(path='fixture.json', lines=[1, 2])")
print(call)
```
▶ inspect(path='fixture.json', lines=[1, 2])
[`compact_call`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_call)
dispatches each tool-use part through `call_renderers`, which maps a
tool name to the renderer for its payload and the argument key holding
it: clikernel and solveit’s `py` render fenced Python, Claude Code’s
`Bash` and solveit’s `bash` render the `!` form. Anything unregistered
falls back to `▶ name(args)`, and a host adds its own tools with one
dict entry rather than a code change.
------------------------------------------------------------------------
source
### compact_call
``` python
def compact_call(
p, max_toks, enc:NoneType=None
):
```
*Render a tool-use part within `max_toks`, dispatching on
`call_renderers`.*
``` python
codex_part = tool_part('tools.mcp__clikernel__execute', code=python_str)
parts = [python_part, codex_part, bash_part, generic_part]
rendered = [compact_call(p, 40) for p in parts]
test_eq(rendered, [simple, simple, bash, call])
test_eq(compact_call(tool_part('py', code=python_str), 40), simple) # solveit runs python as `py` ...
test_eq(compact_call(tool_part('bash', cmd=bash_str), 40), bash) # ... and shell as `bash`, with its own arg key
print('\n'.join(rendered))
```
```python
files = ["a.py", "b.py"]
```
! printf "ready\n" ¶ ls -la
▶ inspect(path='fixture.json', lines=[1, 2])
### Message rendering
User messages are rendered part by part with `§`. Injected skill
contents are infrastructure rather than conversation-specific evidence,
but experience shows a resumed model won’t re-read them unprompted, so
each is replaced by a marker carrying its own instruction - and the
wrapping document ends by listing the exact `Skill(...)` calls to make
(below). Host plumbing parts - slash-command records, command stdout,
hook and system-reminder text - are dropped entirely: they are
self-obsoleting diagnostics, not conversation.
------------------------------------------------------------------------
source
### compact_user
``` python
def compact_user(
m, max_toks, enc:NoneType=None
):
```
*Render the text parts of a user message, dropping plumbing parts.*
------------------------------------------------------------------------
source
### compact_user_part
``` python
def compact_user_part(
p, max_toks, enc:NoneType=None
):
```
*Render one user-text part, replacing injected skill contents.*
``` python
user = compact_user(turn1[0], 100)
skill = compact_user(turn1[5], 100)
test_eq(user, '§ List the project files, use the project skill, then run a shell check. §')
test_eq(skill, '§ …skill text compacted: re-invoke before relying on it… §')
cmd = mk_msg('/compact\ncompact')
test_eq(compact_user(cmd, 100), '')
test_eq(compact_user(mk_msg('Compacted'), 100), '')
print(user)
print(skill)
```
§ List the project files, use the project skill, then run a shell check. §
§ …compacted… §
Assistant messages may interleave prose and tool calls. Text parts use
`»`, while tool-use parts are delegated to
[`compact_call`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_call);
preserving their order keeps the response readable as a sequence.
------------------------------------------------------------------------
source
### compact_asst
``` python
def compact_asst(
m, text_toks, call_toks, enc:NoneType=None
):
```
*Render an assistant message.*
------------------------------------------------------------------------
source
### compact_asst_part
``` python
def compact_asst_part(
p, text_toks, call_toks, enc:NoneType=None
):
```
*Render one assistant part.*
``` python
assistant = compact_asst(turn1[1], 100, 40)
expected = f'» I will inspect the project. »\n{simple}'
test_eq(assistant, expected)
print(assistant)
```
» I will inspect the project. »
```python
files = ["a.py", "b.py"]
```
Tool messages contain tool-result parts. The top-level dispatcher
selects the renderer from the canonical message role, giving the rest of
the compaction pipeline one entry point for every message.
------------------------------------------------------------------------
source
### compact_msg
``` python
def compact_msg(
m, user_toks, asst_toks, call_toks, result_toks, enc:NoneType=None
):
```
*Render one canonical message.*
------------------------------------------------------------------------
source
### compact_tool
``` python
def compact_tool(
m, max_toks, enc:NoneType=None
):
```
*Render the result parts of a tool message.*
``` python
policy = dict(user_toks=100, asst_toks=100, call_toks=40, result_toks=20)
rendered_turn1 = [compact_msg(m, **policy) for m in turn1]
test_eq(rendered_turn1[0], user)
test_eq(rendered_turn1[1], assistant)
test_eq(rendered_turn1[2], '> files = ["a.py", "b.py"]')
test_eq(rendered_turn1[5], skill)
print('\n'.join(rendered_turn1))
```
§ List the project files, use the project skill, then run a shell check. §
» I will inspect the project. »
```python
files = ["a.py", "b.py"]
```
> files = ["a.py", "b.py"]
▶ Skill(name='project-files')
> Skill loaded: project-files
§ …compacted… §
! printf "ready\n" ¶ ls -la
> ready
### Conversation rendering
A compact conversation groups messages into user-led turns and separates
those turns with `***`. Each message normally uses the bounded policy;
the final `last_n` canonical messages use an effectively unlimited
policy so the immediate conversational state remains intact. Only
substantive messages count toward `last_n`: a session often ends with
command plumbing (a `/compact` invocation, its stdout, a context
report), and letting those spend the untruncated slots would push out
exactly the wrap-up prose the rule exists to keep.
------------------------------------------------------------------------
source
### compact_chat
``` python
def compact_chat(
msgs, user_toks, asst_toks, call_toks, result_toks, enc:NoneType=None, last_n:int=0
):
```
*Render canonical messages as compact user-led turns.*
The sample contains three canonical user messages: the initial request,
injected skill text, and the second human request. The skill contents
therefore form a short intermediate turn, but are represented only by
the compact placeholder.
``` python
compact = compact_chat(conversation, enc=enc, last_n=5, **policy)
test_eq(compact.count('\n\n***\n\n'), 2)
test('§ …skill text compacted: re-invoke before relying on it… §', compact, in_)
test('The fixture is short, readable, and ready for the renderer.', compact, in_)
plumbed = conversation + [mk_msg('Compacted'), cmd]
test_eq(compact_chat(plumbed, enc=enc, last_n=5, **policy), compact)
print(compact)
```
§ List the project files, use the project skill, then run a shell check. §
» I will inspect the project. »
```python
files = ["a.py", "b.py"]
```
> files = ["a.py", "b.py"]
▶ Skill(name='project-files')
> Skill loaded: project-files
***
§ …compacted… §
! printf "ready\n" ¶ ls -la
> ready
***
§ Keep the agreed names; do not rename the fixture. §
» Understood. I will verify the final state. »
▶ inspect(path='fixture.json', lines=[1, 2])
> first line ¶ second line ¶ final line
» The fixture is short, readable, and ready for the renderer. »
Different message classes have different informational value. User text
gets the largest allowance because decisions and corrections cannot
safely be reconstructed; assistant prose and tool calls can usually be
re-derived; tool results retain a smaller evidential sample. The final
five messages are rendered without these limits.
A deliberately small policy makes truncation visible in this short
fixture. We set `last_n=0` so every message uses the limits; this is a
demonstration policy, not the production default.
``` python
small_policy = dict(user_toks=8, asst_toks=8, call_toks=8, result_toks=6)
small = compact_chat(conversation, last_n=0, **small_policy)
test(' … ', small, in_)
assert len_toks(small) < len_toks(compact)
print(small)
```
§ List the … check. §
» I will inspect the project. »
```python
files … "]
```
> files … .py"]
▶ Skill(name='project-files')
> Skill loaded: project-files
***
§ …compacted… §
! printf " … ls -la
> ready
***
§ Keep the … fixture. §
» Understood … state. »
▶ inspect(path … 2])
> first … final line
» The fixture … renderer. »
## Compaction documents
The DSL body is wrapped with a short legend and a pointer to the
complete transcript. The body supports orientation and reasoning, but it
is not authoritative source text: truncation, newline substitution, and
later edits mean code and commands must be reopened before reuse.
------------------------------------------------------------------------
source
### compact_content
``` python
def compact_content(
chat, path, notes:str=''
):
```
*Wrap compact conversation text as a continuation document.*
[`compact_content`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_content)
adds the legend, a `## Conversation` section, an optional notes section,
and the path of the authoritative transcript.
``` python
content = compact_content(small, '/tmp/session.jsonl')
test('## Conversation', content, in_)
test('/tmp/session.jsonl', content, in_)
print(content.split('\n\n## Conversation', 1)[0])
```
This session is being continued from an earlier conversation. `§ … §` encloses user text; `» … »` encloses assistant text; fenced `python` is persistent-kernel execution; `!` marks Bash; `▶` marks another tool call; `>` marks its result; `¶` replaces a newline; `***` separates user-led turns; and ` … ` marks removed middle content. The final five messages are preserved untruncated. This representation is for orientation only: never copy code or commands from it; reopen the original transcript or source file first.
------------------------------------------------------------------------
source
### compact_body
``` python
def compact_body(
content
):
```
*Extract the conversation body from a compaction document.*
[`compact_body`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_body)
removes the document wrapper and optional continuation notes, recovering
only the conversation DSL. This lets repeated compaction retain an
earlier immutable segment without recursively truncating it.
``` python
test_eq(compact_body(content), small)
with_notes = compact_content(small, '/tmp/session.jsonl', 'A useful note.')
test_eq(compact_body(with_notes), small)
```
Skill loads are the one thing the DSL compacts away entirely, so the
document ends by instructing their re-invocation. The list derives from
the `▶ Skill(...)` calls preserved in the body - including calls in
earlier immutable segments, so repeated compaction keeps the union - and
[`compact_body`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_body)
discards the instruction along with the rest of the wrapper.
``` python
tailed = compact_content(compact, '/tmp/session.jsonl')
test('re-invoke it: Skill(project-files)', tailed, in_)
test_eq(compact_body(tailed), compact)
assert 'Skill(' not in compact_content('§ hi §', '/tmp/session.jsonl')
```
------------------------------------------------------------------------
source
### join_compacts
``` python
def join_compacts(
*parts
):
```
*Join non-empty compact conversation segments.*
[`join_compacts`](https://AnswerDotAI.github.io/llmsurgery/compact.html#join_compacts)
concatenates non-empty conversation segments with the same `***`
separator used between user-led turns. Empty segments are ignored, which
handles sessions with no earlier synthetic compaction.
``` python
joined = join_compacts('first', '', 'second')
test_eq(joined, 'first\n\n***\n\nsecond')
print(joined)
```
first
***
second
## Dialog compaction
Everything above compacts *transcripts*, where nothing is curated and
every message class needs a budget. A dialog is different: notes, code,
and stored outputs are the way they are because their author chose to
keep them - they are load-bearing narrative, and trimming outputs is
already easy to do in a host or with the ipynb tools. The uncurated bulk
in a dialog is what the AI generated inside prompt messages. So dialog
compaction touches prompts alone: the question is capped by `user_toks`
(a pathology guard that rarely fires), and the reply is re-rendered in
the DSL.
There are two ways to project the result.
[`Dialog.compact`](https://AnswerDotAI.github.io/llmsurgery/compact.html#dialog.compact)
rewrites the replies in place, where message structure already says who
is speaking, so `§` and `»` disappear - reply prose sits bare, and only
tool calls, results, and code wear sigils - while `***` gives way to the
dialog’s own message boundaries. Because prompts keep their ids and
everything else is untouched, a compacted dialog remains a valid,
runnable notebook: re-running it regenerates the real outputs, and all
the message tools keep working on it.
[`dlg2compact`](https://AnswerDotAI.github.io/llmsurgery/compact.html#dlg2compact)
instead returns the whole dialog as one document, where nothing carries
structure and every sigil is back.
``` python
from aidialog.hist import chat2dlg
```
`chat2dlg` turns the familiar fixture into a dialog: one prompt per user
turn, replies carrying the tool calls as details blocks.
``` python
d = chat2dlg(conversation, 'demo')
test_eq([m.msg_type for m in d.messages], ['prompt']*3)
d
```
A reply is internally a mini-transcript: prose interleaved with
tool-call details blocks. `fmt2hist` recovers its canonical parts (the
same parse `dlg2chat` uses), and the DSL renderers above take over -
each call followed by its own result, matched by id. Prose sits bare by
default, because in the in-place form below the message structure
already says whose voice it is; `mark='»'` delimits it for the flat
form, where nothing else does. One wrinkle: `fmt2hist` appends a `'.'`
placeholder text part when a reply ends with a tool result (wire formats
require an assistant turn there); it carries no content, so the renderer
drops it.
------------------------------------------------------------------------
source
### compact_reply
``` python
def compact_reply(
reply, asst_toks:NoneType=None, call_toks:NoneType=None, result_toks:NoneType=None, enc:NoneType=None,
mark:str=''
):
```
*Render a prompt’s reply in the DSL: prose (bare, or delimited by
`mark`), each tool call followed by its result*
```` python
creply = compact_reply(d.messages[0].ai_res)
test_eq(creply, '''I will inspect the project.
```python
files = ["a.py", "b.py"]
```
> files = ["a.py", "b.py"]
▶ Skill(skill='project-files')
> Skill loaded: project-files''')
assert '»' not in creply and '§' not in creply
test_eq(compact_reply(d.messages[0].ai_res, mark='»').splitlines()[0], '» I will inspect the project. »')
print(compact_reply(d.messages[1].ai_res))
````
[`Message.compact`](https://AnswerDotAI.github.io/llmsurgery/compact.html#message.compact)
applies the policy in place, and only to prompts: the question is
mid-truncated by `user_toks`, and the reply is replaced by its DSL
rendering. Every other message type returns unchanged - that content is
curated, so compaction has no business with it. Budgets default to
`compact_policy`.
------------------------------------------------------------------------
source
### Message.compact
``` python
def compact(
user_toks:NoneType=None, asst_toks:NoneType=None, call_toks:NoneType=None, result_toks:NoneType=None,
enc:NoneType=None
):
```
*Compact a prompt message in place: cap the question, re-render the
reply in the DSL; other types are left untouched*
``` python
m = d.messages[0]
mid = m.id
m.compact()
test_eq(m.content, 'List the project files, use the project skill, then run a shell check.')
test_eq(m.ai_res, creply)
test_eq(m.id, mid)
nte = d.mk_message('A curated note with **important** rationale.', msg_type=snote)
test_eq(nte.compact(), nte)
test_eq(nte.content, 'A curated note with **important** rationale.')
m
```
[`Dialog.compact`](https://AnswerDotAI.github.io/llmsurgery/compact.html#dialog.compact)
maps
[`Message.compact`](https://AnswerDotAI.github.io/llmsurgery/compact.html#message.compact)
over the whole dialog and returns it. With the deliberately small
demonstration policy from earlier, truncation becomes visible inside the
replies while every question, note, and id survives; the result still
validates as a dialog.
------------------------------------------------------------------------
source
### Dialog.compact
``` python
def compact(
user_toks:NoneType=None, asst_toks:NoneType=None, call_toks:NoneType=None, result_toks:NoneType=None,
enc:NoneType=None
):
```
*Compact every prompt message in place; the rest of the notebook is
untouched*
``` python
d2 = chat2dlg(conversation, 'demo2')
ids = [m.id for m in d2.messages]
befor = sum(len_toks(m.ai_res) for m in d2.messages)
d2.compact(**dict(small_policy, user_toks=100))
test_eq([m.id for m in d2.messages], ids)
test_eq(d2.messages[2].content, 'Keep the agreed names; do not rename the fixture.')
assert sum(len_toks(m.ai_res) for m in d2.messages) < befor
test(' … ', d2.messages[2].ai_res, in_)
d2.validate()
print(d2.messages[2].ai_res)
```
The other projection is flat: the whole dialog as one document, for
pasting into another conversation’s context. It cannot go through
`dlg2chat`, whose job is the *wire* form - there, notes and code cells
are XML carrying ids, times, and meta, which is what a model needs
mid-conversation and exactly what a reader does not. So the flat
renderer walks the messages itself, and only prompts’ replies route
through the DSL.
[`compact_code`](https://AnswerDotAI.github.io/llmsurgery/compact.html#compact_code)
renders one code message: the source fenced, then its output as a result
line under the same budget a tool result gets. The `#_id` line above the
fence is the message’s Solveit link, and doubles as the signal that this
block is an authored cell rather than a tool call the AI made - a
distinction the fence alone cannot carry, since `py` calls render as
fences too.
------------------------------------------------------------------------
source
### compact_code
``` python
def compact_code(
m, call_toks:NoneType=None, result_toks:NoneType=None, enc:NoneType=None
):
```
*Render a code message: `#_id`, its source fenced, and its output as a
result line*
``` python
cm = Message('files = ["a.py", "b.py"]', msg_type=scode, output=code_output('2 files'))
ccode = compact_code(cm)
test_eq(ccode.splitlines()[0], f'#_{cm.id}')
test('```python', ccode, in_)
test_eq(ccode.splitlines()[-1], '> 2 files')
print(ccode)
```
[`dlg2compact`](https://AnswerDotAI.github.io/llmsurgery/compact.html#dlg2compact)
renders the whole dialog. Notes and prompt questions are `§` text on the
user’s side of the conversation, replies are `»`-marked, and code cells
sit between them as fenced blocks. Turns break after each prompt, so the
notes and cells leading up to a question travel with it. Skipped
messages and tagged raws are session bookkeeping rather than
conversation, so both are dropped, as they are on the way to the wire.
------------------------------------------------------------------------
source
### dlg2compact
``` python
def dlg2compact(
dlg, user_toks:NoneType=None, asst_toks:NoneType=None, call_toks:NoneType=None, result_toks:NoneType=None,
enc:NoneType=None
):
```
*Render a whole dialog as one compact document, breaking turns after
each prompt*
``` python
d3 = chat2dlg(conversation, 'mixed')
d3.mk_message('## Project notes', msg_type=snote, before=d3.messages[0])
d3.mk_message('files = ["a.py", "b.py"]', msg_type=scode, output=code_output('2 files'), after=d3.messages[1])
doc3 = dlg2compact(d3)
test_eq(doc3.count('\n\n***\n\n'), 2) # one turn per prompt
test('§ ## Project notes §', doc3, in_) # a note is a prompt with no reply
test('» I will inspect the project. »', doc3, in_) # the flat form marks reply prose
assert doc3.count('#_') == 1 and '