pages

octavo.pages reads a directory of markdown files and notebooks into Page objects. Each page holds its markdown, frontmatter settings, and HTML output path. For example, api/01_config.ipynb becomes a page with destination api/config.html.

Create a Page directly when you already have markdown, or use scan to read all documents under a directory:

page = Page('api/01_config.md', '# Configuration\n\nProject settings.')
page.dest  # 'api/config.html'

pages = scan('docs')
[(p.rel, p.dest) for p in pages]  # source and output paths

Scanning converts notebooks to markdown using their saved outputs; it does not run their cells unless you supply a processor that does so. Markdown {python} blocks, however, execute during scanning. Hidden files and directories, whose names begin with . or _, are skipped, as are paths matching the exclude globs that source_files takes.

This module reads sources and determines destinations; it does not render or write the site. octavo.site uses the collected pages to build navigation, resolve cross-page links, and write the output.

Frontmatter

Frontmatter is a YAML block at the start of a document, enclosed by --- lines. It supplies settings such as a page title, sort order, and old URLs to redirect.

frontmatter returns the parsed settings and the remaining document body. YAML numbers and lists retain their types.

This tutorial declares a numeric sort order and one old URL in aliases.

text = """---
title: End-To-End Walkthrough
order: 1
aliases:
  - /tutorial.html
---

Build a package from scratch, starting with a single notebook.
"""
meta,body = frontmatter(text)
meta

{'title': 'End-To-End Walkthrough', 'order': 1, 'aliases': ['/tutorial.html']}

Without frontmatter, the settings are empty and the document body is retained.

bare,barebody = frontmatter('# Hand written\n\nNo frontmatter here.\n')
test_eq(bare, {})
barebody

'# Hand written\n\nNo frontmatter here.\n'

Pages

A page’s output keeps its source directory but drops a leading numeric prefix from the filename. For example, api/01_config.md produces api/config.html. This lets authors order source files without including those numbers in public URLs.

Pass nm to choose a different output name without changing the source filename.

Page

def Page(
    rel:str, # Path of the document relative to the source root
    text:str, # Its Markdown, frontmatter included
    meta:dict=None, # Frontmatter; read from `text`, through `snake_keys`, when not given
    nm:str=None, # Output name without extension; derived from `rel` when not given
):

A source document, its metadata, and its output destination

page_nm

def page_nm(
    rel
):

Output name of rel: its stem without any numeric prefix

p = Page('tutorials/01_tutorial.md', text)
test_eq(p.dest, 'tutorials/tutorial.html')
test_eq(p.title, 'End-To-End Walkthrough')
p

tutorials/01_tutorial.md -> tutorials/tutorial.html

Without a frontmatter title, a page takes its title from a leading level-1 heading, as Quarto does. HTML comments before the heading, such as a generated-file notice, are skipped. A page with neither uses its output name. Pass nm="configuration" to produce api/configuration.html from api/01_config.md.

q = Page('explanations/why_nbdev.md', '<!-- Generated from notes.json -->\n# Why nbdev\n\nSome prose.\n')
test_eq(q.title, 'Why nbdev')
cfg = Page('api/01_config.md', 'Settings live here.\n', nm='configuration')
test_eq((cfg.dest, cfg.title), ('api/configuration.html', 'configuration'))
q

explanations/why_nbdev.md -> explanations/why_nbdev.html

Scanning a source tree

A site's source files are every file below its root except hidden paths, octavo.yml, and paths matching the site's exclude globs. A path is hidden when a component begins with . or _, which covers .git, .ipynb_checkpoints and _docs. Each source file becomes either a page or an asset: scan reads the Markdown documents and notebooks, and Site.build copies the rest, such as images, stylesheets and downloads, without rendering them and at the same relative path.

source_files lists them. It does not enter hidden or excluded directories.

source_files

def source_files(
    root:str, # Directory of source documents
    exclude:NoneType=None, # Glob patterns, relative to `root`, of paths to leave out
):

Paths relative to root of the files a site publishes: not hidden, not octavo.yml, and not matching exclude

hidden

def hidden(
    rel
):

Is any part of rel a dotfile or underscore-prefixed?

This source tree holds three Markdown documents, a notebook and an image, plus octavo.yml and a _docs directory left over from an earlier build, which are not source files.

src = Path(tempfile.mkdtemp())
mkdoc(src/'getting_started.md', {'title':'Getting Started'})
mkdoc(src/'api/01_config.md', {'title':'config', 'order':1})
mkdoc(src/'tutorials/tutorial.md', {'title':'End-To-End Walkthrough', 'aliases':['/tutorial.html']})
nb = dict2nb(cells=[mk_cell('# Notebook page', 'markdown')], metadata={}, nbformat=4, nbformat_minor=5)
write_nb(nb, src/'api/02_notebook.ipynb')
mkdoc(src/'_docs/stale.md', {'title':'Left over from an earlier build'})
(src/'octavo.yml').write_text('title: nbdev\n')
(src/'images').mkdir()
(src/'images/logo.svg').write_text('<svg/>')
files = source_files(src)
test_eq(files, ['api/01_config.md', 'api/02_notebook.ipynb', 'getting_started.md', 'images/logo.svg', 'tutorials/tutorial.md'])
files

['api/01_config.md', 'api/02_notebook.ipynb', 'getting_started.md', 'images/logo.svg', 'tutorials/tutorial.md']

exclude patterns are matched against paths relative to the root. A pattern that matches a directory leaves out everything under it, so tutorials drops the walkthrough. In these patterns * also matches /, so *.svg matches SVG files at any depth.

test_eq(source_files(src, ['tutorials', '*.svg']), ['api/01_config.md', 'api/02_notebook.ipynb', 'getting_started.md'])

scan reads the Markdown documents and notebooks among the source files. Markdown {python} blocks execute during reading. Notebook conversion uses saved content and outputs. A caller can supply nbproc to process notebooks first, such as nbdev’s generated API documentation and directives.

scan

def scan(
    root:str, # Directory of source documents
    nbproc:NoneType=None, # Optional callable applied to notebooks before conversion
    exclude:NoneType=None, # Glob patterns of paths to leave out; see `source_files`
):

Every Markdown document and notebook under root, as Pages

ipynb2md

def ipynb2md(
    path:str, # Notebook to convert
    nbproc:NoneType=None, # Optional callable that processes the notebook in place
):

Convert a notebook to document Markdown with code-cell publishing directives

seen = []
def mark_nb(nb): seen.append(len(nb.cells))
pages = scan(src, nbproc=mark_nb)
test_eq(seen, [1])
pages

[api/01_config.md -> api/config.html, api/02_notebook.ipynb -> api/notebook.html, getting_started.md -> getting_started.html, tutorials/tutorial.md -> tutorials/tutorial.html]

Each scanned page already has the destination and title needed for navigation.

{o.dest: o.title for o in pages}

{'api/config.html': 'config',
 'api/notebook.html': 'Notebook page',
 'getting_started.html': 'Getting Started',
 'tutorials/tutorial.html': 'End-To-End Walkthrough'}

The same pages supply redirect destinations for their declared aliases.

{a: o.dest for o in pages for a in o.aliases}

{'/tutorial.html': 'tutorials/tutorial.html'}

Markdown sources can generate content with {python} fences. Blocks share a namespace within one document, while ordinary python fences remain displayed examples. Scanning keeps the frontmatter and the computed Markdown together in Page.text; both output formats use that text without executing it again.

mkdoc(src/'api/01_config.md', {'title':'config', 'order':1}, body=r'''
```{python}
name,score = 'Ada',42
```

```{python}
print(f"| Name | Score |\n| --- | ---: |\n| {name} | {score} |")
```

```python
raise RuntimeError('This is a displayed example')
```

Literal {{name}} remains prose.
''')
computed = next(o for o in scan(src) if o.rel=='api/01_config.md')
test_eq(computed.meta, {'title':'config', 'order':1})
assert '| Ada | 42 |' in computed.text
assert '```{python}' not in computed.text
assert '```python' in computed.text and 'raise RuntimeError' in computed.text
assert '{{name}}' in computed.text
print(computed.text)

---
title: config
order: 1
---


| Name | Score |
| --- | ---: |
| Ada | 42 |

```python
raise RuntimeError('This is a displayed example')
```

Literal {{name}} remains prose.

Generated files

A build can include files produced by another tool. For example, nbdev can execute images/circles.svg.py and pass the resulting SVG to Octavo as a GeneratedFile.

The object records the source path, output path, and text to write. The site builder excludes the source script from copied assets. It rejects generated destinations that collide with pages, copied assets, or other generated files.

GeneratedFile

def GeneratedFile(
    src:str, # Path of the source it was generated from, relative to the source root
    dest:str, # Path to write, relative to the site root
    content:str, # File contents as text
):

A file produced in memory, written verbatim to the output

This generated file writes images/circles.svg and excludes images/circles.svg.py from copied assets.

gen = GeneratedFile('images/circles.svg.py', 'images/circles.svg', '<svg/>\n')
test_eq(gen.src, 'images/circles.svg.py')
test_eq(gen.dest, 'images/circles.svg')
gen

images/circles.svg.py -> images/circles.svg