pshnb IPython magic

Provides psh persistent bash magics in Jupyter and IPython

__file__ = './00_core.ipynb'

Foundations

env = dict(os.environ, TERM='dumb', PS1='$', PS2='$')
eshell = os.environ['SHELL']
sh = pexpect.spawn(eshell, encoding='utf-8', env=env)

This cell creates the initial shell process using pexpect.spawn() with a basic environment configuration. The key setup includes:

• Environment variables: Sets TERM='dumb' to prevent terminal formatting issues, and standardizes prompts with PS1='$' and PS2='$'
• Shell spawning: Creates a persistent shell process using the user’s default shell ($SHELL) with UTF-8 encoding
• Process persistence: The sh object maintains a continuous connection to the shell, allowing commands to be sent and responses captured across multiple interactions

PS1 is the primary shell prompt (what you see when the shell is ready for a command), and PS2 is the secondary prompt (shown when a command spans multiple lines or is incomplete). pexpect is a Python library for controlling interactive command-line programs by automating the sending of inputs and reading of outputs. It’s particularly useful for automating tasks that normally require human interaction. spawn is pexpect’s main function that starts a child process (like a shell) and returns a pexpect object that can communicate with it. Unlike subprocess, spawn maintains an interactive session - you can send commands, wait for responses, and continue the conversation with the same process instance.

This establishes the foundation for persistent shell state - variables, directory changes, and other shell modifications will persist between command executions.

env = dict(os.environ, TERM='dumb', PS1='', PS2='')
sh = pexpect.spawn(eshell, encoding='utf-8', env=env)
sh.sendline('stty -echo')
sh.readline()
echo = os.urandom(8).hex()
echo_re = re.compile(fr'^{echo}\s*$', flags=re.MULTILINE)
sh.sendline(f'export PS1=""')
sh.sendline('set +o vi +o emacs')
sh.sendline('echo '+echo)
sh.expect(echo_re, timeout=2)

0

This cell implements a more sophisticated shell setup that addresses output formatting and command echo issues:

• Echo suppression: Uses stty -echo to prevent command echoing, which is crucial for clean output capture
• Empty prompts: Sets PS1="" and PS2="" to eliminate prompt strings that would interfere with output parsing
• Line editing disabled: Runs set +o vi +o emacs to disable interactive line editing features that can cause issues in programmatic shell interaction
• Synchronization mechanism: Establishes a unique echo token using os.urandom(8).hex() and corresponding regex pattern for reliable command completion detection

The echo token system is particularly important - it provides a reliable way to detect when a command has finished executing by sending a unique marker and waiting for it to appear in the output.

sh.sendline('ls | head -3')
sh.sendline('echo '+echo)
sh.expect(echo_re, timeout=2)
print(sh.before)

00_core.ipynb
CHANGELOG.md
index.ipynb

This cell demonstrates the practical application of the configured shell for command execution and output capture:

• Command execution: Sends ls | head -3 to list the first three files in the current directory
• Completion detection: Uses the echo token mechanism (echo [random_hex]) followed by sh.expect(echo_re) to reliably detect when the command has finished
• Clean output extraction: Captures the command output via sh.before, which contains everything received before the expected echo token

This pattern shows how the kernel can execute arbitrary bash commands while maintaining clean separation between command output and shell control mechanisms. The timeout parameter (2 seconds) provides protection against hanging commands, which is essential for a responsive interactive environment.

---

source

ShellInterpreter

 ShellInterpreter (debug=False, timeout=2, shell_path=None, sudo=False,
                   dumb=True)

Initialize self. See help(type(self)) for accurate signature.

sh = ShellInterpreter()
print(sh('ls | head -3'))

00_core.ipynb
CHANGELOG.md
index.ipynb

NB: requires passwordless sudo to use this:

# sh = ShellInterpreter(sudo=True)

# sh('cd')
# print(sh('pwd'))
# sh('cd ..')
# print(sh('pwd'))
# print(sh('whoami'))

---

source

shell_replace

 shell_replace (s, shell=None)

Replace @{var} refs in s with their variable values, if they exist

b = 1

a = '''asdf
$@{b} @{aa}
fdsa'''

print(shell_replace(a))

asdf
$1 @{aa}
fdsa

---

source

PshMagic

 PshMagic (shell, sudo=False, timeout=2, expand=True, o=None)

Initialize self. See help(type(self)) for accurate signature.

These changes allow -r to work as before (reset with default shell) and also accept an optional shell path like -r /bin/zsh or --reset=/bin/bash.

---

source

create_magic

 create_magic (shell=None)

# Only required if you don't load the extension
create_magic()

%bash pwd

/Users/jhoward/aai-ws/pshnb

%bash ls -h

00_core.ipynb   LICENSE     pshnb.egg-info  settings.ini
CHANGELOG.md    MANIFEST.in pyproject.toml  setup.py
index.ipynb pshnb       README.md   styles.css

%bash cd ..

%bash pwd

/Users/jhoward/aai-ws

%%bash
cat > tmp << EOF
hi
there
EOF

%bash cat tmp

hi
there

%bash rm tmp

%bash ls | head -3

_nbs
_proc
addnew.py

n = 2

%bash echo @{n}

2

%bash ls | head -@{n}

_nbs
_proc

%%bash
echo starting
(sleep 1; echo finished) &

starting
[1] 18411

%bash

finished

[1]+  Done                    ( sleep 1; echo finished )

%bash -h

::

  %bash [-h] [-r [RESET]] [-o] [-x] [-X] [-s] [-S] [-t TIMEOUT]
            [command ...]

Run line or cell in persistent shell

positional arguments:
  command               The command to run

options:
  -h, --help            Show this help
  -r <[RESET]>, --reset <[RESET]>
                        Reset the shell interpreter (optionally choose shell)
  -o, --obj             Return this magic object
  -x, --expand          Enable variable expansion
  -X, --no-expand       Disable variable expansion
  -s, --sudo            Enable sudo
  -S, --no-sudo         Disable sudo
  -t TIMEOUT, --timeout TIMEOUT
                        Set timeout in seconds

%bash pwd

/Users/jhoward/aai-ws

Reset the interpreter:

%bash -r

%bash pwd

/Users/jhoward/aai-ws/pshnb

%bash echo $SHELL

/opt/homebrew/bin/bash

sudo:

%bash -s

%bash whoami

root

no sudo:

%bash -S

%bash whoami

jhoward

timeout:

%bash -t 1

try: get_ipython().run_line_magic('bash', 'sleep 2')
except TIMEOUT: print("timed out")

timed out

---

source

load_ipython_extension

 load_ipython_extension (ipython)

Required function for creating magic

---

source

create_ipython_config

 create_ipython_config ()

Called by pshnb_install to install magic