vpsvip 0.4.0 - design notes

PURPOSE

Gratitude for sweet water and purity of purpose translate into practical design
principles: readable code, accurate native information, administrator control,
and care for existing configuration. Reduce scope before sacrificing those
qualities. Prefer a small honest workflow to an unreviewable promise.

BOUNDARIES

The panel coordinates native OpenBSD tools. It has no resident daemon, remote
control socket, web server, replacement database for native configuration, or
monitoring loop.
Native configuration files, service scripts, package records, and the kernel
remain authoritative. A broad map does not imply a form for every native flag.

system_map.py is data and lookup: 12 top-level categories and 47 exact read-only
command tuples. Codes are explicit stable
paths; additions must not renumber existing entries. A code only navigates.
map_ui.py owns category traversal, search, program review, and the sysctl tree.
Menu positions are keyed by screen and item identity, not a single global
cursor. Numeric navigation is confined to menus/viewers; field digits are text.
Configuration labels put the actual basename before its purpose; full target
paths remain available. Native manual labels begin with name(section).
System > Manuals groups general references, configuration-file syntax, and
administration commands. Topic-specific references remain with their tools.
Panel holds only application functions. Relocated manual codes 12*3, 12*4,
and 12*5 are reserved lookup aliases; they do not duplicate entries in the
tree, searches, or CLI listing. All other released codes remain unchanged.

system_tools.py discovers executable paths, reads native manuals and sysctl
values, checks report membership, and constructs reviewed program argv. It scans
ten fixed directory groups without executing programs for discovery. Only
libexec trees recurse, to depth three. A scan is bounded to 50,000 directory
entries and 20,000 programs; omissions and access problems remain visible.
Dynamic program identity is the absolute path, not an invented map code.

The report allowlist and arbitrary native-program workflow have distinct
contracts. Reports accept exact fixed argv only. A program is rechecked against
the discovered inventory before launch; root invocation additionally checks
trusted executable ownership and path components. This does not sandbox trusted
program behavior or remove native destructive options. native_ui.py hands the
terminal to an explicitly reviewed executable, argv, and working directory.
The native program may have its own command language; the panel itself uses
no shell evaluation. Manuals and synopses come from the installed system.

execution.py owns the shared executable and environment boundary. Runner,
foreground commands, and editors validate the program actually being launched.
When root runs the panel, every executable path component and symlink target
must belong to root and be protected from writes by other accounts. Bare editor
names resolve against a fixed system PATH. Explicit command arguments remain
trusted administrator input; this is not an executable sandbox.

services.py owns command capture and rcctl behavior. Runner accepts an absolute
executable and argument vector, supplies a small fixed environment, and uses /
as the working directory. It preserves exit status and normally
captures up to 1 MiB of combined output. Repository-index downloads explicitly
raise that bound to 16 MiB per index. An incremental decoder preserves split
UTF-8 sequences while streaming. Display code escapes terminal controls.
Observer failures do not abort a child process. Ordinary reads and syntax
checks have deadlines. Service mutations and package writers explicitly use
no deadline; native rc.d lifecycle limits govern service changes.
Pipe creation, nonblocking setup, and selector registration finish before child
launch. Once the immediate child exits, a 250 ms tail-drain window bounds waits
for descendants holding stdout. Incomplete output is marked truncated with the
native exit status preserved. A completed command's group is not killed merely
because output remains open. Closing the capture pipe can still give a later
writer SIGPIPE; custom background programs must redirect their output. Normal
rc.subr actions already redirect it unless debug mode is selected.
Unexpected monitor/read failures after launch can still terminate a live
command group. The no-timer policy cannot guarantee completion through every
local I/O failure; errors require inspecting the actual system state.

Service membership is checked against rcctl before action. Because rcctl uses
native shell service scripts, daemon flags receive an additional conservative
character check. A # is refused because native configuration parsing treats it
as a comment. Valid leading-underscore names are accepted. Boot-enabled and
running-now are separate. Whole-query failure
or unavailable inspection is unknown; successful runtime output follows rcctl's
own checks and cannot independently distinguish every script failure from a
stopped daemon. Commands are never silently forced, retried, or rolled back.
Single-service status uses fresh ls all membership, get NAME status, and, for
root inspecting a daemon script, check NAME. It never runs ls started or ls on
for this path. get status returns its value through an otherwise silent exit
status; a daemon check must match rc.subr's NAME(ok)/NAME(failed) response before
the UI labels it running/stopped. Ambiguous diagnostics remain unknown.

config.py owns one-file transactions in /etc, /var/unbound/etc, and /var/nsd/etc.
Directory descriptors and no-follow opens resist ordinary link substitution.
The file's identity, metadata, and bytes are compared before installation; the
private candidate is checked again after native validation. ConfigReview binds
the displayed diff and text to one candidate's bytes, metadata, and transaction.
The UI passes that same review to commit; changed candidates require review
again. Standalone validation also uses an isolated private copy. Parent/stage
path identity is checked around validation, and both the draft and installation
copy are rechecked after backup work. Nonblocking opens reject substituted FIFOs.
Cleanup removes only the pinned staging directory, never a replacement at its
old name. A durable private
backup precedes atomic replacement. Existing metadata is carried to the new
file. A directory-sync error after replacement is reported as already installed
with incomplete durability, not as an untouched target.

catalog.py registers 47 default paths, 14 native syntax checks, and file-effect
notes. operations.py combines service/configuration operations without curses,
including backup-before-rcctl for startup changes. Installation issues no
service action, but readers can use the new contents immediately or later.
cron's automatic /etc/crontab reload is an explicit example. A validator checks
only its supported main file; includes, derived databases, and external state
are not a multi-file transaction. Alternate daemon paths remain explicit.

templates.py owns two intentionally small configuration languages: a single
HTTP site and local mail. Inputs are validated before rendering. Parsing means
strict recognition of the entire generated format, including comments. A parser
returning None refuses ownership; it is not permission to approximate an
unfamiliar file. Explicit replacement is a separate reviewed operation.
New output uses the vpsvip header. Parsing also accepts the exact legacy
vpsvippanel or rcpanel header with the same complete format; no other differences are
normalized. The program rename itself does not write a service configuration.

tui.py owns reusable terminal widgets, staged editing, confirmations, and the
worker lifecycle. presentation.py formats results and renders terminal controls
visibly. package_ui.py owns package navigation and a bounded progress tail.
Input prompts wrap escaped help text to the current width before positioning
the entry field. Long help uses a bounded viewport with a visible range and
Page Up/Page Down, leaving normal field editing intact. Redraw recalculates
wrapping after a resize; size clamps also cover a shrink during redraw.
Text viewers distinguish Page Up/Page Down paging from n/N search matches and
mark End when the last output row is visible. Space also advances a screen.
viewer_layout.py keeps line identity without duplicating display rows. Unwrapped
text needs no index. Wrapped text uses one compact cumulative row offset per
original line and slices only the visible rows. Resize and search retain their
original-line and character-offset anchors. A search scrolls to the matching
character in wrapped or unwrapped output; resizing preserves the visible
character within a long line. Prompt editing indexes the original text while
cursor placement accounts for the escaped ASCII display. The printable helper returns already-safe ASCII text
directly; all other text still passes through the control-escaping path.
Escape sequence recognition uses a 100 ms delay where curses supports it.
Back keeps service, package, kernel, and directory snapshots instead of issuing
new queries. Successful explicit refreshes replace snapshots and their warning
lists together. Configuration browsing retains only the current ancestry,
discarding a child listing when moving up. Mutation attempts increment local
revisions so failed or partial changes also mark displayed state as stale.
branding.py holds the startup notice and credits. Every interactive launch
shows that notice; text-output modes do not. An ordinary key continues,
resizing does not dismiss it, and normal interrupt/suspend controls remain
available. Preferences cannot disable the notice or action confirmations.
The demo substitutes temporary configuration and in-memory service, package,
kernel, and report data; inventory is illustrative. Native administrative tools
are never called in demo. The user-selected editor is still a real process
editing the temporary draft.

PREFERENCES AND LOCAL STATE

preferences.py keeps panel state separate from native service configuration.
StateStore uses .config/rcpanel beneath the effective user's password-database
home, ignoring inherited HOME. Demo supplies a temporary directory instead.
State access checks ownership, permissions, links, file type, and size; writes
replace a private file atomically. Native configuration policies are not
loosened to accommodate panel state.
First writes synchronize newly created directory links before writing the
file; subsequent writes synchronize the temporary file and final directory.

Preferences is immutable. Its fields are show_hints, remember_positions,
viewer_wrap, page_overlap, start_code, editor, program_directory, favorites,
and recent_limit. Codes must be canonical and present in the static map.
Exact legacy manual codes are normalized only while decoding saved preferences.
Alias/canonical favorite collisions collapse in first-occurrence order;
literal duplicates and oversized lists remain invalid. Loading does not write
or alter the original concurrency baseline. An explicit Save writes the new
canonical codes. Draft validation remains strict.
Favorites are unique and bounded to 100. Page overlap is 0-10; recent_limit
is 0-50, with zero disabling recording. Text fields reject controls and invalid
editor quoting. Settings choose behavior; they are not executable startup jobs.

settings_ui.py edits a local draft. Back discards it; Save validates, persists,
and only then applies it to the session. A save failure leaves the live
preferences unchanged. Missing files yield defaults without writing. An invalid
file is retained and reported; ordinary saving cannot silently replace it.
Only confirmed default reset may replace malformed preference contents, and
that permission never bypasses unsafe filesystem checks.
Changed on-disk preferences are detected before saving. Reload reads and applies
the current saved file; replacing a changed draft requires confirmation and
does not write. A directory-sync error after replacement explicitly reports
that the file was saved; active preferences remain unchanged until a successful
save or reload applies them.

Preferences and favorites persist in preferences.json. Menu positions and
recent-page history remain session state, bounded by the selected preferences.
Viewer wrapping has a saved default and a temporary per-viewer toggle; text
search and next/previous match navigation do not change saved preferences.
The editor override precedes EDITOR, which precedes /usr/bin/vi.

presets.py stores up to 100 named executable/argument/directory records in
commands.json. Loading one reopens the native program page; it never executes
anything. Every invocation still requires review, confirmation, current
inventory membership, and normal trust checks. Default preference reset does
not erase these records or the downloaded package catalog.

PACKAGE PLAN CONTRACT

packages.py owns fixed pkg_info/pkg_add/pkg_delete argv, installed membership,
catalog/search membership, preview issuance, and application. Search accepts
a plain package-name fragment; URLs, paths, leading options, and package-spec
operators are not a second input language. Installation uses an exact result
from the saved compatible catalog or last successful search. Removal/update
use exact installed names.

The full available-name catalog comes from explicit bounded repository-index
downloads, not an empty pkg_info -Q argument. Native fragment search is a
separate operation. The cached available-packages.json records names, source,
and UTC download time. Browsing filters local names against the installed
inventory without repeatedly contacting the mirror. Refresh replaces the cache
only after complete validation; download/parse failure preserves the previous
file and reports its status. A changed mirror, release, or package architecture
makes the cache incompatible for authorizing an install. Cached names still
need a fresh native preview and the native package tools' signature/dependency
checks.

One browser-entry snapshot reads and validates a single catalog record for its
status and names, then queries installed membership once. The decoded record
is not retained across entries or used to authorize later changes. Install
planning still opens and validates the saved catalog independently.

repository_catalog.py resolves /etc/installurl with /usr/bin/arch -s and
/sbin/sysctl -n kern.version. It uses /usr/bin/ftp -V -M -o - with a 120-second,
16-MiB bound per index and extracts validated .tgz names from HTML anchors.
A release merges packages/ARCH and optional packages-stable/ARCH; only an
explicit HTTP 404 makes the optional index absent. Beta/current systems use
snapshots/packages/ARCH. Cached data is limited to 4 MiB and 50,000 names.
Incremental HTML parsing limits unfinished markup to 8 KiB before a complete
oversized tag can expand into a large attribute list. Long ordinary text and
large valid indexes remain supported within the overall byte bound.
Replacement uses StateStore; a post-replacement sync failure reports that the
complete new file was saved rather than claiming the previous file remains.

A plan records native -n/-I preview output and status, final argv, a random
one-use token, and a fingerprint of the complete parsed installed inventory
plus manual-install tags. Both are checked around preview and before apply.
Live search and all repository-dependent plans also bind to bounded, trusted
/etc/installurl contents, checked after search/preview and before apply. A
repository change invalidates those operations. Deletion does not depend on a
working mirror. Frozen records give issued plans and search provenance named
fields rather than positional tuples.
Failed, truncated, forged, altered, stale, foreign, or already-used plans cannot
apply. At most 16 outstanding plans are retained per manager. A consumed plan
must be previewed again, including after an unsuccessful apply attempt.

A per-manager mutex and a root-owned mode-0600 advisory file at
/var/run/rcpanel-packages.lock serialize panel package operations. The lock is
released between preview and confirmation; freshness is checked on apply.
Other package clients need not use that lock. The fingerprint is not a snapshot
of all package metadata or the repository, and native dependency checks and
native locking remain authoritative. Preview is advisory, not a frozen install.

Package writers run without an automatic kill timer. Progress streams to the
UI while final capture remains bounded; a failed observer does not kill the
writer. Ordinary package/script output and the native exit status remain
inspectable. Failure may follow partial changes. There is no automatic retry,
rollback, force flag, unsigned-package override, or panel dependency resolver.
The fixed environment excludes ambient PKG_PATH/PKG_DBDIR settings; native
repository configuration is /etc/installurl. The working directory is /.

KERNEL AND FAILURE SEMANTICS

The sysctl browser groups the actual current names, with bounded parsing and
visible unparsed-output warnings. Presence does not establish writability.
Assignments require an exact currently listed name, root, and confirmation.
A native failure remains a failure. Runtime writes do not update sysctl.conf;
editing that file does not apply its assignments to the running kernel.
An explicit read refreshes the displayed snapshot from actual kernel values.
Successful inventories retain scalar names for targeted reads, never cached
values or write authorization. A selected read queries only that name and
parses the exact successful command response, including indented continuation
lines. Missing framing, mismatched keys, extra records, and incomplete results
leave the displayed snapshot unchanged. Assignments still require a fresh
inventory, and a cancelled inventory does not publish a partial name set.
Writes mark the prior snapshot stale; they do not assume the requested string
was stored or force a new inventory query while the user is backing out.

Panel-managed jobs use a worker while the terminal redraws. Read call sites
explicitly opt into cancellation; the safe default protects writes. jobs.py
binds one CancelToken in thread-local scope. Escape/4/q requests cancellation;
0 also requests a return to the main map. JobCancelled is separate from
PanelError so optional native probes cannot swallow it as a warning.
Navigation consumes cancellation without presenting an error or late result.

Runner checks before launch, during output capture, and in 100 ms wait slices
after pipe EOF. A cancelled read kills and reaps a still-running child using
the existing process-group safeguards. Discovery, parsing, and multi-query
work check cooperatively, including before publishing package searches,
previews, or a replacement downloaded catalog. Atomic cache publication itself
is not interrupted. A filesystem call blocked inside the operating system
must return before the next checkpoint can run.

The UI always joins a worker before transaction contexts close descriptors;
there are no detached workers or stale callbacks. Signal handlers only set
flags, never the locking cancellation Event. Ctrl+C cancels reads and leaves
protected writes running. SIGHUP/SIGTERM cancel reads or await writes, then
close the panel after cleanup. Requested suspension is deferred until the
worker exits.
Service and package writes can take indefinitely long because these native
operations have no panel deadline. A deferred close does not undo a completed
change. Native
foreground programs and the editor use their own terminal interaction and
normal foreground process-group job control. terminal_control.py restores the
panel terminal around an idle suspension and redraws after resumption.

Configuration locks cannot constrain arbitrary root programs or atomically
snapshot includes. Observed conflicts stop installation; another root process
can still race final replacement. Native syntax checks do not prove routing,
credentials, port availability, content access, or future mail delivery.
Failure after a command timeout requires inspection of actual state.

EDITOR is an explicit trust boundary. The chosen editor receives the panel's
privileges and is not sandboxed. Its candidate is reviewed after it exits.
The editor must remain in the foreground, using a wait option when necessary.
It uses the shared executable checks, / as its working directory, an explicit
account environment, and child umask 077. Caller HOME, editor/shell startup
variables, loader/Python overrides, and arbitrary GUI variables are omitted.
An explicit EDITOR passed to an advanced native program is checked and quoted
as literal argv before export. The caller's umask remains in effect for service
tools and advanced native programs, whose native creation modes are deliberate.
Its parent catches SIGINT while the child retains normal interrupt behavior.
Launch/parsing failures and nonzero exits do not install a draft. Terminal
restoration failure is distinguished from ordinary unsupported cursor control.

INSTALLATION AND SCOPE

The installer publishes a complete version directory before replacing the
launcher. An upgrade to 0.4.0 retains older version directories. Identical
application modules may be reinstalled; changed code at an existing version is
refused before launcher replacement. An intentional custom build needs a unique
version instead of overwriting the active directory. The launcher binds
an exact interpreter and uses -I, adding only the selected application path.
Ambient Python import settings do not select privileged application modules.
Live installation checks the interpreter's executable trust before writing.
Writable directories, including sticky directories, are refused within the
installation prefix. The sticky-root exception is only for temporary ancestors
above a private package prefix; it cannot admit extra importable modules into
the installed tree. Version files and directory entries are synchronized before
the launcher switch. A post-replacement directory-sync failure identifies the
file as already replaced.
The install is not a native OpenBSD package and does not register a dependency
on Python; administrators must retain its interpreter.
The installer writes the launcher, application tree, manual, and documentation.
It does not alter services, native configuration, or saved panel state. Upgrade
instructions extract a fresh archive, use the actual versioned Python and the
same installation prefix, and verify the launcher's --version result.

The 0.4.0 rename uses the vpsvip launcher, vpsvip_lib Python package,
libexec/vpsvip version tree, share/doc/vpsvip directory, and vpsvip(8) manual.
Earlier libexec/vpsvippanel and libexec/rcpanel trees remain intact. Both
sbin/vpsvippanel and sbin/rcpanel can be updated as compatibility commands when
their complete contents match this project's trusted generated launchers for
the same prefix. That includes rcpanel launchers already targeting vpsvippanel
0.3.6. Other historical entries are left unchanged with a notice. Clean
installations create only vpsvip. An unrelated or customized primary vpsvip
command is a collision and is refused, not overwritten. The primary command is
rechecked before replacement; an observed concurrent change also stops it.

Version syntax, interpreter spelling, required entry modules, source compilation,
and document reads are checked before publishing a new version. Version files
and documentation are complete and synchronized before the launchers are
replaced. These are separate atomic replacements, not a multi-file transaction;
a compatibility failure identifies the already installed new command.

State and coordination names deliberately do not follow the public rename:
.config/rcpanel, .rcpanel-backups, .rcpanel.lock, .rcpanel-stage-*, and
/var/run/rcpanel-packages.lock retain their meanings. Existing state and backups
remain usable without migration; old and new processes share the same locks.
Changing those names independently would split the configuration/package
coordination boundary and could strand users' saved preferences.

This release supplies broad access to native tools plus a small number of
structured workflows. It has no general parser for hand-maintained service
configuration, certificate issuance workflow, public-mail/IMAP deployment form,
multi-file transaction, package rollback engine, or automatic firewall policy.
Native OpenBSD acceptance remains a separate unfinished gate from the portable
test suite. See OPENBSD-CHECKLIST.txt and TEST-RESULTS.txt.
SECURITY-REVIEW.txt records the security pass introduced in 0.3.1.

0.4.0 REVIEW DETAILS

Built-in draft updates write and fsync a private copy, recheck the old draft,
and replace it only when complete. Failed write/fsync/replace preserves the
previous draft; a concurrent editor change is retained and the update refused.
Identical bytes do not invalidate a review or cause a needless write. This
covers the panel's writes; an external editor still owns its save semantics.
Native validation rejects timed-out, failed, or truncated output.

StateStore records a temporary file as its own only after exclusive creation.
Its cleanup releases coordination descriptors even when unlink also fails.
Package lock cleanup likewise releases the in-process mutex even if descriptor
close fails. Package previews are explicitly discarded when the review ends;
a worker-side holder permits cleanup even when cancellation wins after plan
creation and before busy hands its result to the UI. The one-use preview cap
remains as a backstop for non-UI callers.

setup_ui.py composes the existing reviewed operations rather than implementing
a second mutation backend. It discovers rc.d daemons, retains explicit
snapshots, delegates configuration and service changes, and refreshes status
only on request. Boot state and runtime state remain separate. Setup neither
stores a completed/ready checkbox nor treats successful rcctl configtest output
as proof of service readiness. Known file validators remain authoritative;
unregistered administrator-selected files retain manual review.

The CLI's --status SERVICE path uses ServiceManager.status, including fresh
membership checks and targeted boot/runtime reads. --status without a name
retains the full inventory table. Static map code 8*7 adds guided setup without
renumbering any released entries.
