Tere — web-based shell sessions, securely

Tere is Estonian for hello. Terebra is Latin for a mechanism to bore holes in fortified walls. It's also a genus of sea snails with distinctive shells. Tere also happens to be an abbreviation of terminal emulator.

Tere is a radical rethinking of the terminal emulator + SSH ecosystem. As such, it's an experiment that might go nowhere; that's okay.

We aim to replace OpenSSH with HTTPS and WebAuthn. Those need to be made secure anyway, so this decreases what needs to be trusted. We aim to do this with Rust, without root, and using strong privilege separation and sandboxing.

Any web browser will work as a client. A command-line client can use the same protocol and codebase.

We aim to obsolete (at least) these technical limitations:

• SSH runs as root
• Session lifetime is bound to a single TCP connection.
• Passwords and "keys" are stored in files.
• Remote SSH hosts can fully control your local terminal, which is used also for other things. Avoiding this reduces the attack surface.
• Complete and uncontrolled delegation of authentication to remote host; SSH agent forwarding is fundamentally unable to let you see what you are allowing.

Current status

Just about nothing exists yet. Hold on.

Architecture

Caution: This document contains wishful thinking. Not everything is true yet.

The Tere server is written in Rust. It does not run as root or need CAP_ADMIN.

It uses systemd socket activation (1, 2) to serve HTTPS¹. TLS termination may be done by a proxy in front of the service, if wanted.

The browser client uses hterm, at least for now². User sessions are transported over a WebSocket connection. Sessions survive TCP connection loss and IP address changes.

User authentication is done with WebAuthn. This means user must have suitable hardware, for example a YubiKey.

Sessions are run via systemd, with the same mechanism as machinectl shell.

D-Bus

D-Bus is the systemd-affiliated "message bus". It's basically a baroque IPC mechanism transported over UNIX domain sockets, with a "hub" process in the middle.

We're required to use it in order to programmatically do what machinectl shell does, which lets us open new session the right way, either on the host or in containers. The ability to use D-Bus will be sandboxed and privilege separated.

As this allows our service to start shells as arbitrary users, it requires elevated privileges. Instead of running as root, or needing CAP_ADMIN, we instead simply configure our system account to be allowed to do that. This happens in two steps.

1. Tell dbus-daemon to let us make the relevant D-Bus method call, via /usr/share/dbus-1/system.d/50-tere.conf.

<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
 <policy group="tere-dbus">
  <allow
        send_type="method_call"
        send_destination="org.freedesktop.machine1"
        send_path="/org/freedesktop/machine1"
        send_interface="org.freedesktop.machine1.Manager"
        send_member="OpenMachineShell"
        max_fds="0"
        />
 </policy>
</busconfig>

# systemctl reload dbus.service

2. dbus-daemon confirms that this is allowed via polkit, yet another service. Allow it via /usr/share/polkit-1/rules.d/50-tere.rules.

polkit.addRule(function (action, subject) {
  if ((action.id == "org.freedesktop.machine1.host-shell" ||
    action.id == "org.freedesktop.machine1.shell") &&
    subject.groups.includes("tere-dbus") {
    return polkit.Result.YES;
  }
});

(Reload is automatic.)

Limiting Tere sessions

A site admin can add or adjust polkit rules to suit their needs. For example, you could

• prevent Tere from starting sessions on the host, only allow containers
• prevent Tere from starting sessions as root (or any system account)
• enforce what shell program Tere starts

Starting from systemd v247¹, polkit rules can use action.lookup(key), and systemd-machined defines keys machine, user, program for what session is being started.

It would be nice to be able to pass more Tere-specific metadata to the polkit rules, but that won't happen without systemd-machined changes.

Resources

https://www.freedesktop.org/wiki/Software/dbus/

https://dbus.freedesktop.org/doc/dbus-specification.html

https://dbus.freedesktop.org/doc/dbus-daemon.1.html

Roadmap

Soon

• rewrite the (unpublished) prototype into publishable form
• reach working code
• privilege separation and sandboxing

Later

• reconnect
• sessions survive restart
• TLS without external proxy
• command-line client
• non-interactive sessions

Maybe

• keep terminal state for seamless reconnects

• session sharing

• authenticate from initial server to further one

  • either we can connect directly to final destination, or we can't ("bastion host")
  • can only WebAuthn if browser can connect to final destination

• unattended authentication

  • limit what commands can be run

• switch from hterm to alacritty_terminal in WASM

  This would especially help with keeping terminal state, as we could have the same terminal emulator on both sides, and avoid confusion.

How does Tere compare to OpenSSH?

This probably applies to the SSH protocol in general, but everyone uses OpenSSH, right?

Tere current limitations

These might be lifted with work, later, but for now, Tere definitely cannot do these things:

• non-interactive sessions
  • file transfer: rsync, sftp/scp
  • port forwarding
• authentication forwarding
• unattended authentication (key files)

Non-interactive use should be doable, but first the command-line client needs to exist, as this is beyond the realm of a browser client. After that rsync and such are just a question of using the -e option. Port forwarding should be able to use that, even if the feature is not built in.

Authentication forwarding needs to be completely rethought to be safe. We might need to require the client to be able to connect directly to all servers for authentication, even if the data connection is from one server to another; need to experiment with proxying WebAuthn to know for sure.

We don't have a clear plan yet for unattended authentication and that is somewhat at odds with our security stance of not liking key files. Maybe we'll add a mechanism that limits what can be done with a secret token.

Mosh

Mosh is an SSH wrapper that uses a custom UDP protocol after a handshake conducted via SSH. It's claim to fame with low latency typing feedback, including a predictive mode where your input is echoed in the local terminal even before it has been transmitted to the server.

Mosh is purely an interactive tool, port forwarding and file transfers would still be done with SSH.

Mosh requires a unique open UDP port for every session. Firewalls typically forbid this, and for good reasons. Tere looks like regular web traffic, and servers can be behind any HTTP reverse proxy that supports WebSockets over TLS.

Similarities

• survive temporary network failures and IP address changes

Tere current limitations

These might be lifted with work, later, but for now, Tere definitely cannot do these things:

• we don't even attempt to do predictive input

Tere architectural limitations

We probably won't (be able to) do anything about these, but external factors might change:

• no UDP low latency tricks

  We're aiming to support browser clients, and thus use WebSocket as transport, not a custom UDP protocol, and this puts some limits on our latency. HTTP/3 may help with this, later. A standalone client could avoid browser restrictions, but that doesn't seem worth the effort.

Decisions

These are things that have already been thought about, to various levels. Some go into more detail, as necessary.

Pretty firm

Rust

Tere is programmed in Rust. It gives a nice sense of safety, but even more importantly it allows low-level control over threads, processes & syscalls that makes Seccomp and Landlock style fine-grained sandboxing feasible.

systemd

Why use systemd? Because it gets us features, and it's going to be installed anyway.

• Tere doesn't even need to run as root!
• systemd will let us create sessions, on host or in container, with a simple call, not worry about dropping privileges and what not
• systemd will sandbox our whole service
• systemd will terminate sessions reliably, with cgroups, including if we crash before persisting a pty fd
• we can store pty fds in systemd and have sessions survive restarts

Not so nice trade-offs include

• need to use in-filesystem UNIX domain sockets to connect privsep components, instead of socketpair and fork
• runtime reliability depends on a whole lot of components (then again, most Linux distros these days depend on them anyway)

What about other platforms and niche Linux distributions? We can work on compatibility later, need to get things working first. Nothing in the big picture depends on systemd, it's just conveniences, shortcuts and extra features.

Need multiple uids or gids

Since we use polkit rules to allow the equivalent of machinectl shell without CAP_ADMIN, and because those rules can only enforce based on uid or gid¹, we want to isolate that capability in even the simplest design, as it's the highest risk.

Difficult choices ahead: start as root and fork+setuid, or expose multiple services to the admin and risk version skew. Leaning toward multiple services and detect skew.

systemd template services instead of fork or fork+exec

We could have had parent processes fork/fork+exec worker processes. That could have made it more sure we're using the same executable image, either not exec'ing or using /proc/self/exe or such (Self-exec can help purge resources like open fds, re-randomize some attack mitigations; or use a "nursery" process that spawns children without leftover state). We chose not to, because either all the child processes would run as the same uid, or we'd need CAP_ADMIN to switch groups.

Instead, systemd DynamicUser=yes combined with Accept=yes start each process as a separate uid, with unrelated sandboxes. This works as long as we don't need to pass state from a parent process (or alternatively, we now have to do that via FD socket passing etc).

Downsides: admin has to see internals, might run into version skew, need to connect over UNIX domain sockets instead of just socketpair & fork.

Soft

async or threaded, sandbox granularity

Do we isolate data in/out of each session into their own processes, or do it all in one?

One per user would sound nice, but it's hard to do because we want to support sharing shell sessions.

Especially if and when we start running terminal emulators on the server lots of small sandboxes sounds like a good idea.

Threaded means we can seccomp & landlock individual threads. It also makes the system more manageable to ops, e.g. allowing meaningful strace use. Leaning that way.

Only the non-templated services (without the @ suffix) should be handling a large number of connections or tasks within one sandbox, so those are the ones that would benefit from async. We'll likely sandbox threadpools running async code, there.

Working & secure is better than optimally fast, and it's probably plenty fast anyway. The cost of the actual shell session should be much higher than our overhead.

Authentication is tied to WebSocket connection

Lose a TCP connection (except in HTTP/3 world), or change IP addresses, and you need to reauthenticate. Simplifies things greatly, avoids yet another session token that can be stolen, but might not work out well in practise.

Undecided

D-Bus

We use dbus via the excellent zbus crate.

Troubleshooting

https://dbus.freedesktop.org/doc/dbus-monitor.1.html

dbus-monitor --system "interface='org.freedesktop.machine1.Manager'"

dbus-send --system --dest=org.freedesktop.machine1 --type=method_call --print-reply /org/freedesktop/machine1 org.freedesktop.machine1.Manager.OpenMachineShell string:.host string:root string:/bin/sh array:string:/bin/sh array:string:'TERM=alacritty'

https://dbus.freedesktop.org/doc/dbus-send.1.html

Resources

https://dbus.freedesktop.org/doc/dbus-api-design.html (not very relevant, we're not making new D-Bus APIs)

https://www.freedesktop.org/software/polkit/docs/latest/polkit.8.html

polkit method called by systemd-machined: https://www.freedesktop.org/software/polkit/docs/latest/eggdbus-interface-org.freedesktop.PolicyKit1.Authority.html#eggdbus-method-org.freedesktop.PolicyKit1.Authority.CheckAuthorization

WebSockets

Parsing in a different process

WebSockets (as used over HTTP/1; HTTP/2 will likely change this?) have an interesting property where no information from the handshake is needed to handle the actual WebSocket protocol.

For example, the "frame masking" XOR key is included in each frame, and not in the initial handshake.

(And even if there was some such data, it'd be easy to pass that state along too.)

Thus, WebSockets can be served by a different process, by "hijacking" the TCP connection (beware TLS, HTTP/2¹) from the web server and using fd passing to hand it to a different process.

WebSocket::from_raw_socket

WebSocket::from_partially_read

PTY to stdin/stdout plumbing

This file contains notes about what's needed to properly relay input/output/out-of-band events between an existing tty and a pty. Concurrently copying data in and out from the PTY is harder than it might seem at first. It's made a lot harder by limitations of stdin/stdout. This document is still, and probably forever will be, woefully incomplete.

Raw mode

Stdin needs to be switched to raw mode. In raw mode, for example control-D is read as byte 0x04 = EOT = control-D, not as EOF. The crate raw_tty works fine for this. raw_tty::TtyWithGuard is not Send and is difficult to deal with for threads or async, so do the simpler thing instead and use

#![allow(unused)]
fn main() {
let mut guard = raw_tty::TtyModeGuard::new(stdin.as_raw_fd())?;
guard.set_raw_mode()?;
}

Rust buffers stdout

Hideous wart and a historical waste of person-years of troubleshooting effort, faithfully copied into a greenfield system (sigh). https://github.com/rust-lang/rust/issues/58326

Your choices are:

1. bypass Rust stdout

#![allow(unused)]
fn main() {
let stdout = std::io::stdout();
let mut stdout_unbuffered: File = unsafe { FromRawFd::from_raw_fd(stdout.as_raw_fd()) };
}

2. avoid std::io::copy and write your own loop that flushes the buffer every time.

Probably choose #2, because that buys us cancel-on-interrupt, see Canceling reads.

Stdin/stdout cannot be made non-blocking

Stdin and stdout are file descriptors sharing the open file between many processes, while flags like non-blocking are per-file (not per-fd) properties. It's an old UNIX wart, and we're stuck with it. What that means is that stdin/stdout must not be made non-blocking, as that will break unrelated programs that just happen to use them.

Canceling reads

So if stdin is in blocking mode, a read(2) on it will block until data is available. When the PTY closes (child exits), we need to clean that up somehow.

We can do it with pthreads and signals. We need to reimplement our own copy loop because std::io::copy retries on EINTR.

Using a std::fs::File for PTY FD is too hard

Rust lifetimes won't let a File be handled by separate readers and writers. You can dup(2) the FD but that might lead to other traps, and is wasteful at scale. Probably need to write our own type implementing Read and Write.

SIGWINCH

Relay window size from tty to pty. Also remember to set initial window size.

Sketches

This directory contains ideas and early write-ups, not quite ready yet for greater visibilty. These should not be taken as truth or committed goals.

Privilege separation

The Tere server consists of multiple co-operating, mostly mutually distrusting, processes. (Threads are cheaper and can self-sandbox too, but that boundary doesn't prevent reading secrets.)

OpenSSH just privseps from root into the destination user account. We, however, might be launching sessions in containers, and not on the host itself, we don't switch uid ourself, and the user might not even exist on the host. We don't even run as root to be able to switch users! We need to do things differently.

Services and processes

The services communicate with each other via UNIX domain sockets. Only tere-server serves TCP sockets, for HTTPS and HTTP. All listening sockets are created via systemd socket activation (see systemd.socket).

All of the services use systemd configuration DynamicUser=yes to run as a unique system account. Extra group memberships (via SupplementaryGroups=) are used for access control between services, for UNIX domain sockets (groups tere-socket-*, named after the service connecting to) and D-Bus (group tere-dbus).

Services named with a @ suffix (after the systemd convention) run as multiple instances. They use systemd Accept=yes to make systemd start a new instance for every incoming connection. Additionally, DynamicUser=yes will isolate each connection from each other(1).

tere-server is the entry point from the network

tere-server serves things like static HTTP assets. It holds an extra group membership tere-socket-auth. It has a long-living connection to tere-auth.

tere-server also handles WebSocket connections. It processes the first few initial messages that authenticate the user, and after authentication copies data between WebSocket messages and tere-user@¹ (without parsing). This is to keep unauthenticated actions lightweight, to avoid easy denial of service attacks.

Authentication happens inside the WebSocket connection and is tied to the WebSocket connection lifetime, to avoid the need for session tokens that could be stolen.

All decision-making regarding authentication is delegated to tere-auth.

tere-auth authenticates users

tere-auth makes WebAuthn authentication decisions. It holds extra group memberships tere-socket-user and tere-socket-policy.

After successful authentication, it connects to the tere-policy@ service, which makes systemd spawn a new process. It also connects to the tere-user@ service, and after a handshake passes the just established policy connection to the new tere-user@ instance.² This indirection removes tere-auth from the data path for an authenticated connection.

tere-user@ speaks with with clients

tere-user@ parses most complex potentially hostile input, it is named so because it is the user representative on the server host, and most likely to come under complete user control. Each authenticated WebSocket connection gets a dedicated tere-user@ worker.

It speaks a length-prefixed message protocol. WebSocket frames from browser clients will be translated to these messages by tere-server, and once a command-line client exists, it'll probably bypass the WebSocket protocol and speak this protocol directly.

It transports shell session requests, shell session input/output streams, and such between the client and the tere-policy@ worker, and via further FD passing to tere-sessions and further to tere-pty@.

tere-policy@ makes authorization decisions

tere-policy@ makes authorization decisions based on its configuration. It holds an extra group membership tere-socket-sessions. Each authenticated WebSocket connection gets a dedicated tere-user@ worker.

It reads a message from the connection stating the username, and binds the connection to that username in its state, before parsing any user input.

It processes authenticated end user requests such as "create a new shell sessions", running them through a policy engine, and when allowed relays them to the tere-sessions service, relaying data and passed FDs back to the client.

tere-sessions starts and manages shell sessions

tere-sessions uses D-Bus to talk to systemd-machined to create shell sessions. It holds an extra group memberships tere-dbus (that allows the above) and tere-socket-sessions.

For every created shell session, systemd-machined returns a PTY FD. tere-sessions connects to tere-pty@, which makes systemd spawn a new process, and hands the PTY FD to that new process. It remembers that connection, identified by a random unique session ID. tere-sessions stores both of these FDs in systemd for restarts.³ Later requests to open an existing connection are served by messaging the right tere-pty@ process.

For connecting to already existing shell sessions, tere-sessions proxies the request to the tere-pty@ instance for that session.

tere-sessions exists as separate from tere-pty@ for two reasons: to prevent D-Bus access after a sandbox escape, and to have a place that can store and re-serve PTY FDs after a software restart or crash.

tere-pty@ transports data to and from a PTY

tere-pty@ receives the PTY FD over the incoming connection, and later receives client requests. Each shell session a dedicated tere-pty@ worker. Note that shell sessions and active users do not necessarily map 1:1.

tere-pty@ serves clients connecting to its sessions by creating a socketpair, used for PTY input/output and commands, and passing this FD to the client. This indirection removes tere-sessions from the bulk data path. Commands include updating the terminal size on window resize.

It transports data between the PTY and (once implemented) multiple clients, broadcasting the same PTY output stream to all clients.

Resources

• https://landlock.io/ and https://crates.io/crates/landlock
• https://github.com/openssh/openssh-portable/blob/master/README.privsep
• http://www.citi.umich.edu/u/provos/ssh/privsep.html (archived)

Protocols used in IPC

TODO Extract diagrams from source code. Getting payload descriptions right could be tricky.

Common handshake

We want to avoid attackers being able to confuse a process about its recipient, and having one protocol message be parsed according to a wholly different protocol.

To make this very explicit, all protocol chats begin with a handshake that identifies the protocol being spoken and the executable file hash.

Since this is an IPC protocol where we don't support mismatching versions, to notice any version skew between services, the handshake includes the executable file hash.

sequenceDiagram
    client ->> server: intent_client, build_id_client
    server ->> client: intent_server, build_id_server

Intents use the context string style of BLAKE3 derive_key, for example tere 2021-06-03T10:19:27 user to policy client, and correspondingly tere 2021-06-03T10:19:27 user to policy server terminated by a newline.

(TODO if we buy a nice domain, put that in as application instead of just tere?)

Build IDs are BLAKE3 hashes of the executable (via /proc to ensure it matches what's running), keyed by the corresponding intent. The peer verifies the hash. We use keyed hashes so you can't just reply by echoing what the client sent. We don't use any kind of a challenge mechanism, to allow for precomputing the hash.

tere-user@ to tere-policy@

TODO this is an incorrect draft and even at that probably outdated, update from privsep.md

tere-user@ worker in state Active holds an FD-as-capability that's bound to the currently authenticated user. On the other end of that FD is a tere-policy@ worker.

sequenceDiagram
    user ->> policy: WebAuthnRegisterInitiate{user_name, display_name}
    policy ->> user: WebAuthnRegisterOptions{publicKeyCredentialCreationOptions}
    user ->> policy: WebAuthnRegisterCredential{credential}
    alt ok
        policy ->> user: Ok
    else failure
        policy ->> user: Error
    end

sequenceDiagram
    participant user
    participant policy
    policy ->> user: AuthenticationRequired{publicKeyCredentialRequestOptions}
    user ->> policy: WebAuthnRegisterCredential{credential}
    alt ok
        policy ->> user: Ok
    else failure
        policy ->> user: Error
    end

Big picture

sequenceDiagram
    alt allow
        user ->>+ policy: CreateShellSessionRequest{machine,user,args,env}
        note over policy: policy decision: allow
        policy ->>+ sessions: CreateShellSessionRequest{machine,user,args,env}
        sessions ->>- policy: Created{pty_fds}
        # TODO should create auto-open (subscribe) to session? probably yes.
        policy ->> user: Created
    else deny
        user ->>+ policy: CreateShellSession
        note over policy: policy decision: deny
        policy ->>- user: Denied
    else failure
        user ->>+ policy: CreateShellSession
        note over policy: policy decision: allow
        policy ->>+ sessions: CreateShellSessionRequest{machine,user,args,env}
        sessions ->>- policy: Failed
        policy ->>- user: Failed
    end

Inter-Process Communication (IPC)

If we do privilege separation, we're going to have multiple processes that need to interact. If we're parsing websocket messages in one process, but keeping the PTY fds safe in another, that means we're doing some form of IPC for every chunk of input & output the user experiences; likely more than once. We need an IPC mechanism that won't become the bottleneck. We also want to have one that can pass file descriptors, which eliminates most common RPC mechanisms, that are aimed at networking. Let's explore our options.

Open question: Broadcasting shell session content

Ringbuffers in shared memory

If the mechanism natively supports multiple consumers seeing the same data, how do we kick out slow consumers?

Just send data on UNIX domain socket

Now we need a broadcast pub/sub mechanism, either on the sender side or receiver.

Design sketch: Circuits

circuits is a (TODO to-be-written) module/crate providing a simple mechanism for transporting multiple logical circuits over one (framed, message-based) stream. If a transport is not natively framed, length-prefixed messages are an easy solution for framing messages.

Circuits are comparable to remote procedure calls where each request contains a request ID and responses to refer to those, except each circuit can operate its own multi-stage protocol and can stream data.

Since tungstenite, the websocket library used here, assumes that messages fit in memory, and on the other end bincode/other serde mechanisms often make the same assumption, the circuits API will likely assume that too; but this is not an inherent limitation of the design.

Each transport carries max_circuits circuits, defined by the protocol logic being generic over a suitable integer type. All circuits are "virtually open" at all times, there are no open/close messages. Circuits are "idle" if they are available for use, or "busy" if they are currently in use.

For a communicating pair of endpoints, previously agreed upon circuit IDs may be reserved and used for special purposes. All other circuits are interchangeable when idle.

Circuits must support messages bearing ancillary data, such as UNIX domain socket file descriptor passing, when the transport is capable of such.

Backpressure may or may not be implemented, to be discovered later.

Attacks

TODO This should be expanded to a proper attack tree.

To obtain a root shell, an attacker would have to achieve one of these goals (list is non-exhaustive):

• (somehow) steal TLS secrets and impersonate tere-server
• (somehow) gain a false TLS certificate for the domain name
• make tere-server run arbitrary code, and wait for the next admin connection
• make tere-server run arbitrary code, and attach/hijack an existing admin connection (in-process TLS or HTTP/2 only, otherwise the connection is no longer visible to tere-server)
• convince tere-policy@ of incorrect authentication (as spoken to through tere-user@'s limited protocol)
• convince tere-policy@ of incorrect authorization (as spoken to through tere-user@ and tere-policy@, bound to a successfully authenticated username)

Our insistence on WebAuthn should mean phishing is not a viable attack.

And so on.

Bastion hosts considered harmful

The old school way of "punching through the firewall" from the outside was to set up a "bastion host" in the DMZ, and allow internet->DMZ and DMZ->internal connections.

In such a system, either the bastion host contained SSH key files, or everyone used SSH agent forwarding. Both of those are security disasters, including confused deputies, attackers stealing keyfiles, and SSH agent either a) not prompting at all or b) prompting with no detail whatsoever, letting attackers race a connection to a wholly different server.

Either just expose your systems, set up a Wireguard VPN, or proxy traffic.

If there are enterprise uses for a bastion host (likely: better auditing), let's handle those use cases on their own.

Unattended authentication

We really don't like key files laying around. But the need for that use case is probably too strong to resist.

Privacy

Do NOT offer (fingerprints of) all known public keys to server. That leads to "I know what your github accounts" attacks.

Account store

Our account store needs to store enough information for WebAuth. At the very minimum that means Username -> {CredentialId -> Credential}. Need to read the spec more.

We definitely want minimal dependencies and bureaucracy and maximal troubleshooting ability. We might just go with a JSON file per account, for the data that only changes when users add/remove authenticators.

Need to worry about enterprise use cases, federated authentication, but that's after the basics work.

Write the details here...

Tasks

Not all of these should happen, but they should be thought of.

Small things

• don't do TLS at all, for now, require reverse proxy
• submit mdbook-graphviz bugfixes
• bracketed paste means our input stream from user to pty should have packet types

Vague things

• "session" is a confusing name, as that could mean a logged-in session in the web UI, or the shell session; they don't even map 1:1

• there is a lot of detail in ioctl_tty(2) that should be dealt with later

• potential extra obfuscation for DoS avoidance, if wanted:

  1. Hide the service at non-public URL paths (compare to SSH port change, port knocking, except much more powerful)
  2. CPU-burn challenge, e.g. "give me some 32 bytes that sha256 hash to prefix 0x3571" (can be overcome by botnets)
  3. Dynamic IP reputation management (can be overcome by botnets)
  4. Use a VPN for "good" users (then why are we so keen on encryption and authentication)
  5. Delegate DoS protection to proxy such as Cloudflare (may expose HTTPS contents, depending on implementation)

Architectural decisions to make

WebAuth challenge creation

WebAuthn authentication is based on a server-created challenge. To be conservative, we don't want server creating that challenge, even if the client response was checked in a more trusted component.

auth would need to maintain a state table with expiry etc worries (state spill), or server and auth would have to use connection per authentication attempt to manage the state life cycle more naturally but more expensively. Or, the challenges could be made stateless (as far as auth is concerned) by creating the challenge with a keyed hash from something server owns, or signing them. (Can rotate random keys by also trying the previous key.)

server can also check the completion of the authentication challenge, and forbid clients early, while only being trusted to say no. Doesn't seem worth it with a long-living connection to auth.

WebAuthn register MITM or spoofing

TODO Worry about server faking/MITMing a WebAuthn registration. Maybe that's best solved by demanding hardware attestation?

Resources

Miscellaneous resources that didn't fit elsewhere.

https://www.freedesktop.org/software/systemd/man/systemd-machined.service.html

https://www.freedesktop.org/software/systemd/man/sysusers.d.html