Skip to content

Commit

Permalink
Merge pull request #676 from o-sdn-o/gui-bridge
Browse files Browse the repository at this point in the history
Update docs
  • Loading branch information
o-sdn-o authored Nov 8, 2024
2 parents fd2c71b + 0063f7e commit 7594c9e
Show file tree
Hide file tree
Showing 3 changed files with 63 additions and 69 deletions.
45 changes: 17 additions & 28 deletions doc/apps.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,41 +17,30 @@

### Features

- UTF-8 Everywhere
- TrueColor aware
- Horizontal scrolling
- Infinite* scrollback (40k lines by default, * `< max_int32`)
- Scrollback buffer searching and matching
- Line-based/rect-block text selection:
- Ctrl: Extend selection.
- Alt/Option: Change selection mode (line/block).
- Double left click: Select a word.
- Triple left click: Select paragraph.
- Quadruple left click: Select the entire scrollback buffer or semantic block (when using OSC 133).
- Quintuple left click: Select the entire scrollback buffer.
- Widely used clipboard formats support:
- Horizontal scrolling.
- Configurable scrollback buffer size (40k lines by default, limited by `max_int32` and system RAM).
- Scrollback buffer lookup.
- Linear/box text selection:
- Extending selection with `Ctrl` key pressed.
- Changing selection mode (linear/box) with `Alt` key pressed.
- Selecting a word/paragraph/entire scrollback buffer or a semantic block (when using OSC 133) by Double/Triple/Quadruple/Quintuple left clicking.
- Multiple clipboard formats support:
- Plain text
- RTF
- HTML
- ANSI/VT
- Protected (Windows only: `ExcludeClipboardContentFromMonitorProcessing`, `CanIncludeInClipboardHistory`, `CanUploadToCloudClipboard`)
- [VT-100 terminal emulation](https://invisible-island.net/xterm/ctlseqs/ctlseqs.html) compatible (pass vttest 1 and 2 sections)
- Built-in Windows Console API server:
- Legacy Win32 Console API support
- No Windows Console Host (conhost.exe) dependency
- Fullduplex pass-through VT input/output
- OEM/National, UTF-8 and UTF-16 encoding, even in cmd.exe
- Enforced ENABLE_WINDOW_INPUT mode
- Legacy Win32 Console API support.
- No Windows Console Host (conhost.exe) dependency.
- Fullduplex pass-through VT input/output.
- OEM/National, UTF-8 and UTF-16 encoding support.
- Enforced ENABLE_WINDOW_INPUT mode.
Note: In fact it is a viewport resize event reporting. Viewport dimensions is always equal to the win32 console buffer dimensions.
- Enforced ENABLE_PROCESSED_OUTPUT and ENABLE_VIRTUAL_TERMINAL_PROCESSING modes
- Disabled ENABLE_QUICK_EDIT_MODE mode
- Per process instance (not per process name) cmd.exe input history, aka "line input"/"cooked read"
- Disabled DOSKEY functionality (cmd.exe's F7 input history popups too)
Note: Sharing the input history as well as a bunch of command aliases among processes (which could have different elevation levels) is a huge security threat. So DOSKEY functionality is absolutely incompatible with any sort of sudo-like commands/applications.
- Outside terminal viewport mouse tracking (See #62 for details)
- Stdin/stdout parser log on demand
- Configurable at startup via `settings.xml`
- Configurable in runtime using VT-sequences
- Enforced ENABLE_PROCESSED_OUTPUT and ENABLE_VIRTUAL_TERMINAL_PROCESSING modes.
- Disabled ENABLE_QUICK_EDIT_MODE mode.
- Per process (not per process name) Windows Command Prompt (cmd.exe) input history, aka "line input"/"cooked read".
- Stdin/stdout logging.

### Custom SGR attributes

Expand Down
85 changes: 45 additions & 40 deletions doc/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@ graph TB
subgraph IE10[Generic Text Console 1]
subgraph IE1[Input]
direction LR
C1[keybd, mouse, focus\nwinsize, clipboard,\nos signals]
C1[keybd, mouse, focus<br>winsize, clipboard,<br>os signals]
end
subgraph OU1[Output]
TC1[scrollback\nbuffer]
TC1[scrollback<br>buffer]
end
subgraph CS1[Desktop Client 1]
VTM1[vtm\nprocess 1]
VTM1[vtm<br>process 1]
end
C1 --> CS1
TC1 --- CS1
Expand All @@ -52,13 +52,13 @@ graph TB
subgraph IE20[Generic Text Console 2]
subgraph IE2[Input]
direction LR
C2[keybd, mouse, focus\nwinsize, clipboard,\nos signals]
C2[keybd, mouse, focus<br>winsize, clipboard,<br>os signals]
end
subgraph OU2[Output]
TC2[scrollback\nbuffer]
TC2[scrollback<br>buffer]
end
subgraph CS2[Desktop Client 2]
VTM2[vtm\nprocess 2]
VTM2[vtm<br>process 2]
end
C2 --> CS2
TC2 --- CS2
Expand All @@ -67,43 +67,43 @@ graph TB
subgraph IE30[Generic Text Console 3]
subgraph IE3[Input]
direction LR
C3[keybd, mouse, focus\nwinsize, clipboard,\nos signals]
C3[keybd, mouse, focus<br>winsize, clipboard,<br>os signals]
end
subgraph OU3[Output]
TC3[scrollback\nbuffer]
TC3[scrollback<br>buffer]
end
subgraph CS3[DirectVT Gateway]
VTM3[vtm\nprocess 3]
VTM3[vtm<br>process 3]
end
C3 --> CS3
TC3 --- CS3
end
subgraph APP0x[Standalone DirectVT-aware Application]
APP01[DirectVT App4\napp: process 9]
APP01[DirectVT App4<br>app: process 9]
end
CS3 <-->|DirectVT I/O\nsend: Events\nrecv: Render| APP01
CS3 <-->|DirectVT I/O<br>send: Events<br>recv: Render| APP01
subgraph NP[Connection Point]
TS["system-wide\nnamed pipe"]
TS["system-wide<br>named pipe"]
end
subgraph SS[Desktop Server]
VTMs[vtm\nprocess 0]
VTMs[vtm<br>process 0]
subgraph SE[Desktop Session]
subgraph APPx[running application windows]
subgraph APP1[DirectVT Gateway]
eAPP1[DirectVT App1\napp: process 4]
eAPP1[DirectVT App1<br>app: process 4]
end
subgraph APP2[DirectVT Gateway]
eAPP2["App2 (Teletype+App2)\nvtm: process 5\napp: process 6"]
eAPP2["App2 (Teletype+App2)<br>vtm: process 5<br>app: process 6"]
end
subgraph APP3[DirectVT Gateway]
eAPP3["App3 (Terminal+App3)\nvtm: process 7\napp: process 8"]
eAPP3["App3 (Terminal+App3)<br>vtm: process 7<br>app: process 8"]
end
end
subgraph APPu[connected users]
USR1["Desktop Client 1\nViewport\nTaskbar"]
USR2["Desktop Client 2\nViewport\nTaskbar"]
USR1["Desktop Client 1<br>Viewport<br>Taskbar"]
USR2["Desktop Client 2<br>Viewport<br>Taskbar"]
end
USR1 --->|keyboard focus| APP1
USR2 --->|keyboard focus| APP2
Expand All @@ -112,13 +112,12 @@ graph TB
VTMs === SE
end
CS1 <-->|DirectVT I/O\nsend: Events\nrecv: Render| TS
CS2 <-->|DirectVT I/O\nsend: Events\nrecv: Render| TS
CS1 <-->|DirectVT I/O<br>send: Events<br>recv: Render| TS
CS2 <-->|DirectVT I/O<br>send: Events<br>recv: Render| TS
TS === VTMs
```

- vtm is a text-based desktop environment that comes with a single executable.
- Due to the fact that a typical desktop environment is a dynamic long-living construct of interacting processes, vtm has a number of mutually exclusive runtime modes to parallelize this functionality by launching multiple interacting vtm processes.
- Vtm is a text-based desktop environment that comes with a single executable.
- Due to the fact that a typical desktop environment is a dynamic, long-lived construct of interacting processes, vtm has a number of runtime modes for running multiple vtm processes in parallel to form the environment.
- Each desktop session is a vtm process running in `Desktop Server` mode.
- Desktop environment users connect to an existing desktop session through an additional vtm process running in `Desktop Client` mode.
Expand All @@ -131,19 +130,19 @@ graph TB
- Sessions with different ids can coexist independently.
- To maximize rendering efficiency and minimize cross-platform issues, along with character-oriented xterm-compatible TUI mode called `ANSI/VT`, vtm supports an additional message-based binary TUI mode called `DirectVT`.
- All running applications are connected to the desktop environment using `DirectVT Gateway` windows as DirectVT endpoints.
- DirectVT-aware applications are connected directly to the environment and can seamlessly send and receive the entire set of desktop events, as well as render themselves in binary form, avoiding expensive ANSI/VT parsing.
- To run non-DirectVT applications, an additional vtm host process is launched in `Desktop Applet` mode with the `Teletype Console` or `Terminal Console` applet as a DirectVT bridge to the desktop environment.
- A DirectVT-aware application directly connected to the environment can seamlessly send and receive the entire set of desktop events, as well as render themselves in binary form, avoiding expensive ANSI/VT parsing.
- To run a non-DirectVT application, an additional vtm host process is launched in `Desktop Applet` mode with the `Teletype Console` or `Terminal Console` applet as a DirectVT bridge to the desktop environment.
- The desktop environment server can receive and execute script commands relayed from other vtm processes running on behalf of the session creator.
- In the case of a vtm process with redirected standard input, all standard input is directly relayed to the desktop environment server as a script command flow.

### Runtime modes

Runtime mode | TUI mode | Environment role
----------------|--------------------------|------------------
Desktop Applet | auto detected | Desktop applet of an arbitrary type running in its own process that accepts user input and renders itself. Used to run heavy desktop objects in parallel processes to optimize desktop resource consumption.
Desktop Client | auto detected | Desktop client in its own process that forwards user input to the desktop and renders the corresponding desktop region with a taskbar overlay.
Desktop Applet | auto detected | A desktop applet of an arbitrary type running in its own process that accepts user input and renders itself. Used to place a heavy (complex) desktop object in a separate process in order to optimize desktop resource consumption.
Desktop Client | auto detected | A desktop client running in its own process that forwards user input to the desktop and renders the corresponding desktop region with a taskbar overlay.
Desktop Server | n/a<br>command line only | The desktop environment core that manages connected users, runs desktop applications, routes user input, and forwards renders to desktop clients.
Desktop Monitor | n/a<br>command line only | Desktop monitor that outputs the desktop session log and relays script commands to the desktop server via piped redirection.
Desktop Monitor | n/a<br>command line only | A desktop log monitor which outputs desktop session logs and relays script commands to the desktop server via piped redirection.

The runtime mode can be selected using command-line options. By default, `Desktop Client` mode will be used, with `Desktop Server` implicitly running in parallel if it is not running.

Expand All @@ -160,11 +159,11 @@ DirectVT Gateway with TTY | `dtty` | CUI applications that redirect DirectVT

## TUI modes

An instance of the vtm process in `Desktop Client` or `Desktop Applet` mode can operate in one of two TUI modes, either in `ANSI/VT` mode, or in `DirectVT`(`dtvt`) mode.
A vtm process instance running in `Desktop Client` or `Desktop Applet` mode can operate in one of two TUI modes: either `ANSI/VT` mode or `DirectVT`(`dtvt`) mode.

### DirectVT mode

In DirectVT TUI mode, the vtm process, communicating with the desktop server, multiplexes the following data channels:
In DirectVT TUI mode, vtm process multiplexes the following data channels:
- Keyboard event channel
- Mouse event channel
- Focus event channel
Expand All @@ -179,7 +178,7 @@ The DirectVT stream can be wrapped in any transport layer protocol suitable for

#### Input

In ANSI/VT TUI mode, the vtm process parses input from multiple standard sources, and forwards it to the desktop server using the DirectVT transport. The set of input sources varies by platform.
In ANSI/VT TUI mode, vtm process parses input from multiple standard sources, and forwards it to the desktop server using the DirectVT transport. The set of input sources varies by platform.

##### Unix input sources

Expand Down Expand Up @@ -225,9 +224,9 @@ In ANSI/VT TUI mode, the vtm process parses input from multiple standard sources

#### Output

Console UI applications running as external processes are instantly rendered into their host `DirectVT Gateways ` windows running directly in the desktop server address space.
Console UI applications running as external processes are instantly rendered into their host `DirectVT Gateways` windows running directly in the desktop server address space.

The desktop server receives and caches the window rasters and sends the corresponding projection rendering to desktop clients at each internal timer tick.
The desktop server receives and caches window bitmaps and sends incremental changes to desktop clients every tick of an internal timer.

The binary render stream received from the desktop server to output is converted by the desktop client to the format suitable for the console being used to output. The console type is detected at the desktop client startup and can be one of the following:
- XTerm-compatible terminal with truecolor support
Expand All @@ -241,11 +240,11 @@ The desktop client outputs the received render to the hosting console only when

Term | Description
--------------------|---------------
`colored character` | A character depicted with rendition attributes such as background and foreground color.
`colored character` | A character depicted with rendition attributes such as background and foreground colors.
`text console` | A cellular rectangular surface designed to display colored monospaced characters in cells.
`text cell` | A text console cell containing a colored monospaced character or its fragment.
`bitmap` | A rectangular block of text cells.
`canvas` | A rectangular buffer for text cells output.
`canvas` | A rectangular buffer for bitmap output.

Internally the desktop is represented by the parent-child object tree with a single root object that maintains a desktop-wide configuration, a list of connected users, and a list of running windows. The root object broadcasts a fixed number of ticks every second to update the tree state and to do something else in sync.

Expand All @@ -264,8 +263,8 @@ Desktop object | Description
Teletype Console<br>`teletype` | A solid rectangular truecolor text canvas depicting a freely scrollable buffer of the text runs generated by an xterm-compatible parser from the standard output of an attached CUI application. It can be a very heavy object due to maintaining a scrollback buffer of arbitrary length. Not used directly in the desktop process's address space.
Terminal Console<br>`terminal` | A derivative of `Teletype Console` with additional UI controls.
DirectVT Gateway<br>`dtvt` | A lightweight truecolor text canvas depicting content received from an external dtvt-aware process.
Teletype Console dtvt‑bridge<br>`vtty` | A `DirectVT Gateway` hosting an external standalone `Teletype Console` applet. It is designed to run a heavy `Teletype Console` object in the external process's address space to optimize desktop resource consumption.
Terminal Console dtvt‑bridge<br>`term` | A `DirectVT Gateway` hosting an external standalone `Terminal Console` applet. It is designed to run a heavy `Terminal Console` object in the external process's address space to optimize desktop resource consumption.
Teletype Console dtvt‑bridge<br>`vtty` | A `DirectVT Gateway` hosting an external standalone `Teletype Console` applet. It is designed to run heavy `Teletype Console` objects as external processes to optimize desktop resource consumption.
Terminal Console dtvt‑bridge<br>`term` | A `DirectVT Gateway` hosting an external standalone `Terminal Console` applet. It is designed to run heavy `Terminal Console` objects as external processes to optimize desktop resource consumption.
DirectVT Gateway with TTY<br>`dtty` | A derivative of `DirectVT Gateway` stacked with additional limited `Teletype Console` as a controlling terminal. It is used for CUI applications that redirect DirectVT stream to standard output and require user input via platform's TTY. Depending on activity the corresponding console became active for the user.
Tiling Window Manager<br>`tile` | A window container with an organization of the hosting window area into mutually non-overlapping panes for nested windows.
Desktop Region Marker<br>`site` | A transparent resizable frame for marking the specific desktop region for quick navigation across the borderless workspace.
Expand Down Expand Up @@ -418,12 +417,10 @@ The following examples assume that vtm is installed on both the local and remote

## Standard I/O stream monitoring

vtm allows developers to visualize standard input/output streams of the running CUI applications. Launched in the `Desktop Monitor` mode, vtm will log the event stream of each terminal window with the `Logs` switch enabled.
Vtm allows developers to visualize standard input/output streams of the running CUI applications. Launched in the `Desktop Monitor` mode, vtm will log the event stream of each terminal window with the `Logs` switch enabled.

Important: Avoid enabling the `Logs` switch in the terminal window hosting the `Desktop Monitor` process running, this may lead to recursive event logging of event logging with unpredictable results.

Important: Be careful with enabling the `Logs` switch when working with sensitive information, since all IO events, including keypresses, are logged in this mode.

## Desktop taskbar customization

The taskbar menu can be configured using a settings file `~/.config/vtm/settings.xml` (`%USERPROFILE%\.config\vtm\settings.xml` on Windows):
Expand Down Expand Up @@ -457,7 +454,7 @@ The taskbar menu can be configured using a settings file `~/.config/vtm/settings
</config>
```

The taskbar menu of the running desktop can be configured using shell piped redirection by sending script commands to the running vtm desktop:
Additionally, the taskbar menu of the running desktop can be configured using shell piped redirection by sending script commands to the running vtm desktop:
```
# Delete existing menu items
echo "vtm.del()" | vtm
Expand All @@ -478,3 +475,11 @@ echo "vtm.selected(Term)" | vtm
# Run window with terminals
echo "vtm.run(id=Tile)" | vtm
```

# Desktop Live Panel

- [Desktop Live Panel](panel.md)

# Desktop objects and built-in applications

- [Desktop objects and built-in applications](apps.md)
2 changes: 1 addition & 1 deletion readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ vtm is a text-based desktop environment.
# Supported platforms

- Windows
- Windows 8.1 or later
- Windows 8.1 and later
- Unix
- Linux
- macOS
Expand Down

0 comments on commit 7594c9e

Please sign in to comment.