Windows Console & Processes

A Windows executable carries a subsystem value in its PE header, and that value, not the code, decides whether the loader treats the program as a console program. The linker option /SUBSYSTEM sets it, and the entry point implied by it differs:

If /SUBSYSTEM is not given, the MSVC linker infers it from which entry point the program defines. Other toolchains use their own switches (for example -mconsole and -mwindows for MinGW, #![windows_subsystem = "windows"] for Rust, <OutputType>Exe</OutputType> against WinExe in a .NET project). A CLI is a CONSOLE program. A program that needs no window at all but must not flash a console either (a background helper, a Windows service host) is the usual reason to choose WINDOWS.

A GUI-subsystem program has no standard handles to write to, but it can still print into the console it was started from, which is how one executable can serve both as a desktop app and as a command-line tool. The three calls involved are AllocConsole (create a new console and attach to it), AttachConsole(ATTACH_PARENT_PROCESS) (attach to the parent’s console) and FreeConsole (detach). The C runtime does not notice the change by itself, so the standard streams have to be reopened on the new console’s devices.

Source: AttachConsole and /SUBSYSTEM (Specify Subsystem).

The trade-offs are real. The shell has already returned to its prompt when the GUI-subsystem program starts printing, so output appears after the next prompt, and redirection (shelf > out.txt) does not work because the program’s standard handles were never the redirected ones. The pattern most large tools use instead is to ship two executables (a CONSOLE build for the command line and a WINDOWS build for the desktop), which is what several editors and IDEs do with a small launcher next to the main binary.

Windows has two ways to control a console. The classic Console API is a set of functions that act on the console buffer directly: move the cursor, set attributes, fill a rectangle, scroll a region. Virtual-terminal (VT) sequences are the ECMA-48 and xterm style escape sequences used on every Unix terminal; a program writes them to the output handle and the console interprets them. Microsoft’s documentation recommends VT sequences for all new development and keeps only a small subset of the classic API as still necessary. The pages "Classic Console API versus virtual terminal sequences" and "Console ecosystem roadmap" in the Microsoft documentation describe that direction.

VT handling is a mode of the console, switched on per handle with SetConsoleMode. These are the flags a CLI uses:

Output VT processing is not on by default for a process, so a CLI that wants colours or cursor control has to switch it on, and has to cope with the call failing: SetConsoleMode fails when the handle is not a console (output redirected to a file or a pipe), which is also the Windows answer to "is stdout a terminal?" (the equivalent of isatty). Older consoles that predate VT support also fail the flag, so a fallback to plain text is the right behaviour. The colour policy itself (NO_COLOR, --color) is the same as on every platform and is covered in TTY, Colour & Interactivity.

The classic functions that Microsoft keeps recommending are SetConsoleMode, SetConsoleCP and SetConsoleOutputCP, SetConsoleCtrlHandler, and ReadConsoleInput for window resize events, together with GetConsoleScreenBufferInfo to read the window size. Everything else in the classic API is better done with VT sequences.

The following program is the Windows entry of the shelf list command. It prepares the console, prints in colour only when the output really is a console, and writes Unicode correctly in both situations (console and redirected).

Source: SetConsoleMode, Console Virtual Terminal Sequences and Classic Console API versus Virtual Terminal Sequences.

In .NET, Console.IsOutputRedirected is the isatty answer. The frameworks that wrap these checks for each language are compared in TTY, Colour & Interactivity.

Windows text APIs come in two forms. The W functions take UTF-16 and are the native form; the A functions take bytes in the process’s ANSI code page, a legacy 8-bit encoding chosen by the system locale (for example 1252 for Western European Windows). A console additionally has its own input and output code pages (what chcp shows), which are what the console uses to interpret the bytes a program writes with WriteFile or the CRT narrow functions. By default, that is an OEM code page such as 437 or 850, not UTF-8, which is why a UTF-8 string written with printf shows as garbage on a default console.

There are three independent places where UTF-8 can be switched on, and a program that must work everywhere needs to know what each one does.

The manifest element is declared in the application manifest, embedded in the executable:

Source: Use UTF-8 code pages in Windows apps and Application manifests.

The robust approaches, in order of preference, are:

The C runtime offers _setmode(_fileno(stdout), _O_U16TEXT) to make wprintf write UTF-16 to the console; mixing narrow and wide output on the same stream after that is undefined, so choose one per stream.

Source: Console.OutputEncoding.

On POSIX, execve takes an array of arguments, so the shell does the splitting and the child receives exactly the array the shell built. Windows has no such array. CreateProcess takes a single command-line string, the kernel stores it in the process, GetCommandLineW returns it, and the program itself splits it into argv. The C runtime does this before main, and CommandLineToArgvW does the same for programs that call it directly. Two consequences follow that surprise Unix developers:

Windows has no signals. A console delivers control events to every process attached to it, by running a handler function on a new thread in each process. The events are:

A program registers handlers with SetConsoleCtrlHandler(handler, TRUE). Handlers are called in reverse order of registration until one returns TRUE; returning FALSE passes the event on, and the default handler (which calls ExitProcess) runs last. SetConsoleCtrlHandler(NULL, TRUE) makes the process ignore Ctrl+C, and that setting is inherited by child processes. A handler runs on its own thread, so the usual rules for threads apply, but unlike a POSIX signal handler it may call ordinary functions: set an atomic flag or an event and return. For the close, logoff and shutdown events the system terminates the process once the handler returns or after a short time-out, so cleanup must be quick.

GenerateConsoleCtrlEvent(event, processGroupId) sends CTRL_C_EVENT or CTRL_BREAK_EVENT to a process group sharing the caller’s console. The two have different reach: CTRL_C_EVENT can only be sent to all processes attached to the console (group id 0), while CTRL_BREAK_EVENT can target the process group of a child created with CREATE_NEW_PROCESS_GROUP. That creation flag also makes the new group ignore Ctrl+C, so the only way to ask such a child to stop politely is Ctrl+Break. This is the Windows analogue of kill(pid, SIGINT) and it is much narrower, which is why many tools fall back to TerminateProcess (like SIGKILL, it does not let the target clean up).

Source: SetConsoleCtrlHandler and GenerateConsoleCtrlEvent.

How each runtime maps these events onto its own abstraction differs, and none of the Microsoft pages lists the mappings, so the runtime documentation is the reference:

Sources: Console.CancelKeyPress, Python signal, Node.js signal events and Go os/signal.

The portable pattern, which keeps the shell-level behaviour identical on all three systems, is: handle the interrupt by setting a flag, finish or roll back the current unit of work, exit with a non-zero status, and leave TerminateProcess/SIGKILL for the supervisor. For the longer story of supervised processes, see Long-Running Processes & Services.

CreateProcessW is the one call that creates a process. Where POSIX splits the job into fork (clone) and exec (replace the image), Windows creates the new process from an executable file directly; the closest POSIX counterpart is posix_spawn. Process creation is considerably more expensive than fork on Linux, which matters for CLIs that spawn many short-lived children, but there is no copy-on-write clone to exploit.

The arguments that matter for a CLI:

Not every executable is a program CreateProcess can run directly. It runs .exe files; for a .bat or .cmd file it starts cmd.exe behind the scenes (with the quoting hazards above), and a .ps1 script has to be launched through powershell.exe or pwsh.exe. If no extension is given, CreateProcess appends .exe only; it does not use PATHEXT, which is a convention of cmd.exe and PowerShell.

A program that wants to run another console program inside its own window (a terminal emulator, an SSH server, a test harness, an editor’s integrated terminal) cannot simply redirect the child’s handles: the child would see pipes, not a console, and would turn off colours and cursor control. On Unix the answer is a pseudo-terminal pair (Terminals, TTYs & Pseudo-Terminals). Windows 10 version 1809 added the equivalent, the pseudoconsole (ConPTY). The host creates two pipes, creates a pseudoconsole bound to them, and starts the child attached to that pseudoconsole. The child believes it has a real console; the host reads a stream of VT sequences from one pipe and writes the user’s keystrokes, also as VT sequences, to the other.

The API is three functions and a process attribute:

Source: Creating a Pseudoconsole session and CreatePseudoConsole.

A few facts about ConPTY that matter when testing or embedding a TUI:

Testing the shelf tui screen through ConPTY on Windows CI is covered in Testing Terminal Apps & TUIs; libraries such as portable-pty and expectrl wrap this API behind the same interface they use for POSIX pseudo-terminals.

Two programs can display a console on Windows 10 and 11. conhost.exe is the traditional console host that has existed since Windows NT: it draws the window, owns the console buffer and implements the classic API. Windows Terminal (wt.exe) is a separate, open-source terminal emulator that hosts any command-line program, including cmd.exe, PowerShell and WSL distributions, in tabs and panes. It is built on ConPTY: each tab is a pseudoconsole, and Windows Terminal reads its VT stream and renders it. On Windows 11 22H2 and later it is the default terminal application, and the Windows setting "Default terminal application" lets a user choose which host opens when a console program is started from Explorer or the Run box.

What Windows Terminal adds, in terms a CLI author can rely on:

No supported, documented way exists to detect "am I in Windows Terminal?" from a program. The environment variable WT_SESSION is set by Windows Terminal and is widely used for that purpose, but it is a convention rather than a contract, and the right approach is to detect capabilities (the GetConsoleMode check and TERM/COLORTERM-style hints) instead of identity.

Source: Windows Terminal documentation.

Three smaller platform facts regularly bite portable CLIs:

The table maps the Windows mechanism a CLI uses to its closest POSIX counterpart. "Closest" is deliberate: many pairs differ in important ways, stated in the last column.

Microsoft documents each API above, but there is no single guide to writing a cross-platform command-line tool for Windows. In particular, the documentation does not list how each language runtime maps control events onto its own interrupt model (the runtime documentation linked above is the only source), does not define a safe quoting function for cmd.exe, does not provide a supported way to identify Windows Terminal (as opposed to detecting capabilities), and does not state how a ConPTY host should interpret every rewritten sequence. The advice on this page beyond those API definitions is practical guidance, not a specification, and the Windows-specific code was not run while writing it.

For the .NET side of installing, packaging and developing on Windows, see Development Environment & Tooling and Globalization & Localization.