Welcome to the second post in this "Windows Command-Line" series. In this post we'll discuss some of the background & history behind the Windows Command-Line. Specifically, we’ll explore its humble origins in MS-DOS, to its modern-day incarnation supporting tools like PowerShell and Windows Subsystem for Linux.
Posts in this series:
The evolution of the Windows Command-Line (this post)
In this series' previous post, we discussed the history and fundamentals of the Command-Line and saw how the architecture of Command-Lines in general has remained largely consistent over time, even while terminals evolved from electromechanical teletypes through to modern terminal applications.
Our journey now continues along a rather tangled path, starting with early PC's, winding through Microsoft's involvement with several Operating Systems, to the newly reinvigorated Command-Line of today:
From humble beginnings - MS-DOS
Back in the early years of the PC industry, most computers were operated entirely by typing commands into the command-line. Machines based on Unix, CP/M, DR-DOS, and others tussled for position and market share. Ultimately, MS-DOS rose to prominence as the de-facto standard OS for IBM PC's & compatibles, especially in businesses:
Like most mainstream Operating Systems of the time, Microsoft's MS-DOS' "Command-Line Interpreter" or "shell" provided a simple, quirky, but relatively effective set of commands, and a command-scripting syntax for writing batch (.bat) files.
MS-DOS was very rapidly adopted by businesses large and small, that, combined, created many millions of batch scripts, some of which are still in use today! Batch scripts are used to automate the configuration of users' machines, setting/changing security settings, updating software, building code, etc.
You may never/rarely see batch or command-line scripts running since many are executed in the background while, for example, logging into a work PC. But hundreds of billions of command-line scripts and commands are executed every day on Windows alone!
While the Command-Line is a powerful tool in the hands of those with the patience and tenacity to learn how to make the most of the available commands and tools, most non-technical users struggled to use their Command-Line driven computers effectively, and most disliked having to learn and remember many seemingly arcane/abbreviated commands to make their computers do anything useful.
A more user-friendly, productivity-oriented user experience was required.
The GUI goes mainstream
Many competing GUI's emerged rapidly in the Apple Lisa and Macintosh, Commodore Amiga (Workbench), Atari ST (DRI's GEM), Acorn Archimedes (Arthur/RISC OS), Sun Workstation, X11/X Windows, and many others, including Microsoft Windows:
Windows 1.0 arrived in 1985, and was basically an MS-DOS application that provided a simple tiled-window GUI environment, allowing users to run several applications side-by-side:
Windows 1.01 running on MS-DOS
Windows 2.x, 3.x, 95, and 98, all ran atop an MS-DOS foundation. While later versions of Windows began to replace features previously provided by MS-DOS with Windows-specific alternatives (e.g. file-system operations), they all relied upon their MS-DOS foundations.
Note: Windows ME (Millennium Edition) was an interesting chimera! It finally replaced the MS-DOS underpinnings and real-mode support of previous versions of Windows with several new features (esp. Gaming & Media tech). Some features were incorporated from Windows 2000 (e.g. new TCP/IP stack), but tuned to run on home PC's that might struggle to run full NT. This story might end up being an interesting post in and of itself someday! (Thanks Bees for your thoughts on this :))
However, Microsoft knew that they could only stretch the architecture and capabilities of MS-DOS and Windows so far: Microsoft knew it needed a new Operating System upon which to build their future.
Microsoft - Unix Market Leader! Yes, seriously!
While developing MS-DOS, Microsoft was also busy delivering Xenix - Microsoft's port of Unix version 7 - to a variety of processor and machine architectures including the Z8000, 8086/80286, and 68000.
By 1984, Microsoft's Xenix had become the world's most popular Unix variant!
However, the US Government's breakup of Bell Labs - home of Unix - resulted in the spin-off of AT&T which started selling Unix System V to computer manufacturers and end-users.
Microsoft felt that without their own OS, their ability to achieve their future goals would be compromised. This led to the decision to transition away from Xenix: In 1987, Microsoft transferred ownership of Xenix to its partner The Santa Cruz Operation (SCO) with whom Microsoft had worked on several projects to port and enhance Xenix on various platforms.
Microsoft + IBM == OS/2 … briefly
In 1985, Microsoft began working with IBM on a new Operating System called OS/2. OS/2 was originally designed to be "a more capable DOS" and was designed to take advantage of some of the modern 32-bit processors and other technology rapidly emerging from OEM's including IBM.
However, the story of OS/2 was tumultuous at best. In 1990 Microsoft and IBM ended their collaboration. This was due to a number of factors, including significant cultural differences between the IBM and Microsoft developers, scheduling challenges, and the explosive success and growth in adoption of Windows 3.1. IBM continued development & support of OS/2 until the end of 2006.
By 1988 Microsoft was convinced that its future success required a bigger, bolder and more ambitious approach. Such an approach would require a new, modern Operating System which would support the company's ambitious goals.
Microsoft's Big Bet - Windows NT
In 1988, Microsoft hired Dave Cutler, creator of DEC's popular and much respected VAX/VMS Operating System. Cutler's goal - to create a new, modern, platform-independent Operating System that Microsoft would own, control, and would base much of its future upon.
That new Operating System became Windows NT - the foundation that evolved into Windows 2000, Windows XP, Windows Vista, Windows 7, Windows 8, and Windows 10, as well as all versions of Windows Server, Windows Phone 7+, Xbox, and HoloLens!
Windows NT was designed from the start to be platform independent, having initially been built to support Intel's i860, then the MIPS R3000, Intel 80386+, DEC Alpha, and PowerPC. Since then, the Windows NT OS family has been ported to support the IA64 "Itanium", x64, and ARM / ARM64 processor architectures, among others.
Windows NT provided a Command-Line interface via its "Windows Console" terminal app, and the "Command Prompt" shell (cmd.exe). Cmd was designed to be as compatible as possible with MS-DOS batch scripts, to help ease business' adoption of the new platform.
The Power of PowerShell
While Cmd remains in Windows to this day (and will likely do so for many decades to come), because its primary purpose is to remain as backward-compatible as possible, Cmd is rarely improved. Even "fixing bugs" is sometimes difficult if those "bugs" existed in MS-DOS or earlier versions of Windows!
In the early 2000's, the Cmd shell was already running out of steam, and Microsoft and its customers were in urgent need of a more powerful and flexible Command-Line experience. This need fueled the creation of PowerShell (which originated from Jeffrey Snover's "The Monad Manifesto").
PowerShell is an object-oriented Shell, unlike the file/stream-based shells typically found in the *NIX world: Rather than handling streams of text, PowerShell processes streams of objects, giving PowerShell script writers the ability to directly access and manipulate objects and their properties, rather than having to write and maintain a lot of script to parse and manipulate text (e.g. via sed/grep/awk/lex/etc.)
Built atop the .NET Framework and Common Language Runtime (CLR), PowerShell's language & syntax were designed to combine the richness of the .NET ecosystem, with many of the most common and useful features from a variety of other shells scripting languages, with a focus on ensuring scripts are highly consistent, and extremely ... well ... powerful
To learn more about PowerShell, I recommend reading "PowerShell In Action" (Manning Press), written by Bruce Payette - the designer of the PowerShell syntax and language. The first few chapters in particular provide an illuminating discussion of the language design rationale.
PowerShell has been adopted by many Microsoft platform technologies, and partners, including Windows, Exchange Server, SQL Server, Azure and many others, and provides commands to administer, and control practically every aspect of a Windows machine and/or environment in a highly consistent manner.
PowerShell Core is the open-source future of PowerShell, and is available for Windows and various flavors of Linux, BSD, and macOS!
POSIX on NT, Interix, and Services For UNIX
When designing NT, Cutler & team specifically designed the NT kernel and OS to support multiple subsystems - interfaces between user-mode code, and the underlying kernel.
When Windows NT 3.1 first shipped in 1993, it supported several subsystems: MS-DOS, Windows, OS/2 v1.3, and POSIX v1.2. These subsystems allowed NT to run applications targeting several Operating System platforms upon the same machine and base OS, without virtualization or emulation - a formidable capability even today!
While Windows NT's original POSIX implementation was acceptable, it required significant improvements to make it truly capable, so Microsoft acquired Softway Systems and its "Interix" POSIX-compliant NT subsystem. Interix was originally shipped as a separate add-on, and then later combined with several useful utilities and tools, and released as "Services For Unix" (SFU) in Windows Server 2003 R2, and Windows Vista. However, SFU was discontinued after Windows 8, due largely to a lack of customer interest.
And then a funny thing happened...
Windows 10 - a new era for the Windows command-line!
Early in Windows 10's development, Microsoft opened up a UserVoice page, asking the community what features they wanted in various areas of the OS. The developer community was particularly vociferous in its requests that Microsoft:
Make major improvements to the Windows Console
Give users the ability to run Linux tools on Windows
Based on that feedback, Microsoft formed two new teams:
The Windows Console & command-line team, charged with taking ownership of, and overhauling the Windows Console & command-line infrastructure
A team responsible for enabling genuine, unmodified Linux binaries to run on Windows 10 - the Windows Subsystem for Linux (WSL)
The rest, as they say, is history!
Windows Subsystem for Linux (WSL)
Adoption of GNU/Linux based "distributions" (combinations of the Linux kernel and collections of user-mode tools) had been growing steadily, especially on servers and in the cloud. While Windows had a POSIX compatible runtime, SFU lacked the ability to run many Linux tools and binaries because of the latter's additional System Calls and behavioral differences vs. traditional Unix/POSIX.
Due to the feedback noted received from technical Windows customers and users, along with an increasing demand inside Microsoft itself, Microsoft surveyed several options, and ultimately decided to enable Windows to run unmodified, genuine, Linux binaries!
In Mid 2014, Microsoft formed a team to work on what would become the Windows Subsystem for Linux (WSL). WSL was first announced at Build 2016, and was previewed in Windows 10 Insider builds shortly afterwards.
In most Insider builds since then, and in each major OS release since Anniversary Update in fall 2016, WSL's feature-breadth, compatibility, and stability has improved significantly: When WSL was first released, it was an interesting experiment, ran several common Linux tools, but failed to run many common developer tools/platforms. The team iterated rapidly, and with considerable help from the community (thanks all!), WSL quickly gained many new capabilities, enabling it to run increasingly sophisticated Linux binaries and workloads.
Today (mid 2018), WSL happily runs the majority of Linux binaries, tools, compilers, linkers, debuggers, etc. Many developers, IT Pro's, devops engineers, and many others who need to run or build Linux tools, apps, services, etc. enjoy dramatically improved productivity, being able to run their favorite Linux tools alongside all their favorite Windows tools, on the same machine, without needing to dual-boot.
The WSL team continues to work on improving WSL's ability to execute many Linux scenarios, and improve its performance, and integration with the Windows experience.