draft to enable pwsh to be used as login shell

[SteveL-MSFT]

No description provided.

[iSazonov]

@mklement0 Please look the RFC.

[iSazonov]

In short:

1. When I looked through Posix I saw that it blindly describes the behavior of bash. The /etc/profile file format is a bash script and it is impossible to parse it without bash.
   If we talk about implementation, we need to call bash -l /etc/profile and get PATH value.
   This is already in this RFC.
   But ...

2. If we see PowerShell as a popular shell, then we should describe PowerShell's behavior as a login shell without blindly copying bash implementation details. I mean that we have powershell.config.json and related RFC which we should enhance for login/interactive shell scenarios.
   In particular, we could add to this config PATH option (and option to read PATH from bash on Linux as descibed in previous paragraph or call path_helper on MacOs).
   Also we could add powershell.login.config.json config files if we decide we need it.

3. We say about login shell so -l should be -Login or -LoginShell but not -LoadProfile.

4. From the RFC I'd expect that we descibe not only PATH env problem but full login shell behavior.

[mklement0]

@SteveL-MSFT (I've deleted my previous comment after digging deeper):

I've updated PowerShell/PowerShell#975 (comment) to show that we indeed need to use the call-sh-and-copy-environment-variables on Linux too, to support ssh connections.

However, what I now realize is a must is that PowerShell recognize a - as the first character of argv[0] ($0) as an alternative way (to -l) to request a login shell, because that is the mechanism use by macOS terminal programs and by ssh (and also by Bash's exec -l).

[Environment]::CommandLine doesn't reflect such such a custom invocation name, however (it contains the path to PowerShell DLL, which is a bug yet to be resolved - see https://github.com/dotnet/coreclr/issues/20562 - and even once fixed I'm unclear on whether it would reflect a custom invocation name).
As a workaround, PowerShell could determine its invocation name via the standard ps utility, using command ps -o comm= $PID, which, unfortunately, requires yet another child process.

[SteveL-MSFT]

@mklement0 as always, I appreciate you digging into these details. I don't think we should put in a workaround for argv[0] starts with - == login shell and would prefer to wait for .NET Core support

[mklement0]

Thanks, @SteveL-MSFT; I've asked for clarification in https://github.com/dotnet/coreclr/issues/20562 whether the fix will give us what we need; it makes sense to wait for a proper fix, but just to be clear: without it, whatever functionality this RFC implements will lie dormant in (at least) two important scenarios: creating a new shell interactively on macOS, and using ssh to connect to a machine remotely, on both macOS and Linux.

[mklement0]

@iSazonov

Just a clarification up front:

The shell-related part of POSIX is a specification that compliant shells must adhere to; POSIX-like shell implementations, such as Bash, comply to varying degrees.

Profile files, from what I can tell, are actually not prescribed by POSIX, but de facto POSIX-like shells (with the exception of zsh) do process /etc/profile when invoked as login shells - and situationally also ~/.profile (Bash, when invoked as itself and not as sh, gives precedence to its own ~/.bash_profile or ~/.bash_login file, if present; Dash, which act as sh on Ubuntu, always reads ~/.profile).

Reading /etc/profile and ~/.profile seem to date back to the original Bourne shell, from which the POSIX shell specification was generally derived (but clearly not in all aspects).

Note that these files shouldn't contain Bash code, only code that complies with the POSIX shell specification, which is a subset of Bash. Limiting the code to POSIX-only language features ensure that all POSIX-compatible shells can read it.

The only POSIX-mandated behavior with respect to startup files is a script specified via the optional ENV environment variable, to be sourced in interactive shells only (which may or may not also be login shells; as such, it is comparable to ~/.bashrc, although the latter is only automatically sourced in interactive shells that are not also login shells). In practice, this is rarely used.

[mklement0]

@iSazonov

Re 3.: Agreed.
Re 4: Not sure what you mean.

Re 2:

I get that that the proposed approach is unsatisfying: it is clunky, slow, and only half solves the problem, given that it covers only environment variables, whereas /etc/profile can conceivably contain other process-level modifications that we wouldn't catch, such as umask (default file permissions) and ulimit (setting various resource limits, such as the max. number of open files).

It is desirable, however, to fully honor /etc/profile, given that at least the original intent was to provide one-time initialization that isn't just specific to the shell that processes the file, but sets defaults for all processes, i.e., system-wide configuration.
(This differs from initialization files such as ~/.bashrc, which are for shell-specific customizations in interactive shells; PowerShell profiles are like the latter, despite sharing the name profile with login-shell profile files in the POSIX shell world.)

If we provided a separate, PowerShell-specific configuration, we'd be asking users to duplicate these settings solely to accommodate PowerShell.
(As an aside: we currently lack the ability to provide the functionality of utilities such as umask or ulimit - see PowerShell/PowerShell#6725)

At least Ubuntu seems to be moving away from relying on /etc/profile, providing alternative configuration mechanisms such as /etc/environment (environment variables) and /etc/login.defs (for, e.g., umask values) that do not rely on shells for execution and consistently apply the same setting to both user-created and daemon-created processes, so that using ssh results in the same environment.

On such a system that doesn't rely on /etc/profile anymore, PowerShell wouldn't have to do anything to be a login shell, but the need to support /etc/profile will unfortunately not go away anytime soon.

On Ubuntu, for user-created PowerShell sessions, PowerShell needn't process /etc/profile anyway, because it inherits all environment variables and process attributes from a hidden sh instance that runs on login and itself processes /etc/profile. Unfortunately, that is not the case for a daemon-created shell process, such as when you remote into a machine via ssh.

On macOS, most system configuration is managed via launchd, there is no single sh process that processes /etc/profile on startup from which other user processes inherit, which is why the macOS terminal emulators create every shell as a login shell. Curiously, macOS has no mechanism anymore for creating truly system-wide environment variables (there used to be a launchd-based mechanism, which was removed). The next best thing is to use /etc/profile to define environment variables at least for shell processes and processes launched from them. The addition to $env:PATH that /etc/profile performs can be had directly (using something like $env:PATH =(((/usr/libexec/path_helper -c) -split ' ', 3)[-1] -split '"')[1]), without needing to process /etc/profile, but that would miss other environment variables potentially defined in /etc/profile (there are none by default).

[mklement0]

@SteveL-MSFT:

Here's another approach to consider:

• (On Unix) PowerShell detects if it was called with -l or its argv[0] value starts with a - (once supported by CoreFx).

• If so, as early as possible, it uses an exec*() system call to replace itself with
  /bin/sh -c '. /etc/profile; exec /path/to/pwsh <originalArgs...>', which creates a sh instance that sources /etc/profile and thereby sets all environment variables and process attributes, and then reinvokes PowerShell with the original arguments (sans -l, if present), all in the context of the original process.

While this is obviously slow,

• it solves the problem with the current proposal not preserving process attributes.
• it should be simple to implement.

Note that on Linux the performance penalty would typically only be paid via ssh, given that shell sessions are normally non-login shells.

On macOS, however, the penalty would be paid whenever the user creates a new session by opening a new tab or window in a terminal emulator.

[SteveL-MSFT]

@mklement0 the concern I have with your proposed approach is that pwsh would incur the cost of starting twice which IS slow. Simply starting sh is pretty fast. If we go back to the model Windows PowerShell had which is a native powershell.exe that hosts the CLR, this wouldn't be as big an issue, but starting managed processes is still an order of magnitude slower than starting a native process.

[mklement0]

Yes, pwsh would start twice, but - speaking naively, because I know little about the startup process - perhaps optimizations can be made to detect the login-shell request as early as possible, before modules are loaded, ... - though perhaps that won't make a noticeable difference compared to the startup time of the "plumbing".

Given the relative rarity of login shells - at least on Linux - when given the choice between a faster half-workaround and a slower, but complete workaround, my vote is for the latter.

[SteveL-MSFT]

@mklement0 to give you a sense of the cost of startup for a managed assembly, on my macBook Pro:

PS> (Measure-Command { pwsh -? }).TotalMilliseconds
235.5875
PS> (Measure-Command { pwsh -c exit }).TotalMilliseconds
1097.442

In the first case, all the processing is within pwsh and it doesn't create a PowerShell runspace, simply outputs help. The second case, it creates a runspace and exits.

Simply shelling out to sh to process profile is really fast:

PS> (Measure-Command { /bin/sh -c /etc/profile }).TotalMilliseconds
14.1075

I'm currently just looking for a "good enough" solution so people can start using pwsh as their default shell. umask and ulimit will be an issue in the future, but I suspect for now people who want those will prefer whatever shell they are already using.

[mklement0]

That's illuminating, @SteveL-MSFT, though note that your numbers include processing of PowerShell profile files in the case of the -c command; eliminating that with -noprofile cuts the slowdown to a factor of about 2 on my machine.

However, that's still significant, and what I'm proposing indeed does not require creating a PowerShell runspace on the initial invocation, so the performance impact wouldn't be that bad, right?

[mklement0]

As for:

umask and ulimit will be an issue in the future, but I suspect for now people who want those will prefer whatever shell they are already using.

My sense is that /etc/profile is neither perceived nor used as a shell-specific file; instead, it is about system-wide configuration.

[iSazonov]

I very wonder that we discuss low level implementation details in RFC repository instead of and before defining general conceptions and features of their existence on various platforms.

[SteveL-MSFT]

@mklement0 yes, I forgot about my $profile which inflated the times by quite a bit. Perhaps if I can find some free time I can prototype both approaches and see what the perf difference is.

[iSazonov]

I'd want to get answers from the RFC

• what is interactive vs noninteractive vs login
• what is difference (in particular for login shell) on Windows vs Linux vs MacOs

Also I'd start with defining config files (and GPO for enterprises) for every shell type:

• powershell.config.json
• powershell.login.config.json

In this configs we could have options:

• directly set important environment variables (add/replace values)
• special cases for ModulePath and PATH
• option to run external utility to get values (like path_helper or sh /etc/profile)

Such an approach will give a good understanding of the behavior and will improve adaptation to any (Unix) distributives.

[joeyaiello]

Haven't read the thread yet, but I thought it was worth noting here that we should ignore SHLVL (the shell level) and _ (the last run command, I believe?) Both will be inaccurate when we dump them out and copy them into PS.

[SteveL-MSFT]

SHELL needs to not be inherited. Proposal is to rename to -LoadEtcProfile

[mklement0]

@SteveL-MSFT: SHELL reflects the user's default shell, not the one currently running, so it can safely be inherited.

@joeyaiello: SHLVL would indeed be inappropriate to inherit; _, by contrast is a shell variable (current-process-only), not an environment variable, which is generally an important distinction to make.

With the reinvoke-via-sh approach I've proposed, you wouldn't need to worry about shell variables, because they would invariably be scoped to the sh process.

[mklement0]

@iSazonov:

My sense is that logon shells are a legacy construct, stemming from a time when the shell was the only UI.
System-wide configuration shouldn't rely on a particular shell's startup behavior, and things are moving away from that. A shell's initialization files should be limited to configuring that shell's behavior.

Therefore:

• We should try to be as compatible as is reasonably possible in order to accommodate the remnants of that legacy behavior, most notably on macOS.

• However, we do not want to get into the business of concocting a PowerShell-specific login shell feature, let alone via .json files.

[rjmholt]

@joeyaiello, @JamesWTruher and I discussed this today.

I was a bit dissatisfied with the idea of executing the profile file with sh and sucking up the output.

My first instinct was that maybe starting pwsh as a subprocess of sh so that it inherits the variables, but didn't like the implementation detail of the sh process wrapping pwsh.

@JamesWTruher suggested that this is exactly what exec is for (note that exec decrements SHLVL in bash).

@joeyaiello then suggested that a simple and effective solution is a wrapper script (perhaps /bin/pwsh-login):

#/bin/sh -l
exec pwsh "$@"

Discussing this we were curious about how other sh-incompatible shells are used as login shells, and:

• The prevailing suggestion for fish is to do the same as above
• csh/tcsh have their own .login concept and don't even try to run .profile

[mklement0]

@joeyaiello then suggested that a simple and effective solution is a wrapper script (perhaps /bin/pwsh-login):

We're talking about using using pwsh as a user's default shell.

This means that a single executable - pwsh - must be:

• callable as an interactive non-login shell
• callable as a login shell (explicitly with -l or implicitly with $0's first char set to -), whether interactive or not

So it sounds like a wrapper script with a different executable name is not an option - or am I missing something?

That is, unless you want to:

• have a confusingly named default shell
• burden each and every shell instance with having to process /etc/profile - which is unnecessary in the vast majority of cases on Linux (all locally started sessions); while it is an unfortunate necessity on macOS for interactive shells started by terminal emulators, it also isn't necessary for instances invoked with -c and interactive shells started with just the executable name from exiting shells.

---

Pragmatically speaking, I don't think we need too concerned about the user-level ~/.profile file, given that even Bash ignores it if its own alternatives are present (~/.bash_profile or ~/.bash_login).

By contrast, it is the system-wide, shell-transcendent importance of /etc/profile that makes it important to support.

[iSazonov]

My sense is that logon shells are a legacy construct,

It seems a main feature of login shell is to reset process "config" and create it from scratch (including env).
Note that PowerShell already has a parameter to zero environment variables at startup.
Perhaps it would be useful to have pwsh -login to get mostly clean process "config".

[mklement0]

It seems a main feature of login shell is to reset process "config" and create it from scratch (including env).

Note: I am not an expert in these things, but hopefully the essence of the following is correct:

No. My understanding is that login shells are not about resetting (on demand), but about one-time initializations at the time a user OS session starts (when the user logs on to a machine), and that initialization sets the environment variables and process attributes for all subsequently invoked processes, including other shell instances.

This dates back to a time of the Bourne shell, the predecessor of the POSIX shell specification and modern POSIX-like shells, when a character-mode shell was the shell for a user's OS session (not a GUI shell as is customary today).

At least Ubuntu still seems to be doing that behind the scenes, via a hidden sh instance that processes /etc/profile - which is why we have no problem with user-launched PowerShell instances there (unless a login shell is explicitly requested): the hidden sh instance has already done the work, and PowerShell sees that environment.

Regrettably, macOS doesn't; on macOS - for reasons unknown to me - GUI apps and POSIX-compatible shells are designed to see different environments, with shells expected to make additional modifications to the environment beyond the shared mechanism (launchd), most notably the initial $env:PATH entries via /usr/libexec/path_helper (if we limited our support to that, we could trivially parse its output, without the need for external processes).

The other major scenario in which login shells matter is in remoting: when you use ssh, it is like starting an OS session for that user on that machine, and the same one-time initializations should be performed - which is why ssh creates a login shell based on the user's default shell.

Now, if the SSH daemon on the target machine had (indirectly) also processed /etc/profile on startup, there would be no need to create a login shell either, as in local operation (once the hidden "mother of all users processes" has processed /etc/profile) - there may be good technical reasons not to do that, though (I don't know).

However, my sense is that Ubuntu is trying to move way from /etc/profile precisely in order to unify these two scenarios (interactive OS session, ssh session), via the shell-agnostic /etc/login.defs.
Again: In this day and age, it shouldn't be up to a (user's default) shell to implement what are meant to be system-wide initializations.

---

The above is distinct from wanting to start a shell with a pristine environment on demand; I know of Start-Process -UseNewEnvironment (which is currently broken - see PowerShell/PowerShell#4671) - are you saying that there's an (undocumented) CLI parameter too?

[iSazonov]

My understanding is that login shells are not about resetting (on demand), but about one-time initializations at the time a user OS session starts (when the user logs on to a machine)

Also we can type bash -login in interactive session and again get "clean" session.

---

Ubuntu has its way. MacOs has its way too. And so on.
The lack of a common approach in the Unix world forces us to describe common features and provide opportunities for their easy adaptation to any distribution. This could be done as I wrote above.

[iSazonov]

The above is distinct from wanting to start a shell with a pristine environment on demand; I know of Start-Process -UseNewEnvironment (which is currently broken - see PowerShell/PowerShell#4671) - are you saying that there's an (undocumented) CLI parameter too?

No, I meant this issue only.

[mklement0]

Also we can type bash -login in interactive session and again get "clean" session.

No, it's not a clean session. Any environment variables and process attribute that aren't set in /etc/login will linger (and, obviously, whatever the Bash login shell does wouldn't affect a calling PowerShell session).

But the main point is that that providing a "clean" session is not the purpose of a login shell.

[iSazonov]

But the main point is that that providing a "clean" session is not the purpose of a login shell.

What is definition of login shell? In common what is interactive vs noninteractive vs login?

[mklement0]

I've tried to explain the purpose of login shells in previous comments.

Their purpose is entirely unrelated to interactive vs. noninteractive - in Bash terms, that distinction matters only with respect to the "rc file", ~/.bashrc, whose purpose is to initialize interactive sessions - which is why unattended invocation with -c does not load that file; a distinction that PowerShell regrettably doesn't observe.

The lack of a common approach in the Unix world forces us to describe common features and provide opportunities for their easy adaptation to any distribution

PowerShell cannot and therefore shouldn't even try to fulfill that role:

Neither on Linux (at least based on my Ubuntu experience) nor on macOS does it get a chance to provide system-wide initializations when a user's OS session starts:

• On Ubuntu, it is invariably a /bin/sh instance that processes /etc/profile when a user's OS session starts and determines the environment for all user-created processes.

• On macOS, no shell gets to play that role.

To put it differently: Even making PowerShell a user's default shell does not enable it to perform system-wide initializations.

[SteveL-MSFT]

Completely rewrote the RFC based on feedback and @PowerShell/powershell-committee discussion. Please review the current proposal.

[SteveL-MSFT]

Closing this PR and will submit update as new one as the current comments are outdated.

[SteveL-MSFT]

New PR #186