from Hacker News

Why Create a New Unix Shell?

by shadeless on 1/31/18, 7:45 PM with 292 comments

by ComputerGuru on 1/31/18, 8:32 PM
Lots of overlap in design goals with fish, except fish also places a premium on users interactively using the shell (which means friendlier in-repl experience but a balancing act when it comes to features). Fish’ auto completions are incredible, too.
Best of luck to them. Another interesting shell to check out is elvish, lots of new ideas there (even if awkward to use).
(Disclosure: I’m one of the core fish devs/maintainers. Edit: The entire team is awesome and the others deserve virtually all the credit!)
by jhillyerd on 1/31/18, 8:27 PM
After using fish for 3 years, I'm finding there is very little reason to have my login shell maintain backwards compatibility with bash.
The only time I run into issues is when a command expects manipulate environment variables via bash syntax.
I think the fish documentation WRT to scripting could be much better, but the language is more elegant than bash or PowerShell IMHO.
by simias on 1/31/18, 9:21 PM
The idea to use a real programming language as a shell is a common one but I'm not sure it's really a problem that can be solved without reworking the kernel interface. Whatever you do pipes will still be byte streams, error handling will always be using integer return values, you'll always have stdin, stdout and stderr, job control and signal handling will always work pretty much the same way. The kernel interface exposes an interface, the userland app expect the shell to behave in the certain way, there's not a lot of wiggle room to make things differently in between the two.
Not that the POSIX-like shell syntax is not all sorts of clunky and odd but I almost consider it a feature, it's a deterrent to force you to move to a "real" scripting language when the concepts become too complex to express in a shell script.
by kevlar1818 on 1/31/18, 9:51 PM
> However, Python and Ruby aren't good shell replacements in general. Shell is a domain-specific language for dealing with concurrent processes and the file system. But Python and Ruby have too much abstraction over these concepts, sometimes in the name of portability (e.g. to Windows). They hide what's really going on.
Excellently put. POSIX shell languages have fantastic capabilities you just can't get in most other languages. I would love to see a more safe, more sane shell language gain enough popularity to change the narrative that "shell scripts are dangerous and impossible to maintain."
The contrasts to Python and Ruby made me think of xonsh[1], a Python-based shell that can dynamically switch between Bash-like syntax and standard Python. It's not quite ready to become my daily driver, but I'm still excited about it.
[1]: https://xon.sh
by jernfrost on 1/31/18, 10:13 PM
I essentially solved these problems with fish shell and Julia programming language.
For all sorts of interactive stuff I use fish, because it works the way you want for the most common tasks. I can use it to really quickly match and get back previous statements or do a completion.
Also much easier to configure and grok than bash, because it has saner syntax and is a simpler shell language.
However when writing shell scripts I use a real language like Julia. It integrates very well with the shell world so I don't find it problematic to do this. It is very easy to read output from processes and pipe stuff. Much nicer than say Python or Ruby.
You got built in syntax to deal with shell stuff, but then didn't make it crazy so you end up with a mess like perl. Julia is actually a very clean and nice language. Which also happens to blistering fast and have LISP style macros.
by castis on 1/31/18, 8:34 PM
I see that this person has opted not to use python 3 because it is 'less-suited to shell-like problems' than python 2.
In an effort to understand the reasons for actively choosing against 3, does anyone know what problems those would be?
by zapita on 1/31/18, 8:45 PM
Finally a modern shell that understands the importance of COMPATIBILITY! This gives it a realistic chance of getting real adoption. Shells like zsh and fish will never get mainstream adoption because they are not compatible with bash.
by anderspitman on 1/31/18, 11:19 PM
Love it. I think there's plenty of room for innovation in this space. I'm currently in the process of porting a ~700 line bash script (I didn't write it) to Python. Although longterm I think this will be much better for the project, there are still tradeoffs. There are some things that are just so easy to express in shell language, like composing transformations via pipes. Sure, python can do it, but it feels clunky in comparison to me. I would love to see a language like python (including package ecosystem) written with shell use as a first-class citizen.
by dmix on 1/31/18, 8:59 PM
I checked out Oil previously which looks nice but more of an incremental improvement over Fish/ZSH rather than a significant evolution (it may have changed since then, this was last year).
Im most excited about Elvish shell and the language that's being developed around it. The shell is built with Go and feels super fast compared to my plugin-heavy ZSH. The language design is quite nice too but still very alpha. Looking forward to see what it evolves into...
https://github.com/elves/elvish
by JepZ on 1/31/18, 11:25 PM
@chubot: There is one use-case I come across every once in a while:
https://stackoverflow.com/questions/356100/how-to-wait-in-ba...
So whenever you want to do things in parallel there is probably a limit to the number of processes you would like to execute in parallel (e.g. the famous compiler limit formula: number of CPU cores +1). It would be great if Oil could support such a use-case out of the box, as easy parallelism without the ability to artificially limit the number of parallel executions is often useless.
by z3t4 on 1/31/18, 9:21 PM
I spent the day making a virtual terminal ... I think the terminal is an unnecessary layer. All program UI is limited by this 40+ year old technology that is the terminal. Instead of making a new shell, make a shell + new user-interface.
by tambourine_man on 1/31/18, 9:05 PM
A little tangential, but I keep wondering if Apple is developing its own shell or will adopt one with a more liberal licence such as Oil or Fish.
I mean, they can't keep using Bash 3 forever, right? (Hope)
by unixhero on 2/1/18, 6:59 AM
Ignore the naysayers. I have encountered them when posting my shell activities. You're doing great! Can't wait to try the final release. Maybe even get it into my workflow.
by jiggunjer on 2/1/18, 2:59 AM
So many shells, but how come the virtual terminal hasn't got a revamp or popular alternative? I'd like to see more control over keybinding (e.g. on my machine pressing ctrl+del sends the same character as pressing ctrl+backspace). I understand this would be more a kernel change than a userland one.
Screen tiling and visual 'tabs' would also be welcome additions. Not everyone needs a graphic environment, and I refuse to install X just for better keyboard shortcuts on my terminal.
by JepZ on 1/31/18, 11:07 PM
Sounds pretty cool and everybody who had to learn bash scripting at some point understands why we need a sane language (my favorite are misplaced spaces in if statements...; Disclaimer: I do and love bash scripting but while the language has cool concepts, some things are just broken by design).
Nevertheless, there is one piece in this puzzle I am missing. There does not seem to be a process which manages the 'core software set' across platforms. So after decades we finally have a shell which is available on most operating systems, but how long will it take before Microsoft, Apple, Oracle, etc. will adopt a new shell?
So why don't the large OS corporations form a consortium to define something like a 'cross platform run time environment' standard (maybe together with the Linux Foundation and some BSD guys?). I mean its not so much about which shell someone prefers, but more about a common set of interpreters and maybe tool kits. And even more than that it is not about the state but the process/progress.
What do you think, do we need such a process or is there another way to solve the cross platform dilemma?
by XorNot on 1/31/18, 10:43 PM
The pain of "not quite bash" ultimately is what put me off xonsh as my daily shell. Also losing it whenever I SSH'd somewhere.
The latter problem could probably be solved with a wrapper which would pipe and execute the shell (or a bytecode interpreter ala shuttle?) automatically - but I've seen no alternative shell project take this part seriously for the problem space.
by fnord77 on 2/1/18, 4:41 AM
> Oil is taking shell seriously as a programming language, rather than treating it as a text-based UI that can be abused to write programs.
erm, there's a big difference between a command scripting language and a programming language. These should be treated as different things.
I have years of experience using both, and I really don't want to be doing shell tasks in a programming language and I don't want to write programs in a shell language. Those sorts of hybrids are almost always mediocre. Horses for courses and all that.
There's a reason bash keeps being used - it's mature, it's simple, it's easy and people are productive with it.
by aplorbust on 1/31/18, 11:48 PM
This keeps showing up on the front page. No doubt it will have users.
Lets say there are two uses of a shell: 1. interactive and 2. non-interactive (scripting).
Lets imagine the commandline user is learning about her OS. She learns it is heavily reliant on shell scripts to build and (if desired) to automate starting services.
She realises that to understand the OS she will have to learn the shell that the OS developers used for scripting.
Then she realises that if she chooses another shell for interactive use, she will have to learn two shells.
Finally she realises that any script she writes in the "non-interactive/scripting" shell will also run under the interactive one. But not vice versa.
If she only has enough time in life to master one shell, which one should she choose?
Over time I found I really cared more about the scripting aspect of a shell than the interactive facet.
The scripting shell used by the OS authors might be an Almquist derived shell, for instance.
Occasionally the ash Im using gets a new "feature" but not too often. I like that it stays relatively small. The latest "feature" is LINENO.
But I also use a smaller version of this shell with no command line history, no tabcomplete, etc. IMO, there is no better way to learn how to reduce keystrokes. It has led to some creativity in this regard for which I am thankful.
After "mastering" ash, I started using execlineb, pipeline and fdmove. I am starting to use more components of execline and am continually replacing ash scripts with execline scripts for more and more daily work.
I guess we will never see execline on the front page, which I think would be interesting because I would like to hear whatever harsh critique HN can muster.
Seeking a better non-interactive/scripting experience, I have experimented with many other shells over the years, and written simple execve "program launchers", but in this vein, I have not found anything that compares to execline.
The speed gains and resource conservation are obvious, but with the ability to do "Bernstein-chaining" and the option to use djb low-level functions instead of libc, it is a rare type of project.
The speed and cleanliness of the compilation process is, compared to all the other crud one routinely encounters in open source projects, "a thing of beauty". Humble opinion only, but I think others might agree.
by petre on 2/1/18, 6:02 AM
Oil syntax looks pretty much like Tcl and Th[1] so the author could have probably just used Th. It has saner ways to copy arrays than
```
  b = [ @a ]
```
which pretty much looks like Perl with added line noise. Why are the [] even necessary when it's clear @a is an array?
1. http://www.sqliteconcepts.org/THManual.pdf
by desireco42 on 2/1/18, 4:57 AM
As a fish user, I didn't really get where the advantages lie as clearly. I am for developing new shells, we really need this, innovation is important.
As for new language, I feel like if you want to script things, you can use ruby or python, hell, perl will do and you could be fine. I don't want to be unfair to this effort, I just feel that it is not for me and I am tinkerer.
by tyingq on 2/1/18, 5:52 AM
Applaud the idea, but you are fighting a ton of inertia. I have to wonder if drawing a more clear line of when to go to Perl, Python, Lua, Ansible, Golang, etc, might be more fruitful. Sometimes, a shell script solution is just drawing any kind of shell too far outside it's core competency.
by cestith on 1/31/18, 9:50 PM
In your FAQ you decry Perl as having no ability to redirect around other programs. Yet you don't explain how any of the following fail to meet those needs.:
* http://perldoc.perl.org/functions/open.html
* http://perldoc.perl.org/IPC/Open2.html
* http://perldoc.perl.org/IPC/Open3.html
* http://search.cpan.org/~odc/IPC-Open2-Simple-0.01/lib/IPC/Op...
* http://search.cpan.org/~exodist/Child-0.013/lib/Child.pm
* http://search.cpan.org/~rkrimen/IPC-RunSession-Simple-0.002/...
* http://search.cpan.org/~trski/Proc-Forkmap-0.025/lib/Proc/Fo...
* http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm
* http://search.cpan.org/~ayoung/IPC-Run3-Simple-0.011/lib/IPC...
* http://search.cpan.org/~rjbs/IPC-Run3-0.048/lib/IPC/Run3.pm
* http://search.cpan.org/~djerius/IPC-PrettyPipe-0.03/lib/IPC/...
* http://search.cpan.org/~xan/IPC-Pipeline-1.0/lib/IPC/Pipelin...
* http://search.cpan.org/~sscaffidi/IPC-OpenAny-0.005/lib/IPC/...
* http://search.cpan.org/~glai/IPC-Exe-2.002001/lib/IPC/Exe.pm
* http://search.cpan.org/~zefram/IPC-Filter-0.005/lib/IPC/Filt...
by shmerl on 2/1/18, 3:02 AM
I recently discovered Fish, which is actually not that recent, but also an interesting shell.
by wainstead on 1/31/18, 8:49 PM
> Shouldn't we discourage people from writing shell scripts?
...people frequently ask this? Tip #21 of "The Pragmatic Programmer" states: "Use the Power of Command Shells."
by zimbatm on 2/1/18, 11:13 AM
What I would really like to see happening in Oil shell is a way to run a strict subset of the bash language. Ideally with a tool to convert existing scripts in that subset.
by breatheoften on 2/1/18, 6:14 AM
I would love an interactive shell that applies a type-checker to my input before running it ... is that really an infeasible desire in 2018 ...?
by fiatjaf on 1/31/18, 9:17 PM
What kind of shell can I run on a server without filesystem access that I can open to external untrusted users?
by chrisvalleybay on 2/1/18, 8:18 AM
Why is it called Oil? That name is so loaded. I think it actually will give it less of a chance.
by Sir_Cmpwn on 1/31/18, 9:54 PM
I use fish but I hate it. My perfect shell is strictly POSIX sh but has a better interactive experience (better tab completion and typeahead like fish has).
by fiatjaf on 1/31/18, 9:17 PM
Wasn't there a in-browser Javascript shell somewhere a while ago? It could fetch URLs and do cool stuff with APIs.
by fanf2 on 2/2/18, 1:05 AM
No mention of plan9 rc or Tcl so it is hard to believe it is really breaking the mould
by psibi on 1/31/18, 9:51 PM
Looks nice! Any reason why Apache was used as it's license ?
by oleks on 1/31/18, 9:07 PM
Why is Oil implemented in Python? IMO Python is a terrible language for writing programming languages.
by donatj on 1/31/18, 11:32 PM
I was with the guy right up until the bile laden PHP hated started coming in as a justification.
by devit on 1/31/18, 11:37 PM
Seems an interesting idea, but it's implemented in Python, which means it will never replace bash and probably not achieve any significant adoption unless they rewrite it in Rust first (which they should have written it in to begin with since they started in 2016).
The reasons for that are that shells must start very quickly (due to subshells, local ssh, etc.), be fast, have no complex dependencies since they are used to recover broken systems, be portable but also with full support for OS semantics and be written in a language that allows rapid development of robust software, none of which Python does well.