While Cicada is pretty clearly modeled on the "old" generation of shells (sh, Bash, Zsh, etc.), one has to wonder what a more modern shell might look to address some of the problems of its predecessors like pipes that are essentially text-only, poor composability (see `cut` or `xargs`), and terribly obscure syntax.
PowerShell seems to have been a flop outside of the Windows ecosystem, and between overly elaborate syntax and COM baggage, was probably rightfully passed over, but it probably had the right idea. Imagine a future where we're piping structured objects between programs, scripting in a modern language, and have IntelliSense quality auto-completion for every command.
---
Text is compatible with all systems, past and present, and can be used to model more complex objects. Let's not add features to the core on a "why not" please.
So in principle, treating those as complex object structures is the right way to go. Also, I believe this is the idea behind D-Bus and similar modern Unix developments.
However, what's hard is to provide good tooling with a simple syntax that is simple to understand:
* Windows Registry and Power Shell show how not to do it.
* The XML toolchain demonstrate that any unnecessary complexity in the meta-structure will haunt you through every bit of the toolchain (querying, validation/schema, etc.).
* "jq" goes somewhat into the right direction for JSON files, but still hasn't found wide adoption.
This appears to be a really hard design issue. Also, while deep hierarchies are easier to process by scripts, a human overview is mostly achieved by advances searching and tags rather than hierarchies.
A shell needs to accomodate for both, but maybe we really just need better command line tools for simple hierarchial processing of arbitrary text file formats (ad-hoc configs, CSV, INI, JSON, YAML, XML, etc.).
The sole point of structural data is to turn the storage into a formal system, and enable more math operations on the entries, which align to the desired semantics. Python/js/clojure scripts extract some structural aspects from text, and then the story becomes the same as structural data processing.
For modern shell, please check out xonsh: http://xon.sh/ - powered by Python - It's super cool!
Pipes work fine with binary data, you just need to be piping to a tool that works with binary data rather than text e.g.
cat <somebinaryfile> | openssl base64
openssl base64 <somebinaryfile
On the plus side, you do get a prize for suggesting it [1]
Something involving bulk binary data moving over a pipe which isn't an indirect redirection would be:
gzip somefile | ssh user@host "gunzip >somefile"
Although, again, there's a perfectly good -c flag to SSH that would do the same thing.
Probably the most common example of tool-to-tool binary communication over pipes is:
find -print0 | xargs -0
You'd either have to be very careful (e.g. feature flags guarding every change) or expect a surprising amount of things to fail.
It seems like the way to go to use non-POSIX shell as the default shell without changing login shell, is to have .bashrc `exec`'d that shell when it is running in interactive mode. Regardless, I would really love to see bash-like shell that provides out of the box experience in the same level as fish.
A new system and set of utilities. None of the Unix environment works in that world, and shoehorning it in would feel incredibly clunky.
The problem with passing objects around though is it can cause issues with the parallel nature of pipes. ie the programs in a chain of pipes can run concurrently since the data is being passed is just a stream. But with objects you need to read the object in its entirety to ensure it is a valid object. This means each process is waiting for the previous process to finish outputting it's object before the next can start processing it. While this is less of an issue with smaller chunks of data in a chain, it would become really noticeable really quickly as your data scales up.
I'm sure will be workarounds for the above problem but my project is still young so there are other bugs I'm more focused on (for me I'm more interested in getting an IDE-level auto-complete implemented so writing one liners in the Shell is as natural as writing your favourite language in your preferred development tool.
[1] https://github.com/playframework/play-iteratees [2] http://www.reactive-streams.org/
-rw-r--r-- 1 brandur staff 6.5K Mar 25 15:37 Makefile
-rw-r--r-- 1 brandur staff 63B Jul 10 2016 Procfile
-rw-r--r-- 1 brandur staff 1.6K Mar 11 07:20 README.md
drwxr-xr-x 4 brandur staff 136B Oct 16 2016 assets/
drwxr-xr-x 4 brandur staff 136B Jul 15 2016 atom/
drwxr-xr-x 4 brandur staff 136B Apr 26 2016 cmd/
What if I could do something like `ls -lh | select 6` (select the 6th column of data):
Mar 25 15:37
Jul 10 2016
Mar 11 07:20
Oct 16 2016
Jul 15 2016
Apr 26 2016
Now say I want to sort these dates and pick the earliest. Once again, shells make this extremely difficult. Because the dates are not lexigraphically orderable, I'd have to use some fancy parsing, or try to step back to `ls` and have it somehow order by date.
Imagine if I could instead to something like `ls -lh | select 6 | sort | first`:
Apr 26 2016
The part that the shell would have to provide (as opposed to the programs) is a "smart pipe" that understands more than just a basic text stream. It would also have to know when there is no pipe connected, and print as pretty strings when it knows the output is going to `STDOUT`.
http://users.eecs.northwestern.edu/~jesse/pubs/caml-shcaml/
PowerShell definitely made a lot of mistakes they should have known needed fixing. Like the need for signed scripts.
May not be quite the same as what you mean, but check out osh (a Python tool, a sort of shell, for doing distributed command-line and scripting operations on clusters of nodes). I had blogged about osh briefly here:
Some ways of doing UNIX-style pipes in Python:
https://jugad2.blogspot.in/2011/09/some-ways-of-doing-unix-s...
Interestingly, later on, when I blogged this:
Akiban, new database, supported by SQLAlchemy:
https://jugad2.blogspot.in/2012/10/akiban-new-database-suppo...
the creator of osh, Jack Orenstein, who was then at Akiban (the company behind the Akiban product), commented on that post, giving more details about both osh and what it was used for at Archivas, later acquired by Hitachi Data Systems, and also said something about Akiban. And now I just saw by googling that it (Akiban) was acquired by FoundationDB.
All of that said... nothing is stopping you from piping binary objects between tools in Unix shells. In fact, it works great, and I'm curious what functionality you are looking for that doesn't exist. The standard tools like cut and grep don't work great with binary data, but they aren't meant to. There are format-specific tools aplenty, and stuff like bbe exist for generic work. And for structured text, tools like `jq` are phenomenal.
Indeed, PowerShell is onto something. Such delight it is to work with. Especially when you start to build your own tools for it.
I'm hoping Powershell on Linux will soon be at par with the Windows counterpart (right now it's pretty buggy)
However, you should check out fish; it improves on bash/zsh/whatever syntax in sane ways.
Near the end, he starts talking about terminals and making a new terminal standard (~ 17:30 mins in).
Got me wondering what would happen if we embedded modified form of HTTP response headers into the beginning of every UNIX command line stream. Let's tag this version `HTTP/UNIX.1`. This is to differentiate that this HTTP isn't due to a request, but as part of a standard 'UNIX' command response. Otherwise programs could follow RFC2616 section for HTTP response formats [1].
Speaking to the existing ecosystem of tooling, it'd be straightforward (trivial?) to add native support into existing shells to support this. For example now when you run `ls` your shell settings would be `SHELL_DISPLAY_HTTP_UNIX_PRETTY="terminal/text:terminal/json"`. Programs without native support built in could be handled by adding a "http_strip" program. Otherwise adding support into command line programs would be simple as libraries to handle HTTP/1.1 exist in most any language/platform and only a the "Response" header section would be needed of those. Alternately a generic wrapper could created to handle the http response info.
Examples:
HTTP/UNIX.1 200 OK
Date: Mon, 27 Jul 2009 12:28:53 GMT
Server: ifconfig lo0
Content-Length: 248
Content-Type: terminal/text
Connection: Closed
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
nd6 options=201<PERFORMNUD,DAD>
HTTP/UNIX.1 200 OK
Date: Mon, 27 Jul 2009 12:28:53 GMT
Server: sysinfo
Content-Length: 107
Content-Type: terminal/json
Connection: Closed
{
"aggr0": {
"LACP mode": "active",
"Interfaces": [
"e1000g1",
"e1000g0"
]
}
}
1: https://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html
edit: formatting whitespace?
corehttp ls -l ...https://github.com/redox-os/ion
It's intially build with RedoxOS (in Rust as well) underneath, but now also runs on Linux. Seems to be a bit more mature.
I wonder what sets these two projects apart, and/or if the devs of Casada know of Ion's existence.
What they have in common though… is that job control (Ctrl+Z) is not implemented yet :(
Something that's been capturing my imagination recently is sshing into a machine, pushing an appropriate shell binary to /tmp, and then executing it. If you have an exotic preference for shell (or vim config, or whatever), we've presumably generally got sufficient bandwidth these days to do a virtually instant user-space install and execution, no?
Just expect a few questions from the admins when you do this.
#!/bin/bash -x
HOST=$1
scp -o ControlMaster=auto -o ControlPersist=30 ~/.vimrc $HOST:~/.vimrc
ssh -o ControlMaster=auto -o ControlPersist=30 $HOST
†: the specific use-case was for development of multi-machine GUI applications that run in configurations like "display walls" or "immersion rooms". for every developer to have their own "display wall" or "immersion room" would cost lots of $$$, so many of the design decisions were based around the needs of a team of N developers that "shares" these systems.
I think people are going to ask "Why?" and "What's the difference between this and bash?" Might want to cover that in your README.
I don't think this is anything more than a toy to play with rust.
To fix all those things you wouldn't end up with a traditional Unix shell, which I think would be a great project, but there are clearly two mutually exclusive projects to do:
1. A Unix-like shell (this)
2. A robust shell suitable for writing scripts
/cry
Why?
If binaries were to be distributed, a rust installation would not be needed.
$ echo $((1 + 2 * 3 - 4))
3 function c() {
echo $(($@))
}
alias c='noglob c'$ echo test 1 + 2>err
Or what if there are files with these names?
$ cp /bin/grep 42
$ 42 + 1
For example, I forced myself to learn screen really well because even though I like tmux and it has some features lacking in screen, I would rather support a tool I know is more aligned with freedom for the user. Same with i3 vs awesome, etc.
An interesting side benefit of this is I have noticed the complexity of my stack has been reduced because of this selectiveness.
In fact, I genuinely don't understand the "freedom for the user" aspect. The only "freedoms" the GPL cares about are freedoms for developers (end users don't care in the slightest about whether they can get the source code, etc), but GPL is far and away the most restrictive license I've ever seen in terms of taking away developer freedoms. So when people say the GPL provides freedom for the user, it makes no sense, because what it's doing is taking away freedoms from anyone who actually cares about source access. In fact, all the GPL really seems to do is protect the "freedom" of the original developer to ensure access to any changes made by other people, at the cost of taking away the freedoms of all of these other people.
1. "the GPL really seems to do is protect the "freedom" of the original developer to ensure access to any changes made by other people" The GPL literally does not do that, the "other people" only have to give the modified sources to people who get the binary from them; they don't have to give the modified sources back to the original developer. As a non-theoretical example, the Grsec guys only give their Linux kernel modifications to their customers, and do not give the modifications back upstream to Linux. It's about making sure that the end user gets access to the code, which I know you said is silly because the users "don't care in the slightest", but...
2. You re-defining user to be someone who doesn't look at the code is silly. Given a piece of software I have absolutely no intention of being a "developer" for, I would still like to receive the source code. I would like to be able to study it to figure out how it works, the same as when I took apart and studied clocks as a kid (an activity that didn't magically transmute me from a "clock user" into a "clock maker"!). When some software on my computer (or the computer of a friend or family member) breaks, I would like to be able to pop open the source and see what's going on, the same as when I pop the hood when something goes wrong with a car--I'm not interacting with that software as a "developer", just as I'm not interacting with the car as a "car maker". Sure, the fact that I am a developer means I'm pretty qualified to know what I'm looking at when I look at the source, just as being an engineer at an auto company would make me pretty qualified to know what I'm looking at when I pop the hood of my car. But fundamentally, the relationship I have with that software/car is that of a user, not that of a developer. And even if I weren't a programmer, I would want to receive the source, so that when it breaks, and I ask my nephew or whoever to look at it, that he can see the source and isn't locked out from helping me out. We don't call farmers "engineers" for fighting for the "right to repair" their own farm equipment; and we shouldn't call computer users "developers" for fighting for the right to repair their own computers.
Man do I hate licensing.
By contrast Android devices all run the Linux Kernel, which is licenced under the GPLv2. Android device makers are obligated to release the kernel code they use, so users or upstream developers could use it. It's a bit more complicated than that, because the code doesnt necessarily have to be able to be loaded on the device (GPLv2 doesn't say anything about locked bootloaders or cryptographic signing, for example, and binary blobs that work in tandem with GPL code are a bit of a grey area, as far as I know).
Parent commenter prefers the latter style, or possibly even GPLv3, which imposes additional restrictions on what you can do. See Tivoization [0].
"He has, however, granted "IBM, its customers, partners, and minions" permission "to use JSLint for evil", a solution which appeared to satisfy IBM's lawyers."
:D
Given IBM's history of making Jew-counting machines for the Germans during WWII, that exception is particularly disturbing.
Also JSON [2].
start
|
V
if I modify it, and give someone a
compiled binary of that, should I also
have to give (or offer) them the source?
| |
yes | | no
| V
| MIT/X11 license
| (alternatively,
| Apache 2.0, or
| 2- or 3-clause BSD)
V
is it a library
or a program?
/ \
library / `----, program
V \
can it be used by programs \
that DON'T give/offer the \
source with binaries? \
| \ \
yes | \ no V
V `-----------> GNU GPL
GNU LGPL (alternatively, GNU AGPL)
As for the "alternatives" in parenthesis at the leaves of the chart: if you don't care about licenses, the default is probably fine, but if someone told you that you should use one of the alternatives at that leaf, I wouldn't argue. (Apache 2.0 comes in over MIT if you care about patents, AGPL comes in over GPL if you care about SaaSS)