You just said everything “is a file” and then dismissed out of hand a system that takes that abstraction even further!
PowerShell is more UNIX than UNIX!
What I’m saying is that in *nix tooling, things are typically designed to do one thing well. So no, I don’t want my shell to also have to talk MySQL, Postgres, SQL Server, DB2, HTTP, FTP, SMTP…
What's the point of the shell, if not to manage your databases, your REST APIs, files, and mail? Is it something you use for playing games on, or just for fun?
> designed to do one thing well.
Eeexcept that this is not actually true in practice, because the abstraction was set at a level that's too low. Shoving everything into a character (or byte) stream turned out to be a mistake. It means every "one thing" command is actually one thing plus a parser and and encoder. It means that "ps" has a built-in sort command, as do most other UNIX standard utilities, but they all do it differently. This also means that you just "need to know" how to convince each and every command to output machine-readable formats that other tools on the pipeline can pick up safely.
I'll tell you a real troublshooting story, maybe that'll help paint a picture:
I got called out to assist with an issue with a load balancer appliance used in front of a bunch of Linux servers. It was mostly working according to the customer, but their reporting tool was showing that it was sending traffic to the "wrong" services on each server.
The monitoring tool used 'netstat' to track TCP connections, which had a bug in that version of RedHat where it would truncate the last decimal digit of the port number if the address:port combo had the maximum possible number of digits, e.g.: 123.123.123.123:54321 was shown as 123.123.123.123:5432 instead.
Their tool was just ingesting that pretty printed table intended for humans with "aligned" columns, throwing away the whitespace, and putting that into a database!
This gives me the icks, but apparently Just The Way Things Are Done in the UNIX world.
In PowerShell, Get-NetTCPConnection outputs objects, so this kind of error is basically impossible. Downstream tools aren't parsing a text representation of a table or "splitting it into columns", they receive the data pre-parsed with native types and everything.
So for example, this "just works":
Get-NetTCPConnection |
Where-Object State -EQ 'Bound' |
Group-Object LocalPort -NoElement |
Sort-Object Count -Descending -Top 10
Please show me the equivalent using netstat. In case the above was not readable for you, it shows the top ten TCP ports by how many bound connections they have.This kind of thing is a challenge with UNIX tools, and then is fragile forever. Any change to the output format of netstat breaks scripts in fun and create ways. Silently. In production.
I hope you never have to deal with IPv6.
netstat -aln \
| grep ESTABLISHED \
| awk '{print $4}' \
| grep -Po '\:\d+$' \
| grep -Po '\d+' \
| sort \
| uniq -c \
| sort -r \
| head -n 10
Changing the awk field to 5 instead of 4 should get you remote ports instead of local. But yeah, that will be fragile if netstat's output ever changes. That said, even if you're piping objects around, if the output of the thing putting out objects changes, your tool is always at risk of breaking. Yes objects breaking because field order changed is less likely, but what happens if `Get-NetTCPConnection` stops including a `State` field? I guess `Where-Object` might validate it found such a field, but I could also see it reasonably silently ignoring input that doesn't have the field. Depends on whether it defaults to strict or lenient parsing behaviors.It's for communicating with the operating system, launching commands and viewing their output. And some scripting for repetitive workflows. If I'd want a full programming environment, I'd take a Lisp machine or Smalltalk (a programmable programming environment).
Any other systems that want to be interactive should have their own REPL.
> This kind of thing is a challenge with UNIX tools, and then is fragile forever. Any change to the output format of netstat breaks scripts in fun and create ways. Silently. In production.
The thing is if you're using this kind of scripts in production, then not testing it after updating the system, that's on you. In your story, they'd be better of writing a proper program. IMO, scripts are automating workflows (human guided), not for fire and forget process. Bash and the others deals in text because that's all we can see and write. Objects are for programming languages.
To call other programs to do those things. Why on earth would I want my shell to directly manage any of those things?
I think you're forgetting something: *nix tools are built by a community, PowerShell is built by a company. Much like Apple, Microsoft can insist on and guarantee that their internal API is consistent. *nix tooling cannot (nor would it ever try to) do the same.
> It means that "ps" has a built-in sort command, as do most other UNIX standard utilities, but they all do it differently.
I haven't done an exhaustive search, but I doubt that most *nix tooling has a built-in sort. Generally speaking, they're built on the assumption that you'll pipe output as necessary to other tools.
> This also means that you just "need to know" how to convince each and every command to output machine-readable formats that other tools on the pipeline can pick up safely.
No, you don't, because plaintext output is the lingua franca of *nix tooling. If you build a tool intended for public consumption and it _doesn't_ output in plaintext by default, you're doing it wrong.
Here's a one-liner with GNU awk; you can elide the first `printf` if you don't want headers. Similarly, you can change the output formatting however you want. Or, you could skip that altogether, and pipe the output to `column -t` to let it handle alignment.
netstat -nA inet | gawk -F':' 'NR > 2 { split($2, a, / /); pc[a[1]]++ } END { printf "%-5s %s\n", "PORT", "COUNT"; PROCINFO["sorted_in"]="@val_num_desc"; c=0; for(i in pc) if (c++ < 10) { printf "%-5s %-5s\n", i, pc[i] } }'
Example output: PORT COUNT
6808 16
3300 8
6800 6
6802 2
6804 2
6806 2
60190 1
34362 1
34872 1
38716 1
Obviously this is not as immediately straight-forward for the specific task, though if you already know awk, it kind of is: Set the field separator to `:`
Skip the first two lines (because they're informational headers)
Split the 2nd column on space to skip the foreign IP
Store that result in variable `a`
Create and increment array `pc` keyed on the port
When done, do the following
Print a header
Sort numerically, descending
Initialize a counter at 0
For every element in the pc array, until count hits 10, print the value and key
You can also chain together various `grep`, `sort`, and `uniq` calls as a sibling comment did. And if your distro doesn't include GNU awk, then you probably _would_ have to do this.You may look at this and scoff, but really, what is the difference? With yours, I have to learn a bunch of commands, predicates, options, and syntax. With mine, I have to... learn a bunch of commands, predicates, options, and syntax (or just awk ;-)).
> This kind of thing is a challenge with UNIX tools
It's only a challenge if you don't know how to use the tools.
> Any change to the output format of netstat breaks scripts in fun and create ways
The last release of `netstat` was in 2014. *nix tools aren't like JavaScript land; they tend to be extremely stable. Even if they _do_ get releases, if you're using a safe distro in prod (i.e. Debian, RedHat), you're not going to get a surprise update. Finally, the authors and maintainers of such tools are painfully aware that tons of scripts around the world depend on them being consistent, and as such, are highly unlikely to break that.
> Silently. In production.
If you aren't thoroughly testing and validating changes in prod, that's not the fault of the tooling.