You've mentioned that we do have BTF now in Linux 4.18. I've tried to find if it was leveraged in bpftrace, but it looks like it isn't yet. I see you've opened an issue in one of the bpftrace forks related to this here: https://github.com/ajor/bpftrace/issues/30.
Do you know if there's been any update on this? Support for BTF/CTF would be amazing.
Alastair added struct support for kprobes yesterday, based on the functionality in bcc (which bpftrace uses). That was the final missing piece, and why I'm posting about it now. See the last example here:
https://github.com/iovisor/bpftrace/blob/master/docs/referen...
It's the same struct support we had in bcc, using linux kernel headers. Some things are available, some are not. Just like in bcc, we sometimes need to define some of a missing struct.
BTF will mean that all structs are available, plus, bpftrace should be able to automatically figure out the arguments so you don't have to cast them. It'll be great, but in the meantime, I am able to do tons of stuff.
Now the next time the hot rage of an audio xrun overcomes me I will have a path forward!
In general audio xruns actually is an area that could use a permanent userspace program.
Musicians need low latency audio processing which Jack2 can provide but the random xrun and the internet flimflam/mummery of how to sort it out is waaaaay beyond most of us (I have less of an excuse but yah its a vibe killer when you are trying to make music).
The goal would be "hey you had a/some xrun and its might be mostly this disk or process or..."
Maybe this needs hard ai but maybe it just needs a compass like tool...
DTrace has really helped me out* on macOS. To be without it is to lose a superpower, so I'm really rooting for bpftrace to bring the same tracing experience to Linux. I've been excitedly following bcc, too. It's just the right tool for HTTP metrics. But my most basic experiments so far have failed, and I don't know how to get support.
* DTrace is a real swiss army-knife. It seems intended for observing production systems, but I've found it indispensible for local development. The mysqld provider is great for tracing queries and perf. Opensnoop answers all sorts of questions. And DTrace saved a C++ project of mine: explaining a deadlock that would've otherwise spelled the end of the software. The most fun I've had is using the pid provider to hook text out of visual novels (in WiNE!).
I'd recommend the #iovisor irc channel on oftc. I haven't stopped by in a bit, but the times I did, I usually was able to get assistance from nice people!
When are we getting a Linux performance engineering book with all the bpf/bcc/bpftrace stuff in it?
2. That’s a staggering amount of NIH, from CTF to syntax. Why?
2. bpftrace is built from the ground-up to operate with eBPF, which does much more than DTrace. I made a list in the post, and that's only happened in the last two years. I understand that if I'm narrowing eBPF to just DTrace vs bpftrace comparisons, you might question why we're doing all this, but consider the bigger picture of eBPF.
1. security: eBPF still supports arrays and and is therefore inherently insecure to be used in the kernel. DTrace didn't support that feature on purpose and not because they couldn't do it. see spectre/meltdown.
2. availability: eBPF probes only work within the kernel. DTrace extends the same interface to the libc level and the user space level. you can trace not only kernel calls, also lib calls and with its high-level language support (Java, python, perl, ruby,...) into userspace. Linux-only eBPF will never be able to support that seemless integration. thanksfully we now have Oracle Linux with proper DTrace support on Linux.
2. That's inaccurate, see the bpftrace docs for uprobes and usdt.
2nd: do you see uprobes supported anywhere? I still can only use DTrace with high level languages. and I wouldn't use this island solution anyway. DTrace works everywhere, now even on windows. eBPF is Linux-only. And should have stayed netfilter only BTW.
