great for students. better for researchers. best for unhinged scientists who work at the microscopic level.
it's built to be experimented with. many small hacks take 20 seconds or less from ideation to next completed run. very few take over 5 minutes to implement in the right conditions. using the power of science, version 0.7.0 will be coming soon with one less layer (what?!?!)
------------------------------------------------------------
Rest of the bio:
okay so what i really need tbh is someone to take https://github.com/tysam-code/hlb-gpt and make a plot of a trained network using the top n% of the token mass with banded colors representing the lyapunov exponents of the full following sequences (up to about 128 characters or so) mapped onto a poincare disk
bonus if it is clickable so you can advance one token and look at the subgraph for that plot instead
ill handwrite you a very nice thank you note if you do
if you're confused, the above is a plot that would let us explore visually the output space of language models. the lyapunov exponents represent how much information is added, and are calculated by brute force calculating all of the possible sequences of the network to estimate the entropy added per token over each path. this lets us know how chaotic each token branch is. but to save on exploding compute, only calculating the top X% of tokens (by probability mass) keeps the branching a bit lower, and also bonus, we only really see the tokens we sorta care about (maybe this would shoot us in the foot but who knows). because the longer sequences are precalculated, just clicking in a direction would require only calculating one more round of tokens, and you could do a nice animation from wedge to full circle so you're back to where you were on the first step.
super cool and useful i think. and you know you want a nicely written thank you letter. ill even spill peanut butter on the inside if that's in your taste and helps motivate you some. now it's definitely worth it.
