Anatoly Karatsuba himself was already dead (he died in 2008). I emailed his daughter Yekaterina (who is also a mathematician, btw) asking for a permission to use their last name. She agreed, but asked to be extra careful about potential implicit affiliations, i.e. to be clear that the content has nothing to do with her father's research.
She also expressed an opinion that in the field of mathematics and computation, at least in Russia, actual researchers are rarely involved in writing textbooks, and the textbooks used in universities often contain conflicting or even wrong information about the authorship of research.
In the end a different name was chosen for the project.
[EDITED to add:] Reading the paper a bit more carefully, the above isn't exactly right. It is about doing matrix multiplication on matrices whose entries are "large" but they're interested in using the idea for hardware design where "large" may mean not-very-large, building (say) 16-bit multiplies out of 8-bit multipliers.
But for software implementations (and I doubt anyone will be making special-case hardware for PAR2) small multiplications are not generally faster than large ones. Maybe you might be able to do more of them in parallel using SIMD, but I bet the extra additions and subtractions and general overhead would outweigh any gains in that setting.
https://github.com/trevorpogue/algebraic-nnhw
The hardware architectures focused on in the paper are systolic array designs, an efficient type of hardware design for matrix multiplication (e.g., the Google TPU uses this), as opposed to more SIMD-like vector architectures like GPUs. It may be possible to extend the proposed KMM algorithm to other types of hardware architectures also in future work. Regarding floating point - this work is applicable for integer matrix multiplication acceleration, it may be possible to extend the concept to floating point data types in future work also.
this is wrong. TPUv4 has tensor cores just like NVIDIA has tensor cores just like AMD has tensor cores. no one uses a systolic array because bandwidth/connectivity is much scarcer than compute. the only people that keep talking about them are academics that don't actually fab/sell chips.
https://cloud.google.com/tpu/docs/v4
https://www.nvidia.com/en-us/data-center/tensor-cores/
https://rocm.docs.amd.com/projects/rocWMMA/en/latest/what-is...
ninja edit: before you gotcha me with "a tensor core is a systolic array!!!" - most tensor cores are actually outerproduct engines not riffle shuffle engines (or whatever you wanna call the topology corresponding to a systolic array).
The issue is memory bandwidth. These techniques indeed help you save multiplier area however the performance is still bandwidth limited - you'd need to be able to feed more data per cycle to increase performance.
One thing the paper doesn't talk about is energy. For DNN, at the network level the energy consumed by integer macs is not that high. Localizing data computation would have a much more impact on energy reduction than optimizing MACs.
(ax + b)(cx + d) = acx^2 + [(a + b)(c + d) - ac - bd]x + bd
holds pretty generally; there isn't any new math or algorithm here that I can see. Their own complexity analysis (eqns. 7 and 8) shows this performs about the same as using Karatsuba multiplication on the entries of the matrices (instead of on the matrices themselves).
If it offers improvements in both, why wouldn't one do it in both?
They even has built-in ADC/DAC for communicatiobmn, sensing and data acquisition (DAQ) [2].
On top of that they have native support for dataflow library and framework including linear algebra that is more suitable for data-intensive in which AI is only one of the intended applications [3].
I was recently asked by my colleague why we still need CPU, since GPU is very dominant now and it seems it's all that we need. I just pointed out GPU is only one of the many accelerators available to the CPU, but a very good one that happened to be very useful and biased towards fancy killer applications namely graphic (game), blockchain (bitcoin) and AI (LLM/ChatGPT).
[1] AMD Adaptive SoC Platform: Versal Architecture [pdf]:
https://www.amd.com/content/dam/amd/en/documents/products/ad...
[2] AMD adds RF-sampling data converters to Versal adaptive SoCs (2024):
https://news.ycombinator.com/item?id=42899304
[3] AIEBLAS: Open-Source Expandable BLAS Implementation for AMD/Xilinx Versal Device:
