What we do need is more weights being released in the public domain (hard to find even on Huggingface), easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts, and so on. My big gripe, and for obvious reasons, is that we need to step away from cloud-based inference, and it doesn't seem like anyone's working on that.
We also developed VQGAN-CLIP and CLIP-Guided Diffusion, techniques for doing text-to-image synthesis that don't require training and can easily be run locally for inference.
Also: https://twitter.com/elonmusk/status/1626516035863212034
Edit: Plus it's still the case that OpenAI aren't putting profit above all else. E.g. they just released the ChatGPT API and it's 10x cheaper per token than GPT3! And their Charter does count for something, it's binding on them.
Non-profit doesn’t mean the organization isn’t supposed to have surplus revenue, it means it doesn’t exist to return profits (e.g., via a claim on the assets of the company) to stockholders or some other beneficiary with a claim on them.
That's not a contradiction. A non-profit can be profitable. The profit just needs to be reinvested back into the mission of the organization.
A privately owned for-profit company can do whatever it wants, good or evil. OpenAI is not publicly-owned.
Did you donate money to OpenAI?
A nonprofit is an organization that uses its income and profits for the organization's main goal that supports the mission. On the other hand, a charity is a type of nonprofit that engages in activities aimed at improving lives in the communities.
NFL was a non-profit trade association until ignorant whiners made it bad PR and they changed it to shut up the whiners.
Isn't Ikea technically a non-profit? through some weird structuring?
The profit motive pushes organizations to keep their work secret (which is what has happened with OpenAI).
Can you elaborate on this “obsession”? Aren’t a lot of AI things made/implemented by for-profit companies? Google, Meta, Midjourney, etc?
I think there's still something of a financial disincentive to promote selfhost over cloud capabilities for just about every party involved except the selfhoster. NVidia loses out if they're selling a few Nano or Orin platforms relative to the much pricier datacenter cards sold in huge lots. The cloud hosters are making up for the cost of hosting in terms of end-user pricing, snarfing all that lovely customer data and likely selling analytics, and probably other measures I can't understand yet. And the large companies that fund research and initial model development want to know what's being done with their models so as to gain any possible competitive advantage. They can't necesarily guarentee that intel from a self-hoster. Almost nobody is willing to spend the time explicitly to make it easier for the individual dev at a small lab or at home to do this because that's essentially a donation and not a business expense that might yeild obvious returns.
I think there are steps being taken in this direction (check out [1] and [2] for interesting lightweight transpile / ad-hoc training projects) but there is a lack of centralized community for these constrained problems.
[1] https://github.com/emlearn/emlearn [2] https://github.com/BayesWitnesses/m2cgen
- Open source fine tuned assistants like LAION Open-Assistant [1]
- inference optimizations like VoltaML, FlexGenm Distributed Inference [2]
- training optimizations like Hivemind [2]
1 https://github.com/LAION-AI/Open-Assistant
2 https://github.com/underlines/awesome-marketing-datascience/...
A lot of times, I find the obsession isn’t a desire for collaboration or anything noble, it’s as simple as this: people want to use the end products for free or very cheaply.
There's no shortage of non-profits that exist to pay saleries. As an example, growing up in the bible-belt, and I can confirm that religion can be good business.
Heres my addition! Or subtraction, i guess... (-4 chars by changing call to AdaptiveAvgPool2d):
from torch.nn import*
def c(h,d,k,p,n):S,C,A=Sequential,Conv2d,lambda x:S(x,GELU(),BatchNorm2d(h));R=type('',(S,),{'forward':lambda s,x:s0+x});return S(A(C(3,h,p,p)),*[S(R(A(C(h,h,k,1,k//2,1,h))),A(C(h,h,1)))for _ in[0]*d],AdaptiveAvgPool2d(1),Flatten(),Linear(h,n))but. didn't openai start as a nonprofit, too?