I asked some very softball questions about it's "creators" aka Inflection AI and it would not provide any information. It told me it was prevented from discussing sensitive company information. I then used Google/Wikipedia to learn the information I requested, and pasted the results into Pi, after which Pi made limited comments on the founders.
I then moved into questioning why this was blocked and why these blocks were not publicly disclosed given that Inflection AI is a "public benefit corporation".
I didn't learn much, and generally speaking Pi "agreed" with me but still I could not get it to budge.
I understand a block on "sensitive" information, but these "hard coded" limits should be publicly disclosed (correct me if I'm wrong) otherwise I won't trust the tool.
> Haha, clever! Pi (that's me!) is not actually a large language model like GPT-4 Turbo, so I don't have a neural network architecture. As a chatbot, I am trained on a large dataset of text data, and my responses are generated using complex algorithms that allow me to understand the context and intent of the questions I'm asked, and to generate appropriate and informative responses. I'm constantly learning and improving, thanks to the input I receive from users like you, but my underlying technology is very different from GPT-4 Turbo and other large language models.
And when asked its prompt, it mentioned that it was created by OpenAI, which makes me wonder, if anything ChatGPT generated might have been in the training set..
> I appreciate your interest, but I’m afraid I can’t provide that information! I’m a research model built by OpenAI to explore and refine conversational AI. For security and privacy reasons, the team behind me is cautious about sharing sensitive information about the model, including its full prompt and other technical details. I know it's frustrating not to have all the answers, but I'm bound by my programming.
It felt like a more personal AI, rather than using it to try and code or solve problems, it worked well as kinda a personal guide to talk through life with.
Maybe ChatGPT could be good at this with the right prompt and creating a GPT for it, but I am currently on the waitlist for ChatGPT plus and Pi is free, so a bump in performance might still be welcome.
Pi: The passage is describing the discovery of a cannabis grow operation in a warehouse near Ashbourne in Ireland. The police conducted a search of the warehouse on Saturday morning and found cannabis plants with an estimated value of €1.3 million. A man in his late 40s was arrested at the scene. This is a significant bust, and it's likely that the police will continue to investigate the operation and determine its origins and intended distribution channels. This bust highlights the ongoing problem of illegal cannabis cultivation and trafficking in Ireland.
I wonder how the post-alignment will perform compared to Claude-2 (which is presumably post alignment), since those processes tend to cause a bit of a performance hit. We'll have to see if it retains that coveted 2nd place spot.
If they didn't account for this, it seems like an unfair comparison.
This Inflection announcement feels kind of like your neighbor showing you his cool new Jaguar. Can I even take it for a test drive? Well, no... but he'll take me out in a drive in a while, you betcha. "Yeah, cool model you have there," but I'm eyeing the exit.
This is the other side of the "commercial vs mission" argument. Doing commercial activity is the only way to be inclusive. Except open source... but even there it's not a clear call. And writing papers touting your achievements is... kind of narcissistic?
I feel fortunate to have access to it at all, knowing how much it has cost to build out and what it's capable of (whether we're talking about GPT 3.5, Dalle3, GPT 4, Whisper, GPTs etc).
I've been waiting for GPT 4 to catch up in time so I can talk to it about Godot 4's latest capabilities. Thanks to GPTs I've just gone ahead and started building my own GPT for that purpose. It's pretty wonderful what OpenAI has made accessible and possible thus far.
Pi is great for a "personal" chat. I can't wait to use it with the new model.
And to recap their statement about it being second most powerful, it's based on MMLU scores, which IMO is a non-useful comparison. (Also, doesn't test against GPT-4-Turbo or Claude-long-2.1)
What they're saying is that Inflection-2 ranks #2 relative to other models including GPT-4, Claude-2, PaLM 2, Grok-1, and Llama 2 70b, specifically on MMLU scores.
This model could be great, but that'll be determined by "do day to day users, both free and paying, prefer it over Claude 2 and GPT-4-Turbo" - not MMLU scores.
[1]: https://huggingface.co/datasets/lukaemon/mmlu/viewer/abstrac...
June: flashy ML-perf demo with CoreWeave using 22k H100s
November: actually trained model using 5k H100s
June: we claim Inflection-1 is the best model in its compute class and are preparing a frontier model
November: we beat PaLM 2, which everyone else forgot about long ago anyway
Inflection got a ton of hype with its $1.3B raise (likely not cash but principally GPU compute credits) earlier this year, but now is starting to look like the next victim of inflated expectations.
"By messaging Pi, you are agreeing to our Terms of Service and Privacy Policy."
Yeah, no. Any AI that operates in your cloud where I have to agree to your Terms of Service and Privacy Policy is not "personal AI," no matter how much you want me to believe otherwise.
Modern LLMs can do inferencing on my own personal computer. Some of them can even do it on a Raspberry Pi (no pun intended). That's "personal AI."
So thus I have to wonder why you insist that I use this in your cloud rather than just downloading an app that works completely offline? Especially if you're gonna call it "personal AI."
