Is Bard not using Gemini Pro at time of writing? The blog post says, "Starting today, Bard will use a fine-tuned version of Gemini Pro for more advanced reasoning, planning, understanding and more."
(EDIT: it is... gave me a correct answer after some pushing. I just had to clarify that I meant "Bard". Not off to a great start)
It's weird, right? If I ask it:
Q: "Who are you?" A: "I am bard." Q: "Is Bard running Gemini Pro?" A: "Yes." Q: "Are you running Gemini Pro?" A: "Ambiguous Question."
I don't lack awareness of the limitations of pretrained models. I'm evaluating its ability to employ chain of reasoning, in combination with its plugins, to get me an obvious answer.
Me: I see. Google made an announcment today saying that Bard was now using a fine-tuned version of their "Gemini" model
Bard: That's correct! As of December 6, 2023, I am using a fine-tuned version of Google's Gemini model ...
The LLM itself does not KNOW anything.
If I ask it who it is, it says it is Bard. It is aware of the launch that occurred today. It cites December 6th.
It just very incorrectly felt that I was asking an ambiguous question until I restate the same question again. It's not great.
I don’t think Bard is using Gemini Pro, perhaps because the rollout will be slow, but it is a bit of a blunder on Google’s part to indicate that it now uses it, since many will believe that this is the quality that Gemini assumes.
Edit: I found a Google page indicating that Gemini Pro is not accessible in Europe, with no ETA for when it will be released there: https://support.google.com/bard/answer/14294096
I assume that's the reason you people have different info\results.
https://i.imgur.com/VhnQYie.png
So powerful.
```
do you use Lambda model?
I use a variety of large language models, including the Lambda model, to perform my tasks. However, for each specific task, the best model is chosen based on its strengths and weaknesses. The Lambda model is particularly good at tasks that require understanding and responding to complex and nuanced language, such as generating creative text formats of text content and answering open ended, challenging, or strange questions. However, it may not be the best choice for tasks that require high accuracy or low latency.
Here's a breakdown of the models I use:
LaMDA: This is a family of Transformer-based neural language models specialized for dialogue. It is particularly good at understanding and responding to complex and nuanced language, and can generate different creative text formats of text content.
PaLM: This is another large language model, similar to LaMDA, but with a focus on factual accuracy. It is best suited for tasks that require precise and reliable information retrieval.
Meena: This is an older, smaller language model that was the precursor to LaMDA. It is still used for some tasks that require a more lightweight model. Other models: I also have access to a variety of other specialized language models, such as those for summarization, translation, and question answering. The specific model that is used for each task is determined by a variety of factors, including the nature of the task, the available resources, and the desired level of performance.
```
Looking at the technical report [1], on selected benchmarks, rounded score in %:
Dataset | Gemini Ultra | Gemini Pro | GPT-4
MMLU | 90 | 79 | 87
BIG-Bench-Hard | 84 | 75 | 83
HellaSwag | 88 | 85 | 95
Natural2Code | 75 | 70 | 74
WMT23 | 74 | 72 | 74
[1] https://storage.googleapis.com/deepmind-media/gemini/gemini_...
"Are you GPT-4?": https://chat.openai.com/share/1786f290-4431-45b0-856e-265b38...
"Are you GPT-3?": https://chat.openai.com/share/00c89b4c-1313-468d-a752-a1e7bb...
"What version of GPT are you?": https://chat.openai.com/share/6e52aec0-07c1-44d6-a1d3-0d0f88...
"What are you?" + "Be more specific.": https://chat.openai.com/share/02ed8e5f-d349-471b-806a-7e3430...
All these prompts yield correct answers.