undefined | Better HN

0 pointsjhbadger2y ago0 comments

I don't think so. Consider the following conversation from UndiMix-v2-13B (a local LLM running on my Mac M1 via llama.cpp). It gets it wrong at first, but then gets it right with a little hint. I don't see how the hint could work if it was merely repeating an example it knew.

User: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?

Llama: Hello there! I'd be happy to help you with that question. Let me think... If each of Sally's brothers has two sisters, then together they would have 6 sisters in total (2 x 3 = 6). So, Sally would have 6 sisters as well. Is that the answer you were looking for?

User: no. Think about it. Sally is a sister to the brothers

Llama: Ah, I see what you mean now! In this case, since Sally is one of the siblings, she only counts as one of her own sisters. Therefore, Sally has only one sister - herself. Does that make sense?

User: yes

Llama: Great! I hope that clears up any confusion. Please let me know if you have any other questions or need further assistance.

0 comments

jameshart2y ago

I don’t think ‘Sally has only one sister - herself’ is the right answer.

arnvald2y ago

The initial response had 2 problems - not counting Sally and counting sisters separately for each brother. Your hint pointed out the 1st problem, but then LLM should still provide wrong solution. This makes me think that maybe it would try another answer and explanation no matter what you said. If your reply was just "no, think again" it could also provide the right answer

1024core2y ago

Llama: ... Therefore, Sally has only one sister - herself. Does that make sense?

User: yes

----

Um... no!

nuancebydefault2y ago

She counts as her own sister :D I can be sister of myself?

j / k navigate · click thread line to collapse

0 pointsjhbadger2y ago0 comments

User: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?

User: no. Think about it. Sally is a sister to the brothers

Llama: Ah, I see what you mean now! In this case, since Sally is one of the siblings, she only counts as one of her own sisters. Therefore, Sally has only one sister - herself. Does that make sense?

User: yes

Llama: Great! I hope that clears up any confusion. Please let me know if you have any other questions or need further assistance.

0 comments

jameshart2y ago

I don’t think ‘Sally has only one sister - herself’ is the right answer.

arnvald2y ago

1024core2y ago

Llama: ... Therefore, Sally has only one sister - herself. Does that make sense?

User: yes

----

Um... no!

nuancebydefault2y ago

She counts as her own sister :D I can be sister of myself?

j / k navigate · click thread line to collapse