undefined | Better HN

0 pointszoogeny2y ago0 comments

It sounds like you are doing something different than what I am suggesting. I am not quizzing the AI looking to determine how well the AI knows the subject. I am using the AI as if it doesn't know the subject at all.

I try to explain a subject to the AI as if I am trying to teach a friend who has no knowledge on the subject. Then I am judging how well I can communicate the idea. In many cases as soon as I try to explain the idea I realize that my own knowledge isn't deep. So I am judging myself, not the quality of the AI.

An analogy is the Rubber Duck method of program debugging. Most programers have been in the situation where when they try to explain an answer to another programmer the answer will suddenly pop into their head. The value isn't the knowledge in the other programmers head, it is in the act of trying to explain. LLMs are patient and unopinionated and make for a good recipient.

0 comments

mmcnickle2y ago

I was intrigued by this idea and decided to give it a go on a subject I have only cursory knowledge of. The conversation is here (https://chat.openai.com/share/50fe7120-a4eb-4c01-80bc-43d8f7...)

You're correct that it's an exercise in introspection, rather than relying on the AI's own knowledge. It was clear to me when trying to write explanations to its questions where my understanding was starting to get fuzzy and hand wavy.

A nice bonus was to get it produce a scorecard of correct, nearly correct and incorrect explanations. I could see these as a good jumping off point for me to do more learning/research. Though I suspect the AI would be less accurate at this in a more niche topic than I chose (refrigeration).

zoogenyOP2y ago

That is an interesting conversation. I like that you had the chat bot follow up with questions forcing you to dig deeper. The summary at the end is a good idea as well.

I've been asking the chatbot to point out my assumptions and to point out weaknesses or errors in my reasoning. That has been pretty useful in uncovering aspects of my thought that aren't as foundational or coherent/consistent as I expected. It also doesn't require the AI to be an expert in order to give me feedback, it just points out unchallenged assertions and errors in logic.

I think this is a potentially powerful use for AIs. They can patiently and politely nitpick and point out subtle errors in our thinking process, providing unbiased critical feedback. That is almost as valuable as having them be perfect oracles that can answer any possible question.

mmcnickle2y ago

I like that idea of having it probe the reasoning as much as the content, I'll definitely add that to the mix next time I try this.

ethanbond2y ago

Wow, that is awesome. Kudos to you and @zoogeny's approach. This seems like a much better way to use this educationally than what I was attempting!

Thank you for sharing. It helps that refrigeration has always sort of blown my mind, so this was an interesting topic as well.

mmcnickle2y ago

> It helps that refrigeration has always sort of blown my mind, so this was an interesting topic as well.

Same! It's been in the front of mind because I've recently been watching Hyperspace Pirate's videos on building a DIY cryocooler (and have been struggling to follow it at a technical level). You might enjoy them too: https://www.youtube.com/watch?v=7QZrHzd3RA8

j / k navigate · click thread line to collapse

0 pointszoogeny2y ago0 comments

0 comments

mmcnickle2y ago

I was intrigued by this idea and decided to give it a go on a subject I have only cursory knowledge of. The conversation is here (https://chat.openai.com/share/50fe7120-a4eb-4c01-80bc-43d8f7...)

zoogenyOP2y ago

That is an interesting conversation. I like that you had the chat bot follow up with questions forcing you to dig deeper. The summary at the end is a good idea as well.

mmcnickle2y ago

I like that idea of having it probe the reasoning as much as the content, I'll definitely add that to the mix next time I try this.

ethanbond2y ago

Wow, that is awesome. Kudos to you and @zoogeny's approach. This seems like a much better way to use this educationally than what I was attempting!

Thank you for sharing. It helps that refrigeration has always sort of blown my mind, so this was an interesting topic as well.

mmcnickle2y ago

> It helps that refrigeration has always sort of blown my mind, so this was an interesting topic as well.

j / k navigate · click thread line to collapse