undefined | Better HN

0 pointsjay_kyburz5y ago0 comments

Ok, so many things to comment on here.

Firstly, there are a lot of other countries than America.

Secondly, re grade progression system, the issue is not what broken system you have now, we should strive to improve the education of our children.

There are clearly a million was that software can make assessment better. Software will eventually be able to read an essay and provide feedback on how well the argument was made, as well as the nuts and bolts of how sentences are put together. Software never gets tired after reading 30 other essays. Never grumpy, never phones it in. Teachers are human.

We study intrinsic vs extrinsic motivation in the games industry as well, but to be clear, _almost all_ motivation in education is extrinsic now. Nobody is doing their math homework because its fun. They do it so they won't "fail".

But I agree that just bolting some lame point system on top of exiting work is mostly a waist of time. The work and rewards need to be tightly coupled and the hard part is to blur the line between intrinsic and extrinsic motivations.

Imagine for example the task is to convince an AI to buy you a pony. You have to write a persuasive essay as to why you should have a pony. The AI can respond to your essay with their own arguments. You can then respond to their augments and pushing the discussion forward. You could do it every day in 10th grade. How good would you be at persuasive writing?

0 comments

throwawaygh5y ago

Yes to everything about the US system having some serious problems and being fairly unique relative to the rest of the world.

> Software will eventually be able to read an essay and provide feedback on how well the argument was made, as well as the nuts and bolts of how sentences are put together... Imagine for example the task is to convince an AI to buy you a pony. You have to write a persuasive essay as to why you should have a pony. The AI can...

No, it can't.

Giving good feedback on essays in high school English or History courses is AI-complete.

"Some type of feedback" is already possible (you can go buy essay graders today -- go try one), but if we go that route we'll mostly just be teaching people how to optimize for a (very broken) grading algorithm.

"'The AI' will then <do thing that we spend years teaching people how to do well>" is exactly how you end up with horribly shitty ML code destroying people's lives.

TBH we're not even there with high school math yet, which is a lot easier than high school essays.

I have a CS PhD, publish in AI conferences, and have taught high school. I am fundamentally absolutely certain that high-quality grading of high school English/History essays is not possible. I'm also certain that we could build "something" that "grades essays" and will have catastrophic negative outcomes for a generation of students if adopted. Students will learn how to write for one really dumb algorithm, and then there will be years of cleanup at universities and workplaces afterward teaching them how to actually write for humans.

> but to be clear, _almost all_ motivation in education is extrinsic now. Nobody is doing their math homework because its fun. They do it so they won't "fail".

My classroom experience suggests that a) this isn't true, and b) there are lots of different types of extrinsic motivation; grades are actually not the most important motivator for most students. People are really complex, and everyone has different motivators.

mncharity5y ago

In addition to human graders, and purely automated graders, there's also the option of hybrid human-machine systems.

Some aspects of the grading might be offloaded from humans, as the range of tasks and cases is often diverse. Or multiple less skilled humans substituted for more expensive ones.

In math for instance, there's no need for a human to see a blank answer, or a clearly correct one.

A system can assess graders as well as students. Might suggest to a professor which subset of gradings by graduate students could use a quality check. Using non-AI-complete statistics like "when this person grades late at night, and takes a long time on an answer, but doesn't write much, and the student is someone flagged as being at risk, then the odds increase that the professor will want to have seen it". Or perhaps particular cases are best handled by a particular grader. Or should always get an independent cross check of two graders, to reduce grading variance and bias.

That a hybrid system might hypothetically support good grading, does not imply that that's the direction which often-unfortunate incentives will drive the tech. Years ago I saw a paper, about the post-WWII defense industry investment in CNC tech. One argument was there were other options, like recording and playing back a skilled machinist, which might have been widely deployed decades earlier. But getting rid of the dependency on skilled workmen was a management objective. Creating cheap automated grading may be far more incentivized that attempting harder and less cheap good grading.

throwawaygh5y ago

This industrial model of human capital production consistently fails. It’s literally the root of everything that is broken about the us k12 education system.

“Taylorism for education but with ‘The AI’” will end the same as the factory model for k12 that gave rise to the pathologies that critics like pg are so quick to point out. ‘The AI’ is just the 2020 version of the control theory metaphors that resulted in disasters in the mid 20th century. Ironic.

jay_kyburzOP5y ago

>high-quality grading of high school English/History essays is not possible.

I didn't mean to imply we could do it now, but somebody will eventually spends hundred of millions of dollars on this and we'll get there.

Machine Learning is not what I mean when I say AI. We need AI's that actually understand text in the same way we do.

All jobs will go to machines eventually.

throwawaygh5y ago

> All jobs will go to machines eventually.

IMO this claim is mostly indistinguishable from claims like "God exists".

lmm5y ago

> Secondly, re grade progression system, the issue is not what broken system you have now, we should strive to improve the education of our children.

Better assessments won't magically lead to better education if the system continues to ignore the results of those assessments. The US education system doesn't need better assessments, it needs to start actually using the assessment results that it already has.

j / k navigate · click thread line to collapse

0 pointsjay_kyburz5y ago0 comments

Ok, so many things to comment on here.

Firstly, there are a lot of other countries than America.

Secondly, re grade progression system, the issue is not what broken system you have now, we should strive to improve the education of our children.

0 comments

throwawaygh5y ago

Yes to everything about the US system having some serious problems and being fairly unique relative to the rest of the world.

No, it can't.

Giving good feedback on essays in high school English or History courses is AI-complete.

"'The AI' will then <do thing that we spend years teaching people how to do well>" is exactly how you end up with horribly shitty ML code destroying people's lives.

TBH we're not even there with high school math yet, which is a lot easier than high school essays.

> but to be clear, _almost all_ motivation in education is extrinsic now. Nobody is doing their math homework because its fun. They do it so they won't "fail".

mncharity5y ago

In addition to human graders, and purely automated graders, there's also the option of hybrid human-machine systems.

Some aspects of the grading might be offloaded from humans, as the range of tasks and cases is often diverse. Or multiple less skilled humans substituted for more expensive ones.

In math for instance, there's no need for a human to see a blank answer, or a clearly correct one.

throwawaygh5y ago

This industrial model of human capital production consistently fails. It’s literally the root of everything that is broken about the us k12 education system.

jay_kyburzOP5y ago

>high-quality grading of high school English/History essays is not possible.

I didn't mean to imply we could do it now, but somebody will eventually spends hundred of millions of dollars on this and we'll get there.

Machine Learning is not what I mean when I say AI. We need AI's that actually understand text in the same way we do.

All jobs will go to machines eventually.

throwawaygh5y ago

> All jobs will go to machines eventually.

IMO this claim is mostly indistinguishable from claims like "God exists".

lmm5y ago

> Secondly, re grade progression system, the issue is not what broken system you have now, we should strive to improve the education of our children.

j / k navigate · click thread line to collapse