undefined | Better HN

0 pointsriku_iki1y ago0 comments

> ~=$3400 per single task

report says it is $17 per task, and $6k for whole dataset of 400 tasks.

0 comments

"Note: OpenAI has requested that we not publish the high-compute costs. The amount of compute was roughly 172x the low-compute configuration."

The low compute was $17 per task. Speculate 172*$17 for the high compute is $2,924 per task, so I am also confused on the $3400 number.

bluecoconut1y ago

3400 came from counting pixels on the plot.

Also its $20 on for the o3-low via the table for the semi-private, which x172 is 3440, also coming in close to the 3400 number

bluecoconut1y ago

That's the low-compute mode. In the plot at the top where they score 88%, O3 High (tuned) is ~3.4k

HDThoreaun1y ago

The low compute one did as well as the average person though

ionwake1y ago

sorry to be a noob, but can someone tell me doe sths mena o3 will be unaffordable for a typical user? Will only companies with thousands to spend per query be able to use this?

Sorry for being thick Im just confused how they can turn this into an addordable service?

JohnnyMarcone1y ago

There are likely many efficiency gains that will be made before it's released, and after. Also they showed o3 mini to be better than o1 for less cost in multiple benchmarks, so there're already improvements there at a lower cost than what available.

1 more reply

xrendan1y ago

You're misreading it, there's two different runs, a low and a high compute run.

The number for the high-compute one is ~172x the first one according to the article so ~=$2900

Thorrez1y ago

What's extra confusing is that in the graph the runs are called low compute and high compute. In the table they're called high efficient and low efficiency. So the high and low got swapped.

jhrmnn1y ago

That’s for the low-compute configuration that doesn’t reach human-level performance (not far though)

riku_ikiOP1y ago

I referred on high compute mode. They have table with breakdown here: https://arcprize.org/blog/oai-o3-pub-breakthrough

junipertea1y ago

The table row with 6k figure refers to high efficiency, not high compute mode. From the blog post:

Note: OpenAI has requested that we not publish the high-compute costs. The amount of compute was roughly 172x the low-compute configuration.

gbnwl1y ago

That's "efficiency" high, which actually means less compute. The 87.5% score using low efficiency (more compute) doesn't have cost listed.

bluecoconut1y ago

they use some poor language.

"High Efficiency" is O3 Low "Low Efficiency" is O3 High

They left the "Low efficiency" (O3 High) values as `-` but you can infer them from the plot at the top.

Note the $20 and $17 per task aligns with the X-axis of the O3-low

EVa5I7bHFq9mnYK1y ago

That's high EFFICIENCY. High efficiency = low compute.

j / k navigate · click thread line to collapse

0 comments

binarymax1y ago

"Note: OpenAI has requested that we not publish the high-compute costs. The amount of compute was roughly 172x the low-compute configuration."

The low compute was $17 per task. Speculate 172*$17 for the high compute is $2,924 per task, so I am also confused on the $3400 number.

bluecoconut1y ago

3400 came from counting pixels on the plot.

Also its $20 on for the o3-low via the table for the semi-private, which x172 is 3440, also coming in close to the 3400 number

bluecoconut1y ago

That's the low-compute mode. In the plot at the top where they score 88%, O3 High (tuned) is ~3.4k

HDThoreaun1y ago

The low compute one did as well as the average person though

ionwake1y ago

sorry to be a noob, but can someone tell me doe sths mena o3 will be unaffordable for a typical user? Will only companies with thousands to spend per query be able to use this?

Sorry for being thick Im just confused how they can turn this into an addordable service?

JohnnyMarcone1y ago

1 more reply

xrendan1y ago

You're misreading it, there's two different runs, a low and a high compute run.

The number for the high-compute one is ~172x the first one according to the article so ~=$2900

Thorrez1y ago

What's extra confusing is that in the graph the runs are called low compute and high compute. In the table they're called high efficient and low efficiency. So the high and low got swapped.

jhrmnn1y ago

That’s for the low-compute configuration that doesn’t reach human-level performance (not far though)

riku_ikiOP1y ago

I referred on high compute mode. They have table with breakdown here: https://arcprize.org/blog/oai-o3-pub-breakthrough

junipertea1y ago

The table row with 6k figure refers to high efficiency, not high compute mode. From the blog post:

Note: OpenAI has requested that we not publish the high-compute costs. The amount of compute was roughly 172x the low-compute configuration.

gbnwl1y ago

That's "efficiency" high, which actually means less compute. The 87.5% score using low efficiency (more compute) doesn't have cost listed.

bluecoconut1y ago

they use some poor language.

"High Efficiency" is O3 Low "Low Efficiency" is O3 High

They left the "Low efficiency" (O3 High) values as `-` but you can infer them from the plot at the top.

Note the $20 and $17 per task aligns with the X-axis of the O3-low

EVa5I7bHFq9mnYK1y ago

That's high EFFICIENCY. High efficiency = low compute.

j / k navigate · click thread line to collapse