Show HN: GPT-4-Vision UX audit for your landing page (relaunch) (opens in new tab)

(flawless.is)

15 pointsliorgrossman2y ago15 comments

Last week, I got seriously fascinated by the new GPT-4 Vision API introduced on OpenAI DevDay, so I decided to build a UX Audit tool with this API.

It's pretty simple to use - just go to https://flawless.is and enter your homepage or landing page URL.

I'm using the urlbox API to get a screenshot of your page, then I utilize GPT-4V to analyze the screenshot, find potential usability and conversion issues, suggest fixes, and highlight the relevant areas in the screenshot (although the position is sometimes inaccurate).

Last week, I launched it on Hacker News (https://news.ycombinator.com/item?id=38207892), but I wasn't ready for the 20,000 visitors that came next... Unfortunately, GPT-4V has a rate limit of 100 events daily, so almost no one could use the service - EPIC FAIL.

To manage demand, I now priced the service at $1.99 (still less than a cup of coffee), which should keep the demand at bay, and also cover the costs of GPT-4V, the screenshot API, and hosting.

As a proof of concept, there are some demo audits on the homepage you can click and check out.

Please let me know your thoughts and suggestions on how to improve it!

15 comments

imranhou2y ago

The feedback for Slack is a hit and miss, for example it considers the signup buttons too small, or customer logos too small - is there an issue with its perception or mine? Additionally down below it says button color matches background which isn't true, but other sister buttons may be, which is misleading. At the end of this, I think some parts are useful, but not sure by how much when I have to sift through and ignore the other parts....

Is it possible to adjust the prompts to address some of this? or have a second to run validate the output?

liorgrossmanOP2y ago

All great points, thank you!

I think GPT-4V perception of size might be distorted, since it downsizes everything to be 512px wide before scanning it. I wonder how can this be overcome, I'll try to play with the prompts to instruct it specifically about sizes.

Regarding the "Meet Slack for Enterprise" - you're right, it is weird, looks like it's hallucinating. I'll try and dig deeper.

artur_makly2y ago

Any Senior UX folks here that can review the quality recommendations shown on the audit results posted on this site? A quick glance shows some dubious suggestions. Still better than no UX audit.

potatosalad212y ago

UX here with 10 years experience. It's... not great. Mostly suggestions to fix things that are fine with made up reasons, but some advice that's actively bad. For example:

Problem:The FAQ section requires users to click to view answers, which may reduce engagement

Suggestion:Use an accordion with the first item expanded by default to encourage interaction

liorgrossmanOP2y ago

Yeah, happy to get it roasted by real (human) UX folks for some sanity check.

Broiling31672y ago

It is not clear what aspect of the payment is "one time", please clarify this!

Is it per-domain or per-email?

Rerunning the same domain takes me to checkout again, and I didn't receive an email so I'm lost.

liorgrossmanOP2y ago

Thanks for giving it a try.

One time meaning you are charged $1.99 one time, and you can audit one URL, the one you've chosen on the main page.

After you pay, it takes you to a unique URL with a token. Might be a good idea to send users email with this URL...

There's no concept of logging in or logging out right now, that's why that URL is important. It looks something like this: https://flawless.is/audit?url=https%3A%2F%2Fslack.com%2F&t=1...

If you can't find this URL in your browser history, please ping me at lior at lifehacklabs.org, and I'll restore this URL and send it to you!

Broiling31672y ago

Thank you, I did find that url now.

I’m still confused though, let’s say I “fix” all the problems it gave me, and afterwards if I want to run it again with those fixes, I will need to pay again?

1 more reply

user_named2y ago

I paid for this and tried it on one of our main pages.

It did identify one point that is correct and that I have brought up with the team previously, and we are currently fixing it.

So either points like fine print being too small I think we're not correct and it doesn't understand the use of such a section.

Also the screenshot is of the page not loaded correctly.

liorgrossmanOP2y ago

Thanks for being one of our early adopters!

Glad to hear it did find something that you can correct.

Regarding the points about fine print (and the screenshot not loaded correctly) - would you kindly reply with the URL of the website you audited here, or email it to me to feedback at flawless.is ?

I'd love to take a look and see if these issues could be fixed or improved.

11217mackem2y ago

is the prompt any more sophisticated than? "do a ux audit on this page with the format "problem" and "solution" with tags messaging, cro, usability, marketing, etc"

or rather, significantly more?

this seems super, super simple.

liorgrossmanOP2y ago

Well, it started off this way, but I went through several iterations with the audit prompt and the system prompt to improve the issue detection, the solutions suggested, tag them, reduce hallucinations, provide approximate image coordinates, provide examples to the model, etc. It's currently at over 20 lines of prompting and I guess this will grow over time.

j / k navigate · click thread line to collapse

Show HN: GPT-4-Vision UX audit for your landing page (relaunch) (opens in new tab)

(flawless.is)

15 pointsliorgrossman2y ago15 comments

Last week, I got seriously fascinated by the new GPT-4 Vision API introduced on OpenAI DevDay, so I decided to build a UX Audit tool with this API.

It's pretty simple to use - just go to https://flawless.is and enter your homepage or landing page URL.

To manage demand, I now priced the service at $1.99 (still less than a cup of coffee), which should keep the demand at bay, and also cover the costs of GPT-4V, the screenshot API, and hosting.

As a proof of concept, there are some demo audits on the homepage you can click and check out.

Please let me know your thoughts and suggestions on how to improve it!

15 comments

imranhou2y ago

Is it possible to adjust the prompts to address some of this? or have a second to run validate the output?

liorgrossmanOP2y ago

All great points, thank you!

Regarding the "Meet Slack for Enterprise" - you're right, it is weird, looks like it's hallucinating. I'll try and dig deeper.

artur_makly2y ago

Any Senior UX folks here that can review the quality recommendations shown on the audit results posted on this site? A quick glance shows some dubious suggestions. Still better than no UX audit.

potatosalad212y ago

UX here with 10 years experience. It's... not great. Mostly suggestions to fix things that are fine with made up reasons, but some advice that's actively bad. For example:

Problem:The FAQ section requires users to click to view answers, which may reduce engagement

Suggestion:Use an accordion with the first item expanded by default to encourage interaction

liorgrossmanOP2y ago

Yeah, happy to get it roasted by real (human) UX folks for some sanity check.

Broiling31672y ago

It is not clear what aspect of the payment is "one time", please clarify this!

Is it per-domain or per-email?

Rerunning the same domain takes me to checkout again, and I didn't receive an email so I'm lost.

liorgrossmanOP2y ago

Thanks for giving it a try.

One time meaning you are charged $1.99 one time, and you can audit one URL, the one you've chosen on the main page.

After you pay, it takes you to a unique URL with a token. Might be a good idea to send users email with this URL...

There's no concept of logging in or logging out right now, that's why that URL is important. It looks something like this: https://flawless.is/audit?url=https%3A%2F%2Fslack.com%2F&t=1...

If you can't find this URL in your browser history, please ping me at lior at lifehacklabs.org, and I'll restore this URL and send it to you!

Broiling31672y ago

Thank you, I did find that url now.

I’m still confused though, let’s say I “fix” all the problems it gave me, and afterwards if I want to run it again with those fixes, I will need to pay again?

1 more reply

user_named2y ago

I paid for this and tried it on one of our main pages.

It did identify one point that is correct and that I have brought up with the team previously, and we are currently fixing it.

So either points like fine print being too small I think we're not correct and it doesn't understand the use of such a section.

Also the screenshot is of the page not loaded correctly.

liorgrossmanOP2y ago

Thanks for being one of our early adopters!

Glad to hear it did find something that you can correct.

Regarding the points about fine print (and the screenshot not loaded correctly) - would you kindly reply with the URL of the website you audited here, or email it to me to feedback at flawless.is ?

I'd love to take a look and see if these issues could be fixed or improved.

11217mackem2y ago

is the prompt any more sophisticated than? "do a ux audit on this page with the format "problem" and "solution" with tags messaging, cro, usability, marketing, etc"

or rather, significantly more?

this seems super, super simple.

liorgrossmanOP2y ago

j / k navigate · click thread line to collapse