Ask HN: Unsure which image extraction techniques to utilise

1 pointszemanel7y ago3 comments

For a side project i’m taking a look at how to extract information from Apex Legends (for starters) game play videos. Example screenshot:

https://www.gameprime.org/wp-content/uploads/2019/02/gp-apexlegends-6.jpg

There is a log on top right corner and player status on bottom left corner (health/shields).

Id like to:

- extract <player A username> killed <player B username> with <weapon name> (at time mm:ss) from log, and other information displayed on that log

- from player health extract health/shields percentage (at what time)

Also detect game start/end screens and possibly extract info from them like which characters compose the current player squads.

Apart from OCR, what kind or specific techniques should i learn to implement this ?

3 comments

PaulHoule7y ago

If the text is in a predictable position on the screen, this should be easy.

An image processing pipeline like this usually has a filter up front that drops out the environment and leaves a monochrome image of just the text and associated graphics. The text is white and is usually on a background that provides a lot of contrast since it designed to be readable by you!

zemanelOP7y ago

Yep makes sense. For the OCR part a colleague suggested something like Tesseract or Google Vision but, what about the shapes, like weapon (type) symbol on top right and health/shield bar percentages and etc ?

Maybe i should just get started small with the OCR part

PaulHoule7y ago

The weapons symbols are just very big characters.

The health and shield bar percentages could be measured by counting pixels. For instance you could draw a horizontal line across the bar and see how many pixels are lit up.

j / k navigate · click thread line to collapse

Ask HN: Unsure which image extraction techniques to utilise

1 pointszemanel7y ago3 comments

For a side project i’m taking a look at how to extract information from Apex Legends (for starters) game play videos. Example screenshot:

https://www.gameprime.org/wp-content/uploads/2019/02/gp-apexlegends-6.jpg

There is a log on top right corner and player status on bottom left corner (health/shields).

Id like to:

- extract <player A username> killed <player B username> with <weapon name> (at time mm:ss) from log, and other information displayed on that log

- from player health extract health/shields percentage (at what time)

Also detect game start/end screens and possibly extract info from them like which characters compose the current player squads.

Apart from OCR, what kind or specific techniques should i learn to implement this ?

3 comments

PaulHoule7y ago

If the text is in a predictable position on the screen, this should be easy.

zemanelOP7y ago

Maybe i should just get started small with the OCR part

PaulHoule7y ago

The weapons symbols are just very big characters.

The health and shield bar percentages could be measured by counting pixels. For instance you could draw a horizontal line across the bar and see how many pixels are lit up.

j / k navigate · click thread line to collapse