undefined | Better HN

0 pointsniothiel14d ago0 comments

I've been working on cardcast.gg. It gives you the ability to play Magic: The Gathering with your friends remotely using a webcam.

I got back into MTG back during the pandemic after a long hiatus and Spelltable is what brought me back. My playgroup lamented more features and something tailored to our needs, so curiosity got the better of me and here we are. :)

I've never worked with computer vision before, but I went through a whole journey that started with the classical computer vision techniques and ended with recently migrating to the transformer-based models. Been a really cool adventure!

My playgroup has been consistently preferring it over Spelltable and have been wanting more and more features. I would love for people to try it out and start building a community around it! Discord is on the site.

https://cardcast.gg

0 comments

borealbuilder14d ago

An alternative to spelltable is a great idea! My friend group played extensively a few years back through it but always ran into weird bugs and glitches.

I love that there is no sign-up required! Do you have plans to implement utilizing a mobile phone as a camera? Spelltables implementation leaves much to be desired.

Excited to see where this goes!

niothielOP14d ago

Thanks boreal! I have it on my feature request list. I'm currently using a custom WebRTC + Websocket implementation for connections that I wrote without having this feature in mind, so reworking that will take some effort. Currently focused on client-side inference (runs in the user's browser), followed by continuous tracking (think: Snapshot the entire frame every 5s so all players know what cards are on the board at any given time). Will probably get to that in the next week or two!

If you ever want to follow along or play a game, feel free to hop on Discord (link on site)!

utopiah14d ago

Interesting, seems you are doing the recognition server-side, no reliable way to do it locally using e.g. WASM on the client?

niothielOP14d ago

Actually actively exploring this very topic! I have a feature-flag version where the inference runs via WASM / WebGPU (onnxruntime-web specifically).

My only pause behind rolling this out further is the performance isn't as fast as I'd like (1.5s~ latencies), and the widely varying support for WebGPU / WASM across browsers and OS pairs.

Still testing it out (and learning about ViT performance on various hardware), so hopefully more news on that front soon!

yarbas8914d ago

this is awesome. just to let you know, on your demo page "rats", the swamp is occasionally detected as plains.

niothielOP14d ago

Aha, cool. I'll look into this, thank you!

ramon15614d ago

I had a similar idea, as alternatives were.. eh.

I also had an idea to get a ~12MP camera and set it up on an active game of MTG, just because standing up and having to read other people's deck was bothersome. My eyes are bad, and I end up not reading other people's cards because I feel weird hovering over them when reading.

I would then cast whatever is at the person's deck onto an app so I can manually read the cards. Since my phone is of a similar ratio as a playing card, I figured this might be a nice way to play.

niothielOP14d ago

That's a really cool idea! I'm actually exploring a similar concept right now. On the demo page, I have a "Detect Frame" button which will attempt to identify all of the cards (as well as their bounding boxes). You can hook it up to a webcam to try it out that way right from the demo page.

Today, players have to double click on a card in a webcam stream to identify the specific card, but I'm working on doing full-frame detection on some cadence throughout the course of a match (think 1 scan every 5s so you always have an up-to-date board state, remembering past scans).

What would be super-helpful is to have a few frames from the camera or a video from your intended setup so I could test how well this scenario works. The detection is pretty good overall via webcam, it would probably work even better with 12MP.

I think this would be a really cool application. If you ever want to chat about this, I'd love to talk! Feel free to hop on discord (https://discord.gg/axRtvbsfAU) or DM me! (same username on both)

j / k navigate · click thread line to collapse

0 comments

borealbuilder14d ago

An alternative to spelltable is a great idea! My friend group played extensively a few years back through it but always ran into weird bugs and glitches.

I love that there is no sign-up required! Do you have plans to implement utilizing a mobile phone as a camera? Spelltables implementation leaves much to be desired.

Excited to see where this goes!

niothielOP14d ago

If you ever want to follow along or play a game, feel free to hop on Discord (link on site)!

utopiah14d ago

Interesting, seems you are doing the recognition server-side, no reliable way to do it locally using e.g. WASM on the client?

niothielOP14d ago

Actually actively exploring this very topic! I have a feature-flag version where the inference runs via WASM / WebGPU (onnxruntime-web specifically).

My only pause behind rolling this out further is the performance isn't as fast as I'd like (1.5s~ latencies), and the widely varying support for WebGPU / WASM across browsers and OS pairs.

Still testing it out (and learning about ViT performance on various hardware), so hopefully more news on that front soon!

yarbas8914d ago

this is awesome. just to let you know, on your demo page "rats", the swamp is occasionally detected as plains.

niothielOP14d ago

Aha, cool. I'll look into this, thank you!

ramon15614d ago

I had a similar idea, as alternatives were.. eh.

I would then cast whatever is at the person's deck onto an app so I can manually read the cards. Since my phone is of a similar ratio as a playing card, I figured this might be a nice way to play.

niothielOP14d ago

I think this would be a really cool application. If you ever want to chat about this, I'd love to talk! Feel free to hop on discord (https://discord.gg/axRtvbsfAU) or DM me! (same username on both)

j / k navigate · click thread line to collapse