If you do use the memo field, how do you deal with user error when they copy-paste the wrong number? Banks have humans guessing and correcting wire transfer instructions. Both you and the user will burn a transaction fee each way if you automatically refund unknown transactions.

It’s been a long time since I’ve messed with wallet APIs, but I’m pretty sure you’re using the wrong API. There should be some way to consume a stream of (row_id, transaction) tuples. That way you’re never polling every address. Instead you deal with relevant events that affect any of your addresses as they happen (which is not very often. A whole blockchain might do 15 transactions/second and you see ~0% of that)

How exactly do you think this differs from a single address getting updated? On a lite client this is nothing, on a full client nothing has changed.

There's a reason nearly every single solution involves an individual address tied to a single public key.

On a Mimblewimble blockchain both sender and receiver need to sign for a transaction. So you could interact with the payer to construct a transaction which you as receiver sign last and then you can publish it yourself.