Ask HN: Best Multi-Modal LLM Desktop Controllers?

The Llama 3.1 release is exciting, the smaller models have better context length and decent fine-tuning performance.

It made me curious, what would it take to build a completely local/offline system that can basically automate desktop workflows. e.g. performing research tasks, RPA etc.

Are there any good ones that already exists and seem like healthy projects? Curious what the community may know.