The main difference, however, is you can choose which LLM to talk to. So far OpenPilot supports GPT 3.5/4 and Google PaLM 2. I'd like to add support for more models (Claude 2, etc) but have been stuck on waiting lists so I haven't tested them.
OpenPilot will give context to your prompts by automatically including relevant files from your workspace. You don't need to have the file open or manually highlight the code in question. It does this using 2 techniques:
1. All the files in your workspace are transformed into embeddings and stored in a DB (Chroma). When you enter a prompt, OpenPilot performs a similarity search to try to find the files you are talking about.
2. It uses a two-step process when communicating to the LLM. It first asks the LLM what files it needs to see in order to answer your prompt, passing along the file paths that matched. This lets the model confirm whether the files are really relevant, so we don't pollute the context with irrelevant info. It also gives it a chance to specify any files that didn't semantically match, which it might be able to infer exist based on your prompt and the structure of your project given the file paths it has seen. Finally, the prompt is resubmitted along with the contents of the files it asked for.
Another feature of OpenPilot that Copilot Chat doesn't (yet) have is the ability to apply diffs. In Copilot Chat, you can insert code at your cursor position. I think OpenPilot's diffs have more potential because you can 1) both add and remove lines, 2) make changes in multiple places, and 3) edit a file you don't have open, or even multiple files.
Unfortunately, GPT 3.5 is pretty bad at formatting diffs, in my experience. This is why I've put it behind a toggle. Some work needs to be done here.
I would appreciate any feedback. If there is interest in this, I'd like to continue iterating on it. An obvious next step would be adding support for a locally-running LLM. Along with better strategies for finding relevant context, and better UX in general. Thanks!
