Sending all my data off to two external services might be a bridge too far for me... Then again, I don't trust Microsoft or Google either.
Would need to be about as fast as ChatGPT seems to be for the conversation piece and perhaps utilize some kind of optimistic response while it does things asynchronously
2. Consumers want 0 latency. Every millisecond counts. Ask google about how they optimize search result latency
The concept sounds great though, I have to imagine there is another product out there, or will be soon, that does not require gmail.
Kudos on innovating around applying LLMs to real-world problems and going beyond the bog-standard approach. It'd be interesting to see a more detailed blog on the technical approach you took!
Here are some notes on our architecture:
- We use LLMs at multiple places to choose what data to pull at each step. We use an additive approach rather than a chaining approach to avoid error propagation. We use GPT3.5-turbo with a bunch of hand-rolled prompts for most of this.
- We’re using InstructorXL + Pinecone running on GCP for vector-based search. We combine this with more traditional search methods backed by Postgres & Elasticsearch, to give the assistant the ability to fast searches of multiple types. We use a x-encoding model trained on open source Q&A data from Bing for scoring & reranking to allow us to combine multiple data sources and determine what makes the most sense to feed into the final prompt.
- We hand-rolled a bunch of rule-based algorithms and heuristics on top of the LLMs to deal with email-specific corner cases and other issues we couldn’t resolve reliably in prompts
- Our user-facing output is generated with GPT4.
This enables a bunch of capabilities that other AI assistants can’t match:
- Way better search — Ask a question and get a succinct direct answer, including finding emails that would be tough for you to find through traditional search (ie. you can’t remember a keyword to use).
- Scheduling – Since we can dynamically pull in multiple types of data, we can access calendar data at the right time to help you schedule meetings.
- Analyze across multiple emails & types of data - The assistant can synthesize answers across multiple emails, your calendar, setting, etc to give you an answer (eg. “What are the top 5 issues that customers emails support about last week”, “what are some meeting times that work for me and the other people on this thread”),
- Write in your voice — the assistant can automatically learn your style and tone based on your sent emails. This means it actually sounds like you and, while it still requires some tweaking occasionally, it’ll save you a lot of time.
- Summarize & translate – it can dynamically access the data you have on your screen right now if you reference it, so it can help you with whatever you’re reading.
A note on privacy: We take privacy very seriously. We’re running everything above on our own GPUs + using OpenAI for final outputs. We aren’t training any models on user data.
We’ve put a lot of thought and effort into this one – I hope you like it – either way, let me know what you think in the comments below!
-Andrew
In trying "write it for me" AI tools, the biggest hurdle is always matching my own tone and style - I'm pretty particular about my writing, and I kind of hate the default tone that ChatGPT and Bard use. It seems like you've put a ton of hard work into making sure that isn't the case here.
And the analysis is really a cherry on top - I've been waiting for a tool that I can ask "what are the 3 most important messages that are unread in my inbox?" Excited to try this out!
The nice thing about this is that it's editable, so the user can customize the style to be the style they want, not necessarily exactly the style they have.
We're also investigating doing per-user model training for more refined voice... not launched yet though (and comes with tradeoffs).
Happy to swap notes -- email me: andrew@shortwave.com
When I clicked, I hoped that it could manage the tedium of dealing with emails:
- Negotiating meeting dates and adding events to my calendar.
- Combining pointless corporate emails into an executive summary, or deleting them as they come.
- Surfacing metadata when appropriate: the phone number of the person I should call, directions to a mentioned place, profiles of mentioned people.
A good executive assistant gets stuff out of the way. They reduce friction and let you focus on work that matters. "You have to be in Frankfurt? Here are your tickets. You are checked in. A taxi will pick you up in the afternoon. Your hotel is booked. I sent the address to your phone". There is so much low-level tedium
To me, this tool feels like ChatGPT with access to my emails. You still have to converse with it, ask it specifically for what you want. Instead of magically sorting the things I don't want to deal with, it handles the thing I should be dealing with: working with other humans.
