Launch HN: Midship (YC S24) – Turn PDFs, docs, and images into usable data

This is an interesting use case! We've heard similar stories from people dealing with pensions. Today we are ready to solve out of the box the extract data from a factsheet into excel or CSV step. Shoot me an email at max@midship.dev!

Have you looked into tools https://www.multion.ai or https://www.browserbase.com.

You should check out Twin

Tabula can do that (open source) https://tabula.technology/

worth a shot - has a free tier: https://www.oscar-idp.com/

yes we can! shoot me an email max@midship.dev

This is something we are hyper focused on. shoot me an email at max@midhsip.dev - you can also try our financial analysis template in our playground for free: https://app.midship.ai/demo.

Hi! In our playground we only included a few templates, but in our app you can create any template you'd like! So yes we can extract from academic papers! shoot me an email: max@midship.dev

We think about accuracy in 2 ways

Firstly as a function of the independent components in our pipeline. For example, we rely on commercial models for document layout and character recognition. We evaluate each of these and select the highest accuracy, then fine-tune where required.

Secondly we evaluate accuracy per customer. This is because however good the individual compenents are, if the model "misinterprets" a single column, every row of data will be wrong in some way. This is more difficult to put a top level number on and something we're still working on scaling on a per-customer basis, but much easier to do when the customer has historic extractions they have done by hand.

We have seen a lot of use cases in finance and are currently working with a few firms. Accuracy is generally their primary requirement. We've been focusing on not just accuracy but also the audit experience which includes confidence scoring. Your right about security and we are currently undergoing a soc 2 audit.

Generally firms are outsourcing the data entry and are already manually auditing with shortcuts like summing values.

Relating to formatted in the table they expected. We extract the data directly into their template format so it is always in the table they expect.

I've been working on some ML solutions similar to this with, and the high accuracy is the challenging part. Specially with "creative" layout documents.

I'm curious as well to see how they are handling accuracy, since I had to build an external agent to validate data.

We saw initial traction with real estate firms extracting property data like rent rolls. But we've also seen traction in other verticals like accounting and intake forms. The original idea was very ambitious and when talking to potential customers they all seemed to be happy with the existing players.

We're excited for foundational models to improve because we hope it will unlock a lot more use cases. Things like analysis after extraction, able to accurately extract extremely complex documents, etc!

Curious - have you compared Gemini against Anthropic and OpenAI’s offerings here? Am needing to do something similar for a one-off task and simply need to choose a model to use.

The reducto guys are great! Their benchmark is not exactly how we would index our product because we extract into a user specified template vs. extracting into markdown (wysiwyg). That being said their eval aligns with our internal findings of commercial OCR offerings.

We are currently focused on business customers :/ but that may change in the coming months!

Send me an email here: max@midship.dev so we can learn more. We're in the process of a SOC 2 audit and should be HIPAA by end of the month!

Yes today we support exports to csv or excel from our web app!

We currently support pdf, docx, most image types (jpeg/jpg, png, heic), and excel.

saving the email as a pdf would work!

And if yes, be specific in answering. Emails are a bear! Emails can have several file types as attachemtns. Including: Other emails, Zip files, in-line images where position matters for context.

A friend seems to be doing it for email - https://dwata.com They are early but is promising

Cofounder here.

Great Q - there is definitely a lot of competition in dev tool offerings but less so in end to end experiences for non technical users.

Some of the things we offer above and beyond dev tools: 1. Schema building to define “what data to extract” 2. A hosted web app to review, audit and export extracted data 3. Integrations into downstream applications like spreadsheets

Outside of those user facing pieces, the biggest engineering effort for us has been in dealing with very complex inputs, like 100+ page PDFs. Just dumping into ChatGPT and asking nicely for the structured data falls over in both obvious (# input/output tokens exceeded) and subtle ways (e.g. missing a row in the middle of the extraction).

Yes there is definitely a boom in document related startups. We see our niche as focusing on non technical users. We have focused on making it easy to build schemas, an audit and review experience, and integrating into downstream applications.

Hey we're on that list! Congrats on the launch Max & team!

I could definitely point to minor differences between all the platforms, but you're right that everyone is tackling the same unstructured data problem.

In general, I think it will be a couple years before anyone really butts heads in the market. The problem space is just that big. I'm constantly blown away by how big the document problem at these mid sized businesses. And most of these companies don't have any engineers on staff. So no attempt has ever been made to fix it.

"accurate document extraction is becoming a commodity with powerful VLMs"

Agree.

The capability is fairly trivial for orgs with decent technical talent. The tech / processes all look similar:

User uploads file --> Azure prebuilt-layout returns .MD --> prompt + .MD + schema set to LLM --> JSON returned. Do whatever you want with it.

Execution is everything. Not to drop a link in someone else’s HN launch but I’m building https://therapy-forms.com and these guys are way ahead of me on UI, polish, and probably overall quality. I do think there’s plenty of slightly different niches here, but even if there were not, execution is everything. Heck it’s likely I’ll wind up as a midship customer, my spare time to fiddle with OCR models is desperately limited and all I want to do is sell to clinics.

Do you know if there any good (pref C++) libraries for extracting data tables from PDFs?

TableFlow co-founder here - I don't want to distract from the Midship launch (congrats!) but did want to add my 2 cents.

We see a ton of industries/use-cases still bogged down by manual workflows that start with data extraction. These are often large companies throwing many people at the issue ($$). The vast majority of these companies lack technical teams required to leverage VLMs directly (or at least the desire to manage their own software). There’s a ton of room for tailored solutions here, and I don't think it's a winner-take-all space.