The drama in trying to convert election PDFs to Spreadsheets (opens in new tab)

(markessien.com)

716 pointsmarkessien3y ago141 comments

141 comments

Nice read. It's important to note

1.The 2020 protesters did not begin vandalizing property, but government infiltrated the protests by burning cars and maiming people.

2. The Obidient movement encompassed multiple sub movements of which a part of the #EndSARS was one of them. A vast majority of Peter Obi's supporters were not #EndSARS activists.

3. Elections in Nigeria are fraught with treacherous behavior so everyone suspects everything. It's important to be very careful with your communication. There is a lot of desperation in the land and so if in a position of information leverage, the responsible thing is to handle the privilege with care and transparency.

crazygringo3y ago

First of all, what a fantastic and inspiring read.

But, I'm left greatly confused -- the article never states whether this changed the result.

It says that halfway through counting Obi was in the lead, but nothing about when finished counting.

And when I look at the spreadsheet, the last row (#3380) appears to be the totals, which lists:

  APC     LP     PDP     NNPP
  149014  85748  329030  8305

Which shows LP (Obi) in third place, just like the official results.

So what point is the article trying to make at the end of the day? Or have I misunderstood the numbers?

sd93y ago

Those are the results for just one state, Adamawa.

However, like you I don't know what the overall results are; I agree that the article could make this clearer.

crazygringo3y ago

Oh thanks for clarifying. Turns out the link to the folder for all the states is here:

https://drive.google.com/drive/folders/173oHgms6wYy5WKz_i3Lh...

But there doesn't appear to be any file that calculates the nationwide totals.

It just seems like such a strange omission but I'm on mobile and can't add up the numbers from across a ton of different files myself.

1 more reply

karagenit3y ago

I totaled up the results from only the "crosschecked" CSV files, here's what I saw:

  APC:  5928825
  LP:   4731127
  PDP:  4555334
  NNPP: 1019045

I tried to manually verify about a dozen rows myself, half were so blurry/low res they were illegible but the ones that were legible were all correct.

And for the "unsure" CSVs:

  APC:  1308067
  LP:    578482
  PDP:   736183
  NNPP:  513245

Also checked about a dozen, and all but one of them were wildly inaccurate so I wouldn't trust these much.

error5033y ago

I collected all the _crosschecked CSVs and got:

  LP       PDP      APC      NNPP
  4731127  4555334  5928825  1019045

Obi seems to make second place here, but far from first.

https://i.imgur.com/UaZbXz6.png

pxc3y ago

There are some conditions under which a presidential election in Nigeria can trigger a runoff election, so that may still matter, right?

1 more reply

jxramos3y ago

probably because it was submitted as evidence and not yet accepted by the courts or wherever things wound up next.

djoldman3y ago

Checking one at random:

https://docs.google.com/spreadsheets/d/1HhV9iJxXTU9liAZPIDoM...

...shows 0s in the first row for all candidate parties. But the corresponding photo shows votes for all three:

https://inec-cvr-cache.s3.eu-west-1.amazonaws.com/cached/res...

I hope it's not a mistake and that there's some arcane law/technicality to explain it.

edit: another mistake on row 21, LP should get 25 but it was credited to NNPP:

https://docs.inecelectionresults.net/elections_prod/1292/sta...

dan-robertson3y ago

Yeah looks weird. When I scrolled to a random part, the numbers seemed to line up. They didn’t say things were entirely correct though. Perhaps the data quality is sufficient for a challenge. Odd that the first rows seem more wrong though.

MontagFTB3y ago

So the bug where the first voting sheet shown to a user was from the same 10% of the photos turned out to be a feature, serving as a CAPTCHA of sorts to weed out the bad actors from the good.

If memory serves, some CAPTCHA techniques include showing two numbers to transcribe, where one’s value is already known. If that number is transcribed incorrectly, then the other number’s result isn’t used, and the CAPTCHA fails. Perhaps a similar technique may have also helped here?

Spare_account3y ago

This approach was part of their strategy:

>Then we started showing some results we knew to the bots - if they entered wrong numbers, we would stop accepting the results.

didgetmaster3y ago

It seems to me that when combating bots or hackers, the wrong approach is to provide immediate negative feedback. Giving an immediate error code lets them know that their current strategy is not working and to try something different.

It seems like a better approach would be to make them think you were accepting the results, when in fact they were going to the bit bucket. Hackers trying to get into your corporate database should be presented with a table full of false (but plausible) data rather than an error. Let them waste time trying to use all those fake SS numbers or account numbers before they figure out they got duped.

3 more replies

dan-robertson3y ago

I think the bug was that your first sheet came from a small set and the people entering bad data would refresh instead of doing the actually random next sheet, so entries for most of the sheets came only from people who had long sessions who were apparently more likely to enter good data.

malborodog3y ago

Can you explain that again differently? I didn’t understand that captcha point. It feels important though.

czx4f4bd3y ago

I think they're referring to the old reCAPTCHA v1 approach.

From https://en.wikipedia.org/wiki/ReCAPTCHA:

> The original iteration of the service was a mass collaboration platform designed for the digitization of books, particularly those that were too illegible to be scanned by computers. The verification prompts utilized pairs of words from scanned pages, with one known word used as a control for verification, and the second used to crowdsource the reading of an uncertain word.

wodenokoto3y ago

Original captcha was built around transcribing text that ocr tools failed at

So I give you two words to transcribe to prove you are human. I know one of them and I want to know the other.

churchill3y ago

Oh, and Mark didn't mention that Bola Ahmed Tinubu was indicted for heroin charges in the US in 2003, forfeited $460k & is just too old to run a democracy this size.

Atiku Abubakar (second candidate) was a former VP and the president he served under (Obasanjo) still insists the dude remains a monument to corruption.

There's been a coordinated campaign at all levels to rig this election massively and we saw voter intimidation, manipulation in broad daylight, and the acquiescence of foreign governments to it all.

churchill3y ago

Proofs:

To explain the $460k he forfeited to the feds for his heroin trafficking indictment [0][1], Tinubu claims to have worked at Deloitte as a consultant & made $850k in pre-tax bonuses a year. Problem is, Deloitte claims he's never worked for them [2] and a director at Deloitte earns $340k, according to Glassdoor [3].

[0]: https://www.bbc.com/news/world-africa-61732548 [1]: https://www.scribd.com/document/345742027/Bola-Tinubu-Heroin [2]: https://pbs.twimg.com/media/FhhgxX2WQAAWOVo?format=jpg [3]: https://www.glassdoor.com/Salary/Deloitte-Director-Salaries-...

JumpCrisscross3y ago

> a director at Deloitte earns $340k, according to Glassdoor

This in no way undermines your post, broadly. But narrowly, these are sales roles. Two people with the same title at Deloitte can make vastly different incomes depending on their production.

bschne3y ago

> is just too old to run a democracy this size

Ahem, somebody tell the U.S. that

churchill3y ago

I meant heroin trafficking

1 more reply

charles_f3y ago

> run a democracy this size.

From the looks of it, if he runs it, it won't be a democracy

lostlogin3y ago

> is just too old to run a democracy this size.

Bola Ahmed Tinubu was born 29 March 1952. He is 70.

Joe Biden was Born November 20, 1942. He is 80.

There are plenty of world leaders that are old and I completely agree with you. Why aren’t there upper age limits? The UK House of Lords, US Congress and US Supreme Court have this problem too.

toyg3y ago

Until recently, old age was supposed to be a Good Thing in leaders, because experience would (or rather should) provide perspective to go beyond hysterical everyday squabbles and actually produce Laws that could survive in the long run. It would also mean such leaders had gone through challenging jobs, and demonstrated themselves worthy of higher office.

churchill3y ago

He claims to be 70 but it's been disputed widely - I don't have the energy to filter signal from noise though.

themitigating3y ago

Proof?

kevviiinn3y ago

Wow what a cliffhanger, it sounds like they have to deal with the courts now. I hope we get an update

https://www.msn.com/en-us/news/world/opposition-files-petiti...

mtrovo3y ago

Is the access to the original photos open? It might be fit for a good Kaggle competition, although maybe a little too late for this current election.

jasonjayr3y ago

From the article, it seems like the rush was to collect enough evidence to file a challenge within the legal timeframe. With a challenge filed, it seems like there is a bit more time to verify claims + other evidence. (I know nothing of the system of government there, but) -- it seems like the prudent thing to do would be for the courts to mandate a neutral verification of each of those paper sheets. (ie, 10 trusted representatives from each party re-key the figures manually).

olabyne3y ago

If you want, you have exactly the same issue to solve with Kenya last year.

The pictures of all of the voting sites are available, but the country went to chaos to pick a winner. It is crazy , because on the lower level (in voting offices), the vote process was respected and the numbers are trustworthy, but the higher you go and the more corruption happens, as each aggregation of data removes trust to the system.

davedx3y ago

Incredible story.

Some more background: https://ng.usembassy.gov/nigerias-2023-elections/

dec0dedab0de3y ago

This would have been a good use for hn style shadow banning. Especially if they didn't publish the current tally, then the original easy to detect bots may have never realized you were on to them

rqtwteye3y ago

I still don't understand how we ended up with PDF as sort of standard to archive data. PDF is already pretty bad for things like manuals but for things like spreadsheets we basically collect the data, then we destroy all the structure by putting it in into POF, and later on we painstakingly try to restore the data from PDF which is often almost impossible to do with accuracy.

It just shows that bad solutions often win.

spacebanana73y ago

I've thought about this and come round to think that the flaws of PDF are actually essential to the success of the document format.

- Non-responsive (compared to HTML). Allows PDFs to serve as a common standard between other document formats with different resizing logic, like Latex and Word.

- Difficultly of network access from code running inside document. Allows PDFs to generally operate offline. Nobody's brave enough to try to write a single page application in a PDF

- Destroying data structure. Allows forward compatibility with anything that can be displayed statically on a screen. New applications can have different ideas about how tables, text or charts should work but if there's static visual output then it'll convert to PDF. Awareness of say, the structure of tables is precisely what makes it so difficult for say google sheets and excel to stay compatible with each other's new table features. If somebody develops a new language with new characters not even in Unicode it'll still work on a PDF

It's also worth noting that most PDF limitations have the characteristic of making things hard but not absolutely impossible. These escape hatches prevent people with hard requirements from actually moving to a new format.

If it were truly impossible to get invoice data from PDFs people might've shifted to a different format for business transactions. But if it's merely difficult some company will come up with an API that works as a good enough extraction solution whose cost is justified by the other compatibility benefits of PDFs, so the ecosystem stays with PDFs.

salawat3y ago

>Difficultly of network access from code running inside document. Allows PDFs to generally operate offline. Nobody's brave enough to try to write a single page application in a PDF.

You can absolutely do so. Most times however, the desire is to embed the latest cut of info into the PDF, then hand it off to somebody who will not have network access.

t. Been there, done that. Had the end product thrown out because of Adobe's licensing terms. I also met one of the people responsible for the tooling I had to suffer through. I have their address, but they apologized, and explained the internal politics at the time; so I've chilled on the whole crushing their genitalia with a large wrench bit.

Long story short: doable, but

  Do Not Follow. 
  This is not a place of honor. 
  No great deed was once commemorated here
  That which remains is repulsive to us, in our time, as it will be in yours.

Seriously. If I could fill this post with spikes and sick faces, I would.
Vvvvvvvvvvvvvvvvvvvvvvvvvvvvv ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
XFA was the dream of madmen, and sadists, that decent men thought they could wrangle some positive utility out of. They were wrong.

The trefoil is not an angel. The weird ring things are symbols for infectious waste.

zo13y ago

Oh but there is:

https://en.wikipedia.org/wiki/Apache_Flex

Not sure if I linked to the right article, but it was basically compiled scripts/code that was embedded into PDF's that could run arbitrary code.

""Apache Flex, formerly Adobe Flex, is a software development kit (SDK) for the development and deployment of cross-platform rich web applications based on the Adobe Flash platform.""

varenc3y ago

For this particular case, the use of PDFs seems irrelevant. Photos were just taken of each polling unit’s results. These photos happened to then be embedded into PDFs for distribution, but the core underlying data is just an image embedded into that PDF. No important data was destroyed when these photos were placed into PDFs.

seunosewa3y ago

They should probably have used JPEGS.

layer83y ago

> how we ended up with PDF as sort of standard to archive data.

I don’t think we really did. They are a standard for archiving typeset page-based documents.

Of course, paper documents used to be standard for archiving data, and some continue to do so in the form of PDF.

In principle, it is possible to integrate all the structure you want in a PDF (using Marked Content, Structure Attributes and User Properties), but for data (as opposed to document structure) you’d need custom software to generate and interpret that.

anigbrowl3y ago

Because PDF shows you a page on screen that will look the same if you print it out, and print layouts have been optimized for reading convenience over centuries. And if you give someone with no technical expertise a pdf file, it's virtually certain that they're going to be able to open it because some kind of viewer is built into most operating systems.

You're totally right about PDF being a massive pain in the butt for any other purpose, but unless you have an alternative that handles the basic use case at least as well and other use cases way better, PDF is here to stay.

chrisfinazzo3y ago

It's old, and sometimes things don't come out right, but this is one way out of that hornet's nest.

https://tabula.technology

There's also a CLI if that is more to your liking. If that doesn't do it, there's always the brute-force option of scripting in your language of choice to pull the data out.

codeulike3y ago

these are just photos embedded in a PDF, which actually isn't that bad an idea, because it lets you scan multiple pages and join them together as a 'document'

(not sure if the documents in OP had several pages, but if you've scanned/photographed a multi-page document, PDF is not that bad of a solution)

SilverCode3y ago

A better option would be to use the TIFF format. You can use it as a container format to store lossless and lossy image formats, and handles multiple images in a single container.

It was the standard for scanners until PDF seemed to dominate the scene.

2 more replies

gus_massa3y ago

I'm a teacher in the first year of the university. During the remote classes in the pandemic, we made almost mandatory to upload the photos of the take homes and questions using camscaner [1].

The student just download the app, and it fix the orientation, rotation, bad light, contrast, and many other horrible things that a jpg may have. In particular the orientation and ordering multiple sheets. Also, Moodle has a little more support for pdf than jpg [2].

I don't know how many three letter agencies are reading the stream, but I'm happy that many three letter agencies operative now have a better formation in algebra and calculus.

[1] https://www.camscanner.com/

[2] It depends on how many optional packages your sysadmin installed.

manv13y ago

Back in the day there were at least two programs competing for the role that PDF fills today that I remember: diskpaper and PDF. Apple also had one for its developer docs, but it was never released commercially, I believe.

PDF provided more fidelity for printing, had better tooling (it was by Adobe after all), it was cross-platform, could be displayed on the desktop, so it won. The reader was cross-platform so end-users didn't have to mess with installing plugins for various image types. And because everyone in the document creation division(1) used Postscript to print, printing to PDF was super-easy. And at some point everyone had a postscript printer driver on their machine, so printing to PDF because super-easy as well.

It's not an archiving tool, but people use it for archiving...just like the way a spreadsheet isn't a project management tool, but millions of people use it for project management.

At this point the network effects for the PDF file format would make it difficult to replace. With PDF you can practically guarantee(2) that the file will look the same on any device.

(1) This was more true back then than today, probably (2) assuming that you embedded the fonts, and that the reader doesn't suck.

What's funny is I don't think Adobe really makes any money off of PDF; it's an accidental de-facto standard.

lostlogin3y ago

> PDF provided more fidelity for printing, had better tooling

This might have been true once, but using Acrobat now is so painful. Of all the apps that work, Apples Preview is my editor of choice and when I’m on Windows I really miss it.

1 more reply

davedx3y ago

It depends. There are PDFs with rasterized images of text (like in the article, when it’s a scan or photo of a document), then there are PDFs with vector positioned text runs (when it’s usually a result of some digital process). The latter are way easier to process than the former.

andrewio3y ago

Try https://parsio.io.

It converts PDFs into a structured JSON format that you can export anywhere using a Zapier or Make automation:

redman253y ago

This might be a sensitive question but I wonder if something like this would work in the United States? With all of the fears of election interference why not trust but verify?

charles_f3y ago

Would you trust the recount? I mean, the only way to engage the number of people you need to do that kind of recount is by having them very pissed, so most likely feeling like their party was wronged and therefore the thing is partisan by essence. If you're on the winning party you wouldn't trust the numbers the others give you anyhow, so what's the point

pjc503y ago

Genuinely the US would do better if it had paper elections with a handcount with observers. The system works in the UK just fine. Unfortunately, there's a category of people in both the US and Nigeria who use "election interference" to mean "accurately counting the votes".

bunabhucan3y ago

The distrust is not based on evidence. Actual election fraud is incredibly rare in the US, typically things like someone owning property in two states and voting the state ballot in one and federal in the other. Getting two ballots is legal but using both is jail time. Typical solo offender is a conservative white male.

harvey93y ago

This is some compelling writing. I know this has real life implications for real people so I hope it's not in poor taste to say it would make a good movie.

cwkoss3y ago

I agree, but still needs an ending! Will this be a story of triumph or tragedy?

davedx3y ago

More background. OP is an impressive entrepreneur! Massive kudos. https://markessien.com/projects/hotels-ng/

tr33house3y ago

I'd tried something like this with the Kenyan election but our setup was to use OCR (google cloud) -> text -> parse -> sqlite

We started late so the results were out when we finished but I think it'll be a good idea to develop software that can parse the PDF results and display them faster than the electoral bodies can. In Kenya, and Nigeria, the delays cause a lot of anxiety

hoseja3y ago

Silly, you don't malcount the actual votes, you brainwash the population and pervert the process until they vote the way you want them to, like in the advanced first world democracies.

avodonosov3y ago

That's not the worst case, if wise elite brainwashes (manufactures consent of) the population.

Worse is when the elite is not so wise (sometimes plainly crazy), or the elite loses control to crazy people, adversaries. Or self-induced mass hysteria of the population.

The direct "democracy" that very soon will inevitably be enabled by technology, poses great dangers in the situation where masses are so easily manupulateable, and their collective intelligence seems not raising above individual level, but degrading below it for some reason. Violent chaos, lynch courts, etc.

mattlutze3y ago

This was thrilling.

Sometimes, one person's bug is another person's feature :)

londons_explore3y ago

Isn't things like this the reason that the UN provide election observers?

By spot checking just a random 100 votes are correctly tallied, you can be pretty sure the outcome of the election is legit in a > 10M voter country.

Someone3y ago

> By spot checking just a random 100 votes are correctly tallied

How do you do that? I think the only error you could detect is when the tally has fewer votes for a party than what’s in that sample. If so, a fraudster could report 100 votes for every party, and add the remaining to whatever party they want to win.

londons_explore3y ago

You have to design the election system with this in mind.

One such design would be for every vote to have a unique id. When announcing the results, you also publish a list of which vote ids were tallied for which candidate.

Then you have 100 random ids, and the checkers watch those votes all the way from the voter casting them to the final tally.

throwaway815233y ago

I've done stuff like this semi manually. Use pdftotext to get the text tables out of the pdf, eyeball it and massage with emacs keyboard macros, and in some cases python scripts. It's not that big a deal but it is somewhat ad hoc.

I know that OCR software is able to read stuff like magazine articles and figure out column layout, embedded charts, etc. It's weird if is nothing to do that with a pdf. Maybe I'll look around or see if I can hack up something.

infinityio3y ago

unfortunately in this case the text content was handwritten, not computer-generated

hardlianotion3y ago

That is a great job - well done from a grateful Nigerian.

bundie3y ago

I did not know that Nigerians used Hacker News :-D Most people I encounter on this site are oyinbos.

hardlianotion3y ago

We are everywhere and cannot be avoided.

1 more reply

mmmuhd3y ago

Elupee 75, To be frank, you did a great job and i am proud of someone from my country pulling this off, but the bitter truth is President Elect Bola Ahmed Tinibu won this election. Peter Obi's youth support is predominantly in the south, and Christian majority parts of the country, he clearly lack support in the Muslim north, where I am from. I voted for Kwankwaso though.

bmsleight_3y ago

Can you expand on " he clearly lack support". Bonus points for facts over opinions.

mmmuhd3y ago

Clearly means even his Vice Presidential Candidate could not win his own polling unit, polling unit, not ward, not Local Government, not State.

https://punchng.com/nigeriaelections2023-datti-loses-polling...

2 more replies

YeGoblynQueenne3y ago

>> We had a brainstorming meeting, and decided to try a new approach. We would simply ask the Obidients to help us do the conversion. If hundreds of Obidients did the transcription, it would go fast.

What would guarantee that the Obidients would not, in turn, try to inflate the score of the Labor candidate?

munchler3y ago

They planned to transcribe each PDF multiple times in order to validate the results.

seventytwo3y ago

Wow, this was a fantastic read!

I have no idea what’s going on in Nigeria, but I hope the truth (whatever it is) will prevail!

nivenkos3y ago

This is a great example of why electronic voting is important and can help secure democracy.

logifail3y ago

> This is a great example of why electronic voting is important and can help secure democracy.

If those in power are against change, I wouldn't want to have to put my trust in electronic voting if I was hoping for change.

I was left with the impression that it is the paper records in this story that led to the unravelling of an attempt to forge the results.

Long live paper ballots.

SkeuomorphicBee3y ago

> I was left with the impression that it is the paper records in this story that led to the unravelling of an attempt to forge the results.

The manual tallying of paper records is what lead to the attempt to forge the results in the first place. If the results were electronically tallied to generate an official result, then they wouldn't need to recount the whole election to verify the result, just doing a statistically significant random sampling of the polls to recount would be enough.

2 more replies

TazeTSchnitzel3y ago

On the contrary, eletronic voting doesn't create the paper trail necessary to dig up frauds like this. You can simply program or hack the system to report any vote total you want.

SkeuomorphicBee3y ago

First of all, hacking the electronic system is much much harder than hacking the paper process. In the case at hand the paper tallying process was the one hacked.

And second, electronic systems can create a paper trail, just make the electronic machine spit out a paper receipt. Then you have the best of both worlds, you can have instant electronic totals, and then do some random sampling recounts of the receipts to validate the result.

2 more replies

gdelfino013y ago

You introduce technology to increase transparency and fight corruption. You increase transparency by having video recordings of human counting votes linked to the electronic record of the totals.

When you introduce technology to eliminate manual counting and paper trails, then transparency is eliminated and you give a green light to fraud, corruption, very juicy contracts and death.

cwkoss3y ago

Wouldn't electronic voting just create a means for the ruling party to deliver the result without releasing evidence of vote tampering?

I don't understand what you think electronic voting solves...

jxramos3y ago

> Then ominously, on the 20th of October of 2020 some people drove there in unmarked cars and removed all the Cameras installed at the tollgate.

They at least capture some photos of the equipment. I wonder if anyone communicated with the individuals.

pjc503y ago

Striking reminder of how big the world is that while I had heard of #EndSARS, I hadn't realised the scale of the political violence in Nigeria nor that it had its own Bloody Sunday-scale massacre.

londons_explore3y ago

The votes surprise me... In many regions one party gets 90+% of the vote.

Assuming the numbers are correct, then it suggests that most people are easily swayed by their local peers.

Is that common in say the USA?

crazygringo3y ago

> it suggests that most people are easily swayed by their local peers.

That feels like a particularly uncharitable interpretation to me.

I think it's more along the lines of that parties and their policies have very different impacts on different regions. So it makes sense to vote on what is beneficial to your region, and a lot of people will agree on that.

So it's not about susceptibility to being "swayed", but genuine policy affecting regions differently.

muyuu3y ago

It happens in the US too. Tribalism and ideological clustering are so similar, they are being used interchangeably these days. But in some traditional countries there are literal clans and tribes voting in blocks.

anigbrowl3y ago

Yep, bloc voting can be habitual or strategic. There's a town in Northern California where the majority of the seats on the council is held by people who all happen to attend the same megachurch.

mmmuhd3y ago

Exactly! and this mostly happened in the regions where the OP's preferred candidate won. This is clear scam.

blntechie3y ago

What was the final result numbers from the transcription?

toyg3y ago

They're probably going to be similar to the 14k sample he tweeted: a solidified Labour Party getting 50-55% of the votes, and the establishment candidates splitting the rest.

thread_id3y ago

Fantastic story. What an excellent example of democratization from technology. And also a perfect example of how the blade cuts both ways. Digital warriors battling it out in real time and the stakes are enormous. Great respect for Mark and his ingenuity and adaptive responses!!!!

pxc3y ago

I'm impressed by the courage of the protesters here, and the tenacity of the youth voters.

I hope they get a clear answer and a fair count, and whether they win this time or not, a real shot at cracking up their corrupt, two-party system.

neves3y ago

Is it true that USA does not have a open data law to make everybody publish in CSV?

JumpCrisscross3y ago

> Is it true that USA does not have an open data law to make everybody publish in CSV?

American elections are de-centralised. Each state comes up with its methods. In some, each county. (I'm not sure how publishing a CSV of vote totals would help.)

SergeAx3y ago

Pdf is a very unfortunate format. It is proprietary, it is paper-oriented, its almost single goal is to keep precise printing layout. But for the last 30 years world didn't come up with anything that could compete.

segfaultbuserr3y ago

PDF isn't the actual problem in this particular case. The documents here are photographs taken at different camera angles, embedded in PDFs.

jxramos3y ago

I was going to say, using alt drag to select vertical columns is usually how I extract useable tables out from pdfs with embedded tables.

orf3y ago

Fantastic story! Did the results get used in a claim?

vincheezel3y ago

I hope for (but do not expect) a positive outcome

jgtrosh3y ago

The context should be dated to 2020, not 2023 Edit: it was now corrected, no need to downvote

Great story! Looking forward to some follow up

public_defender3y ago

I don't understand. The article says the SARS protests started in 2020 and the election was in 2023. This seems correct.

jgtrosh3y ago

Yes, it was now corrected

dejongh3y ago

Wow. Wild story. Thanks for sharing. Cool twist that a bug ended up identifying the bad guys.

clipper_janosch3y ago

What an exceptional story. You are a legend.

prhrb3y ago

What a scam by the ruling political party

roschdal3y ago

The people who cast the votes don't decide an election, the people who count the votes do. - Stalin.

pxc3y ago

In case the downright cartoonish character of this quotation made anyone else wonder if it were fake...

that quotation is, indeed, fake: https://web.archive.org/web/20220128105324/https://www.polit...

snvzz3y ago

Not providing CSV is at the level of criminal negligence.

churchill3y ago

pxc3y ago

@dang are these '-' comments an attempt to evade showdead?

mmmuhd3y ago

David Hundeyin is a deceitful, lying criminal, so don't bring his "Content" as any kind of evidence.

https://www.icirnigeria.org/controversy-as-oxford-terminates...

1 more reply

j / k navigate · click thread line to collapse

141 comments

OoTheNigerian3y ago

Nice read. It's important to note

1.The 2020 protesters did not begin vandalizing property, but government infiltrated the protests by burning cars and maiming people.

2. The Obidient movement encompassed multiple sub movements of which a part of the #EndSARS was one of them. A vast majority of Peter Obi's supporters were not #EndSARS activists.

crazygringo3y ago

First of all, what a fantastic and inspiring read.

But, I'm left greatly confused -- the article never states whether this changed the result.

It says that halfway through counting Obi was in the lead, but nothing about when finished counting.

And when I look at the spreadsheet, the last row (#3380) appears to be the totals, which lists:

  APC     LP     PDP     NNPP
  149014  85748  329030  8305

Which shows LP (Obi) in third place, just like the official results.

So what point is the article trying to make at the end of the day? Or have I misunderstood the numbers?

sd93y ago

Those are the results for just one state, Adamawa.

However, like you I don't know what the overall results are; I agree that the article could make this clearer.

crazygringo3y ago

Oh thanks for clarifying. Turns out the link to the folder for all the states is here:

https://drive.google.com/drive/folders/173oHgms6wYy5WKz_i3Lh...

But there doesn't appear to be any file that calculates the nationwide totals.

It just seems like such a strange omission but I'm on mobile and can't add up the numbers from across a ton of different files myself.

1 more reply

karagenit3y ago

I totaled up the results from only the "crosschecked" CSV files, here's what I saw:

  APC:  5928825
  LP:   4731127
  PDP:  4555334
  NNPP: 1019045

I tried to manually verify about a dozen rows myself, half were so blurry/low res they were illegible but the ones that were legible were all correct.

And for the "unsure" CSVs:

  APC:  1308067
  LP:    578482
  PDP:   736183
  NNPP:  513245

Also checked about a dozen, and all but one of them were wildly inaccurate so I wouldn't trust these much.

error5033y ago

I collected all the _crosschecked CSVs and got:

  LP       PDP      APC      NNPP
  4731127  4555334  5928825  1019045

Obi seems to make second place here, but far from first.

https://i.imgur.com/UaZbXz6.png

pxc3y ago

There are some conditions under which a presidential election in Nigeria can trigger a runoff election, so that may still matter, right?

1 more reply

jxramos3y ago

probably because it was submitted as evidence and not yet accepted by the courts or wherever things wound up next.

djoldman3y ago

Checking one at random:

https://docs.google.com/spreadsheets/d/1HhV9iJxXTU9liAZPIDoM...

...shows 0s in the first row for all candidate parties. But the corresponding photo shows votes for all three:

https://inec-cvr-cache.s3.eu-west-1.amazonaws.com/cached/res...

I hope it's not a mistake and that there's some arcane law/technicality to explain it.

edit: another mistake on row 21, LP should get 25 but it was credited to NNPP:

https://docs.inecelectionresults.net/elections_prod/1292/sta...

dan-robertson3y ago

MontagFTB3y ago

So the bug where the first voting sheet shown to a user was from the same 10% of the photos turned out to be a feature, serving as a CAPTCHA of sorts to weed out the bad actors from the good.

Spare_account3y ago

This approach was part of their strategy:

>Then we started showing some results we knew to the bots - if they entered wrong numbers, we would stop accepting the results.

didgetmaster3y ago

3 more replies

dan-robertson3y ago

malborodog3y ago

Can you explain that again differently? I didn’t understand that captcha point. It feels important though.

czx4f4bd3y ago

I think they're referring to the old reCAPTCHA v1 approach.

From https://en.wikipedia.org/wiki/ReCAPTCHA:

wodenokoto3y ago

Original captcha was built around transcribing text that ocr tools failed at

So I give you two words to transcribe to prove you are human. I know one of them and I want to know the other.

churchill3y ago

Oh, and Mark didn't mention that Bola Ahmed Tinubu was indicted for heroin charges in the US in 2003, forfeited $460k & is just too old to run a democracy this size.

Atiku Abubakar (second candidate) was a former VP and the president he served under (Obasanjo) still insists the dude remains a monument to corruption.

There's been a coordinated campaign at all levels to rig this election massively and we saw voter intimidation, manipulation in broad daylight, and the acquiescence of foreign governments to it all.

churchill3y ago

Proofs:

JumpCrisscross3y ago

> a director at Deloitte earns $340k, according to Glassdoor

This in no way undermines your post, broadly. But narrowly, these are sales roles. Two people with the same title at Deloitte can make vastly different incomes depending on their production.

bschne3y ago

> is just too old to run a democracy this size

Ahem, somebody tell the U.S. that

churchill3y ago

I meant heroin trafficking

1 more reply

charles_f3y ago

> run a democracy this size.

From the looks of it, if he runs it, it won't be a democracy

lostlogin3y ago

> is just too old to run a democracy this size.

Bola Ahmed Tinubu was born 29 March 1952. He is 70.

Joe Biden was Born November 20, 1942. He is 80.

There are plenty of world leaders that are old and I completely agree with you. Why aren’t there upper age limits? The UK House of Lords, US Congress and US Supreme Court have this problem too.

toyg3y ago

churchill3y ago

He claims to be 70 but it's been disputed widely - I don't have the energy to filter signal from noise though.

themitigating3y ago

Proof?

kevviiinn3y ago

Wow what a cliffhanger, it sounds like they have to deal with the courts now. I hope we get an update

https://www.msn.com/en-us/news/world/opposition-files-petiti...

mtrovo3y ago

Is the access to the original photos open? It might be fit for a good Kaggle competition, although maybe a little too late for this current election.

jasonjayr3y ago

olabyne3y ago

If you want, you have exactly the same issue to solve with Kenya last year.

davedx3y ago

Incredible story.

Some more background: https://ng.usembassy.gov/nigerias-2023-elections/

dec0dedab0de3y ago

This would have been a good use for hn style shadow banning. Especially if they didn't publish the current tally, then the original easy to detect bots may have never realized you were on to them

rqtwteye3y ago

It just shows that bad solutions often win.

spacebanana73y ago

I've thought about this and come round to think that the flaws of PDF are actually essential to the success of the document format.

- Non-responsive (compared to HTML). Allows PDFs to serve as a common standard between other document formats with different resizing logic, like Latex and Word.

- Difficultly of network access from code running inside document. Allows PDFs to generally operate offline. Nobody's brave enough to try to write a single page application in a PDF

salawat3y ago

>Difficultly of network access from code running inside document. Allows PDFs to generally operate offline. Nobody's brave enough to try to write a single page application in a PDF.

You can absolutely do so. Most times however, the desire is to embed the latest cut of info into the PDF, then hand it off to somebody who will not have network access.

Long story short: doable, but

  Do Not Follow. 
  This is not a place of honor. 
  No great deed was once commemorated here
  That which remains is repulsive to us, in our time, as it will be in yours.

The trefoil is not an angel. The weird ring things are symbols for infectious waste.

zo13y ago

Oh but there is:

https://en.wikipedia.org/wiki/Apache_Flex

Not sure if I linked to the right article, but it was basically compiled scripts/code that was embedded into PDF's that could run arbitrary code.

""Apache Flex, formerly Adobe Flex, is a software development kit (SDK) for the development and deployment of cross-platform rich web applications based on the Adobe Flash platform.""

varenc3y ago

seunosewa3y ago

They should probably have used JPEGS.

layer83y ago

> how we ended up with PDF as sort of standard to archive data.

I don’t think we really did. They are a standard for archiving typeset page-based documents.

Of course, paper documents used to be standard for archiving data, and some continue to do so in the form of PDF.

anigbrowl3y ago

chrisfinazzo3y ago

It's old, and sometimes things don't come out right, but this is one way out of that hornet's nest.

https://tabula.technology

There's also a CLI if that is more to your liking. If that doesn't do it, there's always the brute-force option of scripting in your language of choice to pull the data out.

codeulike3y ago

these are just photos embedded in a PDF, which actually isn't that bad an idea, because it lets you scan multiple pages and join them together as a 'document'

(not sure if the documents in OP had several pages, but if you've scanned/photographed a multi-page document, PDF is not that bad of a solution)

SilverCode3y ago

A better option would be to use the TIFF format. You can use it as a container format to store lossless and lossy image formats, and handles multiple images in a single container.

It was the standard for scanners until PDF seemed to dominate the scene.

2 more replies

gus_massa3y ago

I'm a teacher in the first year of the university. During the remote classes in the pandemic, we made almost mandatory to upload the photos of the take homes and questions using camscaner [1].

I don't know how many three letter agencies are reading the stream, but I'm happy that many three letter agencies operative now have a better formation in algebra and calculus.

[1] https://www.camscanner.com/

[2] It depends on how many optional packages your sysadmin installed.

manv13y ago

It's not an archiving tool, but people use it for archiving...just like the way a spreadsheet isn't a project management tool, but millions of people use it for project management.

At this point the network effects for the PDF file format would make it difficult to replace. With PDF you can practically guarantee(2) that the file will look the same on any device.

(1) This was more true back then than today, probably (2) assuming that you embedded the fonts, and that the reader doesn't suck.

What's funny is I don't think Adobe really makes any money off of PDF; it's an accidental de-facto standard.

lostlogin3y ago

> PDF provided more fidelity for printing, had better tooling

This might have been true once, but using Acrobat now is so painful. Of all the apps that work, Apples Preview is my editor of choice and when I’m on Windows I really miss it.

1 more reply

davedx3y ago

andrewio3y ago

Try https://parsio.io.

It converts PDFs into a structured JSON format that you can export anywhere using a Zapier or Make automation:

redman253y ago

This might be a sensitive question but I wonder if something like this would work in the United States? With all of the fears of election interference why not trust but verify?

charles_f3y ago

pjc503y ago

bunabhucan3y ago

harvey93y ago

This is some compelling writing. I know this has real life implications for real people so I hope it's not in poor taste to say it would make a good movie.

cwkoss3y ago

I agree, but still needs an ending! Will this be a story of triumph or tragedy?

davedx3y ago

More background. OP is an impressive entrepreneur! Massive kudos. https://markessien.com/projects/hotels-ng/

tr33house3y ago

I'd tried something like this with the Kenyan election but our setup was to use OCR (google cloud) -> text -> parse -> sqlite

hoseja3y ago

Silly, you don't malcount the actual votes, you brainwash the population and pervert the process until they vote the way you want them to, like in the advanced first world democracies.

avodonosov3y ago

That's not the worst case, if wise elite brainwashes (manufactures consent of) the population.

Worse is when the elite is not so wise (sometimes plainly crazy), or the elite loses control to crazy people, adversaries. Or self-induced mass hysteria of the population.

mattlutze3y ago

This was thrilling.

Sometimes, one person's bug is another person's feature :)

londons_explore3y ago

Isn't things like this the reason that the UN provide election observers?

By spot checking just a random 100 votes are correctly tallied, you can be pretty sure the outcome of the election is legit in a > 10M voter country.

Someone3y ago

> By spot checking just a random 100 votes are correctly tallied

londons_explore3y ago

You have to design the election system with this in mind.

One such design would be for every vote to have a unique id. When announcing the results, you also publish a list of which vote ids were tallied for which candidate.

Then you have 100 random ids, and the checkers watch those votes all the way from the voter casting them to the final tally.

throwaway815233y ago

infinityio3y ago

unfortunately in this case the text content was handwritten, not computer-generated

hardlianotion3y ago

That is a great job - well done from a grateful Nigerian.

bundie3y ago

I did not know that Nigerians used Hacker News :-D Most people I encounter on this site are oyinbos.

hardlianotion3y ago

We are everywhere and cannot be avoided.

1 more reply

mmmuhd3y ago

bmsleight_3y ago

Can you expand on " he clearly lack support". Bonus points for facts over opinions.

mmmuhd3y ago

Clearly means even his Vice Presidential Candidate could not win his own polling unit, polling unit, not ward, not Local Government, not State.

https://punchng.com/nigeriaelections2023-datti-loses-polling...

2 more replies

YeGoblynQueenne3y ago

>> We had a brainstorming meeting, and decided to try a new approach. We would simply ask the Obidients to help us do the conversion. If hundreds of Obidients did the transcription, it would go fast.

What would guarantee that the Obidients would not, in turn, try to inflate the score of the Labor candidate?

munchler3y ago

They planned to transcribe each PDF multiple times in order to validate the results.

seventytwo3y ago

Wow, this was a fantastic read!

I have no idea what’s going on in Nigeria, but I hope the truth (whatever it is) will prevail!

nivenkos3y ago

This is a great example of why electronic voting is important and can help secure democracy.

logifail3y ago

> This is a great example of why electronic voting is important and can help secure democracy.

If those in power are against change, I wouldn't want to have to put my trust in electronic voting if I was hoping for change.

I was left with the impression that it is the paper records in this story that led to the unravelling of an attempt to forge the results.

Long live paper ballots.

SkeuomorphicBee3y ago

> I was left with the impression that it is the paper records in this story that led to the unravelling of an attempt to forge the results.

2 more replies

TazeTSchnitzel3y ago

On the contrary, eletronic voting doesn't create the paper trail necessary to dig up frauds like this. You can simply program or hack the system to report any vote total you want.

SkeuomorphicBee3y ago

First of all, hacking the electronic system is much much harder than hacking the paper process. In the case at hand the paper tallying process was the one hacked.

2 more replies

gdelfino013y ago

You introduce technology to increase transparency and fight corruption. You increase transparency by having video recordings of human counting votes linked to the electronic record of the totals.

When you introduce technology to eliminate manual counting and paper trails, then transparency is eliminated and you give a green light to fraud, corruption, very juicy contracts and death.

cwkoss3y ago

Wouldn't electronic voting just create a means for the ruling party to deliver the result without releasing evidence of vote tampering?

I don't understand what you think electronic voting solves...

jxramos3y ago

> Then ominously, on the 20th of October of 2020 some people drove there in unmarked cars and removed all the Cameras installed at the tollgate.

They at least capture some photos of the equipment. I wonder if anyone communicated with the individuals.

pjc503y ago

Striking reminder of how big the world is that while I had heard of #EndSARS, I hadn't realised the scale of the political violence in Nigeria nor that it had its own Bloody Sunday-scale massacre.

londons_explore3y ago

The votes surprise me... In many regions one party gets 90+% of the vote.

Assuming the numbers are correct, then it suggests that most people are easily swayed by their local peers.

Is that common in say the USA?

crazygringo3y ago

> it suggests that most people are easily swayed by their local peers.

That feels like a particularly uncharitable interpretation to me.

So it's not about susceptibility to being "swayed", but genuine policy affecting regions differently.

muyuu3y ago

anigbrowl3y ago

Yep, bloc voting can be habitual or strategic. There's a town in Northern California where the majority of the seats on the council is held by people who all happen to attend the same megachurch.

mmmuhd3y ago

Exactly! and this mostly happened in the regions where the OP's preferred candidate won. This is clear scam.

blntechie3y ago

What was the final result numbers from the transcription?

toyg3y ago

They're probably going to be similar to the 14k sample he tweeted: a solidified Labour Party getting 50-55% of the votes, and the establishment candidates splitting the rest.

thread_id3y ago

pxc3y ago

I'm impressed by the courage of the protesters here, and the tenacity of the youth voters.

I hope they get a clear answer and a fair count, and whether they win this time or not, a real shot at cracking up their corrupt, two-party system.

neves3y ago

Is it true that USA does not have a open data law to make everybody publish in CSV?

JumpCrisscross3y ago

> Is it true that USA does not have an open data law to make everybody publish in CSV?

American elections are de-centralised. Each state comes up with its methods. In some, each county. (I'm not sure how publishing a CSV of vote totals would help.)

SergeAx3y ago

segfaultbuserr3y ago

PDF isn't the actual problem in this particular case. The documents here are photographs taken at different camera angles, embedded in PDFs.

jxramos3y ago

I was going to say, using alt drag to select vertical columns is usually how I extract useable tables out from pdfs with embedded tables.

orf3y ago

Fantastic story! Did the results get used in a claim?

vincheezel3y ago

I hope for (but do not expect) a positive outcome

jgtrosh3y ago

The context should be dated to 2020, not 2023 Edit: it was now corrected, no need to downvote

Great story! Looking forward to some follow up

public_defender3y ago

I don't understand. The article says the SARS protests started in 2020 and the election was in 2023. This seems correct.

jgtrosh3y ago

Yes, it was now corrected

dejongh3y ago

Wow. Wild story. Thanks for sharing. Cool twist that a bug ended up identifying the bad guys.

clipper_janosch3y ago

What an exceptional story. You are a legend.

prhrb3y ago

What a scam by the ruling political party

roschdal3y ago

The people who cast the votes don't decide an election, the people who count the votes do. - Stalin.

pxc3y ago

In case the downright cartoonish character of this quotation made anyone else wonder if it were fake...

that quotation is, indeed, fake: https://web.archive.org/web/20220128105324/https://www.polit...

snvzz3y ago

Not providing CSV is at the level of criminal negligence.

churchill3y ago

pxc3y ago

@dang are these '-' comments an attempt to evade showdead?

mmmuhd3y ago

David Hundeyin is a deceitful, lying criminal, so don't bring his "Content" as any kind of evidence.

https://www.icirnigeria.org/controversy-as-oxford-terminates...

1 more reply

j / k navigate · click thread line to collapse