From the first article in the series [0]:
> Insiders ... understand that a research paper serves ... in increasing importance ... Currency, An advertisement, Brand marketing ... in contrast to what outsiders .. believe, which is ... to share a novel discovery with the world in a detailed report.
I can believe it's absolutely true. And yikes.
Other than the brutal contempt, TFA looks like pretty good advice.
In the private sector you can choose your patrons and your dissemination mechanism. Many, many scientists publish papers, publish code, give talks, write blogs, and otherwise distribute technical details about their work product.
In academia the Federal Government is your only serious patron and you must disseminate in academic journals/conferences, which generally do a piss poor job of providing incentives for either doing good work or communicating well about that work.
Any time I hire a junior PhD I have to UNDO a ton of academic writing/provlem-solving propaganda and reteach both common sense and normal writing style.
The harsh truth is that private sector scientists tend to do better science and disseminate it in more useful and lasting ways. They are paid better for it.
The academic scientists who are up to private sector standards tend to have diverse funding mechanisms and therefore rely far less heavily on prestige publication for their labs revenue stream. But most professors must publish papers because they are unable to do good work and/or communicate the value of that work to anyone other than their inner circle of friends (who sit on the grant review panels or take stints at federal agencies).
The most disturbing thing about it is the way advice to forget about science and optimize for the process is mixed with standard tips for good communication. It shows that the community is so far gone that they don't see the difference.
If anyone needs a point of reference, just look at an algorithms and data structures journal to see what life is like with a typical rather than extreme level of problems.
Chemists are extremely brand-aware regarding their figures.
In synthetic chemistry many chemists could guess the author based just on the color scheme of the paper's figures.
For instance, look at the consistency here: https://macmillan.princeton.edu/publications/
And it comes with rewards! The above lab is synonymous with several popular techniques (one, organocatalysis, which garnered a Nobel prize) - the association would be much less strong if the lab hadn't kept a consistent brand over so many years.
The number of accepted papers is absolutely currency and measure of worth in academia.
> Could you imagine someone saying, "be sure that the graphic for the molecule in figure 1 is 3D and has bright colors?"
I doubt the reviewers asked for that, but yes that kind of thing happens all the time prior to publishing and there’s nothing wrong with it. If it reduces the amount of time it takes to understand the paper then do it..
This tends to not manifest as "We need one of these" but "If we have one of these, lets be sure to use it."
This article is spot on. what are you talking about? have you ever published a research paper and gone through peer review?
The most fun in science can be had when done at home and shared with friends.
Which is why it's so funny when you see non skeptical appeals to "the god of science" which apparently exists in a vacuum of correctness and ethical purity.
As long as It has some capacity to self correct, it’s a stable function.
Thankfully the scientific process is incredibly resilient to nonsense, because a bad result will eventually screw up someone's future work when they come to rely on it. But it's not pretty.
Although there's plenty of critique to go around about the review system, machine learning here typically uses double-blind peer review for the major conferences. That blinding is often imperfect (e.g. if a paper very obviously uses a dataset or cluster proprietary to a major company), but it's not precise enough to reject a paper based on the author being an unknown.
So I assume that it is not done to keep outsiders out of your garden…
Honestly, I don’t find any other reason to don’t apply it.
> And it’s not just a pace thing, there’s a threshold of clarity that divides learned nothing from got at least one new idea.
But these days, ideas are quite cheap: in my experience, most researchers have more ideas than students to work on them. Many papers can fit their "core idea" in a tweet or two, and in many cases someone has already tweeted the idea in one form or another. Some ideas are better than others, but there's a lot of "reasonable" ideas out there.
Any of these ideas can be a paper, but what makes it science can't just be the fact that it was communicated clearly. It wouldn't be science unless you perform experiments (that accurately implement the "idea") and faithfully report the results. (Reviewers may add an additional constraint: that the results must look "good".)
So what does science have to do with reviewers' fixation on clarity and presentation? I claim: absolutely nothing. You can pretty much say whatever you want as long as it sounds reasonable and is communicated clearly (and of course the results look good). Even if the over-worked PhD student screws up the evaluation script a bit and the results are in their favor (oops!), the reviewers are not going to notice so long as the ideas are presented clearly.
Clear communication is important, but science cannot just be communicating ideas.
As an academic I need to be up to date in my discipline, which means skimming hundreds of titles, dozens of abstracts and papers, and thoroughly reading several papers a week, in the context of a job that needs many other things done.
Papers that require 5x the time to read because they're unnecessarily unclear and I need to jump around deciphering what the authors mean are wasting me and many others' time (as are those with misleading titles or abstracts), and probably won't be read unless absolutely needed. They are better caught at the peer review stage. And lack of clarity can also often cause lack of reproducibility when some minor but necessary detail is left ambiguous.
In the end, getting a paper accepted is a purely social game, and has not much to do with how clear your science is described, especially for truly novel research.
By “idea” researchers usually imply “idea for a high-impact project that I’m capable of executing”. It’s not just about having ideas, but about having ideas that will actually make an impact on your field. Those again come in two flavors: “obvious ideas” that are the logical next step in a chain of incremental improvements, but that no one yet had time or capability to implement; and “surprising ideas” that can really turn a research field upside down if it works, but is inherently a high-risk/high-reward scenario.
Speaking as a physicist, I find the truly “surprising ideas” to be quite rare but important. I get them from time to time but it can take years between. But the “obvious” ideas, sure, the more students I have the more of them I’d work on.
> Any of these ideas can be a paper, but what makes it science can't just be the fact that it was communicated clearly. It wouldn't be science unless you perform experiments (that accurately implement the "idea") and faithfully report the results. (Reviewers may add an additional constraint: that the results must look "good".)
I kinda agree with this. With the caveat that I’d consider e.g. solving theoretical problems to also count under “experiment” in this specific sentence, since science is arguably not just about gathering data but also developing a coherent understanding of it. Which is why theoretical and numerical physics count as “science”.
On the other hand, I think textbooks and review papers are crucial for science as a social process. We often have to try to consolidate the knowledge gathered from different research directions before we can move forward. That part is about clear communication more than new research.
I think it's still the case that there's lots of ideas that (if they worked!) would be surprising. Anyone can state outlandish ideas in a paper -- imo the contribution is proving (e.g. with sound "experiments", interpreted broadly) that they actually work. Unfortunately, I think clarity of writing matters more to reviewers than the soundness of your experiments. I think in CS this could very well change if the reviewers willed it (i.e. require artifact submission with the paper, and allow papers to be rejected for faults in the artifact)
The value lies in getting true ideas in front of your eyeballs. So communicating the idea clearly is crucial to making the value available.
I can write anything I want in the paper, but at the end of the day my experiments could do something slightly (or completely) different. Where are reviewers going to catch this?
I would be interested to hear other perspectives.
In particular, the lines between science and some industry is blurring.
Eg. Machine learning where universities appear almost lazy compared to their industrial counter parts.
There is a half-joke in our lab that the more times a paper is rejected, the bigger or more praised it will be once it's accepted. This simply alludes to the fact that many times reviewers can be bothered with seeing value in certain ideas or topics in a field unless it is "novel" or the paper is written in a way that is geared towards them, rather than being relegated to "just engineering effort" (this is my biased experience). However, tailoring and submitting certain ideas/papers to venues that value the specific work is the best way I have found to work around this (but even then it takes some time to really understand which conferences value which style of work, even if it appears they value it).
I do think there is some saving grace in the section the author writes about "The Science Thing Was Improved," implying that these changes in the paper make the paper better and easier to read. I do agree very much with this; many times, people have bad figures, poor tables or charts, bad captions, etc., that make things harder to understand or outright misleading. But I only agree with the author to a certain extent. Rather, I think that there should also be changes made on the other side, the side of the reviewer or venue, to provide high-quality reviews and assessments of papers. But I think this is a bit outside the scope of what the author talks about in their post.
But I think your most significant change was changing the "what" to "why".
Reading the original, we can see that most sentences start with "we did..." "we did..." and my impression as a reader was, "Okay, but how is this important?" In the second one, the "what" is only in the first part of the sentence, to name things (which gives a sense of novelty), and then only "whys" come after it.
"Whys" > "Whats" also applies to good code comments (and why LLM's code sometimes sucks). I can easily know "what" the code does, but often, I want to know "why" it is there.
In the old days, scientific careers were largely restricted to the independently wealthy or those who could secure patrons.
I also feel like there's a sort of tension with what Hacker News broadly wants out of science. There's often a lament that there aren't enough staff science positions, or positions where people can have a career beyond a postdoc that's just devoted to research.
Those things have to be paid for. Postdocs are expensive. Staff scientists are expensive - and terrifying, because they have careers and kids and mortgages. Postdocs are expensive.
That ends up eating a lot of a PIs time, because the success rate on proposals are low. Even worse now.
Would I love to be able to just sit in my office, think my thoughts, and occasionally write those thoughts up? Sure. But I'd also like to give people an opportunity to have careers in science where they can get paid.
Sure it's framed in terms of "helping you get published" (which feels kind of gross) but I think ultimately it's really about tips for authors to get their points across in a clear and engaging way.
It's the difference between being a Cassandra or the Oracle at Delphi. Maybe the only difference between the two was presentation? (Classicists, feel free to roast my metaphor).
> "The primary objects of modern science are research papers. Research papers are acts of communication. Few people will actually download and use our dataset. Nobody will download and use our model—they can’t, it’s locked inside Google’s proprietary stack."
The author is confusing the concept of 'science as a pursuit that will earn me enough money and prestige to live a nice life' - in which, I'd say, we can replace 'science' with 'religion' and go back to the 1300s or so - with science as the practice of observation, experiment and mathematical theory with the goal of gaining some understanding of the marvelously wonderful universe we exist in.
Yes, the academic system has been grotesquely corrupted by Bayh-Dole, yes, the academic system is internal blood sport politics for a limited number of posts, yes, it's all collapsing under the weight of corporate corruption and a degenerate ruling class - but so what, science doesn't care. It can all go dormant for 100 years, it has before, hasn't it? 125 years ago you had to learn to read German to be up on modern scientific developments.
Wake up - nature doesn't care about the academic system, and science isn't reliant on some decrepit corrupt priesthood.
P.S. Practically speaking, new graduate students should all be required to read Machiavelli as an intro to their new life.
1. Avoid overly general citations. The rejected paper leads with references to image captioning tasks in general and visual question-answering, neither of which is directly advanced by the described study. The accepted paper avoids these general citations in favour of more specific literature that works directly on the image-comparison task.
2. Don't lead with citations. The accepted paper has its citations at the end of the introduction, on page 2.
I think that each change is reasonably justified.
In avoiding overly-general citations, the common practice in machine learning literature is to publish short papers (10 pages or fewer for the main body), and column inches spent in an exhaustive literature review are inches not spent clearly describing the new study.
Placing citations towards the end of the introduction is consistent with the "inverted pyramid" school of writing, most commonly seen in journalism. Leaving the review process out of it for the moment, an ordinary researcher reading the article probably would rather know what the paper is claiming more than what the paper is citing. A page-one that can tell a reader whether they'll be interested in the rest of the article does readers a service.
My least favourite type of citations in introductions, that I often see from more junior researches are ones that look like:
"In this paper we use a Machine Learning [1][2][3] technique known as Convolutional [4] Neural Networks [5][6][7][8] to..."
In academia the equivalent is prestige. Who gets it and how? Who are the players? There are college students, PhD students, professors, administrators, grant committees, corporation-university industrial collaborations and consortiums, individual managers at corporations and their shareholders, university boards, funding agency managers, politicians allocating taxpayer money to research funding, journal editors, reviewers, tenure committees, pop science magazine editors, pop science magazine readers, general public taxpayers.
You should be able to put yourself in the shoes of each of these and have a rough idea of how they can obtain prestige as input from some other actor and how they can pass on prestige to yet another actor. You must understand the flow of prestige, and then it will be much less mysterious. (Of course understanding the flow of money also helps, but people tend to overlook prestige because one of the least prestigious things is to overtly care about prestige, it's supposed to seem effortless and unacknowledged)
Max should publish this in a book and it will probably sell by truckloads.
If I've to choose by ranking in usefulness, it will probably topic no. 4 is the best part "Don't Make Things Actually Work". Topic no. 3 is the second. This particular topic no. 5 is the third. Topic no.1 is the fourth. The topic no. 2 is the fifth ranking in usefulness but overall great advises nonetheless.
Perhaps the last one for the topic is when and how to wrap up the PhD research since research is a never ending endeavor.
"Is the scientific paper a fraud?"
I found a PDF online here: https://www.weizmann.ac.il/mcb/alon/sites/mcb.alon/files/use...
When it comes to papers, I always reminded myself and others that people also _read_ with their eyes.
It is easy to be cynical about this (with some justification!), but if the findings are more clearly and quickly communicated by a pretty-looking paper, then the paper has objectively improved.
If you're submitting to a control theory journal, you better have some novel theorems with rigorous mathematical proofs in that "rest of the paper" part. That's a little nontrivial.
It seems to go 180 degrees against what a smart starry-eyed junior grad student would believe. Surely, it's all about actually making things work, right? We are in the hard sciences, we don't just craft narratives about our ideas, we make cold hard useful things that are objectively and measurably better and can be used by others, building on top of it, standing on our shoulders, and what could be more satisfying than seeing the fruits of our research being applied and used.
However, for an academic career you want to cultivate the profile of a guru, a thought leader, a visionary, a grand ideas person. Fiddling with the details to put a working system together is lowly and kinda dirty work, like fixing clogged toilets or something. Not like the glorious intellectual work of thinking up great noble thoughts about the big picture.
If you want to pivot to industry, it could help you to build a track record of having created working systems, sure. But I've often seen grad students get stuck on developing bepoke internal systems that are not even really visible to potential future employers. Like improving the internal compute cluster tooling, automating the generations of figures in Latex, building a course management system to keep track of assignment submissions and exam grading and so on. Especially when you're at a phase where your research project is getting rejections and you feel stuck, you are most prone to dive into these invisible, career-killing types of work. In academia, what counts is your published research, your networking opportunities obtained through going to conferences where you have papers, getting cold emailed because someone saw your paper etc. I've seen very smart PhD students get stuck in engineering rabbit holes and it's sad. It happens less if your parents were already in academia, and you kinda get the gist of how things work via osmosis. But outsiders don't really grok what actually makes a difference and what is totally invisible (and a waste from a career perspective). Another such trap is pouring insane amounts of hours into teaching assistance and improving the materials, slides, handouts and so on. The careerists will know to spend just as much on this sort of stuff as they absolutely have to. Satisficing, not optimizing. Do enough to meet the bar, and not one minute more. It is absolutely invisible to the wider academic research community whether your tutorial session on Tuesday to those 20 students was stellar or just OK. Winners of the metagame ruthlessly optimize for visible impact and offload everything else to someone else or just not do them. A publication is visible. A research semester at a prestigious university is visible. Getting a grant is visible. Being the organizer of a workshop is visible. Meticulously grading written exams is invisible. Giving a good tutorial session is invisible. Improving the compute infrastructure of the lab is invisible. Being the goto person regarding Linux issues is invisible.
Packaging your research in a way that works well out of the box is in the middle on this spectrum. It may be appreciated by another stressed PhD student somewhere in some other university, and it may save them some time in setting things up. But that other PhD student won't sit on your grant committee or promotion board. So it might as well be invisible. Unless your work is so stellar and above and beyond other things that it goes viral and you become known to the community through it. But it's a double edged sword, because being known for having packaged your work in an easy to use manner will get you pigeonholed into the "software engineer technician" category, and not the "ideas person" category. Execution is useful but not prestigious. Like the loser classmate whose homework gets copied but isn't invited to parties.
The metagame winner recognizes that their work is transient. Any time spent on packaging up the research software for ease of use or ease of reproducibility once the publication is accepted is simply time stolen from the next project that could get you another publication. Since you'll likely improve the performance in the next slice of the salami anyway, there would be no use in releasing that outdated software so nicely. The primary research output is the paper itself, and the talks and posts you can make to market it to boost its citations, as well as the networking opportunities that happen around the poster and the conference. Extras beyond that are nice, but optional.
While you're working on making something "really" work, you're either delaying the publication, making it risky to get scooped (if done before publication), or you're dumping time into a dead project (dead in the sense that the paper is already published and won't be published-er by pouring more time into it post-publication).
This won’t get you a Stanford professorship. That’s something you can cry about from your mountain chalet or beachfront vacation home.
>The tweaks that get the paper accepted—unexpectedly, happily—also improve the actual science contribution. >The main point is that your paper’s value should be obvious, not that is must be enormous.
This is slightly oversimplified, but from the outside, science may look like researchers are constantly publishing papers sort of for the sake of it. However, the papers are the codified ways in which we attempt to influence the thinking of other researchers. All of us who engage in scientific research aim to be on the literal cutting edge of the research conversation. Therefore it's imperative to communicate how our work can be valuable to specific readers.
Let's take a look at the two abstracts:
(Version 1, Rejected): Given two distinct stimuli, humans can compare and contrast them using natural language. The comparative language that arises is grounded in structural commonalities of the subjects. We study the task of generating comparative language in a visual setting, where two images provide the context for the description. This setting offers a new approach for aiding humans in fine grained recognition, where a model explains the semantics of a visual space by describing the difference between two stimuli. We collect a dataset of paragraphs comparing pairs of bird photographs, proposing a sampling algorithm that leverages both taxonomic and visual metrics of similarity. We present a novel model architecture for generating comparative language given two images as input, and validate its performance both on automatic metrics and visa human comprehension.
(Version 2, Accepted): We introduce the new Birds-to-Words dataset of 41k sentences describing fine-grained differences between photographs of birds. The language collected is highly detailed, while remaining understandable to the everyday observer (e.g., “heart-shaped face,” “squat body”). Paragraph-length descriptions naturally adapt to varying levels of taxonomic and visual distance—drawn from a novel stratified sampling approach—with the appropriate level of detail. We propose a new model called Neural Naturalist that uses a joint image encoding and comparative module to generate comparative language, and evaluate the results with humans who must use the descriptions to distinguish real images. Our results indicate promising potential for neural models to explain differences in visual embedding space using natural language, as well as a concrete path for machine learning to aid citizen scientists in their effort to preserve biodiversity.