One of the first things we did after all NIER 2013 papers were in, was identifying papers that should be desk rejected. What is a desk reject? Why are papers desk rejected? How often does it happen? What can you do if your paper is desk rejected?
A desk reject means that the program chairs (or editors) reject a paper without consulting the reviewers. This is done for papers that fail to meet the submission requirements, and which hence cannot be accepted. Filtering out desk rejects in advance is common practice for both conferences and journals.
To identify such desk rejects for NIER 2013, program co-chair Sebastian Elbaum and I made a first pass through all 160+ submissions. In the end, we desk rejected around 10% of the submissions (a little more than I had anticipated).
Causes for reject included problems in:
- Formatting: The paper does not meet the 4 page limit;
- Scope: The paper is not about software engineering;
- Presentation: The paper contains, e.g., too many grammatical problems;
- Innovation: The paper does not explain how it builds upon and
extends the existing body of knowledge.
Of these, for NIER the formatting was responsible for half of the desk rejects.
Plagiarism
A potential cause that we did not encounter is plagiarism (fraud), or its special form self-plagiarism (submitting the same, or very similar, papers to multiple venues).
In my experience, plain plagiarism is not very common (I encountered one case in another conference, where we had to apply the IEEE Guidelines on Plagiarism).
Self-plagiarism is a bigger problem as it can range from copy-pasting a few paragraphs from an earlier paper to a straight double submission. While the former may be acceptable, the latter is considered a cardinal sin (your paper will be rejected at both venues, and reviewers don’t like reviewing a paper that cannot be accepted). And there are many shades of grey in between.
Notifications
We sent out notifications to authors of desk rejected papers within a few days after the submission deadline (it took a bit of searching to figure out that the best way to do this is to use the delete paper option from EasyChair). Thus, desk rejects not only serve to reduce the reviewing load of the program committee, but also to provide early feedback to authors whose papers just cannot make it.
Is there anything you can do to avoid being desk rejected?
The simple advice is to carefully read the submission guidelines. Besides that, it may be wise to submit a version adhering to all criteria early on when there is no immediate deadline stress yet. This may then serve as a fallback in case you mess up the final submission (uploading, e.g., the wrong pdf). Usually chairs have access to these earlier versions, and they can then decide to use the earlier version in case (only) the final version is a clear desk reject (for NIER this situation did not occur).
Is there anything you can do after being desk rejected?
Usually not. Most desk rejects are clear violations of submission requirements. If you think your desk reject is based on subjective grounds (presentation, innovation), and you strongly disagree, you could try to contact the chairs to get your paper into the reviewing phase anyway. The most likely outcome, however, will still be a reject, so it may not be in your self-interest to postpone this known outcome.
Submission times
And … are desk rejects are related to the paper submission time? Yes, there is a (mild) negative correlation: For NIER, there were more desk rejects in the earlier than in the later submissions. My guess is that this is quite common. There seem to be authors who simply try their same pdf at multiple conferences, hoping for an easy conference with little reviewing only.
Acceptance rates
This brings me to the final point. Conferences are commonly ranked based on their acceptance ratio. The lower the percentage of accepted papers, the more prestigious the conference is considered. The most interesting figure is obtained if acceptance rates are based on the serious competition only — i.e., the subset of papers that made it to the reviewing phase. Desk rejected papers do not qualify as such, and hence should not be taken into account when computing conference acceptance rates.
Arie van Deursen 
Do you use special software like Turnitin to scan for plagiarism?
No, we did not. Sounds like a useful service/tool ACM/IEEE could offer to organizers of conferences they sponsor!
I think the broader question of innovation is probably best left to the reviewers themselves, considering that it’s something that is quite difficult to judge.
You rejected only 10% of the papers after making a pass on all 160. Two questions. One, would you say it was worth your effort (I don’t know how much effort it took you to make a pass on them all)? Two, balance that against giving all papers a fair shot. Would it not have been better to simply let each paper get three reviews?
Thanks for your comments.
The reason for desk reject is not that the innovation is considered insufficient (which would indeed be difficult to judge). The reason for desk reject in the innovation category is that the paper does not even include a comparison against the state of the art (an easy to spot indicator might be: a paper with no references).
Concerning the effort:
As program co-chairs, we need to go through all papers anyway — be they accepted, rejected, or desk rejected. Therefore turning desk rejects into regular rejects would not save the program chairs any work.
Concerning the fair shot:
If three reviewers have to evaluate a paper that falls in one of the desk reject categories they will all write the same short review (‘paper too long’). Hearing that from three reviewers is not better than hearing it once from the chairs. But hearing it fast (which is what you get with a desk reject) is better. (see also the nice post Please Reject Me)
But your basic point is that desk rejects should be beyond doubt, with which I fully agree. And yes, there are papers for which there is no doubt at all.
Pingback: How to avoid desk rejections | Achilleas Kostoulas
… desk rejection because of presentation problems:)
“Is there anything you can do after BEING desk rejected?”
Fixed, thanks!