An assessment associated with the Ninth SOSP Submissions -or- How (and just how perhaps maybe maybe Not) to create an excellent systems Paper
Roy Levin and David D. Redell, Ninth SOSP Program Committee Co-chairmen
Introduction
On March 21, 1983, this program committee for the 9th Symposium on running System Principles, having browse the eighty-three documents submitted, chosen sixteen for presentation during the symposium. This acceptance ratio of approximately one in five approximates those of previous SOSPs, even though the true quantity of submissions ended up being notably less than in the past few years. A few people in this program committee discovered it interestingly an easy task to split up the good documents from the bad people; certainly, the ten committee users quickly decided on the disposition of over 80% for the papers. Because the acceptance ratio shows, a lot of these had been rejections.
Following the committee had finished its selectio n process, a few users indicated dissatisfaction into the quality that is overall of submissions. A number of the refused documents exhibited comparable weaknesses, weaknesses that the committee felt needs to have been obvious into the writers. When you look at the hope of increasing the caliber of future SOSP submissions, and systems documents generally speaking, the committee made a decision to explain the requirements found in assessing the documents it received. This short article combines the requirements utilized by all the known people of the committee, not only the writers.
To try and avoid sounding preachy or pedagogic, we’ve cast this presentation in the 1st and 2nd individual and adopted a light, periodically funny design. Nonetheless, the intent is severe: to indicate the problems that are common look over over repeatedly in technical papers in a manner that is likely to make it easier for future writers to prevent them. Yourself to be a prospective author for the 10th SOSP or for TOCS as you read this article, then, suppose. You have done some ongoing work you desire to publish, so that you sit back to create a paper. Just exactly just What questions for anyone who is thinking about as you compose? They are additionally the questions that individuals, the reviewers of one’s paper, may be asking to ascertain its suitability for book.
Classes of Papers
Your paper will probably fall obviously into certainly one of three groups:
- It presents a system that is real either by an international study of a complete system or by way of a selective study of particular themes embodied in the machine.
- It presents a method this is certainly unimplemented but uses ideas or strategies which you have the community that is technical know.
- It addresses a subject into the theoretical areas, as an example, performance modelling or protection verification.
Clearly, a solitary group of assessment requirements may not be used uniformly across these groups; however, many requirements use similarly well to all or any three. We will try to emphasize the classes of papers to which it applies as we describe each one below. Frequently it shall be obvious from context.
Requirements for Evaluation of Submissions
Original Tips
Would be the a few ideas into the paper brand new? There’s no point in publishing a paper to a seminar or journal focused on initial work unless the paper contains one or more idea that is new.
How will you understand? You should be knowledgeable about their state associated with the art and current research in the region included in your paper so that you can understand that your projects is initial. Possibly the many failing that is common the submissions in the 1st category (genuine systems) ended up being a lack of the latest tips; the systems described were usually isomorphic to at least one of only a few pioneering systems well-documented into the literary works.
Are you able to state the brand new concept concisely? In the event the paper is always to advance their state of real information, your audience needs to be capable of finding the brand new some ideas and realize them. Decide to try writing each concept down in a paragraph that somebody generally speaking versed within the appropriate area can realize. If you fail to, look at the possibility you don’t actually understand the concept your self. If you have the paragraphs, make use of them in the abstract when it comes to paper.
What is the issue being resolved? Your audience may not be anticipated to guess the issue you encountered given just a description of this solution. Be particular. Make sure to explain why your trouble could not be resolved simply aswell by formerly posted strategies.
Would be the basic tips significant sufficient to justify a paper? Often, papers explaining systems that are real 1 or 2 little improvements of founded methods. The newest idea(s) may be described in a couple of paragraphs; a twenty-page paper is unneeded and frequently obscures the innovation that is actual. Since construction of a genuine system will be a lot of work, mcdougal of this paper often unconsciously confuses the full total work utilizing the work this is certainly really brand new. (“My group labored on this technique for 2 years and now we’re finally done. Let us inform the global globe just exactly just how wonderful it really is.”) In the event that innovation is tiny, a paper that is small technical note in an appropriate log developmental psychology topics for research paper is much right than an SOSP submission.
May be the work described notably different from current associated work? a apparent expansion to a formerly posted algorithm, method, or system, doesn’t generally justify publication. Needless to say, the label “obvious” must certanly be applied very carefully. ( recall the tale of Columbus sexactly howing steps to make an egg stand at a time (by carefully crushing it): “it’s apparent once i have shown you how”.) you need to show that your particular work represents an important departure from their state for the art. You should ask yourself why you are writing the paper and why anyone except your mother should want to read it if you can’t.
Is all related work referenced, and also you actually browse the cited material? You’ll have trouble persuading the skeptical audience regarding the originality of one’s efforts if you do not particularly differentiate it from formerly posted work. This calls for citation. Also, you will discover it harder to persuade your audience of this superiority of one’s approach if he’s got read the cited works along withn’t.
Are evaluations with past work explicit and clear? You can’t simply state: “Our approach varies notably from that used within the BagOfBits system 3.” Be particular: ” Our memory that is virtual management utilizes magnetic news in place of punched paper tape like in the BagOfBits system 3, aided by the expected improvements in transfer price and janitorial expenses.”
Does the task comprise an extension that is significant validation, or repudiation of previous but unverified tips? Execution experiences supporting or contradicting a formerly posted paper design are really valuable and worthy prospects for book. Designs are low priced, but implementations (specially those centered on unsound designs) are costly.
What’s the earliest paper you referenced? The most recent? Have actually you referenced similar work on another organization? Have actually you referenced technical reports, unpublished memoranda, individual communications? The responses to those concerns help alert one to blind spots in your knowledge or understanding. Often, papers with just references that are venerable recently posted work of that your writer is unaware. Papers with only current references frequently “rediscover” (through ignorance) old ideas. Documents that cite just unpublished or material that is unrefereed to have problems with narrowness and parochialism. Understand that citations not just acknowledge a financial obligation to other people, but additionally act as an abbreviation procedure to spare your audience a complete development from very very first concepts. In the event that audience has to get some good of the development, nonetheless, he must certanly be in a position to transform your citations into supply product they can read. Private communications and memoranda that are internal this test. Technical reports are often posted in restricted amounts, out-of-print, and hard to get. Consequently, such citations as supply product ought to be prevented whenever we can.
In the event that system is implemented, exactly just exactly how has it been utilized, and exactly just just what has this usage shown concerning the practical need for the some ideas? Once more, a numerous implementation that is man-year will not of it self justify publication of the paper. In the event that system that is implemented new a few ideas, you should explain the way they worked down in training. a seemingly good notion that don’t pan away is at minimum because interesting as you that did. It’s important to be precise and specific. “Our climate prediction system is installed and operating with no you’ve got reported about its periodic forecasts that are inaccurate is a lot less convincing than “everytime we neglect to forecast rain, the users hang their damp tops on the tape drives to dry”. When you look at the case that is latter at minimum we understand that individuals are utilising and with regards to the system.