Home » CHI 2018 » Paper length in the reformulated Papers track

Champion Sponsors

Alibaba Group

Bloomberg

Facebook

Google

IBM Research

Microsoft

Oath logo

Past Deadlines

14 September 2017
(Originally 12 September 2017)
Papers: Title, abstract, authors, subcommittee choice, and all other metadata
Updates on PCS problems

19 September 2017
Papers: Submission files

11 October 2017
Doctoral Consortium
Case Studies
Courses
Accepted Courses

13 October 2017
Workshops/Symposia
Accepted Workshops/Symposia

27 October 2017
Art Exhibition

22 November 2017
Paper rebuttals are due at 20:00 EST (8pm in Montreal)

2 January 2018
Student Design Competition
Student Research Competition

15 January 2018
alt.chi
Early Career Development Symposium
Late-Breaking Work
Panels & Roundtable Discussions
Special Interest Groups (SIGs)
Video Showcase

25 January 2018
Demonstrations

8 March 2018
1 March 2018
Early Registration Deadline

Paper length in the reformulated Papers track

Headlines

  1. There are no Notes at CHI 2018. Papers with a length proportional to their contribution were invited in the CfP.
  2. There are no Note-length papers in the programme this year – the shortest papers have five pages of content.
  3. Only 7% of accepted submissions have 7 or fewer content pages.
  4. Papers with 10 pages of content make-up 81% of all accepted submissions.
  5. Looking at rejected submissions, it looks like longer submissions are more likely to be accepted.
  6. Length somewhat predicts scores. Scores predict acceptance. Length somewhat predicts acceptance.

The Papers track at CHI

CHI Notes, short submissions with a four-page limit, were introduced at CHI 2006. In a twist of fate, CHI 2006 took place in Montréal and with CHI 2018, which is also in Montréal, CHI Notes are no longer part of the programme. The last 12 CHIs have had Papers and Notes side by side from submission through to presentation. Papers had a 10 page limit. Notes had a 4 page limit. (Latterly, these limits have excluded references.) The reviewing process for Notes and Papers was the same. They were published in the same format. Notes, however, often fared badly in terms of relative acceptance rates. Table 1 shows the relative acceptance rates for Notes and Papers at past CHIs. It has been more challenging to get notes accepted at CHI.

Table 1. Historical acceptance rates of Papers and Notes

YearPapers acceptance rateNotes acceptance rate
201627.3%14.5%
201524.8%17.8%
201426.7%13.6%
201323.5%12.3%

Furthermore, the rates of Paper and Note submissions have diverged over recent CHIs. The number of submissions of Papers has increased substantially. Notes, already submitted in fewer numbers have not kept up.

Finally, Notes and Papers were given different lengths of talk slots at previous CHI conferences. Having to accommodate two different durations of talks complicated the planning process for the sessions at the conference.

With all of this in mind, the decision was taken to drop Notes from the conference, and instead accept papers of any length up to the limit of ten pages to the Papers track.

Paper lengths – accepted submissions

Have Notes ‘lived on’ in the new Papers track? Are we still getting those kind of short papers? First we will consider accepted papers. When authors of accepted submissions were preparing their final submissions, we asked them to record how many pages of content were in their manuscript. This allows us to augment our data from the number of pages in the submitted PDF, which also includes references.

The mean number of content pages was 9.6 (SD=1.14). The mean number of pages, including references, was 12.1 (SD=1.8). On average, papers came with 2.6 pages of references (SD=1.1). The shortest submissions were 4.5 pages (2). So many of the submissions (512, 77%) report 10 pages of content that a histogram is not informative. These data are tabularized instead, rounded to the nearest whole page (e.g., 4.5 pages and 5 pages are both rounded to five whole pages).

Table 2. Author indicated content length of accepted submissions

Number of content pagesFrequency% of all accepted submissions
5102%
6183%
7203%
8264%
9518%
1054181%

These data seem show that there are no Note-length papers being presented at CHI this year. Shorter papers (5-7 pages of content) account for only 7% of all of the papers being presented at the conference. Does this reflect reviewer’s historical preference for longer papers, or has the loss of the Note format encouraged people to submit generally longer papers?

Paper lengths – rejected submissions

Is there a preference for longer papers, or do acceptances simply reflect the underlying distribution of submissions? We do not have the number of content pages for rejected papers, because this information was not solicited from authors. However, from the accepted submissions we know that, on average, 79% of the total length of accepted submissions is made up of content. We can use this to estimate the number of content pages in the rejected submissions (NB – rejects only, not withdrawn, quick reject papers etc) made to the papers track from the length of the PDFs submitted. This yields Table 3.

Table 3. Estimated content length of rejected submissions

Estimated number of content pagesFrequency% of all rejected submissions
4623%
521111%
61166%
71659%
840321%
924113%
1067736%

Recall that these are only estimates. Nevertheless, these estimates seem to suggest a much more even distribution of paper content lengths amongst the rejected submissions. The accepted papers seem to be disproportionately 10 pages in length, compared to rejected papers.

Relationship between score and length

Are longer papers more likely to be accepted, then? We looked at the relationship between the average score a paper received and its total (i.e., including references) length.

Relationship between mean submission score and submission length in pages

Figure 1. Relationship between mean submission score and submission length in pages

This can be inferred from what has already been reported, but higher scores imply acceptance and longer submissions imply higher scores.

It seems clear that CHI reviewers want to see a CHI-sized contribution to get them tending toward accepting submissions. Whether this preference reflects a preference for longer papers or a preference for the more substantial contribution that longer papers should be able to make is not something that we can assess using the data at our disposal, but we might speculate.

It might be that the kind of disciplines that CHI encompasses generally don’t produce outputs that are well suited to low page limits (consider, e.g., work reporting the results of an interview study). It might also be that experienced researchers perceive full-length papers to have more cachet (e.g., in the context of tenure or national research assessments) and so focus their  efforts on full-length papers. It might also be the case that shorter submissions represent work that has been quickly –and therefore not fully– developed. These submissions might be of lower quality. All of these factors might play a role in the relationship between length and acceptance, amongst many others. We can’t be sure of the answer, but perhaps all of these things are worth considering when you come to submit your CHI 2019 papers in September.

Anna Cox and Mark Perry
Technical Programme Chairs, ACM CHI 2018

Sandy Gould
Analytics Chair, ACM CHI 2018


Leave a comment

Your email address will not be published. Required fields are marked *

News

24th April 2018
Breaking news! CHI est dans la press: comment combattre les incessantes distractions au bureau?
(How to fight the incessant distractions at the office).

21st April 2018
Registration has closed.

19th April 2018
Long flight? Download the papers and extended abstracts video previews to watch offline on the plane.

11th April 2018
Planning your trip to Montréal? Check our Montréal and Things to do in Montréal pages along with the Montréal tourist board CHI Delegates page.

9th April 2018
The CHI Expo will feature Demonstrations on Monday night. Get a sneak peak of this exciting event!

14th March 2018
The CHI18 video teaser is available, a quick overview of what's coming at CHI.

27th February 2018
The CHI 2018 Technical Program is now live.

26th January 2018
Traveling to Canada? Check out the visa requirements page.

25th January 2018
New this year! Book your hotel through the CHI registration page.

24th January 2018
CHI 2018 is excited to announce three dynamic keynote speakers who will address our theme - Engage - from differing perspectives.

23rd January 2018
Registration for CHI 2018 is open! Register now on the CHI registration site.

21st January 2018
Have children? Need support? Check out support for attendees with families at CHI 2018.

8th January 2018
Getting ready for CHI? Have a look at the CHI 2018 "Health Blog" by m.c. schraefel

4th January 2018
Direct registration for exhibitors is now open. For additional information, read “For Exhibitors”.

20th December 2017
Check out the CHI 2018 accepted courses

18th December 2017
Check out the CHI 2018 accepted workshops and symposia

Scroll Up