Home » CHI 2018 » CHI 2018 – Looking at the big picture

Champion Sponsors

Alibaba Group

Bloomberg

Facebook

Google

IBM Research

Microsoft

Oath logo

Past Deadlines

14 September 2017
(Originally 12 September 2017)
Papers: Title, abstract, authors, subcommittee choice, and all other metadata
Updates on PCS problems

19 September 2017
Papers: Submission files

11 October 2017
Doctoral Consortium
Case Studies
Courses
Accepted Courses

13 October 2017
Workshops/Symposia
Accepted Workshops/Symposia

27 October 2017
Art Exhibition

22 November 2017
Paper rebuttals are due at 20:00 EST (8pm in Montreal)

2 January 2018
Student Design Competition
Student Research Competition

15 January 2018
alt.chi
Early Career Development Symposium
Late-Breaking Work
Panels & Roundtable Discussions
Special Interest Groups (SIGs)
Video Showcase

25 January 2018
Demonstrations

8 March 2018
1 March 2018
Early Registration Deadline

CHI 2018 – Looking at the big picture

The gargantuan effort of soliciting, submitting and assessing submissions at CHI is over for the 2018 programme. (The process for 2019 has already started!) In this blog post we look across all of the tracks to look at CHI’s big picture. How many submissions did we get? How many were accepted? For whatever reasons you might want to know these statistics – comparisons across different years, assessment of your own success rates against track averages, comparison to other conferences, or something else – we have tried to be as open as possible here about the conference data for the broad set of submissions and reviews.

Overall submissions

CHI 2018 solicited submissions to 14 tracks. Across all of these tracks we had a total of 3955. The conference is, unsurprisingly, dominated by the Papers track (66% of all submission to the conference). Late Breaking Work is also a significant contributor (16%). The rest of the tracks combined account for 18% of the submissions to the conference.

TrackSubmissionsAcceptRate
Papers259066625.7%
LBW64225639.9%
Demonstrations1307759.2%
Workshops and Symposia1084339.8%
Student Design Competition791215.2%
Video showcase691521.7%
Doctoral Consortium682029.4%
Case studies642132.8%
alt.chi611626.2%
Student Research Competition432455.8%
Courses382565.8%
Art241458%
SIGs231982.6%
Panels 16850%

Table 1: Submissions and acceptance rate by track.

A complete listing of all tracks, sorted by submission rate is provided in Table 1. Note that some of these numbers vary slightly from data you may have seen previously: this is because some submissions may have been removed from the submission system (for example, if they were withdrawn).

The Student Design Competition had the lowest acceptance rate (15.2%). SIGs were the most likely to be accepted (82.6%). Of course, the overall acceptance rate is skewed by the fact that we have so many submissions to the Papers track. The average unweighted acceptance rate (i.e., the mean of the individual track acceptance rates) is 43%.

Thank you everyone who submitted their work to CHI 2018! High quality submissions are the lifeblood of any conference.

Overall external reviews

Of course, all 3955 submissions had to have some kind of review, whether through the Paper track’s full peer-review, or through a slightly more lightweight approach. Here we consider just the external reviewers in the process – we’ll consider committee members in the next section.

Number of reviews contributed by each reviewer in the Papers track.

Figure 1: Number of reviews contributed by each reviewer in the Papers track. This is a plot of the data from a previous blog post.

The conference defines ‘Refereed’ content as that which is rigorously reviewed by members of the program committee and peer experts. The process includes an opportunity for authors to respond to referees’ critiques. Submitters can expect to receive formal feedback from reviewers, and the program committee may ask authors for specific changes as a condition of publication. Reviewers are also involved in ‘Juried’ content  (Late Breaking Work, Workshops/Symposia, Case Studies, alt.chi, Student Design and Research Competitions). Note that some tracks do not have external reviewers because they are ‘Curated’ (Panels, Courses, Doctoral Consortium, EXPO, Special Interest Group (SIG) meetings, Video Showcase), and while these tracks are highly selective, they do not usually follow a committee-based reviewing process. Fuller details on these processes can be found in Selection Processes on the CHI website.

In total, submissions to the tracks that made use of external reviewers generated 1.93 external reviews per submission. The Papers track attracted the lion’s share of reviews (73.9%), averaging 2.01 external reviews per submission. Outside the Papers and Late Breaking Work tracks, fewer external reviewers (516) were involved in the process (7.5%), and averaging 1.65. reviews per submission. Although 3837 non-committee reviewers are listed here across all tracks, we have not collated the contributions of individual reviewers across tracks, so we cannot yet say how many unique CHI external reviewers we had this year across all tracks.

Table 2 contains information for each track on the number of reviewers for a track and the total number of reviews that they produced. Note once again that these breakdowns are based on final-end-of process data and may vary very slightly from what has been reported previously (e.g., due to reviewers being added, or incomplete reviews being removed from the system).

TrackReviewersReviews
Papers26485061
LBW8741267
Workshops and Symposia131219
alt.chi100138
Case studies4083
Student Design Competition4476

Table 2: External reviewers and number of reviews by track

Thank you everyone who provided an external review for CHI 2018. People know that when they see work at CHI, it has been rigorously reviewed by experts. Without your reviews we wouldn’t be able to put together a programme of this quality.

Overall committee reviews

Submissions need reviewers. Committee members help to find them. Once reviewers give their opinion, someone has to synthesise them to come to a decision. Committee members do this too.

Between them CHI 2018 committee members wrote 6,321 reviews, meta-reviews or opinions. This is not far off the total number of external reviews – committee members work extremely hard in their roles. Once again, the Papers track generates the vast majority of the work (83.8%) for the committee in terms of reviews or meta-reviews written.

TrackCommittee
 members
Committee/Meta reviews
Papers3195295
LBW112664
Workshops and Symposia28108
Student Design Competition373
alt.chi1261
Case studies861
Video showcase759

Table 3: Committee members and number of reviews by track (NB – includes only PC members with meta-reviewing responsibility and not, for instance, track chairs).

Table 3 shows the contributions of committee members to the different tracks. Note that, again, these figures may not match up precisely because of changes over time (or, for instance, data not appearing in the correct place, a switching of 1AC and 2AC roles during the process, for instance). Also note that not all tracks have a process that involves committee members writing formal reviews.

The CHI 2018 committee members have worked meticulously, often on a large number of submissions, to make sure that the right decisions are made for the the conference programme. We thank all of them for their efforts.

Summary

As you can see, the CHI process generates huge volumes of work – for those working on committees, and as reviewers, as well as for authors, and while we have tried to minimise the effort for everyone involved, the review process requires vast amounts of time to ensure its sustained quality of outputs and topical relevance. Here we have only quantified the number of submissions and the amount of work that these submissions generate for reviewers and committee members. Of course, not all the tracks at CHI generate work that can be measured using the metrics we have used in this post, nor is this a complete account of thousands of hours of organizational work and planning that also has to take place for all of these parts to come together. We can only run a conference of this scale and quality with the dedicated effort of the researchers and practitioners that offer their time, energy, and expertise. The ACM CHI conference stands at the forefront of its discipline, but its pillars are supported by the community it serves. We thank those who have stepped up to do their bit, and encourage the next generation of researchers emerging to help take on these demanding but rewarding roles. The future of the conference lies in your hands!

 

Anna Cox and Mark Perry
Technical Programme Chairs, ACM CHI 2018

Sandy Gould
Analytics Chair, ACM CHI 2018


Leave a comment

Your email address will not be published. Required fields are marked *

News

24th April 2018
Breaking news! CHI est dans la press: comment combattre les incessantes distractions au bureau?
(How to fight the incessant distractions at the office).

21st April 2018
Registration has closed.

19th April 2018
Long flight? Download the papers and extended abstracts video previews to watch offline on the plane.

11th April 2018
Planning your trip to Montréal? Check our Montréal and Things to do in Montréal pages along with the Montréal tourist board CHI Delegates page.

9th April 2018
The CHI Expo will feature Demonstrations on Monday night. Get a sneak peak of this exciting event!

14th March 2018
The CHI18 video teaser is available, a quick overview of what's coming at CHI.

27th February 2018
The CHI 2018 Technical Program is now live.

26th January 2018
Traveling to Canada? Check out the visa requirements page.

25th January 2018
New this year! Book your hotel through the CHI registration page.

24th January 2018
CHI 2018 is excited to announce three dynamic keynote speakers who will address our theme - Engage - from differing perspectives.

23rd January 2018
Registration for CHI 2018 is open! Register now on the CHI registration site.

21st January 2018
Have children? Need support? Check out support for attendees with families at CHI 2018.

8th January 2018
Getting ready for CHI? Have a look at the CHI 2018 "Health Blog" by m.c. schraefel

4th January 2018
Direct registration for exhibitors is now open. For additional information, read “For Exhibitors”.

20th December 2017
Check out the CHI 2018 accepted courses

18th December 2017
Check out the CHI 2018 accepted workshops and symposia

Scroll Up