Reviewing Process and Criteria - ECTEL 2019

ECTEL knows multiple paper and submissions formats. Below we list the different expectations and review criteria for each category, as well as the best paper / demo / poster awards.

For all reviews the following holds true:

The review should firstly serve as constructive feedback to the authors. Authors have typically invested a lot of time and energy in this paper, and deserve a respectful, constructive reply. Reviewing is part of the ongoing scientific discourse, in the sense that researchers engage with each others’ works.  Secondly, the review should be informative to the program chairs who will ultimately need to decide on paper acceptance.

All papers need to be correctly formatted and have a correct length. If they are not, they may be rejected without further review.

Review criteria: Research papers

Contribution should be commensurate with length (6-14 pages)

Criteria

  • Relevance to the conference topics and appropriateness of keywords

  • Quality of writing/clarity of presentation

  • Research question

  • State of art regarding the research question

  • Contribution to the Field / Novelty

  • Methodology (suitable to the research question)

Overall evaluation  (reject, weak reject, neutral, weak accept, accept)

Reviewers are expected to provide a rating to each paper according to the above criteria and a justification for their respective ratings in a written comment.

Especially, reviewers are asked to identify within the full review whether they would recommend the paper for a best paper award.

Review Criteria: Demo papers

Reviewers are asked for an overall evaluation  (reject, weak reject, neutral, weak accept, accept)

In addition, reviewers are expected to provide a full-text review that justifies their rating and addresses the relevant criteria from the below list:

Demo Criteria

  • Relevance to the conference topics

  • Quality of writing/clarity of presentation

    • Clear description of the presented prototype (how is the prototype made? Which technologies does it use? How does it work?)

    • Clear purpose of the presented prototype (what aspects of learning does the described prototype support?)

  • State of art regarding the demo prototype - can be briefer than in full research papers, or poster papers, but authors need to make clear in what way the demo prototype is interesting and novel to see for conference participants.

  • Contribution and novelty: Clear contribution or expected contribution of the prototype to the state-of-the-art research or practice (what is new about the prototype? how does it compare to existing systems? How does it inform research?)

  • Demo: Clear description on how the prototype will be used by learners (how are learners supposed to interact with the prototype?)

Poster papers

Reviewers are asked for an overall evaluation  (reject, weak reject, neutral, weak accept, accept)

Reviewers are expected to provide a full-text review that justifies their rating and addresses the relevant criteria from the below list:

  • Relevance to the conference topics

  • Quality of writing/clarity of presentation

  • Research question

  • State of art regarding the research question

  • Contribution to the Field / Novelty - What does the poster contribute (or intend to contribute) to the state of the art research – what is new about the proposed approach?

  • Methodology (suitable to the research question - at least planned). We expect a poster paper to present at least some idea of how the research is going to mature. Alternatively, a poster may describe preliminary research, in which case both preliminary methods, results, and future methodologies may be described.

  • Preliminary or expected results - as the contribution in a poster may be more of the type “intended contribution” than “already made contribution”, we are looking for preliminary or expected results.

Practitioner papers

(Long/short paper) - the length should be proportionate to the contribution

Reviewers are asked for an overall evaluation  (reject, weak reject, neutral, weak accept, accept)

Reviewers are expected to provide a full-text review that justifies their rating and addresses the relevant criteria from the below list:

  • Relevance to the conference topics

  • Quality of writing/clarity of presentation

  • Motivation: Does the submission address a real world problem or practical use case?

  • State of the art in practice - what is currently the state of the art in practice?

  • Take-Away Message and Contribution: What does the paper contribute - does it identify a relevant practical problem that would warrant more research? Does it describe a best practice that shall be shared in the community? In what way is the identified problem or practical solution interesting and relevant to practitioners and researchers at ECTEL?



Workshop proposals

Reviewers are asked for an overall evaluation  (reject, weak reject, neutral, weak accept, accept)

Reviewers are expected to provide a full-text review that justifies their rating and addresses the relevant criteria from the below list:

  • Relevance to the conference topics

  • Quality of writing/clarity of presentation

  • Existing community: Is the workshop preceded by previous workshops? Is there some other form of community that is addressed by the workshop proposal?

  • Target audience: Who is the target audience? Does the proposal clearly state how sufficient submissions and participants could be achieved?

  • Format: Is the proposal implementing innovative and interactive formats?

  • Reasonable planning: Is the proposal overall reasonably and well planned?

  • Publication: [Recommended] Does the proposal include a reasonable plan to make workshop results publicly and persistently available (e.g., workshop proceedings - but other reasonable plans are welcome)?

Host Organisation

Centre for Education and Learning

Supporting Organisations

Delft University of TechnologyErasmus Universiteit RotterdamUniversiteit LeidenDIPF Bildungsforschung und BildungsinformationInstitut JOŽEFA STEFANKing's College LondonKnow Center TU GrazOpen UniversiteitTallinn UniversityTU GrazUniversité Toulouse III Paul Sabatierhttc e.V.

Supporters:

European Association On Technology Enhanced Learning

Sponsors:

Springer

Social Media: