Paper summarizes community input to development of potential standards and recommended practices for research assessment metrics
(9 June 2014) The National Information Standards Organization (NISO) has released a draft white paper summarizing Phase I of its Alternative Assessment Metrics (Altmetrics) Project for public comment. The Initiative was launched in July 2013, with a grant from the Alfred P. Sloan Foundation, to study, propose, and develop community-based standards or recommended practices for alternative metrics. In Phase 1 of the project, three in-person meetings were held and 30 in-person interviews conducted to collect input from all relevant stakeholders, including researchers, librarians, university administrators, scientific research funders, and publishers. The draft white paper is the summary of the findings from those meetings and interviews, along with the identification of potential action items for further work in Phase II of the project.
“Citation reference counts and the Journal Impact Factor have historically been the main metric used to assess the quality and usefulness of scholarship,” explains Martin Fenner, Technical Lead Article-Level Metrics for the Public Library of Science (PLOS) and consultant to NISO for the project. “While citations will remain an important component of research assessment, this metric alone does not effectively measure the expanded scope of forms of scholarly communication and newer methods of online reader behavior, network interactions with content, and social media. A movement around the use of alternative metrics, sometimes called ‘altmetrics,’ has grown to address the limitations of the traditional measures. With any new methodology, however, issues arise due to the lack of standards or best practices as stakeholders experiment with different approaches and use different definitions for similar concepts. NISO’s Altmetrics project gathered together the variety of stakeholders in this arena to better understand the issues, obtain their input on what issues could best be addressed with standards or recommended practices, and prioritize the potential actions. This white paper organizes and summarizes the valuable feedback obtained from over 400 participants in the project and identifies a road forward for Phase II of the project.”
“More than 250 ideas were generated by participants in the meetings and interviews,” states Todd Carpenter, NISO Executive Director. “We were able to condense these to 25 action items in nine categories: definitions, research outputs, discovery, research evaluation, data quality and gaming, grouping and aggregation, context, stakeholders’ perspectives, and adoption. The highest priority items focused on unique identifiers for scholarly works and for contributors, standards for usage statistics in the form of views and downloads, and building of infrastructure rather than detailed metrics analysis. We are now soliciting feedback on the draft white paper from the wider community prior to its completion. The white paper will then be used as the basis for Phase II: the development of one or more of the proposed standards and recommended practices.”
The White Paper is open for public comment through July 18, 2014. It is available with a link to an online commenting form on the NISO Altmetrics Project webpage (www.niso.org/topics/tl/altmetrics_initiative/), along with the detailed output documents and recordings from each of the meetings and related information resources.
