Interim Guidance Document Governing Code of Conduct, Peer Review, and Information Quality Correction for National Park Service Cultural and Natural Resource Disciplines
/s/ Herbert C. Frost
Herbert C. Frost
Acting Associate Director
Natural Resource Stewardship and Science
January 31, 2008
This guidance document will remain in effect until amended or superseded.
Interim Guidance Document Governing Code of Conduct, Peer
Review, and Information Quality Correction for National Park Service
Cultural and Natural Resource Disciplines
I. Background and Purpose
II. Authority to Issue this Reference Manual
III. Code of Scientific and Scholarly Conduct
IV. Information Quality
V. Thresholds for Peer Review
VI. Peer Review Policies and Instructions
IX. Legal Effect
I. BACKGROUND AND PURPOSE
Sound management and interpretation of National Park System resources and National Park Service technical assistance programs depend on authoritative information from scientific and scholarly activities. Peer review provides the appropriate process for ensuring technical quality and accuracy of these activities, which include inventory, monitoring, research, assessment, and management projects. This peer review must be applied appropriately at all stages, which may include proposals at the funding decision point, detailed proposals or plans of action at the point of initiating an activity, progress of an activity at key points during its duration, and results and draft reports of accomplishments of the activity.
The scientific and scholarly guidance presented in this document ensures that the review requirement is met using a consistent, conscientious, and appropriate level of effort. Such technical peer review is essential to demonstrate the professional stature and ensure the accountability of the National Park Service's acquisition and application of scientific and scholarly information. Such scientific peer review complements, and is part of, administrative review.
A. Enhance the quality, objectivity, utility, and integrity of scientific and scholarly information generated or supported by the NPS;
The NPS shall ensure appropriate peer review of all
scientific and scholarly information prior to use in decision-making, regulatory
processes, or dissemination to the public and regardless of media (i.e., print,
digital, audiovisual, or Web). Any information that the NPS determines to be
either “influential scientific information” and/or “highly influential
scientific assessment” will require more specific, independent peer review
consistent with the OMB Peer Review Bulletin.
NPS will place these guidelines, its peer review agendas for influential
scientific information and highly influential scientific assessment (as defined
by the OMB Peer Review Bulletin), and its information quality correction site
on a publicly available webpage. These
guidelines become effective on an interim basis
Peer review is the critical evaluation of the scientific or scholarly merits of an activity conducted by impartial subject-matter experts who are not directly associated with the activity. Peer review may be formal or informal. It may be authored or anonymous. It may involve open exchange of views and ideas between reviewers and managers of the activities being reviewed during the process of the review, or it may consist of reviewers conducting their observations and writing their reports without any contact with those managers. No matter how it is conducted, it must be objective to achieve its purpose of ensuring that the quality, objectivity, utility, and integrity of scientific and scholarly information meets scientific, scholarly, and technical community standards.
Peer review may be conducted by scientists and scholars from the National Park Service, other federal or state agencies, universities, or other organizations. Peer reviewers may submit their comments separately to the National Park Service or they may submit their comments jointly as part of a team report, subject to appropriate provisions of the Federal Advisory Committee Act.
Peer review is different from public comment and other stakeholder processes, which are not addressed in this document. The selection of peer reviewers is based on their demonstrated expertise, independence, and lack of conflict of interest.
I.f. What is Administrative Review?
The scientific and scholarly peer review process outlined here does not eliminate the need for administrative review of scientific and scholarly activities by managers. Managers conduct administrative review to ensure that:
• proposed activities are compatible with Park Service policies and regulations;
• proposed activities are pertinent to park purposes, programs, and needs; and
• personnel, logistical (e.g., housing, permits, and transportation), and supplementary (e.g., equipment and supplies) operating needs are identified and authorized prior to commencement of a project.
II. AUTHORITY TO ISSUE THIS REFERENCE MANUAL
II.a. Guidance for Science, Peer Review, and Information Quality
· 305 DM 2 requires that science must be integrated into and used in setting regulatory and management policies in the Department and its bureaus;
· E.O. 12866 establishes a government-wide policy that each agency shall base its regulatory decisions on the best reasonably obtainable scientific, technical, economic and other information; and
· OMB Final Information Quality Bulletin for Peer Review (70 FR 2664-2677) establishes government-wide requirements for the peer review of “influential scientific information” and “highly influential scientific assessments.” It also establishes requirements for posting peer review plans on a website and annual reporting of associated peer review activities to the OMB.
II.b. Guidance for Scientific Code of Conduct
· 43 CFR 20.501 requires employees of the Department to comply with all Federal statutes; Executive Orders; and Office of Government Ethics, Office of Personnel Management, and Departmental regulations;
· 43 CFR 20.502 states that employees are required to carry out the announced policies and programs of the Department;
· 43 CFR 20.502(a) states that an employee is subject to appropriate disciplinary action if he or she fails to comply with any lawful regulations, orders, or policies;
Federal Policy on Research Misconduct,
65 FR 76260-76264,
II.c. General Authority
· 16 U.S.C. 1 through 4 (the National Park Service Organic Act).
II.d. Other Relevant Policy and Guidance
Chapter 1, The Foundation, Sections 1.8 (Managing Excellence); 1.9.4 (Public Information and Media Relations);
Chapter 2, Park System Planning, Sections 2.1.2 (Scientific, Technical, and Scholarly Analysis), 2.1.3 (Public Participation); 188.8.131.52 (Science and Scholarship);
Chapter 4, Natural Resource Management, Sections 4.1.2 (Natural Resource Information); 4.2 (Studies and Collection); 184.108.40.206 (Paleontological Resources and Their Contexts); 220.127.116.11 (Caves);
Chapter 5, Cultural Resource Management, Sections 5.1.1 (National Park Service Research); 5.2.3 (Confidentiality); 18.104.22.168.2 (Sacred Sites); 22.214.171.124.4 (Acquisition, Management, Disposition, and Use); 126.96.36.199.6 (Archives and Manuscripts);
Chapter 7, Requirements for All Interpretive and Educational Services, Section 7.5.4 (Research and Scholarship);
Chapter 8, Use of the Parks, Sections 8.5 (Use by American Indians and Other Traditionally Associated Groups); 8.11 (Social Science Studies);
Chapter 10, Commercial Visitor Services, Section 10.2.4.9 (Natural and Cultural Resource Management Requirements).
· DO #19 Records Management; and
· DO #78 Social Science, Section III.I (Peer Review).
III. Code of Scientific AND SCHOLARLY Conduct
To enhance their contribution to quality, objectivity, utility, and integrity of such information, all NPS employees working with scientific and scholarly information will, in performing their duties:
IV. INFORMATION QUALITY
IV.a. Administrative Record
Whenever the NPS relies on influential scientific information (including a highly influential scientific assessment) to support a regulatory action or policy decision, it shall include in the administrative record for that action a certification explaining how the NPS has complied with the requirements of the OMB Peer Review Bulletin and the Information Quality Act.
IV.b. Information Quality Correction Procedures
IV.b.1. Complaints About Quality of Scientific and Scholarly Information
Persons who have complaints about NPS-provided scientific and scholarly information may avail themselves of any of the four methods outlined in D.O. 11B for notifying the NPS of their complaints. These four methods include informally discussing their complaint in person with park or program office staff, informally directing complaints about the quality of disseminated information to the superintendent of the park or manager of the program office responsible for disseminating the information, formally directing complaints about the quality of disseminated information by mail to the NPS Washington Administrative Program Center, or formally directing complaints to the Correspondence Control Unit by e-mail.
Persons submitting a formal request for correction of such information must submit all information identified in D.O. 11B, the technical components of which include the specific reasons for believing the information fails to meet OMB, DOI, or NPS standards, along with any supporting documentation; a detailed description of the specific material in question, including where the material is located (that is, publication title, date, and publication number, if any; the website and web page address; or other source where the material exists); and the specific recommendations for corrective action.
Upon receipt of a formal request for correction of information, NPS shall post the request on the appropriate park or program office Information Quality web page with a link to the NPS web page (www.nps.gov/notices.htm). All interim and final responses also shall be posted on the appropriate web page at the time they are sent to the complainant.
IV.b.2. Processing Complaints
The CCU will route formal complaints it receives regarding scientific or scholarly information to the park or office that disseminated the information and track response to assure that the response complies with the requirements of Director's Order 11B. The park or office receiving the complaint, regardless of the manner of receipt, will notify the complainant of receipt within 10 working days. The disseminating office will evaluate the complaint within 60 calendar days of the day it is received by the NPS, in accordance with the OMB guidelines, and notify the complainant as to whether the information has been corrected, deleted, or confirmed to be accurate. The Service will respond to additional complaints on the same subject according to the processes delineated in D.O. 11B.
IV.b.3. Appeals Process
If a complainant does not receive the notice or the response within the time frames described above, or wishes to appeal a determination of merit, or wishes to appeal the proposed correction of information, the complainant may appeal to the Director, National Park Service. Appeals must follow the requirements in D.O. 11B. All appeal requests shall be posted on the park or program office Information Quality web page upon receipt, with a link to the NPS Information Quality web page.
If the Director determines that an appeal of a determination has merit or the proposed correction of information has merit, the affected program office or park will be notified. The challenged information will be withdrawn, to the extent practicable, from the public domain and will not be used in any NPS decision-making process until it is corrected. The Director will make a decision on the final appeal within 60 calendar days.
V. THRESHHOLDS FOR PEER REVIEW
V.a. When is Peer Review Required?
Although peer review generally benefits all scientific and scholarly activities, existing workloads do not permit conducting the same intensity of peer review in every case. As a result, supervisors apply professional judgment to each activity to determine what level of peer review is required at any step in the implementation process. In general the need for peer review should be decided by a supervisor or program leader at least one level above the person who conducts the work. The following factors are examples of considerations used in making this professional judgment:
· the amount of funding involved;
· the number of researchers involved;
· the length of time involved;
· whether or not the activity involves significant experimentation or use of new technology or methods;
· the degree to which the information may contradict prior findings and results or is likely to be precedent-setting;
· potential for the activity to have applicability to other similar projects;
· the geographical scale involved;
· specific properties of, and values assigned to, natural and cultural resources involved;
· the potential for risk to both target and non-target resources;
· the potential for risk to visitor enjoyment or park operation;
· whether the act of conducting the activity or the probable results of the activity have potential to generate broad implications beyond the project, itself;
· the probable degree of controversy likely to be associated with the activity or the management, regulatory, or policy decision that it informs;
· the potential for societal and resource impacts associated with management, regulatory, or policy decisions that the information might influence; and/or
· the presence of legal mandates or unusual, unanticipated, or time-sensitive requirements that might preclude carrying out peer review.
V.b. Scientific and Scholarly Information Not Requiring Peer Review.
Activities that generally do not require independent peer review are those that are unlikely to be controversial or of a sensitive nature; involve routine, well-established practices; and pose no unacceptable impact to resources, visitor enjoyment, or operations. In such cases, administrative, NEPA, and Section 106 review may provide a sufficient alternative to peer review. When in doubt, however, it is better to conduct peer reviews.
Other information that does not require peer review under this guidance includes:
A. Items found in the list of exemptions under Section IX of the OMB Peer Review Bulletin:
· related to certain national security, foreign affairs, or negotiations involving international trade or treaties where compliance with this Bulletin would interfere with the need for secrecy or promptness;
· disseminated in the course of litigation, adjudication or a permit proceeding (including a registration, approval, licensing, or a site-specific determination), unless the agency determines that peer review is practical and appropriate and that the influential dissemination is scientifically or technically novel or likely to have precedent-setting influence on future adjudications and/or permit proceedings;
· a health or safety dissemination where the agency determines that the dissemination is time-sensitive (e.g., findings based primarily on data from a recent clinical trial that was adequately peer reviewed before the trial began);
· an agency regulatory impact analysis or regulatory flexibility analysis subject to interagency review under Executive Order 12866, except for underlying data and analytical models used;
· routine statistical information released by federal statistical agencies (e.g., periodic demographic and economic statistics) and analyses of these data to compute standard indicators and trends (e.g., unemployment and poverty rates);
· accounting, budget, actuarial, and financial information, including that which is generated or used by agencies that focus on interest rates, banking, currency, securities, commodities, futures, or taxes; or
· information disseminated in connection with routine rules that materially alter entitlements, grants, user fees, or loan programs, or the rights and obligations of recipients thereof.
B. Information published either in a peer-reviewed journal listed on the Thomson Scientific Master Journal List or in an official report of the National Academies of Sciences.
C. Routine statistical data used to compute standard indicators and trends that are gathered using methods based on well-established, peer-reviewed protocols and are analyzed and interpreted within the guidelines of the protocols.
D. An Environmental Impact Statement prepared as part of the NEPA process, except for underlying scientific and scholarly products and analytical models that provide information serving as the principal basis for a decision. In such cases, peer review that was conducted must have been appropriate to the context in which the information will be used or an additional peer review will be required.
E. Information performed by other federal agencies following the requirements of the OMB Peer Review Bulletin.
VI. PEER REVIEW POLICIES AND INSTRUCTIONS
VI.a. The Peer Review Process
The peer review process begins when the project manager (or supervisor) of a scientific or scholarly information activity, such as a proposal, nomination, progress report, review of previous studies, or product, identifies the need for peer review. Project managers or supervisors may be program scientists or scholars, resource managers, science coordinators, or other professionals.
The submission, which may be in the form of a proposal, study plan, inventory and monitoring plan, conceptual model, protocol, interim report, draft report, database, action plan, or scientific assessment should be peer reviewed in accordance with the following peer review process prior to funding, authorization, execution, publication, or incorporation into a decision:
A. The project manager assigns the
submission to a peer review manager, who is not the performer of the activity
but may be the project manager, to provide appropriate technical oversight and
ensure appropriate quality performance throughout the life of the activity. The
project manager consults appropriately with park, regional office, and
B. The peer review manager identifies the structure of the peer review and recruits one or more highly qualified individuals with technical expertise in the specific appropriate disciplines to provide peer review of each relevant component of the activity. Peer reviewers may be from within or outside the Park Service. Peer reviewers may have neither direct involvement with the activity nor a direct stake in the outcome of the review. The peer review manager may obtain recommendations for appropriately skilled reviewers from other knowledgeable people, in addition to using competent reviewers known to the peer review manager. Depending on the nature of the peer review, the peer review manager may serve as one peer reviewer if otherwise qualified. The reviewers carry out their tasks independently, without interaction among themselves, or together, depending on the structure of the review process.
The peer review manager may determine that an alternative peer review process may be accepted whenever it can be demonstrated that a reputable scientific body has already completed a formal peer review on a scientific or scholarly activity or product. (See Section IV.b. Scientific and Scholarly Information Not Requiring Peer Review.) Documentation of reliance on this alternative review process should be included in the peer review file.
C. The peer review manager ensures that peer review adequately addresses the technical merits of the methods, the professional techniques, data analysis, conclusions, and suggested management outcome scenarios and that activities requiring significant data analysis or sampling design receive additional review by peer reviewers with expertise in statistics and experimental design.
D. Because reviews of funding proposals must be performed in a timely manner, given the necessity to obligate funds within a given fiscal year and the short time usually available for obligating funds the peer review manager should schedule peer reviews to allow enough time for the development of cooperative agreements or contracts, including solicitor review. Therefore, it is critical that reviewers be selected who can provide fast turnaround as well as scientifically sound reviews.
E. Proposals solicited through competitive procurement or cooperative agreement processes will be subjected to a technical review process selected by the contracting/agreement officer upon recommendation of the contracting/agreement officer's technical representative. Typically, the peer review panel and the technical review panel are one and the same.
F. The peer review manager will negotiate with the performer of the activity regarding revisions to the information as recommended by peer reviews, will ensure that the project manager is notified of these recommended revisions, and will maintain a record of the reviews, activity revisions, and approvals in accordance with NPS records disposition requirements.
G. Once all appropriate changes and modifications have been completed in response to peer review comments, the peer review manager recommends technical acceptance of the activity in a written record documenting the process. The project manager has the final authority to either approve or reject an activity.
In those cases where peer reviewers recommend rejection of the activity, the peer review manager should write a summary of the reasons for the recommendation and be prepared to supply the author with reviewer comments.
VI.b. Peer Review Objectives
The peer review manager should define clearly the objectives of each peer review using questions that all reviewers are given and are expected to answer. Examples of potential peer review questions include:
· Are purpose for and methodology underlying the activity clearly presented, appropriate, and valid?
· Are appropriate field, laboratory, library, and statistical methods used for data collection, analysis, and characterization of uncertainty, and are they sufficiently described to allow for replication?
· Does the communication plan address communication needs and audiences appropriately?
· Is the combination of involved scientific, scholarly, or technical disciplines sufficient to adequately measure and test the hypothesis or to meet the activity’s stated objectives?
· Is the work original?
· Does the activity address significant, new, or novel circumstances?
· Does the activity include an appropriate literature review that puts the work in the context of previous work?
· Are conclusions reasonable and supported by data and other evidence and are alternative conclusions or counter arguments taken into account?
· Is written information well-written, logical, and understandable?
· What are the strengths, limitations, and potential usefulness to park managers and others of the activity?
· Does the performer of the activity have a level of recognized authority, technical and project management experience, and past record of success in the field of effort to adequately accomplish the stated objectives?
VI.c. Number of Peer Reviewers
One or more qualified individuals with technical expertise in the appropriate discipline(s) should be asked to provide peer review of informative scientific or scholarly information. Each review of “Influential Scientific Information” and “Highly Influential Scientific Assessment” must include a minimum of two independent peer reviewers. (See VI.k for procedures governing reviewer selection.)
VI.d. Peer Review Structure
Peer reviews may be structured in various ways, ranging from one or more individual reviewers (probably the most appropriate model for most NPS endeavors) to review panels or advisory committees. The choice will depend on the novelty and complexity of the scientific information and whether the desired result is individual reviews or a consensus among reviewers. In determining the structure of a given review, the NPS must observe the requirements of the Federal Advisory Committee Act (FACA), where applicable. If the review does not involve convening a group of reviewers as a committee, the review is probably not subject to FACA. Consensus panels may not be subject to FACA if established, controlled, and managed by an outside organization, such as a contractor.
VI.e. Peer Review Manager
In general, reviews for information that will benefit from peer review but are not deemed “influential” or “highly influential” will be managed by a park, region, or other office peer review manager following the guidelines in this document. The peer review manager for “influential” and “highly influential” NPS scientific or scholarly information activities will be selected by the responsible Senior Executive Service (SES) manager.
VI.f. Required Disclaimer
The peer review manager must ensure that each product distributed for peer review under this chapter is imprinted with the following disclaimer:
This information is distributed solely for the purpose of pre-dissemination peer review under applicable information quality guidelines. It has not been formally disseminated by the National Park Service. It does not represent and should not be construed to represent any National Park Service determination or policy.
VI.g. Disposition of Peer Review Comments
The peer review manager must make all review comments available to the authors of the work being reviewed, usually without identifying the reviewer’s identity. Authors may disagree with peer reviewer comments on scientific or scholarly grounds. Where comments of a substantial nature are not incorporated in a revision:
VI.h. Peer Review Costs
Traditionally, peer review is carried out at no cost. The NPS recognizes that increasing demands for reviewer time and associated travel may occasionally necessitate payment for reviewer time and for associated travel, especially for review of “influential” and “highly influential” activities.
VI.i. Peer Review Confidentiality
Peer review must be conducted in a manner that respects confidential business information and intellectual property. Reviewers must agree to not disclose or divulge any results or conclusions, or make any public statements regarding the reviewed information before it is published and released.
VI.j. Peer Review Documentation
Each peer review manager must maintain a written administrative record for all formal peer reviews.
A. Documentation must be sufficient for an uninvolved person to understand the process used and any changes made as a result of the review.
B. Peer review records will be appropriately maintained under direction of the responsible SES manager, who should describe where and how long each administrative record remains in the working files and identify the archives where the record will be stored.
C. At a minimum, the records must include:
· The name and position of the responsible peer review manager;
· The name, affiliation, and pertinent qualifications of each peer reviewer;
· Applicable confidentiality and conflict of interest documentation for each peer reviewer (see Appendix B);
· The objectives and structure of the peer review;
· A copy of all peer reviewer comments, and
· Explanation of how peer review comments were addressed.
VI.k. Selection of Peer Reviewers
A. Peer reviewers must be true peers who are selected on the basis of their relevant scientific, scholarly, and technical expertise and their objectivity. They must not be associated directly with the work being reviewed. Reviewers should represent a range of viewpoints, especially where legitimate scientific controversy exists. Reviewers should be able to ensure that the information is effectively presented with the intended audience in mind and be cognizant of controversial or high-visibility issues that may be relevant to public policy;
B. Potential reviewers may be recommended by NPS staff members, scientific and professional societies, members of the public, or authors of scientific and scholarly products, but actual selection of reviewers rests solely with the peer review manager and should be based on specific criteria for peer reviewers. Supervisors of potential NPS reviewers are expected to encourage their employees to serve whenever feasible.
D. If standing panels are created to perform consistent peer review, membership should rotate across the pool of qualified reviewers.
E. For all influential scientific information (including highly influential scientific assessments), peer review managers must ensure that all reviewers comply with federal ethics requirements and must adopt or adapt the National Academy of Sciences policy for committee selection with respect to evaluating the potential for conflicts (e.g., those arising from investments; employer and business affiliations; grants, contracts and consulting income). The NAS policy is available at: http://www.nationalacademies.org/coi/bi-coi_form-0.pdf. All peer reviewers for influential NPS information products must agree to be bound by the strictest scientific ethics and sign the NPS Conflict-of-Interests and Confidentiality statement (see Appendix B). For scientific information relevant to specific regulations, the NPS shall examine a reviewer’s financial ties to regulated entities (e.g. businesses), other stakeholders, and the NPS.
F. For highly influential scientific assessments, the use of formal peer reviewers employed by the National Park Service is prohibited unless the reviewer is employed only for the purpose of conducting the peer review (i.e., special government employees). (This restriction does not exclude informal, internal reviews in addition to the formal peer review process.)
i. The only exception to this prohibition would be the rare case where the NPS determines, using the criteria developed by the National Academy of Sciences for evaluating use of “employees of sponsors,” that a premier government specialist is not in a position of management or policy responsibility and possesses essential expertise that cannot be obtained elsewhere.
ii. To be eligible for this exception, the specialist must be employed by a different park, region, or office of the NPS than is disseminating the scientific information. The use of such an exception must be documented in writing and approved, on a non-delegable basis, by the Secretary or Deputy Secretary prior to the specialist’s appointment. In addition, peer review managers shall avoid repeated use of the same reviewer on multiple assessments unless his or her participation is essential and cannot be obtained elsewhere.
VI.l. Peer Reviewer Anonymity
Peer review managers should clearly inform prospective peer reviewers whether –
A. Reviewer names will be disclosed and comments will be specifically attributed;
B. Reviewer names will be disclosed but comments will not be specifically attributed;
C. Reviewer names will not be disclosed but comments will be disclosed; or
D. Neither reviewer names nor comments will be disclosed.
To the extent information about a peer reviewer (name, credentials, affiliation) will be disclosed along with his or her comments or analysis (see also Section VI.q), the peer review manager must comply with the requirements of the Privacy Act, 5 U.S.C. § 522a as amended, and OMB Circular A-130, Appendix I, 61 Fed. Reg. 6428 (February 20, 1996) to establish appropriate routine uses in a published System of Records Notice.
VI.n. Public Participation
Whenever feasible and appropriate, the peer review manager must make a draft of each highly influential scientific assessment available to the public for comment at the same time it is submitted for peer review (or during the peer review process).
A. The office producing the assessment will add an explicit prefatory statement to the disseminated material that states the material being disseminated to the public is concurrently being submitted for peer review and, therefore, may be subject to change and then reiterate this prefatory statement when sponsoring any public meeting where interested members of the public can provide the peer reviewers with oral presentations on scientific or scholarly issues of concern to the public.
B. When employing a public comment process as part of the peer review, the peer review manager shall, whenever practical, provide peer reviewers with access to public comments that address significant scientific, scholarly, or technical issues. To ensure that public participation does not cause undue delay, the peer review manager shall specify time limits for public participation throughout the peer review process.
VI.p. Peer Review Plans for Influential Scientific and Scholarly Information
Each SES manager will post a list of the manager’s Park, Region, or Office’s planned and ongoing influential scientific and scholarly information along with associated peer review plans on an Information Quality webpage accessible to the public. This webpage shall be updated at least every six months. The webpage will provide contact addresses to allow the public to comment on the adequacy of the peer review plans. Each peer review plan shall include:
A. The title, subject and purpose of the planned scientific information along with an electronic link to the information (when available) and a contact person.
B. The timing of the review (including deferrals).
C. The structure of the review (e.g., individual letters, panel) including the number of planned reviewers and the primary disciplines or expertise needed.
D. The method for reviewer selection and whether the public can nominate prospective reviewers.
E. Whether there will be opportunities for the public to comment on the information to be peer reviewed, and if so, how and when these opportunities will be provided.
F. Whether the agency will provide significant and relevant public comments to the peer reviewers before they conduct their review.
VI.q. Peer Review Reports for Scientific and Scholarly Information
The peer review manager shall prepare a report that describes the nature of the review along with findings and conclusions. The report shall either include a verbatim copy of each reviewer’s comments (with or without specific attribution) or represent the views of the group as a whole, including any dissenting views. The report shall contain the names of reviewers and their organizational affiliations. Reviewers must be notified in advance regarding the extent of disclosure and attribution. For influential scientific information (including highly influential scientific assessment), NPS shall disseminate the peer review report (including the peer reviewer information) on a webpage accessible to the public. In addition, where scientific or scholarly information is specifically used to inform a rulemaking, the peer review report shall be discussed in the preamble to that rulemaking and included in the administrative record for any related NPS action.
VI.r. NPS Annual Peer Review Report
With respect to influential scientific information and highly influential scientific assessments, each SES manager shall provide appropriate information for compilation into the Service’s report to the Office of the Secretary by December 1 of each year. This information should include the following:
A. The number of peer reviews conducted for influential scientific information and highly influential scientific assessments;
B. The number of times alternative procedures (e.g., relying on refereed journal reviews) were invoked;
C. The number of times waivers or deferrals were invoked (and in the case of deferrals, the length of time elapsed between the deferral and the peer review);
D. Any decision to appoint a reviewer pursuant to any exception to the applicable independence or conflict of interest standards, including determinations by the Secretary under provisions of the OMB Peer Review Bulletin;
E. The number of peer review panels that were conducted in public and the number that allowed public comment;
F. The number of public comments provided on the agency’s peer review plans; and
G. The number of peer reviewers that the agency used that were recommended by professional societies.
The Associate Director, Natural Resource Stewardship and Science, will consolidate the SES managers’ reports into the Service report and will submit the Service report to the Department.
VII.a. The Director
The Director addresses appeals of responses to information quality complaints. The Director may waive or defer some or all of the peer review requirements where warranted by a compelling rationale. If the Director defers the peer review requirements before dissemination, peer review must be conducted as soon as practicable.
VII.b. Associate and Regional Directors
The Associate and Regional Directors are responsible for ensuring that their staffs implement these policies and procedures.
VII.c. Senior Executive Service Official Responsible for Conducting Scientific Reviews
The Senior Executive Service manager directly responsible for a park, region, or directorate will designate an official (the SES manager) to have overall responsibility for implementation of these guidelines. Where the responsibility for a specific action or actions may not be delegated to a subordinate, this should be so stated.
VII.d. Project Manager
The project manager, as the overseer of an activity, applies these guidelines to determine whether or not the activity requires peer review and, for an activity that does require peer review, is the person who designates the peer review manager and who oversees implementation of these guidelines with respect to the specific activity.
VII.c. Peer Review Manager
The peer review manager:
A. Determines what level of peer review is required;
B. Establishes the peer review process by setting objectives, structure, and timeframe for completing the review, as appropriate, by:
i. Submitting the information to an appropriate refereed journal or other, credible, professional peer review program.
ii. If the information is deemed “influential” or “highly influential,” conducting a formal peer review using the process outlined in this guidance.
iii. If the information is not deemed “influential” or “highly influential,” continuing a peer review process according to these guidelines.
C. Provides review findings to authors.
D. Ensures that the authors address comments adequately and fairly.
E. Ensures that proper records are kept.
F. Oversees independent entities or contractors commissioned to manage the peer review process.
Parks and offices responsible for releasing information to the public must follow the requirements of D.O. 11B by:
VII.e. The NPS Correspondence Control Unit (CCU) in the Washington Administrative Program Center is responsible for:
VII.f. Chief, Office of Policy
The Chief, Office of Policy ensures that Servicewide information quality policies and procedures are available on the web, including links to regional, network, office, and park information quality web pages that are maintained by the respective officials responsible for conducting scientific reviews.
The following definitions apply to this document:
Conflict of Interest. Any financial or other interest which conflicts with the actions or judgments of a reviewer because it (1) could significantly impair the individual’s objectivity or (2) could create an unfair competitive advantage for any person or organization. National Park Service employees must avoid conflicts of interest and the appearance of a lack of impartiality, in accordance with 18 U.S.C. 208 and 5 CFR 2635 Subparts D and E.
Dissemination. NPS initiated or sponsored distribution of information to the public. Dissemination does not include distribution limited to government employees or NPS contractors or grantees; intra- or inter-agency use or sharing of government information; and responses to requests for agency records under the Freedom of Information Act, the Privacy Act, the Federal Advisory Committee Act, or other similar law. This definition also does not include distribution limited to correspondence with individuals or persons, press releases, archival records, public filings, subpoenas or adjudicative processes.
Fabrication. Making up data or results and recording or reporting them.
Falsification. Manipulating research materials, equipment, or processes; or changing or omitting data or results such that the research is not accurately represented in the research record.
Highly Influential Scientific or Scholarly Assessment. A scientific or scholarly assessment that could have a potential impact of more than $500 million in any year, or is novel, controversial, or precedent-setting or has significant interagency interest. Such an assessment is a subset of “influential scientific or scholarly information.”
Independent peer reviewer. An internal or external peer reviewer who the NPS determines to meet the requirements of section II (3)(c) of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664-2677), and is not associated with the information under review –
1. by direct involvement in its development;
2. indirectly, by significant consultation during development or by supervising the personnel who conducted it; or
3. by significant personal relationship to persons directly involved in development of the information.
Influential Scientific or Scholarly Information. Scientific or scholarly information that the Service reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions but that does not meet the definition of “highly influential scientific assessment.” Influential scientific information is a subset of “informative scientific information.” Information is influential in determining important policies or decisions if the same decision would be difficult to reach in the absence of the information. Information has a clear and substantial impact when the specific information serves as the principal basis for a decision that affects significant numbers of private sector entities outside parks or not associated with NPS assistance activities.
Information. Any communication or representation of knowledge (e.g., fact or data), in any medium or form, including textual, numerical, graphic, cartographic, narrative, or audiovisual forms. This definition includes information that an office disseminates from a web page, but does not include the provision of hyperlinks to information that others disseminate. This definition does not include opinions, where the office's presentation makes it clear that what is being offered in someone's opinion rather than fact or the office's views.
Informative Scientific or Scholarly Information. Scientific or scholarly information that serves to inform scientific, scholarly, and management awareness and decision-making generally but that does not provide the sole or major component of information used in decision-making and does not, by itself, lead to a change in the direction of decision-making or to a decision that creates a clear and substantial impact on important public policies or private sector decisions.
Integrity. Systematically and objectively developed and maintained information that is protected from unauthorized access or revision to ensure that the information reflects the circumstances of its acquisition and is not compromised through corruption or falsification.
Inventory. Process of acquiring, managing, and analyzing information on park resources, including but not limited to the presence, distribution, and condition of plants, animals, soils, water, air, natural features, biotic communities, cultural resources, and natural and cultural resource processes.
Mitigation. Maintenance of the existing form and integrity of natural and cultural resource systems or system components, consistent with park management objectives, in the face of harm or potential harm from human activities within or outside the park. Mitigation is also the conversion of a resource, altered by human activity, to a more functional or desired state consistent with management objectives. As such, mitigation encompasses preservation and restoration activities.
Monitoring. Systematic collection and analysis of natural and cultural resource data at regular intervals, in perpetuity, to predict or detect natural and human-induced changes, and to provide the basis for appropriate management response.
Resource Management. Understanding of natural and cultural resource processes and human-induced effects; mitigation of potential and realized effects; monitoring for ongoing or future trends; protecting existing natural physical, biological, and cultural resources, systems, and processes; and interpreting these resources, systems, and processes to people. This function can be broken down into seven major activities: research, inventory, monitoring, mitigation, protection, interpretation, and administration.
Objectivity. "Objectivity" involves two distinct elements, presentation and substance. It includes whether disseminated information is being presented in an accurate, clear, complete, and unbiased manner. This involves whether the information is presented within a proper context. In addition, "objectivity" involves a focus on ensuring accurate, reliable, and unbiased information. In a scientific, financial, or statistical context, the original and supporting data shall be generated, and the analytic results shall be developed, using sound statistical and research methods.
Peer Review. A quality control process in which the scientific merit of scientific information is critically evaluated by independent peers, meaning persons who are not associated directly or indirectly with the information under review and whose background and expertise puts them on par technically and scientifically with the authors of the information.
Plagiarism. The appropriation of another person’s ideas, processes, results, or words without giving appropriate credit.
Quality. An encompassing term comprising utility, objectivity, and integrity. Therefore, the guidelines sometimes refer to these three statutory terms, collectively, as "quality."
Reporting. Dissemination or disclosure of the results of scientific or scholarly activities. Dissemination and disclosure may be oral or in any media, including print and digital media.
Research. Investigation aimed at the discovery and interpretation of facts, the revision of accepted theories in light of new facts, or the development of practical applications of such new revised theories.
Research Misconduct. Fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results. Research misconduct does not include honest error or differences of opinion. (This definition is quoted from The Federal Policy on Research Misconduct [65 FR 76260-76264].)
Science. Knowledge obtained and tested through the use of the scientific method. Science may also include the observation and classification of facts with the goal of establishing verifiable knowledge derived through induction and hypothesis.
Scientific or Scholarly Activities. Activities involving inventorying, monitoring, experimentation, study, research, modeling, and scientific assessment. Scientific or scholarly activities are conducted in a manner specified by standard protocols and procedures and include any of the physical, biological, cultural, or social sciences as well as landscape architecture, engineering, and mathematics that employ scientific or scholarship methods. Inspections for regulatory compliance and resulting records are not included, even though they may require use of complementary methods.
Scientific or Scholarly Assessment. Scientific or scholarly information constructed to provide an evaluation of a body of scientific or technical knowledge, typically by synthesizing multiple factual inputs, data, models, assumptions, and/or by applying best professional judgment to bridge uncertainties in the available information.
Scientific or Scholarly Information. Scientific and scholarship outputs that consist of proposals, hypotheses, models, written documents, records of all kinds, and assessments. This definition does not include opinions, where the presentation of an output makes clear that what is being offered is someone’s opinion rather than fact or the agency’s views. Scientific or scholarly information includes all of the following:
(1) Factual inputs, data, models, analyses, technical information, or scientific or scholarly assessments based on the behavioral, cultural, and social sciences, health and medical sciences, life and earth sciences, engineering, or physical sciences.
(2) Any communication or representation of knowledge such as facts or data, in any medium or form, including textual, numerical, graphic, cartographic, narrative, or audiovisual forms.
(3) Information that an agency disseminates from a web page, but does not include the provision of hyperlinks to information that others disseminate.
Scientific Method. A method of research in which a question is identified, relevant data are gathered, a hypothesis is formulated from these data, and the hypothesis is empirically tested.
Scientific or Scholarly Misconduct. Fabrication, falsification, or plagiarism in proposing, performing or reviewing scientific or scholarly activities and their products.
Utility. Usefulness of the information to its intended users, including the public. In assessing the usefulness of information that the NPS disseminates to the public, the office needs to consider the uses of the information not only from the perspective of the office, but also from the perspective of the public. As a result, when transparency of information is relevant for assessing the information's usefulness from the public's perspective, the office must take care to ensure that transparency has been addressed in its review of the information.
IX. LEGAL EFFECT
guidelines are intended only to improve the internal management of the National
Park Service relating to information quality. Nothing in these guidelines is
intended to create any right or benefit, substantive or procedural, enforceable
by law or equity by a party against the
NATIONAL PARK SERVICE PEER REVIEW TEMPLATE
The following material provides potential questions and rankings that may be appropriate to ask peer reviewers to address. The peer review manager should not feel constrained by these suggestions and should add, modify, or delete questions or text, as appropriate to the specific informative or influential scientific or scholarly product under review.
The disclaimer at the foot of each page is required by OMB Final Information Quality Bulletin for Peer Review (70 FR 2664-2677).
Thank you for agreeing to provide your experience and expertise in evaluating this information for the National Park Service. Your efforts help enhance the quality of this NPS generated scientific/scholarly product and increase the credibility of decisions managers may make based on this information. Your review ranking and comments will be provided to the author(s) and others preparing this material. However, your name will remain confidential unless you specifically give permission to provide your name or we specifically tell you that your name will not remain confidential.
Title of product: ______________________________________________________________
Reviewer # ______
I give permission to include my name on reviews provided to the author(s) (Optional statement) Signed by ________________________________________________________
1. Are the purpose and methodology of this information clearly presented, appropriate and valid? Are appropriate methods used for data collection, analysis, and characterization of uncertainty, and are they sufficiently described to allow for replication?
Strongly disagree ↔ Strongly agree*
Methodology is appropriate and valid
*Double click on each box to select ‘checked’ as the default value
2. Is the work novel? Does the information present significant, new or novel findings?
Strongly disagree ↔ Strongly agree*
The work and findings are significant
The work and findings are new
The work and findings are novel
*Double click on each box to select ‘checked’ as the default value
3. Does this information include an appropriate literature review that puts the work in the context of previous work?
Strongly disagree ↔ Strongly agree*
An appropriate literature review is included
*Double click on each box to select ‘checked’ as the default value
4. Are the conclusions reasonable and supported by the data and other evidence? Are alternative conclusions or counter arguments discussed and taken into account?
Strongly disagree ↔ Strongly agree*
Conclusions are reasonable & supported
*Double click on each box to select ‘checked’ as the default value
5. Is the information well-presented, logical, and understandable?
Strongly disagree ↔ Strongly agree*
Paper is well-presented
*Double click on each box to select ‘checked’ as the default value
6. Please comment on the strength and limitations of the overall product and its potential usefulness to managers and others.
Strongly disagree ↔ Strongly agree*
Product is strong & useful to managers & others
*Double click on each box to select ‘checked’ as the default value
7. Other concerns or comments?
This information is distributed solely for the purpose of pre-dissemination peer review under applicable information quality guidelines. It has not been formally disseminated by the National Park Service. It does not represent and should not be construed to represent National Park Service determination or policy.
conflict-of-interests and confidentiality statement for INFLUENTIAL nps scientific INFORMATION reviewers
1. Potential conflicts-of-interests: As a National Park Service scientific information reviewer, you must declare any potential conflict situations. Please read the examples of potentially biasing affiliations or relationships on the second (or back) page and identify any concerns to the NPS peer review manager.
2. Your obligation to maintain confidentiality: A peer reviewer for a potentially influential NPS information product must agree to be bound by the strictest scientific ethics. For this reason, you must not copy, quote, or otherwise use or disclose to anyone any material revealed during this review process. If you believe a colleague can make a substantial contribution to the review, please obtain permission of the NPS peer review manager before disclosing any portion of the contents of the material under review.
3. Availability of the reviewers names and identities: Unless otherwise agreed upon, the National Park Service will be free to provide a list of peer reviewer names and affiliations to the author and the public. The NPS may, also, provide the author and the public with access to the reviewer’s comments and rankings, either verbatim or in a summary report. The Park Service may or may not directly attribute comments, reviews, or rankings to a specific reviewer.
I have read the list of affiliations and relationships that could prevent my participation in matters involving such individuals and instititions. To the best of my knowledge, I have no affiliation or relationship that would prevent me from performing my review duties. I understand that I must contact the NPS peer review manager if a conflict exists or arises during my service. I further understand that I must sign and return this Conflict Statement to the peer review manager before I may serve.
I will not divulge or use any confidential information that I may become aware of during my service.
I understand my identity as a reviewer of specific NPS scientific or scholarly products and/or my comments may be made available to the author and the public. Unless I am told otherwise, I understand my comments will not be directly attributed to me.
Reviewer’s Name (Please print or type) __________________________________________________________________
Reviewer’s Signature ___________________________________ Date_______________________________
Name of product: __________________________________________________________________________
Reviewer’s Position and Affiliation ___________________________________________________________
INFLUENTIAL nps scientific products reviewers
(First bullet in Item 3 of this section explains the role of the italicized entries)
1. Your affiliations with the product’s author(s)/creator(s) institution
You may have a conflict if you have/hold/are:
· Current employment at the park, division, or institution (including consulting or advisory arrangement);
· Previous employment with the park, division, or institution within the last 12 months;
· Being considered for employment at the park, division, or institution;
· Formal or informal re-employment arrangement with the park, division, or institution;
· Ownership of securities of firms involved in the product;
· Any office, governing board membership, or relevant committee chairpersonship with the park, division, institution, or their cooperating associations;
· Current enrollment as a student at the institution (applicable when an author/creator is affiliated with an educational institute);
· Received and retained an honorarium or award from the institution within the last 12 months.
1. Your relationship with an author, investigator, project director, or other person who has a personal or financial interest in the product.
3. Your other affiliations or relationships
The National Park Service relies on both internal and external reviewers to ensure the quality of NPS generated and sponsored scientific and scholarly information products and to validate the credibility of decisions land managers may make based on these documents. We invite you, as a specialist to review the following information product, _______________________________________________, as we believe you can provide important experience and knowledge about this topic. The XX page manuscript will be accompanied by a form that asks you to rank the paper on six qualities and provide any additional comments you believe will be helpful. We ask that reviews be returned by _____.
I will be coordinating the reviews. Please let me know if you are able to assist us in this important task and I will e-mail you the material. Please indicate to me and provide a mailing or delivery address if you prefer receiving a paper copy of the manuscript and review form.
We recognize that professional reviews take time from your busy schedule and appreciate your willingness to consider helping with this important process.