The Secretary reviews an application by an SEA or consortium of SEAs seeking innovative assessment demonstration authority consistent with § 200.104(c) based on the following selection criteria:
(a) Project narrative. The quality of the SEA's or consortium's plan for implementing the innovative assessment demonstration authority. In determining the quality of the plan, the Secretary considers—
(1) The rationale for developing or selecting the particular innovative assessment system to be implemented under the demonstration authority, including—
(i) The distinct purpose of each assessment that is part of the innovative assessment system and how the system will advance the design and delivery of large-scale, statewide academic assessments in innovative ways; and
(ii) The extent to which the innovative assessment system as a whole will promote high-quality instruction, mastery of challenging State academic standards, and improved student outcomes, including for each subgroup of students described in section 1111(c)(2) of the Act;
(2) The plan the SEA or consortium, in consultation with any external partners, if applicable, has to—
(i) Develop and use standardized and calibrated tools, rubrics, methods, or other strategies for scoring innovative assessments throughout the demonstration authority period, consistent with relevant nationally recognized professional and technical standards, to ensure inter-rater reliability and comparability of innovative assessment results consistent with § 200.105(b)(4)(ii), which may include evidence of inter-rater reliability; and
(ii) Train evaluators to use such strategies, if applicable; and
(3) If the system will initially be administered in a subset of schools or LEAs in a State—
(i) The strategies the SEA, including each SEA in a consortium, will use to scale the innovative assessment to all schools statewide, with a rationale for selecting those strategies;
(ii) The strength of the SEA's or consortium's criteria that will be used to determine LEAs and schools that will initially participate and when to approve additional LEAs and schools, if applicable, to participate during the requested demonstration authority period; and
(iii) The SEA's plan, including each SEA in a consortium, for how it will ensure that, during the demonstration authority period, the inclusion of additional LEAs and schools continues to reflect high-quality and consistent implementation across demographically diverse LEAs and schools, or contributes to progress toward achieving such implementation across demographically diverse LEAs and schools, including diversity based on enrollment of subgroups of students described in section 1111(c)(2) of the Act and student achievement. The plan must also include annual benchmarks toward achieving high-quality and consistent implementation across participating schools that are, as a group, demographically similar to the State as a whole during the demonstration authority period, using the demographics of initially participating schools as a baseline.
(b) Prior experience, capacity, and stakeholder support.
(1) The extent and depth of prior experience that the SEA, including each SEA in a consortium, and its LEAs have in developing and implementing the components of the innovative assessment system. An SEA may also describe the prior experience of any external partners that will be participating in or supporting its demonstration authority in implementing those components. In evaluating the extent and depth of prior experience, the Secretary considers—
(i) The success and track record of efforts to implement innovative assessments or innovative assessment items aligned to the challenging State academic standards under section 1111(b)(1) of the Act in LEAs planning to participate; and
(ii) The SEA's or LEA's development or use of—
(A) Effective supports and appropriate accommodations consistent with § 200.6(b) and (f)(1)(i) and section 1111(b)(2)(B)(vii) of the Act for administering innovative assessments to all students, including English learners and children with disabilities, which must include professional development for school staff on providing such accommodations;
(B) Effective and high-quality supports for school staff to implement innovative assessments and innovative assessment items, including professional development; and
(C) Standardized and calibrated tools, rubrics, methods, or other strategies for scoring innovative assessments, with documented evidence of the validity, reliability, and comparability of annual summative determinations of achievement, consistent with § 200.105(b)(4) and (7).
(2) The extent and depth of SEA, including each SEA in a consortium, and LEA capacity to implement the innovative assessment system considering the availability of technological infrastructure; State and local laws; dedicated and sufficient staff, expertise, and resources; and other relevant factors. An SEA or consortium may also describe how it plans to enhance its capacity by collaborating with external partners that will be participating in or supporting its demonstration authority. In evaluating the extent and depth of capacity, the Secretary considers—
(i) The SEA's analysis of how capacity influenced the success of prior efforts to develop and implement innovative assessments or innovative assessment items; and
(ii) The strategies the SEA is using, or will use, to mitigate risks, including those identified in its analysis, and support successful implementation of the innovative assessment.
(3) The extent and depth of State and local support for the application for demonstration authority in each SEA, including each SEA in a consortium, as demonstrated by signatures from the following:
(i) Superintendents (or equivalent) of LEAs, including participating LEAs in the first year of the demonstration authority period.
(ii) Presidents of local school boards (or equivalent, where applicable), including within participating LEAs in the first year of the demonstration authority.
(iii) Local teacher organizations (including labor organizations, where applicable), including within participating LEAs in the first year of the demonstration authority.
(iv) Other affected stakeholders, such as parent organizations, civil rights organizations, and business organizations.
(c) Timeline and budget. The quality of the SEA's or consortium's timeline and budget for implementing the innovative assessment demonstration authority. In determining the quality of the timeline and budget, the Secretary considers—
(1) The extent to which the timeline reasonably demonstrates that each SEA will implement the system statewide by the end of the requested demonstration authority period, including a description of—
(i) The activities to occur in each year of the requested demonstration authority period;
(ii) The parties responsible for each activity; and
(iii) If applicable, how a consortium's member SEAs will implement activities at different paces and how the consortium will implement interdependent activities, so long as each non-affiliate member SEA begins using the innovative assessment in the same school year consistent with § 200.104(b)(2); and
(2) The adequacy of the project budget for the duration of the requested demonstration authority period, including Federal, State, local, and non-public sources of funds to support and sustain, as applicable, the activities in the timeline under paragraph (c)(1) of this section, including—
(i) How the budget will be sufficient to meet the expected costs at each phase of the SEA's planned expansion of its innovative assessment system; and
(ii) The degree to which funding in the project budget is contingent upon future appropriations at the State or local level or additional commitments from non-public sources of funds.
(d) Supports for educators, students, and parents. The quality of the SEA or consortium's plan to provide supports that can be delivered consistently at scale to educators, students, and parents to enable successful implementation of the innovative assessment system and improve instruction and student outcomes. In determining the quality of supports, the Secretary considers—
(1) The extent to which the SEA or consortium has developed, provided, and will continue to provide training to LEA and school staff, including teachers, principals, and other school leaders, that will familiarize them with the innovative assessment system and develop teacher capacity to implement instruction that is informed by the innovative assessment system and its results;
(2) The strategies the SEA or consortium has developed and will use to familiarize students and parents with the innovative assessment system;
(3) The strategies the SEA will use to ensure that all students and each subgroup of students under section 1111(c)(2) of the Act in participating schools receive the support, including appropriate accommodations consistent with § 200.6(b) and (f)(1)(i) and section 1111(b)(2)(B)(vii) of the Act, needed to meet the challenging State academic standards under section 1111(b)(1) of the Act; and
(4) If the system includes assessment items that are locally developed or locally scored, the strategies and safeguards (e.g., test blueprints, item and task specifications, rubrics, scoring tools, documentation of quality control procedures, inter-rater reliability checks, audit plans) the SEA or consortium has developed, or plans to develop, to validly and reliably score such items, including how the strategies engage and support teachers and other staff in designing, developing, implementing, and validly and reliably scoring high-quality assessments; how the safeguards are sufficient to ensure unbiased, objective scoring of assessment items; and how the SEA will use effective professional development to aid in these efforts.
(e) Evaluation and continuous improvement. The quality of the SEA's or consortium's plan to annually evaluate its implementation of innovative assessment demonstration authority. In determining the quality of the evaluation, the Secretary considers—
(1) The strength of the proposed evaluation of the innovative assessment system included in the application, including whether the evaluation will be conducted by an independent, experienced third party, and the likelihood that the evaluation will sufficiently determine the system's validity, reliability, and comparability to the statewide assessment system consistent with the requirements of § 200.105(b)(4) and (9); and
(2) The SEA's or consortium's plan for continuous improvement of the innovative assessment system, including its process for—
(i) Using data, feedback, evaluation results, and other information from participating LEAs and schools to make changes to improve the quality of the innovative assessment; and
(ii) Evaluating and monitoring implementation of the innovative assessment system in participating LEAs and schools annually.
[81 FR 88969, Dec. 8, 2016]