Instructional Materials Development (IMD)

• Instructional Materials for Students—supports the creation and substantial revision of comprehensive curricula and supplemental instructional materials that are research-based; enhance classroom instruction, preK-12; and reflect standards for science, mathematics, and technology education developed by national professional organizations.

• Dissemination and Implementation Sites—builds the capacity of the educational system to use research-based instructional materials and provides a range of services to interested schools, districts, and states that address issues related to awareness, selection, adoption, and implementation of these materials.

• Assessment—supports the creation of tools for assessing student learning that are tied to nationally developed standards and reflect the most current thinking on how students learn mathematics and science; projects can also focus on teacher assessment, assessment in informal settings, and assistance to schools and districts in implementing new assessments.

• Applied Research[1]—supports the research for development of Elementary, Secondary, and Informal Education (ESIE) programs, provides feedback for strengthening the portfolio, and identifies possible new directions in instructional materials, assessment, teacher education or informal programs.  

Proposals may be submitted for projects in any field of science, technology, or mathematics (STM) typically supported by NSF.  

[1] While Applied Research projects can address areas within all ESIE programs (Instructional Materials Development [IMD], Teacher Enhancement [TE], and Informal Science Education [ISE]), proposals to conduct applied research must be submitted to IMD.

Goals and Objectives.  Science, technology, and mathematics (STM) education, pre-Kindergarten through grade 12 (preK-12), lays the foundation of knowledge and skills needed by future researchers, educators, and technologists; students pursuing post-secondary education in other disciplines; and individuals directly entering the technological workforce.  The Division of Elementary, Secondary, and Informal Education (ESIE) supports the National Science Foundation’s mission of providing leadership and promoting development of the infrastructure and resources needed to improve preK-12 STM education throughout the United States.  

ESIE’s comprehensive and coherent, research-based program portfolio develops the nation’s capacity to support high-quality STM education.  Innovative instructional materials and student assessments, as well as new models for the delivery of teacher professional development contribute to STM classroom environments that enable all students to achieve their full potential.  Moreover, ESIE’s informal learning opportunities via media, exhibit, and community-based programs increase scientific and technological literacy, as well as develop life-long learning skills that benefit students of all ages.  All ESIE programs contribute to development of a knowledge base that informs practice and of partnerships that leverage expertise and other resources of major education stakeholders nationwide, including higher education, state and local education agencies, school districts, informal science education institutions, and industry. 

Program Overviews.  Programs administered by ESIE in FY 2002 include:

• Instructional Materials Development (IMD)  -- IMD develops high-quality, research-based instructional and assessment materials for students that enhance knowledge, thinking skills, and problem-solving abilities of all students, as well as incorporate recent advances in disciplinary content, research on teaching and learning, and instructional technologies.  IMD materials are intended to be implemented nationwide and address learning in diverse settings. 

• Teacher Enhancement (TE) – TE develops models for strengthening skills of the teacher workforce by expanding and deepening their understanding of content, pedagogy, and instructional technologies; by heightening awareness and deepening understanding of the diverse learning needs of students; by grounding continued professional development in the context of school structure and organization; and by developing a cadre of teachers and administrators who can effectively lead the reform of STM education.

• Centers for Learning and Teaching (CLT) – CLT focuses on the advanced preparation and professional development of STM practitioners and educators, as well as establishment of complex, meaningful partnerships among education stakeholders, especially Ph.D.-granting institutions, school systems, and informal education performers.  Its goals are to rebuild and diversify the national infrastructure for STM education; to increase the number of K-16 educators capable of delivering high-quality content, instruction, and assessment; and to provide substantive research opportunities into the nature of learning, teaching strategies, as well as education reform policies and outcomes.

• Presidential Awards for Excellence in Mathematics and Science Teaching (PAEMST) – PAEMST, administered on behalf of the White House, recognizes exemplary careers of elementary and secondary teachers of mathematics and science.

• Informal Science Education (ISE) – ISE provides stimulating experiences for STM learning outside of formal classroom environments through media, exhibits, and community-based programming.  Its goals are to increase understanding of, and participation in, STM disciplines by individuals of all ages; to establish linkages between informal and formal education; and to stimulate parents and others to support their children’s STM learning endeavors and to become informed proponents for high-quality, universally available STM education. 

• Advanced Technological Education (ATE) – Jointly managed by the Division of Undergraduate Education (DUE) and ESIE, ATE promotes improvement in technological education at the undergraduate and secondary school levels by supporting curriculum development; preparation and professional development of college faculty and secondary school teachers; internships and field experiences for faculty, teachers, and students; and other activities. With an emphasis on two-year colleges, the program focuses on the education of technicians for the high-technology fields that drive our nation's economy. 

ESIE has identified three issues that should be integrated, as appropriate, into funded projects across its programs.   

• Evaluation – ESIE has a strong commitment to evaluation that assesses the effectiveness and impact of its projects in enhancing learning and instruction in STM disciplines in both formal and informal education settings.  All projects should be based upon the research in cognitive science. 
    

• Instructional Technology – Projects should capitalize on educational technologies and electronic networking in the development of materials, instructional strategies and professional development in a manner that uses the technology thoughtfully and ensures accessibility to a quality education for all students.    

• Promoting Representation in STM Education and Careers – All ESIE programs have the goal of increasing access, participation, and success of women, underrepresented minorities, and persons with disabilities in STM education.  ESIE is particularly interested in increasing the numbers of such students who will pursue advanced study and careers in STM areas, including the teaching of science, technology, and mathematics, grades K-12. 

ESIE programs, other than IMD, are described in separate program announcements.  Updates may be issued, as needed, to announce relevant changes or additions.  To stay current with ESIE program offerings, periodically visit the ESIE Web Site, http://www.ehr.nsf.gov/ehr/esie/; for ATE, the DUE Web Site, http://www.ehr.nsf.gov/ehr/due/.  All NSF publications referenced in this document are available via the NSF Online Document System, http://www.nsf.gov/cgi-bin/pubsys/browser/odbrowse.pl.

B. INSTRUCTIONAL MATERIALS DEVELOPMENT (IMD)

OVERVIEW OF IMD COMPONENTS

The goal of the IMD program is to enhance science, technology, and mathematics (STM) content knowledge, as well as the thinking skills and problem solving abilities, of all students, pre-Kindergarten through grade 12 (preK-12), regardless of background, ability, or future education plans.  The IMD program encourages learning by all students by promoting positive dispositions toward science and technology and of themselves as learners.  A broader cross section of students is led to pursue education in scientific and technological disciplines through the use of real-world contexts and by providing an understanding of the role of science and technology in the workplace.  These goals are implemented through support of the development, revision, evaluation, and dissemination of materials and assessments that guide instructional decisions and provide information on depth of student learning of concepts and ideas.  In pursuing enhanced student learning of STM content with depth and understanding, IMD recognizes the need to align teacher content knowledge and pedagogical strategies with these materials and assessments.  IMD promotes applied research that increases understanding of how teachers, materials, and assessments support student learning.  The IMD program also supports awareness and use of materials and assessments in interested school districts, and supports the dissemination of information for administrators, parents and other caregivers to promote the adoption and enhance the effectiveness of the implementation of research-based classroom-tested instructional materials.

AREAS FOR INVESTIGATION

The major emphasis of the IMD program is the development, dissemination, and implementation of instructional materials and assessments in science, technology education,[2] and mathematics.  Attention is called to two additional areas that merit investigation.

• Impact Assessments.  Proposals are encouraged for studies of the effectiveness and impact of instructional materials developed with NSF support.  Effectiveness and impact may include such things as student achievement, persistence in course taking beyond school and state requirements, and documentation of increased interest in science and technology disciplines.  Research could range from meta-analyses of existing studies, to large-scale studies involving the design and administration of new evaluation tools, which could include the use of non-traditional measures (examples include indirect measures, embedded data collection, usage patterns, etc.).  The studies should be conducted by external investigators who are not connected with the individual materials development projects.  Investigators contemplating such studies should discuss their ideas with an IMD Program Officer prior to writing a preliminary proposal.

• Emerging Technologies.  IMD also seeks to support a small number of innovative, high-risk projects that develop and test prototypes of instructional materials using state-of-the-art technologies such as hybrid course materials that combine text with dynamic elements, assessment tools embedded in student materials that provide timely feedback, adaptive and flexible guides that provide support for teaching, and others.  Proposals for such high-risk projects must demonstrate promise for advancing and testing the limits of instructional and assessment materials to promote student understanding of STM concepts and processes.  Typically, the funding is limited yet sufficient to test the efficacy of the innovation.  Proposals to be considered for this category should be clearly defined as such in the project summary.

[2] Technology education refers to the study of the human-made environment using the design process that leads to the development of technological literacy.  It is separate from uses of instructional technology.

There are four components in the IMD program.  These are Instructional Materials for Students, Dissemination and Implementation Sites, Assessment, and Applied Research.

A.  Instructional Materials for Students

PROJECT CHARACTERISTICS

Proposed instructional materials must exhibit a coherent content framework that is aligned with standards developed by national professional organizations (American Association for the Advancement of Science, 1993; International Technology Education Association, 2000; National Research Council, 1996; National Council of Teachers of Mathematics, 2000); foster inquiry, including critical thinking, problem solving, decision-making, and communication at increasing levels of complexity; and focus on appropriate and important topics at each grade level.  Projects should be grounded in recent research on teaching and learning (e.g., Bransford, Brown, & Cocking, 1999; Wiggins & McTighe, 1998) and further an understanding of the connections among science, technology, engineering, and mathematics disciplines.  Projects should be planned and implemented by teams consisting of appropriate combinations of practicing scientists, mathematicians, and engineers, STM educators, classroom teachers, professional curriculum developers, assessment experts, and instructional technology experts.  Projects should include research to understand better the effectiveness of the materials in increasing student learning and changing teacher practice.  

Projects are supported that are national in scope and significance.  These projects should have the potential to enhance student learning and make a significant and noticeable impact on the national market for instructional materials.  Projects range from the creation of new materials to the substantial revision of effective materials; from development of comprehensive curricula for several school years to the development of a single module for one grade level; and from the integration of several disciplines to addressing a single topic.  

Particularly encouraged are projects that develop and implement research-based instructional materials that ameliorate achievement gaps between students and lead to improved understanding of and participation in STM by members of underrepresented groups.  Strategies for ensuring equity and accessibility in materials use and in student learning should be part of the development and implementation process.  Proposals should describe specific research-based strategies for engaging target groups.

Incorporation of instructional technologies should be considered, especially when those technologies are used to provide learning experiences that enhance student understanding.  Digital libraries, computational tools, modeling and visualization, virtual environments, connectivity, and communication are among a growing catalog of capabilities that change what and how students can learn.  All IMD-funded projects should use instructional technology in appropriate ways, anticipating a future where these technologies are used widely to enhance education.  

Also supported are innovative, high-risk projects that develop and test prototypes of instructional materials and instructional technologies.  Proposals for such high-risk projects must demonstrate promise for advancing the state-of-the-art development of curriculum and assessment materials and for testing the limits of instructional and assessment materials to promote student understanding of science, mathematics, and/or technology concepts and processes.

In addition, when appropriate, the projects should include products designed to help parents understand, and teachers implement, the materials.  It is important that projects include strategies and tools to assess the impact of the instructional materials, particularly on student learning.  These data should be presented in ways that help stakeholders, including teachers, administrators, and parents, make informed decisions about curriculum adoptions.

PROJECT DESCRIPTION 

Exemplary projects will contain the following elements, which should be addressed in the Project Description section of the proposal.  Proposal reviewers will examine the extent to which these elements are effectively incorporated in the overall project plan.

• Goals and Objectives.  Describe the major goals for the project, as well as the anticipated outcomes, for the students and for the teachers.  

• Project Evaluation.  Describe the evidence that will be accepted to determine the extent to which goals are achieved, and the evaluation strategies that will be used to obtain that evidence.  Each major aspect of the project should be evaluated -- the development process, fidelity of implementation, student learning, teacher professional development, etc. Formative evaluation, designed to effect development efforts, could be conducted by an internal evaluator.  External evaluation should provide evidence of the effectiveness of the developed projects.  The proposal should provide evidence of the qualifications of the evaluators.  

• Anticipated Products.  Describe the materials to be produced (e.g., workbooks, textbooks, software, videos, CD-ROMs, scholarly publications, monographs), including the specific learning activities to be developed (experiments, student projects, assessments, etc.).  

• Rationale.  Describe how the proposed materials will address learning goals for students and teachers throughout the nation better than existing materials.  Relevant literature should be referenced to indicate knowledge of disciplinary and pedagogical issues.  The proposer should describe how the instructional materials build on, and relate to, previous and on-going efforts in the field.  A search of the Eisenhower National Clearinghouse (ENC) database is recommended (see: http://enc.org/partners/fed/mfinder/nsf.htm).

Proposals for the substantial revision of successful IMD-funded materials must provide data on classroom use of the earlier materials.  Data such as market share, total number of copies sold or in use, or other pertinent measures, should be cited.  Evidence of positive student outcomes, in terms of student achievement, persistence in course taking beyond school, district, or state requirements, and/or other measures must be provided.  Portions of the materials to be revised (e.g. units, modules, etc.) must be identified and a description of the nature of the changes must be given.  Proposals must provide a clear rationale for the changes being proposed, based on research/studies of the implementation of the materials to be revised.  Finally, evidence of strong support of the publisher, including substantial financial contributions to the revision process, must be included.

• Work Plan.  Explain how the materials will be created, reviewed, pilot-tested, field-tested, evaluated, and published.  A detailed plan, including a complete timeline that indicates who is responsible for each facet, helps reviewers understand the flow of work.  Draft materials must be pilot-tested with master teachers, and field-tests must include a broad range of teachers with diverse backgrounds.  It is expected that results of these trials will be used to inform revisions of the materials, and that both the results of the trials and the revisions will be submitted to NSF.  

• Content and Pedagogical Strategies.  Describe how the materials’ content and pedagogical strategies are aligned with national standards; how the materials will prepare and motivate students to continue to study STM at higher grade levels; and how the materials will account for potential differences in students’ prior knowledge.  Resources (e.g., computer expertise) that will be needed at the school level for implementation of the materials also must be specified.  If the materials are supplemental in nature, they must have a clear and limited content focus; utilize appropriate scientific, computational, and instructional technologies; and demonstrate connections to comprehensive curricula.  All materials should undergo independent review to ensure accuracy of the content, appropriateness of the pedagogy, and suitability of the contexts, language, etc., for the intended audience.  For materials that span one or more years of instruction, the review will be conducted by reviewers external to the project who are selected in consultation with the cognizant Program Officer at NSF.   

• Assessment.  Describe tools and strategies for student assessment that will be included in the instructional materials.  It is critical that student assessments be clearly aligned with the desired student learning outcomes and be informed by the nationally-developed standards in mathematics, science, and/or technology.  Assessments should address both formative and summative aspects of learning.  That is, assessments should include strategies for teachers to use during instruction as a guide for making instructional decisions as well as to determine what students are learning.  Development and validation of assessment tools should occur in both the pilot- and field-testing of the materials.  To the extent possible, there should be a variety of assessment strategies that are responsive to the different ways that students communicate understanding of content.

• Professional Development.  Describe the products to be developed (e.g., print, CD-ROM, web-based) that will support teachers and administrators in accurately and effectively implementing the materials.  This element should include teaching guides to accompany the student materials, but may also include other products.  If the materials span one or more years of instruction, there must be separate professional development materials to help teachers develop a deep understanding about the content and pedagogy of the materials.  There should be plans for sustaining the use of the professional development materials after the end of the project and for informing teacher educators about the new materials to facilitate incorporation into pre-service teacher education programs.  The creation of the professional development materials must meet the guidelines for Teacher Enhancement (TE) program’s professional development materials (see NSF 01-60).

• Community Involvement.  Describe ways to communicate to the community how the materials are designed to enhance learning of significant subject matter content and to increase student interest in science, mathematics, and technology.  Attention should be given to communicating how the materials may differ from those with which community members may be familiar.  Further, parent/community materials should assist parents and caregivers in helping their children in STM education.  

• Dissemination and Implementation.  Explain how information about the materials will be shared with professionals in STM education communities both during and after the project.  Instructional materials typically will be published and distributed commercially, although in a few instances “free” distribution (e.g., through a refereed and highly visible Web Site) might be an appropriate outlet.  Proposals should provide a timeline for securing a publisher (or identifying another distribution outlet).  This step typically should occur within the first two years of multi-year projects, with distribution outlets firmly established (e.g., through a contract with a publisher or distributor) by the end of the third year of the project.  Dissemination plans that project potential sales income during the duration of the grant should specify how that income will be used to support the implementation, revision, or continued development of materials.  

• Personnel.  Describe the expertise and experience of the key personnel.  It is expected that the development team will include, as appropriate, scientists, mathematicians, and engineers; STM educators; classroom teachers; assessment and evaluation experts; technology experts; instructional technologists; and professional developers.  The proposal should include a detailed description of the role and commitment level of each of the key personnel.  

• Results of Prior NSF Support.  Describe results of prior NSF support for educational projects in which senior personnel have been involved.  For projects that have developed materials related to the proposed work, the proposal must include a summary of the past project evaluation that provides compelling evidence of the quality and effectiveness of the materials developed.

B.  Dissemination and Implementation Sites

PROJECT CHARACTERISTICS

Dissemination and Implementation Sites increase access to the use of high-quality instructional materials and thus increase the opportunities for all students to learn mathematics, science, and technology.  There is a growing body of evidence that curriculum implementation is an effective strategy for districts and schools to use in launching a standards-based reform effort (Berns, Kantrov, Pasquale, Makang, Zubrowski, & Goldsmith, 2000; Cohen & Hill, 1998; Weiss, Arnold, Banilower, & Soar, 2000; Weiss, Knapp, Hollweg, & Burrill, 2001; Winkler & Mark, 2001).  The primary goal of Dissemination and Implementation Sites is to increase the expertise of state, district, and school level educators to select, adopt, and implement high quality instructional materials.  Dissemination and Implementation projects are designed to leverage the investment made in instructional materials development in interested schools and districts.

Key elements of Dissemination and Implementation Sites are: (a) knowledge transfer (diffusing knowledge broadly), (b) systems change (working with districts and schools to improve practice in mathematics and science), (c) expert consultation (acting as “brokers” and linking districts and schools with others), and (d) integration (providing support to districts and schools so that new materials are integrated into the system).

Based on these key elements, each Dissemination and Implementation Site is expected to:

• Build awareness of research-based instructional materials.  Each site will become the primary national resource for information and expertise on curriculum materials selection and implementation for a particular subject area and grade band.  This includes providing opportunities for educators to examine instructional materials and to see them in action in selected schools.  Information should be made widely available through web pages and print material and should include evaluative data when they are available.  Collectively, these sites provide an ongoing support system for integrating and promoting research-based practices.         

• Maintain strong connections with curriculum developers of the materials that are being disseminated.  This ensures better fidelity of implementation (expert consultation).         

• Enhance local curricular leadership (school boards, superintendents, curriculum specialists in mathematics, science, and technology, teachers and parents) to sustain an ongoing process of curriculum improvement.  This can include strategic planning, building effective partnerships to support school change, and planning for professional development that helps teachers in the use of the materials (systems change).       

• Connect districts and schools with related projects and resources from other NSF programs that can specifically help them implement new curricula.  These include the Math and Science Partnership, Local Systemic Change, Centers for Learning and Teaching, new instructional materials development, and applied research projects (expert consultation).         

• Connect districts and schools with recent research on curriculum implementation, including student outcome data (expert consultation).        
• Identify gaps in the research base around curriculum implementation and student learning, and propose projects to address those gaps (systems change, integration).       

• Provide connections with other national curriculum development efforts such as those initiated by professional organizations, publishers, mathematicians and scientists, and innovative preservice programs (expert consultation).       

• Work collaboratively and synergistically with other dissemination and implementation projects.  Those contemplating the preparation of proposals for a Support Site should consult the ESIE web page, http://www.ehr.nsf.gov/ehr/esie/.         

• Assist a limited number of strategically chosen local districts and schools ready to undergo the curriculum implementation process to gain a deeper knowledge base about the implementation process (expert consultation and integration).       

• Provide feedback about the implementation process in schools, including approaches to instructional materials and professional development.       

• Support contacts among school districts, so that schools can learn from each other.  These Sites should support the sharing of models of successful practice among schools.

Two lessons have been learned from the first set of Dissemination and Implementation Site projects (Tushnet, et al., 2000).  First, Sites should focus on either mathematics or science.  Second, Sites appear to be more successful when they focus on a particular grade band (elementary, middle grades, high school).  There are characteristics that differ between curriculum, instruction, and pedagogy at different levels of schooling.  School organization at the different levels means that different strategic approaches to implementation also come into play.  For information on existing Sites, see the ESIE web page, http://www.ehr.nsf.gov/ehr/esie/.  

PROJECT DESCRIPTION

Exemplary projects will contain the following elements, which should be addressed in the Project Description section of the proposal.  Proposal reviewers will examine the extent to which these elements are effectively incorporated in the overall project plan.

• Goals and Objectives.  Provide goals that clearly address the Dissemination and Implementation Site’s purpose.  Identify the disciplinary and grade-level band focus of the proposed site.         

• Project Evaluation.  Describe the Site evaluation plan and the criteria that will be used to judge the success of the site.  External evaluation data are required of all supported activities to provide formative and summative information on the quality of the partnerships developed, effectiveness of the approach, and impact of the site.  External evaluation data must also be collected in districts where sites work intensively, and should examine changes in teachers’ knowledge, in the quality of instruction, and in student achievement.  In addition, Sites will be required to participate in an evaluation of the entire portfolio of projects.        

• Rationale and Anticipated Outcomes.  Provide a justification for the site’s organization, approaches, and products.  In addition, a description must be provided ensuring that all selected materials to be disseminated by the site are based on accepted standards, are research-based, and pilot- and field-tested.  Instructional materials that are supported by the site should be sensitive to issues of gender, race, class, ethnicity, and cultural differences.  It is anticipated that the curricula disseminated will change over time, as evidence is generated about their effectiveness or as new materials are developed.         

• Work Plan.  Describe how the Site will operate.  For example, some sites may choose to work through a regional structure with satellites distributed throughout the country.  Site personnel should be aware of regional differences in needs for instructional materials and have access to on-line conferencing or video-conferencing to assist remote school districts.  All Sites must form working relationships with institutions of higher education, school districts, curriculum developers, publishers, other dissemination and implementation projects, and major national reform initiatives.  The lead institution should provide evidence that it is “neutral” with regard to the curricula it intends to disseminate and recommend for adoption/implementation.         

• Dissemination.  Describe the plan for dissemination of the Site’s products, such as awareness and planning materials, professional development materials, results of research, etc.        

• Personnel.  Describe the expertise and experience of key personnel.  It is expected that key personnel will include mathematics and/or science educators with experience in curriculum dissemination and implementation, professional development experts, and university faculty.  

In addition to Dissemination and Implementation Sites that focus on grade levels, there is a need for K-12 Support Sites.  A K-12 Support Site is intended to ensure that there is reasonable articulation across the Dissemination and Implementation Sites and to provide the field with one place to go for information in either mathematics or science.  Functions of a Support Site include: (a) providing information (through brochures and Web Sites) to interested participants about the work of the Dissemination and Implementation Sites; (b) helping districts or schools to focus on articulation issues across grade levels; (c) providing regular opportunities for the site PIs to meet around issues of common concern; (d) developing technical assistance materials for the sites to use in common; (e) providing assistance around general issues such as strategic planning, building partnerships, and working with the community; (f) providing information on adoption opportunities and constraints in schools and districts; (g) providing expertise on working with schools; and (h) acting as a repository of information on current research around curriculum implementation including student outcomes.

It is anticipated that one or two Support Sites (one that supports both mathematics and science, or one that supports mathematics and one that supports science) will be active at all times.  Support Sites need to ensure collaboration with the existing Dissemination and Implementation Sites and provide coherence to the overall effort.  Those proposing a separate site for either mathematics or science should pay particular attention to the issue of coherence.  In either case, it is strongly advised that prospective Principal Investigators communicate with a Program Officer before preparing a preliminary proposal, and provide a strong rationale for the option selected.  

The Project Description section of proposals for Support Sites should include the bulleted items listed for the Project Description for Dissemination and Implementation Sites, with appropriate modifications.  In particular, items (a)-(h) cited in the paragraph above should be addressed and the work plan should describe how the proposed support site will work collaboratively and synergistically with the Dissemination and Implementation Sites and provide for a coherent effort.

C.  ASSESSMENT 

PROJECT CHARACTERISTICS

Accurate assessment of student learning is critical for determining the effects of education reforms.  New assessment tools must be developed that are tied to accepted standards, as well as to specific reform goals.  Similarly, schools and districts need support for new types of assessment; for documenting the quality of new instructional materials; for assessing teacher knowledge and practice; and for assessing learning in informal settings.  IMD encourages the development and implementation of new directions in assessing student and teacher learning in both formal and informal settings (see ESIE Program Solicitation [NSF 01-60] for description of the Teacher Enhancement [TE] and the Informal Science Education [ISE] programs).  Assessments may range from those embedded in instructional materials to the creation of items for general use by districts and states.

Assessment projects: (1) are based on current research and include a model of cognition and learning as the cornerstone of the assessment design process (Pellegrino, Chudowski, & Glaser, 2001); (2) provide reliable and valid information that leads to a better understanding of how student learning can be enhanced and how instructional practice can be improved; (3) are developed in collaborative teams with appropriate expertise in the content area, in cognitive and learning theory, in assessment development and psychometrics, and that include target users (e.g., teachers); and (4) are regional or national in scope.  

Assessment projects should address one or more of the following areas:

Assessment Development

• Student Learning.  Assessment that focuses on measuring student learning and achievement.  Assessment development that focuses on comprehensive tests is needed.  These tests should be consistent with standards developed by national professional organizations, and be cost effective and easy to administer, score, and interpret.  Critical areas of need include: (a) assessment instruments that focus on NSF-funded materials and key concepts or strands in middle grades mathematics; (b) projects that develop various tools to assess and guide student learning and instruction (e.g., embedded assessments); (c) comprehensive science (e.g., middle school science); and (d) technology education assessments (e.g., concept acquisition through project-based learning).       

• Teacher Knowledge and Performance.  New and effective approaches to assess teachers’ knowledge and performance aligned with effective instructional strategies.       

• Informal Education.  Assessment instruments that examine learning in informal settings, such as museum exhibits, television, and films.  Projects should consider connections with classroom assessment efforts in terms of both content and psychometric approaches.        
• Technology-Based.  Effective applications of technologies for assessment purposes.  Studies examining unique contributions of various technologies to assessment are encouraged.

Assessment Assistance 

• Technical Assistance.  New approaches are needed to provide assistance to schools and districts in judging, adapting, and implementing available assessment tasks/tests and in using the information they provide.       

• Communication Assistance.  Materials and strategies may be developed to help schools and districts communicate effectively with teachers, administrators, school board members, parents, and the general community about the role of assessment.  Materials and strategies that help public to be critical consumers of available assessments and assessment results are also welcome.

PROJECT DESCRIPTION

Exemplary projects will contain the elements listed below, which should be addressed in the Project Description section of the proposal.  Proposal reviewers will examine the extent to which these elements are effectively incorporated in the overall project plan.

• Goals and Objectives.  Provide a description of the major goals for the project and for targeted audiences (e.g., students, teachers, general public).  

• Project Evaluation.  Describe the evidence that will be accepted to determine the extent to which the goals are achieved and the activities that will be used to obtain that evidence.  An advisory board or external review team with expertise in the content area, STM education, assessment development, and measurement is recommended to oversee the project.  The proposal should provide evidence of the qualifications of the advisory board or review team members.        

• Anticipated Products.  Describe the assessment materials to be produced (e.g., type of assessments, monographs for helping teachers to implement assessments).       

• Rationale.  Provide evidence that the proposed assessment materials meet the needs of students and teachers better than the existing ones.  The project should be based on clear theoretical foundations and include a thorough overview of relevant research and literature to indicate knowledge of disciplinary, learning, cognition, and assessment issues.  The proposal must describe how the assessment materials build on and relate to previous and on-going efforts in the field, and the contribution they will make to the field of assessment.  A search of the Eisenhower National Clearinghouse (ENC) database is recommended(see: http://enc.org/partners/fed/mfinder/nsf.htm).         

• Work Plan.  Explain how the assessments or materials will be created, reviewed, pilot-tested, field-tested, evaluated, and disseminated.  The use of appropriate assessment development methodologies and psychometrically sound evaluations is expected.  The proposal should contain a detailed plan of work, including a complete timeline.       

• Dissemination.  Explain how information about the assessments and materials will be shared with professionals in SMT education communities both during and after the project.  A dissemination plan that projects potential sales income should specify how that income will be used to support the implementation, revision, or continued development of assessment materials.      

• Personnel.  Describe the expertise and experience of the key personnel.  It is expected that the assessment development team will include, as appropriate, scientists, mathematicians, and experts in technology; STM educators; classroom teachers; curriculum developers; and assessment and psychometric-experts.  The proposal should include a detailed description of the role and commitment level of each of the key personnel.       

• Results of Prior NSF Support.  Describe results of prior NSF support for educational projects in which senior personnel have been involved.  Proposers who have developed assessments and materials related to the proposed work should include a summary of the past project evaluation that provides compelling evidence of the quality and effectiveness of the materials developed.

D.  APPLIED RESEARCH

PROJECT CHARACTERISTICS

ESIE has a strong commitment to applied research that assesses the effectiveness and impact of funded efforts in enhancing learning and instruction in STM in both formal and informal educational settings.  The purpose of applied research projects is to ensure that all ESIE programs are firmly grounded in research and that ESIE projects benefit from this knowledge.  Applied research provides important feedback for strengthening ESIE’s portfolio and for identifying new programmatic directions.  Research studies may be of three different types:  

• separate applied research efforts that grow out of completed projects or from questions that arise through analysis of an issue of priority to ESIE;
  
• well-defined elements of a proposed IMD, ISE, or TE project; or 
  
• supplements to an existing project to address questions that have arisen during project implementation.  In these cases, proposed studies must generate important feedback to the on-going project.  

Examples of questions that might be addressed include:  In what ways and under what circumstances is enhanced student learning an outcome of using NSF-funded instructional materials?  Do NSF-funded instructional materials enhance other desirable student outcomes (e.g., increased enrollments in upper level STM courses)?  What characteristics of teachers and classroom environments lead to student achievement gains?  What elements (e.g., time, reflection, activities) are needed in programs that educate new and experienced teachers to implement instructional materials with fidelity in classrooms and increase student learning?  What characteristics of professional learning communities of teachers help to sustain reform?  What learning outcomes result from the use of instructional technologies?  How does informal learning promote better understanding of concepts, processes, and thinking in scientific and technical disciplines for learners of all ages?

PROJECT DESCRIPTION

Exemplary projects will contain the following elements listed below, which should be addressed in the Project Description section of the proposal.  Proposal reviewers will examine the extent to which these elements are effectively incorporated in the overall project plan.

• Goals and Objectives.  Provide clear research questions that address areas of interest fundamental to ESIE’s mission and goals.  

• Project Evaluation.  Describe the evidence that will be accepted to determine the extent to which the goals are achieved and the activities that will be used to obtain that evidence.  An advisory board or external review team with expertise in the content area, STM education, methodology (qualitative or quantitative), and measurement is recommended to oversee the project.  The proposal should provide evidence of the qualifications of the advisory board or review team members.

• Anticipated Products.  Describe the materials to be produced (e.g., workbooks, software, assessment tools, scholarly publications, monographs, etc.).  

• Rationale.  Provide evidence that the project is based on clear theoretical foundations, and include a thorough overview of relevant research and literature to indicate knowledge of disciplinary, pedagogical, and evaluation issues.  The proposal must describe how the research endeavor builds on, and relates to, previous and on-going efforts in the field.  The proposal must provide evidence that the research project has a high likelihood of generating data that will contribute to the cycle of design, research, and redesign of ESIE’s programs and project portfolios.  A search of the Eisenhower National Clearinghouse (ENC) database is recommended (see: http://enc.org/partners/fed/mfinder/nsf.htm).

• Work Plan.  Explain clearly the research plan and the methodology to be applied.  Explain how the research plan will provide answers to the research questions.  Issues related to internal validity (correctly concluding about the effects of treatment) and external validity (can the research findings be generalized) of the data generated should be considered.  An explanation of how data will be collected and analyzed and the appropriateness of the methods used should be included.        

• Dissemination.  Explain how research results will be shared with professionals in STM education communities both during and after the project.  

• Personnel.  Describe the expertise and experience of key personnel.  The proposal should include a detailed description of the role and commitment level of each of the key personnel.

• Results of prior NSF support.  Describe results of prior NSF support for educational projects in which senior personnel have been involved.  In cases where previous projects have resulted in materials related to the proposed work, include a summary of the past project evaluation that provides compelling evidence of the quality and effectiveness of the materials developed.

Principal Investigators interested in supplementing a current project with a related applied research effort should contact their cognizant NSF Program Officer.  Approximately $1.5 million will be available for separate projects in fiscal year 2003.  Stand-alone proposals will be reviewed in collaboration with the EHR Division of Research, Evaluation and Communication (REC).  ESIE will not accept proposals under current consideration for funding by REC.  

E.  REFERENCES

American Association for the Advancement of Science (1993). Science for all Americans.  Washington, DC: Author.

Berns, B.B., Kantrov, I., Pasquale, M., Makang, D.S., Zubrowski, B., & Goldsmith, L.T. (2000). Guiding curriculum decisions for middle-grades science.  Newton, MA:  Education Development Center, Inc.

Bransford, J.D., Brown, A.L., & Cocking, R.R. (1999). How people learn:  Brain, mind, experience, and school.  Washington, DC:  National Academy Press.

Cohen D.K., & Hill, H.C. (1998). State policy and classroom performance: Mathematics reform in California [CPRE Policy Brief].  Philadelphia, PA:  Consortium for Policy Research in Education.

International Technology Education Association (2000). Standards for technological literacy.  Reston, VA:  Author.

National Council of Teachers of Mathematics (2000). Principles and standards for school mathematics.  Reston, VA:  Author.

National Research Council, National Academy of Sciences (1996). National science education standards.  Washington, DC:  National Academy Press.

Pelligrino, J., Chudowski, N., & Glaser, R. (2001). Knowing what students know:  The science and design of educational assessment.  Washington, DC:  National Academy Press.

St. John, M., ESIE Seminar, March 15, 2001.

Tushnet, N.C., Millsap, M.A., Abdullah-Welsh, N., Brigham, N., Cooley, E., Elliott, J., Johnson, K., Martinez, A., Nierenberg, M., & Rosenblum, S. (2000). Final report on the evaluation of the National Science Foundation’s Instructional Materials Development Program.  Arlington, VA:  National Science Foundation.

Weiss, I.R., Arnold, E.E., Banilower, E.R., & Soar, E.H. (2001). Local systemic change through teacher enhancement:  Year six cross-site report.  Chapel Hill, NC:  Horizon Research, Inc.

Weiss, I.R., Knapp, M.S., Hollweg, K., & Burrill, G. (Eds.). (2001). Investigating the influence of standards.  Washington, DC:  National Academy Press.

Wiggins, G.P., & McTighe, J. (1998). Understanding by design.  Alexandria, VA:  Association for Supervision and Curriculum Development.

Winkler, K., & Mark, J.  (Eds.). (2001). Perspectives on curricular change:  Interviews with teachers, administrators, and curriculum developers.  Newton, MA:  Education Development Center, Inc.

Full proposals must be submitted via FastLane no later than 5:00 PM local time on the specified deadline date.  Submission of full proposals via FastLane requires completion of the following FastLane forms:

Cover Sheet.  (See GPG, Chapter II, Section C) The Cover Sheet must contain all requested information.  If project funds are requested from another Federal agency or another NSF program, it must be indicated on the cover sheet.  If such funds are requested subsequent to proposal submission, a letter should be sent to the relevant program in ESIE, identifying the proposal by its NSF number.  Proposers are reminded to identify the Program Solicitation number in the Program Solicitation block.  Compliance with this requirement is critical to determining the relevant proposal processing guidelines.  Failure to submit this information may delay processing.  The related preliminary proposal number should be entered, when appropriate.  

The Authorized Organizational Representative (AOR) must electronically sign the proposal Cover Sheet to submit the required proposal certifications.  The AOR must provide the required certifications within five working days following the electronic submission of the proposal.  Further instructions regarding this process are available on the FastLane Web Site at: http://www.fastlane.nsf.gov.

Project Summary.  A one-page (250 word) Project Summary should be prepared, suitable for publication, which presents a self-contained description of the activity that would result if the proposal were funded.  The initial sentences should describe the IMD program component to which the proposal is submitted, the disciplinary content of the project, the grade level of the target audience, and the pedagogical strategies to be used.  This information is used to place the proposal in the appropriate review panel.  The summary should be written in the third person, in the present tense, and include an indication of the need being addressed, a statement of objectives, methods to be employed, potential contribution to the advancement of knowledge, and a description of the products or outcomes resulting from the project.  The summary should be informative to other persons interested in developing projects or using instructional materials.

Table of Contents.  (See GPG, Chapter II, Section C.2) The Table of Contents is automatically generated in FastLane.

Project Description (including results from prior NSF support and data sheet).  (See GPG, Chapter II, Section C.3) The Project Description presents most of the information that determines whether or not a grant will be awarded.  It should be written to respond to criteria provided in the section on each component.  Reviewers will use this information in judging the merit of the proposal as described in this document.  In addition, it should communicate potential project impact and general project characteristics.  The information provided should help to direct proposals to appropriate reviewers.  The maximum number of pages allowable for the Project Description is 15 pages, with the following exceptions:  planning and conference grant proposals should be no longer than 10 pages in length, including visual materials (e.g., charts, graphs, maps, photographs, and other pictorial presentations); proposals for comprehensive, multi-year curricular materials should be no longer than 20 pages in length.  Page formats should be single-spaced with a clear and legible type size of no smaller than 12-point type and with no less than 2.5 cm margins on all sides.  See the section Supplementary Documents below, and individual program sections for any exceptions to this limitation.

Results from Prior Support.  (See GPG, Chapter II, Section C.3) If the prospective PI or co-PI(s) received support for related NSF activities within the past five years, a description of the project(s) and outcomes must be provided in sufficient detail to enable reviewers to assess the value of results achieved.  Past projects should be identified by NSF award number, funding amount, period of support, title, summary of results, and a list of publications and formal presentations that acknowledge the NSF award (do not submit copies of the latter).  Evaluation data should be clearly described.  Details regarding evaluation data should be put into an appendix.  PIs must have submitted a final report for any completed NSF-funded project, or no new grant may be awarded.

Project data sheets:  A completed project data sheet must accompany the proposal.  Data sheets may be downloaded from the ESIE Web Site at: http://www.ehr.nsf.gov/ehr/esie/datasheet.doc.  Data sheets should be included in the supplementary document section of FastLane.  

References Cited.  (See GPG, Chapter II, Section C.4) Any literature cited should be specifically related to the proposed project, and the Project Description should make clear how each reference has played a role in the motivation for or design of the project.  

Supplementary Documents.  (See GPG, Chapter II, Section C.9) Reviewers are often asked to read and assess a substantial number of competing proposals.  For this reason, the Project Description alone should provide sufficient information so that a reviewer unfamiliar with the context of the project can make an informed judgment.  It may be critical to convey more detailed information to demonstrate levels of competence or expertise, to document commitment of personnel or other resources, to demonstrate the quality of instructional materials, or to provide details of the evaluation of previously developed materials.  Such material can be included in appendices that are clearly referenced in the proposal.  Appendices, except sample materials, should be uploaded into the Supplementary Documents section on FastLane.  Additionally, the proposal may refer to Web Sites that contain this type of supplementary material.  Presentation of such materials should be thoughtful and concise.  Reviewers are not required to read appendices.   

Proposals for the development of student materials should include examples of the proposed materials and samples of past work, if they exist.  These sample materials should be sent directly to the program, clearly labeled with the proposal number.  If the materials are to be sent to the reviewers, at least ten copies are needed.  Prospective PIs are encouraged to contact NSF Program Officers if they have questions about submission of appendix materials.  

Biographical Sketches.  (See GPG.  Chapter II, Section C.5) Biographical information (no more than two pages) must be provided for each person listed as senior personnel on the budget form; include career and academic credentials and a mailing address.  

Current and Pending Support.  (See GPG, Chapter II, Section C.7).

Budgetary Information.  (See GPG, Chapter II Section C.6) Proposals must contain a budget for each year of requested support.  The proposal may request funds under any budget category so long as the item is considered necessary to perform the proposed work and is not precluded by program guidelines or applicable cost principles.  All budget requests must be documented and justified in the Budget Justification section of no more than three pages.  Ordinarily, no funds are made available for equipment or facilities or for continued operational expenses. Estimates of calendar months of activity must be reported for categories of key personnel.

Special Proposals:

·         Small Grants for Exploratory Research (SGER).  (See GPG, Chapter II, Section C.11a) SGER grants are for small-scale exploratory, high-risk research. Investigators are strongly encouraged to contact an NSF Program Officer before submitting the proposal. 

·         Collaborative Proposals.  (See GPG Chapter 11, Section C.11b) Proposals in which investigators from more than one institution wish to collaborate on a unified project. 

·         Proposals for Conferences, Symposia and Workshops.  (See GPG, Chapter 11, Section C.11f) IMD may support a few well-focused conferences or workshops whose products inform the community. 

Proposers are reminded to identify the program solicitation number (NSF-02-067) in the program announcement/solicitation block on the proposal Cover Sheet. Compliance with this requirement is critical to determining the relevant proposal processing guidelines. Failure to submit this information may delay processing.

The proposed cost sharing must be shown on Line M on the proposal budget. Documentation of the availability of cost sharing must be included in the proposal. Only items which would be allowable under the applicable cost principles, if charged to the project, may be included as the awardee's contribution to cost sharing. Contributions may be made from any non-Federal source, including non-Federal grants or contracts, and may be cash or in-kind (see OMB Circular A-110, Section 23). It should be noted that contributions counted as cost-sharing toward projects of another Federal agency may not be counted towards meeting the specific cost-sharing requirements of the NSF award. All cost-sharing amounts are subject to audit. Failure to provide the level of cost-sharing reflected in the approved award budget may result in termination of the NSF award, disallowance of award costs and/or refund of award funds to NSF.

Submission of Electronically Signed Cover Sheets. The Authorized Organizational Representative (AOR) must electronically sign the proposal Cover Sheet to submit the required proposal certifications (see Chapter II, Section C of the Grant Proposal Guide for a listing of the certifications). The AOR must provide the required certifications within five working days following the electronic submission of the proposal. Further instructions regarding this process are available on the FastLane website at: http://www.fastlane.nsf.gov.

Informal Science Education - (Elementary, Secondary, and Informal Education Program Solicitation and Guidelines [NSF 01-61] http://www.nsf.gov/cgi-bin/getpub?nsf0160).

Presidential Awards for Excellence in Mathematics and Science Teaching (http://www.ehr.nsf.gov/pres_awards/)

Advanced Technological Education (ATE) – (Advanced Technological Education (ATE) Program Solicitation [NSF 02-035] http://www.nsf.gov/cgi-bin/getpub?nsf02035).

Centers for Learning & Teaching (CLT) – (Centers for Learning and Teaching (CLT) [NSF 02-038]
 http://www.nsf.gov/cgi-bin/getpub?nsf02038).

Programs in other EHR Divisions that may be of interest to proposers to IMD include: 

Assessment of Student Achievement in Undergraduate Education (ASA) – development and dissemination of assessments and practices to guide efforts to improve effectiveness of undergraduate science, technology, engineering, and mathematics (STEM) courses.  (http://www.ehr.nsf.gov/EHR/DUE/programs/asa/)

Course Curriculum and Laboratory Improvement (CCLI) - instructional materials, courses for undergraduates, and professional development for faculty.  (http://www.ehr.nsf.gov/EHR/DUE/programs/ccli/)

Graduate Teaching Fellows in K-12 Education (GK-12) - opportunities for graduate and upper division undergraduate students in STEM disciplines to work with mentor teachers in the nation's K-12 schools.  (http://www.ehr.nsf.gov/dge/program/gk12)

Interagency Education Research Initiative (IERI) - Supports the establishment of a strong research base for education and learning, particularly in investigating scaling-up of proven interventions and the role of instructional technologies.  IERI is a joint research activity between the NSF, the National Institute of Child Health and Development of NIH, and the Department of Education.  (http://www.nsf.gov/cgi-bin/getpub?nsf0192)

Math and Science Partnership (MSP) – more information about this new grant program can be found at http://www.ehr.nsf.gov/ 

National Science, Mathematics, Engineering, and Technology Education Digital Library (NSDL) - online networks of learning environments and resources for STEM education at all levels.  (http://www.ehr.nsf.gov/EHR/DUE/programs/nsdl/)

Research on Learning and Education (ROLE) - research in four areas: (1) Brain research as a foundation for research on human learning; (2) Fundamental research on behavioral, cognitive, affective, and social aspects of human learning; (3) Research on STEM learning in formal and informal educational settings; and (4) Research on STEM learning in complex educational systems.  (http://www.ehr.nsf.gov/EHR/REC/)

Science, Technology, Engineering, and Mathematics Teacher Preparation (STEMTP) - development of exemplary science, technology, engineering, and mathematics teacher preparation models.  (http://www.ehr.nsf.gov/EHR/DUE/programs/stemtp/)