Links for Portland Parents of Talented and Gifted Children

Mathematics Instruction in Portland

Materials approved by the Oregon State Board of Education for 2009-15

My comments about the math adoption by Portland Public Schools in 1999

Skip my comments and go to the reviews/samples

About Singapore Math

go to other mathematics links on this page

HOME

Mathematics Links

Carolina Biological Supply Co., Math Out of the Box, c. 2009

Sites related to the "math wars" or mathematics education

NEW What's Wrong with the New Math Curriculum? a post by Karen Smith, a parent in Maryland

NEW More Oregon students are getting math class, By Betsy Hammond, The Oregonian November 07, 2009, 7:50PM
"Oregon math teachers have moved middle schoolers far enough ahead in math that the typical eighth-grader now can do math at nearly the same level as many high school sophomores.
Middle school students in every racial, ethnic and income group show greater mastery of mathematics -- including algebraic reasoning, statistics and geometry -- than they did three years ago.
Educators attribute much of the progress to Oregon's embracing a national recommendation to drastically scale back the number of math topics covered in each grade. ......"
http://www.oregonlive.com/education/index.ssf/2009/11/more_oregon_students_are_getti.html

NEW Report of the National Mathematics Advisory Panel http//www.ed.gov/about/bdscomm/list/mathpanel/index.html

NEW Education Panel Lays Out Truce In Math Wars Effort to Fix 'Broken' System Sets Targets for Each Grade, Avoids Taking Sides on Method By JOHN HECHINGER for the Wall Street Journal March 5, 2008; Page D1

"A presidential panel, warning that a "broken" system of mathematics education threatens U.S. pre-eminence, says it has found the fix A laserlike focus on the essentials. The National Mathematics Advisory Panel, appointed by President Bush in 2006, is expected to urge the nation's teachers to promote "quick and effortless" recall of arithmetic facts in early grades, mastery of fractions in middle school, and rigorous algebra courses in high school or even earlier. Targeting such key elements of math would mark a sharp departure from the diverse priorities that now govern teaching of the subject in U.S. public schools."

http://online.wsj.com/article/SB120465579132610785.html

Final Report on the National Survey of Algebra Teachers for the National Panel,  September 27, 2007. Recommended

Most Algebra teachers felt their students' preparation was "weak" and the skill areas of greatest concern were rational numbers, word problems and study habits.  A slight majority of the teachers consider a lack of ability grouping to be a moderate or serious problem--and teachers in schools that did not offer ability grouping were more likely to consider this a serious problem (See Research Question 11)

http://www.ed.gov/about/bdscomm/list/mathpanel/final-report-algebra-teachers.pdf

National Council of Teachers of Mathematics, Focal Points http://www.nctm.org/focalpoints/intro.asp  The new grade-based skills recommended by the NCTM

Content Review of CPM mathematics by Wayne Bishop, Department of Mathematics and Computer Sciences, California State University, Los Angeles:

 "Much of Volume 1 actually detracts from developing algebraic competence. Almost all of the mathematical content is at the level of the Grade 7 standards or below, e.g., the equations to be solved all are, but the activities are still very time consuming and sometimes frustrating. The worst of all, however, is not teaching the power of algebra itself. Unit 4: 123 is TOOL KIT CHECK UP and it is mandated that it contain Guess and Check tables, and "cups and tiles to model solving equations."  This is not algebra and it is not college preparatory math, no matter what it calls itself. Eventually, Volume 2 starts teaching some algebra but it is too little and too late." http://mathematicallycorrect.com/cpmwb.htm

Conference Summary: Does Two plus Two Still Equal Four: What should our children know about math?  Sponsored by the American Enterprise Institute for Public Policy Research http://www.aei.org/cs/cs020304.htm

 "Some of my ideas about mathematics programs" by W. Stephen Wilson, professor of mathematics at Johns Hopkins University. http://www.math.jhu.edu/~wsw/ED/  Includes his comments on what students must know to be prepared for college mathematics http://www.math.jhu.edu/~wsw/ED/panel

 "Basic Skills versus Conceptual Understanding: A Bogus Dichotomy in Mathematics Education" by H. Wu from the American Educator.  This article is especially valuable for its discussion of the fundamental concepts embedded in common mathematics algorithms such as performing division of fractions by inversion and multiplication.

 http://www.aft.org/pubs-reports/american_educator/fall99/wu.pdf

 "Another Everyday math horror story" http://groups.yahoo.com/group/GTAletters/message/6821

  The Fordham Report on State Mathematics Standards: http://www.edexcellence.net/standards/math/math.htm

An Appraisal of Math Standards in 46 States, the District of Columbia, and Japan by Ralph A. Raimi and Lawrence S. Braden. A scathing series of comments on various state mathematics curriculum standards by two well-qualified individuals. Raimi is professor emeritus of mathematics at the University of Rochester and former chairman of the math department (and graduate dean) at that institution. Braden has taught mathematics and science in elementary, middle, and high schools for many years in Hawaii, in Russia, and now in New Hampshire, at St. Paul's school. He is a recipient of the Presidential Award for Excellence in Science and Mathematics Teaching. He holds a bachelor's degree in mathematics from the University of California and an M.A.T. in mathematics from Harvard. [from the website]. Oregon received a "D" from these authors.

The report is written in rather formal English and requires some patience but their comments deserve serious consideration by our state and local policymakers. Recommended.

League of Women Voters report on math education in California http://www.hobel.org/lwved/index.htm

The Education Committee of the Los-Altos-Mountain View League of Women voters has prepared a presentation entitled "Algebra And The New California High School Exit Exam: Will Our Children Be Prepared?" It contains numerous links to other reputable sources and seems to be the closest thing available to an unbiased history of the California math conflicts. Recommended.

2+2, Mathematically Correct website maintained by critics of "whole math" http://www.mathematicallycorrect.com/

GTAforum: the entire saga of an attempt by Montgomery County Public Schools (MCPS) to gain teacher training tied to the Connected Mathematics Program. Includes letters, newspaper articles both for and against. The opposition was led by the Gifted and Talented Association of Montgomery County (GTA). http://groups.yahoo.com/group/GTAletters/files/Mathematics/FUZZYMATH/

Math Forum From Swarthmore, provides a generally positive view of "math reform." also provides a Discussion list and many other links http://forum.swarthmore.edu/

"Math wars" from the McNeil/Lehrer news hour. Contains many references to Oregon. A typical News Hour report, with all sides represented http://www.pbs.org/newshour/bb/education/jan-june98/math_5-11.html

Parents Skilled at Math protest new Curriculum article from the L.A. times. Despite its title, a relatively balanced discussion of the California "math wars" http://mathematicallycorrect.com/colvin1.htm

The Politics of California School Mathematics: The Anti-Reform of 1997-99  By Jerry P. Becker and Bill Jacob.  From the Phi Delta Kappan. A criticism of the views and actions of the "whole math" opponents by two professors of education. http://www.pdkintl.org/kappan/kbec0003.htm

NEW: Where's the Math.com:  Washington State advocacy group http://www.wheresthemath.com/

See also: "Middle School Math Comparison for Singapore Mathematics ......" below
 

 A few of my favorite mathematics links

( These are not related to the math wars.)

NEW Abacus: the Art of Counting with Beads  http://www.ee.ryerson.ca:8080/~elf/abacus/

NEW Algebasics.  A straightforward introduction to Algebra with narrative and examples http://www.algebasics.com/

American Mathematical Society http://www.ams.org/

AMSER Math Links:  From the Scout Report's Applied Math and Science Education Repository, the section for Mathematics http://amser.org/SPT--BrowseResources.php?ParentId=972655

Dave's Short Trig Course A Short Course in Trigonometry

Dr. Mike's Math Games http://www.dr-mikes-math-games-for-kids.com/

Euclid's Elements http://aleph0.clarku.edu/~djoyce/java/elements/elements.html

Harvard Math Circle: website created by a group that started math enrichment classes in Cambridge

http://www.themathcircle.org/history.php

Historical Activities for the Calculus Classroom: a set of problems that challenged mathematicians in the past with illustrations and solutions

 http://mathdl.maa.org/convergence/1/?pa=content&sa=viewDocument&nodeId=1581

History of Mathematics website: From Clark University. Not as well organized as it might be, but a storehouse of information, including biographies of over 1,000 mathematicians. http://aleph0.clarku.edu/~djoyce/mathhist/mathhist.html

Hoagies Kids and Teens Math  Links http://www.hoagiesgifted.org/math.htm a very entertaining collection of math contests, games, and child-friendly sites from the Hoagies site for gifted children.  Recommended

The Integrator http://integrals.wolfram.com/

Internet Resources in Mathematics: From Langara College in British Columbia, a very large, well organized collection of sites that have been reviewed and annotated. Recommended. Note especially the pages for "resources, lists, and catalogues--General" which is a very large set of resources organized by topic. http://www.langara.bc.ca/mathstats/resource/internet.htm

NEW Kathy Shrock's Guide for Educators. Straightforward list of Math websites http://school.discovery.com/schrockguide/math.html

NEW The KnotPlot site http://www.pims.math.ca/knotplot/

NEW Life of Fred: a series of mathematics textbooks popular with home-schooling families.  Fred encounters problems in his life and then used mathematics from arithmetic to calculus to solve them.  Contains some mild Christian references. http://www.stanleyschmidt.com/FredGauss/index2.html

MacTutor History of Mathematics from St. Andrew's University.  Includes an Index of Famous Curves, Birthplace Maps, Biographies, Mathematical Societies, timelines and so on.  Well-organized and informative.

Mathcasts: A collection of demonstrations of math operations from elementary school through calculus.  Requires the latest version of the Macromedia Flash Player.  http://www.mathcasts.org/index.php?title=Main_Page

Mathematical Association of America Classroom Capsules collection of material issued by the MAA in printed form, now online http://mathdl.maa.org/mathDL/20/

Mathematical Fiction compiled by  Alex Kasman, Associate Professor in the College of Charleston Department of
Mathematics, listed by category (e.g. mystery, fantasy, children's books).

http://math.cofc.edu/faculty/kasman/MATHFICT/default.html

Mega-Math:  A colorful and entertaining introduction to such topics as knot theory, the four color theorem etc .http://www.c3.lanl.gov/mega-math/

NEW Mathematical Imagery http://www.ams.org/mathimagery/  A great collection of links to math images but the one for the Visual Mathematics Journal (the bottom on the left) is LOADED with objectionable pop-up ads.  The others seem fine.

NEW Napier's bones. From the Mathematical Association of America, a site about Napier and a demonstration of how his "bones" functioned .http://mathdl.maa.org/mathDL/3/?pa=content&sa=viewDocument&nodeId=1514

Online Teachers Resource Network in Australia, Math Links  http://www.otrnet.com.au/Weblinks/lesson_resources.html

The Opinionator: Entertaining math blog from the New York Times, aimed at the "common reader" (i.e. not math specialists) http://opinionator.blogs.nytimes.com/author/steven-strogatz/

The Pacific Institute for the Mathematical Sciences http://www.pims.math.ca/ at the University of British Columbia offers a host of appealing mathematical webpages.  Select "Education" to find copies of their newsletter "Pi in the sky" which offers a wide range of math articles, problems and activities including cartoons http://www.pims.math.ca/education/  Topologists might want to install their Knot Plot.at http://www.pims.math.ca/knotplot/download.html

Mathematical Fiction
 

Plus Magazine  From Great Britain.  Lively and enjoyable.  Don't miss their article on the Parliamentary debate on the quadratic equation in issue 29.  http://plus.maths.org/index.html

Sphere Eversion a cool 20 minute video on topology:

http://video.google.com/videoplay?docid=-6626464599825291409

Wolfram MathWorld http://mathworld.wolfram.com/ created and edited by Eric W. Weisstein.  See also the section called MathWorld Classroom http://mathworld.wolfram.com/classroom/

Zona Land  http://id.mind.net/~zona/

About Singapore Math

Singapore (the country) astonished educators by the very high performance of its students in the TIMSS (Third International Math and Science) tests--the international equivalent of NAEP in 1995 and again in 1999. Some people wondered what they were doing right and decided to take a look at the curriculum they were actually using in Singapore schools. Singapore students are taught in English. A company here in Oregon City decided to make these books available to American families and they have become very popular with TAG families and homeschooling families across the country. A few school districts have experimented with the series and have had very good results.

The Singapore Primary Math series is one of many different approaches to math that are on the list of materials approved for adoption by the State of Oregon

http://www.ode.state.or.us/search/page/?id=1565

It was also approved by California

I am not associated in any way with the Singapore math books--I don't know anyone in the company and have read only one of the books all the way through. If you want to learn more about it, go to the company site at http://www.singaporemath.com/Default.asp

They also offer a very limited set of news articles on the site. My understanding, however, is that Singapore math is not simply a "back to basics" approach but also teaches thinking skills.

It's hard to find unbiased stories about Singapore Math. I have been following the saga of its use in Montgomery County, Maryland because I know someone there. It was adopted on a pilot basis in 2001.  An article about the pilot appeared in the Washington Post on October 17, 2001--see

http://www.washingtonpost.com/ac2/wp-dyn?pagename=article&node=&contentId=A9422-2001Oct17

 The principal supporters of the program were TAG parents who were frustrated by the district's slow math curriculum.
A study by the school district's own evaluation staff found that students in the Singapore pilot program schools significantly outperformed students in control programs after two years
And then....the Superintendent dropped the program on the grounds that it was inadequately aligned with the state curriculum standards

    See also the testimony of John Hoven which provides a detailed comparison of the work expected of American students (as evidenced by the questions on the National Assessement of Educational Progres, NAEP) and that routinely expected in Singapore "Testimony of John Hoven On Behalf of The Center for Education Reform"  This testimony was given in 2001. 

http://edreform.com/_upload/NAEPmath.pdf

and "Miracle Math" by Barry Garelick, an article on the Montgomery County story

http://www.hoover.org/publications/ednext/3853357.html

Wikipedia article http://en.wikipedia.org/wiki/Singapore_Math_Method

For the use of Singapore Math in the Ingenuity program in Baltimore see http://www.ingenuityproject.org/

"Middle School Math Comparison for Singapore Mathematics, Connected Mathematics Program (CMP) and Mathematics in Context, a Summary (Including Comparisons with the NCTM Principles and Standards 2000"  A summary of the November 2000 report submitted to the National Science Foundation by the Department of Applied Mathematics University of Washington.

http://www.wheresthemath.com/images/Singapore_Report_condensed.pdf

Finds that the best match with the NCTM standards is in the CMP and that the Singapore series fails to teach "higher order thinking skills" and relies on knowledgeable teachers.  However, it also includes the following comments:
 "The Algebra level in CMP and MIC appear to be almost two grade levels lower than in the Singapore materials.  ... It is also our prediction that students wishing to take calculus before the end of their 12^th grade year are likely not to be on track to do so after completing 8^th grade CMP or MIC, but would be ready to do so after completing Singapore s SL2.  We are not advocating that calculus in high school should be a goal for all students, but if this is the desired goal for certain students, the proper supplementation of CMP and MIC at an accelerated pace cannot be ignored.  Moreover, we are skeptical about the possibility of maintaining the interest of high-end students while progressing at the pace necessitated by the discovery process, if care is not taken to individualize these discoveries for the students" (pp. 43- 49)

NEWSPAPER STORIES:

Making Math Lessons as Easy as 1, Pause, 2, Pause by Winnie Hu in the New York Times, Sept. 30, 2010 http://www.nytimes.com/2010/10/01/education/01math.html

...In contrast to the most common math programs in the United States, Singapore math devotes more time to fewer topics, to ensure that children master the material through detailed instruction, questions, problem solving, and visual and hands-on aids like blocks, cards and bar charts. Ideally, they do not move on until they have thoroughly learned a topic. Principals and teachers say that slowing down the learning process gives students a solid math foundation upon which to build increasingly complex skills, and makes it less likely that they will forget and have to be retaught the same thing in later years. And with Singapore math, the pace can accelerate by fourth and fifth grades, putting children as much as a year ahead of students in other math programs as they grasp complex problems more quickly....

Singapore math a success so far in Fayette Co. By Jim Warren, Lexington Kentucky Herald Leader http://www.kentucky.com/142/story/1109176.html

.....Polly Anna Cox taught a calendar math lesson to her fifth-grade class at Liberty Elementary School, one of nine Fayette County schools that instituted a full Singapore math curriculum this year. Cox says she is teaching math at a "much higher level now."... Participating students are using a textbook called Math in Focus, which essentially is identical to the text used by about 80 percent of elementary students in Singapore, a tiny Asian city-state whose kids have been hitting the ball out of the park on international math assessments since the late 1990s. Fayette County is hoping for similar results.

At L.A. school, Singapore math has added value  By Mitchell Landsberg Los Angeles Times Staff Writer March 9, 2008

"Here's a little math problem. In 2005, just 45% of the fifth-graders at Ramona Elementary School in Hollywood scored at grade level on a standardized state test. In 2006, that figure rose to 76%. What was the difference?

If you answered 31 percentage points, you are correct. You could also express it as a 69% increase.

But there is another, more intriguing answer The difference between the two years may have been Singapore math.

At the start of the 2005-06 school year, Ramona began using textbooks developed for use in Singapore, a Southeast Asian city-state whose pupils consistently rank No. 1 in international math comparisons. Ramona's math scores soared.

"It's wonderful," said Principal Susan Arcaris. "Seven out of 10 of the students in our school are proficient or better in math, and that's pretty startling when you consider that this is an inner-city, Title 1 school."

http://www.latimes.com/news/local/la-me-math9mar09,1,2133870.story?track=rss&ctrack=1&cset=true

http://articles.latimes.com/2008/mar/09/local/me-math9

Comments made at a public Portland School District Math Forum, April 21, 1999 by Margaret DeLacy

[Corrections/comments are in square brackets]

My concerns are over the following issues:

• 1) PROCESS

• 2) CONSONANCE WITH STATE BENCHMARKS/GOALS/REQUIREMENTS

• 3) IMPLEMENTATION and TIMING

• 4) OVERALL QUALITY 0F THE PROGRAM

1. Process for adoption:

A committee of teachers recommended the adoption without any opportunity for parent or community imput. This is in accord with the district's written policies and procedures, but not in accord with state law. OAR 581-022-1640 states that "The school district process for selecting and adoption instructional material shall include opportunities for citizen and parent involvement." The Superintendent has already informed us that he intends to recommend the adoption of this book to the School Board on Monday--before parents/community members had any opportunity to review the material or to comment on it the decision has been made.

In addition, parents have not had an adequate opportunity to review these materials because, contrary to assurances we were given, they were not made available in the Professional library prior to tonight's meeting. I was able to track down a partial set in the office of a school district employee and to review that part. [In subsequent discussions, we heard that the materials had been available in the library for some time prior to the publication of the article in the Oregonian concerning the math adoption--but parents testified that they were not made aware of this].

2. State law further requires that the adopted materials "shall contribute to the attainment of district, program, and course or grade level goals." After reviewing the materials that were available for the fifth grade sequence, I feel that they do not provide the instruction students will need to meet the benchmarks. In several critical elements, for example, in finding and using the concept of "perimeter" in the geometry section, or in understanding and using the concept of temperature, there is no curriculum at all. More important, the materials are inappropriate for use with many students including ESL students, disabled students, and TAG students. The materials are so inappropriate for TAG students that in my judgment they should not even be used if they were accelerated by a full set.

3. The materials pose very serious implementation problems which raise doubts in my mind as to whether they could ever be adequately used in most classrooms. For example, the geometry sequence requires a very extensive use of computers equipped with geo/logo. In schools with fewer than five computers in the classroom, the students cycle through the computer activities throughout the day, causing considerable disruption to other instructional programs. The geo/logo instructions must be posted next to the computer, making it more difficult to use a shared computer lab for this project. One middle school project requires that students have bicycles equipped with multiple gears and derailleurs.

There are very burdensome preparation requirements. For example, teachers must first learn how to install, run and program geo/logo themselves. The district is planning to print a set of assignment masters for each student to save teachers from having to copy them but I haven't seen cost estimates for printing, binding, and distributing approximately 40,000 sets. [In retrospect this is probably not correct because I am not certain the middle school students will also require copied sets--The true number is probably more like 23,000 sets] In addition, I haven't seen any cost estimates for having a teacher in every school who is responsible for ensuring that there is communication about the program as promised by the adoption committee--these individuals would require extended responsibility pay. The elementary materials are very bulky and awkward to handle and there is NO INDEX.

In addition, the timing is unfortunate, because the teachers are already struggling with the new literacy adoption. The district does not have enough planning time for both in addition to the time needed for CIM/CAM implementation. The materials are so lacking in practice opportunities that even the adoption committee has agreed that the district must also supply workbooks by another publisher. However there is no curriculum that integrates these workbooks and no textbook that accompanies them--they are clearly an afterthought and will be treated as such by our teachers.

4. The overall quality of the materials is poor. The pace is very slow, the level of sophistication embarrassing. For example, although the Oregon baseline standards expect students to "use appropriate mathematical terminology" the materials for FIFTH grade geometry refer to an angle measuring device as "A TRANSPARENT TURTLE TURNER." The word "protractor" is not used. This same project requires that a teacher spend two days on the definition of a triangle even though the baseline standards expect our THIRD GRADE students to be able to "recognize, describe, and provide real-world examples of line segments, angles, triangles, quadrilaterals, pentagons, hexagons, octagons, spheres and cubes." Most of the rest of the fifth grade unit is devoted to developing the concept of angle--again, our THIRD GRADE students are supposed to have accomplished this. The suggested homework assignment for that week is an essay on the topic, "are all triangles three-sided polygons?"

There are no constructions using compass and ruler. There is a discussion of right angles and triangles, but I did not see any reference to the Pythagorean theorem, or to congruent and similar triangles, or to the use of triangulation for measuring distances. (Maybe because I didn't take long enough with the materials--I couldn't look for these topics because there was NO INDEX).

A sixth grade project in the middle school sequence provides a table of 20 metropolitan areas and has students order them by size, describe how they could be compared, and locate them on a map. There are two questions, "what geographic factors seem to lead to large population centers?" and "How do you think the locations of these large metropolitan areas affect national and state government and business decisions? The level of mathematics required for this project would be appropriate for a fourth grade student, although the student would find sorting 20 large numbers somewhat tedious. The other questions are suitable for social studies, but have nothing to do with developing math competence.

Other reviewers have commented that the elementary sequence does not explain long division and does not include paper and pencil multiplication in second grade. I cannot comment on this because the appropriate section of the curriculum was missing. When I asked a teacher here tonight to show me the section that dealt with multiplication, she had trouble finding it, because there is no index. I did find that, when multiplication of two digits by one digit was introduced at last (in the fourth grade set), the only discussion that included the numbers wrote them horizontally--children were not shown how to write the multiplication problem vertically and then to carry the number to the tens place.

5. Classroom projects and materials of the sort included in this book might make an appropriate supplement to a math curriculum. They do not constitute one. Moreover, there are other "whole math" sequences that might provide better quality material.

Reviews of these materials by other parents:

From parent authors opposed to the "whole math" in California

Second Grade Books:

Fifth Grade Books

Seventh Grade Books

From another California parent, comparing this sequence to two other "whole math" sequences. This parent spent eight months studying the books for a math book selection committee and interviewing teachers who had used them in the classroom. Comments from teachers indicate dissatisfaction with the program and seriously lowered scores for students in math computation "the lowest we had ever seen" although scores in conceptual thinking rose. Comments for Everyday Math, another "whole math" sequence indicated a much higher level of satisfaction.

The parent concluded: "I tried to present the highlights from three activities in Investigations. I did not pick these examples to show minor mistakes with Investigations. Rather I picked them to illustrate how its philosophy reduces the effectiveness of many of its activities, and makes it fall short of its potential. The authors start with good ideas, but they often don't execute them very well. If the authors could moderate their philosophy and improve their mathematics, Investigations could become an excellent supplement.

Whole Review

High School/Algebra books:

Algebra I, (California parent reviewers rated it second from the bottom of the 12 standard Algebra books reviewed.) The Overall Summary from this review says: "The book provides a poor opportunity for student learning, with support for only basic levels of achievement. The appropriateness of the use of technology and the emphasis on analytic methods are fair, but the mathematics content coverage and depth are seriously insufficient.The presentation style is poor in terms of the mathematical understanding supported, and the presentation and the practice exercises seriously lack sufficient breadth and depth."

Geometry

Algebra II

Check out the materials for yourself : Sample lessons from the publisher's web pages:

Elementary grades: Elementary Investigations in Number, Data, and Space

Middle School: Connected Mathematics

High School: Explorations and Applications,

[Note: this h.s. file appears to be unavailable now.  Information can be found on the PPS mathematics department site]  

The U.S. Dept. of Education has designated "Connected Mathematics" an "exemplary program," and "Investigations in Number, Data and Space" a "promising program." The reviews require an Adobe Acrobat reader. They were partly based on the congruence of the program with the NCTM and AAAS standards. To be found "exemplary" the publisher must submit convincing evidence that the program improves student performance.

HOME