Dealing With Social Promotion's Ills
To review our services please scroll down to Asora® Report Card Services.

Background and Summary
Based partly on tradition and partly on the requirements of state and federal legislation, including that of No Child Left Behind, achievement testing in mathematics and reading is administered in nearly every state. They report on every district and every school within their public education systems. Separately, at the federal level, there is the National Assessment of Educational Progress (NAEP)- also known as the Nation’s Report Card. Important characteristics of the NAEP include:

• A long track record of 39 years, establishing itself as the defacto national standard for achievement.
• The NAEP achievement level of “proficient” defines what it is to be at “grade level.”
• Content standards tested by the NAEP are drawn from the National Assessment Governing Board.
• NAEP tests children in 4
th and 8th grades and reports proficiencies at the state level.
• In 12
th grade NAEP tests and reports only at the national level.
• The use of statistical sampling techniques prevents reporting at district and school level.

The tests administered by the states also claim to determine which students have become “proficient” in the tested subject areas. Generally, the state administered achievement tests:

• Examine sufficient numbers of children to report statistically valid results for schools and districts.
• But measure against content standards and use cut-scores inconsistent with the NAEP.
• Most states significantly inflate the number of children deemed “proficient” compared to the NAEP.
• Inflation is always accompanied by differential inflation in which disadvantaged groups suffer markedly higher levels of inflation than the state average. The cynic views this as optimally applying the "cosmetic" where it is needed the most to cover up the most dysfunctional groups. Others, bent on helping such children, seek realistic proficiency measures to show their dire needs.
• Some grossly inflate their proficiencies to as much as 300% or more above the NAEP results. The median state inflates at about 100%.
• In three New England states, where the NECAP test is used, a number of problems have been identified. We believe the NECAP has too many flaws. Rather than overhauling this assessment regimes we think replacing it makes more sense. Click
here to learn more- including a speech about its dysfunctional nature.

For stakeholders at the school and district levels, the well-respected NAEP results are not available and the state generated results are unreliable, due to the inflation. To address this lack of useful information:

• We developed a conversion formula (mapping procedure) to estimate what the NAEP proficiencies would have been in these localities. It essentially removes the inflation from the state reported scores.
• Error measurements of this procedure in simulated environments have shown good accuracy.
• Where actual results are known independently we have verified that they are reasonably accurate.
• The method’s errors are measured to be of the same order as the NAEP reported sampling errors.
• In terms of proficiency percentages, the measured errors have rarely exceeded 5%.

By applying these mapping procedures we have constructed report cards for every public school and every school district within the states of Oklahoma, New Hampshire, New Mexico, Pennsylvania, Rhode Island, Tennessee and Vermont. Additionally we have studied Ventura County in California, Bristol County in Massachusetts, and Hudson County in New Jersey. Our findings include:

• In the worst performing schools, less than 5% of children are proficient in both reading and math.
• Among the “best” performing schools, it is rare to find more than 70% of the students proficient.
• Mathematics proficiency percentages are generally lower than those for reading during the high school years.

Our evaluation of the analysis conducted thus far suggests:

• In all jurisdictions studied, social promotion is the key dysfunction leading to the low proficiencies.
• Age based group instruction hinders the ability to properly “place” students consistent with skill levels.
• Strict retention policies, such as used in India, could correct and eliminate social promotion, but we think such methods are too draconian.
• However, replacing group instruction with self-paced (probably online) instruction would be effective.
• Reform the math curriculum first and extend it to all math courses. A few such efforts are underway, including our own
Stellar Schools project.

Asora® Report Card Services:
We are ready to work with interested clients who seek to obtain more realistic estimates of student proficiencies in the K-12 public schools within their states or in other client defined regions.

To conduct its analysis, Asora Consulting uses as its input data:

• Recently published results from the Nation’s Report Card- also known as the National Assessment of Educational Progress (NAEP), which it obtains from trusted data repositories or from the U.S. Department of Education directly.

• Recently published results from the state’s achievement test as reported by the states directly or through the Council of Chief State School Officers' data repository at SchoolDataDirect.

District Proficiencies
After applying our mapping procedure to the input data, we can generate estimates as to how students would have performed on the Nation’s Report Card on a district-by-district basis. We generally provide estimates at the primary school level (4th grade), the middle school level (8th grade) and at a high school grade level (which varies from state to state according to the level tested by the state’s examination). Results are presented in terms of the percentages of children measured to be proficient or better. According to the definition of proficiency used by the NAEP it also denotes the percentage of students who are at or above grade level. Given the fact that proficiency levels are nearly always “inflated” by the state administered examinations, the proficiency estimates we generate are low and troubling. Former President Bush’s remark about educators being engaged in “the soft bigotry of low expectations” is consistent with these estimates.

School Proficiencies
In a similar manner we can produce estimates for individual schools. These estimates are useful in showing the degree to which schools are dysfunctional. It is not uncommon to find schools with proficiency percentages well less than 5%. We believe that a properly operated school will maintain student proficiencies above 90%. However, among the public schools we have checked none has ever reached that level.

Private School Proficiency Estimates
In most states, private school students do not take the state’s achievement tests. Where they do, we can generate NAEP scale estimates for them in the same way we generated the public school estimates. For those schools that do not take the state’s achievement tests but take the SAT or ACT tests we can make a rough estimate of the 12th grade proficiency levels of those schools. For example, we performed such an analysis in Rhode Island and found only two private schools for which we estimate proficiency percentages above 90%. Many private schools do not release their SAT or ACT scores. In those cases we are unable to make estimates.

Commissioned Reports
Depending on the client’s needs and wishes we can provide reports based on the performance estimates generated. The report could range in length and complexity from:

• A short description of the findings with selective presentation of the estimates.
• To a more in-depth analysis of the findings with a complete presentation of the estimates generated.

We will work with clients to produce reports suitable for their needs.

Clients will hold the copyright to reports generated for them and will be permitted to revise and extend them subject to the requirement that Asora Consulting will be cited for its contributions.

Some Examples
We have analyzed student proficiencies in Oklahoma, New Hampshire, New Mexico, Pennsylvania, Rhode Island, Tennessee, and Vermont where we provided NAEP estimates for every public school in these states. We also looked at a small number of private schools for which SAT scores were available.

Many of the reports resulting from these studies can be downloaded from our
Reform Reports page.

The derivations of our mapping method can be reviewed in the report ELQ-Mappings.docx and its accompanying spreadsheet ELQ-Derivation.xlsx which are also downloadable from our Reform Reports page.


If you are interested in reviewing the "big picture" of Asora's full breadth of activities,
click here to reach our home page.