Thank you for sharing!

Your article was successfully shared with the contacts you provided.
Daubert and its progeny apply to testing protocols used to evaluate alleged racial discrimination by real estate agents, a federal judge held Feb. 16 ( Metropolitan St. Louis Equal Housing Opportunity Council, et al. v. Gordon A. Gundaker Real Estate Co., No. 4:98CV837SNL, E.D. Mo., Eastern Div.). Senior U.S. District Judge Stephen N. Limbaugh of the Eastern District of Missouri found no reported cases in which behavioral and social science evidence was challenged under Daubert. “Given Rule 702′s broad field of ‘expertise’ and Kumho‘s broadening of the application of Daubert to all fields of expert testimony regardless of the field of expertise (i.e. no longer limited to fields historically considered to be ‘scientific’), the Court determines that the gate-keeping function of this Court as first set out in Daubert is applicable to the testimony that plaintiffs and the defendant seek to elicit through their respective experts,” he said. After applying Daubert, the judge determined that the testing protocol was seriously flawed and lacked any reasonable indicia of reliability. Therefore, the judge barred the plaintiffs’ experts from testifying that the design, implementation and data obtained from the tests were reliable. STUDY LAUNCHED Plaintiff City of Florissant, outside St. Louis, launched an investigation with the Metropolitan St. Louis Equal Housing Opportunity Council (EHOC) in 1996 to determine if area real estate agents were violating the Fair Housing Act by steering potential homeowners to specific neighborhoods because of their race. The first EHOC “audit” found five of seven area agencies engaged in racial steering, including two offices of Gordon A. Gundaker Real Estate Co. A follow-up audit in 1997 again found violations by Gundaker. Armed with the 1996 and 1997 test results, the EHOC and Florissant sued Gundaker for FHA violations. The tests involved sending black and white couples, who were supposed to be similar in all respects but race, to the agencies looking for similar homes. The testers were to report back on the location of homes they were shown and how they were treated. In finding the tests unreliable, the judge noted that neither test administrator had an extensive background in fair housing matters, or fair housing testing. Neither were especially familiar with U.S. Department of Housing and Urban Development audit studies nor with Heights Community Congress v. Hilltop Realty, Inc. (629 F. Supp. 1232 [N.D. Ohio 1983], aff’d in part, rev’d in part 774 F.2d. 135 [6th Cir. 1985]). Neither used the testing protocol established in the HUD studies or modified in Hilltop as a standard for developing their testing protocols. There was ambiguity in the test instructions as to the target housing area; some forms targeted the “City of Florissant,” while others targeted the “Florissant area.” Some test forms contained an initial telephone contact form, others did not. Some test forms contained a debriefing form to be completed by the test coordinator, some did not. INCONSISTENCIES “More worrisome was the inconsistencies in the execution of the tests. Several of the testers failed to follow the fabricated script; i.e. significantly altering their personal background, the reason for contacting the agent, changing the type of house sought,” the judge said. “Some failed to complete the test form or left parts blank. The testers differed as to their response to follow-up calls by the agents; some viewed additional homes, some did not. On some test forms, the number of homes shown were not the same for both testers of a ‘couple’ team. “In the 1996 tests, one agency was audited by repeated contact with the same agent (who happened to be the owner) while the other agencies were audited by repeated contact with different agents. The debriefing of testers varied greatly. Especially in 1997, there was little or no consistency in the timing or method of debriefing. In fact, at least one tester was never debriefed.” The judge rejected arguments by the city and EHOC that there was a distinction between “research based testing” and “enforcement based testing.” They contended that because their testing was “enforcement based testing,” social science research standards, HUD audit standards, and the standards espoused in Hilltop are inapplicable. The judge found the plaintiffs offered no evidence that supported their contention that a distinction exists between enforcement and social science fair housing testing. TECHNIQUE FAULTED “Whether the test results are used as the basis of HUD research study, a social science study, or as evidence in a lawsuit, the techniques employed by testers in gathering the data is the same,” the judge said. “More importantly,” the judge said, “the plaintiffs failed to articulate a set of standards which govern the validity and reliability of an ‘enforcement test.’ “There simply was no standardized protocol utilized in the design and execution of the EHOC tests,” the judge said. “The Court is not dismissing the fact that tester evidence is probably the best means by which to demonstrate fair housing violations. However, the production of such evidence must be done under some set of standards which assure a reasonable level of validity and reliability.” EXPERTS LIMITED While the judge made this determination as to the tests themselves, he did not conclude that the unreliability of the testing methodology alone rendered the plaintiffs’ experts testimony inadmissible. Turning to the experts, the judge found inadmissible the opinions of Shanna Smith, executive director of the National Fair Housing Alliance, regarding voice identification of race over the telephone. Without any training or supporting studies, Smith is unqualified to testify in this regard, the judge said. Also, her lack of training as a statistician or social scientist barred her from testifying about the EHOC’s methodology. The judge likewise barred the testimony of Clifford Schrupp, executive director of the Fair Housing Center in Detroit, regarding the EHOC test data. Schrupp contended that the EHOC tests were on solid foundation for “enforcement testing” because they followed a protocol developed through consultation with attorneys. The judge noted that Schrupp has no social science degree and is not a lawyer or statistician. “As for the standards he espouses for an ‘enforcement test’ he fails to point to any authority, treatise, journal, or any written body of knowledge that supports his ‘legal’ standards for the testing done in this case,” the judge said. “Mr. Schrupp’s theory on fair housing testing has not been subject to any peer review or publication, its known potential rate of error has not been tested, and it has not been shown to be generally accepted as reliable by the relevant social science community.” Dr. William Peterman, a doctor in urban geography, performed three analytical tests on the data provided by EHOC. The judge noted that Peterman did not review any of the original underlying data and assumed that the information the EHOC provided from the testers was accurate. FAULTY TESTS One test Peterman admitted he applied incorrectly. Another, the chi-square statistical test, when applied to this type of data, had not been tested or subject to peer review or publication or been generally accepted in the relevant scientific community for use with matched pair data, the judge said. Peterman’s final test was the “Nearest Neighbor Test,” which studied the geographic distribution of the various properties. The judge noted that the test is widely accepted in the field of geography, not in any field of social science. The judge determined that Peterman’s testimony and expert report are inadmissible for failure to meet the Daubert criteria on several grounds — his failure to review the underlying data for accuracy, his use of an incorrect test because he was not familiar with statistical testing in the area of fair housing, and the irrelevance of the Nearest Neighbor Test. “Although Dr. Peterman could be considered qualified as an expert in statistical analysis of geographic data, the Court finds that he lacks sufficient knowledge through training, education, or experience to be considered qualified to offer an expert opinion regarding matched pair data analysis and/or whether the EHOC tests provide evidence of racial steering,” the judge said. The judge said he would permit rebuttal testimony from plaintiff expert Susan Scovill, an attorney and director of fair housing for Housing Opportunities Made Equal, regarding education of real estate agents. While the judge barred the plaintiff experts from testifying about the reliability of the tests, it concluded defense experts could testify about reliability of methodology. DEFENSE EXPERTS ALLOWED Dr. David Graeven, a sociologist who has conducted social science research for 35 years, reviewed EHOC’s methodology in designing and executing the tests, the depositions of the testers, materials supplied by EHOC regarding its tester training program, and the depositions of EHOC trainers. Graeven did not assess the credibility of the EHOC staff and/or testers. He opined that in order to conduct a competent and reliable study, whether it be for research or enforcement, certain fundamental precepts of social science must be followed. Foremost, Graeven said that in order to determine validity, one must be able to measure reliability and dependability of data against certain social science research standards, including but not limited to: testers must be properly trained, all data must be included, all variables except one (in this case, race) must be controlled to isolate the one variable as making the difference in results, test results must be properly and consistently analyzed through debriefing and the process of collecting data via the testers must be consistent. He opined that the HUD studies’ protocol should have been considered by EHOC because it has generally been accepted as a valid method for investigating alleged racial housing discrimination. Graeven concluded that the EHOC test protocols were severely flawed as to design and execution because the EHOC staff failed to use any standardized testing procedure other than to implement the suggestions of attorneys as to preservation of evidence for trial. There were marked differences in training of the testers over the two years. There were inconsistencies in the information sought on the test forms. Graeven found that testers executed the tests in a variety of ways, including changing the script. Some testers did not complete the test form. Debriefing of testers became significantly inconsistent and took a variety of forms. EHOC and the city argued Graeven’s testimony regarding the plaintiffs’ testing methodology should be inadmissible because he used the “wrong” standards. However, the judge again said that the plaintiffs failed to articulate the standards that should be applied to assess the validity and reliability of the EHOC testing methodology, even if the tests would be considered for “enforcement purposes.” ACCEPTED METHODS “Dr. Graeven’s proposed social science research testing standards are widely accepted in the social science research community and have been subjected extensively to peer review and publication,” the judge said. “His testimony is based upon approved testing procedures. The Court finds that Dr. Graeven’s testimony and expert report meet the criteria set down under Daubert and its progeny for scientific validity and reliability. Furthermore, the Court finds such testimony to be relevant to the facts in this case and of assistance to the jury in adjudging such facts. Dr. Graeven’s testimony and expert report is admissible under Rule 702.” Similarly, the judge said he would allow the testimony of Dr. Mike Tiktinsky, who has doctorate degrees in psychology and mathematical/statistical psychology. Tiktinsky reviewed the EHOC testing methodology and the analytical methodology used by the plaintiffs’ experts, Graeven’s and Peterman’s expert reports, numerous articles on measuring discrimination in housing studies, the HUD studies and the Hilltop decision, the data set compiled by EHOC staff and the testers’ report forms from which the data was derived. Tiktinsky concluded that the EHOC failed to follow the systematic protocol of the HUD studies and failed to follow the systematic protocol of established field research. While not embracing the HUD studies as the standardized protocol for racial housing discrimination testing, he recognized that they followed a generally reliable testing protocol which would have provided some measurement of validity and reliability to the EHOC tests. He concurred with Dr. Graeven as to the EHOC test design flaws and execution inconsistencies. FAULT FOUND He faulted Peterman for failing to review the underlying data for accuracy and for simply relying on the assertion that the testers’ data was properly collected. He found that the number of properties listed by the tester or tester couple did not always equal the number recorded in the EHOC data set. Testers working as a couple did not record the same number of properties shown or referred. He found that the test coordinators regularly recorded on the debriefing forms a different number of houses shown or referred than the testers actually listed on the test report form. Furthermore, he found that Peterson’s methodology for analyzing the EHOC test data lacked a systematic approach as what data would be included or excluded. He also opined that the EHOC tests were a “matched pairs” test and should not have been subjected to a chi-square test. The EHOC and city asserted that Tiktinsky’s testimony and expert report are inadmissible because he improperly attacked the credibility of the testers and he used the wrong standards for judging the testing methodology. The judge concluded that the arguments were meritless. Tiktinsky’s testimony regarding the reliability of the EHOC’s evidence is a challenge to the reliability of the testing protocol, the judge said, finding it goes to the core of the reliability of the plaintiffs’ experts’ analytical methodology. AMICUS BRIEF ALLOWED Finally, the judge said he would permit attorney and Realtor Donald MacPherson to testify as to the professional and ethical standards agents follow in Missouri. He may not give a legal opinion as to whether the EHOC followed the standards. The judge also granted a motion of the United States to intervene and file an amicus brief. Counsel to the EHOC are Kenneth M. Chackes and M. Susan Carlson of Van Amburg, Chackes, Carlson & Spritzer, LLP in St. Louis and Michael A. Ferry of Gateway Legal Services in St. Louis. John Gianoulakis of Kohn, Shands, Elbert, Gianoulakis & Giljum in St. Louis represents the city. Alan N. Zvibleman, of Rothman, Sokol, Adler & Sarachan, P.C., in St. Louis, Paul F. Devine of Vatterott, Shaffar & Dolan in Maryland Heights, Mo., and Daniel K. Barklage of Barklage, Barklage, Brett, Martin & Wibbenmeyer, P.C. in St. Charles, Mo., represent Gundaker. Counsel to the United States are Joan A. Magagna, Isabelle M. Thabault and Eric I. Halperin of the U.S. Department of Justice in Washington, D.C. � Copyright 2001 Mealey Publications, Inc.

This content has been archived. It is available through our partners, LexisNexis® and Bloomberg Law.

To view this content, please continue to their sites.

Not a Lexis Advance® Subscriber?
Subscribe Now

Not a Bloomberg Law Subscriber?
Subscribe Now

Why am I seeing this?

LexisNexis® and Bloomberg Law are third party online distributors of the broad collection of current and archived versions of ALM's legal news publications. LexisNexis® and Bloomberg Law customers are able to access and use ALM's content, including content from the National Law Journal, The American Lawyer, Legaltech News, The New York Law Journal, and Corporate Counsel, as well as other sources of legal information.

For questions call 1-877-256-2472 or contact us at [email protected]

Reprints & Licensing
Mentioned in a Law.com story?

License our industry-leading legal content to extend your thought leadership and build your brand.


ALM Legal Publication Newsletters

Sign Up Today and Never Miss Another Story.

As part of your digital membership, you can sign up for an unlimited number of a wide range of complimentary newsletters. Visit your My Account page to make your selections. Get the timely legal news and critical analysis you cannot afford to miss. Tailored just for you. In your inbox. Every day.

Copyright © 2021 ALM Media Properties, LLC. All Rights Reserved.