severity

Announcement: CFP Synthese Topical Collection:  Severity and Learning from Error

.

I hope that many readers of this blog will consider contributing to this!

ANNOUNCEMENT SEV26

 

Synthese Topical Collection CFP:  Severity and Learning from Error

This Topical Collection examines how inquiry learns from error by focusing on a basic principle of evidence in science, statistics, medicine, law, epistemology, and day-to-day learning: a claim is not well-tested, known or epistemically warranted, if it is based on a method that makes it easy to accept, conclude or infer the claim, even if it is false. Such a claim may accord well with the data, but it has not passed a stringent or severe test. While this overarching intuition is widely shared, the problem of how to understand or satisfy it remains unsolved. C. S. Peirce emphasizes randomization and (what is now called) pre-designation to achieve self-correcting methods. Popper viewed severity in terms of satisfying novel predictive success and surviving stringent attempts at falsification. Deborah Mayo (1996, 2018) combines elements from Popper and Peirce with the use of error probabilities from statistical methods: proposed solutions to problems earn warrant by surviving probes that were capable of showing them wrong or inadequate. This Topical Collection takes “severity” to be a broad meta-level concept according to which a claim – whether a report of a perception, a prediction, a hypothesis, or part of a model – is assessed according to whether, and how readily, its errors and inadequacies would have been found, if present. Continue reading

Categories: Error Statistics, SEV 26, severity | Leave a comment

“Are Controversies in Statistics Relevant for Responsible AI/ML? (My talk at an AI ethics conference) (ii)

Bayesians, frequentists and AI/ML researchers

1. Introduction

I gave a talk on March 8 at an AI, Systems, and Society Conference at the Emory Center for Ethics. The organizer, Alex Tolbert (who had been a student at Virginia Tech), suggested I speak about controversies in statistics, especially P-hacking in statistical significance testing. A question that arises led to my title:
Are Controversies in Statistics Relevant for Responsible AI/ML?”

Since I was the last speaker, thereby being the only thing separating attendees from their next destination, I decided to give an overview in the first third of my slides. I’ve pasted the slideshare below this post. I want to discuss the main parallel that interests me between P-hacking significance tests in the two fields (sections 1 and 2), as well as some queries raised by my commentator, Ben Jantzen, and another participant Ben Recht (section 3). Let me begin with my abstract: Continue reading

Categories: AI/ML, Ben Janzen, Ben Recht, biasing selection effects, severity | 18 Comments

Blog at WordPress.com.