Skeptical and enthusiastic Bayesian priors for beliefs about insane asylum renovations at Dept of Homeland Security: I’m skeptical and unenthusiastic

Danver State Hospital

Danvers State Hospital

I had heard of medical designs that employ individuals who supply Bayesian subjective priors that are deemed either “enthusiastic” or “skeptical” as regards the probable value of medical treatments.[i] From what I gather, these priors are combined with data from trials in order to help decide whether to stop trials early or continue. But I’d never heard of these Bayesian designs in relation to decisions about building security or renovations! Listen to this….

You may have heard that the Department of Homeland Security (DHS), whose 240,000 employees are scattered among 50 office locations around D.C.,has been planning to have headquarters built at an abandoned insane asylum St Elizabeths in DC [ii]. See a recent discussion here. In 2006 officials projected the new facility would be ready by 2015; now an additional $3.2 billion is needed to complete the renovation of the 159-year-old mental hospital by 2026 (Congressional Research Service)[iii].The initial plan of developing the entire structure is no longer feasible; so to determine which parts of the facility are most likely to be promising, “DHS is bringing in a team of data analysts who are possessed” said Homeland Security Secretary Jeh Johnson (during a DHS meeting, Feb 26) –-“possessed with vibrant background beliefs to sense which buildings are most probably worth renovating, from the point of view of security. St. Elizabeths needs to be fortified with 21st-century technologies for cybersecurity and antiterrorism missions” Johnson explained.

Failing to entice private companies to renovate the dilapidated west campus of the historic mental health facility that sits on 176 acres overlooking the Anacostia River,they can only hope to renovate selectively:  “Which parts are we going to overhaul? Parts of the hospital have been rotting for years!”Johnson declared.

Read more:

Skeptical and enthusiastic priors: excerpt from DHS memo:

The description of the use of so-called “enthusiastic” and “skeptical” priors is sketched in a DHS memo released in January 2014 (but which had been first issued in 2011). Here’s part of it:

Enthusiastic priors are used in evaluating the portions of St. Elizabeths campus thought to be probably unpromising, in terms of environmental soundness, or because of an existing suspicion of probable security leaks. If the location fails to be probably promising using an enthusiastic prior, plus data, then there is overwhelming evidence to support the decision that the particular area is not promising.

Skeptical priors are used in situations where the particular asylum wing, floor, or campus quadrant is believed to be probably promising for DHS. If the skeptical opinion, combined with the data on the area in question, yields a high posterior belief that it is a promising area to renovate, this would be taken as extremely convincing evidence to support the decision that the wing, floor or building is probably promising.

But long before they can apply this protocol, they must hire specialists to provide the enthusiastic or skeptical priors. (See stress testing below.) The article further explains, “In addition, Homeland Security took on a green initiative — deciding to outfit the campus’ buildings (some dating back to 1855) with features like rainwater toilets and Brazilian hardwood in the name of sustainability.” With that in mind, they also try to get a balance of environmentalist enthusiasts and green skeptics.

Asked how he can justify the extra 3 billion (a minimal figure), Mr. Johnson said that “I think that the morale of DHS, unity of mission, would go a long way if we could get to a headquarters.” He was pleased to announce that an innovative program of recruiting was recently nearly complete.

Stress Testing for Calibrated “Enthusastic” and “Skeptical” Prior Probabilities

Perhaps the most interesting part of all this is how they conduct stress testing for individuals to supply calibrated Bayesian priors concerning St. Elizabeths. Before being hired to give “skeptical” or “enthusiastic” prior distributions, candidates must pass a rather stringent panoply of stress tests based on their hunches regarding relevant facts associated with a number of other abandoned insane asylums. (It turns out there are a lot of them throughout the world. I had no idea.) The list of asylums on which they based the testing (over the past years) has been kept Top Secret Classified until very recently [iv]. Even now,one is directed to a non-governmental website to find a list of 8 or so of the old mental facilities that apparently appeared in just one batch of tests.

Scott Bays-Knorr, a DHS data analyst specialist who is coordinating the research and hiring of “sensors,” made it clear that the research used acceptable, empirical studies: “We’re not testing for paranormal ability or any hocus pocus. These are facts, and we are interested in finding those people whose beliefs match the facts reliably. DHS only hires highly calibrated, highly sensitive individuals”, said Bays-Knorr. Well I’m glad there’s no hocus-pocus at least.

The way it works is that they combine written tests with fMRI data— which monitors blood flow and, therefore, activity inside the brain in real time —to try to establish a neural signature that can be correlated with security-relevant data about the abandoned state hospitals. “The probability they are attuned to these completely unrelated facts about abandoned state asylums they’ve probably never even heard of is about 0. So we know our results are highly robust,” Bays-Knorr assured some skeptical senators.

Danvers State Hospital

Take for example,Danvers State Hospital, a psychiatric asylum opened in 1878 in Danvers, Massachusetts.

“We check their general sensitivity by seeing if any alarm bells go off in relation to little known facts about unrelated buildings that would be important to a high security facility. ‘What about a series of underground tunnels’ we might ask, any alarm bells go off? Any hairs on their head stand up when we flash a picture of a mysterious fire at the Danvers site in 2007?” Bays-Knorr enthused. “If we’ve got a verified fire skeptic who, when we get him to DC, believes that a part of St. Elizabeths is clear, then we start to believe that’s a fire-safe location. You don’t want to build U.S. cybersecurity if good sensors give it a high probability of being an incendiary location.” I think some hairs on my head are starting to stand up.

Interestingly, some of the tests involve matching drawn pictures, which remind me a little of those remote sensing tests of telepathy. Here’s one such target picture:

Target: Danvers State Hospital

They claim they can ensure robustness by means of correlating a sensor’s impressions of completely unrelated facts about the facility. For example, using fMRI data they can check if “anything lights up” in connection with Lovecraft’s Arkham Sanatorium”, a short story “The Thing on the Doorstep”, or Arkham Asylum in the Batman comic world.

Bays-Knorr described how simple facts are used as a robust benchmark for what he calls “the touchy-feely stuff”. For example, picking up on a simple fact in connection with High Royds Hospital (in England) is sensing its alternative name: West Riding Pauper Lunatic Asylum.They compare that to reactions to the question of why patient farming ended. I frankly don’t get it, but then again, I’m highly skeptical of approaches not constrained by error statistical probing.

 

Yet Bays-Knorr seemed to be convincing many of the Senators who will have to approve an extra 3 billion on the project. He further described the safeguards, “We never published this list of asylums, the candidate sensors did not even know what we were gong to ask them. It doesn’t matter if they’re asylum specialists or have a 6th sense. If they have good hunches, if they fit the average of the skeptics or the enthusiasts, then we want them.”  Only if the correlations are sufficiently coherent is a ‘replication score’ achieved. The testing data are then sent to an independent facility of blind “big data” statisticians,Cherry Associates, from whom the purpose of the analysis is kept entirely hidden.”We look for feelings and sensitivity, often the person doesn’t know she even has it,” one Cherry Assoc representative noted.Testing has gone on for the past 7 years ($700 million) and is only now winding up. (I’m not sure how many were hired, but with $150,000 salaries for part time work, it seems a god gig!)

Community priors, skeptical and enthusiastic, are eventually obtained based on those hired as U.S. Government DHS Calibrated Prior Degree Specialists.

Sounds like lunacy to me! (but check the date of this post!)

image-1

High Royds Hospital

 

 

[i]Spiegelhalter, D. J., Abrams, K. R., & Myles, J. P. (2004). Bayesian approaches to clinical trials and health care evaluation. Chichester: Wiley.

[ii]Homepage for DHS and St. Elizabeths Campus Plans.

http://www.stelizabethsdevelopment.com/index.html

GSA Development of St. Elizabeths campus:“Preserving the Legacy, Realizing Potential”

[iii]

U.S. House of Representatives
Committee on Homeland Security
January 2014

Prepared by Majority Staff of the Committee on Homeland Security

http://homeland.house.gov/sites/homeland.house.gov/files/documents/01-10-14-StElizabeths-Report.pdf

[iv] The randomly selected hospitals in one standardized test included the following:

Topeka State Hospital

Danvers State Hopital

Denbigh Asylum

Pilgrim State Hospital

Trans-Allegheny Asylum

High Royds Hospital

Whittingham Hospital

Norwich State Hospital

Essential guide to abandoned insane asylums: -http://www.atlasobscura.com/articles/abandoned-insane-asylums

 

 

Categories: junk science, Statistics, subjective Bayesian elicitation | 11 Comments

Post navigation

11 thoughts on “Skeptical and enthusiastic Bayesian priors for beliefs about insane asylum renovations at Dept of Homeland Security: I’m skeptical and unenthusiastic

  1. you almost got me there…!

  2. Anonymous

    Xi’an: I believe it’s true, I have a friend who does data analysis at DHS, and he didn’t deny knowing about these sensor tests on abandoned asylums.

  3. john byrd

    For a fraction of the budget, I am offering to tell them my priors assigned to the probability of that approach being worth the costs…

    • john byrd (off-topic): I never got back to you on Spanos’s base rate fallacy thing; I have an extended reply half-written, and I should update it in light of your (highly relevant!) presentation and send it off to you…

    • John: You might have made a good candidate for them, but I think all the positions are filled. Still, they expect to need replacements: part of their strategy, they said, was to make sure sensors had “skin in the game”. What I don’t understand is how they can show any of them were “wrong” about what they predict regarding St. Elizabeths. Like if there’s a fire in 10 years, are they going to go back to the “enthusiasts” and get their money back? That’s if it’s even done in 10 years, they’re predicting 2026. Apparently the bulk had been so far paid using the “stimulus”. I guess it was “shovel ready”.

  4. By the way, the answer to the question, “Why did patient farming end at High Royd?” is that as the use of medcations grew, patients were no longer able to til the fields. This is absolutely true, you can look it up.

  5. Readers: so is this an April Fool’s joke or not? The best April fool’s jokes are those that really fool, which means they must have enough believability, until it hits you. It’s worrisome when things like this zany “testing” on other abandoned insane asylums are credible to people….isn’t it? On the other hand, it’s not so very different from some published things I read.Please let me know if you hear of this actually taking place.

    • Anonymous

      Obvious tom-foolery, took me 5 seconds to spot. That’s before it became hilarious!

  6. David Pattison

    “The best April Fool’s jokes….”: Also interesting would be a true episode, presented on April 1, just bizarre enough that it would be assumed by most readers to be a joke. Is this a Type I joke or a Type II joke?

I welcome constructive comments for 14-21 days

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com. The Adventure Journal Theme.

Follow

Get every new post delivered to your Inbox.

Join 460 other followers