Lies, Damned Lies, & Statistics: Mental Health Association-Forsyth blew its money on an moth eaten report funded by the United Way
What a mess this report is. So how much money did we spend on this? United Way, you blew your money. How much public money went into this?
Here is a statement about the presumed results of the MHA-Forsyth report:
"...What do the survey results show?
Some areas in the delivery of publicly-funded mental health services still need attention."
http://www.forsythunitedway.org/pilotFiles/bulletinBoard/files/2009%20MHA%20IN%20FORSYTH'S%20SURVEY%20PRESENTATION%20%20%20%20%20.pdf
Gosh. Really? No you cannot even say that. Indeed, you can't say much of anything for the gaping holes associated with the data collection.
I do not see the entire report online anywhere. If you want a copy of the pdf as forwarded to me by a mental health advocate, come to me back channel and I will send it to you: hammondmv@netzero.com
Save us from the reports. Or at least save us from these endless mental health reports that are of poor quality. If this had been a doctoral dissertation I was marking up, I would have punted it back to you and told you to recollect your data----so many holes are there in terms of unanswered questions. No conclusions can be made on the basis of the data set which is spelled out here:
Moreover, where's this blog so I can put my statements on it?: : 'The MHA has suggested as a result of the data collected through this survey, additional recommendations from the February 23rd presentation audience included creating a blog for fielding comments from the community, "
The biggest need, stated by consumers, was individual therapy, however only 50 of the 220 participants receive it. And if you're a state funded consumer, you get 8 sessions and THAT'S IT for many of the LME's across NC. Specifically, in western NC, associated w/ Smoky Mountain Center LME, state funded consumers get NO individual therapy. As associated with Western Highlands Network LME, you get 8 sessions and then you have to harass Marsha Ring, the Utilization Review manager to get anymore (which you won't as she believes that most people don't want more than this-----something she stated to me on a phone call about a year ago).
The study implies that it is a longitudinal study of mental health providers and consumers associated with the Winston Salem area of NC. Since the 2006 data set is not associated w/ the 2008 data set information, there is no way to verify that this is true or false.
There are lots of problems, in terms of clarity of the data and their presumed findings based on data problems as per the long-awaited , "ONGOING IMPACT OF NC MENTAL HEALTH REFORM ON CONSUMERS AND STAFF/PROVIDERS IN FORSYTH COUNTY conducted by The Mental Health Association in Forsyth County, Inc. "
The Mental Health Association of Forsyth County (MHA-Forsyth county) appears to want to declare that changes re: mental health care has 'stabilized' or at least not gotten worse but their statements cannot be corroborated as per their data which is listed. This is what MHA-Forsyth stated as a conclusion (well, they have encrypted it so you cannot cut and paste: how clever and how indicative it is that they do not want people to utilize their document):
"...the 2008 survey indicates that there have been very minimal improvements for consumers and staff...."
I don't think they can state that AT ALL. I think they have a flawed report and these are the reasons why I think that:
1. the 2008 MHA-Forsyth is being compared to 2006 MHA-Forsyth. Where is the 2006 data? The n or number of consumers =220. What was the number of 2006 participants as associated w/ comparing? Were the consumers more or less matched in terms of being 'similar' as associated with demographic and any other salient characteristics that could have been speculated to skew the data----or not?
2. On p. 19 of the report, there are 50 consumer participants that are not identified as being from any agency. Where did they find them?
3. 3/4 of the consumers surveyed had been receiving services for less than a year. So, were the participants/ subjects matched in terms of demographic and other salient data in order to compare them? (again, where is the 2006 data).
4. This below table is most problematic because the numbers do not add up in terms of If n= 220, how can you have 74 uninsured and 207 w/ Medicaid? You either have one or the other.
5. Its not clear how many providers were surveyed. The n or number of participants, associated with the various questions ranged from a few dozen to 135. And, how many providers were surveyed in 2006 and where are those survey questions? They're not in the appendix.
6. There were only 5 provider companies surveyed. How did the five separate companies get CHOSEN as associated with Centerpoint divesting its clinical arm?: "HopeRidge provided clinical services from July 1, 2004 until August 31, 2005 when it went out of business and most of its services were divested to five separate private companies. " Who chose the companies? Are they same ones that were surveyed on 2006? How many of the same providers responded? For what reason? What were the politics associated with that? Why only five companies?
7. pretty darn important matter not to have collected data on: "Although we did not collect data on the refusal rate, we estimate that the refusal rate was approximately 25% across agencies." Which agencies? and which agencies participated or can't you say as re: anonymity or was the anonymity at the level of INDIVIDUALS within the agencies or as per the agencies themselves? Again, an important point.
8. The MHA- Forsyth report stated: "In addition to the questions adapted from the state survey, we added demographic questions and some open-ended questions that allowed staff and consumers to provide additional information about their experiences and opinions. " What did you do w/ the additional demographic data gathered, apparently, only in 2008? What did you do w/ the open ended questions?
9. The MHA-Forsyth report stated: "Quantitative data were analyzed using standard descriptive statistical methods with Excel 2003. To analyze qualitative data from open-ended survey questions, we developed a standardized, explicit coding system using content analysis. " So, you utilized a mixed methods analysis. EXCEL is simply an organizing tool. And so how did you compare 2006 to 2008?
10. This is very problematic, this statement, to my mind as you have mixed Basic Level Services with Enhanced Benefits services: "In regards to service needs, case management, community support, and individual therapy/counseling were all services that approximately 20% of consumers surveyed stated they needed, but did not currently receive. " So, your statement indicates that case management is grouped, in terms of NC mental health reform and Service Definitions, with Community Support Services. These are termed ' Enhanced Services' and there is very very little of this available to state funded/ IPRS consumers and less and less available to Medicaid or Medicaid/ Medicare dual eligibility consumers. Basic level services, which is essentially individual therapy, for state funded clients is available in many of the LME's for 8 sessions. That's IT unless the provider wants to go to a great deal of trouble to obtain more. Medicaid consumers gets as much individual therapy as they want, for the most part. And so you have mixed services that have a huge range in terms of availability.
Therefore, I do not know what use this statement serves: "20% of consumers surveyed stated they needed, but did not currently receive.' Wre these Medicaid clients? State funded/ IPRS clients? what?
11. You stated: "Staff/provider data overall indicated that overall satisfaction rates with service provision and availability for consumers remained quite low and had dropped a bit when compared to the data collected from the MHA 2006 survey ..." Was this a statisically SIGNIFICANT difference? 'dropped a little' has no meaning when comparing two different samples that may or may not be matched in terms of demographic or other characteristics which matter.
12. This statement is attempting to link apples and oranges: "And although 65.9% of staff were in agreement with the statement “I give clients a choice of providers” only 28.9% agreed with the statement “I have been pleased with the consistency of the providers I have seen or referred clients to”. " What does 'choice' have to do w/ 'consistency of providers'?
13. The MHA-Forsyth report stated: "In regards to overall systems changes, we saw a very small rise in staff satisfaction when compared to our 2006 survey (please see MHA 2006 survey report for full details, the MHA survey PowerPoint presentation also has some comparisons listed). " Was this a STATISTICALLY SIGNIFICANT DIFFERENCE? 'small rise' has no meaning. You have to know the n of the 2006 data and you have to know if the participants/ subjects were matched across the variables that might be inferred to affect that matching.
14. How can you compare Virginia when you have given us no information as to the comparability of that sample to the ones you obtained?":The comparisons to the satisfaction scores from Virginia were made to give us all a view of a comparable system and to give us a benchmark for a comparable state. "
15. Let's all have a big hurrah for this---as it really does not matter except as associated w/ 'perception' : "Although the presentation presented findings that showed little improvement in regards to satisfaction scores in our community, it was stated, on a very positive note, that “the release of the survey results didn't come with the usual finger-pointing that has slowed progress on mental-health delivery” in our area. "
16. Consumers numbered 220. As regards a question attempting to solicit whether consumers believes there were improvements or not in terms of mental health care, 40 of those 220 did not answer the question. Therefore you cannot say anything because you did not collect more complete data. And there is also no information about the 2006 data set.
Here is a statement about the presumed results of the MHA-Forsyth report:
"...What do the survey results show?
Some areas in the delivery of publicly-funded mental health services still need attention."
http://www.forsythunitedway.org/pilotFiles/bulletinBoard/files/2009%20MHA%20IN%20FORSYTH'S%20SURVEY%20PRESENTATION%20%20%20%20%20.pdf
Gosh. Really? No you cannot even say that. Indeed, you can't say much of anything for the gaping holes associated with the data collection.
I do not see the entire report online anywhere. If you want a copy of the pdf as forwarded to me by a mental health advocate, come to me back channel and I will send it to you: hammondmv@netzero.com
Save us from the reports. Or at least save us from these endless mental health reports that are of poor quality. If this had been a doctoral dissertation I was marking up, I would have punted it back to you and told you to recollect your data----so many holes are there in terms of unanswered questions. No conclusions can be made on the basis of the data set which is spelled out here:
Moreover, where's this blog so I can put my statements on it?: : 'The MHA has suggested as a result of the data collected through this survey, additional recommendations from the February 23rd presentation audience included creating a blog for fielding comments from the community, "
The biggest need, stated by consumers, was individual therapy, however only 50 of the 220 participants receive it. And if you're a state funded consumer, you get 8 sessions and THAT'S IT for many of the LME's across NC. Specifically, in western NC, associated w/ Smoky Mountain Center LME, state funded consumers get NO individual therapy. As associated with Western Highlands Network LME, you get 8 sessions and then you have to harass Marsha Ring, the Utilization Review manager to get anymore (which you won't as she believes that most people don't want more than this-----something she stated to me on a phone call about a year ago).
The study implies that it is a longitudinal study of mental health providers and consumers associated with the Winston Salem area of NC. Since the 2006 data set is not associated w/ the 2008 data set information, there is no way to verify that this is true or false.
There are lots of problems, in terms of clarity of the data and their presumed findings based on data problems as per the long-awaited , "ONGOING IMPACT OF NC MENTAL HEALTH REFORM ON CONSUMERS AND STAFF/PROVIDERS IN FORSYTH COUNTY conducted by The Mental Health Association in Forsyth County, Inc. "
The Mental Health Association of Forsyth County (MHA-Forsyth county) appears to want to declare that changes re: mental health care has 'stabilized' or at least not gotten worse but their statements cannot be corroborated as per their data which is listed. This is what MHA-Forsyth stated as a conclusion (well, they have encrypted it so you cannot cut and paste: how clever and how indicative it is that they do not want people to utilize their document):
"...the 2008 survey indicates that there have been very minimal improvements for consumers and staff...."
I don't think they can state that AT ALL. I think they have a flawed report and these are the reasons why I think that:
1. the 2008 MHA-Forsyth is being compared to 2006 MHA-Forsyth. Where is the 2006 data? The n or number of consumers =220. What was the number of 2006 participants as associated w/ comparing? Were the consumers more or less matched in terms of being 'similar' as associated with demographic and any other salient characteristics that could have been speculated to skew the data----or not?
2. On p. 19 of the report, there are 50 consumer participants that are not identified as being from any agency. Where did they find them?
3. 3/4 of the consumers surveyed had been receiving services for less than a year. So, were the participants/ subjects matched in terms of demographic and other salient data in order to compare them? (again, where is the 2006 data).
4. This below table is most problematic because the numbers do not add up in terms of If n= 220, how can you have 74 uninsured and 207 w/ Medicaid? You either have one or the other.
5. Its not clear how many providers were surveyed. The n or number of participants, associated with the various questions ranged from a few dozen to 135. And, how many providers were surveyed in 2006 and where are those survey questions? They're not in the appendix.
6. There were only 5 provider companies surveyed. How did the five separate companies get CHOSEN as associated with Centerpoint divesting its clinical arm?: "HopeRidge provided clinical services from July 1, 2004 until August 31, 2005 when it went out of business and most of its services were divested to five separate private companies. " Who chose the companies? Are they same ones that were surveyed on 2006? How many of the same providers responded? For what reason? What were the politics associated with that? Why only five companies?
7. pretty darn important matter not to have collected data on: "Although we did not collect data on the refusal rate, we estimate that the refusal rate was approximately 25% across agencies." Which agencies? and which agencies participated or can't you say as re: anonymity or was the anonymity at the level of INDIVIDUALS within the agencies or as per the agencies themselves? Again, an important point.
8. The MHA- Forsyth report stated: "In addition to the questions adapted from the state survey, we added demographic questions and some open-ended questions that allowed staff and consumers to provide additional information about their experiences and opinions. " What did you do w/ the additional demographic data gathered, apparently, only in 2008? What did you do w/ the open ended questions?
9. The MHA-Forsyth report stated: "Quantitative data were analyzed using standard descriptive statistical methods with Excel 2003. To analyze qualitative data from open-ended survey questions, we developed a standardized, explicit coding system using content analysis. " So, you utilized a mixed methods analysis. EXCEL is simply an organizing tool. And so how did you compare 2006 to 2008?
10. This is very problematic, this statement, to my mind as you have mixed Basic Level Services with Enhanced Benefits services: "In regards to service needs, case management, community support, and individual therapy/counseling were all services that approximately 20% of consumers surveyed stated they needed, but did not currently receive. " So, your statement indicates that case management is grouped, in terms of NC mental health reform and Service Definitions, with Community Support Services. These are termed ' Enhanced Services' and there is very very little of this available to state funded/ IPRS consumers and less and less available to Medicaid or Medicaid/ Medicare dual eligibility consumers. Basic level services, which is essentially individual therapy, for state funded clients is available in many of the LME's for 8 sessions. That's IT unless the provider wants to go to a great deal of trouble to obtain more. Medicaid consumers gets as much individual therapy as they want, for the most part. And so you have mixed services that have a huge range in terms of availability.
Therefore, I do not know what use this statement serves: "20% of consumers surveyed stated they needed, but did not currently receive.' Wre these Medicaid clients? State funded/ IPRS clients? what?
11. You stated: "Staff/provider data overall indicated that overall satisfaction rates with service provision and availability for consumers remained quite low and had dropped a bit when compared to the data collected from the MHA 2006 survey ..." Was this a statisically SIGNIFICANT difference? 'dropped a little' has no meaning when comparing two different samples that may or may not be matched in terms of demographic or other characteristics which matter.
12. This statement is attempting to link apples and oranges: "And although 65.9% of staff were in agreement with the statement “I give clients a choice of providers” only 28.9% agreed with the statement “I have been pleased with the consistency of the providers I have seen or referred clients to”. " What does 'choice' have to do w/ 'consistency of providers'?
13. The MHA-Forsyth report stated: "In regards to overall systems changes, we saw a very small rise in staff satisfaction when compared to our 2006 survey (please see MHA 2006 survey report for full details, the MHA survey PowerPoint presentation also has some comparisons listed). " Was this a STATISTICALLY SIGNIFICANT DIFFERENCE? 'small rise' has no meaning. You have to know the n of the 2006 data and you have to know if the participants/ subjects were matched across the variables that might be inferred to affect that matching.
14. How can you compare Virginia when you have given us no information as to the comparability of that sample to the ones you obtained?":The comparisons to the satisfaction scores from Virginia were made to give us all a view of a comparable system and to give us a benchmark for a comparable state. "
15. Let's all have a big hurrah for this---as it really does not matter except as associated w/ 'perception' : "Although the presentation presented findings that showed little improvement in regards to satisfaction scores in our community, it was stated, on a very positive note, that “the release of the survey results didn't come with the usual finger-pointing that has slowed progress on mental-health delivery” in our area. "
16. Consumers numbered 220. As regards a question attempting to solicit whether consumers believes there were improvements or not in terms of mental health care, 40 of those 220 did not answer the question. Therefore you cannot say anything because you did not collect more complete data. And there is also no information about the 2006 data set.
posted by Marsha V. Hammond, PhD at 12:29 AM
0 Comments:
Post a Comment
<< Home