Validation and Evaluation

"Is the account valid, and by whose standards?" 

Creswell opens Chapter 10 with this question. In qualitative research there are multiple perspectives, voices, and accounts that figure into the final product, how then is the product judged as acceptable?

Creswell summarizes multiple authors' work in Chapter 10 and discusses various approaches to validation, reliability, and evaluation.

Of course one must be cautious so as not to evaluate qualitative research with a quantitative lens.

Several times Creswell notes that thick description is necessary for all of the approaches.

I particularly appreciated, possibly due to my strength of "strategic," the citation by Creswell on page 204 where Eisner (1991) stated "The researcher compiles bits and pieces of evidence to formulate a "compelling whole." At this stage, the researcher looks for recurring behaviors or actions and considers disconfirming evidence and contrary interpretations.

I believe one of my strengths is the ability to see connections between separate works. Of course in qualitative work this can be both a blessing and a curse!  These seeming connections would be the exact fodder for this topic!

Thus there would be a high need for me to use consensual validation so that I might solicit the opinion of others to assure I am not making up things!

Lather (1991) presented four types of validation:  triangulation, construct validation, face validation, and catalytic validation. In 1993, Lather presented four additional frames  (related to feminist research): ironic, paralogic, rhizomtic, and situated validation (p. 205).

On page 206 Creswell writes "a synthesis of validation perspectives" and this made me wonder if this is what a "meta-analysis" is?

Creswell presents eight validation strategies:

1. Prolonged engagement in the field
   
2. Triangulation
   
3. Peer review
   
4. Refining hypotheses until all cases fit
   
5. Clarifying researcher bias from the outset
   
6. Member checking (the most critical technique for establishing credibility-from the famous Lincoln & Guba)
   
7. Rich, thick descriptions
   
8. External Audits
   

Creswell recommends engaging in at least two of these strategies in any study.

Evaluation Criteria:

Howe and Eisenhardt suggest five standards to be applied to all research:

1. Assess a study in terms of whether the research questions drive the data collection and analysis rather than the reverse
   
2. Examine the extent to which the data collection and analysis techniques are competently applied in a technical sense
   
3. Are the researchers' assumptions made explicit
   
4. Does the study have overall warrant 
   
5. Is there value for informing and improving practice and in protecting confidentiality
   

Overall I am surprised at the thinness of this chapter. After having spent 200 pages in the teaching of how to gather data, how to design a study and so forth, I am surprised that only 20 pages are dedicated to validity. I surmise that Creswell believes if the researcher spends considerable time in the data from the first 200 pages, the study will be well done enough that validating and evaluating is a more smooth process!

Methodological Memo? Analytical? Or Just Plain Arguing With Data?

Recently I read Redesigning for Collaboration in Higher Education by Kezar (2009). Kezar used the case study methodology to consider collaboration in higher education and writes "case study methodology was chosen in order to explore exemplary institutions that had developed an organizational context to support collaboration. This methodology is often used when a unique phenomenon can be identified and examples investigated in detail to describe and articulate the issues.)

Kezar's reason for selecting case study made me give more thought to my own study and my reason for selecting the phenomenological approach.  Creswell (2007) writes "a phenomenological study describes the meaning for several individuals of their lived experiences of a concept or a phenomenon" (p. 57). Creswell suggest that a phenomenological study's goal is to describe the very essentiality or soul of an experiences.

So I ask myself:  what's the difference?  Isn't case study the description of several individuals' experience?  

I returned to Creswell for guidance and he didn't let me down. Creswell explains that a case study is actually "detailed, in-depth data collection involving multiple sources of information (e.g., observations, interviews, audiovisual material, and documents and reports), and reports a case description" (p. 73).  This helped me grasp the difference between the two approaches. In my study, I'm using one source of information:  interviews. And it seems that I will attempt to drill down into one thing, the experience of these individuals which is a strengths-based philosophy. My goal is to feel what they felt in short so that the "phenomenon" can be described in the writing.

In Kezar's work she explored collaboration and used the multiple sources for case study but there is a feeling of broadness to learning about collaboration. My study seems to be more specific and less bounded. Kezar used multiple universities who excel at a collaborative style but she could have just focused on one university. I am focusing on the experience of multiple folks and they do not have to have been in a same setting. Creswell's example is that all may have experienced grief. Grief is the shared phenomenon similar to my strengths-based philosophy.

Argument completed-Elizabeth wins!

Threaded Discussions

I LOVE the threaded discussions!!!  I am so into "seeing" my peers' perspective and having my thoughts pushed into new areas!  My peers are so smart and I'm so lucky to have them surround me!

The BEST Way to Do Observation

God's Love

Isaiah 40:31

but those who hope in the Lord will renew their strength

they will soar on wings like eagles;

they will run and not grow weary,

they will walk and not be faint.

When I am the most worn down, the most depleted, it is always when I have failed to keep my closeness to God. It is always me who wonders off, like a little kid who is supposed to stay near but things capture my attention like a floating butterfly and I follow it, not realizing that I've gotten out of sight. I look up, scared, confused, and realize I'm alone and lost. Luckily I'm finally learning that all I have to do is call out and my Savior is right there and takes my hand.

As I have ventured into the fitness world in training for a 5k, I am learning how to push myself-going from walking to jogging then back to walking to catch my breath. To think about running and to not grow weary-what amazing energy God provides-sustaining even when demanding so much as forcing the body to run. There are times when I've not eaten for the day and after working all day, I head out for a walk/jog.  I have termed this wogging. On the days when I've not eaten anything then I do a warm up that takes about 12 minutes and it's uphill, then I try to jog. I can feel my body trying hard but there isn't any energy to pull. It's like trying to make the car go without any fuel.  I push but all I can do is stop. I am thankful my Lord never runs out of energy. I am blessed that regardless of my own choices-did I fuel up? did I make the best choices? the Lord will give me energy to complete His will.

Even as I stumble and am my own biggest obstacle, the Lord cherishes me and will provide me energy and I will soar with wings.

Daylight Savings Time

I HATE changing the clocks. It takes me forever to adjust and I think William Willett (the guy who suggested it in the first place) was a lunatic. 

What is wrong with consistency?

Writing a Qualitative Study

One of the structural techniques I really like is to instruct the reader on qualitative research methods or techniques while also presenting data. In previously discussing My Freshman Year, I noted how author, Rebekah Nathan,  educated the reader both on the data/findings and on qualitative research. Creswell writes "Another structural device is to "educate" the reader through a discussion about phenomenology and its philosophical assumptions" (p. 189).  

This is a skill I hope to develop well. I have thoroughly enjoyed reading information presented in this manner so it's certainly something to which I aspire!

At some point (when I have free gray matter) I'm going to think about qualitative research and the collision with "reality" TV (which is scripted and edited as we all know) and how it is somewhat of a "National Enquirer" of qualitative research yet there may be potential for both using qualitative research in a more broad appeal to heighten the average person's awareness to important topics.

Rhetorical Issues-Chapter 9

Creswell discusses rhetorical issues as he begins Chapter 9 and I believe one of his best images is "to narrow and expand the focus, evoking the metaphor of a camera lens that pans out, zooms in, then zooms out again" (p. 178). 

In reading some of the qualitative works we are enjoying this semester, I had noted the theme of the narrow, broad, narrow again but hadn't realized it was an actual technique.

I like the discussion of how today's qualitative research admits the impact of the work on the researcher, the participants, and the reader. I have found the qualitative process to be an iterative process and like the ability to document the thoughts, changes in beliefs, or how discovery has impacted self and others.  I believe it marks the difference between research and fiction.

Creswell's discussion regarding the audience is important but I also believe the researcher has to have a broad appeal and recognize that there is likely a blurring of the lines between audiences. Granted writing for a group of college professors versus a report for a government agency is going to be different, however when writing a qualitative study, there is a likely cross-appeal.

Qualitative Ruminations

Last semester Dr. Longman suggested I read the book Eats, Shoots & Leaves by Lynne Truss since I seem to suck at grammar so badly. While on the cruise with Joshlyn this week (Saturday - Thursday) I finished reading the following books:

• Rethinking the L word in Higher Education by Kezar
  
• My Freshman Year by Rebekah Nathan
  

Caught up and am ahead in reading

• Qualitative Inquiry & Research Design; Choosing Among Five Approaches by John Creswell
  

Thus I felt I could handle, briefly, beginning to read Eats, Shoots & Leaves. I'm about one-third of the way into the book and am delighted to not only be learning more about grammar but Ms. Truss's (I just learned that I can properly do the s's and it be correct) work is qualitative.  Throughout her book Ms. Truss provides qualitative data from her research and does so in a somewhat whimsical way that make the book a light and engaging read.

On page 48 Ms. Truss begins detailing some of the qualitative data regarding grammar. She writes, "Hundreds of emails and letters arrived, all of them testifying to the astonishing power of recall we sticklers have when things have annoyed us ("It was 1987, I'll never forget, and it said "CREAM TEA'S"); and also to the justifiable despair of the well educated in a dismally illiterate world. Reading the letters, I was alternately thrilled that so many people had bothered to write and sunk low by such overwhelming evidence of Britain's stupidity and indifference. The vast majority of the letters concerned misplaced apostrophes, of course, in potato's and lemon's.

As I read now, I am enjoying being able to identify the types of data and to consider how I am responding to the data. I am beginning to consider things such as "what type of approach would this be, what type of questions would the researcher ask, what would be the reporting format for this?"

My Hero Kezar Goes EVERYWHERE With Me

Here I am sitting on the BEAUTIFUL Mexican beach with the clear blue waters calling to me.

But no way-Kezar is calling. That's right-from Galveston, Texas to Cozumel, Mexico, to Progreso, Mexico, then back to Galveston, everywhere I went, Kezar went!

Kezar + Wallace = BFFI

The Claims of Community-Part II

I have been thinking further about the claims community makes upon a person. In the New Testament Christians read Paul's words to indicate that we should live in community, sharing our giftedness with one another as we are each uniquely created and have some wonderful gift to offer the whole.

I believe and often say to others:

With great freedom, comes great responsibility.

I believe the more freedom you have, the more "claim" there is upon you, to give of self in order to share your gifts. 

I also believe and often say to others:

It is only your choice for so long...

This statement started out as a employee/employer type of conversation but I realized it had more broad reading impact. I often allow students to "choose" what a response will be for certain actions-however they sometimes struggle with that and I give them a time limit in order to create the real world complexity. Then I tell them-it is only your choice for so long, then your lose the privledge of making the choice and it becomes my choice and you have to simply live with it.

I've carried that over (although not that successfully) to parenting as well.

Ultimately I believe what I'm doing is pointing out the construct of chaos theory-regardless-enduring values will surface and make themselves known.

A Mexcian Student Agreed

At the bottom of page 70, Nathan writes "A Mexican student agreed: I've met people who are interested in me, but for a lot of other people it's ...'whatever'! My [car] mechanic is more interested in my life and my background than other students."

I wonder if Nathan actually asked the Mexican student if he/she agreed/disagreed with the statement about whether or not American students "cared" for him/her or if the coding allowed Nathan to clearly see the "agreement?"

The Claims of Community

Nathan's statement on page 47-

They genuinely want to have a close community while at the same time they resist the claims that community makes on their schedule and resources in the name of individualism, spontaneity, freedom, and choice.

so easily that statement could read:

I genuinely want to have a close community while at the same time I resist the claims that community makes on my schedule and resources in the name of individuality, spontaneity, freedom, and choice.

I believe Nathan is gently brushing the top off an ant pile here!  There is so much that can be taken from this statement. Is it really so unusual that this generation just woke up and became like individualistic?  I don't think so really. As I note above, I could personalize the statement as an introvert who loves people, loves talking, laughing, and interacting, but I have no dimmer switch-when I'm done I am DONE. I am very black and white in my allowances of self to be claimed. 

However, when I inspect this notion more closely, the arms length I keep is not always there and is really only physical. I allow the claims of community to impact my mental thoughts, I think about things others need me to think, or pray, or contemplate. I offer advice, laughter, and feedback. 

Maybe like Nathan noted, when students make a choice and do not attend an event, they are labeled apathetic, we all have claims on our lives from community but based upon our own unique make-up, we respond to community in different ways.

Maybe the person walking out of step with the rest of the group is walking to the beat only he hears. And that's ok.