In qualitative research, topic choices are sometimes based on personal investment and a desire to solve a known problem rather than a desire to add to the scholarly knowledge of the issue

Considerations for the Qualitative Researcher

Introduction

Tell us about yourself and what drew you to your research topic.

I am Laurie Hinze, and I am an assessment specialist. I had a topic that I was very interested in, in feedback, faculty feedback from my work as an assessment specialist with faculty. I graduated in 2013, and am still involved in ongoing course development and assessment specialist work on faculty feedback.

Controlling for Initial Bias

How did you assess for initial bias as you honed in on the topic and started considering research design?

The conversations that I've had with faculty over time really dealt with the assessments in the course or the assignments and the scoring guides. And in the online setting, there's always a challenge of providing effective feedback for the learners but doing it in an efficient manner. So my initial thought was, well, what scoring guide do we use in order to make that efficient and effective? And as I went through a number of steps of the process I learned there was a lot more to it than that. But that was my interest, you know how can we help faculty, how can we help learners, and how can we help the field really in online, being able to identify what's an effective and efficient tool for providing feedback. And my initial focus was on the scoring guides, but as I go through it a little bit more that moved over a little bit and now it's definitely more of the course design component of it. But that's the topic, faculty feedback.

I came in with ideas that I was sure that's exactly what I was going to do. And I found out that would probably be a lot bigger, a lot more involved than I really needed to be doing, in a beginning component of researching this. So one of the things for checking about biased, because I knew that obviously there was a biased that I was bringing to the table, my thought was well, rubrics are the way to do that. Faculty that I talked to thought checklists are the way to do that. So in talking to my mentor, she was instrumental in saying you really need to start with a lit review. In order to really set the stage is kind of how I call it. So, setting the stage really shed a light on what do I really want to do. But it was very helpful in making sure that the biases weren't there. That I didn't bring in, well this is the way I think it should be or that I was going with you know well, this is what the faculty said. So, starting with the lit review, ongoing conversations with her were very important to monitor that bias, that initial thought of what I wanted to explore.

I guess going back to the lit review, before you even do Chapter One, that was her initial direction. Start with that literature review really became my tool. The lit review that I did, was also a way that I could create an outline that would really help create what research was out there, what was it saying, and how did that really combine with my thoughts and the topic I wanted to explore. So much so that I really had to narrow it down, and that was the other piece, the two things I think that you'll probably hear me mention a lot are the use of the lit review and the outline that I actually created from that, and the mentor conversations that I had. And sometimes it was even beyond that, committee members.

Sometimes it was colleagues in my department. And just questions that or I would share a comment and they would ask questions, it would help me think about, is this really what we should be looking about? Is this really what the problem is? So it helped me really bring it down to a much more narrow focus that I ended up really looking at more the course design. Really what is being delivered by faculty? Not so much what do they think is effective and efficient, but let's start at a baseline of what is really being, how are they going about it now, without even thinking, what's effective and efficient, what are they doing? So that really after many conversations, many reviews of the literature, and creating that outline, that really kind of showed me how things were flowing. That became more of my topic, what's the perception of the faculty from course design components and how they deliver feedback. And that's really what developed into both what I've kind of felt what the problem was.

Still effective and efficient feedback, although I wasn't going to be answering that specifically. I was going to be looking now more at the perception. And then the problem, again of the effectiveness and the efficiency, really looking at the course design features, so that became more of the, not so much the problem, but we didn't know what we'd be looking at. And then I think the last one was the participants that you had asked about, and that would really be where again I looked at the literature review and looked at who were other researchers involving in their literature, I mean in their research. So it was helpful for me to narrow down who I was going to be looking at, who I was going to be working with, again, working with very closely with my mentor.

We ended up doing people that have had experience, so that I was getting somebody that could reflect back a little bit more than just, oh this is how I do it. But two to three years of experience in the online teaching world. And then, at the higher ed level I really wanted to keep it at, at first we started with masters and PhD level, and I did a snowball technique where people recommend each other that they think would be good participants, and that wasn't giving me the number that I needed to reach a saturation point. So that we did go back through the IRB and change that so that I was also doing undergrads. But I don't think that made a big difference at all in the study.

Participant Selection

How did your alignment problem, purpose, and research question(s) affect participant selection?

I did the interview questions; there were about eight or nine of them if I remember correctly. I did keep my study to one research question. And it really didn't change that a lot because I was looking for perception. But the challenges with the, the snowball technique were a little bit of making sure that I still had a sample size like you said that I would know when I got saturation. And maybe this was a benefit too, that I was able to start coding immediately after the interviews. So I was able to start seeing, slow but sure, was I reaching saturation on that. So, it's a matter of it takes a little while to do, but you really can start your coding and start getting a feel for your saturation point. And I reached that about after about 10 or 11 participants.

I was getting the same types of responses and those types of things. Of course I think any type of sampling technique can have its challenges, but this is the one that really fit what I felt I was looking for. It gave me, with the online component of it, it gave me people that were geographically mixed, which was very good, and then as I was talking about the different levels and that I did finally go back into the IRB for the undergraduate level gave me a nice spread too of the different levels. As well as the different disciplines that they taught in, which, as I got into the coding could be challenging, by the way that they may have responded feedback wise, say in a math course, they were much more linear with those types of feedback. As opposed to a different course that the feedback would be more of a critical thought type of requirement that the faculty wanted them to search out. So the subject matter was good, but challenging.

Developing Initial Codes

How did you develop your initial codes? How did they change over the course of your data analysis process? Did you consider the use of qualitative software?

The initial recommendations to start the snowball sampling were from my committee members. They reached out to somebody that they felt would be a good participant based on what I had put as requirements. And you know I think some of it is, it's such an additional thing to do, so you get some people that say, oh yeah I'd be happy to, and then you just don't always hear back from them. So, those people were probably more at the masters and PhD level that I was interviewing, so that was a little bit similar, and as I mentioned I know the math one was definitely an undergrad component, where he talked a little bit more about the specifics of feedback. So it probably in the long run broadened what I was looking for. Because I was doing the coding along the way, I think there was probably still a convergence of thoughts. It's just a matter of maybe expanding my list of codes as opposed to deleting them. Or any kind of divergence of coding.

One of the tools, I'm a big person for using tools and researching things. I don't want to reinvent the wheel always, and I had found a document that really listed the coding and I did broad coding. Then the next one was patterns, axial coding, and the last one really was themes or the selective coding. So, that first component was broad, very broad and that's where I would talk about the convergence. That's where I would go back to the literature review and really see, where does this fit? How can this fit in there? So I was using that list through Excel to really construct that and then using my literature review outline to help with that. But then also really looking at what the participants were saying, I wanted to say the respondents, but what the participants would say, and then I had two things.

One, when looking at what I was going to highlight, did it apply to the study, and could it stand on its own? So I think at times there might have of been more that I might have elected to put as a code, but that really didn't fit that criteria. And that was a comment that I had found as part of how you do the coding in that manner, where you do broad coding, and then what patterns or axial, and selective coding that really needs, that those two criteria pieces were really important to kind of keep in mind. So, not sure if that answered the convergence question, but it really, you go through it over and over and over and I think you just start really getting a feel of what is going to fit for your study. And those two items were really, really helpful for me.

I always want to fill in the gaps. So learning to ask the question, and this is just basic interviewing, and there again I looked at what types of tips for that, and there's tons of them out there. But really letting them respond, otherwise I think I was too eager and I think the biases would have come in there, so I really had to be careful of that. And that was a really conscientious effort to make sure that I did that. So it's hard I think, but it's an important one or otherwise your biases, unknowingly I think, can sway the participants response and I did not want that all.

So, catching myself, and having a document that had the questions, I just made sure that I stuck to that and just had to not say anything for a little bit. And the fact they're recorded, you know that helps so you, you know after listening to the first one if I felt like I walked over that line I really had to take a step back and think okay I can't do that. And by the time you get towards the end you're feeling pretty comfortable about not feeling you're leading or asking. And that actually in the development of the questions too that's really important. There again my mentor, using the lit review, those two items. You know I can't say enough about that helps keep a balance on bias all the way through. It's important to remember that. It's not that you don't know it right off the top of your head, having those conversations is really important.

I did explore them I can't even think of the names anymore; it's been a couple of years. There again, my mentor was like, you don't want to do that, just use Excel, so it's really bringing it down to that baseline level. I enjoyed coding, and I always thought I wanted to a quantitative research study. So to find out that I really liked qualitative, I really like coding, I do coding in my job now. And using it as an Excel document it's, I don't want to say easy because you still have to kind of think through the different levels of it. You know what's the broad categories, what's the patterns, and what's the themes. But I also find that energizing, and that starts answering those questions that we ask. So, yeah, no I used just a basic Excel document and reviewed it over and over and over. So there's probably, there were probably many, many versions of it. But yeah, it's very, I think you have a little, I shouldn't say this cause I haven't done it the other way, but I would like to say that it's easier, you know, for me. It was just easy to control.

Developing Findings

How did you determine when there was sufficient evidence to develop a finding?

I felt some of my hunches were accurate. Yeah I do think, and you say never left, I think I was probably a little bit of that, always checking back in. And that actually inspired me as I developed that outline from the literature review. Oh, this is what I would have been expecting, this is what I'm hoping for, this is where I hope my research goes. And in my situation, that's where it did go. Were there other components of it that I learned and were able to add to that, which you want to do, you want to contribute to the, to the world of online and for me it was course design and how faculty felt that influence, their feedback practices.

But yeah, I felt like I was able to really confirm my initial thoughts, initial hunches. But with it in a much more detailed understanding in manner, and be able to support that as I share that with either faculty training, faculty discussions, and the work that I do with other assessment specialists, those types of things, so. Yeah, it was, I don't think I ever left it but I was very happy that I felt like it matched and enhanced, the findings matched and enhanced what I initially thought and where I wanted to go with it.

I don't think there were as many surprises as differences of the way people do things and approach. But again that's me wondering why. I wonder why they do that? So I can't say that it was a surprise but it was like, oh, that's an interesting way of looking at that. So it's always so easy to think, well this is the way it is and this is how everybody thinks. So in some ways it's a pleasant surprise to really look at that breadth of information. But I'm trying to think if there was anything that really caught me off guard, but I can't think of anything other than you know really especially with my mentor consistently getting me to narrow down, so maybe the surprise would be that I'm always too broad of a thinker and I really needed to focus on building the blocks as opposed to looking at that high level end product right away.

,

Interview With Sharan Merriam Presented by Mike Worthington, Core Faculty

Basic Qualitative Research

Dr. Mike Worthington

Thank you Sharan for participating in this interview. One of the things that I have noticed is that the most common design that you — you have also stated the most common design used in the field of education as basics qualitative design. It is kind of a center piece of the course, so the learners are going to learn a great deal about that.

Well, why do you think design is so common and it is such a good fit for qualitative researchers in education setting?

Dr. Sharan Merriam

Well, I think that it is good for people in lots of settings besides this education because essentially the basic qualitative design is for those kinds of questions we — you are trying to understand, the experience of the other. You are trying to make sense about the people's lives and experiences. And that actually is the basic structure of all qualitative research, so the way I sort of made sense out of this arena is that you know the basic qualitative study is trying to make sense of other people's experiences.

Now, if you add another dimension, another aspect to that that is when you get into these different types of qualitative research. For example, if you want to look at the social and cultural structure, if you want to understand the meaning and experience but within the social cultural context, then you are likely to be doing an ethnographic study. Or if you are interested in the essence and underlying structure of the lived experience, then you are likely to be doing a phenomenal logical qualitative study. Or, if you want to build a theory or build a substantive theory, then you are likely to be doing a grounded theory study.

So, my sense is that all these types of qualitative research, what underlies all qualitative research is the search for understanding, the search for meaning, making sense out of other people's experiences. And it seems to me that usually, graduate students when they first get introduced to qualitative research that is unclear paradigm and to me, it is enough to sort of master a basics qualitative study without having to get into yet another variation of it because all of these other types of qualitative research have their own strategies and their own procedures and the way they do things. So, I think that is why basic— why I see the biggest percent of qualitative studies in education which is in my field also is that it is because that it is enough to master that and actually get to act what qualitative research is designed to do which is to get you understand the meaning of experience for people that you are interested in their lives and their experiences.

Dr. Mike Worthington

That is really a help in it. What you have said is really validated the approach we are taking here because we are building upon that basic design.

Conducting Effective Interviewse

Dr. Mike Worthington

We will be conducting interviews with colleagues at least two interviews and for some of them, they may never have done interviewing before. What advice do you have for this novice qualitative researcher in conducting an effective interview?

Dr. Sharan Merriam

Well, first, maybe do this as part of the course or the assignment. I will have them practice with a friend. I do workshops all over the world basically on qualitative research and in the work — in the section on data collection and particularly interviewing, I have them tear off and the person who is being interviewed just role plays whatever the topic is. If you are interviewing a school administrator or something, that person even bully may not be that. Just pretend that they are so that you as the interviewer can get a chance to try out your questions, just practice and then, I have them reverse role. So, I think it is — first of all, I think practicing in a comfortable environment, not sort of a nonthreatening situation is really good.

And then, I might suggest that they practice with someone who is in the role that they are interested in interviewing, but that is not going to be in the study. That might just be a friend that can give him some feedback and a real, a real possible trial interview with somebody. So, I think the practice is where it is happening. There are couple of other things that are equally important here. One is that the — and this is why it is fun to do like dyads in the class or something, that you should strike — you should like your research questions or have someone else look at them. And any research, any interview question that can be answered yes or no probably will be if somebody answered yes or no. We still did not give you that any information.

When I do the segment in my workshop with stuff, I always sort of demonstrate this. I pick somebody in the group and I ask him. I said, "Now, I want you to answer as few words as possible, few words as possible." And so, I will say, "Do you like this workshop so far?" And then, they will say like, "Yes or no." So, do you think might use qualitative research in your next study? You know what I mean so I demonstrate it. So, I said to someone until I get done with this little mock interview. I said, "So, what do we learn from this interview?" It is virtually nothing, because those questions do not go anywhere. So then, I asked them. "How can we reword these questions such that you will get more information?" So, you can say, "What activity have you liked the best in the workshop so far?" And we just sort of get them saying something besides yes and no.

So, I think trying out the questions to make sure that there is — getting a devil's advocate on the other side to make sure you have got a question that says they are going to get you something beside yes and no. And then, a comfortable site, you know, you just — and that to me means noise and whether it is cold or hot, wherever your interviewee chooses like the office or home or wherever. It should be a place to stay we choose that they will fee comfortable in and that is another thing.

I mean, you have to be a good listener when you are interviewing. The tendency for new interviewers and actually even myself, I do a lot of studies, but always the first couple of interviews that any new study I tend to do this which is I am thinking ahead to the next questions I want to ask. And so, I am not listening really well to the question they are answering. So, that is another thing, you know, trying to really listen to what they are saying because they might be saying something which would then prompt you to ask another question to follow up and get even richer detail.

Here is absolutely the biggest thing for good interviews, a minimum of open-ended questions. If you have 25 questions, you are going to be so busy looking at your list of questions. You would not be hearing them. You will be worried about asking them. You will be thinking, "Oh, which one did I miss?" You would not be listening very well. You might die. My PhD students, I would say to them, "Okay. I do not want any more than 10 questions, 10 questions maximum." And I think in the book, my qualitative research book I gave them an example of somebody. The whole dissertation thing was, I think should be open-ended questions.

And so, I don't let them ask a ton of questions if I have in their interview schedule where you got the questions, but you do have some, but you keep those to a minimum because when you ask the first question if it is an open-ended question, "Tell me about your experience or whatever." Then, whatever they say is going to lead you to your next question. You do not have to be tied to your interview. I found that if this does take a little practice and it does take maybe, I do not know, two or three interviews to free yourself up from the interview schedule, that list of questions. And I found in my own research that by the time I am into the fourth or fifth interviews, I do not even know if I brought it with me because I am familiar enough with the topic. And also, I have already used open-ended questions and which ones were good and which ones do not seems to work so well, so a minimum of open-ended of questions — with no yes and no kind of questions practice at.

And you know, being patient and having the tolerance for some ambiguity here, you will not have the answer right away. So, just kind of keep it open. And then, there is a couple excellent questions that I found work really well to get really good data and I always have my students who whatnot incorporate a couple of questions. And one of those questions is. Tell me about a time when you encountered a severe discipline problem or whatever your topic is. Tell me about a time when or can you give me an example of that or tell me more about that. What you want to do, you want to get instances. You want to get little stories. You want to get descriptions of the behavior or the perspective that you are trying to uncover here.

So, let us say you are doing a study of leadership or something in administration leadership. So, you select somebody who is identified as a real leader of other administrators and you want to find out their leadership style, so you could ask them. "How would you classify your leadership style?" Well, maybe they will say, "Well, I tried to engage the people who I am dealing with." Follow that up immediately with. Can you give me an example of time when you actually employed that strategy? Tell me about a time when and that is when you get the best data, really, really good examples of the phenomenon that you are trying to understand and uncover.

Dr. Mike Worthington

Those are really helpful tips.

Conducting Qualitative Data Analysis

Dr. Mike Worthington

Well, learners in this class are being asked to do, they are being asked to conduct at least two interviews to learn more about their colleague's educational processes and strategies, practices that are working well. They are going to be asked to use the — and we will follow the inductive and compare to data analysis you have outlined in your qualitative research textbook. But what advice do you have for learners in conducting qualitative data analysis?

Dr. Mike Worthington

Well, it is not going to be an issue with doing two interviews, but the thing that I would absolutely, absolutely — it is never been required as if you can is that they do one interview and analyze that before they do the second interview. Because in qualitative research, ideally, you analyze as you go because if you go pull — if you collect all your data — and sometimes as I understand it, sometimes you have to go to a different part of the country or something and you can only get three people in one day or something. I understand that but do the best that you can is to try not to collect all of your data and then start analyzing because you probably would not be able to do it and you also are missing a good opportunity to improve the data you are collecting.

So let us say you do one interview and I will see if it is possible. I would require this of students that they are going to agree with. They do one interview and they type up the transcript and they should go down through that interview first before they do with preliminary data analysis. Two things happen then. One is that they can see where they could have done better in the interviewing. For example, they will say, "Oh, my gosh. I cut the speaker off. I cut the person off." There, I should have let her keep talking or I should have followed up with another question or it looks like I talk too much, if you got a lot of interviewer and not much interviewee.

So, those are the things that you can see when you look at a transcript that I have found even after doing many, many interviews in years of experiences. I have found that it is very hard for me to tell how good an interview is until I look at the transcript. You can come often in interviewing. You can say, "Wow! That was really a good interview. I got lots of information, blah, blah." And then you look at the transcript and say, "Well, they sure talk a lot but there is not much here or vice versa."

That was really harder to get a person. I did not get much at all and then you look at the interview. Well actually, they did say quite a few important things. So, it improves your interviewing by looking at the interviews as you go. Second thing is if you do even just a preliminary analysis of your interview, then you get sort of a sense of what — somebody answers to your research questions that might be. Those are in the back of your mind when you go to interview the next person.

So, you go to interview the next person and maybe that person does not mention something the first person mentioned. Well, you can check that out or you can say something like, "You know, I have interviewed someone else on this topic and they made this interesting point." And what do you think about that?" So it gives you a chance to check out something you heard from the first interview.

And so that's really, really important to analyze as you go because it then and improves your data. It improves your interviewing and then it improves your analysis. So then after you do the second interviewee, you do the same thing. And after you have done two interviews, you have looked at them carefully and you sort of kind of pull together from both interviews what you think some of your tentative analysis is, some of your tentative answers.

And then you go to your third interview and by the time you get done interviewing, you already have tentative findings. And so as you go along in your interviewing at the end or other data collection, observation which works well too. At the end, you are confirming it. But if you have not analyzed as you have gone along, I mean you do not know what is in there, basically. So, that is crucial.

And then, I think the main strategy here in analysis is that you are trying to answer the question you raised. So I always have people say, "Well, what is the answer to your question about how they do something or what their perspective is on something or what is the answer?" So you go down through the interview and you say, "What in here helps to answer my research questions, question or questions, whatever you have?"

That answer is a tentative finding. I mean it is a finding and then become a category or a theme or whatever you want to call these things. But your findings in a qualitative study are the answers to your research questions. So let us say you are looking at, I don't know, whatever, a process of some sort, how do teachers handle difficult discipline problems? And anything that is a how is a process.

So, I would look in my answers, I would look — well, what is the first thing. What is the first thing in this process? What is the first step in this process? What is the first part of this process? Well, let us say the educator has to correctly identify the discipline problem itself. Maybe that is the first step, then what? I mean, then what is the second and what is the third? So, you are looking for answers to your research questions and those answers are your findings or your themes or categories, whatever you want to call them. That is what it is.

And in your data which is the interview or your field notes or documents in the data, an answer can be as short as a word or it can be several pages which is the recounting of an incident of a story that illustrates an answer to whatever question you are answering. So, you go down through your data and you — you know, I have them often write inside the margin. I just have them write — it can be the same words that are in the data or it can be something

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.