Now that I'm really working on the Final Project.....

GO TO THE ON-LINE COURSE RESERVES FOR THIS CLASS AND LOOK AT THE 6 SAMPLE RESEARCH PROJECT NIGEM HAS POSTED!!!!!!!!!!!!!!!!!!

Okay, don't make me say I told you so later. I also want to say thank you to those of you who encouraged me to go and look at those myself. I am so not even worried about this thing anymore. I'm serious!

The first four samples are alright. Kind of good for getting a better feel for this thing. They're all about 13-15 pages long and the Code Books in a couple of them are beautiful. The good thing about these four is that you can really see what mistakes others have made so you don't end up making them too.

I have been paying special attention to Sample Paper 5 (the check plus on the first page caught my attention - especially since Sample Paper 6 had a check minus on it). I have found this one to be so very helpful. The Codebook.... just gorgeous! (Sample Paper 6 reads more like a "what not to do" example.)

And I'll tell you what... I don't give a damn if this is supposed to be written in traditional paper style, I'm formatting mine like Sample 5. It just seems so much more clear to me and there's no way he can miss the fact that you have done EVERYTHING! But... that's just me.

My point is... if you have not already... go and look at the Sample Papers in the Course Reserves. You will be thankful that you did!

Article Review

Are we supposed to turn in the article along with our review?

How elaborate is this thing supposed to be? Is it like a paper or do we just answer his questions going number by number?

I just wondered what anyone else's impression was of how this is supposed to be.

Why have we never talked about this in class?

Research Project

Does he want a "faux paper" or does he want us to prepare an informative/explanatory/descriptive outline based on the outline-guideline he provided us with early on?

And Now on to Bigger and Better Things...

Now, I can procrastinate with the best of them, but if I can get (some)thing(s) done reasonably early or on time, I much prefer to go that route as I find it to be much less stressful in the long run.

I consider this post-exam time to be a bit of a breather period in Nigem's course and therefore a prime opportunity to address the Research Project and Article Review and their rapidly upcoming due dates of April 27, 2005. I (admittedly) have been procrastinating on this, but NO MORE!! (at least for tonight) Truth be told, I'm not even sure what we're supposed to be doing exactly.

I have this "Suggested Outline for a Research Prospectus" handout from way-back-when. Is this what he wants? (note that there are two sides to this handout - the second side being the center of my concern)

Soooooooo.....

Has anyone done anything beyond the original "proposal" and perhaps the actual creation of their survey? If so, please share. I think we can all be a great resource for one another since (as usual) the information exchanged in the classroom is in need of some discussion.

I'm going to get started on this thing (which, in actuality, if I had a better understanding of what it is I am supposed to be doing, I believe I could do in an afternoon's time) and wing it up and away until I hear from y'all.

Also, there is the Article Review. Do we just pick a random article or what?

One final note...
On the syllabus, in the Grading/Weights section it reads as follows:
Your final grade will be based (evaluated) on examinations, assignments, participation, and attendance. Weights are allocated as follows:
Four Examinations 75%
Two Quizzes* 5%
Research Project 20%
Article Review 5%

Do I need to retake some basic math course again, because according to my calculations, that's 105%.

Granted, below the percentage area, in reference to the Two Quizzes, it is stated:
[If the quizzes are not administered, their 5% allocation will be added to the examinations.]

It says ADDED! What is going on? Are we working on a crazy 105% scale here? I know that he will probably just drop the 5% from the non-existent quizzes, so don't think I'm freakin' out over here or anything, but the point is.... Hello? WTF? Don't get me wrong, I love the man! I just find this "mistake" (if you'd call it that) oddly out of character. I'm curious and amused, that's all.

So, let's get the ball rolling on these April 27 things now, while we still have a bit-o-time, and figure it out together as we move along.

La-Dee-Da,
Hope

Three Words (dashed into one) on the Multiple Choice Exam...

Piece-O-Cake!

Thanks for the pre-study session, Crystal. I am ready to do a happy dance right here in the Student Union South Lounge!!!

-hope

Ch9 Notes and Qs

Notes on my Homework from Nigem discussion of answers:

Secondary analysis' biggest problem is VALIDITY. The biggest benifit besides cost is in scale testing.

Types of Interview:
Structured - don't deviate from Q on survey
Informal - ??
Analytical - go into more depth, also look around and collect own data about person


Had to kick myself on #9 - I didn't finish reading the Q and answered A when it should be that (D) hand delivery/pick up is the best way to improve self-administered survey response rates.

#10 I have a Q on return rate graphs -- I thought they were useful for BOTH examining history effects AND estimating nonresponse bias...? I vaguely remember that this got discussed in class but didn't make notes about what Nigem said. Anyone remember why they AREN'T useful for history effects?

Note to self (#12): Interviews do NOT offer increased reliability over questionares. Or - Interviews and Questionaries have EQUAL reliability.


My notes on ch:
CAPI (Personal Interviewing): face-to-face interview where interview uses computer
CASI (Self Interviewing): 'face-to-face' interview where respondant uses computer
CSAQ (Self-Administered Questionaire): dumb acronyms - person obtains software that surveys them and they return data. (Online surveys Could, I think, be this?)
CATI (Telephone Interviewing): telemarketers and modern telephone surveys; computer dials and provides script for interviewer allowing instant data analysis
TDE (Touchtone): keep reading this as touchStone?!! - automated hell
VR (Voice): automated hell with spoken responses

Data Archives: collection of survey data available for secondary analysis. Ex is General Social Survey (GSS)
Response Rate: 50% adequate, 60% good, 70% V good
Questionnaire: "Survey used to elicit information"
Bias: Any property of Q that encourages as certian answer (wording is critical)

Clarification on Ch8 -- also Solomon vs classic vs post-only control group designs

(Edited to add Hope's clarifications)

One-shot v. Static Group
- One Shot has ONE GROUP with stimulus and a post test
- Static Group Comparison has TWO GROUPS, one with the stimulus and both with post tests.
- They're both lousy. Neither has pretests.

The question came about because of Q9 on the homework about TV on emotional health in public vs Montessori school kids. Besides being lousy experimental design (what about emotional health as a result of going to different schools or one being public and one private and the economic status of families as a result of that?!!) I noticed that there's a lack of pretest so I'm assuming it's static-group comparison. However, it could also (I *think*) be a one-shot case study (why isn't it?). Because it's two groups.
---

I have some issues with 'natural experiments'.

In the example described on p233 about three-mi-island WHY is their 'natural experement' NOT classified as survey research. They say it's 'quasi-experimental and after-the-fact' and they use interviews and surveys to collect their data.

Is it because of subject selection differences? Help?!
---

Additionally, from the homework: Q15: Natural experiments are most likely to resemble which design?
a) static group comparison (YES?) (Correct, TWO GROUPS are compared, one with stimulus, one w/o, no pretests, both posttested).
b) classical (NO)
c) Solomon (NO)
d) 1group pre-post (No)
e) post-only control group (No, because of lack of randomization of subjects?)
---

Solomon 4-Group:
1: P, S, Po
2: P, _, Po
3. _, S, Po
4. _, _, Po

All have post-tests. Groups 1 and 2 have pretesting and are (exactly) classical design. Groups 3 and 4 lack pretests and are exactly Post-Test-Only Control group design.

Solomon allows comparison between control and experemental groups both with and without pretesting.

So: Classic + PoTCG --> Solomon
---

Why does classical theory suck? (Or, what's it bad at?). Can't control for interactions between pretest and stimulus (sensitization taking place).

Also, extra (unnecissary) expense of pretesting (and addl source of error) when can (theoretically) obtain same results with PostTest Only Control Group design because you've randomized assignment of subjects to control and experemental groups.

Here's What I Remember...

Here are the questions from our Study Guides that (I think) I remember seeing on the MC Exam Preview. I'm sure I don't have all of them (and I may have a few that I think I saw, but really weren't there) but here's a start:

Chapter 8

5. An experimenter wanted to see the effects of caffeine intake on the arousal state of her subjects. She randomly assigned subjects to experimental and control groups and administered caffeine-rich soda to one group of subjects and caffeine-free soda to the other group; she then compared the arousal states of the members of both groups. This is an example of a
a. posttest-only control group
b. pretest-posttest design
c. double-blind experiment
d. classical design
e. static-group comparison

8. Matching refers to
a. linking subjects in the pretest group with those in the posttest group
b. selecting pairs of subjects who are included and not included in an experiment
c. selecting similar pairs of subjects and assigning each member randomly to the experimental and control groups
d. linking pairs of subjects on the independent variable with those on the dependent variable
e. assigning similar pairs of subjects to different settings for the same experiment

11. Elmer, a subject in Professor Jencken's experiment testing the effects of certain films on a person's emotional state, has just undergone a break-up with his girlfriend. He continues with the experiment, however. Which one of the following threats to internal validity is reflected in this example?
a. history
b. maturation
c. selection biases
d. statistical regression
e. experimental mortality

14. What is the basic difference between the classical design and the Solomon four-group design?
a. There is no difference.
b. More time elapses between the stimulus and the second observation in the Solomon four-group design.
c. The Solomon four-group design has randomization.
d. The Solomon four-group design repeats the classical design but adds groups that are not pretested.
e. The Solomon four-group design repeats the classical design but adds groups that are not posttested.


Chapter 9

7. Professor Kaled wishes to ask three additional questions only of those respondents who have been active in a political organization in the previous year. Best to use would be
a. contingency questions
b. matrix questions
c. matched questions
d. separate questionnaires
e. different response sets

8. Response set is most likely to occur in which kinds of questions?
a. matrix questions
b. contingency questions
c. closed-ended questions
d. open-ended questions
e. interview questions

9. A particularly useful strategy for improving rates to self-administered questionnaires is to
a. offer an inducement
b. use commemorative stamps
c. use colored paper
d. use hand delivery and/or pick up
e. use a jazzy cover letter

11. According to Babbie, a 60 percent return rate is considered
a. poor
b. adequate
c. good
d. very good
e. excellent

13. Interviewers can be helpful in dealing with confusing situations regarding a given item through the use of clarifying comments known as
a. specifications
b. elaborations
c. matrix questions
d. response set formats
e. conversations

14. Which one of the following is false regarding telephone interviews?
a. They are cheaper than in-person interviews.
b. They save time over in-person interviews.
c. They enhance the safety of the interviewer.
d. They make it harder for the respondent to terminate the interview.
e. They have a bad reputation.


Chapter 10

5. Professor Sullivan performed an observational study of the norms that govern interactions between cab drivers and their passengers. Which one of the following does this example reflect?
a. roles
b. encounters
c. episodes
d. groups
e. settlements

9.Which one of the following is false regarding field notes? Or are they all true?
a. Don't trust your memory more than you have to.
b. Take notes in stages.
c. Get the major points, but don't worry about getting as many details as you can.
d. Rewrite your notes before going to sleep.
e. All are true.

13. In comparison to surveys and experiments, field research has
a. high validity and high reliability
b. high validity and low reliability
c. low validity and low reliability
d. low validity and high reliability
e. high reliability, but only when the validity is high


Those are the ones I remember so far from the Study Guides, but it's not these ones that I am worried about. It's the ones that I have not seen before, can not remember, and don't know the answers to.

Any help on remembering any other questions - or even just the subject or topic that might trigger my own memory would be greatly appreciated!

-hope

Expts: tell me about em and use as much termanology as possible

So Experiments are the thing, so far, that I'm most rusty on... so tell me about em...

What part of the design process are there? What decisions are to be made? Why would you make those decisions? What're they useful for and why? What's some expt-related terminology that you think is important?

Would you ever want to run one? Why? What'd you want to test? How would you do it? Why?

--Crystal

ExamPartTwoYuck

Soooooooo. It's 9a Sunday AM and my day will be spent writing various research papers and studying for nigem. Who-hoo.

I'll check in periodically or more frequently if I/other folks have questions. The conversational aspect of this blogthing seem to help the material stick best for me, so later on I may post a general question or two if there's some material (experements!) that I'm having trouble remembering all aspects of.

Also, wanted to thank everyone for participating - I hope everyone else felt a much needed breath of confidence after our discussions (I know I did)!

--Crystal

Ethical problems in survey research?

Any examples of ethical problems in survey research?

I can think of general problems of misrepresented results and intentinally bad sampling but neither are survey-specific....?

Confidentiality? Always a problem.

??

Still Looking for Causal Time-Order and Compensation Information

Still Looking for Causal Time-Order and Compensation Information to fill in the hole in Question Number 1. If anyone has anything on this, now's the time to give it up.

Selection Biases

Can someone provide a BRIEF explanation of Selection Biases. I know what they are, but it takes me like 5 sentences to explain it and I know I don't have that kind of time.

What is a Regression Artifact?

what is a regression artifact?

anyone?.... anyone?.... anyone?.....

Is everyone still working on the questions they volunteered for?

Just wondering if everyone is still working on the question they chose earlier today so I know if I should be working on anything in particular. (That is, other than the 500 million billion other things I am trying to do - which does NOT include joining my friends at the bar around the corner on the patio where they continue to harass me from via the telephone, trying to get me to cave! All I can do is picture The Tiny Man looming over me like a giant fig tree, just waiting to rain figs down all over my GPA parade.)

So, are there any questions that need to be covered - we have 2 (Numbers 1 & 3) + 1/2 (My poor attempt at Number 5) + 1/2 (Terminology) posted so far, but I'm not worried. We're going to open up 5 big ol' cans of whoop-ass tomorrow when that essay sheet hits our desks.

Yeee-Haaaw!

What's the story with Internal Extrinsic Validity?

That's all. I just want to know a bit more about Internal Extrinsic Validity....

Three Methods of Data Collection: Survey, Telephone, Personal Interview (Strengths and weaknesses)

This is that table that he wants us to regurgitate w/o you know, regurging it. Yea.

The easiest way for me to remember it was to sit down and look at what each method was particularly strong and particularly weak on and then figure out the rest ('moderate') by common sense.

Additionally, ya'll get a sample of my stellar spelling skills - Nigem'll get the full effect tomorrow with my chicken scrawl handwriting.

I also made flash cards out of this stuff, cause it's the only bloody way I'll remember it.

What type of survey can you ask the most Q on?
Face to Face Interview

What survey type has limited contingency Qs?
Mail Survey

What type of survey has the greatest bias from social desirability?
Face to Face Interview

What is field research bad for?
Weak on internal validity
Low Control
High Risk of ethical errors
Low Reliability (of results)
Bad for Hypothesis Testing
Bad for Explanatory Research

What is field research good for?
Very natural
High External Validity
Allows moderate #s of subjects
Useful for long descriptive research

What are surveys strong on?
Large populations
High #s of subjects
(note: those two things are different, I had to stare at them for a while before I had it straight)
Allow Hypothesis testing
Useful for both explanatory and descriptive research

What are surveys bad at?
Only moderatly:

• Natural
• Xternal Validity
• Internal Validity
• Control (did one person fill out your survey, was it the person you wanted?, did they watch the news while they were doing it?)

Moderate ethical problems (Ex of what these are?)

What are experements strong on?
Internal validity
Control
High Reliability
Allow formation and testing of hypotheses
Useful for explanatory research and INTRODUCED phenomenia

A Note: Experements allow introduced phenomonia, you control the action so you can test before, after, and during. Surveys are ex post facto - you can only ask about stuff that's already happened (have you ever discriminated against someone in a job interview?) OR hypothetical actions (if given the opty, would you descriminate in job interview?). Field research is 'in the now' -- it's ongoing, complex phenomonia.

Just answered this, but in flashcard format:
Phenomenon in experements, survey and field research?
Introduced: Exp
Past: Survey
Ongoing: Field

What are experments weak on?
Ethical errors are common
External validity is a problem
unnatural
Small populations, low #s of subjects (with ass sample selection)
Not useful for descriptive research
Short duration

What methods can be used for explanatory research?
Experement or survey

What methods can be used for descriptive research?
Survey or field research

What's low on external validity?
Experement

What's high on internal validity?
Experement

Where are the most problems with ethics in research?
Both experements and field research

Where can you have hypotheses?
Experements and surveys

Discuss Advantages and Disadvantages of Specific Methods Used in Qualitative Research

Let's see.... there are what 7 methods? I don't even know that for sure! Let's start with what I do know...

Overall Advantages and Disadvantages of Qualitative vs. Quantitative Research

Advantages
1. The chief strength of qualitative field research lies in the depth of understanding it permits.
2. Flexibility is another advantage of field research.
3. Field research is (usually) relatively inexpensive.
4. Field research has greater validity than do survey and experimental measurements.


Disadvantages
1. It is not an appropriate means for arriving at statistical descriptions of a large population.
2. Low in reliability.
3. Often very personal

I Think These Are The Methods:
(then again, I could be way off here... I know these are the paradigms, but hell if I know if they're the "methods" or not)
1. Naturalism (Ethnography)
2. Ethnomethodology
3. Grounded Theory
4. Case Studies
5. Extended Case Method
6. Institutional Ethnogrpahy
7. Participatory Action Research

Not Every Girl Can Explain Insane Pain
(hey, gimme a break, it rhymes and "insane" correlates well with "institutional" and you can remember that this tip/trick goes with the qualitative question because it has the word "Explain" in it. (Even though it should be "descriptive" but just remember that the explaining is more related to qualitative that quantitave and you'll be fine.)


Note: At the bottom of page 287 in the Second Edition, our buddie Babbie states, "There aren't any specific methods attached to each of these paradigms..... The important distinctions of this section are epistemological, that is, having to do with what data mean, regardless of how they were collected." WTF?!?!?

1. Naturalism: an approach based on the assumption that social reality is "out there," ready to be observed and reported by the researcher as it "really is." An ethnography is a type of naturalist method.

2. Ethnomethodology: an approach that examines the rules that govern everyday life, often by breaking the rules

3. Grounded Theory: deriving theories from an analysis of the patterns, themes, and common categories discovered in observational data

4. Case Studies: an approach that focuses attention on one or a few instances of some social phenomenon

5. Extended Case Method: the purpose of this approach is to discover flaws in, and to modify existing social theories

6. Institutional Ethnogrpahy: an approach in which members of subordinated groups are asked about "how things work" so that researchers can discover the institutional practices that shape their realities

7. Participatory Action Research: with this paradigm, the researcher's function is to serve as a resource to those being studied, typically disadvantaged groups, as an opportunity for them to act effectively in their own interest


Okay, now that I've done everything BUT answer the question, I'm hoping someone can actually do what I was supposed to do - and that is Discuss Advantages and Disadvantages of Specific Methods Used in Qualitative Research! :-(

Some Terms Nigem Mentioned

While the tiny man have been tryng to scare us for his own amusement, here are the only two terms I caught (along with what Alicia captured as well) as he mentioned the possibility of their appearing on the (most likely not-an-optional-but-a- required-to-answer) Terminology Question List:

Hawthorne Effect: people are influenced by the presence of the experimentor

Symbolic Realism: indicates the need for social researchers to treat the beliefs they study as worthy of respect rather than as objects of ridicule

Regression Artifact: i have stuff on statistical regression and regression to the mean, but nothing on this (c'mon... someone has to have this!)

Reflexivity: things acting on themselves. your own characteristics can affect what you see and how you interpret it while doing field research.

Mortality: Experimental mortality (a source of internal invalidity) refers to experimental subjects dropping out of the experiment before it's completion, and this can affect statistical comparisons and conclusions.

Selection: (was he talking about selection biases?)

Intrinsic: inherent, being an innaate or essential part

Probe: request for elaboration

As a matter of intuition, I just want to put the possible need for knowing the O X O diagrams for the various testing designs and their proper names because I can so see "Solomon four-group design", "posttest only control group design", "one-group pretest-posttest design", etc. and static-group comparison being on that Terminology list.

Discuss Research (In)Validity

Research Validity
1. External Validity
2. Internal Validity
a. Extrinsic Validity
b. Intrinsic Validity
- History
- Maturation
- Testing
- Instrumentation
- Statistical Regression
- Selection Biasis
- Mortality
- Diffusion of Treatments
- Causal Time-Order
- Compensation
- Compensatory Rivalry
- Demoralization


Nigem's not too concerned with details on External Validity. In fact, he said, "Now watch this. External Validity deals with generalization and representation. Can you generalize your findings to the outside or "real" world? That is all that you need to say for the essay."

Internal Validity on the other hand is EXTREMELY important (it even earned its own handout: See Sources of Invalidity for Designs 1 through 6).

Internal Invalidity refers to the possibility that the conclusions drawn from EXPERIMENTAL results may not accurately reflect what actually occured during the experiment. You MUST have control over all factors affecting your experiment.

Side Note for people who like the x,y stuff:
Independent Variable (x) = stimulus
Dependent Variable (y)
MAKE SURE "x" (and ONLY "x") influences "y"

Internal Validity has both INTRINSIC and EXTRINSIC components.

The Extrinsic bit just deals with the selection of people used in the experiment. (that's all i've got on this... gimme what ya' got)

INTERNAL INTRINSIC VALIDITY is what it's all about!!!
All we're talking about here, really, is "What could go wrong?" Hmmmmmm.....
These are the sources that can interfere with your Independent Variable (x)
(and of course, we need to BRIEFLY explain these a bit and give an example)

a. History
- something historical happens during the experiment that has some effect
- ie: an assassination of an African-American leader during an experiment on prejudice

b. Maturation
- people change
- ie: short-term=get tired, bored long-term=grow older and wiser

c. Testing
- the actual testing/retesting process
- ie: people become influenced by the tests and their responses change

d. Instrumentation
- what is used to do the measuring/testing
- ie: must use same survey and pre & post-test so one is not more sensitive than another

e. Statistical Regression (to the mean)
- when subjects are so extreme the results will erroneously be attributed to the stimulus
- ie: kids are so bad in math, they can't get any worse or tall people have shorter children

f. Selection Biases
- experimental and control groups MUST be comparable
- ie: use proper methods for subject selection (we studied this before)
- Note: when subjects come from all volunteer group you lose External Validity

g. Mortality
- subjects die or drop out of experiment
- ie: bigots leave during film leaving higer ratio of less prejudiced people for post-testing

h. Diffusion of Treatments
- the more groups you use the more complications arise
- common sense - more groups = more opportunities for Internal Invalidity

i. Causal Time-Order
j. Compensation

k. Compensatory Rivalry (thank you Alicia for noting/adding that... Compensatory Rivalry is when one group competes against the other in an attempt to better performance.)

(i've got nothing on these three... i think i was still recovering from the repeated use of the kittens in the dark experiment... I actually have written in my notes, "Why does he keep referring to the deprivation of light experiment with 1 mo/2, 3, 6, etc. w/the maze and the food? It is so upsetting and disturbing!")

l. Demoralization
- oh, i'm upset because I'm in the Control Group and I feel left out, boo-hoo
- demoralized kids in educational studies may stop studying, act up, or get angry


Now for the fun stuff... how are we supposed to remember all of the factors? I mean, I can tell you what each of them is if you ask me, but listing them all from memory to begin with? Not so much. Sooooo.....


Hairy Men Don't Do Much Shaving To Create Sexy Clean Cut Images

That's right... HMDDMSTCSCCI (I know that's a lot, but hell, there are 11+1 (so that's 12) types of Internal Intrinsic Invalidity Issues - I point that out as your "double checker"... the I's at the beginnning of the words "Internal Intrinsic Invalidity" should remind you that there are 11+ 1 (three III's = three 111's = 11+1 = 12) factors. That way you don't skip a word in Hairy Men Dont Do Much Shaving To Create Sexy Clean Cut Images.)


History
Maturation
Diffusion
Demoralization
Mortality
Selection Bias
Testing
Causal Time-Order
Statistical Regression
Compensation
Compensatory Rivalry
Instrumentation

If you're a cynic and have an easier time remembering negative things instead of funnier stuff, try this one:

How Many Deeply Troubled Marriages Can Improperly Supervised Couples Counseling Sessions Destroy?

Does he want more than this? What do you think?

Third Time's A Charm... (hopefully)

Okay, here we go, one more time, yadda yadda yadda...

As I understand it, the following are the expected essay questions (however he seemed to make alot of changes Monday and I don't know if I'm on the right page anymore or not - so correct me where I'm wrong):

1. Discuss Research (In)Validity: External and Internal (see handout)

2. Compare 3 Modes of Observations and their Strengths and Weaknesses (see handout)

3. Three Methods of Data Collection: Survey, Telephone, Personal Interview (Strengths and weaknesses)

4. Problems You Encounter in Survey Research (see handout)

5. Discuss Advantages and Disadvantages of Specific Methods Used in Qualitative Research

6. Terminology (woo-hooo fun!)



Whatever! Narrow these down or clarify them more and I am going to start to answer the damn things....

New classfolk, comment here with e-mail to be added to admin of blog

So that you too can write posts!

--Crystal (clstal.at.gmail)