Point by Point IRR Considerations

NOTE:  This post is a follow up to my posts on calculating Point by Point Inter Rater Reliability (if you are not familiar with my scripting method for this, go back and watch that video first).

I recently received an email from a Studiocode user asking a great question about those instances where neither rater coded any instances of a particular code.  Using the scripts I originally provided, the output would say “IRR = 0%” if neither rater coded anything.  This is an inaccurate representation because, in actuality, both raters were in 100% agreement that there were no instances of that code.  In order to remedy this, we need to add a conditional IF statement into our output.  I explain how to do this in the following video.  Also note some of the considerations I raise at the end of the video regarding your interpretation of 100% IRR when there are no instances vs. 100% IRR when ther are coded instances.


Posted on July 23, 2015, in Uncategorized. Bookmark the permalink. 1 Comment.

  1. Nice explanation Tara. I love how you make scripting to see output in code window seem completely manageable. Thanks for showing how measuring inter-rater reliability, intra-rater reliability is a great benefit of completing your video research through Studiocode. Keep it up!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: