Point by Point IRR Considerations

NOTE:  This post is a follow up to my posts on calculating Point by Point Inter Rater Reliability (if you are not familiar with my scripting method for this, go back and watch that video first).

I recently received an email from a Studiocode user asking a great question about those instances where neither rater coded any instances of a particular code.  Using the scripts I originally provided, the output would say “IRR = 0%” if neither rater coded anything.  This is an inaccurate representation because, in actuality, both raters were in 100% agreement that there were no instances of that code.  In order to remedy this, we need to add a conditional IF statement into our output.  I explain how to do this in the following video.  Also note some of the considerations I raise at the end of the video regarding your interpretation of 100% IRR when there are no instances vs. 100% IRR when ther are coded instances.