In this video I show you how to calculate Kappa with Studiocode when there are 3 category choices for instances. Basically the main difference is in the matrix of agreement options. The 4-cell matrix when there are 2 choices becomes a 9-cell matrix (see matrix below):
The “agreement” cells are now cells a, e, and i.
Attached to this post I have included a new PDF document as a reference as well as a copy of the code window with the scripts already in them. Feel free to copy and adapt these resources to fit your research needs. This same pattern can be used for 4 or more category Kappa calculations. HINT: Make the matrix first and identify the agreement cells (the downward diagonal).
New Kappa Code Window:
3 Group Kappa Directions:
As I mentioned in the video, make sure you are thinking about sample size (frequency of code), especially with larger matrices. I would recommend calculating Kappa after you combine timelines for multiple occasions. For infrequent codes, there are better IRR options.