Percent Agreement In Statistics
where in is the relative correspondence observed between advisors (identical to accuracy), and pe is the hypothetical probability of a random agreement, the observed data being used to calculate the probabilities of each observer who sees each category at random. If the advisors are in complete agreement, it`s the option » 1″ «textstyle» «kappa — 1.» If there is no agreement between advisors who are not expected at random (as indicated by pe), the «textstyle» option is given by the name «. The statistics may be negative, which implies that there is no effective agreement between the two advisers or that the agreement is worse than by chance. As Marusteri and Bacarea (9) have found, there is never 100% certainty about the results of the research, even if the statistical significance is reached. The statistical results used to test hypotheses about the relationship between independent and dependent variables are meaningless when there are inconsistencies in the evaluation of variables by evaluators. If the agreement is less than 80%, more than 20% of the data analysed is wrong. With a reliability of only 0.50 to 0.60, it is understandable that 40 to 50% of the data analyzed is wrong. If Kappa values are less than 0.60, the confidence intervals around the received kappa are so wide that it can be assumed that about half of the data may be false (10). It is clear that statistical significance does not mean much when there are so many errors in the results tested. The percent deal and Kappa have strengths and limits. Percentage chord statistics are easy to calculate and directly interpretable. Its main restriction is that it does not take into account the possibility that councillors guess on partitions. It may therefore overestimate the true agreement between the advisors.
The Kappa was designed to take into account the possibility of rates, but the assumptions it makes about the independence of advisers and other factors are not well supported, and it can therefore reduce the estimate of the agreement excessively. In addition, it cannot be interpreted directly, and it has therefore become common for researchers to accept low levels of kappa in their interrater reliability studies. The low level of reliability of the Interrater is unacceptable in the field of health or clinical research, especially when the results of studies can alter clinical practice in a way that leads to poorer patient outcomes. Perhaps the best advice for researchers is to calculate both the approval percentage and kappa. While there are probably a lot of rates between advisors, it may be helpful to use Kappa`s statistics, but if the evaluators are well trained and low rates are likely, the researcher can certainly rely on the percentage of consent to determine the reliability of the Interraters. A final concern about the reliability of advisors was introduced by Jacob Cohen, a leading statistician who, in the 1960s, developed key statistics to measure the reliability of interrater, Cohens Kappa (5). Cohen indicated that there will likely be some degree of match among data collectors if they do not know the correct answer, but if they simply guess. He assumed that a number of conjectures would be speculated and that insurance statistics should be responsible for this fortuitous agreement. He developed Kappa`s statistics as an understanding of this random agree factor. The concept of «advisor agreement» is quite simple and, for many years, the reliability of Interraters has been measured as a percentage of match among data collectors.