A calibrated confusion matrix based on positive and background data

Remote sensing classification has been widely used to monitor the land use land cover change. The situation under which the users are only interested in a single land use type is called as one-class classification. In one-class remote sensing classification， researchers usually use one-class classification methods to reduce the cost of manual labeling， i.e. only samples from the target class （or positive class） are collected without requiring samples from other classes （or negative class）. However， traditional accuracy assessment methods require both positive and negative data to establish a confusion matrix and calculate accuracy measures. In the report， a calibrated confusion matrix based on the positive and unlabeled background data was proposed， and the two case of mapping impervious surface from a WorldView-3 image and a Landsat8 image were used to evaluate its effectiveness in accuracy assessment. The results showed that the calibration constant c obtained from the receiver operating characteristic curve falls into the range of 0.301 7 ~ 0.310 3 for WorldView-3 image and 0.289 5 ~ 0.313 2 for Landsat8 image， which are close to the corresponding true values of 0.279 0 and 0.300 0， respectively. There were big differences between the accuracy values obtained from the naive confusion matrix with positive and background data and that from the benchmark values obtained from the traditional confusion matrix with positive and negative data， whereas the accuracy values obtained from the calibrated confusion matrix are very similar to the benchmark values obtained from the traditional confusion matrix with positive and negative data. The results indicated that the proposed calibrated confusion matrix from positive and background data is effective for accuracy assessment of one-class classification， without requiring negative data.