A Benchmark for 3D Interest Point Detection Algorithms

This benchmark aims to provide tools to evaluate 3D Interest Point Detection Algorithms with respect to human generated ground truth.

Please refer to the paper for more information about this benchmark: Helin Dutagaci, Chun Pan Cheung, Afzal Godil: “Evaluation of 3D interest point detection techniques via human-generated ground truth", The Visual Computer, 2012. [Bib Tex]

3D Model Data Set and Interest Points Marked by Human Subjects

Using a web-based subjective experiment, human subjects marked 3D interest points on a set of 3D models. The models were organized in two datasets: Dataset A and Dataset B. Dataset A consists of 24 models which were hand-marked by 23 human subjects. Dataset B is larger with 43 models, and it contains all the models in Dataset B. The number of human subjects who marked all the models in this larger set is 16.

Some of the models are standard models that are widely used in 3D shape research; and they have been used as test objects by researchers working on the best view problem. Examples are Armadillo, David’s head, Utah teapot, Bunny, etc. We chose some of the models from The Stanford 3D Scanning Repository and some others from the Watertight Models Track of SHREC 2007 (see the license).The Stanford 3D Scanning Repository and some others from the Watertight Models Track of SHREC 2007 (see the license).

The data set can be downloaded from the link: DATA SET. The triangular mesh models are stored in MAT files. Please refer to README for details.

The interest points marked by human subjects' can be downloaded from the link: HUMAN SUBJECT’s INTEREST POINTS.

The text files are named with respect to the following template:
subjectname-modelname_points.txt

An example is jck-ant_points.txt. Where jck is the ailas for the human subject and camel is the name of the 3D model. Each row in the text file gives x, y, z coordinates of an interest point marked by the subjet on the 3D model.



3D Interest Point Detection Algorithms

We have compared five 3D Interest Point Detection algorithms. The interest points detected on the 3D models of the dataset can be downloaded from the link next to the corresponding algorithm. Please refer to README for details.

The interest points for all of the six algorithms can also be downloaded as a single zip file: ALGORITHMs_INTEREST_POINTS

Evaluation Codes

There are two steps for evaluating an “interest point detection algoritm” with respect to the interest points marked by human subjects:

The MATLAB code can be downloaded from the following link: MATLAB code. The details about how to use the code are present in the pdf document : README.


Downloads

README : Document explaining the formats of data files and how to use the evaluation codes.
DATA SET : 51 3D models in MAT format. The lists of the models for Dataset A and Dataset B are stored in two MAT files; exp_model_list_A and exp_model_list_B, respectively.
HUMAN SUBJECT’s INTEREST POINTS : Interest points of the 51 models marked by human subjects. The lists of the subjects who marked on Dataset A and Dataset B are stored in two MAT files; subject_list_A and subject_list_B, respectively.
ALGORITHMs_INTEREST_POINTS : Interest points detected by five algorithms.
MATLAB code : MATLAB code for generating the ground truth and evaluating interest point detection algorithms.
OUTPUT data : MAT files generated by the codes

The entire benchmark, including the output MAT files generated by the codes, can be downloaded as a single archive file:
IP_BENCHMARK

Citation

Helin Dutagaci, Chun Pan Cheung, Afzal Godil, “Evaluation of 3D interest point detection techniques via human-generated ground truth”, The Visual Computer, 2012. [Bib Tex]

References

Contact Information

For more information and feedback, please contact Helin Dutagaci or Afzal Godil