SHREC 2012 - Sketch-Based 3D Shape Retrieval
The objective of this track is to evaluate the performance of different sketch-based 3D model retrieval algorithms using both hand-drawn and standard line drawings sketch queries on a watertight 3D model dataset.
Sketch-based 3D model retrieval is to retrieve 3D models using a 2D sketch as input. This scheme is intuitive and convenient for users to search for relevant 3D models and also important for several applications including sketch-based modeling and sketch-based shape recognition. However, most existing 3D model retrieval algorithms target the Query-by-Model framework, that is, using existing 3D models as queries. Much less research work has been done regarding the Query-by-Sketch framework. In addition, until now there is no comprehensive evaluation or comparison for available sketch-based retrieval algorithms. Considering of this, we organize this track to foster this challenging research area by providing a common sketch-based retrieval benchmark and soliciting retrieval results from current state-of-the-art retrieval methods for comparison. We will also provide corresponding evaluation code for computing a set of performance metrics similar to those used in the Query-by-Model retrieval technique.
3D target dataset
Our 3D benchmark dataset is built based on the Watertight Model Benchmark (WMB) dataset  which has 400 watertight models, divided into 20 classes, with 20 models each. The 3D target dataset contains two versions: Basic and Extended.
(1) Basic versionIt comprises 13 selected classes from the WMB dataset with each 20 models (in summary, 260 models). In the basic version, all 13 classes are considered relevant for the retrieval challenge. Fig. 1(c) shows one typical example for each class of the basic benchmark.
(2) Extended versionIt adds to the basic version all remaining 7 classes of the WMB dataset (each 20 models). These additional classes, however, are not considered relevant for the retrieval challenge but added to increase the retrieval difficulty of the basic version. Fig. 1(d) illustrates typical examples for these remaining 7 irrelevant classes. The Extended version is utilized to test the robustness performance of a sketch-based retrieval algorithm.
2D query set
The 2D query set comprises two subsets, falling into two different types.
(1) Hand-drawn sketches
We utilize the hand-drawn sketch data compiled by TU Darmstadt and Fraunhofer IGD . It contains 250 hand-drawn sketches, divided into 13 classes. One typical example for each class is shown in Fig. 1(a).(2) Standard line drawings
We also select relevant sketches from the Snograss and Vanderwart's standard line drawings dataset . Some examples are shown in Fig. 1(b).In this track, the two subsets will be tested separately. However, users can also form a query set by combining these two to form a query set which contains diverse types of sketches.
- The participants must register by sending a message to SHREC@nist.gov. Early registration is encouraged, so that we get an impression of the number of participants at an early stage.
- The database will be made available via this website. Test dataset.
- Participants will submit the dissimilarity matrix (also named as distance matrix) for the test database. Up to 5 matrices per group may be submitted, resulting from different runs. Each run may be a different algorithm, or a different parameter setting. More information on the dissimilarity matrix file format. More information on the dissimilarity matrix file format.
- The evaluations will be done automatically.
- The organization will release the evaluation scores of all the runs.
- The participants write a one page description of their method with two figures and send their comments on the evaluation results.
- The track results are combined into a joint paper, published in the proceedings of the Eurographics Workshop on 3D Object Retrieval.
- The description of the tracks and their results are presented at the Eurographics Workshop on 3D Object Retrieval (May 13, 2012).
|January 28||- Call for participation.|
|February 1||- Few sample models of the test database will be available on line.|
|February 5||- Please register before this date.|
|February 7||- Distribution of the whole database. Participants can start the retrieval.|
- Submission of results (dissimilarity matrix) and a one page description of their method(s).
(Extended to February 15)
|February 17||- Distribution of relevance judgments and evaluation scores.|
|February 23||- Track is finished, and results are ready for inclusion in a track report.|
|February 26||- Camera ready track papers submitted for printing.|
|May 13||- Eurographics Workshop on 3D Object Retrieval including SHREC'2012.|
Tobias Schreck - University of Konstanz
We would also like to thank Daniela Giorgi who built the Watertight Shape Benchmark for SHREC 2007.
We would like to thank Snograss and Vanderwart who built the standard line drawings dataset.
 S. M. Yoon, M. Scherer, T. Schreck, and A. Kuijper. Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In ACM Multimedia, pages 193-200, 2010.
 J. G. Snodgrass and M. Vanderwart. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6(2):174-215, 1980.
 B. Li, Y. Lu, A. Godil, T. Schreck, B. Bustos, A. Ferreira, T. Furuya, M.J. Fonseca, H. Johan, T. Matsuda, R. Ohbuchi, P.B. Pascoal, J.M. Saavedra, A comparison of methods for sketch-based 3D shape retrieval, Computer Vision and Image Understanding (2013), doi: http://dx.doi.org/10.1016/j.cviu.2013.11.008.