This paper presents an anisotropic denoising/smoothing algorithm for point-sampled surfaces. Motivated by the impressive results of mean shift filtering on image denoising, we extend the concept to 3D surface smoothing by taking the vertex normal and the curvature as the range component and the vertex position as the spatial component. Then the local mode of each vertex on point-based surfaces is computed by a 3D mean shift procedure dependent on local neighborhoods that are adaptively obtained by a kdtree data structure. Clustering pieces of point-based surfaces of similar local mode provides a meaningful model segmentation. Based on the adaptively clustered neighbors, we finally apply a trilateral point filtering scheme that adjusts the position of sample points along their normal directions to successfully reduce noise from point-sampled surfaces while preserving geometric features.