In this article, we propose a slice-based interactive segmentation of spectral CT datasets using a bag of features method. The data are acquired from a MARS scanner that divides up the X-ray spectrum into multiple energy bins for imaging. In literature, most existing segmentation methods are limited to performing a specific task or tied to a particular imaging modality. Therefore, when applying generalized methods to MARS datasets, the additional energy information acquired from the scanner cannot be sufficiently utilized. We describe a new approach that circumvents this problem by effectively aggregating the data from multiple channels. Our method solves a classification problem to get the solution for segmentation. Starting with a set of labeled pixels, we partition the data using superpixels. Then, a set of local descriptors, extracted from each superpixel, are encoded into a codebook and pooled together to create a global superpixel-level descriptor (bag of features representation). We propose to use the vector of locally aggregated descriptors as our encoding/pooling strategy, as it is efficient to compute and leads to good results with simple linear classifiers. A linear support vector machine is then used to classify the superpixels into different labels. The proposed method was evaluated on multiple MARS datasets. Experimental results show that our method achieved an average of more than 10% increase in the accuracy over other state-of-the-art methods.