Twenty Newsgroups Classification任务之二seq2sparse(三)
rmAttribute.class);
StringTuple document = new StringTuple();
stream.reset();
while (stream.incrementToken()) {
if (termAtt.length() > 0) {
document.add(new String(termAtt.buffer(), 0, termAtt.length()));
}
}
System.out.println("key:"+key.toString()+",document"+document);
}
}
得出的结果如下：
[plain]
key:4096,document[today, also, late.what, about, tomorrow]
其中，TokenStream有一个stopwords属性，值为：[but, be, with, such, then, for, no, will, not, are, and, their, if, this, on, into, a, or, there, in, that, they, was, is, it, an, the, as, at, these, by, to, of]，所以当遇到这些单词的时候就不进行计算了。