Automatic Document Classification Applied to Swedish News
Independent thesis Basic level (professional degree), 20 points / 30 hpStudent thesis
The first part of this paper presents briefly the ELIN system, an electronic newspaper project. ELIN is a framework that stores news and displays them to the end-user. Such news are formatted using the xml format. The project partner Corren provided ELIN with xml articles, however the format used was not the same. My first task has been to develop a software that converts the news from one xml format (Corren) to another (ELIN).
The second and main part addresses the problem of automatic document classification and tries to find a solution for a specific issue. The goal is to automatically classify news articles from a Swedish newspaper company (Corren) into the IPTC news categories.
This work has been carried out by implementing several classification algorithms, testing them and comparing their accuracy with existing software. The training and test documents were 3 weeks of the Corren newspaper that had to be classified into 2 categories.
The last tests were run with only one algorithm (Naïve Bayes) over a larger amount of data (7, then 10 weeks) and categories (12) to simulate a more real environment.
The results show that the Naïve Bayes algorithm, although the oldest, was the most accurate in this particular case. An issue raised by the results is that feature selection improves speed but can seldom reduce accuracy by removing too many features.
Place, publisher, year, edition, pages
Institutionen för datavetenskap , 2005. , 86 p.
ELIN, automatic text classification, Naïve Bayes network, Rocchio
IdentifiersURN: urn:nbn:se:liu:diva-3065ISRN: LITH-IDA-EX--05/038--SEOAI: oai:DiVA.org:liu-3065DiVA: diva2:20343