I'm in the process of planning a NNTP robot for KDE 4.x. The plan is for it to have feature parity with NewsLeecher, which I've been using for quite a while. Unfortunately, NewsLeecher has issues.
What I'm planning will not be simply an NZB downloader, but a fully fledged NNTP client that stores headers locally and allows local searching. However, some newsgroups can have header counts reaching into the hundreds of millions and sometimes further - I've seen a header count of over 3 billion for one particular newsgroup.
So what I'm looking for is an embeddable database capable of handling that many records, with stability and a decent level of performance. I've been looking at NoSQL databases, but I'm not sure whether key/value or document databases would be a better fit.
Or should I tie the project directly into Akonadi and use that to index the headers instead?
Any advice is appreciated.
Edited by parityboy - 9/5/12 at 4:32am
What I'm planning will not be simply an NZB downloader, but a fully fledged NNTP client that stores headers locally and allows local searching. However, some newsgroups can have header counts reaching into the hundreds of millions and sometimes further - I've seen a header count of over 3 billion for one particular newsgroup.
So what I'm looking for is an embeddable database capable of handling that many records, with stability and a decent level of performance. I've been looking at NoSQL databases, but I'm not sure whether key/value or document databases would be a better fit.
Or should I tie the project directly into Akonadi and use that to index the headers instead?
Any advice is appreciated.
Edited by parityboy - 9/5/12 at 4:32am
Mythica
(14 items) |
|
Mythica
(14 items) |
)
