Newsleecher tutorial

Support, Discussion, Reviews
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Newsleecher tutorial

Post by Winnow »

I've been wanting to try out Macromedia (Adobe) Capitvate for awhile now and so figured I'd make a Newsleecher tutorial/demo to give myself some practice. I get lots of questions about the newsgroups since I can't shut up about them so this might show people how downloading binaries works.

It's a 9.8mb flash interactive tutorial I just made. Nothing fancy but should give a good idea of how to do things.

I might make one on DC++ and Copernic if this turns out to be helpful.

I made this a little irritating as you must click the red boxes to continue through the tutorial.

http://winnow.veeshanvault.org/files/no ... cher-2.swf
Ebumar
Way too much time!
Way too much time!
Posts: 1302
Joined: July 8, 2002, 8:22 pm
Location: YOUR MOM'S HOUSE

Post by Ebumar »

Pretty good. Will help people to figure certain things out.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

Here's the latest on Newsleecher's roadmap.

For those holding out for text message handling, it looks like that's coming soon. The continued improvements with Super Search are welcomed as well including date range, and more possible category groupings.

Newsleecher = the best binary newsreader with the best, and fastest, continued development.
Latest News:

Oct 23, 2006

It's been quite a long time since the last NewsLeecher status report, so here's a quick roadmap if what's to come in the (near) future:

NewsLeecher v3.8 Final

NewsLeecher v3.8 Final, which implements encrypted usenet connections (SSL), among other new features will be released within the next couple of days.

NewsLeecher v3.9x

The v3.9 series will implement a new feature called "Repair'n'Extract". Catchy name, eh!?;). Well anyway ... The feature will make it possible to repair damaged downloads, using .PAR2 repair files, and automatically extract downloaded archives, using various customizable settings, such as extraction destination folder, etc.. We've already put a lot of work into the repair'n'extract feature, and it's actually already half finished as a basic feature. We expect the first v3.9 beta to be out soon after the v3.8 final release.

NewsLeecher v4.x

The v4.x series will primarily focus on text reader capabilities! Our goal is to implement an easy-to-use, powerful and *non-bloated* text reader feature-set into NewsLeecher without breaking the current look and feel of the application. Besides text reader capabilities, the v4x series will also improve the Repair'n'Extract feature aswell as other parts of the application of course.

The SuperSearch Service will of course also be improved along the way. Longer article retention, combined search categories, text group search support and other interesting things are on the to-do list already!
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

It's time to put away your childish news readers and move to Newsleecher if you haven't yet.

3.9b1 is out which now includes the Repair and Extract feature.

Newsleecher now automatically downloads only the PAR files that are needed to repair a set of files, and then automatically unRars the files for you, cleaning up/deleting the par and rar files in the process and allows you to select a destination folder if you wish for the finished files.

This rocks for anyone that downloads large files like DVD images. Extracting those takes time so it's nice to have it automated when you go to bed so you don't have to wait for that kind of stuff later. I downloaded all 11 seasons of Married with Children (259 episodes + special reunion) which was 42 GB in size. It took about eight hours and had PAR files linked to the entire 260 episodes. Can you imagine how long it takes to fix a 42 GB set of files with Pars? ...I can't tell you because they all downloaded perfectly but if they hadn't it would have been nice to have the Par process run automatically afterwards, one, to only download a few pars instead of the several GBs of pars posted and two, because it would be done when I woke up.

For binaries, Newsleecher is so far ahead of all other Newsreaders I'm not sure you can even put them in the same category. Works great in Vista too!

I suppose the last holdouts will be those that actually "read" newsgroups still. I haven't done that in years as most text messages are posted in forums like this on these days but I suppose it's stil a legit reason to use another Newreader although the Supersearch and overwhelming load of nice options/features in Newsleecher would make me use two different Newsreaders if I had to.

Version 4 will take care of that:
NewsLeecher v4.x

The v4.x series will primarily focus on text reader capabilities! Our goal is to implement an easy-to-use, powerful and *non-bloated* text reader feature-set into NewsLeecher without breaking the current look and feel of the application. Besides text reader capabilities, the v4x series will also improve the Repair'n'Extract feature aswell as other parts of the application of course.
For those that already use Newleecher, first reports are good with 3.9b1 but you might want to wait if you only like final versions. 3.8 final was just released.
User avatar
Siji
Way too much time!
Way too much time!
Posts: 4040
Joined: November 11, 2002, 5:58 pm
Gender: Male
XBL Gamertag: mAcK 624
PSN ID: mAcK_624
Wii Friend Code: 7304853446448491
Location: Tampa Bay, FL
Contact:

Post by Siji »

The par support sounds nice. The same level of watch/delete filter support as Agent would be nicer. Allowing the user to set whatever color they want for various text/windows would be nicer. Accurately showing the number of headers in each group or even each virtual group would also be nice.

No compelling reasons to switch from Agent, which is much more stable and allows watch filters and lets me set whatever color I want for headers. I hate inverse colors.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

Siji wrote:The par support sounds nice. The same level of watch/delete filter support as Agent would be nicer. Allowing the user to set whatever color they want for various text/windows would be nicer. Accurately showing the number of headers in each group or even each virtual group would also be nice.

No compelling reasons to switch from Agent, which is much more stable and allows watch filters and lets me set whatever color I want for headers. I hate inverse colors.
Whatever you're talking about seems to relate to text messages. Newsleecher doesn't support text groups/browsing.

Newsleecher is amazingly stable considering certain newsgroups that have eight million headers like alt.binaries.movies.divx. I'm also not sure if you're inferring that Newsleecher doesn't have accurate header info. It does, and that's also dependant on your news provider as well. Get Giganews and you'll always be in good shape.

Agent is a joke compared to Newsleecher when it comes to binaries. I keep an eye on Agent to see how things progress with it (very slowly). I used Free Agent for several years before BNR2. BNR2 is ancient and buggy but years ahead of Agent when dealing with binaries.

There's a laundry list of features I use with Newsleecher each day which makes searching, downloading and handling binaries unbelievably easy and quick.

I guess it comes down to time. I always look for the most efficient and fastest way to get things done in order to get to the real purpose of enjoying the results. Destination, not journey in this case although Newsleecher is a blast to use. If I want to listen to a song, I want to be listening to it in a few seconds, not hours later after finding it. Same with searching the net, searching my hard drives, or finding toilet paper in the closet.

I suppose where I'm wrong here is thinking most people want to optimize things for speed and efficiency. My philosophy is the faster I gather data, the more time I have to digest the data I've gathered. A basic example would be download speeds. The faster you can download, the faster you get to the "enjoyable" part...but speed isn't everything...multiple connections to maximize speeds...better organization to find what you're looking for hidden in masses of data...a faster computer to decompress files...customized applications with UI's and capabilities geared toward speeding up the mundane search/execute/move tasks in order to watch that movie, listen to that song, read that pdf, pay those bills...whatever...any task that doesn't involve me consuming data in some way I want to be a fast as possible. I can sit in SL and blow hours doing nothing productive but that's my own choice. Time spent enjoying multimedia, life in general, friends, sex, etc is no rush.

As for text newsgroups, I've found faster ways to find information I'm looking for than text newsgroups and if I'm desperate, I'll use google's newsgroup search...not that there's anything wrong with text newsgroups for those that like them.

To summarize:

Gathering Data: Destination > Journey
Enjoying the data you've gathered: Journey > Destination

The Tao of Winnow
User avatar
Ashur
Way too much time!
Way too much time!
Posts: 2604
Joined: May 14, 2003, 11:09 am
Location: Columbus OH
Contact:

Post by Ashur »

Winnow wrote:It's time to put away your childish news readers and move to Newsleecher if you haven't yet.
I laughed when I read this. I bought Agent back in like 1995 and have been using it faithfully, but I've noticed creep of PAR2 and nbz(?) files into the newsgroups and scractched my head.

I went to download headers from a.b.m to grab Rome and "Operation Timed Out", due to too many headers and there's no way to grab headers just on date, just by last X number in Agent.

So when I read this, I went and downloaded the trial. Tried it out in my normal groups, worked great, then hit a.b.m and it was still downloading headers when I left. So far it's looking very good and I may be due for a switch.

The $19.95 package looks reasonable as well, all upgrades for a year. Let's see if I can grab last night's Rome and then we'll see. I may also wait to see how it behaves on my new ISPs newservers first since that's this coming weekend.
- Ash
User avatar
Ashur
Way too much time!
Way too much time!
Posts: 2604
Joined: May 14, 2003, 11:09 am
Location: Columbus OH
Contact:

Post by Ashur »

Multiple channels open to the server is also a big plus. With Agent everything was done serially and I could be in the middle of a big download and see a small file I want to check out. With Agent I was stuck, I could stop the download, grab the file, then resume it (usually losing whatever file is in progress) or just wait.

But yeah, it needs text support badly (I understand that's not it's focus), but even binary groups have some amount of chatter and having formatted usenet messages pop open in text files is crude and it seems like they could do so much better, even if to just format the messages in a UI pop-up.
- Ash
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

Ashur wrote:Multiple channels open to the server is also a big plus. With Agent everything was done serially and I could be in the middle of a big download and see a small file I want to check out. With Agent I was stuck, I could stop the download, grab the file, then resume it (usually losing whatever file is in progress) or just wait.

But yeah, it needs text support badly (I understand that's not it's focus), but even binary groups have some amount of chatter and having formatted usenet messages pop open in text files is crude and it seems like they could do so much better, even if to just format the messages in a UI pop-up.
for the rare text message I want to read in a binary newsgroup, I just open in notepad.

I love the extreme flexibility in downloading, sorting and prioritizing downloads in Newsleecher. I will typically start a large download and then either use "open" or "leech first" to view smaller files while I wait or grab an mp3, etc. Easy to manipulate the queue as well. Always make good use of virtual groups as well.

Of course, Supersearch is the best feature of all.
Last edited by Winnow on January 29, 2007, 2:40 pm, edited 1 time in total.
User avatar
Ashur
Way too much time!
Way too much time!
Posts: 2604
Joined: May 14, 2003, 11:09 am
Location: Columbus OH
Contact:

Post by Ashur »

I hope WOW's news servers are as good as time warner's. I dislike having to log in though, as I saw on a FAQ may be required. I may go to a third-party newserver if it sucks, but my budget is negligible.
- Ash
User avatar
Xatrei
Way too much time!
Way too much time!
Posts: 2104
Joined: July 22, 2002, 4:28 pm
Gender: Male
Location: Boringham, AL

Post by Xatrei »

I've always been a fan of agent for text groups, but for binaries I've used a combination of stuff. A couple different nzb grabbers (BNR/BNR2 for windows and a few different linux-based tools), and Xnews for general binary browsing and downloading, and ExtractNow + QuickPar for extraction. It works for my needs, but some of the more recent improvements to newsleecher have me on the verge of taking the plunge.

edit to add: I've found that most ISP provided news servers have either throttled bandwidth, limited retention, limited or no simultaneous connections or usually all of the above. If you download much of anything via binary groups, your best bet is to upgrade to a 3rd party service. Some can be had very inexpensively if you don't have requirements for enormous bandwidth, transfer or simultaneous connections. My service is up to 4 simultaneous connections, 50 day retention, unlimited bandwidth and only costs 15 bucks a month. You can spend more or less depending on your needs.
"When I was a kid, my father told me, 'Never hit anyone in anger, unless you're absolutely sure you can get away with it.'" - Russel Ziskey
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

Xatrei wrote:I've always been a fan of agent for text groups, but for binaries I've used a combination of stuff. A couple different nzb grabbers (BNR/BNR2 for windows and a few different linux-based tools), and Xnews for general binary browsing and downloading, and ExtractNow + QuickPar for extraction.

It works for my needs, but some of the more recent improvements to newsleecher have me on the verge of taking the plunge.
I think there's a 30 day trial version of Newsleecher that allows you to use Super Search. It will give the search results but won't display which newsgroup they're in (can still download them though directly using Supersearch. Just keep in mind that the unrestricted Super Search displays the newsgroup the results come from as well which is great for finding newsgroups with odd names that turn out to be worth browsing directly. Also keep in mind that Super Search's extremely fast results contain up to 100+ days of binary files. If you're not using something like Giganews that matches that kind of retention, you'll need to adjust Super Search to display only the amount of retention your particular newsgroup provider has.
User avatar
Ashur
Way too much time!
Way too much time!
Posts: 2604
Joined: May 14, 2003, 11:09 am
Location: Columbus OH
Contact:

Post by Ashur »

Time Warner just over the holiday season apparently upgraded to 4 max connections and 30 day retention (used to be a few days max), a shame to lose it, but I'll be saving about $60 a month.and getting DVR to boot.
- Ash
User avatar
Xatrei
Way too much time!
Way too much time!
Posts: 2104
Joined: July 22, 2002, 4:28 pm
Gender: Male
Location: Boringham, AL

Post by Xatrei »

Didn't see a trial that included the Super Search, but I didn't really spend much time looking around. When I saw a $2.99 a month subscription option, I figured that's a good way to try it out.

Super Search == $$$ damn, this thing is awesome.
"When I was a kid, my father told me, 'Never hit anyone in anger, unless you're absolutely sure you can get away with it.'" - Russel Ziskey
User avatar
Mak
Almost 1337
Almost 1337
Posts: 834
Joined: August 5, 2002, 4:13 pm
Location: Tucson, AZ
Contact:

Post by Mak »

Based on Winnow's recommendation, I've been using NL for a couple of months now and I'm a HUGE fan of the supersearch feature.

Two thumbs up from me.
Makora

Too often it seems it is the peaceful and innocent who are slaughtered. In this a lesson may be found that it may not be prudential to be either too peaceful or too innocent. One does not survive with wolves by becoming a sheep.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

Siji wrote: No compelling reasons to switch from Agent, which is much more stable and allows watch filters and lets me set whatever color I want for headers. I hate inverse colors.
Just FYI, your colors can be anything you want. NewsLeecher has the best option of any Binary Newsreader:

Image

Image

It's also great for font style, spacing, column adjustments, etc.
User avatar
Ashur
Way too much time!
Way too much time!
Posts: 2604
Joined: May 14, 2003, 11:09 am
Location: Columbus OH
Contact:

Post by Ashur »

I cried. I laughed. I liked it. I bought it.
- Ash
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

For Newsleecher users, the newly added ability to search for posts by poster is a great addition.

This was mentioned in the news tab of Newsleecher awhile back but if you haven't tried it, it's great.

When I'm looking for something that has only general descriptions and it's hard to find, once I find a poster that has material I like, I search for all of their posts.

All you do is type into the search field:

poster:"poster name"

and it will pull all of that persons posts. It works just like the supersearch so you don't need to be exact, just need enough of the posters email address or their user name

Lotsaposts Jim <blah>

so could search various ways:

poster:"Lotsaposts Jim"
poster:"blah@blah.com"

works well!
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

edit: outdated info!
Last edited by Winnow on May 25, 2007, 9:51 am, edited 1 time in total.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Post by Winnow »

Here's a tip for Newsleecher Supersearch users.

A lot of times you can find MP3's located in the "main" search category as opposed to the "music" search. Supersearch separates categories by what newsgroup the file is in and not by what the file extension is. Also, you'll sometimes find rar'd albums which show in the main category and not the music.

That leads to searching for "discography" in the main search group which I do every so often to see what's out there. I don't care if you're straight as an arrow and buy all of your music legit from iTunes, you're going to miss some things that aren't on iTunes or some other service. The peeps that put together these discographies are pretty thorough.

Here's a sample of some of the discographies I've grabbed:

Beatles
Image

Dire Straights
Image

Led Zeppelin
Image

Red Hot Chili Peppers
Image

If you're a big fan of a certain group, check for a discography using Supersearch! You can go buy them all on iTunes if you end up liking them (but some you may not be able to get using a music service)
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

New beta for Newsleecher is out. b4 is actually out now with a few fixes but b3 has the additions listed so I'll quote from that:

Version history log:

Hoping this version put the advanced filters back.

http://www.newsleecher.com/forum/viewtopic.php?t=6211
3.9 Beta 3 - Released 08 Jun 2007

---------------------------------

+: Added an option to enable/disable the single-click activation

when clicking the tray icon. The option can be accessed from

the tray popup menu.

+: The Article Subject Ignore Filter (which can be accessed from

'Options->Filters->Ignore Filter') now also applies to

SuperSearch articles results and Repair'n'Extract set files.

*: The filter system has been much improved. All main filters now

support boolean searching ("and", "or", "not" search queries)

and new toolbars with subfilters makes it possible to narrow

the search queries even further if wanted.

*: Changed the functionality of the Repair'n'Extract Auto-Unpause

feature. Instead of auto unpausing *any* needed set files from

the transfer queue, it will now only auto unpause PAR repair

files. This makes it possible to manually pause files in the

transfer queue, without the RnE 'auto unpause' feature unpauses

them again.

*: The Pause Dl. Filters can now temporarily be ignored, by

holding down the F12-key on the keyboard, while adding

articles to the transfer queue.

*: When Repair'n'Extract encounters an error with a set, such as

when it's unable to extract archives due to password protection

or lack of diskspace, the exact error will now be shown in the

set 'status' listview column.

*: When using the 'Perform Action when Queue is Empty' feature,

the action will now be delayed until the transfer queue is

empty *and* all eventual jobs in the Repair'n'Extract queue

have finished:

http://www.newsleecher.com/forum/viewtopic.php?t=12636

*: Lots of smaller additions and improvements.

!: Since the previous beta, the article filters didn't always

work as they should. This has been fixed.

!: Fixed "When Empty > Run Commandline does not correctly parse

input string" bug:

http://www.newsleecher.com/forum/viewtopic.php?p=32682

!: Fixed "Post confirmation dialog chops off long subject

titles" bug:

http://www.newsleecher.com/forum/viewtopic.php?p=51238

!: Fixed bug where articles were not saved to the disk, if the

specified download folder didn't exist, and the default

download folder didn't exist either. In such case, articles

will now be saved to the '%USERPROFILE%\Downloads\' folder:

http://www.newsleecher.com/forum/viewto ... 4179#54179

!: Fixed "Resuming a partially download UUEncoded multipart

results in a failed download" bug:

http://www.newsleecher.com/forum/viewtopic.php?t=9261

!: Fixed an issue, where toolbar items sometimes were inaccessible

if the toolbar was wider than the NewsLeecher window:

http://www.newsleecher.com/forum/viewtopic.php?t=11689

!: Fixed "Explorer tab displays file sizes of large files (> 4GB)

incorrectly" bug:

http://www.newsleecher.com/forum/viewtopic.php?t=10905

!: Fixed "Exported NZBs have an incorrect codepage defined" bug:

http://www.newsleecher.com/forum/viewtopic.php?t=8210

!: Fixed bug where the Auto-Import-NZB feature sometimes would

import incomplete NZB files:

http://www.newsleecher.com/forum/viewtopic.php?t=12070

!: Fixed bug where Repair'n'Extract sometimes would delete files

which shouldn't be deleted, after having processed the set.

!: Fixed bug where NewsLeecher would sometimes hang when using the

the "When Queue is Empty > Shutdown" feature:

http://www.newsleecher.com/forum/viewtopic.php?t=13509

!: Lots of other minor bugfixes here and there.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Man, I'm not worthy. Look at this person's newsgroup usage per month:
44369_transfer_122_544lo.jpg
I'm trying to figure out if there's a way to reset my connection every 20 seconds to exploit Cox's "Powerboost" which provides 30Mbps during the initial transfer for each connection. I'm thinking if I can get Newsleecher to restart one of the 10 pipes I have open every 20 seconds, it might do the trick.

Giganews has an Accelerator beta app as well but I don't think those of us that have the SSL plan need it:

http://www.giganews.com/accelerator.html

I have noticed fast header downloads since switching to SSL.

Some notes on Newsleecher's Forums about it:

http://www.newsleecher.com/forum/viewtopic.php?t=13796
You do not have the required permissions to view the files attached to this post.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Image

The search boxes are what was missing from 3.9b2. That was driving me crazy. I like to quick filter by size a lot (not sort) and poster filtering comes in handy often as does the day filtering.

3.9 b4's looking good!

BTW, you can see in the image, HD version of the Firefly episodes were posted in the 360 newsgroup 5-6 days ago. There's 25-30mb sample video of each episode if you want to see if it's worth grabbing them.
User avatar
Mr Bacon
Way too much time!
Way too much time!
Posts: 2108
Joined: September 27, 2002, 4:57 pm
Location: Down the street
Contact:

Re: Newsleecher tutorial

Post by Mr Bacon »

What the hell is out there to download 1.5-2+ tb a month? IT'S NOT PORN IS IT?

Do you actually view half the stuff you download? I don't watch more than a gig or two of content a month and I have plenty of time on my hands ... what gives? (You need to find a new mmo)
miir and I are best friends. <3
User avatar
Waikiki
Gets Around
Gets Around
Posts: 133
Joined: January 20, 2003, 10:56 pm
XBL Gamertag: FeistyPea
PSN ID: clmolnar
Location: Boston, MA

Re: Newsleecher tutorial

Post by Waikiki »

Winnow wrote: BTW, you can see in the image, HD version of the Firefly episodes were posted in the 360 newsgroup 5-6 days ago. There's 25-30mb sample video of each episode if you want to see if it's worth grabbing them.
You can buy the entire series for $20 now, don't download it!

Support Firefly and hope for it's return!
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Waikiki wrote:
Winnow wrote: BTW, you can see in the image, HD version of the Firefly episodes were posted in the 360 newsgroup 5-6 days ago. There's 25-30mb sample video of each episode if you want to see if it's worth grabbing them.
You can buy the entire series for $20 now, don't download it!

Support Firefly and hope for it's return!
You can buy the entire Firefly series in HD for $20.00? I have the Widescreen Firefly series. Check the samples and see if there's a difference
Mr Bacon wrote:What the hell is out there to download 1.5-2+ tb a month? IT'S NOT PORN IS IT?

Do you actually view half the stuff you download? I don't watch more than a gig or two of content a month and I have plenty of time on my hands ... what gives? (You need to find a new mmo)
That wasn't my download stats. I'm usually somewhere in the 60-100GB/month range with the largest chunk coming from 7.05GB Xbox 360 images followed by ~700MB and 1.4GB Divx/Xvid movies.

(this is the first time I used the "quote" button to add a second quote to a post from the topic review. Nice!)
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

A new beta version of Newsleecher 3.9 came out with mostly enchancements to the repair and extract feature.

Newsleecher 4 is coming soon. Here's an update from Spiral (NL programmer)
Jul 28, 2007
Here's a quick update on some of the topics we will look into in the, not so far, NewsLeecher/SuperSearch future.

SuperSearch Retention
The SuperSearch Index Retention is currently at around 140 days, but to keep up with the ever increasing retention offered by usenet providers, we will soon add another 30-40 days of retention to the index. Maybe a bit more.

Later on, when we have the possibility to compile the SuperSearch engine for a 64bit environment, we'll be able to bump the retention to 1000+ days if needed.

NewsLeecher V4
Even though NewsLeecher v3.9 hasn't been released as final yet, we've already begun working on NewsLeecher v4. When NewsLeecher V4 is final, it will implement fullblown usenet text reader support, including advanced filtering, threaded article view, advanced usenet text posting system, etc.. Lots of enhancements to the 'binary' part of NewsLeecher will be added aswell of course.

SuperSearch Comment System
This is not certain yet. But we're looking into making it possible to attach searchable comments to the article stored in the SuperSearch index. One of the advantages of a comment system, is that it will be much easier to locate articles with odd, non-descriptive subject lines. A team of editors will be in charge of the comment system, if we decide to implement it.

Header Retrieval Compression
We'll soon contact some of the major usenet server developers, in an effort to make and, natively incorporate, a freely available specification for retrieving compressed article headers from usenet servers. Currently headers are sent to usenet clients as raw, non-compressed data, but by implementing article header data compression, it will be possible to increase the header retrieval speed by several hundred percent, and save bandwidth as the same time! Our goal is to come up with a solution which can be implemented natively in any application, developed by any software author who'd like to support it.

Much More
Besides the topics mentioned above, we will of course continue to implement new, and optimize current, parts of NewsLeecher and SuperSearch aswell.
I'm not sure who still reads text newsgroups (doesn't just about every topic known to man have a phpBB type bulletin board forum on the net now?) but there you go! Newsleecher 4's text features will by default be better than Agents because Agent is programmed by sloths and Spiral can do in a week what Agent programmers muster up in a year or two. He actively works with usenet server developers to increase speeds etc.
User avatar
Fash
Way too much time!
Way too much time!
Posts: 4147
Joined: July 10, 2002, 2:26 am
Gender: Male
XBL Gamertag: sylblaydis
Location: A Secure Location

Re: Newsleecher tutorial

Post by Fash »

Winnow wrote:
Spiral wrote:We'll soon contact some of the major usenet server developers, in an effort to ... natively incorporate ... retrieving compressed article headers from usenet servers.
He actively works with usenet server developers to increase speeds etc.
and I work with my state representatives to fight government spending. :lol:



<3 newsleecher
Fash

--
Naivety is dangerous.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

pr0n wins the day!

This change may help speed up searches for peeps. I think it's a good idea to separate adult material from the rest as it does much up searches sometimes.

I like that Spiral is going to add some sub filters though as it's still nice to search by media format:
Aug 14, 2007
Due to the vast number of erotic/adult related usenet groups indexed by SuperSearch, we've changed the SuperSearch categories a bit.

SuperSearch now has two main categories: "General" and "Erotica". Basically all erotic related articles indexed by SuperSearch, will be put into the "Erotica" category, and the rest will be put into the "General" category.

The new "Erotica" category replaces/merges the old "Pictures" category, so it will be possible to search the new category from within current releases of NewsLeecher, by selecting the "Pictures" SuperSearch category. Future releases of NewsLeecher will of course reflect the changes to the SuperSearch categories.

The new category has been introduced, because we've noticed that nearly half of all groups indexed by SuperSearch are erotica/adult related(!). So it definitely seems like there is a need for such a category. We are planning on introducing various SuperSearch filters in the future, which will make it easier to find specific article types, such as mp3 files, pictures, etc., within the two main categories.

Notice that the category changes only apply to articles indexed by SuperSearch from now on. Articles which were indexed before todays date, will still belong to the categories where they were originally indexed.
User avatar
Xatrei
Way too much time!
Way too much time!
Posts: 2104
Joined: July 22, 2002, 4:28 pm
Gender: Male
Location: Boringham, AL

Re: Newsleecher tutorial

Post by Xatrei »

I don't like the fact that they've merged music searches into general, and I'm getting annoyed with the alert box telling me about it after every search. Hopefully newsleecher will be updated to include the supersearch changes soon so that the annoying pop-up alerts stop.
"When I was a kid, my father told me, 'Never hit anyone in anger, unless you're absolutely sure you can get away with it.'" - Russel Ziskey
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Xatrei wrote:I don't like the fact that they've merged music searches into general, and I'm getting annoyed with the alert box telling me about it after every search. Hopefully newsleecher will be updated to include the supersearch changes soon so that the annoying pop-up alerts stop.
yeah, I don't get the music move back to general but if he adds format filters before switching it should be fine again. i do kind of like having the option to search for something and find all formates associated with it though. It's the same argument I made with Copernic...even though you search for music, it's nice to know there are some music videos out there, maybe some still images, a pdf book about the artist, etc. It's all good as long as the subfilters are well done in the next release.
User avatar
Aslanna
Super Poster!
Super Poster!
Posts: 12417
Joined: July 3, 2002, 12:57 pm

Re: Newsleecher tutorial

Post by Aslanna »

I didn't like the music split to begin with because you had to do two searches. The regular mp3 groups seemed to be under general and the lossless ones under audio. Or maybe the other way around. All I know is I'd search one category and get certain results and then the audio category and get completely different results.
Have You Hugged An Iksar Today?

--
User avatar
Xatrei
Way too much time!
Way too much time!
Posts: 2104
Joined: July 22, 2002, 4:28 pm
Gender: Male
Location: Boringham, AL

Re: Newsleecher tutorial

Post by Xatrei »

I guess to clarify, I don't like the split without having the format filters in place on NL. Once they're in place, it will definitely be better not having to look in both search categories, as I do now. I always got both mp3 and lossless flac files in the music search, but I did notice that there seemed to be a ton of flac files that were only indexed under the general search.
"When I was a kid, my father told me, 'Never hit anyone in anger, unless you're absolutely sure you can get away with it.'" - Russel Ziskey
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Aslanna wrote:I didn't like the music split to begin with because you had to do two searches. The regular mp3 groups seemed to be under general and the lossless ones under audio. Or maybe the other way around. All I know is I'd search one category and get certain results and then the audio category and get completely different results.
Yeah, that too. I sort of alluded to that point when I made the post above about discographies. Those are typically always rar'd so would end up on in the general category. Also, because Spiral split things up based on the group name, it had nothing to do with the extensions so files could still be anywhere.

Good filtering will do the trick. I love the file size filters and like the more advanced typed in searches by poster, etc.

Filtering is key even when you download using normal groups instead of Supersearch. The dcp comics forum nicely labels their posts by date buy post like 5 extra par files per comic which I cont' want so filtering by "cb" gets me all cbr/cbz files without all the extra crap.

One thing I notice is that pictures get neglected. I like to use supersearch to find particular drawings by artists but Spiral tends to not index all jpegs due to the large quantities. I just entered some basic searches to test it. searched for "california" and "birds" and the oldest date I could find under the picture category is 13-15 days. Doing a "california" search in general finds jpegs dating back 90 days but when you look at what category they're from it's non picture newsgroups. So, when Spiral boasts about going back 200 days, I'd prefer ~120 and include pictures. I can see why he doesnt though as the spam for jpegs is unbelievable. On top of that, people will spam 40000 images in a newsgroup because they have issues with the group and try to spam the group out of existence which doesn't really matter when you have a good filtering app like Newsleecher but it's probably hell on his indexing.

Newsleecher/Supersearch rock...nice to have an active developer though that reads his forums!
User avatar
Xatrei
Way too much time!
Way too much time!
Posts: 2104
Joined: July 22, 2002, 4:28 pm
Gender: Male
Location: Boringham, AL

Re: Newsleecher tutorial

Post by Xatrei »

3.9 Beta 7 was released today. I just downloaded it, but haven't played around with the new search filters too much yet.
3.9 Beta 7 - Released 21 Aug 2007

---------------------------------

#: In preparation for the NewsLeecher Version 4 release, there're

lots of major internal changes in this beta (still invisible

to the end-user). Lots of changes mean lots of new

possibilities for bugs, so it's recommended that you stay with

the NewsLeecher Final Release if you don't want to experiment.

+: *BIG ADDITION*: NewsLeecher now implements a new, and much

improved header fetching cache, which is used when fetching

headers from usenet groups. In earlier versions, NewsLeecher

wrote to the disk *extensively* while fetching headers. But

due to the new cache, the disk writes are reduced by several

hundred percent, meaning much lower resource usage and much

higher responsiveness while fetching group headers. Especially

when a fast broadband connection is used.

+: *BIG ADDITION*: The SuperSearch 'Audio' category has been

merged with the 'General' catetegory, and the 'Picture'

category has been replaced by a new 'Erotica' search category,

due to the vast amount of erotic related groups on usenet.

Furthermore, a new SuperSearch subfilter system, which makes it

easy to find specific articles types (such as audio related

articles), has been added.

+: The SuperSearch 'Result Filter' now supports boolean searching.

*: Lots of internal changes have been made to NewsLeecher and

SuperSearch in preparation for the upcoming NewsLeecher v4

version.

!: Fixed a serious bug, introduced in the previous beta, where

NewsLeecher would consume all system memory under certain

circumstances:

http://www.newsleecher.com/forum/viewtopic.php?t=14279

!: Smaller bugfixed here and there.
"When I was a kid, my father told me, 'Never hit anyone in anger, unless you're absolutely sure you can get away with it.'" - Russel Ziskey
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Good results with the new beta so far. The sub filters work as they should although I like using the "all" option as much as I can. I found an artist's entire collection rar'd while searching in "All" which I would have missed with a pure mp3 (audio) search.

As for the separate erotica search, I don't entirely trust which groups he decided to put in there. alt.binaries.amp contains all sorts of photos and artwork but there's porn mixed in. I'm going to have to search erotica as well when dealing with images to be sure I don't miss non porn pics I'm looking for.

I wonder if he has a list of which groups are associated with the erotica tab.

This release seems snappier than the last one I was using. I was beta or two behind the latest release before this. He's got the full 200 day drop down option for searches now.

Overall, a nice improvement.
User avatar
Xanupox
Almost 1337
Almost 1337
Posts: 518
Joined: July 5, 2002, 2:15 pm
Gender: Male
PSN ID: TheRealScarr
Contact:

Re: Newsleecher tutorial

Post by Xanupox »

I probably gave you virtual items once upon a time...
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

That's better than torrents I guess. I'll take Supersearch over that any day though for finding things on the newsgroups. Don't know why you'd want to limit yourself to nzb tagged posts.

I use Pirate Bay and Release log (http://www.rlslog.net/) if I want to see what the latest (popular) postings to the newsgroups are and then grab them fast with Supersearch. With Pirate Bay, just use the browse by category option: http://thepiratebay.org/browse and you'll see the latest offerings and then go use the much better newsgroups (for your hard drive and time) to grab the files.

That site would have some use if you can access it for free as one source to see what's out there but there are free alternatives for that.

You can also use places like this if you're just bored and want to see what's been posted recently:

http://www.alt-bin.com/index.php?pg=fd& ... ovies.divx (sortable by any column)
User avatar
Siji
Way too much time!
Way too much time!
Posts: 4040
Joined: November 11, 2002, 5:58 pm
Gender: Male
XBL Gamertag: mAcK 624
PSN ID: mAcK_624
Wii Friend Code: 7304853446448491
Location: Tampa Bay, FL
Contact:

Re: Newsleecher tutorial

Post by Siji »

Winnow wrote:I wonder if he has a list of which groups are associated with the erotica tab.
I'll admit I haven't paid attention to how supersearch works, but wouldn't it be an awesome feature if you could set up the groups included in a supersearch filter manually?

Would still be nice if they added filter options like Agent has.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Siji wrote:
Winnow wrote:I wonder if he has a list of which groups are associated with the erotica tab.
I'll admit I haven't paid attention to how supersearch works, but wouldn't it be an awesome feature if you could set up the groups included in a supersearch filter manually?

Would still be nice if they added filter options like Agent has.
You can limit the search to groups with a certain name in them like this:
Q: What if I want to search certain group?

A: You can use the group: option for this.

Star Wars group:pictures
I suppose you could also further limit it to a single group with:

star wars group:alt.binaries.amp

You might suggest that on the Newsleecher forums. I don't think there'd be an easy way to request that on the initial search but the returning data could be filtered pretty easy on the client side if Spiral set it up.

Having used this new setup for awhile now, I haven't found anything too out of place. It's mostly alt.binaries.amp which has a combination of adult and non adult images which causes me to search twice when looking for images but all that really takes is clicking one button and searching again.

As for filtering, I noticed that if you select the minimum or maximum size, the search is filtered before providing the results...which is important for searches that would normally return more than the 5000 max hits if they weren't filtered beforehand.

Some of the advanced search features of Supersearch:
Input: mercedes
Output: All subjects with the word "mercedes" in it.

Input: mercedes slk
Output: All subjects with the word "mercedes" AND the word "slk" in it. The new thing here, is that it's not an "exact string" search, so subjects like "slk model - mercedes.jpg" will also hit on this searchstring.

Input: mercedes -slk
Output: All subjects with the word "mercedes", *not* containing the word "slk".

Input: "mercedes slk"
Output: The quotes mean exact string matching. Just like the old SuperSearch did it.

---

True wildcard searching:

SuperSearch uses true wilcard for the search terms. If you search for ikin abe, SuperSearch will perform a search for *ikin* *abe*, and will match "Babe in Bikini", "Bikini Babe", "AbeInBikini", etc.

---

Q: What if I want to search certain group?

A: You can use the group: option for this.

* Like: Mercedes group:pictures

This will search for articles with mercedes in the subject line in all groups with the word pictures in the group name.

Note: This will not override the category, so searching for pictures in the general category will not give you results from the picture category.

--------

Q: What if I want to search for posts from a certain poster?

A: You can use the poster: option for this.

* Like: Mercedes poster:mercfan

This will search for articles with mercedes in the subject line posted by someone with mercfan in their name.

-------

Q: How can I search a specific range of days?

A: To set the minimum article age for returned articles, you can now use the new "MinDays" property. The max age can be set using the new "MaxDays" property, or by using the usual days dropdown menu in NewsLeecher.

Example Search:
Mercedes minDays:10 maxDays:40
The above example will return all articles posted between 10 and 40 days ago, matching the *mercedes* keyword.

--------

Q: How can I search a specific file size range?

A: You can use the minKB: and maxKB: properties. This will search for files of specific sizes in kilobytes

Example Search:
Mercedes minKB:1024 maxKB:12000
User avatar
Aslanna
Super Poster!
Super Poster!
Posts: 12417
Joined: July 3, 2002, 12:57 pm

Re: Newsleecher tutorial

Post by Aslanna »

No matter how you slice it the filter features of Agent are far superior to Newsleecher. Which is understandable considering it's main purpose (reading news not just binary retrieval)and how long it's been around.
Have You Hugged An Iksar Today?

--
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Aslanna wrote:No matter how you slice it the filter features of Agent are far superior to Newsleecher. Which is understandable considering it's main purpose (reading news not just binary retrieval)and how long it's been around.
They might be better for text but the filters in Newsleecher work well for binaries.

Image

I use the secondary filter field ("result search" field above) quite often to narrow search results. The results search field is client side though and won't prevent a 5000+ limit return while the poster, group, min/max days/size fields will filter before returning results.

Also, don't be afraid to use the min/max days to limit search results! If you're getting 5000+ max warnings, try searching in chunks:

Max Days: 200 , Min Days 160
Max Days: 159, Min Days 120

It's nice to be able to limit the search to a specific time frame and not just limit if from the present day to a max day.

Supersearch will only allow you to select 1000 results at one time for download but that's easy to over come with multiple selections and the "leech specify" option to make sure they all save to the same subfolder.

---

When Newsleecher 4 is released, I'm guessing Spiral will include filters to search within the bodies of posts as well, etc.
User avatar
Xatrei
Way too much time!
Way too much time!
Posts: 2104
Joined: July 22, 2002, 4:28 pm
Gender: Male
Location: Boringham, AL

Re: Newsleecher tutorial

Post by Xatrei »

Allowing regular expressions in the NL filters would be supremely cool.
"When I was a kid, my father told me, 'Never hit anyone in anger, unless you're absolutely sure you can get away with it.'" - Russel Ziskey
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Xatrei wrote:Allowing regular expressions in the NL filters would be supremely cool.
Spiral's saving that for the linux version! :twisted:

We're talking levels of uberness now. Supersearch is already the best. That it is actively developed and continues to improve is even better.

I'd encourage anyone that wants a feature to actually make a post and explain why it would be useful on Newsleecher's forum.

Same with Faststone's Image viewer although with that, you need to email the developer and he'll email you back and let you know how he feels about your suggestion.
User avatar
Winnow
Super Poster!
Super Poster!
Posts: 27586
Joined: July 5, 2002, 1:56 pm
Location: A Special Place in Hell

Re: Newsleecher tutorial

Post by Winnow »

Looks like we're in no danger of people discovering the usenet.

A recent poll:

Image

Thank god for dumb people.
User avatar
Ashur
Way too much time!
Way too much time!
Posts: 2604
Joined: May 14, 2003, 11:09 am
Location: Columbus OH
Contact:

Re: Newsleecher tutorial

Post by Ashur »

I'm amazed torrents are so popular. My wife couldn't figure out Bittorrent and I consider her the "average" (AKA not overly Internet-saavy) computer user. She uses Google, iTunes, Email and needs protection from herself for malicious links. She only ever used Usenet when I set up Agent for her years ago so she could download MP3s (although the whole concept of Usenet was alien to her, she didn't understand where these files actually came from).

Where was that poll conducted at? I imagine at a site that caters to people who regularly download from torrents.
- Ash
User avatar
Aslanna
Super Poster!
Super Poster!
Posts: 12417
Joined: July 3, 2002, 12:57 pm

Re: Newsleecher tutorial

Post by Aslanna »

Winnow wrote:Looks like we're in no danger of people discovering the usenet.

A recent poll:

Image

Thank god for dumb people.
If you disregard things like Rapidshare, which I do since it's not really 'mass appeal', than Usenet is ranked #2. Hardly what I would consider being well concealed. Even if you do consider those than that's still over 10%. Besides which the poll doesn't give any indication that the people that voted for something else had no idea what usenet was. They just preferred to download a different way.
Have You Hugged An Iksar Today?

--
Fairweather Pure
Super Poster!
Super Poster!
Posts: 8509
Joined: July 3, 2002, 1:06 pm
XBL Gamertag: SillyEskimo

Re: Newsleecher tutorial

Post by Fairweather Pure »

Well, I'm a comcast subscriber and tried to put in thier server settings, but it's not taking. I don't know where to go in order to get the info on more news servers, so I guess I'm at an impass. I simply cannot connect to a server to try it out.

EDIT: Comcast also has a 2G monthly limit.
User avatar
Aslanna
Super Poster!
Super Poster!
Posts: 12417
Joined: July 3, 2002, 12:57 pm

Re: Newsleecher tutorial

Post by Aslanna »

Most people go with a third party news provider like giganews, supernews, etc. ISPs typically don't carry many of the binary groups. And if they do the retention and completion is usually pretty lousy.
Have You Hugged An Iksar Today?

--
User avatar
Fash
Way too much time!
Way too much time!
Posts: 4147
Joined: July 10, 2002, 2:26 am
Gender: Male
XBL Gamertag: sylblaydis
Location: A Secure Location

Re: Newsleecher tutorial

Post by Fash »

I'm always amused when someone posts something that flies in the face of their argument.

Usenet is quite popular... There are other reasons why it has not come under much fire from the RIAA or other agencies. They are unable to acquire the identity or IP addresses of downloaders, without serious government interference which hasn't happened. It's a lot easier to grab an IP from a P2P client or attach to a torrent and log all the other IP's on it, than it would be to compile massive lists of Usenet article ID's and then sue providers to expose their members.

Not saying it won't change in the future, but for now, Usenet is the last remaining paradise.

Fair, try giganews.com I think they have a free trial. Newsleecher.com for the best software (their Supersearch is gods gift to Usenet)
Fash

--
Naivety is dangerous.
Post Reply