Well You were asking about starting one species...
I always start my plants indoors early with
these in Jan/Feb with bottom heat and grow lights. These systems are great if you want to get a jump on the season. The white inner trays are modular you can get various sizes based on the number of plants you want to start and buy the sponges from Parks and other suppliers. Parks also sells grow light racks and heat mats etc etc. You can build your own racks with fluorescent fixtures but Parks sells rack systems with multiple trays/lights and heating pads which fit perfectly with their seed starter systems. The lighting height is easily adjustable as the seedlings grow. Bottom heat is key to seed germination so I would look into heat mats for sure. The sponge systems allow easy transplant with no shock.
Peat pots are crappy unless you are working on an industrial scale with a timed mist system plus they tend not to decay well with the low water on peppers. Jiffy Pellet systems are decent but cheaply made and fall apart in a year if you use them be sure and remove the polyester wrapper when transplanting the roots tend to bind if you don't.
The best general source for veggie/herb seeds is probably
http://www.burpee.com you might also try some heirloom tomatoes from
http://www.tomatobob.com/ tomatoes require a bit more attention and work but they are pretty damned easy and you should be able to grow enough to keep your self supplied during the growing season.
Also if you want guaranteed variety characteristics...buy seed every year (or actually I usually use a packet I buy for two years (after two years the germination rates drop off considerably). Capsicum and Tomatoes both cross-pollinate so saved seed can produce odd hybrids (which can be good or bad). You can always hand pollinate some flowers and cover them til they set fruit to guarantee variety or even cross pollinate and play Mendel.
Cleaning and saving tomato seeds is about the same as peppers. Small tupperware container, a cup of water but you squeeze the fruit into the water then add a Tbsp sugar and let it ferment about 4 weeks then skim the fermented gunk off the top and pitch any floating seeds. Then rinse and dry. Heirloom tomatoes will make you think twice about every buying those mushy wet things that pass for tomatoes from stores ever again.
And read up on tomato pruning (not necessary but desirable for some varieties).
Oh and always label seed trays...you think you will remember...but you'll always screw it up somehow...

. And start a least 6 of each variety...you can always find someone who will appreciate any surplus babies.
And finally...don't try growing fruits/veggies from store bought fruit or store bought hybrid plants. A lot of these are polyploid and infertile or if fertile the seeds will not produce the plant the fruit came from.