Get a decent microphone
You need a good microphone. There is no argument about that. It doesn't matter whether you're starting your own show or planning to be a guest on someone else's podcast. A great microphone boosts your voice and helps you achieve the depth and richness you hear on the radio and on popular shows like Radiolab . While you could record with your phone or your PC's webcam microphone, nobody wants to hear that every week.
Pro Tip: RTFM
You should actually read the instructions and make sure that you know what each dial and button does. Above all, make sure that you speak in the right direction! For most microphones, including the Blue Yeti, you want to target the label's side. Some models, especially dynamic microphones, have to be addressed from above. Yes, I know it all sounds easy, but I've met dozens of people who end up aiming at the wrong part of their microphones when they start.
It's also worth buying a few accessories to make your recordings sound great. Get a pop filter or foam cover to avoid explosives (the annoying titutal pop when you produce "p" tones). If you want to record regularly, it's worth investing in a table arm to keep your microphone in an optimal position (and also to avoid the extra noise you get from desktop stands).
You can of course start exploring more expensive microphone options, but I would recommend postponing them until you are more committed to podcasting life. The next big step in USB capacitors is the world of XLR microphones, the same interface that is used for professional audio devices. You will also need a USB audio interface such as the Tascam US-2×2 ($ 137) or the FocusRite Scarlett 2i2 ($ 160) to connect these microphones to your computer. From this point on, you can check options like the Rode Procaster ($ 228). It sounds noticeably richer than the Yeti, and since it's a dynamic microphone, it can also rule out unwanted noise better than a condenser.
Here is some advice: You can save a lot by buying all of these used or refurbished devices. I saved $ 100 for the superb Shure PG42 USB microphone by going through eBay.