Here's a quick answer: if you're having pain (wither constant or just occasional) and/or if you notice swelling (in the area of your wisdom teeth or you can see swelling outside your mouth), then the answer is almost always YES. Severe pain and swelling that is noticeable outside your mouth are dental emergencies and you should speak to your dentist as soon as possible.
But what if you rarely have any discomfort or if you have no discomfort at all? Maybe your dentist recommended you have them extracted as a preventative measure even though they're not bothering you. What should you do?
There are basically three scenarios for your wisdom teeth:
(1) Full impaction - the tooth is totally buried under your gums and bone
If your teeth are totally encased in gum and/or bone, and if there's no infection, pain, or other pathology, then you can probably leave them be. Your dentist will want to monitor them every now and then.
(2) Full eruption - the tooth is in your mouth like the rest of your teeth
If your wisdom teeth have erupted into your mouth as normal, and if there's no infection, pain, or other pathology, then you can probably leave them be. If can be hard to brush and floss all the way back there, so try to keep them clean to avoid problems. Your dentist will check on these teeth the same way they check up on the rest of them.
(3) Partial eruption - your tooth is stuck with some of it in the jaw and some of it exposed in your mouth
This is the scenario where you will probably want to have them out. Even if you currently don't have any infection, pain, or pathology, there is a good chance they will cause problems for you in the future. Every case is different, so talk with your dentist about your specific situation.
There are a few reasons why you'll want partially erupted wisdom teeth removed. There's probably some thin gum tissue partially covering the stuck tooth; this gum tissue (called an operculum) can become painful and swollen for various reasons. You may also get food and bacteria stuck down around the stuck tooth and it will be almost impossible to remove it. So stuck teeth can get cavities and gum disease more frequently. These teeth can also make it hard to clean your molars directly in front, which can leave to nasty cavities on the back side of your molars just because of the stuck wisdom tooth next door.
Most people have four wisdom teeth and you may have them all fully impacted, fully erupted, partially impacted, or some combination of all three scenarios. Your dentist or oral surgeon should be able to explain the specifics of your situation so you can make an informed decision about whether to take them out or leave them be.