That's a good question and we understand your confusion, since the days when Las Vegas was an evening-dress destination are long gone and today, appropriate attire can range from your best tiara to a wet suit, depending on the show and which ticket option you choose.
To answer your question, there are a number of factors to consider, including the type of show and your own personal preferences. When it comes to the more casual entertainment options, like casino comedy clubs, the room tends to be more casual (table seating rather than a theater setting) and pretty much anything goes, both off stage and on (many of the performers are far from dressy and the whole ambiance is pretty informal).
Smaller shows, like those in V Theater in the Miracle Mile Shops, or at off-Strip properties like Hooters or Palace Station, or downtown, for example, also tend to be more casual, since the theaters are smaller and the shows are often more casual in presentation. Some shows, including Evil Dead The Musical (Planet Hollywood), Blue Man Group (Monte Carlo), and Le Reve (Wynn) actually encourage casual attire for audience members seated in some seats, due to the interactive nature of the show and the chance of certain guests getting showered in water or splattered with fake blood. Le Reve, in fact, offers an option called the "Divers Dream" package for scuba enthusiasts, who may enjoy the show from the vantage point of the dive pool/"lake" (no, we weren't kidding about the wet suits). If any special dress is recommended for your seats, you'll be made aware of this at the time of booking and will have the option to select safe seating if you don't desire that degree of participation.)
The experience of filming the Ocean's Eleven remake evidently got some of the principals nostalgic for the Rat Pack days, with rumors that George Clooney and Brad Pitt were entertaining the idea of opening their own Las Vegas casino, with a strictly enforced dress code. That idea came to naught, but while it's no longer de rigueur to don pearls or tux, some shows do attempt to keep it classy with some kind of minimal dress code. The official word from Wynn is that a property-wide standard is requested: "While jackets and ties are not required, we suggest casual evening attire. Tee Shirts, tank tops, shorts, and athletic shoes are not appropriate."
How rigorously such policies are enforced, however, is a different matter entirely, of course: We read of one couple who checked before they saw Le Reve and were concerned because they only had sneakers to wear, when the dress code stipulated trousers and shoes for men. "We needn't have been [worried]," they explained, "as there were folk there in shorts and flip-flops."
While Penn and Teller themselves dress in suits for their show, Penn's still basically a huge long-haired thug, regardless of what he's wearing, and the show is pretty anarchic, with a different crowd than you'll encounter at David Copperfield, for example, or Donny & Marie. Hence, the attire of the audience will likely be more casual at P&T.
In general, it's pretty much a matter of common sense and of what makes you feel comfortable. For many visitors, going to a show in Las Vegas is a special occasion and getting dressed up is part of what makes the experience a break from the norm. Even if it's not a big deal for you, or if you're being dragged along to see Celine or Sir Elton against your will, we'd urge everyone to have consideration both for the performers and for the other audience members and choose a level of attire that's respectful and won't burst someone else's bubble: If you've made an effort to get dressed up and you're sitting next to stranger who's wearing a ripped-up pair of cut-off jeans and a wife beater, it tends to kill the mood and can make you feel overdressed, which can be just as uncomfortable, or even more so, than being too casually attired.
Take showroom size and ticket price (both for the show in general, and for the seating tier you're in) as a good indicator of what's appropriate and if in doubt, check with the box office or the show's official website, which will indicate if there is a desired code of dress.
We'd like to end with a delightfully tongue-in-cheek response we found on an amazon.com forum, in reply to a query about appropriate attire for a Cirque du Soleil show:
"Tiaras are de rigeur. For the ladies, feather boas, sequined gowns, harlequin patterns, and dominoes (half-masks you hold to your face with a little stick) are all appropriate. For the men, dramatic capes, riding boots, jester's hats, and ballet tights make a nice effect.
"Makeup should be heavy and dramatic (for men and women). Eye shadow should extend to the eyebrows. If possible, glue spangles to your eyelids.
"For accessories, look at albino boa constrictors, ostriches on jeweled leashes, and golden spheres you are juggling (when you aren't juggling them, put them away. Don't just hold the golden spheres in your hands like a dodo.)
"If possible, perform daring feats of strength and agility as you enter."