Snow camping? Why the heck am I talking about snow camping at an ultralight backpacking site? Two good reasons, actually. First off, weight is still something that can be safely pared down for a winter backcountry trip (though the safety margin needs to be MUCH wider than a typical 3-season trip). By paring down that unneeded weight, you can enjoy the same benefits that 3-season ultralight adventuring can bring you. Second, odds are if you're interested in ultralight backpacking, you probably have a good number of miles/days under your belt and may well be ready for exploring another dimension of the backcountry experience.
Snow camping brings on a whole new set of challenges and rewards. First and foremost is extra attention to safety. Under other seasonal conditions, you can hike out of nearly any bad weather situation if you have to. When a sudden winter storm hits, however, you may well be stuck for awhile. A long while. Under extreme conditions. Forgetting some gear on a summer hike can mean a wet, cold night. Forgetting gear on a winter backcountry trip can mean a quick end to your life. Or a long, drawn-out one....
I don't recommend anyone try a winter backcountry adventure unless they, 1) Are already skilled at "fair weather" backcountry travel, and 2) Have a good amount of winter weather experience (via nordic skiing, snowshoeing, snow camping in the backyard, etc.). Come to think of it, snow camping in the backyard is an excellent way to get a feel for what snow camping can be like. Also, seek training in winter safety, such as how to recognize and avoid avalanche conditions. They are a very serious threat that most folks simply don't take seriously. At least until they find themselves entombed in an ice-cement hell. For more information, see "Why on earth would I worry about avalanches?".
So why on earth would anyone bother? The total solitude, incredible beauty and deafening silence that you can only find in a snowy wilderness. No logging in the distance, no trails clogged with fellow hikers, no crowds at a prime backcountry campsite, even at the most popular national parks. (Though I'm told the snowmobilers at Yellowstone have overrun a few spots....)
Until you've experienced a perfect sunlit morning overlooking a snowy meadow, the mist clinging above a small creek while a gentle breeze fills the air with diamond-dust sparkles of dry snow, you simply can't imagine how incredible it can be. Until you've weathered a major winter storm on a lonely mountainside, snug and warm the entire time, then dug out the next morning to find several feet of fresh powder for you to play in, you may not fully appreciate how great simply being alive really is.
One thing you may want to give special consideration to - bringing a GPS receiver in addition to a compass. Orienteering in the snowy woods can challenge even the seasoned map-and-compass freak. Visibility can drop to nearly nothing without much warning, and heavy snows mask terrain features that may otherwise have shown you the way in a tricky spot. I still use my orienteering compass, but for snow camping I'm very close to buying my first GPS unit ever as a backup.
Hopefully this intro, combined with the links below, will get you started in the right direction. Also, see if you can find any trail huts in your area. These are permanent shelters that in some cases link together to allow you to spend an entire week or more in the snowy backcountry as you tour from one to the next. Some nordic centers are also providing snowshoe trails as well. That way you can visit one and try both nordic (cross-country) skiing as well as snowshoeing to see which one suits your needs best. I like both myself. Much cheaper than downhill skiing too!