Withdrawing money from your retirement account to buy a house has several drawbacks. By Kimberly Lankford, Contributing Editor May 12, 2008 We withdrew money from my husband's 401(k) plan so that we could come up with a down payment for a home. Now we're getting hit with the taxes on that money. Are there any exclusions of taxes on that income when the money is used for the purchase of a primary residence?Sorry, but no. Even though you may be allowed to take a hardship withdrawal from a 401(k) to pay for a down payment on your main home, it isn't a good idea -- as you've discovered. You have to pay taxes on the money at your top income-tax rate, plus a 10% early-withdrawal penalty if you're under age 59½. You can only withdraw your contributions and not any earnings or employer match, and you generally have to prove that you don't have another source for the cash. You could have avoided the penalties and taxes -- at least for a while -- by borrowing the money from your 401(k) instead of withdrawing it. The specific rules vary by plan, but you may be able to borrow up to 50% of your vested account balance, with a maximum of $50,000. Advertisement You'll generally pay interest at the prime rate plus one or two percentage points, which goes back into your account. But there's a big downside to 401(k) loans: If you lose your job before age 55, you usually have to pay the loan back immediately, or get stuck with the same taxes and penalties that you'd have for withdrawals. The rules aren't as harsh for IRAs. You can take up to $10,000 from a traditional IRA penalty-free to buy a first home (you can qualify as long as you haven't owned a house in the past two years). However, you still will owe taxes on the withdrawal. A Roth is even more flexible: You can withdraw contributions from a Roth IRA for any reason without taxes or penalties, and you can withdraw up to $10,000 in earnings for a first-home purchase tax- and penalty-free if you've had a Roth for at least five years (you'll owe taxes, but not the 10% penalty, if you haven't had a Roth for five years). Even if you can access any of this money for a down payment, it's generally better to search for other sources of cash -- and keep the funds growing in the account for your retirement. For an update on down payment requirements and assistance, see An Open Door for First-Time Home Buyers. For more information about buying a home, see the Home Buyer's Survival Kit. For more information about retirement-plan rules, see Why You Need a Roth IRA and Max Out Your 401(k). Got a question? Ask Kim at email@example.com.