When you fail to file your tax return on time or pay your tax due, not only will you face failure-to-file and failure-to-pay penalties, but interest charges as well.
The IRS calculates interest based on the amount of unpaid tax you owe. Currently the interest rate for underpayment stands at 4% per year. The IRS charges interest for every day the tax goes unpaid.
Note that the interest rates on unpaid tax are variable. They change every financial quarter and are the equivalent of the federal short-term rate plus 3%.
Furthermore, while penalties may be abated by the IRS, depending on your circumstances, interest charges are almost never abated and will continue to accrue until all the owed tax, penalties, and interest are fully paid.