The only thing worse than willingly to go to Confession and intentionally withhold confessing a grave sin, is choosing not to go to Confession at all (the Church has a minimum requirement of once a year). Apart from those things, there are only a few others that can actually invalidate a Sacramental Confession. Here is a list:
1) If the Priest does not give an absolution "God the Father of mercies...I absolve you from your sins..." This has happened more than once. If the Priest does not do so, you are right to ask, "Father, may I please have the absolution?" If he refuses, then it is invalid. If he does a sloppy job of it, then you can probably consider it invalid and go to another Priest and start over.
2) If you are not truly penitent. You know your own heart, and if you do not really intend to stop the sin (i.e. "I used artificial contraception and plan on continuing to use it and am going to make no effort to stop") then even if the Priest does give an absolution it is not valid. The entire Sacrament is dependent on the repentant heart of the person confessing. This means you must intend not to return to the sin (even though you might fail in this intent) and you must intend to seek greater holiness in your life (even though you might fail in this as well).
3) To refuse to perform your penance. Assuming you understood it and it was something that was possible (I have heard of penances that were irrational!), then you must do it. If you forgot and then remembered later, do it as soon as possible, and then mention it when you go to Confession the next time.
There are also things that are problematic, and would make the Confession illicit, but not actually make it invalid. Those include the following: if someone other than the Priest overhears your Confession (in other words, do not yell in the confessional); if the Priest gets confused about what you are confessing (it is not dependent on the brilliance of the Priest, and sometimes people are not clear in their confessions); if you forgot to perform your penance (it is likely only venial if you forgot, but if it was intentional, that would make it invalid); or forgot something you intended to confess (just make sure to go to Confession as soon as possible and confess it).
I cannot say this enough: Go To Confession.