There are a few things that have to happen for this "regression" to show up. First, you have to be working in .net 2.0 WinForms. Second, you must be using a custom application context. Third and most importantly, you must have at least one form open in your application that handles its closing event and sets CancelEventArgs.Cancel = true.
What's the behavior? While operating under a custom ApplicationContext, what would be my Main form is closed by the user. Since the application context is not associated with that form, I decide to call "Application.Exit()" in an override of OnClosing. My Main form closes, but any hidden forms stick around, thus leaving the application running until you open up task manager (or something similar) and manually kill the process.
Why does it happen? WinForms 2.0 introduced an overload to Application.Exit that accepts a CancelEventArgs parameter. This is what is called by Application.Exit(). If even one of your open forms doesn't think it needs to close itself for whatever reason and sets its CancelEventArgs to true, Application.Exit stops processing and returns control to the application.
Why did I call it a "regression"? This behavior has changed between .net 1.1 and 2.0. In 1.1, if I call Application.Exit, by golly the application exits! I understand the reasoning behind the change. I just wish that there was some indication to it other than Reflector. A comment for intellisense perhaps? Something.
Will it matter to most WinForm projects? Probably not. If you use the default application context (by calling Application.Run(new MainForm()), which is what VS.NET spits out for you from the template) you do not have the same problem. Closing the main form exits the application as expected. If you're using a custom ApplicationContext, tho, watch out.