feat: support for exception context propagation

[guibou]

This change is in order to have exception context propagation.

Everywhere we "re"-throw exception with throw, we instead use rethrow which does not remove the exception context. I've introduced rethrowIO' which is just a backward compatibility wrapper over rethrowIO and throwIO.

I will use this commit at work for a bit of time in order to gather some feedback and maybe comeback with a more robust solution.

Example of usage / changes:

The following code:

{-# LANGUAGE DeriveAnyClass #-}
import Control.Concurrent.Async
import Control.Exception
import Control.Exception.Context
import Control.Exception.Annotation
import Data.Typeable
import Data.Traversable
import GHC.Stack

data Ann = Ann String
  deriving (Show, ExceptionAnnotation)

asyncTask :: HasCallStack => IO ()
asyncTask = annotateIO (Ann "bonjour") $ do
  error "yoto"

asyncTask' :: HasCallStack => IO ()
asyncTask' = annotateIO (Ann "bonjour2") $ do
  error "yutu"

main = do
  -- withAsync asyncTask wait
  concurrently asyncTask asyncTask'
  -- race asyncTask asyncTask'

When run without this commit leads to:

ASyncException.hs: Uncaught exception ghc-internal:GHC.Internal.Exception.ErrorCall:

yoto

HasCallStack backtrace:
  throwIO, called at ./Control/Concurrent/Async/Internal.hs:630:24 in async-2.2.5-50rpfAJ7BEc1o5OswtTMUN:Control.Concurrent.Async.Internal

When run with this commit:

*** Exception: yoto

Ann "bonjour"
HasCallStack backtrace:
  error, called at /home/guillaume//ASyncException.hs:15:3 in async-2.2.5-inplace:Main
  asyncTask, called at /home/guillaume//ASyncException.hs:23:16 in async-2.2.5-inplace:Main

[mrkline]

Is there anything else that needs to be done while this is WIP? It would be excellent for exception annotations to propagate through.

[guibou]

@mrkline

Sorry, the only reason is that I had not took time to work on that.

I'm using this MR since month at work and did not had any surprising behavior.

Could be turned to "ready", if it builds with older GHC (I think so), I don't see any reason why not processing any further. Even if I missed some proper rethrow, it can be added in a future work.

[mrkline]

No worries, and thanks for the work!

I tried backporting this to base-4.20 so that it could also work with LTS-24. (mrkline@bb9d703) I hoped it would be simple since

rethrowIO e = throwIO (NoBacktrace e)

But sadly GHC 9.10 seems noisier - your example code gives

async-test: yoto
CallStack (from HasCallStack):
  error, called at Main.hs:15:3 in tacview-0.4.2.0-inplace-async-test:Main
  asyncTask, called at Main.hs:23:16 in tacview-0.4.2.0-inplace-async-test:Main
Ann "bonjour"
HasCallStack backtrace:
  collectBacktraces, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:169:13 in ghc-internal:GHC.Internal.Exception
  toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:204:5 in ghc-internal:GHC.Internal.Exception
  error, called at Main.hs:15:3 in tacview-0.4.2.0-inplace-async-test:Main
  asyncTask, called at Main.hs:23:16 in tacview-0.4.2.0-inplace-async-test:Main


HasCallStack backtrace:
  collectBacktraces, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:169:13 in ghc-internal:GHC.Internal.Exception
  toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/IO.hs:260:11 in ghc-internal:GHC.Internal.IO
  throwIO, called at libraries/ghc-internal/src/GHC/Internal/Control/Exception/Base.hs:195:43 in ghc-internal:GHC.Internal.Control.Exception.Base

If you spot any obvious problem, let me know.

[guibou]

No worries, and thanks for the work!

I tried backporting this to base-4.20 so that it could also work with LTS-24. (mrkline@bb9d703) I hoped it would be simple since

rethrowIO e = throwIO (NoBacktrace e)

My understanding of NoBacktrace is that it is useful at catch site, to remove the backtrace, but not at throw site.

As always with exception, I need to experiment to really understand what is going on (I have an infinite respect for people who can reason about exception in their head without experimenting ;)

If you spot any obvious problem, let me know.

I need to get back into this MR actually.

It seems that your branch mrkline@bb9d703 is based on one commit that is not what's inside this MR. Your commit is bb9d703, which is really alike the commit on my local repo, 5cbf92365b4501d9c102b810e8f6159b778ae615, they all are on branch exception-context, however this MR is the branch novainsilico:no_discard_exception_context.

I'm lost in the history there, sorry, it may be my fault (not worked on this MR since a long time and I'm used to try a lot of thing locally, change branch, force push, ...). Do you remember how you ended with commit bb9d703 and branch exception-context?

[guibou]

@mrkline

Using GHC 9.12:

Indeed, throwIO with NoBacktrace does not include backtrace.

λ gecko ~ → cat Ex.hs 
import Control.Exception

main = do
  throwIO (NoBacktrace (ErrorCall "test"))
  -- throwIO (ErrorCall "test")
λ gecko ~ → runhaskell Ex.hs 
Ex.hs: Uncaught exception ghc-internal:GHC.Internal.Exception.ErrorCall:

test

With no NoBacktrace, there is a backtrace:

λ gecko ~ → cat Ex.hs       
import Control.Exception

main = do
  -- throwIO (NoBacktrace (ErrorCall "test"))
  throwIO (ErrorCall "test")
λ gecko ~ → runhaskell Ex.hs
Ex.hs: Uncaught exception ghc-internal:GHC.Internal.Exception.ErrorCall:

test

HasCallStack backtrace:
  throwIO, called at Ex.hs:5:3 in main:Main

(So I'm wrong, throwing with NoBacktrace does indeed not include the backtrace).

However, throwing an exception caught with ExceptionWithContext leads to a duplication of the backtrace, if NoBacktrace is not there:

λ gecko ~ → cat Ex.hs       
import Control.Exception

main = do
  -- throwIO (NoBacktrace (ErrorCall "test"))
  Left e <- try @(ExceptionWithContext SomeException) (throwIO (ErrorCall "test"))
  throwIO (NoBacktrace e)

λ gecko ~ → runhaskell Ex.hs
Ex.hs: Uncaught exception ghc-internal:GHC.Internal.Exception.ErrorCall:

test

HasCallStack backtrace:
  throwIO, called at Ex.hs:5:56 in main:Main

λ gecko ~ → cat Ex.hs       
import Control.Exception

main = do
  -- throwIO (NoBacktrace (ErrorCall "test"))
  Left e <- try @(ExceptionWithContext SomeException) (throwIO (ErrorCall "test"))
  throwIO e

λ gecko ~ → runhaskell Ex.hs
Ex.hs: Uncaught exception ghc-internal:GHC.Internal.Exception.ErrorCall:

test

HasCallStack backtrace:
  throwIO, called at Ex.hs:6:3 in main:Main

HasCallStack backtrace:
  throwIO, called at Ex.hs:5:56 in main:Main

So as a first approximation, it seems that your implementation should be correct.

I'll continue the experimentation, maybe that's different with ghc 9.10.

[guibou]

With GHC 9.10.2:

λ gecko ~ → cat Ex.hs
import Control.Exception
import GHC.Internal.Exception.Type

main = do
  -- throwIO (NoBacktrace (ErrorCall "test"))
  Left e <- try @(ExceptionWithContext SomeException) (throwIO (ErrorCall "test"))
  throwIO (NoBacktrace e)

λ gecko ~ → runhaskell Ex.hs
Ex.hs: test
HasCallStack backtrace:
  collectBacktraces, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:169:13 in ghc-internal:GHC.Internal.Exception
  toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/IO.hs:260:11 in ghc-internal:GHC.Internal.IO
  throwIO, called at libraries/exceptions/src/Control/Monad/Catch.hs:308:12 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at compiler/GHC/Driver/Monad.hs:167:54 in ghc-9.10.2-ef99:GHC.Driver.Monad
  a type signature in an instance, called at compiler/GHC/Driver/Monad.hs:167:54 in ghc-9.10.2-ef99:GHC.Driver.Monad
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at ghc/GHCi/UI/Monad.hs:288:15 in ghc-bin-9.10.2-4bb7:GHCi.UI.Monad
  a type signature in an instance, called at ghc/GHCi/UI/Monad.hs:288:15 in ghc-bin-9.10.2-4bb7:GHCi.UI.Monad
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/haskeline/System/Console/Haskeline/InputT.hs:53:39 in haskeline-0.8.2.1-0f10:System.Console.Haskeline.InputT
  a type signature in an instance, called at libraries/haskeline/System/Console/Haskeline/InputT.hs:53:39 in haskeline-0.8.2.1-0f10:System.Console.Haskeline.InputT
  throwM, called at ghc/GHCi/UI/Monad.hs:215:52 in ghc-bin-9.10.2-4bb7:GHCi.UI.Monad


λ gecko ~ → cat Ex.hs       
import Control.Exception
import GHC.Internal.Exception.Type

main = do
  -- throwIO (NoBacktrace (ErrorCall "test"))
  Left e <- try @(ExceptionWithContext SomeException) (throwIO (ErrorCall "test"))
  throwIO e

λ gecko ~ → runhaskell Ex.hs
Ex.hs: test
HasCallStack backtrace:
  collectBacktraces, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:169:13 in ghc-internal:GHC.Internal.Exception
  toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/IO.hs:260:11 in ghc-internal:GHC.Internal.IO
  throwIO, called at libraries/exceptions/src/Control/Monad/Catch.hs:308:12 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at compiler/GHC/Driver/Monad.hs:167:54 in ghc-9.10.2-ef99:GHC.Driver.Monad
  a type signature in an instance, called at compiler/GHC/Driver/Monad.hs:167:54 in ghc-9.10.2-ef99:GHC.Driver.Monad
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at ghc/GHCi/UI/Monad.hs:288:15 in ghc-bin-9.10.2-4bb7:GHCi.UI.Monad
  a type signature in an instance, called at ghc/GHCi/UI/Monad.hs:288:15 in ghc-bin-9.10.2-4bb7:GHCi.UI.Monad
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/exceptions/src/Control/Monad/Catch.hs:427:21 in exceptions-0.10.9-f51c:Control.Monad.Catch
  throwM, called at libraries/haskeline/System/Console/Haskeline/InputT.hs:53:39 in haskeline-0.8.2.1-0f10:System.Console.Haskeline.InputT
  a type signature in an instance, called at libraries/haskeline/System/Console/Haskeline/InputT.hs:53:39 in haskeline-0.8.2.1-0f10:System.Console.Haskeline.InputT
  throwM, called at ghc/GHCi/UI/Monad.hs:215:52 in ghc-bin-9.10.2-4bb7:GHCi.UI.Monad


λ gecko ~ → 

There is no difference between NoBacktrace and no NoBacktrace.

This is surprising.

[guibou]

Haa, I think I got it. The problem is that in newest base, ErrorCall does not duplicate the callstack.

From base 4.12 changelog: https://hackage.haskell.org/package/base-4.21.0.0/changelog

Get rid of the HasCallStack mechanism manually propagated by ErrorCall in favour of the more general HasCallStack exception backtrace mechanism, to remove duplicate call stacks for uncaught exceptions.

So the reason for the duplicate call stack is that it is part of the context AND part of ErrorCall.

To be honest, I think we can keep that. This is painful, indeed, but that's part of a work in progress in the exception handling logic. Adding more complexity would, IMHO, just lead to problem in the future.