Uploaded image for project: 'Erlang/OTP'
  1. Erlang/OTP
  2. ERL-1143

Eunit doesn't report a timeout in parallel tests

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Help Wanted
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 21.3
    • Fix Version/s: None
    • Component/s: eunit
    • Labels:
      None

      Description

      When a parallel test is cancelled by a timeout before it's reported, it's totally ignored in the result.

      -module(ignored_cancel).
      -include_lib( "eunit/include/eunit.hrl" ).
      
      foo(N) ->
          timer:sleep(N),
          ok.
      
      step1_test_() ->
          {inparallel, [
           {"foo1",                ?_assertEqual(ok, foo(800))},
           {"foo2",                ?_assertEqual(ok, foo(700))},
           {"foo3", {timeout, 0.2, ?_assertEqual(ok, foo(400))}}
          ]}.
       

       Sample run:

      Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]
      
      Eshell V10.3  (abort with ^G)
      1> c(ignored_cancel).
      {ok,ignored_cancel}
      2> eunit:test(ignored_cancel,[verbose]).
      ======================== EUnit ========================
      module 'ignored_cancel'
        ignored_cancel:11: step1_test_ (foo1)...[0.801 s] ok
        ignored_cancel:12: step1_test_ (foo2)...[0.701 s] ok
        [done in 0.804 s]
      =======================================================
        2 tests passed.
      ok
      

      Interestingly, if foo3 is moved to the top

      step1_test_() ->
          {inparallel, [
           {"foo3", {timeout, 0.2, ?_assertEqual(ok, foo(400))}},
           {"foo1",                ?_assertEqual(ok, foo(800))},
           {"foo2",                ?_assertEqual(ok, foo(700))}
          ]}.
      

      it works as expected:

      ======================== EUnit ========================
      module 'ignored_cancel'
        ignored_cancel:11: step1_test_ (foo3)...*timed out*
        ignored_cancel:12: step1_test_ (foo1)...[0.802 s] ok
        ignored_cancel:13: step1_test_ (foo2)...[0.701 s] ok
        [done in 0.805 s]
      =======================================================
        Failed: 0.  Skipped: 0.  Passed: 2.
      One or more tests were cancelled.
      error
      

      Similarly, increasing the timeout and time spent in the test over the time spent in other tests makes the whole report correct:

           {"foo1",                ?_assertEqual(ok, foo(800))},
           {"foo2",                ?_assertEqual(ok, foo(700))},
           {"foo3", {timeout, 0.9, ?_assertEqual(ok, foo(1000))}}
      
      ======================== EUnit ========================
      module 'ignored_cancel'
        ignored_cancel:11: step1_test_ (foo1)...[0.801 s] ok
        ignored_cancel:12: step1_test_ (foo2)...[0.701 s] ok
        ignored_cancel:13: step1_test_ (foo3)...*timed out*
        [done in 0.901 s]
      =======================================================
        Failed: 0.  Skipped: 0.  Passed: 2.
      One or more tests were cancelled.
      error
      

        Attachments

          Activity

            People

            Assignee:
            otp_team_ps Team PS
            Reporter:
            choroba E. Choroba
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: