Skip to content

Audio tests are flaky under firefox headless in CI #24205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sbc100 opened this issue Apr 28, 2025 · 1 comment
Open

Audio tests are flaky under firefox headless in CI #24205

sbc100 opened this issue Apr 28, 2025 · 1 comment

Comments

@sbc100
Copy link
Collaborator

sbc100 commented Apr 28, 2025

Since enabled audio tests in FF headless more in CI in #23701 we have been seeing flakes like this:

======================================================================
FAIL [184.888s]: test_sdl_audio_beep_sleep (test_browser.browser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/project/test/common.py", line 2355, in run_browser
    self.assertContained(expected, output)
  File "/root/project/test/common.py", line 1653, in assertContained
    self.fail("Expected to find '%s' in '%s', diff:\n\n%s\n%s" % (
  File "/usr/lib/python3.8/unittest/case.py", line 753, in fail
    raise self.failureException(msg)
AssertionError: Expected to find '/report_result?exit:0
' in '[no http server activity]
', diff:

--- expected
+++ actual
@@ -1 +1 @@
-/report_result?exit:0
+[no http server activity]



During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/project/test/common.py", line 896, in resulting_test
    return func(self, *args)
  File "/root/project/test/test_browser.py", line 216, in decorated
    f(self, *args, **kwargs)
  File "/root/project/test/test_browser.py", line 3352, in test_sdl_audio_beep_sleep
    self.btest_exit('test_sdl_audio_beep_sleep.cpp', emcc_args=['-Os', '-sASSERTIONS', '-sDISABLE_EXCEPTION_CATCHING=0', '-profiling', '-lSDL', '-sASYNCIFY'] + args, timeout=90)
  File "/root/project/test/common.py", line 2406, in btest_exit
    return self.btest(filename, *args, **kwargs)
  File "/root/project/test/common.py", line 2439, in btest
    self.run_browser(url, expected=['/report_result?' + e for e in expected], timeout=timeout, extra_tries=extra_tries)
  File "/root/project/test/common.py", line 2360, in run_browser
    return self.run_browser(html_file, expected, message, timeout, extra_tries - 1)
  File "/root/project/test/common.py", line 2362, in run_browser
    raise e
  File "/root/project/test/common.py", line 2355, in run_browser
    self.assertContained(expected, output)
  File "/root/project/test/common.py", line 1653, in assertContained
    self.fail("Expected to find '%s' in '%s', diff:\n\n%s\n%s" % (
AssertionError: Expected to find '/report_result?exit:0
' in '[no http server activity]
', diff:

--- expected
+++ actual
@@ -1 +1 @@
-/report_result?exit:0
+[no http server activity]



======================================================================
FAIL [186.825s]: test_sdl_audio_beep_sleep_safeheap (test_browser.browser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/project/test/common.py", line 2355, in run_browser
    self.assertContained(expected, output)
  File "/root/project/test/common.py", line 1653, in assertContained
    self.fail("Expected to find '%s' in '%s', diff:\n\n%s\n%s" % (
  File "/usr/lib/python3.8/unittest/case.py", line 753, in fail
    raise self.failureException(msg)
AssertionError: Expected to find '/report_result?exit:0
' in '[no http server activity]
', diff:

--- expected
+++ actual
@@ -1 +1 @@
-/report_result?exit:0
+[no http server activity]



During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/project/test/common.py", line 896, in resulting_test
    return func(self, *args)
  File "/root/project/test/test_browser.py", line 216, in decorated
    f(self, *args, **kwargs)
  File "/root/project/test/test_browser.py", line 3352, in test_sdl_audio_beep_sleep
    self.btest_exit('test_sdl_audio_beep_sleep.cpp', emcc_args=['-Os', '-sASSERTIONS', '-sDISABLE_EXCEPTION_CATCHING=0', '-profiling', '-lSDL', '-sASYNCIFY'] + args, timeout=90)
  File "/root/project/test/common.py", line 2406, in btest_exit
    return self.btest(filename, *args, **kwargs)
  File "/root/project/test/common.py", line 2439, in btest
    self.run_browser(url, expected=['/report_result?' + e for e in expected], timeout=timeout, extra_tries=extra_tries)
  File "/root/project/test/common.py", line 2360, in run_browser
    return self.run_browser(html_file, expected, message, timeout, extra_tries - 1)
  File "/root/project/test/common.py", line 2362, in run_browser
    raise e
  File "/root/project/test/common.py", line 2355, in run_browser
    self.assertContained(expected, output)
  File "/root/project/test/common.py", line 1653, in assertContained
    self.fail("Expected to find '%s' in '%s', diff:\n\n%s\n%s" % (
AssertionError: Expected to find '/report_result?exit:0
' in '[no http server activity]
', diff:

--- expected
+++ actual
@@ -1 +1 @@
-/report_result?exit:0
+[no http server activity]
@dschuff
Copy link
Member

dschuff commented Apr 28, 2025

I like searchParams.has_and_v

leftover paste from our conversation on the previous PR? :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants