Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test on liveness for MySQL #6656

Open
tchiotludo opened this issue Jan 7, 2025 · 0 comments
Open

Flaky test on liveness for MySQL #6656

tchiotludo opened this issue Jan 7, 2025 · 0 comments
Assignees
Labels
area/backend Needs backend code changes bug Something isn't working

Comments

@tchiotludo
Copy link
Member

tchiotludo commented Jan 7, 2025

  io.kestra.runner.mysql.MysqlServiceLivenessCoordinatorTest ✘ shouldReEmitTasksWhenWorkerIsDetectedAsNonResponding() (30.2s)

    java.lang.AssertionError: 
    Expected: is <true>
         but: was <false>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
        at io.kestra.jdbc.runner.JdbcServiceLivenessCoordinatorTest.shouldReEmitTasksWhenWorkerIsDetectedAsNonResponding(JdbcServiceLivenessCoordinatorTest.java:129)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.micronaut.test.extensions.junit5.MicronautJunit5Extension$2.proceed(MicronautJunit5Extension.java:142)
        at io.micronaut.test.extensions.AbstractMicronautExtension.interceptEach(AbstractMicronautExtension.java:162)
        at io.micronaut.test.extensions.AbstractMicronautExtension.interceptTest(AbstractMicronautExtension.java:119)
        at io.micronaut.test.extensions.junit5.MicronautJunit5Extension.interceptTestMethod(MicronautJunit5Extension.java:129)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)


    2025-01-07 14:09:16,239 ERROR Test worker  io.kestra.core.secret.SecretService Could not decode secret 'SECRET_NON_B64_SECRET', make sure it is Base64-encoded: Illegal base64 character 20


    2025-01-07 14:09:16,608 INFO  worker_0     f.2PUJTFI02LWTHB4E5MV5mG.unit-test [namespace: io.kestra.unit-test] [flow: 2PUJTFI02LWTHB4E5MV5mG] [task: unit-test] [execution: SMfK7IpUtyGKqtQqpycWR] [taskrun: 6TRbqxVOjoPy6rnuZEKYOY] Type Sleep started
    2025-01-07 14:09:16,640 INFO  worker_0     f.2.S.6TRbqxVOjoPy6rnuZEKYOY Waiting for PT10S
    2025-01-07 14:09:16,650 ERROR worker_0     f.2.S.6TRbqxVOjoPy6rnuZEKYOY sleep interrupted
    java.lang.InterruptedException: sleep interrupted
    	at java.base/java.lang.Thread.sleep0(Native Method)
    	at java.base/java.lang.Thread.sleep(Thread.java:558)
    	at java.base/java.util.concurrent.TimeUnit.sleep(TimeUnit.java:446)
    	at io.kestra.plugin.core.flow.Sleep.run(Sleep.java:53)
    	at io.kestra.plugin.core.flow.Sleep.run(Sleep.java:21)
    	at io.kestra.core.runners.WorkerTaskCallable.doCall(WorkerTaskCallable.java:78)
    	at io.kestra.core.runners.AbstractWorkerCallable.call(AbstractWorkerCallable.java:56)
    	at io.kestra.core.runners.WorkerSecurityService.callInSecurityContext(WorkerSecurityService.java:10)
    	at io.kestra.core.runners.Worker.callJob(Worker.java:808)
    	at io.kestra.core.runners.Worker.runAttempt(Worker.java:767)
    	at io.kestra.core.runners.Worker.run(Worker.java:618)
    	at io.kestra.core.runners.Worker.handleTask(Worker.java:323)
    	at io.kestra.core.runners.Worker.lambda$run$8(Worker.java:252)
    	at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:49)
    	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    	at java.base/java.lang.Thread.run(Thread.java:1583)
    2025-01-07 14:09:16,659 INFO  worker_0     f.2PUJTFI02LWTHB4E5MV5mG.unit-test [namespace: io.kestra.unit-test] [flow: 2PUJTFI02LWTHB4E5MV5mG] [task: unit-test] [execution: SMfK7IpUtyGKqtQqpycWR] [taskrun: 6TRbqxVOjoPy6rnuZEKYOY] Type Sleep with state RUNNING completed in 00:00:00.026
    2025-01-07 14:09:22,327 WARN  service-liveness-coordinator-task s.AbstractServiceLivenessCoordinator Detected non-responding service [id=29yrr0ftE1QeuAAErNKMab, type=WORKER, hostname=fv-az1755-281] after timeout (5696ms).
    2025-01-07 14:09:23,328 WARN  service-liveness-coordinator-task s.AbstractServiceLivenessCoordinator Detected non-responding service [id=29yrr0ftE1QeuAAErNKMab, type=WORKER, hostname=fv-az1755-281] after termination grace period (992ms).
    2025-01-07 14:09:23,331 INFO  service-liveness-coordinator-task k.j.r.JdbcServiceLivenessCoordinator Trigger task restart for non-responding workers after termination grace period: [29yrr0ftE1QeuAAErNKMab].

@tchiotludo tchiotludo added bug Something isn't working area/backend Needs backend code changes labels Jan 7, 2025
@tchiotludo tchiotludo added this to Issues Jan 7, 2025
@github-project-automation github-project-automation bot moved this to Backlog in Issues Jan 7, 2025
@fhussonnois fhussonnois moved this from Backlog to In progress in Issues Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backend Needs backend code changes bug Something isn't working
Projects
Status: In progress
Development

No branches or pull requests

2 participants