This is a workaround to failures we occasionally see on self-hosted #25613

Sashan · 2024-10-05T15:42:37Z

runner when running tests on FreeBSD. These failures are caused by some kind of race condition in perl code which controls test suite run. The typical symptom is:

Test Summary Report
-------------------
70-test_tls13messages.t               (Wstat: 0 Tests: 17 Failed: 0)
  Parse errors: No plan found in TAP output

The racecondition is time dependant. The slower the machine is the more likely is we trigger it. Our current self hosted runners run on google cloud as E2 instances. If I run simple shell script here:


GO=0
COUNT=0

while [ $GO -eq 0 ] ; do
        make test TESTS='test_tls13messages test_tls13hrr' HARNESS_JOBS=2
        GO=$?
        COUNT=$(($COUNT + 1))
done

echo "Failure after $COUNT tries"

The script seems to alwasy stop after 13 - 45 iterations due to 'No plan found in TAP output' error. Another possible error I could see was 'tests run out of sequence'. Both errors are occasionally seen.

If I upgrade google instance from E2 to N2 the script above continues to run overnight until I stop it.

Another option is to set number of harness jobs to 1. This also keeps script above running with no issues.

When using 1 job instead of 4 jobs makes test to run 11 minutes instead of 8. So it runs like 40% slower.

xnox · 2024-10-06T09:12:54Z

Typo in PR title & the commit message.

Note I think I can also observe similar on Linux...... when running HARNESS_JOBS set to nproc and the machine for tests getting randomly allocated a 128 CPU core instance with multi-threading. Thus it does seem like tests are not isolated enough.

For the tests that do use network, I do wonder if it would help to spin up a user-network namespace or a freebsd-jail, as it seemed like multiple tests were talking to the wrong client/server, or getting cleaned up in contention.

paulidale · 2024-10-06T21:22:28Z

I'd still really like to see the tests in question made idempotent.

Sashan · 2024-10-07T09:16:41Z

I'd still really like to see the tests in question made idempotent.

I understand. But the glitch seems to be coming from perl test framework. Output lines seem to be mixed up and TAP parser interprets that as a failure. I still have to find a way how to get better insight of what's happening with test logs for individual tests. The only machine where bug is triggered quite often is E2 instance on google. I was not able to find suitable settings for other virtualization means (proxmox, vmd).

kroeckx · 2024-10-07T14:54:32Z

Is it possible to see why it goes wrong with something like strace? Is there something that runs parallel, and mixes output from different writers? Does it need some kind of lock on stdout?

Sashan · 2024-10-07T15:52:54Z

Is it possible to see why it goes wrong with something like strace? Is there something that runs parallel, and mixes output from different writers? Does it need some kind of lock on stdout?

that's what I'm trying to figure out. there is a dtrace on FreeBSD, so tools do exist but I need to better understand what's going on in perl first. I would like to see how things look like in log parsed by TAP::Parser first. I don't know how to obtain a copy of log as seen by TAP parser, so that's where I'm at now.

Sashan · 2024-10-10T07:33:27Z

FYI I'm just testing change which might be considered as better workaround. Diff reads as follows:

diff --git a/util/perl/TLSProxy/Proxy.pm b/util/perl/TLSProxy/Proxy.pm
index 48c55f4822..9fd6cae358 100644
--- a/util/perl/TLSProxy/Proxy.pm
+++ b/util/perl/TLSProxy/Proxy.pm
@@ -291,7 +291,7 @@ sub start
         print STDERR "Server command: $execcmd\n";
     }
 
-    $pid = IPC::Open2::open2(my $sout, my $sin, $execcmd) or die "Failed to $execcmd: $!\n";
+    $pid = IPC::Open3::open3(my $sin, my $sout, my $serr, $execcmd) or die "Failed to $execcmd: $!\n";
     $self->{serverpid} = $pid;
 
     # Process the output from s_server until we find the ACCEPT line, which

Yesterday I've managed a way how to inspect data seen by TP::Parser() I've managed to capture three logs (log-tap-ok.txt, log-out-of-sequence.txt and log-no-plan.txt). When things do work and test passes the log-tap-ok.txt shows all subtests results appear at the beginning of line. If I understand perl code in Test::More et.al. right, then this is what TAP::Parser expects to see to conclude tests are successful:

sashan@daemon:~/openssl $ egrep -e '^ok|^1\.\.17'  log-tap-ok.txt
ok 1 - Server ciphersuite changes%s
1..17%s
ok 1 - Default handshake test%s
ok 2 - Resumption handshake test%s
ok 2 - Client ciphersuite changes%s
ok 3 - Server duplicated HRR%s
ok 3 - status_request handshake test (client)%s
ok 4 - status_request handshake test (server)%s
ok 4 - Invalid group with HRR%s
ok 5 - status_request handshake test%s
ok 5 - supported_versions missing from HRR%s
ok 6 - status_request handshake with client auth test%s
ok 7 - Client auth handshake test%s
ok 8 - Server name handshake test (client)%s
ok 9 - Server name handshake test (server)%s
ok 10 - Server name handshake test%s
ok 11 - ALPN handshake test (client)%s
ok 12 - ALPN handshake test (server)%s
ok 13 - ALPN handshake test%s
ok 14 - SCT handshake test%s
ok 15 - HRR handshake test%s
ok 16 - Resumption handshake with HRR test%s
ok 17 - Acceptable but non preferred key_share

If test fails with No plan... message, then TAP::Parser could not find line which matches ^1\.\.17 pattern (line which begins with 1..17). The line is still present in file but it is not found at the beginning of line (does not match ^1\.\.17 pattern). It's found here in log-no-plan.txt file:

84:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA%s
Ciphersuite: TLS_AES_128_GCM_SHA256%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:1..17%s
RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512%s
No peer certificate or raw public key%s

See line which begins with ^Signature Algo pattern. That line is actually broken by 1..17%s string into two lines. This makes also the string (1..17) to appear at the end of the line, therefore TAP::Parser can not see it and thinks plan is missing in test.

The Out of sequeence error comes with similar symptoms:

Client cipher list:     ok 1 - Message type check. Got 1, expected 1%s                                                                                                                                                                                                                                                      
TLS_AES_256_GCM_SHA384    ok 2 - Extension presence check (Message: 1 Extension: 4, 0)%s                                                                                                                                                                                                                                    
:    ok 3 - Extension presence check (Message: 1 Extension: 0, 1)%s                                                                                                                                                                                                                                                         
TLS_CHACHA20_POLY1305_SHA256    ok 4 - Extension presence check (Message: 1 Extension: 7, 2)%s                                                                                                                                                                                                                              
:    ok 5 - Extension presence check (Message: 1 Extension: 7, 3)%s                                                                                                                                                                                                                                                         
TLS_AES_128_GCM_SHA256    ok 6 - Extension presence check (Message: 1 Extension: 7, 4)%s                                                                                                                                                                                                                                    
:    ok 7 - Extension presence check (Message: 1 Extension: 0, 5)%s                                                                                                                                                                                                                                                         
ECDHE-ECDSA-AES256-GCM-SHA384    ok 8 - Extension presence check (Message: 1 Extension: 0, 6)%s                                                                                                                                                                                                                             
:    ok 9 - Extension presence check (Message: 1 Extension: 7, 7)%s                                                                                                                                                                                                                                                         
ECDHE-RSA-AES256-GCM-SHA384    ok 10 - Extension presence check (Message: 1 Extension: 7, 8)%s                                                                                                                                                                                                                              
:    ok 11 - Extension presence check (Message: 1 Extension: 7, 9)%s                                                                                                                                                                                                                                                        
DHE-RSA-AES256-GCM-SHA384    ok 12 - Extension presence check (Message: 1 Extension: 7, 10)%s                                                                                                                                                                                                                               
:    ok 13 - Extension presence check (Message: 1 Extension: 7, 11)%s                                                                                                                                                                                                                                                       
ECDHE-ECDSA-CHACHA20-POLY1305    ok 14 - Extension presence check (Message: 1 Extension: 7, 12)%s                                                                                                                                                                                                                           
:    ok 15 - Extension presence check (Message: 1 Extension: 0, 13)%s                                                                                                                                                                                                                                                       
ECDHE-RSA-CHACHA20-POLY1305    ok 16 - Extension presence check (Message: 1 Extension: 0, 14)%s                                                                                                                                                                                                                             
:    ok 17 - Extension presence check (Message: 1 Extension: 7, 15)%s                                                                                                                                                                                                                                                       
DHE-RSA-CHACHA20-POLY1305    ok 18 - Extensions count mismatch (11, 11)%s                                                                                                                                                                                                                                                   
:    ok 19 - Message type check. Got 2, expected 2%s                                                                                                                                                                                                                                                                        
ECDHE-ECDSA-AES128-GCM-SHA256    ok 20 - Extension presence check (Message: 2 Extension: 7, 34)%s                                                                                                                                                                                                                           
:    ok 21 - Extension presence check (Message: 2 Extension: 7, 35)%s                                                                                                                                                                                                                                                       
ECDHE-RSA-AES128-GCM-SHA256    ok 22 - Extension presence check (Message: 2 Extension: 0, 36)%s                                                                                                                                                                                                                             
:    ok 23 - Extensions count mismatch (2, 2)%s                                                                                                                                                                                                                                                                             
DHE-RSA-AES128-GCM-SHA256    ok 24 - Message type check. Got 8, expected 8%s                                                                                                                                                                                                                                                
:    ok 25 - Extension presence check (Message: 8 Extension: 0, 37)%s                                                                                                                                                                                                                                                       
ECDHE-ECDSA-AES256-SHA384    ok 26 - Extension presence check (Message: 8 Extension: 0, 38)%s                                                                                                                                                                                                                               
:    ok 27 - Extension presence check (Message: 8 Extension: 0, 39)%s                                                                                                                                                                                                                                                       
ECDHE-RSA-AES256-SHA384    ok 28 - Extensions count mismatch (0, 0)%s                                                                                                                                                                                                                                                       
:    ok 29 - Message type check. Got 11, expected 11%s                                                                                                                                                                                                                                                                      
DHE-RSA-AES256-SHA256    ok 30 - Extension presence check (Message: 11 Extension: 0, 41)%s                                                                                                                                                                                                                                  
:    ok 31 - Extension presence check (Message: 11 Extension: 0, 42)%s                                                                                                                                                                                                                                                      
ECDHE-ECDSA-AES128-SHA256    ok 32 - Extensions count mismatch (0, 0)%s                                                                                                                                                                                                                                                     
:    ok 33 - Message type check. Got 15, expected 15%s                                                                                                                                                                                                                                                                      
ECDHE-RSA-AES128-SHA256    ok 34 - Message type check. Got 20, expected 20%s                                                                                                                                                                                                                                                
:    ok 35 - Message type check. Got 20, expected 20%s                                                                                                                                                                                                                                                                      
DHE-RSA-AES128-SHA256ok 1 - Default handshake test%s                                                                                                                                                                                                                                                                        
:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA%s                                                                                                      
Ciphersuite: TLS_AES_128_GCM_SHA256%s

As you can see Client cipher list: line is interrupted by ok 1 - Message type... string which is expected to be found at the beginning of the line.

The lines which interfere with output expected by TAP::Parser come from print_raw_cipherlist() in apps/lib/s_cb.c:

1179     BIO_puts(bio_err, "Client cipher list: ");
1180     for (i = 0; i < rlistlen; i += num, rlist += num) {
1181         const SSL_CIPHER *c = SSL_CIPHER_find(s, rlist);
1182         if (i)
1183             BIO_puts(bio_err, ":");
1184         if (c != NULL) {
1185             BIO_puts(bio_err, SSL_CIPHER_get_name(c));
1186         } else if (memcmp(rlist, scsv_id, num) == 0) {
1187             BIO_puts(bio_err, "SCSV");
1188         } else {
1189             size_t j;
1190             BIO_puts(bio_err, "0x");
1191             for (j = 0; j < num; j++)
1192                 BIO_printf(bio_err, "%02X", rlist[j]);
1193         }
1194     }
1195     BIO_puts(bio_err, "\n");

What's worth to note in snippet above is that the output should go to stderr. The process which produces such output is invoked by Proxy::start here:

290     if ($self->debug) {
291         print STDERR "Server command: $execcmd\n";
292     }
293
294     $pid = IPC::Open2::open2(my $sout, my $sin, $execcmd) or die "Failed to $execcmd: $!\n";
295     $self->{serverpid} = $pid;
296
297     # Process the output from s_server until we find the ACCEPT line, which
298     # tells us what the accepting address and port are.
299     while (<$sout>) {
300         print;
301         s/\R$//; # chomp does not work on windows.
302         next unless (/^ACCEPT\s.*:(\d+)$/);
303         $self->{server_port} = $1;
304         last;
305     }

I suspect that if file descriptor for stderr is not defined then output for stderr gets mixed up with stdout. This idea made me to think of giving a open3() try to keep stderr separated from stdout. The oneliner change seems to work. I'm running the while loop for ~3 hours now with no issues. I'll let it run for two more hours and then I will update this pr with this better workaround.

log-tap-ok.txt
log-no-plan.txt
log-out-of-sequence.txt

This avoids false psotivie failures on FreeBSD-CI which suffers most from this issue. Fixes openssl#23992

Sashan · 2024-10-10T09:25:32Z

I've just forced pushed a change which reverts changes to github workflows and adds a one-liner tweak to Proxy.pm mentioned in my earlier comment.

mattcaswell · 2024-10-10T09:57:50Z

Nice work @Sashan!

nhorman · 2024-10-10T10:53:33Z

Woo hoo! Well done!

Sashan · 2024-10-10T11:13:25Z

This comment adds some debugging notes worth to remember.

the failure was triggered on google E2 instances running FreeBSD 13.2 only
we were lucky we could find a reproducer which could hit the issue within few minutes (see shell script in the first comment)
the hardest part here was to figure out how to obtain raw input for TAP parser, this is the diff here:

lifty$ diff -u /usr/libdata/perl5/TAP/Parser.pm Parser.pm                                                                                                                                                                                                                                                                  
--- /usr/libdata/perl5/TAP/Parser.pm    Tue Aug 13 07:42:16 2024
+++ Parser.pm   Thu Oct 10 13:08:14 2024
@@ -310,9 +310,15 @@
 
 =cut
 
+open(TAP_FH, '>', '/tmp/log-tap-stream.txt');
 sub next {
     my $self = shift;
-    return ( $self->{_iter} ||= $self->_iter )->();
+    my $tmp = ( $self->{_iter} ||= $self->_iter )->();
+    if (defined($tmp)) {
+       print TAP_FH "%s\n", $tmp->as_string();
+    }
+    return $tmp;
+    #return ( $self->{_iter} ||= $self->_iter )->();
 }
 
 ##############################################################################

levitte · 2024-10-10T12:13:21Z

A quick detail:

+       print TAP_FH "%s\n", $tmp->as_string();

print != printf

(I was wondering why your output had all those %s all over the place)

Sashan · 2024-10-11T10:05:01Z

All tests are green now. Looks like open3() does not work quite well on Windows. I tried to debug it but I had no luck this time. I was trying to place breakpoint around call to open3() but windows command line fails to interact with perl properly.

The easiest option is to let perl on windows to use open2() and open3() everywhere else.

Also note I'm using undef for stderr file descriptor when calling to open3() this still works. I did test that on FreeBSD machine. Using grep in log-tap-master.txt shows Signature Alogorithms which goes to stderr ends up in stdout

sashan@daemon:~/openssl $ grep ^Signature\ Alg log-tap-master.txt 
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:  Message type: NewSessionTicket(4)%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSSForwarded packet length = 1055%s
Signature Algorithms: ECDSAReceived server packet%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSAForwarded packet length = 239%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:Forwarded packet length = 239%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSSForwarded packet length = 255%s
Signature Algorithms: ECDSA%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA Record 2 (server -> client)%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512%s
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512%s

We don't want to mix stderr with stdout not to confuse TAP::parser(). So to check using undef in call to open3() still works we check the log file for FreeBSD-CI branch. We expect grep finds nothing:

sashan@daemon:~/openssl $ grep ^Signature\ Alg log-tap-FreeBSD-CI.txt
sashan@daemon:~/openssl $ sudo su 
You have mail.
root@daemon:/usr/home/sashan/openssl # halt -p
Connection to freebsd.sashan.net closed by remote host.
Connection to freebsd.sashan.net closed.

I think this is good to go now.

openssl-machine · 2024-10-12T12:00:33Z

This pull request is ready to merge

shahsb

Good job - @Sashan

Nice, detailed analysis and the fix.!

t8m · 2024-10-23T13:41:39Z

This should be merged to all the active branches.

This avoids false psotivie failures on FreeBSD-CI which suffers most from this issue. Fixes #23992 Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Bernd Edlinger <[email protected]> (Merged from #25613)

This avoids false psotivie failures on FreeBSD-CI which suffers most from this issue. Fixes #23992 Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Bernd Edlinger <[email protected]> (Merged from #25613) (cherry picked from commit 3d3bb26)

This avoids false psotivie failures on FreeBSD-CI which suffers most from this issue. There was conflict when merging openssl#25613 from master to openssl-3.2 and earlier. The openssl-3.2 branch (and earlier branches) are missing 4439ed1 which introduces `open2()` to create s_server process. Fixes openssl#23992

Sashan · 2024-10-23T20:31:07Z

This should be merged to all the active branches.

it got merged to master, 3.4 and 3.3. 3.2 and earlier merge ended up with conflict which I've resolved in PR here:
#25782

tmshort · 2024-10-28T19:51:21Z

Closing as this has been merged to master, openssl-3.4, and openssl-3.3. #25782 is the followup for openssl-3.2 and earlier.

This avoids false psotivie failures on FreeBSD-CI which suffers most from this issue. Fixes openssl#23992 Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Bernd Edlinger <[email protected]> (Merged from openssl#25613)

Sashan force-pushed the FreeBSD-CI branch from c6a38b8 to 0141fc8 Compare October 5, 2024 16:43

Sashan marked this pull request as ready for review October 5, 2024 18:40

Sashan mentioned this pull request Oct 5, 2024

Freebsd CI failure #23992

Closed

bernd-edlinger previously approved these changes Oct 7, 2024

View reviewed changes

tom-cosgrove-arm changed the title ~~This is a woraround to failures we occasionally see on self-hosted~~ This is a workaround to failures we occasionally see on self-hosted Oct 7, 2024

nhorman linked an issue Oct 8, 2024 that may be closed by this pull request

Freebsd CI failure #23992

Closed

Do not confuse TAP::Parser by mixing up stderr with stdout.

dfe5963

This avoids false psotivie failures on FreeBSD-CI which suffers most from this issue. Fixes openssl#23992

Sashan dismissed bernd-edlinger’s stale review via dfe5963 October 10, 2024 09:18

Sashan force-pushed the FreeBSD-CI branch from 0141fc8 to dfe5963 Compare October 10, 2024 09:18

t8m previously approved these changes Oct 10, 2024

View reviewed changes

mattcaswell previously approved these changes Oct 10, 2024

View reviewed changes

mattcaswell added approval: done This pull request has the required number of approvals and removed approval: review pending This pull request needs review by a committer labels Oct 10, 2024

Sashan requested review from levitte, t8m, mattcaswell and bernd-edlinger October 11, 2024 10:05

mattcaswell approved these changes Oct 11, 2024

View reviewed changes

bernd-edlinger approved these changes Oct 11, 2024

View reviewed changes

t8m approved these changes Oct 11, 2024

View reviewed changes

t8m added approval: done This pull request has the required number of approvals and removed approval: review pending This pull request needs review by a committer labels Oct 11, 2024

openssl-machine added approval: ready to merge The 24 hour grace period has passed, ready to merge and removed approval: done This pull request has the required number of approvals labels Oct 12, 2024

shahsb approved these changes Oct 14, 2024

View reviewed changes

t8m added branch: 3.0 Merge to openssl-3.0 branch branch: 3.1 Merge to openssl-3.1 branch: 3.2 Merge to openssl-3.2 labels Oct 23, 2024

Sashan mentioned this pull request Oct 23, 2024

Do not confuse TAP::Parser by mixing up stderr with stdout. #25782

Closed

Sashan removed branch: 3.0 Merge to openssl-3.0 branch branch: 3.1 Merge to openssl-3.1 branch: 3.2 Merge to openssl-3.2 labels Oct 23, 2024

tmshort closed this Oct 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

This is a workaround to failures we occasionally see on self-hosted #25613

This is a workaround to failures we occasionally see on self-hosted #25613

Sashan commented Oct 5, 2024

xnox commented Oct 6, 2024

paulidale commented Oct 6, 2024

Sashan commented Oct 7, 2024 •

edited

Loading

kroeckx commented Oct 7, 2024

Sashan commented Oct 7, 2024

Sashan commented Oct 10, 2024

Sashan commented Oct 10, 2024

mattcaswell commented Oct 10, 2024

nhorman commented Oct 10, 2024

Sashan commented Oct 10, 2024

levitte commented Oct 10, 2024 •

edited

Loading

Sashan commented Oct 11, 2024

openssl-machine commented Oct 12, 2024

shahsb left a comment

t8m commented Oct 23, 2024

Sashan commented Oct 23, 2024

tmshort commented Oct 28, 2024

This is a workaround to failures we occasionally see on self-hosted #25613

This is a workaround to failures we occasionally see on self-hosted #25613

Conversation

Sashan commented Oct 5, 2024

xnox commented Oct 6, 2024

paulidale commented Oct 6, 2024

Sashan commented Oct 7, 2024 • edited Loading

kroeckx commented Oct 7, 2024

Sashan commented Oct 7, 2024

Sashan commented Oct 10, 2024

Sashan commented Oct 10, 2024

mattcaswell commented Oct 10, 2024

nhorman commented Oct 10, 2024

Sashan commented Oct 10, 2024

levitte commented Oct 10, 2024 • edited Loading

Sashan commented Oct 11, 2024

openssl-machine commented Oct 12, 2024

shahsb left a comment

Choose a reason for hiding this comment

t8m commented Oct 23, 2024

Sashan commented Oct 23, 2024

tmshort commented Oct 28, 2024

Sashan commented Oct 7, 2024 •

edited

Loading

levitte commented Oct 10, 2024 •

edited

Loading