Good day,
I am trying to dig into an issue after a recent Oracle update. The
application has been running flawlessly from version 9.x to 18.5, and
has only broken on the most recent client library updates.
Attached is a cut down application which demonstrates the issue.
The forked child processes never exit. Attaching to them with GDB gives:
(gdb) t a a bt
Thread 1 (Thread 0x7f0c48a62c40 (LWP 773676) "test.pl"):
#0 futex_wait (private=<optimized out>, expected=14,
futex_word=0x55c5ef1c96e4) at ../sysdeps/nptl/futex-internal.h:146
#1 futex_wait_simple (private=<optimized out>, expected=14,
futex_word=0x55c5ef1c96e4) at ../sysdeps/nptl/futex-internal.h:177
#2 __GI___pthread_cond_destroy (cond=0x55c5ef1c96c0) at
pthread_cond_destroy.c:53
#3 0x00007f0c35cac072 in sltspcdestroy () from
/usr/lib/oracle/21/client64/lib/libclntshcore.so.21.1
#4 0x00007f0c36ecf68e in kpucpstopthr () from
/usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#5 0x00007f0c39e9a42e in kpufhndl0 () from
/usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#6 0x00007f0c48a1028a in ora_db_destroy (dbh=dbh@entry=0x55c5ef32c5b0,
imp_dbh=imp_dbh@entry=0x55c5ef035390) at
/root/.local/share/.cpan/build/DBD-Oracle-1.83-2/dbdimp.c:1263
#7 0x00007f0c48a10752 in XS_DBD__Oracle__db_DESTROY
(my_perl=0x55c5eee272a0, cv=<optimized out>) at ./Oracle.xsi:432
#8 0x00007f0c48a4d106 in XS_DBI_dispatch (my_perl=0x55c5eee272a0,
cv=0x55c5ef13b9c8) at
/usr/src/debug/perl-DBI-1.643-15.fc38.x86_64/DBI.xs:3783
#9 0x00007f0c487265aa in Perl_pp_entersub (my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/pp_hot.c:5353
#10 0x00007f0c4867c7a6 in Perl_call_sv (my_perl=<optimized out>,
sv=<optimized out>, flags=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/perl.c:3057
#11 0x00007f0c4872c643 in S_curse (my_perl=my_perl@entry=0x55c5eee272a0,
sv=sv@entry=0x55c5ef34af38, check_refcnt=check_refcnt@entry=true) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6973
#12 0x00007f0c4872cd88 in Perl_sv_clear
(my_perl=my_perl@entry=0x55c5eee272a0,
orig_sv=orig_sv@entry=0x55c5ef079858) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6536
#13 0x00007f0c4872d3a2 in Perl_sv_free2 (my_perl=0x55c5eee272a0,
sv=0x55c5ef079858, rc=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:7073
#14 0x00007f0c486fc878 in Perl_SvREFCNT_dec (sv=<optimized out>,
my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/inline.h:405
#15 S_mg_free_struct (my_perl=0x55c5eee272a0, sv=<optimized out>,
mg=0x55c5ef34bb70) at /usr/src/debug/perl-5.36.1-497.fc38.x86_64/mg.c:563
#16 0x00007f0c486fc8c9 in Perl_mg_free (my_perl=0x55c5eee272a0,
sv=0x55c5ef34aec0) at /usr/src/debug/perl-5.36.1-497.fc38.x86_64/mg.c:585
#17 0x00007f0c4872cdc3 in Perl_sv_clear
(my_perl=my_perl@entry=0x55c5eee272a0,
orig_sv=orig_sv@entry=0x55c5ef34aec0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6544
#18 0x00007f0c4872d3a2 in Perl_sv_free2 (my_perl=0x55c5eee272a0,
sv=0x55c5ef34aec0, rc=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:7073
#19 0x00007f0c48722646 in Perl_SvREFCNT_dec_NN (sv=<optimized out>,
my_perl=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/inline.h:419
#20 do_clean_objs (ref=0x55c5ef338ce0, my_perl=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:448
#21 S_visit (mask=2048, flags=2048, f=<optimized out>,
my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:391
#22 Perl_sv_clean_objs (my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:542
#23 0x00007f0c48680160 in perl_destruct (my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/perl.c:880
#24 0x000055c5eda0f324 in main (argc=<optimized out>, argv=<optimized
out>, env=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/perlmain.c:121
I have instrumented dbdimpl.c as follows:
/* free global environment handle during
destruction of last connection */
} else if ( (imp_dbh->envhp == imp_drh->envhp)
&& (SvTRUE(perl_get_sv("DBI::PERL_ENDING",0))) ) {
PerlIO_printf(PerlIO_stderr(), "EXIT1\n");
OCIHandleFree_log_stat(imp_dbh,
imp_dbh->envhp, OCI_HTYPE_ENV, status);
PerlIO_printf(PerlIO_stderr(), "EXIT2\n");
if ( status == OCI_SUCCESS ) {
PerlIO_printf(PerlIO_stderr(), "EXIT3\n");
imp_dbh->envhp = NULL;
imp_drh->envhp = NULL;
}
}
And the child prints 'EXIT1', but never 'EXIT2'. If I comment out the
call to OCIHandleFree_log_stat() then the child process exits without
error (although there are random segfaults after long run times).
1. A complete log of all steps of the build, e.g.:
Atatched (build.log)
2. Full details of which version of Oracle client and server you're using
# rpm -qa | grep oracle
oracle-instantclient-basic-21.11.0.0.0-1.x86_64
oracle-instantclient-devel-21.11.0.0.0-1.x86_64
3. The output of perl -V (that's a capital V, not lowercase)
Attached (perl-V.log)
Thanks,
Justin
I am trying to dig into an issue after a recent Oracle update. The
application has been running flawlessly from version 9.x to 18.5, and
has only broken on the most recent client library updates.
Attached is a cut down application which demonstrates the issue.
The forked child processes never exit. Attaching to them with GDB gives:
(gdb) t a a bt
Thread 1 (Thread 0x7f0c48a62c40 (LWP 773676) "test.pl"):
#0 futex_wait (private=<optimized out>, expected=14,
futex_word=0x55c5ef1c96e4) at ../sysdeps/nptl/futex-internal.h:146
#1 futex_wait_simple (private=<optimized out>, expected=14,
futex_word=0x55c5ef1c96e4) at ../sysdeps/nptl/futex-internal.h:177
#2 __GI___pthread_cond_destroy (cond=0x55c5ef1c96c0) at
pthread_cond_destroy.c:53
#3 0x00007f0c35cac072 in sltspcdestroy () from
/usr/lib/oracle/21/client64/lib/libclntshcore.so.21.1
#4 0x00007f0c36ecf68e in kpucpstopthr () from
/usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#5 0x00007f0c39e9a42e in kpufhndl0 () from
/usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#6 0x00007f0c48a1028a in ora_db_destroy (dbh=dbh@entry=0x55c5ef32c5b0,
imp_dbh=imp_dbh@entry=0x55c5ef035390) at
/root/.local/share/.cpan/build/DBD-Oracle-1.83-2/dbdimp.c:1263
#7 0x00007f0c48a10752 in XS_DBD__Oracle__db_DESTROY
(my_perl=0x55c5eee272a0, cv=<optimized out>) at ./Oracle.xsi:432
#8 0x00007f0c48a4d106 in XS_DBI_dispatch (my_perl=0x55c5eee272a0,
cv=0x55c5ef13b9c8) at
/usr/src/debug/perl-DBI-1.643-15.fc38.x86_64/DBI.xs:3783
#9 0x00007f0c487265aa in Perl_pp_entersub (my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/pp_hot.c:5353
#10 0x00007f0c4867c7a6 in Perl_call_sv (my_perl=<optimized out>,
sv=<optimized out>, flags=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/perl.c:3057
#11 0x00007f0c4872c643 in S_curse (my_perl=my_perl@entry=0x55c5eee272a0,
sv=sv@entry=0x55c5ef34af38, check_refcnt=check_refcnt@entry=true) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6973
#12 0x00007f0c4872cd88 in Perl_sv_clear
(my_perl=my_perl@entry=0x55c5eee272a0,
orig_sv=orig_sv@entry=0x55c5ef079858) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6536
#13 0x00007f0c4872d3a2 in Perl_sv_free2 (my_perl=0x55c5eee272a0,
sv=0x55c5ef079858, rc=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:7073
#14 0x00007f0c486fc878 in Perl_SvREFCNT_dec (sv=<optimized out>,
my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/inline.h:405
#15 S_mg_free_struct (my_perl=0x55c5eee272a0, sv=<optimized out>,
mg=0x55c5ef34bb70) at /usr/src/debug/perl-5.36.1-497.fc38.x86_64/mg.c:563
#16 0x00007f0c486fc8c9 in Perl_mg_free (my_perl=0x55c5eee272a0,
sv=0x55c5ef34aec0) at /usr/src/debug/perl-5.36.1-497.fc38.x86_64/mg.c:585
#17 0x00007f0c4872cdc3 in Perl_sv_clear
(my_perl=my_perl@entry=0x55c5eee272a0,
orig_sv=orig_sv@entry=0x55c5ef34aec0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6544
#18 0x00007f0c4872d3a2 in Perl_sv_free2 (my_perl=0x55c5eee272a0,
sv=0x55c5ef34aec0, rc=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:7073
#19 0x00007f0c48722646 in Perl_SvREFCNT_dec_NN (sv=<optimized out>,
my_perl=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/inline.h:419
#20 do_clean_objs (ref=0x55c5ef338ce0, my_perl=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:448
#21 S_visit (mask=2048, flags=2048, f=<optimized out>,
my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:391
#22 Perl_sv_clean_objs (my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:542
#23 0x00007f0c48680160 in perl_destruct (my_perl=0x55c5eee272a0) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/perl.c:880
#24 0x000055c5eda0f324 in main (argc=<optimized out>, argv=<optimized
out>, env=<optimized out>) at
/usr/src/debug/perl-5.36.1-497.fc38.x86_64/perlmain.c:121
I have instrumented dbdimpl.c as follows:
/* free global environment handle during
destruction of last connection */
} else if ( (imp_dbh->envhp == imp_drh->envhp)
&& (SvTRUE(perl_get_sv("DBI::PERL_ENDING",0))) ) {
PerlIO_printf(PerlIO_stderr(), "EXIT1\n");
OCIHandleFree_log_stat(imp_dbh,
imp_dbh->envhp, OCI_HTYPE_ENV, status);
PerlIO_printf(PerlIO_stderr(), "EXIT2\n");
if ( status == OCI_SUCCESS ) {
PerlIO_printf(PerlIO_stderr(), "EXIT3\n");
imp_dbh->envhp = NULL;
imp_drh->envhp = NULL;
}
}
And the child prints 'EXIT1', but never 'EXIT2'. If I comment out the
call to OCIHandleFree_log_stat() then the child process exits without
error (although there are random segfaults after long run times).
1. A complete log of all steps of the build, e.g.:
Atatched (build.log)
2. Full details of which version of Oracle client and server you're using
# rpm -qa | grep oracle
oracle-instantclient-basic-21.11.0.0.0-1.x86_64
oracle-instantclient-devel-21.11.0.0.0-1.x86_64
3. The output of perl -V (that's a capital V, not lowercase)
Attached (perl-V.log)
Thanks,
Justin