Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segmentation fault on v8.1.3 #14069

Closed
stefanosala opened this issue Jul 4, 2017 · 30 comments
Closed

Segmentation fault on v8.1.3 #14069

stefanosala opened this issue Jul 4, 2017 · 30 comments
Labels
v8 engine Issues and PRs related to the V8 dependency.

Comments

@stefanosala
Copy link

  • Version: v8.1.3
  • Platform: Linux f0022a84-19f5-4890-ac94-af66da6c237f 3.13.0-112-generic Added DS_Store to gitignore #159-Ubuntu SMP Fri Mar 3 15:26:07 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  • Subsystem:

Hi there,
we noticed that we're receiving a Segmentation Fault on node v8.

Here's the backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000d5d9d8 in v8::internal::compiler::StateValuesAccess::size() ()
(gdb) backtrace
#0  0x0000000000d5d9d8 in v8::internal::compiler::StateValuesAccess::size() ()
#1  0x0000000000c699ed in v8::internal::compiler::InstructionSelector::GetFrameStateDescriptor(v8::internal::compiler::Node*) ()
#2  0x0000000000c705d2 in v8::internal::compiler::InstructionSelector::VisitCall(v8::internal::compiler::Node*, v8::internal::compiler::BasicBlock*) ()
#3  0x0000000000c727db in v8::internal::compiler::InstructionSelector::VisitNode(v8::internal::compiler::Node*) ()
#4  0x0000000000c7346b in v8::internal::compiler::InstructionSelector::VisitBlock(v8::internal::compiler::BasicBlock*) ()
#5  0x0000000000c73757 in v8::internal::compiler::InstructionSelector::SelectInstructions() ()
#6  0x0000000000d0abf9 in void v8::internal::compiler::PipelineImpl::Run<v8::internal::compiler::InstructionSelectionPhase, v8::internal::compiler::Linkage*>(v8::internal::compiler::Linkage*) ()
#7  0x0000000000d0e637 in v8::internal::compiler::PipelineImpl::ScheduleAndSelectInstructions(v8::internal::compiler::Linkage*, bool) ()
#8  0x0000000000d1166d in v8::internal::compiler::PipelineImpl::OptimizeGraph(v8::internal::compiler::Linkage*) ()
#9  0x0000000000d119f7 in v8::internal::compiler::PipelineCompilationJob::ExecuteJobImpl() ()
#10 0x0000000000d95360 in v8::internal::CompilationJob::ExecuteJob() ()
#11 0x0000000000d9c4e3 in v8::internal::(anonymous namespace)::GetOptimizedCode(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ConcurrencyMode, v8::internal::BailoutId, v8::internal::JavaScriptFrame*) ()
#12 0x0000000000d9cd42 in v8::internal::(anonymous namespace)::GetLazyCode(v8::internal::Handle<v8::internal::JSFunction>) ()
#13 0x0000000000d9d16c in v8::internal::Compiler::Compile(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ClearExceptionFlag)
    ()
#14 0x0000000001129605 in v8::internal::Runtime_CompileLazy(int, v8::internal::Object**, v8::internal::Isolate*) ()
#15 0x000026325978437d in ?? ()
#16 0x00002632597842c1 in ?? ()
#17 0x00007fffffff7e20 in ?? ()
#18 0x0000000000000006 in ?? ()
#19 0x00007fffffff7e78 in ?? ()
#20 0x00002632597846b9 in ?? ()
#21 0x0000331b07904609 in ?? ()
#22 0x00003419a6682311 in ?? ()
#23 0x0000331b07904609 in ?? ()
#24 0x0000000100000000 in ?? ()
#25 0x0000263259784441 in ?? ()
#26 0x000000000000001a in ?? ()
#27 0x00007fffffff7ed0 in ?? ()
#28 0x000026325a958389 in ?? ()
#29 0x000008bc77192499 in ?? ()
#30 0x00003419a6682311 in ?? ()
#31 0x00003419a6682201 in ?? ()
#32 0x0000227c15536c51 in ?? ()
#33 0x00003419a6682311 in ?? ()
#34 0x0000331b07904609 in ?? ()
#35 0x000009f4f378ac19 in ?? ()
#36 0x0000227c15536bc9 in ?? ()
#37 0x0000227c15536b71 in ?? ()
#38 0x00007fffffff7f00 in ?? ()
#39 0x000026325978579b in ?? ()
#40 0x00003419a6682311 in ?? ()
#41 0x0000000100000000 in ?? ()
#42 0x0000227c15536bc9 in ?? ()
#43 0x000000000000001e in ?? ()
#44 0x00007fffffff7f60 in ?? ()
#45 0x000026325ab6adf0 in ?? ()
#46 0x000008bc77192499 in ?? ()
#47 0x00003419a6682311 in ?? ()
#48 0x00007fffffff7fe0 in ?? ()
#49 0x0000000000aa6992 in v8::Object::Get(v8::Local<v8::Context>, v8::Local<v8::Value>) ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Here's a list of compiled modules:

~ $ find node_modules/ -name "*.node"
node_modules/fsevents/build/Release/.node
node_modules/fsevents/build/Release/fse.node
node_modules/fsevents/lib/binding/Release/node-v11-darwin-x64/fse.node
node_modules/fsevents/lib/binding/Release/node-v57-darwin-x64/fse.node
node_modules/fsevents/lib/binding/Release/node-v48-darwin-x64/fse.node
node_modules/fsevents/lib/binding/Release/node-v47-darwin-x64/fse.node
node_modules/fsevents/lib/binding/Release/node-v46-darwin-x64/fse.node
node_modules/node-sass/vendor/linux-x64-57/binding.node
node_modules/kerberos/build/Release/obj.target/kerberos.node
node_modules/kerberos/build/Release/kerberos.node

We tried to downgrade to latest v7 and the issue is not there, so it must be something with node-v8.

Thanks!
Stefano
@targos targos added the v8 engine Issues and PRs related to the V8 dependency. label Jul 4, 2017
@bnoordhuis
Copy link
Member

@stefanosala Can you post the output of disassemble and info registers? Does running with --noturbo make a difference? Can you exclude kerberos and node-sass? (fsevents won't be loaded on your system, it's MacOS-only.)

@mscdex mscdex added the v8.x label Jul 4, 2017
@stefanosala
Copy link
Author

Thanks @bnoordhuis for looking into the issue!

--noturbo doesn't make any difference.

Here's the disassemble output: https://gist.github.com/stefanosala/f76b0d3d577714fb3542a20f8930224b

Here's the info registers output:

(gdb) info registers
rax            0x80000000f	34359738383
rbx            0x16	22
rcx            0x9	9
rdx            0x313d038	51630136
rsi            0x7fffffff6e90	140737488318096
rdi            0x7fffffff6f00	140737488318208
rbp            0x7fffffff6ff0	0x7fffffff6ff0
rsp            0x7fffffff6e80	0x7fffffff6e80
r8             0x4	4
r9             0x373c6f0	57919216
r10            0x0	0
r11            0x349bf40	55164736
r12            0x31430b0	51654832
r13            0x7fffffff7310	140737488319248
r14            0x313d2d0	51630800
r15            0x170095e	24119646
rip            0xd5d9d8	0xd5d9d8 <v8::internal::compiler::StateValuesAccess::size()+136>
eflags         0x10246	[ PF ZF IF RF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0

Removing all compiled modules doesn't make any difference either.

Thanks!
Cheers

@targos
Copy link
Member

targos commented Jul 4, 2017

Could you isolate the function that triggers this bug?
You can use call _v8_internal_Print_StackTrace() to get the JS stack trace.

@bnoordhuis
Copy link
Member

Thanks. Looks like SparseInputMask::InputIterator::GetReal() returns a bad node pointer: 0x80000000f.

Can you try a debug build and see what happens? ./configure --debug && make -j8 - the binary is called node_g.

Is there an easy way for us to reproduce?

@bnoordhuis
Copy link
Member

Looks like SparseInputMask::InputIterator::GetReal() returns a bad node pointer: 0x80000000f.

But just to be sure: what does p *('v8::internal::compiler::Node'*)$rax print?

@stefanosala
Copy link
Author

@targos here's the JS stack trace: https://gist.github.com/stefanosala/1345fa59d02a77f89531eca059f0d03c

@stefanosala
Copy link
Author

@bnoordhuis I'll try compiling a debug version.

@stefanosala
Copy link
Author

@bnoordhuis it prints:

(gdb) p *('v8::internal::compiler::Node'*)$rax
No symbol "v8::internal::compiler::Node" in current context

@bnoordhuis
Copy link
Member

@stefanosala What about simply p *(long*)$rax?

I'd delete that gist again if I were you, it contains sensitive information.

@stefanosala
Copy link
Author

@bnoordhuis thanks, deleted.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000d5d9d8 in v8::internal::compiler::StateValuesAccess::size() ()
(gdb) p *(long*)$rax
Cannot access memory at address 0x6e6f697461727473

@stefanosala
Copy link
Author

Hi @bnoordhuis,
this is the output from node_g:

#
# Fatal error in ../deps/v8/src/compiler/node-properties.cc, line 69
# Check failed: 1 == OperatorProperties::GetFrameStateInputCount(node->op()) (1 vs. 0).
#

==== C stack trace ===============================

    node-v8.1.3/node_g(v8::base::debug::StackTrace::StackTrace()+0x1d) [0x2916d3d]
    node-v8.1.3/node_g(V8_Fatal+0xf9) [0x2912b68]
    node-v8.1.3/node_g(v8::internal::compiler::NodeProperties::GetFrameStateInput(v8::internal::compiler::Node*)+0x64) [0x1ca1db2]
    node-v8.1.3/node_g(v8::internal::compiler::JSBinopReduction::CreateFrameStateForLeftInput()+0x1c) [0x1c2748a]
    node-v8.1.3/node_g(v8::internal::compiler::JSBinopReduction::ConvertInputsToNumber()+0xde) [0x1c2624a]
    node-v8.1.3/node_g(v8::internal::compiler::JSTypedLowering::ReduceSpeculativeNumberBinop(v8::internal::compiler::Node*)+0x7c) [0x1c2869c]
    node-v8.1.3/node_g(v8::internal::compiler::JSTypedLowering::Reduce(v8::internal::compiler::Node*)+0x5d9) [0x1c33595]
    node-v8.1.3/node_g(v8::internal::compiler::GraphReducer::Reduce(v8::internal::compiler::Node*)+0x9e) [0x1b845b8]
    node-v8.1.3/node_g(v8::internal::compiler::GraphReducer::ReduceTop()+0x22f) [0x1b84967]
    node-v8.1.3/node_g(v8::internal::compiler::GraphReducer::ReduceNode(v8::internal::compiler::Node*)+0xe1) [0x1b8431b]
    node-v8.1.3/node_g(v8::internal::compiler::GraphReducer::ReduceGraph()+0x32) [0x1b84518]
    node-v8.1.3/node_g(v8::internal::compiler::TypedLoweringPhase::Run(v8::internal::compiler::PipelineData*, v8::internal::Zone*)+0x4df) [0x1cb672f]
    node-v8.1.3/node_g(void v8::internal::compiler::PipelineImpl::Run<v8::internal::compiler::TypedLoweringPhase>()+0x4f) [0x1cbc08d]
    node-v8.1.3/node_g(v8::internal::compiler::PipelineImpl::CreateGraph()+0x4b8) [0x1cb9060]
    node-v8.1.3/node_g(v8::internal::compiler::PipelineCompilationJob::PrepareJobImpl()+0x2d9) [0x1cb4c7f]
    node-v8.1.3/node_g(v8::internal::CompilationJob::PrepareJob()+0x264) [0x1db5630]
    node-v8.1.3/node_g() [0x1db8633]
    node-v8.1.3/node_g() [0x1db9497]
    node-v8.1.3/node_g() [0x1dba42a]
    node-v8.1.3/node_g(v8::internal::Compiler::Compile(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ClearExceptionFlag)+0x158) [0x1dbb364]
    node-v8.1.3/node_g() [0x23168cf]
    node-v8.1.3/node_g(v8::internal::Runtime_CompileLazy(int, v8::internal::Object**, v8::internal::Isolate*)+0x112) [0x2316723]
    [0xdf91a784204]
Illegal instruction

Do you need anything else?

@bnoordhuis
Copy link
Member

@stefanosala Thanks. Can you check in gdb what p *node prints in frame 3 (the GetFrameStateInput frame) and likewise its members (p *node->op_, p *node->type_, etc.)?

If you have the time and inclination, it would be helpful to know if you also experience the issue with the master branch which bundles a newer V8 version. If it's fixed there, we can look into isolating and back-porting the fix.

@stefanosala
Copy link
Author

@bnoordhuis the issue is not present building from master.

Can you check in gdb what p *node prints in frame 3 (the GetFrameStateInput frame) and likewise its members (p *node->op_, p *node->type_, etc.)?

I'm not sure how to do that, I'm sorry :(

@bnoordhuis
Copy link
Member

In gdb, type bt to get the backtrace, select the right frame with f <n>, then type p *node, p *node->op_, etc.

@stefanosala
Copy link
Author

Thanks, here you are:

(gdb) f 2
#2  0x0000000001ca1db2 in v8::internal::compiler::NodeProperties::GetFrameStateInput (node=0x4219c38) at ../deps/v8/src/compiler/node-properties.cc:69
69	  DCHECK_EQ(1, OperatorProperties::GetFrameStateInputCount(node->op()));
(gdb) p *node
$1 = {static kOutlineMarker = 15, static kMaxInlineCount = 14, static kMaxInlineCapacity = 14, op_ = 0x3bd47d0 <_ZN2v88internal8compilerL6kCacheE+5840>, type_ = 0xe3f,
  mark_ = 14, bit_field_ = 1140851114, first_use_ = 0x4219ce8, inputs_ = {inline_ = {0x4211eb0}, outline_ = 0x4211eb0}}
(gdb) p *node->op_
$2 = {<v8::internal::ZoneObject> = {<No data fields>},
  _vptr.Operator = 0x2b12370 <vtable for v8::internal::compiler::SimplifiedOperatorGlobalCache::SpeculativeNumberDivideOperator<(v8::internal::compiler::NumberOperationHint)3>+16>, static kMaxControlOutputCount = 65535, opcode_ = 118, properties_ = {mask_ = 56 '8'}, mnemonic_ = 0x2b110db "SpeculativeNumberDivide", value_in_ = 2, effect_in_ = 1,
  control_in_ = 1, value_out_ = 1, effect_out_ = 1 '\001', control_out_ = 0}
(gdb) p *node->type_
Cannot access memory at address 0xe3f
(gdb) p *node->mark_
Cannot access memory at address 0xe
(gdb) p *node->bit_field_
Cannot access memory at address 0x440001aa
(gdb) p *node->first_use_
$3 = {next = 0x4219d30, prev = 0x0, bit_field_ = 9}
(gdb) p *node->inputs_
Attempt to take contents of a non-pointer value.
(gdb) p *node->inputs_->inline_
$4 = (v8::internal::compiler::Node *) 0x4211eb0
(gdb) p *node->inputs_->outline_
$5 = {node_ = 0x3bccdd8 <v8::internal::compiler::kCache+3448>, count_ = 84096000, capacity_ = 0, inputs_ = {0x6300010f0000000f}}

@bnoordhuis
Copy link
Member

Thanks. I had a working hypothesis but seeing SpeculativeNumberDivide in there sadly invalidated it. :-)

It's interesting that you didn't see the issue with V8 5.9 because I don't think this particular code path changed much since 5.8.

@bmeurer Perhaps you have a suggestion?

@bmeurer
Copy link
Member

bmeurer commented Jul 5, 2017

Is there some kind of asm.js code involved? I'm a bit puzzled about the stack trace.

@stefanosala
Copy link
Author

Nope @bmeurer, no asm.js involved in the project :(

@bmeurer
Copy link
Member

bmeurer commented Jul 6, 2017

Can you check with the latest Node Canary?

cc @mathiasbynens @schuay @nodejs/v8

@stefanosala
Copy link
Author

@bmeurer not happening on canary

@bmeurer
Copy link
Member

bmeurer commented Jul 6, 2017

Can you also try with Node 8.2.0RC1 please?

@stefanosala
Copy link
Author

No issue on 8.2.0RC1 .... /shrug :)

@bmeurer
Copy link
Member

bmeurer commented Jul 7, 2017

Ok, thanks. @targos do we also have downloadable Node w/ V8 6.0?

@targos
Copy link
Member

targos commented Jul 7, 2017

I don't think so. But if 5.9 doesn't have the issue, do we need to check 6.0?

@bmeurer
Copy link
Member

bmeurer commented Jul 7, 2017

Oh, right, sorry. I was confused.

@stefanosala
Copy link
Author

I guess everything will be magically fixed on 8.2 then? 👯 :)

@georgecrawford
Copy link

georgecrawford commented Jul 7, 2017 •
edited
Loading

I don't follow the above - it's totally out of my league! - but if it helps, I've observed the following:

docker run -it node:8.1.3-alpine /bin/sh -c 'npm i bcrypt && node -e '"'"'require("bcrypt").genSalt(10)'"'" produces output:

<SNIP>
npm WARN enoent ENOENT: no such file or directory, open '/package.json'
npm WARN !invalid#1 No description
npm WARN !invalid#1 No repository field.
npm WARN !invalid#1 No README data
npm WARN !invalid#1 No license field.

+ [email protected]
added 115 packages in 7.118s
npm info ok
Segmentation fault

@georgecrawford georgecrawford mentioned this issue Jul 7, 2017
Closed
@stefanosala
Copy link
Author

Hey folks, wait a min... I tried with 8.2.1 and the issue is still there :(

@bnoordhuis
Copy link
Member

8.2.1 is still using V8 5.8. See #13515 and #14004 for in-progress updates to 5.9 and 6.0 respectively.

@bnoordhuis
Copy link
Member

#14004 landed and will be released in v8.3.0. I'll close this out, cheers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@mscdex @georgecrawford @stefanosala @bnoordhuis @bmeurer @targos