Skip to content

Spurious failures in sccache #40240

Closed
@alexcrichton

Description

@alexcrichton
Member

Outstandling failures:

Failed to spawn child broken pipe

maybe fixed in #40809?

error: failed to execute compile
caused by: failed to spawn child
caused by: Broken pipe (os error 32)
make[3]: *** [bin/llvm-config] Error 2

Notes:

Error 254

Suspected bug in sccache. Hidden by mozilla/sccache#82 by accident. Sccache update in #40676

maybe fixed in #40809?

notes:

  • One job failed nul bytes in env, args, or program passed to Command when creating preprocessor. This seems crazy. Memory corruption?
  • OSX or Linux so far, no Windows.
  • Most indicate just one compilation failing, the last log indicates the entire server crashed w/ a segfault or something

Fixed failures

Unexpected EOF

Hoping to be fixed by mozilla/sccache#79

error: failed to execute compile
caused by: error reading compile response from server
caused by: IoError(Error { repr: Custom(Custom { kind: Other, error: StringError("unexpected EOF") }) })
caused by: unexpected EOF
make[3]: *** [tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o] Error 2

Forcibly closed by remote host

Suspected fixed in mozilla/sccache@5f6932b

PR sent in #40676

error: failed to execute compile
caused by: Failed to send data to or receive data from server
caused by: IoError(Error { reprError { : Os { code: 10054repr: Os { code: 10054, message: "An existing connection was forcibly closed by the remote host." } })

notes:

  • Only windows so far
  • Both 32 and 64 bit
  • Ignore segfaults, they're likely Remove dependency on named_pipe mozilla/sccache#83
  • Both beginning and end of the build
  • appears that the server process is crashing
  • suspect, crossbeam segfaulting (there are known segfaults in there)

Wedged in build logs

wedged in mio, likely fixed in #40809

Errors not at the fault of sccache

ar.exe/ranlib errors

More tracked at #40546 suspected to be unrelated to sccache and instead related to MinGW makefiles (and how they're not always the best...)

Hoping to be fixed by #40548

Activity

added
A-spuriousArea: Spurious failures in builds (spuriously == for no apparent reason)
on Mar 3, 2017
alexcrichton

alexcrichton commented on Mar 11, 2017

@alexcrichton
MemberAuthor

First attempt to debug this landed in #40240, although that didn't cover OSX/Windows which seem to be failing a lot as well. Second attempt to debug this is #40442

57 remaining items

added a commit that references this issue on Apr 28, 2017
alexcrichton

alexcrichton commented on Apr 28, 2017

@alexcrichton
MemberAuthor

I've updated sccache's mio, next step is binaries.

I'll be flying for the next ~20 hours and won't have the ability to do so until I land unfortunately :(

alexcrichton

alexcrichton commented on Apr 28, 2017

@alexcrichton
MemberAuthor

oops didn't mean to close

added a commit that references this issue on Apr 29, 2017
aa891a5
added a commit that references this issue on Apr 29, 2017

Auto merge of #41623 - alexcrichton:update-sccache, r=TimNN

12e76e2
kennytm

kennytm commented on Aug 9, 2017

@kennytm
Member

There has been two sccache failure this week, both talking about invalid checksum.

43652 (failure on Linux):

ERROR:sccache::server: ["ARMFastISel.cpp.o"] fatal error: Error(Io(Error { repr: Custom(Custom { kind: Other, error: StringError("Invalid checksum") }) }), State { next_error: None })

43732 (failure on Windows):

ERROR:sccache::server: ["LoopInfo.cpp.obj"] fatal error: Error(Io(Error { repr: Custom(Custom { kind: Other, error: StringError("Invalid checksum") }) }), State { next_error: None })

Reopen?

alexcrichton

alexcrichton commented on Aug 9, 2017

@alexcrichton
MemberAuthor

I think this issue is probably no longer useful, @kennytm mind opening a new issue? You can cc mozilla/sccache#171, the upstream report tracking this.

kennytm

kennytm commented on Aug 9, 2017

@kennytm
Member
alexcrichton

alexcrichton commented on Aug 9, 2017

@alexcrichton
MemberAuthor

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-spuriousArea: Spurious failures in builds (spuriously == for no apparent reason)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @alexcrichton@kennytm@luser@aturon@TimNN

      Issue actions

        Spurious failures in sccache · Issue #40240 · rust-lang/rust