Re: Multiple as-installed test suite failures under different circumstances with 5.1-test-1

[Axel Beckert <abe@xxxxxxxxxxxxxxx>]

Hi,

On Wed, Sep 09, 2015 at 01:04:04PM +0200, Axel Beckert wrote:
> > > This looks to me as if the wrong zsh binary is being executed for some of
> > > the tests.  Are there two zsh packages installed?
> > 
> > Yep, those tests are new, to cover recent changes to the shell.
> > I'm 99% convinced you're just testing an older version, else it
> > doesn't really make sense.
> 
> Thanks for that hint. Will check the remainder of the log if e.g. an
> automated installation step failed and has been unrecognised.

Indeed, in the Jenkins job, at least in all cases in the past two
days, 5.1 was installed instead of 5.1-test-1. We're still
investigating why this happens (and why it didn't happen in the past).

We though had a test suite failure at _build_ time (i.e. "make test"
directly after the build) once and only once withing 5 build runs so
far, so it may be a kind of race condition:

Running test: SH_NULLCMD option
*** /tmp/zsh.ztst.err.3773	Wed Sep  9 13:06:16 2015
--- /tmp/zsh.ztst.terr.3773	Wed Sep  9 13:06:16 2015
***************
*** 0 ****
--- 1,2 ----
+ fn:echo:2: write error: broken pipe
+ fn:2: write error: inappropriate ioctl for device
Test ../../Test/E01options.ztst failed: error output differs from expected as shown above for:
  print this is bar >bar
  fn() {
    local NULLCMD=cat READNULLCMD=cat
    echo hello | >foo
    cat foo
    <bar
  }
  setopt shnullcmd
  print option set
  fn
  unsetopt shnullcmd
  print option unset
  fn
  rm -f foo bar
Was testing: SH_NULLCMD option
../../Test/E01options.ztst: test failed.
../../Test/E02xtrace.ztst: starting.

I wonder if that's another case of a missing "2>/dev/null" like we
fixed a few before 5.1?

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@xxxxxxxxxxxxxxx  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@xxxxxxxxx (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)