A better way to build with Junit test failures

[Ian Black]

Hi,

Has anyone found a way to let people know that a Unit test has failed during
continuous integration, without failing (halting) a build?

I'm using Maven, and we want to see the Unit Test report with and without
unit test failures...but we don't just want the build to pass with unit test
failures, which can be achieved by setting the maven.test.failure.ignore
param.

We want the build to succeed but an email to be sent to the author of the
unit test.

Anyone done this already?



Thanks,

Ian.

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.665 / Virus Database: 428 - Release Date: 21/04/2004

[Sri Sankaran]

Maybe someone else has some more insight into this matter.

Sri

_____

From: Ian Black [mailto:***@pacemetrics.com]
Sent: Friday, April 23, 2004 9:06 AM
To: Sri Sankaran
Subject: RE: [Cruisecontrol-user] A better way to build with Junit test failures


Thanks Sri. I did think of the case whereby we periodically go and look at the reports. Not very nice though :o(

I guess we're looking to have our cake AND eat it here. We want a test report that shows both passes and errors and some sort of notification.
Ah well...we can always dream!

-----Original Message-----
From: Sri Sankaran [mailto:***@sas.com]
Sent: 23 April 2004 13:51
To: Ian Black
Subject: RE: [Cruisecontrol-user] A better way to build with Junit test failures


We too are using Maven to run the system builds. Since the console output -- also the contents of CC email -- doesn't provide much for debugging unit test failures, we have a low tech solution of knowing to look at the build/test-reports directory on the build server. Sigh!

That being said I don't think there is a way (without some work) to figure out the author of the unit test. Currently we use the default CC functionality of notifying all committers whose changes triggered the build. One of them invariably is responsible for the error.

Sri


_____

From: Ian Black [mailto:***@pacemetrics.com]
Sent: Friday, April 23, 2004 8:35 AM
To: cruisecontrol-***@lists.sourceforge.net
Subject: [Cruisecontrol-user] A better way to build with Junit test failures



Hi,

Has anyone found a way to let people know that a Unit test has failed during continuous integration, without failing (halting) a build?

I'm using Maven, and we want to see the Unit Test report with and without unit test failures...but we don't just want the build to pass with unit test failures, which can be achieved by setting the maven.test.failure.ignore param.

We want the build to succeed but an email to be sent to the author of the unit test.

Anyone done this already?



Thanks,

Ian.

[Joris Kuipers]

----- Original Message -----
From: Ian Black
To: cruisecontrol-***@lists.sourceforge.net
Sent: Friday, April 23, 2004 2:35 PM
Subject: [Cruisecontrol-user] A better way to build with Junit test
failures

Post by Ian Black
Has anyone found a way to let people know that a Unit test has failed
during continuous integration, without failing (halting) a build?
I'm using Maven, and we want to see the Unit Test report with and
without unit test failures...but we don't just want the build to pass
with unit test failures, which can be achieved by setting the
maven.test.failure.ignore param.
We want the build to succeed but an email to be sent to the author of
the unit test.

What is the reason you don't want to fail the build? You mention "without
failing (halting)". If halting prematurely is the problem, then you can
easily fix this. I don't know how this works in Maven, but this is what
you would do in Ant (I'm sure Maven has something similar):
set a propery to indicate that the junit tests have failed in the
junit-task. Don't immediately fail the build, but do all the other stuff
that you want to do. At the end of the build, check the property (and
maybe some similar properties from other tasks too) to see if everything
went OK. If not, fail the build at the end.

Really, failing the build is the appropiate way to inform your developers
that something went wrong. As an alternative, you can define two projects
in CruiseControl: one for just building, and one (that is dependent on the
first) for testing. Then you can fail the test-project without failing the
build-project.

Joris



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek
For a limited time only, get FREE Ground shipping on all orders of $35
or more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297

[Ian Black]

Hi Joris,

The reason we don't want to fail the build when unit tests fail, is bacuse
if we fail the build at that stage then we don't get to generate the website
and thus seethe unit test report. At present, the only way we get to see the
report is if all tests pass (useless report) or if we set a property in
project.properties that says don't fail the build on test failure. This
means we get a meaningful report, but it also means we don't get an
automatic notification of unit test failure.
That's a good idea with the two projects. Thanks.

-----Original Message-----
From: Joris Kuipers [mailto:***@xs4all.nl]
Sent: 23 April 2004 14:44
To: Ian Black; cruisecontrol-***@lists.sourceforge.net
Subject: Re: [Cruisecontrol-user] A better way to build with Junit test
failures



----- Original Message -----
From: Ian Black
To: cruisecontrol-***@lists.sourceforge.net
Sent: Friday, April 23, 2004 2:35 PM
Subject: [Cruisecontrol-user] A better way to build with Junit test
failures

Post by Ian Black
Has anyone found a way to let people know that a Unit test has failed
during continuous integration, without failing (halting) a build?
I'm using Maven, and we want to see the Unit Test report with and
without unit test failures...but we don't just want the build to pass
with unit test failures, which can be achieved by setting the
maven.test.failure.ignore param.
We want the build to succeed but an email to be sent to the author of
the unit test.

What is the reason you don't want to fail the build? You mention "without
failing (halting)". If halting prematurely is the problem, then you can
easily fix this. I don't know how this works in Maven, but this is what
you would do in Ant (I'm sure Maven has something similar):
set a propery to indicate that the junit tests have failed in the
junit-task. Don't immediately fail the build, but do all the other stuff
that you want to do. At the end of the build, check the property (and
maybe some similar properties from other tasks too) to see if everything
went OK. If not, fail the build at the end.

Really, failing the build is the appropiate way to inform your developers
that something went wrong. As an alternative, you can define two projects
in CruiseControl: one for just building, and one (that is dependent on the
first) for testing. Then you can fail the test-project without failing the
build-project.

Joris


---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.665 / Virus Database: 428 - Release Date: 21/04/2004

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.665 / Virus Database: 428 - Release Date: 21/04/2004



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek
For a limited time only, get FREE Ground shipping on all orders of $35
or more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297

[Jeffrey Fredrick]

why not write a custom publisher that sends notifications of errors it finds
in the log?

Jtf

-----Original Message-----
From: cruisecontrol-user-***@lists.sourceforge.net
[mailto:cruisecontrol-user-***@lists.sourceforge.net] On Behalf Of Ian
Black
Sent: Friday, April 23, 2004 7:05 AM
To: Joris Kuipers; cruisecontrol-***@lists.sourceforge.net
Subject: RE: [Cruisecontrol-user] A better way to build with Junit test
failures


Hi Joris,

The reason we don't want to fail the build when unit tests fail, is bacuse
if we fail the build at that stage then we don't get to generate the website
and thus seethe unit test report. At present, the only way we get to see the
report is if all tests pass (useless report) or if we set a property in
project.properties that says don't fail the build on test failure. This
means we get a meaningful report, but it also means we don't get an
automatic notification of unit test failure. That's a good idea with the two
projects. Thanks.

-----Original Message-----
From: Joris Kuipers [mailto:***@xs4all.nl]
Sent: 23 April 2004 14:44
To: Ian Black; cruisecontrol-***@lists.sourceforge.net
Subject: Re: [Cruisecontrol-user] A better way to build with Junit test
failures



----- Original Message -----
From: Ian Black
To: cruisecontrol-***@lists.sourceforge.net
Sent: Friday, April 23, 2004 2:35 PM
Subject: [Cruisecontrol-user] A better way to build with Junit test failures

Post by Ian Black
Has anyone found a way to let people know that a Unit test has failed
during continuous integration, without failing (halting) a build?
I'm using Maven, and we want to see the Unit Test report with and
without unit test failures...but we don't just want the build to pass
with unit test failures, which can be achieved by setting the
maven.test.failure.ignore param. We want the build to succeed but an
email to be sent to the author of the unit test.

What is the reason you don't want to fail the build? You mention "without
failing (halting)". If halting prematurely is the problem, then you can
easily fix this. I don't know how this works in Maven, but this is what you
would do in Ant (I'm sure Maven has something similar): set a propery to
indicate that the junit tests have failed in the junit-task. Don't
immediately fail the build, but do all the other stuff that you want to do.
At the end of the build, check the property (and maybe some similar
properties from other tasks too) to see if everything went OK. If not, fail
the build at the end.

Really, failing the build is the appropiate way to inform your developers
that something went wrong. As an alternative, you can define two projects in
CruiseControl: one for just building, and one (that is dependent on the
first) for testing. Then you can fail the test-project without failing the
build-project.

Joris


---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.665 / Virus Database: 428 - Release Date: 21/04/2004

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.665 / Virus Database: 428 - Release Date: 21/04/2004



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek For a
limited time only, get FREE Ground shipping on all orders of $35 or more.
Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Cruisecontrol-user mailing list Cruisecontrol-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-user




-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek
For a limited time only, get FREE Ground shipping on all orders of $35
or more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg297