[lively-kernel] Parts testing

[Fabian Bornhofen]

Hi there -

we hope all of you had a great start into 2012.

One issue that came up at the end of last year was parts testing. We
do have test cases for classes and traits, but we are not covering
most (core) tools from the PartsBin. Obviously this is a problem. We
ran into a regression just today. While we are sure _that_ we want to
test them, we are not sure about the _how_ yet.

Here are a few scenarios:

1) Write regular class-based test cases
mechanism:
- write a lively module with classes subclassing TestCase
- setup method: downloads part from PartsBin
- testXYtestName methods performing all manipulations and asserts()
pro:
- no need for additional tools or infrastructure
contra:
- test cases and tested parts are separated (conceptually and locally)
- need a class browser to write tests for a part (which we develop
with our ObjectEditor); tests would be run in TestRunner
- where to put these test cases?

2) Store test cases in parts
mechanism:
- add methods to each part that are marked as tests which is easy now
that we can tag methods (alternatively, we can run all methods named
"testXY...")
- provide a tool that runs tests:
  (a) extend TestRunner so it can download parts and run their tests
(this could be triggered e.g. by a button in the PartsBin browser)
  (b) extend ObjectEditor so it runs tests
- integrate parts tests into existing Jenkins script
pro:
- you know where to look for a part's tests
- no matter where you get a part from, you'll always get its specs
(provided somebody wrote tests ;) ).
- different versions of the same part can have different tests
contra:
- a small or moderate amount of work is needed to make it work

3) A mechanism suggested by you.

We are experimenting and would appreciate additional opinions on that.
Do you feel that we missed an important point in the suggestions in
this e-mail? Please let us know!

Regards,
Lauritz and Fabian