[aosd-discuss] AOP and testing

[leeca at pnambic.com]

Juri, Richard, et al.,

Juri missed :-) a point here from the blog ....

> From: Juri Memmert
> Subject: Re: [aosd-discuss] AOP and testing
>
>
> --- Rickard Cberg <rickard at dreambean.com> wrote:
> Hi Rickard,
>
>
> > I made a blog entry about AOP and testing, and I'm curious
> if anyone on
> > this list have any comments and ideas on this topic. How do
> You deal
> > with the issues that are mentioned?
>
> I'd like to go through your blog and give you my replies on the way...
>
>
> > AOP is great. It allows us to do things that aren't really
> possible with any
> > other coding paradigm. AOP is also a big responsibility,
> because it means
> that
> > there's no single place to look for bugs when bugs occur.
> And bugs do,
> despite
> > occasional claims to the contrary, occur.
>
> Sorry... there is exactly one place to look. ;-)
> I do not use an AspectJ-like approach but a Hyperspaces-based
> one, so my
> thoughts and practices might not be applicable, but any
> application I write
> (beyond a very few classes) is captured in a Cosmos schema.
> Within that schema, I define the interactions of the various
> aspects/concerns
> of my application. There, I define what code is merged where.
> When a bug occurs, I go back to the schema and check which
> concerns were part
> of the code that created the bug (I also go back the stack
> trace to see what
> concerns were there).
> In 99% of the cases, the error occured where two+ concerns
> converge in one
> method or class. So, finding the concerns in the schema
> (which can be done with
> a simple search on the schema), this points me directly to
> the concerns that
> caused the problem and from there to the code that I wrote.
> From there,
> debugging normally is really simple.
>
>
> > If you've built an entire system on AOP, like we have, then
> how do you make
> > sure it actually works? How do you get the "green light"
> that TDD'ers crave
> > for?
> >
> > To be honest: I don't have the slightest idea.
>
> IMHO, you need a meta-level description of your application
> that tells you what
> converges where and why... something you can not find in the code.
> You can find the "what" and with more work the "where"...
> often a _lot_ more
> work... but definitely not the "why" (unless you document
> _way_ more than I've
> ever seen in any commercial project).
>
>
> > You could test each advice independently, but that might be
> difficult since
> > they usually rely on some particular context which may be
> impossible to
> > replicate for testing purposes.
>
> This is different with hyperslices. Each of the slices in
> itself is complete
> enough to test it separately and I just merge in a test
> context that is based
> on my best estimate of possible use cases.
>
>
> > Also, even if an advice is considered to work by a JUnit
> test, maybe it will
> > break when combined with other advice?
>
> I do integration tests on all hyperslices that are combined
> as defined in the
> Cosmos schema. The number is not too big.
> These integration tests are basically nothing more than the
> unit tests for each
> hyperslice plus a bit.
> If the unit tests for the various hyperslices do not work on
> the combined
> hypermodule anymore, you have modified the contract and need
> to document this.
> Often, this points to a problem as there could be a third
> hyperslice in this...
> But if it's not a problem, this also means that any class
> calling any method
> the resulting hypermodule needs to adhere to this new
> contract. That way the
> new contract is propagated and all integration tests need to
> reflect that (this
> is the "bit", I mentioned above).
>
>
> > Or, what if the pointcuts are incorrectly defined and the
> advice is never
> > applied in the first place?
>
> This is a large problem with the way pointcuts are defined in
> AspectJ. I've
> seen pointcut definitions longer than my arm that still were
> wrong. :-(
> It is much easier with hyperslices and a Cosmos schema from
> which to derive the
> mappings.
>
>
> > This will be a common issue if you're using method regexps to define
> pointcuts,
> > and do refactoring without using a tool.
>
> Regex are, in my opinion, a really bad idea. They are very
> powerful and
> flexible... and often you match more or less than you want.
> Unfortunately, you
> find out about that too late, too.
>
>
> > And even if you use a tool, just how is it supposed to
> work? Let's say a
> method
> > is being adviced because of a regexp including a "*". If
> that method is
> renamed
> > so that it's not covered anymore, should the regexp be
> changed so that it is
> > indeed still included? E.g. "foo*" -> "foo* | someMethod"
> when "fooBar" is
> > renamed to "someMethod". That could easily become a real nightmare.
>
> It _is_ a nightmare. It's one of the reasons I really dislike
> using that kind
> of method to define pointcuts.
>
>
> > The issue of what advice is applied where can be made more
> transparent by
> using
> > tools that allow you to introspect a system to see how it
> is actually
> > constructed.
>
> Yes, tools to inspect the application are helpful, but
> they're nowhere enough.
> You need to delve into the concerns that make up the
> application before they
> became code...
>
>
> > AspectJ relies on plugins to IDE's, and personally I'm
> leaning towards
> creating
> > a JavaDoc extension that creates HTML showing the actual
> structure of the
> > program. This, of course, relies on the idea that there are
> classes of
> objects
> > that work the same way, and some other AOP implementations
> do not have this
> > property (JBoss AOP comes to mind).
>
> JavaDoc is good... when you can ascertain that your code
> adheres to the
> standards needed by the doclet... and as you point out, that
> this might be
> problematic...
>
>
> > But the above is only related to advice. What about
> introductions and mixins?
> > Well, it's the similar problems there really, since using
> introductions is a
> > kind of weak typing approach where the only way to know
> whether an object has
> a
> > particular interface is to do instanceof or casting at
> runtime. The JavaDoc
> > thing can help again though, to make things a little more
> manageable and
> avoid
> > the simple errors.
>
> Again... Doclets can help... I use them myself... but they
> rely entirely too
> much on the documentation of the code base and not enough on
> the concerns that
> build the underlying conceptual base. Many of the problems
> with concern
> interaction are already visible in the concern model and can
> be resolved there,
> they don't have to be resolved in the code... unless you have
> nothing but the
> code to do it all...
>
>
> > Our own experience is that we have indeed been bitten by
> > some of the above problems, in particular the advice interaction and
> refactoring parts. All you
> > can do really, at this point, is to be aware of the problem
> and take care of
> it
> > in any way possible. And hope for understanding customers
> and a quick
> > turnaround time when bugs occur, both of which we have so far.
>

> > So, there are definite drawbacks with this approach.

and the rest of it ..

> > So, there are definite drawbacks with this approach.
> > Would I ever consider doing it any other way? Hell no. Why?
> > Because there is no other way. To do the things that we do
> > any other way would mean incredibly verbose and error-prone
> > coding, which would not be any better. Some features we have
> > would be so expensive to do without AOP that we'd probably
> > simply don't do them. And, of course, unwritten code have no bugs but
that's cheating.

>
> I don't see that as a drawback. Most of all since I can
> insert probes into my
> application to find / analyze / fix the bug.
>
>
> > There are two ways that I can think of to minimize the
> pains involved here.
> One
> > is to have tools that allow you to see the structure of the
> final program.
>
> The structure of the final program is interesting, but imho,
> not half as
> important as understanding what concerns you merge, where and why. The
> resulting structure is then a direct mapping of those concerns.
>
>
> > Second, using runtime attributes instead of regular
> expressions to define
> > pointcuts would minimize the risk of getting pointcut
> definition problems,
> > since defining runtime attributes is a much more
> semantically clear way to
> > define what should happen than using method naming conventions.
>
> I shudder at the "runtime" term... I do not think that
> "compile time" is too
> early... but regex definitely is problematic.
>
>
> > This is also how we do it. The pointcut definitions in our
> system that are
> > regexp either look like this: "set*|add*|remove*" or like this:
> > "create*|clone*|remove|", and we intend to replace them
> both with "!readonly"
> > and "lifecycle" runtime attribute declarations down the
> road to avoid any
> > potential problems.
>
> If you can get away with these regex, you're _very_ lucky. ;-)
> It has been said that the regex support of Hyper/J is lacking
> because it can
> not match all the things you want... but I found that not to
> be a limitation.
> I rather do explicit mappings all over my application or use
> even more simple
> expressions than the ones you listed above. That way, I have
> much more control
> over what I do... I do not like "automagically"...
> "automatically" is ok... but
>  as soon as you need to check a few thousand methods to make
> sure that noone
> messed up the naming conventions, I'm rather unhappy.
>
>
> I hope that helps... (otherwise ignore my ramblings. ;-) )
>
>
>      Juri
>
> __________________________________
> Do you Yahoo!?
> The New Yahoo! Search - Faster. Easier. Bingo.
> http://search.yahoo.com
> __________________________________________________
> AOSD Discuss mailing list    -    discuss at aosd.net
> To be removed send mail to  discuss-admin at aosd.net
> or visit http://aosd.net
>
>

---
http://www.pnambic.com/leeca
... The real cycle you're working on is a cycle called 'yourself'.