The Java.net project for Intercepts (YAIL?) was finally approved, so we have real project infrastructure now =)
The mailing lists etc are all up, but I need to re-import CVS there as Chris wasn't able to work for a while, so we switched back to the self-hosted server after the initial import. I will be moving tarballs, CVS, and docs over tonight and this weekend.
0 writebacks [/src/java/commons-intercept] permanent link
Thu, 08 Jan 2004
Support for Mocking Classes as well as Interfaces
MockObjects have a great dyanamic proxy based mock object implementation available, but it is limited to mocking interfaces up. A coworker needed (well, wanted) to dynamically mock up a class instead of an interface so I tossed together support in intercepts for mocking arbitrary objects. If you want to mock a class it requires an instance of that class, as Java doesn't allow you to instantiate subclasses without being able to instantiate the super class, but it guarantees that no methods will be invoked on the object it is given.
Also fixed a pretty major bug where interceptors could be invoked out of order if objects were wrapped, and interceptors added, in a specific way. I fixed it via changing a map to a list of tuples. Sometimes Java's static typing instead of dynamic typing really bugs me.
Posted new snapshots as this involved a fix to the first bug found!
7 writebacks [/src/java/commons-intercept] permanent link
Mon, 05 Jan 2004Bo requested mixin support for Yet-Another-Intercept-Library (YAIL) so I just checked in support.
The mixin support is completely orthogonal to the interceptor support, and works like:
public void testMixer()
{
Mixer mixer = new Mixer();
mixer.addMixin(Map.class, new HashMap());
Object base = new Object();
Object mixed = mixer.mix(base);
assertTrue(mixed instanceof Map);
}
It works fine with the interception library as well:
public void testInterceptMixins()
{
InterceptionBroker broker = new InterceptionBroker();
MockInterceptor mock = new MockInterceptor();
broker.addInterceptor(mock, new ClassSignature(Map.class));
Mixer mixer = new Mixer();
mixer.addMixin(Map.class, new HashMap());
Object base = new Object();
Object mixed = mixer.mix(base);
Map wrapped = (Map) broker.wrap(mixed);
wrapped.put("1", "2");
assertTrue(mock.value);
}
The mixin support is little more than a thin veneer around cglib's mixins. Adding additional utility to the Mixer class will probably happen once I actually use mixins. I can see places where it will be awfully convenient for passing information down interceptor stacks though.
Finally, unrelated, YAIL and Nanning are getting pretty similar, which bugs me. Jon mentioned that he is considering adding cglib support back into Nanning for people who don't mind aspected classes being non-serializable. This seems like silly duplication, but ah well, is all fun. On a side note I implemeneted a quick-and-dirty Nanning backend to the InterceptionBroker and it worked pretty transparently. None of the unit tests passed though as I tend to use HashMap for the base object, and nanning cannot intercept classes, just interfaces.
Pushed another development snbapshot with this.
0 writebacks [/src/java/commons-intercept] permanent link
Over the holidays I spent some time looking at the other interception frameworks available for the JVM directly (ie, AspectJ and AspecktWerkz are out because they require special compilation or classloaders). As Intercepts-Without-A-Name (IWAN) grows I want to try to avoid reinventing wheels, after all.
Nanning I particularly like. It does a little more than interception, also supporting interface based mixins, but is primarily an interception tool. It is purely DynamicProxy based so it only intercepts calls against interfaces, and it requires finer grained setup than the broker in IWAN, but it is perty nice. Big kudos for not having any dependencies!
The Spring Framework is busy re-implementing a better designed J2EE stack without the EJB's and includes an AOP type thingie. It is a DynamicProxy provides a DyanmicProxy based interceptor facility as well. It works via the AOP Alliance interfaces. Dependencies on the AOP Alliance specification jar, and commons-logging. Not too bad, but commons-logging dependency is annoying.
Both of these libraries are designed to play nicely with IoC containers, Pico and Spring respectively. The more I play with interception the more I conclude that interception and IoC go hand in hand -- half of the standard interceptors I have done use type 2 IoC to configure the interceptor at execution time, not to mention the factory aspect of building intercepted objects.
In comparison both of these libraries are a little bit more limited (intercept interfaces only), and underperform IWAN by a factor of 3-5 times before we even get baking working. Spring is being actively developed, Nanning is in maintenance mode. Both have an IoC container associated with them to build tools with to get the beautiful IoC+Interception synergy.
All of that said, I think that IWAN (which I will never call it after this post as I hate the acronym, we need a real name) is worth continueing to persue. It offers a couple key features over these other two -- mainly that it is code generation based instead of dyanmic proxy based so is a lot more flexible and performant. The more I do (the security and transaction stuff, for instance) the more I realize that to really get full benefit I need to provide factory hooks. I hate to tie it to any given container, so maybe I will just try to provide factories alongside interceptors that make use of them that play nicely with generic type 2 and 3 containers (what I have been doing thus far).
On the other hand, the mixin stuff that nanning does would be pretty easy to add to IWAN, but I don't think I want to add it to the core as I want to keep the focus on interception. Ah, fun fun stuff.
5 writebacks [/src/java/commons-intercept] permanent link
Thu, 01 Jan 2004
Object Transactions and Lunch Conversations
I am quite lucky to work with very smart and creative people. This leads to wonderful lunch converstaions (though they ramble enough that we have, at times, been concerned about our waitress calling the FBI...). One of the things we discussed this past week was object transactions, identity, etc.
Now, my greatest familiarity with object space transactions actually being implemented outside of EJB containers is with OJB. OJB does it pretty nicely, but... it only really does it for persistent objects. We were talking general purpose object-space transactions such that the following test will pass:
public void testExample() throws Exception
{
final Map base = new HashMap();
Thread child = new Thread(new Runnable()
{
public void run()
{
// current() starts transaction if one hasn't already
Transaction tx = Transaction.current();
tx.setOptimistic(true);
Map bound = (Map)tx.writeLock(base);
bound.put("a", "1");
try { Thread.sleep(500); } catch (Exception e) {fail("Interrupted!!!");}
assertNull("base shouldn't be modified yet", base.get("a"));
tx.commit();
}
});
child.start();
Transaction tx = Transaction.current();
tx.setOptimistic(true);
Map bound = (Map)tx.writeLock(base);
// let child set "a"
Thread.sleep(200);
assertNull("Shouldn't have value from other tx", bound.get("a"));
// Set "a" in our transaction
bound.put("a", "2");
// Let child finish
Thread.sleep(1000);
assertEquals("child has committed, assert that base is updated", "1", base.get("a"));
try
{
tx.commit();
fail("should have thrown transaction exception");
}
catch (TransactionException e)
{
assertTrue("optimistic transaction failed", true);
}
assertEquals("base has child's operation", "1", base.get("a"));
assertEquals("parent wrapped object matches base now", "1", bound.get("a"));
}
This basically starts a child thread and a parent thread and times things to try to make sure that the child wins the optimistic lock race for a transaction. This should be pretty straightforward to do by a copy-on-transaction type semantic using the intercept library, but what if we want to support objects that don't have clone() for convenient copying?
I am thinking that a copy-on-tx solution may not work as well as intercepting every invocation downstream of a transaction bound interception and binding the result of the invocation to the transaction as well. I am not sure how well this will work for certain types of mutations, but for code adhering even closely to the law of demeter it should be an effective way to do it.
0 writebacks [/src/java/commons-intercept] permanent link
Wed, 24 Dec 2003
Looking for a Home for Intercepts
I grepped through the access logs counting downloads of the intercept-library-without-a-cool-name and discovered people actually seem interested in it. Pretty cool. That brought up another thought, which is that I should probably move it somewhere with public CVS, bug tracking, mailing lists, and all that happy stuff.
My initial thought was to move it to the Apache Jakarta Commons Sandbox if the project caught on, but I am not sure that is a good place for it. For one, only 50% of the contributors are Apache committers, so we would have to get Chris an account with Apache (shouldn't be a problem just based on the committer list for cglib) but it seems like a lot of hassle. Also, Jakarta Commons is intended for projects that grew out of, and are used in, existing Apache projects. This library isn't, and may never be. Though it is already used in non-Apache projects (scary as it is still very young and not set in its interfaces).
I dislike the idea of going to SourceForge as their cvs and tools seem to be down as much as they are up. They provide a lot of nice services though, when those services are working. SF does make it easy to add committers without rigamarole, and provides the basic lists, bug tracking, etc though.
Java.net is another option, but (despite the Collab.net goodness) their layout, cvs access, etc are a royal mess. Their services are decent, but... I have had so much trouble getting to cvs for java.net projects in the past that I hesitate to introduce that barrier to new contributors. Like SF.net, java.net does make it easy to add committers and manage resources though.
Right now it is self-hosted, and that is working for now. I just don't feel like setting up serious project infrastructure (mailing lists, wiki, public cvs, etc) for it on my server.
I guess I am just cranky, and shouldn't be so demanding of free services. Anyone have any preference for where this project goes?
0 writebacks [/src/java/commons-intercept] permanent link
Mon, 22 Dec 2003
Method-Invocation Level ACL Style Security via Interception
Checked in (and pushed a snapshot) of the interception library which Chris Nokleberg and I are working on. I started to put together an iptables style method invocation filter based upon actor-rule filtering. Using just interception, as compared to full AOP via AspectJ or whatnot doesn't allow access to the calling context, so we have to cheat a little bit and require registering the actor with the rule broker -- but it is done as a ThreadLocal so as to be easily used in a servlet or other J2EE environment.
Right now there are two levels of filtering through the security filter and broker. The first is a very course grained filtering, which is also much more efficient, based solely on static invocation information -- this is what you have the SecurityInterceptor match on:
InterceptionBroker broker = new InterceptionBroker();
SecurityInterceptor intercept = new SecurityInterceptor();
broker.addInterceptor(intercept, new NamedSignature("execute"));
The code above tells the interception broker that we we want to intercept and run through the security tests on any method named "execute" -- sudh as would be used on a typical Command style web application. If the method signature doesn't match the test (method name is "execute") than it isn't intrercepted and is allowed to continue.
The next level of filtering is done at invocation time, so is slower. To use this we need to register a SecurityBroker with the SecurityInterceptor
SecurityBroker sec = new SecurityBroker();
intercept.setSecurityBroker(sec);
sec.appendRule(new Rule()
{
public boolean allow(InvocationContext ctx)
{
User user = (User)ctx.getActor();
if (!user.isAuthenticated()) return false;
else return true;
}
});
This sets a rule that makes sure the actor is authenticated.
All of this is fun, but until we see it used it isn't all that useful, so here is an example of using it. This is a quick-and-dirty Struts Action which delegates to a more flexible Command
package org.skife.kim.struts.tools;
import org.skife.intercept.interceptors.security.Actor;
import org.skife.intercept.interceptors.security.SecurityBroker;
import org.skife.intercept.interceptors.security.AccessError;
import org.skife.intercept.interceptors.security.Rules;
import org.skife.intercept.interceptors.security.Rule;
import org.skife.intercept.interceptors.security.InvocationContext;
import org.skife.kim.model.User;
import org.apache.struts.action.Action;
import org.apache.struts.action.ActionForm;
import org.apache.struts.action.ActionForward;
import org.apache.struts.action.ActionMapping;
import org.apache.struts.action.ActionErrors;
import org.apache.struts.action.ActionMessage;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class DelegatingAction extends Action
{
private static final CommandFactory factory;
private static final SecurityBroker broker;
static
{
broker = new SecurityBroker();
broker.appendRule(new Rule()
{
public boolean allow(InvocationContext ctx)
{
User user = (User) ctx.getActor();
if (user.getFirstName().equalsIgnoreCase("George"))
{
return false;
}
return true;
}
});
broker.appendRule(Rules.ALLOW);
factory = new CommandFactory(broker);
}
public ActionForward execute(ActionMapping mapping,
ActionForm form,
HttpServletRequest request,
HttpServletResponse response) throws Exception
{
String cname = mapping.getParameter();
Actor actor = (Actor) request.getSession().getAttribute("current-actor");
if (actor == null) return mapping.findForward("login");
broker.setActor(actor);
Command comm = factory.build(cname);
BeanUtils.copyProperties(comm, form);
try
{
return mapping.findForward(comm.execute());
}
catch (AccessError e)
{
ActionErrors errors = new ActionErrors();
errors.add("general", new ActionMessage(e.getMessage()));
this.saveErrors(request, errors);
return mapping.findForward("SecurityError");
}
}
}
The CommandFactory does the fun work:
package org.skife.kim.struts.tools;
import org.skife.intercept.InterceptionBroker;
import org.skife.intercept.signatures.NamedSignature;
import org.skife.intercept.interceptors.security.SecurityInterceptor;
import org.skife.intercept.interceptors.security.SecurityBroker;
import java.io.IOException;
import java.util.Properties;
public class CommandFactory
{
private Properties registry;
private InterceptionBroker broker;
public CommandFactory(SecurityBroker sec)
{
broker = new InterceptionBroker();
broker.addInterceptor(new SecurityInterceptor(sec),
new NamedSignature("execute"));
registry = new Properties();
try
{
registry.load(CommandFactory.class.getClassLoader()
.getResourceAsStream("commands.properties"));
}
catch (IOException e)
{
throw new RuntimeException("Unable to load commands.properties", e);
}
}
public Command build(String name)
{
String cname = registry.getProperty(name);
try
{
return (Command)broker.wrap(Class.forName(cname).newInstance());
}
catch (Exception e)
{
throw new RuntimeException("Error building Command: " + name, e);
}
}
}
The security broker is configured in a static initialization block in this action. In a real application it would be done in a seperate policy configurator, but this is just top demonstrate. FYI, the rules established in this broker block "George" from doing anything, but allow anyone else to execute any command they want to. Not realistic, but then this is just a sample application ;-)
Right now there isn't much in the way of sophisticated security rule handling. There is a single chain that is executed for everyone. It will be easy to chaneg this to support iptables style chains and jumping, or real rules systems, or delegating back to JAAS.
Right now I am wanting to rethink some of the filtering. I would like to make the static rule elements filter via the InterceptionBroker as that is much faster than executing the whole chain (until failure anyway) every time. I also need to add short-circuting for allowing progression as well as denying. The boolean return probably won't last as it really needs a PAM style "sufficient" to short circuit for authenticated admins etc.
0 writebacks [/src/java/commons-intercept] permanent link
Fri, 19 Dec 2003
Intercept Standard Library - Transactions
I have started to implement standard interception tasks and idioms in the intercept library such as transaction management. Transaction management is a fun one to bite off first because the most popular O/R frameworks use connection-oriented transactions. OJB ties transactions to connections in PB and OTM API's (I know how to cheat the OTM API for this, but I don't want to cheat), Hibernate ties them to the Session (which is tied to a DB connection).
The base implementation is just around an abstract factory designed to be configured via type 2 or 3 IoC. The only implementation of the Transactionfactory in CVS right now is for ODMG, which works with both the aforementioned API's (though I have never used Hibernate's). Any Hibernate guys want to catch up with me on the "right way" to do this for Hibernate transactions? I am afraid I am going to have to leap off the deep and and provide a whole wrapper around Hibernate the way the Spring developers did -- ick (not at Spring, but at limiting how they use it). The transaction-per-thread model from JTA is handy. On that note, JTA transactions will go in when I figure a nice way to not break the build for people who don't want to jump through Sun's legal hoopla to get the interfaces.
Having toyed with implementations of the standard type stuff people talk about with interception (and aspects) the obviousness of interception going hand in hand with type 2 IoC leaps out hard.
2 writebacks [/src/java/commons-intercept] permanent link
Fri, 12 Dec 2003Put together a quick benchmark to test some changes Chris made to the intercept library. He realized that instead of caching per class and doing a hash look up that we could just cache per-method and not even have to do an array index. He also reworked the matrix to make more use of method filtering in CGLIB. I need to get better at CGLIB as the results speak for themselves:
Release 0.1.2 (the last release)
[junit] Testsuite: org.skife.intercept.Benchmark
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 131.226 sec
[junit] ------------- Standard Output ---------------
[junit] # Direct (ms) Intercepted (ms) Overhead (%)
[junit] 1 6423.0 10847.0 68.87747158648607
[junit] 2 6400.0 10298.0 60.90625000000001
[junit] 4 6373.0 10560.0 65.69904283696846
[junit] 8 6385.0 11067.0 73.32811276429129
[junit] 16 6402.0 11932.0 86.37925648234928
[junit] 32 6404.0 13735.0 114.47532792004998
[junit] 64 6370.0 17538.0 175.32182103610677
[junit] ------------- ---------------- ---------------
Release 0.2 (current)
[junit] Testsuite: org.skife.intercept.Benchmark
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 124.792 sec
[junit] ------------- Standard Output ---------------
[junit] # Direct (ms) Intercepted (ms) Overhead (%)
[junit] 1 6376.0 7220.0 13.237139272271015
[junit] 2 6366.0 7331.0 15.158655356581852
[junit] 4 6365.0 7579.0 19.07305577376277
[junit] 8 6383.0 8015.0 25.567914773617417
[junit] 16 6372.0 9464.0 48.5247959824231
[junit] 32 6626.0 10821.0 63.3111983096891
[junit] 64 6375.0 14674.0 130.18039215686272
[junit] ------------- ---------------- ---------------
The first column is the number of interceptors on the invocation (I am afraid of an app with a 64 interceptor deep stack). The interceptor does nothing but immediately yield to the next interceptor in the stack. Eventually a virtual toString() is called that defaults back up to Object.toString(). Also, it requires a build of CGLIB against cvs at the moment, everything required should be in CGLIB 2.0 RC1 coming up though.
The benchmark is available in the source distro via ant benchmark
Managed to talk Chris into accepting cvs access =)
0 writebacks [/src/java/commons-intercept] permanent link
Mon, 08 Dec 2003Bug fix to fix an NPE when attempting to wrap an already wrapped object. Also includes improved documentation.
Source Tarball, Binary Tarball, and Javadocs
A comment I missed a while back asked about the performance of Signature.matches(...). Bo's concern was that calling this on every interceptor on every invocation would be painfuly slow. Rest easy, this isn't done. The broker builds match information when the interceptor is added and when a new class (not new instance, just new class) is first wrapped. It caches the interception information on a per-class/per-method basis so that the invocation sequence looks like:
MethodInterceptMatrix is key as it allows a hash lookup speed interceptor stack generation (plus interceptor instantiation time for the "new interceptor for every invocation" style interceptors). As mentioned, I am pretty sure I can reduce this to an array index, but havent had time/need to do so yet. The MethodInterceptMatrix is responsible for keeping track of what interceptors care about what methods and builds this information when new interceptors are added, or when new MethodInterceptMatrix instances are created when a class is generated.
The benefit to this design is faster interceptor stack invocation. The drawbacks are that adding a new interceptor to a broker with a lot of classes registered is slow, and adding a class to a broker with a lot of interceptors is slow. An additional drawback is that you only have java.util.reflect.Method information in the Signature.matches(...) test. There have been cases where I want to match calls against a specific instance only intead of all calls to that type's method. This has to be filtered at invocation time (as Bo was afraid of) so needs to be done in the interceptor instead of the Signature which is ugly =/ I have thought about splitting signatures into two types -- compile-time and invocation-time in order to make this cleaner, but haven't yet as I haven't had the need and no one has asked for it.
0 writebacks [/src/java/commons-intercept] permanent link
Mon, 01 Dec 2003Chris Nokleberg, one of the cglib developers, sent a patch in to optimize the use of cglib's proxies in the interceptor library I posted this morning. I replaced the tarballs on my site with the new improved version. No API changes, but the patch removed the last remnant of reflection in the invocation sequence.
Thank you!
2 writebacks [/src/java/commons-intercept] permanent link
Intercept Library Initial Release
This past weekend, my girlfriend being out of town, I got to hack a lot. It was nice. At one point I needed a general purpose interceptor tool and went digging through the various ones out there. XWork is tightly coupled to its Command stuff. Spring is tightly coupled to its IoC stuff and XML configuration. HiveMind is temporarily unavailable. AspectJ and AspectWerkz are not simple interceptor libraries, they are full fledged languages for all practical purposes.
So, in good old itchy fashion I tossed together a bare-bones interception library. It doesn't do IoC, it doesn't do XML configuration, it doesn't include a web application framework, it doesn't include an EJB container, or service definitions, or dependencies on commons-*. It just does interception of arbitrary method calls with arbitrary interceptors in 332 lines of code =)
It provides all the spiffies I want, and that others seem to want in very few classes. Its very optimized for wrapping objects and executing invocation stacks (ie, intercepted method calls) rather than for adding new interceptors. Making a typical call to an intercepted method doesn't involve any reflection (it uses cglib). The first time an instane of a given class is wrapped it is slow (bytecode generation) and when a new interceptor is added it is slow (needs to traverse its cache and figure out what it applies to) but the common stuff is fast and that's what counts =)
Using it is pretty easy, and works like:
InterceptionBroker broker = new InterceptionBroker();
broker.addInterceptor(new LoggingInterceptor(), new Signature()
{
public boolean match(Method m)
{
// Match every method call on every object
return true;
}
});
Map example = new HashMap();
Map same = broker.wrap(example);
// same is the interception wrapped version of example
example.put("foo", "bar"); // Will not be intercepted
same.get("foo"); // Will be intercepted and logged
Source code , Binary Distribution and javadocs are all available. It's BSD licensed for now. If anyone else uses it or contibutes then I may look into moving it into the Apache Commons Sandbox, but I won't bother as long as I am the sole contributor =)