Thu 9 Oct 2008
Complex event processing, my move, and other things
Posted by joeldg under coding , database , serverNo Comments
First) I moved to Boston, it was not a choice against NY it was a choice for Boston and I am glad I did, I have found a great new place to work and I will be starting with them later this month. The only bad, I cannot find a Chase ATM anywhere here, and they have no branches, I have been relegated to opening an account with BofA which is not making me happy as they are one of ‘those’ banks.
Second) I originally came here working remotely, yes I thought that it was not fun this time and chose to switch to full time employ here in Boston (for more money than I was making at my ‘relaxing job in NY’ as well) ..
Now on to other things, I would like to talk a bit about CEP (Complex Event Processing) and it’s relationship to what developers and engineers do.
Basically, CEP is a query that runs continually on an sql server (that is very general I know) and that server has several inputs which continually input data… Okay, we got that. Now, many of you know about stored procedures and triggers in modern databases, these are replicates of this idea that are ’selfishly’ based. CEP is one of these new buzzwords and ideas, and to think “oh, that is it” is folly, there are people charging over $10k/month for this (I am serious).. The entire point is that you have streams which feed into a database, the continual query has a ruleset that it follows and your business logic is implemented in real time. Okay, cool.. BUT has the world forgot about TRIGGERS in databases, and stored procedures? I realize that this is designed to circumvent DB programmers who have been doing what they do for a long time, but hey…
Following that I don’t want any comments about how I rail against {x paradigm), I love you guys, but just read on..
I suspect that mysql will implement their own version shortly instead of using sqlstream.com (which is linked off the mysql site) and postgres has telegraphCQ (google it) and so on and so forth.. These guys need to get on the ball and explain to business that they have had this ability for a long time and display they are in fact the DB companies we trust and not trying to hamper some new technology that people are charging insane amounts per month for, the same things they could easily program in triggers and stored procs..
CEP is simple, continual queries, and rulesets to work on those queries.. so, if you have stock data on an input and it is flowing in, along with other stock data, you can set a rule saying “if stock A is below 85 and stock B is above 34 then buy stock A” … (that is a very simplified example), but these are not neural networks tied in with GA folks, these are “business rules” programs.. and if you know ANYTHING about modern databases and triggers and stored procedures then you are already doing this stuff…
I spent a good deal of my day today reading and researching what these things do and have concluded they are NOT good for true expert systems, but they are good for many others and are quick to build and deploy, but if you are building an expert system you will be better served to write your own rule set inside the database that stores your data, they all support them now.
The only position I could find myself in using a CEP is with a neural-net committee system as it would be easier to program the CEP to follow rulesets of a committee consensus and fire off programs than to write the logic code, that is it.. And this is ACTUALLY what I am doing with CEP…
However, I keep finding myself thinking that a trigger in mysql would do the same damn thing.
If you have some thoughts on that, please let me know if I am wrong..