I’m 100% sure there is no dent in my reputation. What’s the reason of its appearance? Yes, I’ve made a test suite showing many lacks in ORM products. Taking into account BLToolkit results, even our own product isn’t ideal there. So what kind of dent to you see?

Before I reply to everything else, can you tell me, are you talking just about performance tests? Test suite consists of two parts: LINQ and performance tests. So I’d like to hear something about LINQ tests as well.

For me it’s just geek entertainment, and I learn something about NH along the way. But even still, it’s a time sink. Can’t imagine how much time wasted for those who are active in responding to everyone about this.

Your comment was what I tried to explain to Alex, but better worded.

Alex,
> So think about politics. Blaming the others, giving obvious advices, providing no proofs – that’s you.
At least I’m not the only one, huh Alex?
It is not my intention to make you look like a fool.

What I find funny is that you advice others to stop wasting time and start improving their product, while you keep discussing with everyone as if your time is in abundance. Maybe it is, I don’t know.

If you agree with most of Seans answer, maybe also with the part “they are meant as insight into how you are unknowingly hurting your cause, in my mind at least.”? That is kind of what I meant with the dent in your reputation and filling it up.

Davy,
> you know, if nobody responds to his comments, he would just go away…
I find this entertaining so I don’t mind. But if you want him to go away, just say so, I’ll stop responding then.

I know such a web site significantly polarizes people’s opinion, and this isn’t related just to my company and product. You decided you will study our product at last mainly because of my position. But many others decided to study it first. The same is about other products listed there. Some people reading Oren’s blog decided to stay with NH independently of results we’ve shown, others have started to think about alternatives. I think that’s great.

There are few other aspects I could discuss, but for now there is really no time for this.

Aside from the fact that I get put off by your argumentative and abrasive tone, I think the point you are generally missing is that you are [blindly?] obsessed with proving that ORMBattle.Net is the “one true way” to compare ORMs. You write countless essays (er, I mean comments) about how test X is correctly written and measurement Y is the true key to measuring “the best” ORM on the market. Arguing on the internet has got to be the biggest time-sink there is so do yourself and us an favour and stop obsessing. If you truly believe in your site and product then they should not need to be so vigorously defended and the competition need not be attacked. Focus on your site and product and stop “educating” us via huge comment threads, which, honestly, it isn’t doing you any good, public-image wise.

I know I for one will gladly consider any ORM — I’ve used SubSonic and EntitySpaces on a few projects and have recently begun experimenting with NHibernate and Fluent NHibernate. I mean to try LLBLGen Pro too but with it having a similarly steep learning curve to NHibernate I can only choose one to learn, for now. I would probably have tried DO too, but to be honest I was put off by the *appearance* of a possible hidden agenda on the comparison site. I know you’ll say that wasn’t the intent and that you removed DO from the running but the damage was done.

Even that could have been overlooked though, but when I read your posts/diatribes other blogs and sites — like the “leave the industry” one above — it really sealed the deal for me. I’ll pretty much have to exhaust all other options before I turn to your ORM. Those options have grown for me thanks to your site by the way, so thanks for that. Unfortunately it also means I have more tools to play with before I try yours.

Good luck with your product and site, and please don’t take my comments as insults. They are not intended as such. Rather they are meant as insight into how you are unknowingly hurting your cause, in my mind at least.

> BTW your reputation already got a big dent in it I think.

I don’t think so. But I’m pretty sure the reputation of few persons that were thinking about exclusion of their products from tests will suffer.

> Maybe you can fill up that dent with improving your product, instead of saying the same things over and over again

So you just tied together two independent things and think it’s ok? Or you seriously think we don’t work on DO4?

> If you want to waste time, argue all you want.

Obviously, I don’t want waste the time. Ridiculous statement.

Now read your own post – none of sentences there is related to the subject. They all are about me personally. Have you said something important? I don’t think so. Just flood. But you THOUGHT you said, yes?

So think about politics. Blaming the others, giving obvious advices, providing no proofs – that’s you.

Further I’ll reply to comparison-related questions only.

@ John:

> if the scenario isn’t real-world, then the results are irrelevant in the real-world

They are relevant. Yes, real-world results might be different, but the numbers we show describe real world cases pretty well, you must just clearly know how to apply them.

Let’s take my last post as an example: http://ormbattle.net/index.php/blog/105-a-honest-comparison-of-entity-framework-and-nhibernate.html

Why NH with prefetch is just 2 times faster than EF without prefetch? How can it be true that 1 query is just 2 times faster than 2500 fetching the same data? The answer is obvious: slow materialization + slow query pipeline. That’s what we’ve shown.

So dealing with large amount of queries or data isn’t the case where NH will be the best choice. Is it related to real-world? I think, yes.

See also:
- http://ormbattle.net/index.php/faqs/85-what-results-of-your-benchmark-show.html
- http://ormbattle.net/index.php/faqs/84-when-results-of-these-benchmarks-are-applicable.html

> if there’s anything in my first paragraph that’s not obvious to someone, then that person is clearly either an entry-level developer or not a developer at all

There is actually third option: you’re simply wrong. I’m leaving the opportunity to prove this for you.

For a benchmark to be valid, it needs to represent real-world scenarios, and in the case of databases, that means the DB is on its own server and any and all accesses to it has network overhead. Sure, you can change the scenario to measure other things, but if the scenario isn’t real-world, then the results are irrelevant in the real-world.

Of course, the point we’re all missing is that if there’s anything in my first paragraph that’s not obvious to someone, then that person is clearly either an entry-level developer or not a developer at all. Therefore, like Davy, I’m going to ignore anything you have to say as well, but because of your lack of expertise rather than your child-like refusal to consider the possibility that a benchmark could be irrelevant.

BTW your reputation already got a big dent in it I think. Maybe you can fill up that dent with improving your product, instead of saying the same things over and over again about the competition and its people.

Then again, it’s your choice of course. If you want to waste time, argue all you want. I just sit back and enjoy the soap opera.

To Davy:
Some people will always love hand-coded SQL, SP’s etc… Better to focus on the people that want to learn, instead of the unwilling who stick their head into the sand.
Oh and I love your blog!

I’m sure that all of us who are in favor of the participants in the ORM industry working together to adopt fair and standard benchmarks want to discourage any public wrangling. It is tedious for us to have to read. If these guys say stupid things about you then we are quite capable of reading what they write, and judging, without your elaborate responses/goadings. It is much better to leave it, let it go, and allow us make our minds up.

The established benchmarking sites in the IT industry are all run by people who have to be able to resist considerable pressure from misguided pressure groups, if ever it happens, without flinching.

With good wishes to everyone in the ORM industry,

Phil

I think I speak for a lot of people that followed the discussion at Oren’s blog when I say “dude, let it go”.

> And you’ve already shown on Ayende’s blog that you are impossible to talk to.

You tell me I’m the person impossible to talk, but facts are completely different:
- I’m ready to discuss everything. You aren’t: “Because of this, i have zero intention in even trying to explain the differences in our thoughts and i won’t waste time trying to argue with you.”.
- The same about Oren. But I’m glad he answers a bit more carefully now.

“i think there is something seriously wrong with your thought process in general” – so tell me, what is wrong there?

The problem I see is that you’re trying to play with the words instead of proving your position. Great, go further with this. But that’s purely your problem, not my.

I can add that I answered on most of questions forwarded to me (except may be the most ridiculous ones). And I’m still ready to do this. Yes, I don’t promise anyone to agree with him, and that’s what people must accept. I never simply say “no”. I’m ready to waste a lot of my time explaining why something that was offered won’t work. In if there are no strong arguments on your side, I won’t agree with you.

Speaking about Oren, try to count how many times he said “I agree with this” in these discussions (actually I can’t remember even a single point). Compare this to my “I agree with this”. Now compare the numbers and think who’s really impossible to talk here.

> i have no problem being compared to others…
I didn’t mean exactly you. It was about anyone.

> What does it truly matter if, in real world code, ORM A requires more roundtrips to the database to achieve what you need to do than ORM B would?

There are lots of factors in real world, and we will never be able to measure everything. I hope this is clear. On the other hand, we’re trying to measure the most important ones.

Here you’re talking about quality of query, fetch and preload pipeline. Yes, currently we don’t measure its quality in count of queries its send; moreover, our current tests measure pretty basic stuff now. But:
1) We’ll go further with this. Bulk load tests involving future queries and preload pipelines will be there some day.
2) Even what’s was measured now shows the difference pretty well. E.g. NH and Subsonic (that was claiming to be super-fast & super-simple) have exposed their query pipeline is the slowest one. Do you think running a simple query returning 1 page 7 (or 20!) times slower than competitors is ok? I don’t think so.
3) Moreover, I think it’s hardly possible these tools “save” so many queries that they can compete with leaders. I hardly believe that in real applications they run even 2 times less queries.
4) IMHO, 3) makes me think there can be absolutely the same situation with queries (possibly not with NH, but quite likely – with Subsonic): bad raw performance just proves they never been seriously profiled & compared to other products.

So why I must BELIEVE they are better from this point? Because they’re POPULAR? I know this does not work. DO1.X-2.X was pretty popular. But if I’d benchmarked it @ ORMBattle.NET, likely, it would be the worst one.

Popularity is related to acceptance by average developer (likely, it implies simplicity or minimal learning curve), costs and many other factors (documentation, etc.). Moreover, popularity is related to popularity itself: the more tool is known, the faster it gets new fans. But performance does not play any role here, until it is acceptable for this average developer.

And if there are no benchmarks, this acceptance limit can be very low. On the other hand, when benchmarks appears, performance requirements are raising up. The same happened many times in history.

The best thing is that customers win because of this.

P.S. Good luck with analysis of my wrong thought process.

Anyway just wanted to share some positives we have had using nHibernate and appreciation for people like yourself who contribute to this project, thank you.

Mike

i’m not saying that there shouldn’t be ORM benchmarks, only that people shouldn’t base decisions on bad benchmarks. Take the whole entity hydration cost thing… let’s say that ORM A has a lower overhead when it comes to hydrating entities than ORM B. What does it truly matter if, in real world code, ORM A requires more roundtrips to the database to achieve what you need to do than ORM B would? Running tests like that in an environment where the database is present on the same machine as the code is useless because in a real world system, nobody in their right mind would have the DB installed on the same machine. The overhead of network roundtrips definitely changes a lot of things.

Also, there is a lot more to it than simply performance results in silly benchmarks. Why is nobody paying to attention to features, power, flexibility? Current acceptance and user community, not to mention documentation are things that really need to be taken into account as well. If performance in benchmarks is what we should base our decisions on, none of us should be writing code in C# anyway.

As for my comment about managers, i truly feel that they shouldn’t even be involved in deciding between ORMs or any library for that matter, unless there are licensing issues attached to it (either through a non acceptable OSS license, or a license cost)

i have no problem being compared to others… but you and i think very, very differently about a lot of things. And you’ve already shown on Ayende’s blog that you are impossible to talk to. Not only that, but i think there is something seriously wrong with your thought process in general (i’m basing this on everything i’ve read from you so far), not to mention the manner in which you try to engage people in conversation. Because of this, i have zero intention in even trying to explain the differences in our thoughts and i won’t waste time trying to argue with you. Any future attempts from your side to begin any kind of discussion with me will simply be ignored. So do yourself a favor, and don’t waste time on me

Likely, there was a performance test similar to ours, since real life it can’t differ so much. Although maximal possible difference must be close to 3-4 times. Compare e.g. BLToolkit and Entity Framework performance – there is nearly the same case. BLToolkit is much closer to DAL than to ORM (as well as LINQ 2 SQL), but it is really fast.

> As I read it, he’s saying that benchmarking will actually help acceptance of ORMs as it will put an end to vague statements about performance deficits.

++

As I understand it, you are saying that there shouldn’t be ORM benchmarks. You also seem to think that Tony is against ORMs. Hold on. As I read it, He’s saying that benchmarking will actually help acceptance of ORMs as it will put an end to vague statements about performance deficits. He is warning that current language being used in the debate about ORM benchmarking will serve just to put managers off. If so, Yes. It will. Even your comments about managers in the blog made me wince.

I’m not close enough to the issues to join in the debate on what type of benchmarks would be fair, and how they should be administered, but I can’t help but agree that clear and objective benchmarks would help me a great deal, and would surely serve to highlight weaknesses in individual ORMs.

It’s fun to see how weak players are walking around its “bad quality”, “wrong tests” & “framework misuse” further and further.

Take a look at Telerik or BLToolkit. Or read this: http://ormbattle.net/index.php/blog/93-great-quote-from-todays-e-mail.html

I think if you don’t want to be compared to others, may be it’s your time to leave the market.