Could anyone please advise me on this design of a data integration solution:

We’re doing a data integration project between a MS Sql Server database and a Microsoft CRM system (through its web services).

We’re trying to build a “service” layer on top of the database. The design of our current solution is to use web services for CRUD, with xml being the format of data.

Views are created to consolidate related tables into one entity, and we query these views, using the “SELECT * FROM someview ” + “For XML” to generated xml that will be returned from our web services.

For update we’re trying to use the same approach – using SQL XML to map updates views, we have “instead of” triggers defined on top of these views, and in these “instead of” triggers we update the underlying tables.

The views/triggers are generated by tools so don’t be too concerned with coding efficiency here…

what do you think if we use WCF data Provider to publish a enterprise data model (essentially DTOs) ? p.s., we don’t have a BL layer for now, it’s all in the stored procedures!!!

What’s your opinion on this / any better design? much appreciated!

these links should get you started:
http://davybrion.com/blog/2010/05/why-you-shouldnt-expose-your-entities-through-your-services/
http://davybrion.com/blog/2009/11/requestresponse-service-layer-series/

I am all fairly new to this and am actually glad that Ben posted his comments – its always good to hear two viewpoints when trying to learn.

@Davy. That being said your post is interesting and I’d like to learn more about your way of providing access to the database. Do you have any links/resources you could post here to help me learn?

seriously, go waste someone else’s time… i already said i have no problem with being wrong about the filtering, the other stuff however, still holds and you continue to ignore it while you keep focusing on one sentence. it certainly makes your last paragraph all the more ironic.

By the way, please explain exactly how something done so poorly in the first place would become so widely used that other people would have to “clean up the mess”, as you put it. Are you tired of cleaning up other people’s messy code, or is this some general humanitarian concern? You must have hated the 20th century, when we actually used pointers and all that really dangerous stuff. (I suppose that should have been kept top-secret, too.)

@Davy: A database-backed service layer is not all about querying, but it is very much about querying, and RIA Services makes that part of it–the part that otherwise requires much tedious boilerplate code–a cinch. Nowhere does it state this is the only way to do things.

Don’t get all stuffy about my poor attitude: your post, rich with snit and cynicism, practically invites it. I came here looking for useful information about RIA Services, and if I hadn’t known any better before I read your post, afterwards I’d have thought it was a turd. I mean, really: what modern services layer technology set would facilitate something so odious as client-side filtering? Guess what, buddy: every one of those Microsoft demos you laughed at was demonstrating exactly the opposite of what you assumed.

So, yes: in my mind, it is exceptionally arrogant for anyone to spend fifteen minutes poking through some Visual Studio wizards, immediately form a wrong opinion, and rush to WordPress to crow about it to the world. If that’s not deserving of mild censure, I’m not sure what is.

The fact that some ‘Prime Directives’ (read up on your PARC history) are violated when it comes down to encapsulation worries me far more.

“There’s good software, and there’s bad software. If you find yourself working on a lot of the latter, maybe it’s time to ask whose fault that really is.”

It’s probably partly your fault, if you consider this kind of development good practice.
It just falls to pieces ‘in the wild’ and then there’s other people that have to clean up the mess.

Indeed i haven’t, because i haven’t verified your statement yet. I really don’t have a problem with me being wrong about that, but the other problems that i believe are inherent with using RIA Services are still there… if you really wanna make this discussion interesting, perhaps you want to talk about those? Or is a service layer all about querying for you?

As for my previous statement being a cop-out, that’s the kind of response you get with the attitude that you display. You talk about arrogance in my point of view, but your second comment was just filled with it. You can’t really expect anyone to talk to you seriously if that is how you present yourself…

I notice you haven’t corrected the misrepresentations in your post.

“There’s good software, and there’s bad software. If you find yourself working on a lot of the latter, maybe it’s time to ask whose fault that really is.”

i’m pretty lucky in that i hardly ever have to deal with bad software so by following your logic, i could conclude that i must be doing something right in my decisions and choices

and you are indeed right: there is good and bad software, but that doesn’t mean we shouldn’t try to do better. We’d all end up as frustrated as you appear to be.

If I can’t be sure that you made the whole scenario up, I’m at least certain that you’re blind to the simple fact that changes are batched in the context until that method is called.

Study up, junior.

var albums = domainContext.GetAlbum().Select(a => a.Name == "Back in Black" && a.ReleaseYear == 1980);
var loadOperation = domainContext.Load(albums);

The albums returned (asynchronously) in the operation are only those that match the criteria specified in LINQ. So, RTFM.

While I’m at it, griping about how a technology might allow lesser developers to shoot themselves in the foot is the height of arrogance. What exactly is your concern–that a bunch of newbs are going to write a huge system using poor methodology and with no oversight, and that somehow, somewhere, a long-suffering senior developer–or worse, society–will be forced to deal with the consequences? Please. There’s good software, and there’s bad software. If you find yourself working on a lot of the latter, maybe it’s time to ask whose fault that really is.

The good thing about WCF RIA Services is that it provides a place (your domain services classes) for you to add business logic that is separate from your presentation layer, and it provides a great deal of flexibility for you to structure your business interface and logic however you want.

Therefore I see no reason why you can’t define a sound architecture using WCF RIA Services, and implement it productively.

I initially had the same opinion about a WCF Data Services. However, I eventually realized that what it really does is provide a standard REST interface to any IQueryable. It just so happens that out of the box most of the examples are using the Entity Framework as the IQueryable implementation. It is similar to how people think that LINQ is just a tool for ORM.

Data Services also has reflection based provider that you could use to expose your domain model (not your database) as services. I am currently writing a custom provider (using the 1.5 update) that allows us to load metadata on the fly and generate different web services depending on customer customizations. There are certainly limitations to the model.

if the data service layer would only be used to import the data into your system, then i guess there’s no problem with that. anything else that consumes that data through another service that you do have control over, i’d hide the actual db schema from them and provide the data in a form that is the most optimal for the specific client(s)

I’ve been going back and forth on the best way to do this. Either model out the business objects then put another interface layer on top of that, or just create classes/types for the metrics we want and call get methods with the sql on those to return the data we want.

personally, i don’t want any generated code at all… i want something that allows me to deal with cross-cutting concerns in as few places as possible, something that doesn’t advocate sending your business enitities to clients, something that doesn’t encourage you to expose your database (or parts of it) and something that does not automatically lead to a maintenance nightmare once you need more than straightforward CRUD functionality.