First, let us review what is the concept behind services, with particular mention of the web services themselves.

A service is a business functionality exposed by a simple interface. Note that it is not an object, nor a method, nor a module, not a procedure. It is pure business functionality.

Services are consumed, not called, nor executed nor invoked. We define the service consumption as the interaction between agents, a client and a provider, which will communicate using a messaging system, where the data unit is a document. Consumption may be local or remote (meaning a service may not require remote communication).

A service may live in the web, where they are modeled as resources, identified by a URI. If not living in web, as suitable endpoint implementation should be provided.

A service is described at business level with a contract, where a glossary of the terms is agreed upon, and the semantics of all business concepts and processes are clearly stated and known by interacting parts (this is important). The service implementation is defined by the service name, its contract, an endpoint (unique entry point/port) where to send messages, the operations the service offers, the message interaction flow, the message composition (including the document definition) and optionally a binding information.

Nice definition. What is wrong with it?. Not much, actually. Just that when Web Services Architecture group decided to make those concepts into a standard, they did a great job, but also committed a couple of fatal mistakes.

The first one was the actual selection of SOAP as the messaging protocol. Yes, SOAP is a big mistake, not because it is SOAP, but because of what it was created for. You can read here a very complete explanation of the SOAP origins by one of the people that worked on it, Don Box. There, we can understand the main problem to solve was creating a protocol to better support RPC (actually RMI) in a object distributed system.

Yes, SOAP (Simple Object Access Protocol) was not created to support services documents and messaging. It was created to provide a technology neutral data/class definition system, plus supporting remote method access (that actually meant Remote Method Invocation).

So, why is this a mistake? Well, unless your idea of services was, from the beginning, that of distributed objects and the exposition of their methods, SOAP was no fit for a Services architecture. Why is SOAP there, not sure, but my common sense tells me it was not its place. If the idea was to support document oriented messaging, an RPC protocol was the weirdest decision I can think of. You will need to tweak the protocol to support what it was not supporting before. The only reusable aspect is that it was meant for HTTP transportation, which by the way is not a constrain of the WSA (it is the only implementation though).

History tells that soon after, SOAP was modified to allow document style definition, relaxing the literal RPC format. The SOAP acronym meaning was actually changed! So it is not an object access oriented protocol anymore, but a service oriented one. That should prevent people using it for RPC, was the hope.

Ok, if that was fixed (SOAP adjusted), we may have no more problems with that, right?
What is the second mistake?

Well, a subtle one: they kept the RPC idea in SOAP. Backwards compatibility? Not sure, but the RPC literal was there, in SOAP. Worse, the WSDL included that option and WSA did not discouraged the practice! I may think it was because there were tools supporting this and the idea was not banning anyone from using them. So, people started (or continued) using RPC and calling them services!
Is that wrong? Well, a little:

a. The problem is RPC is not Services,
b. RPC does not share the service metaphor,
c, RPC has different implementation side effects,
d. RPC usually forces fine grain while Services try the contrary
e. RPC couples (per method name, parameters and interaction)
f. RPC is in the IT domain, while services should be in the business one (this is a big one).
g. RPC usually works in the distributed environment, but services not are always distributed.
h. RPC forces request-response flow, while services may have any combination of flows.
i. RPC is usually blocking, synchronous. Services are asynchronous per messaging basic concepts.
j. RPC parameters usually require serialization of objects (to be passed as parameters). Services do not even know what objects are nor the parameter concept, they deal with plain messages.
k. In HTTP, as a particular implementation, RPCs mapping of concepts adds overhead to the call, while Services uses HTTP just as a transport and should not add more overhead.

Ok but, what about that? In case the RPC was kept due to tools, the existing tools at that time may be kept as RPC tools, and new tools to work with services and their concepts should had been created. Easy solution.

But that didn’t happen.

Actually, the tools took over and started to fight back! There was almost no support for document style. There was an RPC style non encoded. There was no standard way to represent a document (that you can feed the service call with). Yes, the tools claim they supported document style. But when I did some research projects to compare the tools, I found the services created using document styles didn’t work at all! I created a service, then I created a client using the same tool, and the client failed talking to the service! I made it work by manually structuring the XML document to have the first element named as the method that implemented the service! It was RPC all along, and since nobody complained, I assume no one was expecting to work document style. Well, there were a few.

But the worst comes afterward, with the creation of the Wrapped and Bare modifiers. They are no were in the WSDL nor the WSA. The wrapped is a way to use document style, but doing what I did manually in the lines above: formatting the XML to reassemble an RPC format message! You know what, testing the BARE modifier yield what I suspected: didn’t work.

That means the tools and server vendors are still selling RPC, creating RPC “services” and RPC clients. The ones that supported the document style, did a great job making people stay away from it, like Axis, whose example of RPC is a couple of lines long, but the same example using document style is gross and long, not even I could read.

Ok, granted, the web services implementation are just RPC in disguise. So what? We’ve been using RPC all our lives and we are happy with it, right?

Well, SOA is a style that looks forward to have its structure and components made out of business functionality. Under that assumption, we can build lots of additional features, like business composition (business processes), orchestration, governance, etc. But, for that we need services as business functionality, decoupled by an uniform interface. RPC adds coupling to the mix, since we are adding the object or library concepts (for many, those two are the same!) , the method invocation interaction, the data coupling in the arguments lists, and the stiffness of non-variable method definitions (just change one argument and all breaks!).

Point (f) above is important. All other features of SOA are built on top of the concept of a business functionality exposed as a service. Governance, composability, visualization, all those work with services, not with function calls. So, as mentioned in point f, what surfaces is IT, not business, and thus the features may not work as expected.

Add to the mix that, to actually create the services, you must be a programmer. Since they reflect the RMI concept, we assume distributions, method and parameter passing knowledge, etc. Even more, the actual code is converted into exposed services, bottom up approach. This will create services shaped as the actual legacy implementation, which may not match the actual business at hand. The top down approach, which should start with the definition of the service at the business domain level, is discouraged. Tools offer not help there (or very little), and only a few architects follow that path. So, the RPC influence is breaking the possibilities of actually taking advantage of the Services metaphor.

Can I say that the whole failure of SOA as a agent of change, an encourager of business over IT in systems, a provider of business valuable concepts like business processes, governance and product composing, is just the intrusion of RPC in the whole dance? May be, but there are other causes as well. Still, SOA may not raise from he tomb if no paradigm shift is achieved.

Wait a minute. REST is in the title, it is another problem in your list? Or is it just the enemy of SOA?

Well, not really. REST is another architectural Style. It was made for a very different problem and I can assure you it is not the replacement of SOA.

It is in the title because of two reasons:
1. People think the problem of SOA were the Web Services using SOAP,
2. and also people believe REST is a simple way of creating the next generation of services.

So, they are adding another point of failure to the new SOA by trying to construct SOA using what they call “RESTFull Services“.

Why is it a fail point? I’ve heard REST services rock!

Well, first I own REST an apology. You see, actually REST has nothing to do with this. The problem is with the wrongly named REST services, which are no more than an API based on HTTP. Now, there could be REST services, but they are much more complex that the ones most people use. And bigger. So, let’s simply call those “Not-So-REST Services” as Web API Services, (WAS? Nice acronym, a provider for one of those would be a WASP!).

Well, our WAS are, as usual, more oriented to functional calls (yes, RPC). There could be a way the WAS achieves a truly service meaning, when using HTTP as the messaging transport, and the Hypermedia as the documents. Yep, it is a nice match. But that requires people thinking on documents and interactions through messages, and they are not doing that. If you check, it is the same old RPC story again.

So, what to do then? Are we doomed, I mean, is SOA doomed?

Not really. We need to understand what a service is about, and provide a suitable implementation that allows developers to work with it without mapping. I mean, why does a service need to implemented as an object method call? It breaks the metaphor and thus breaks design and implementation, making it flawed.
I’m working on a DOSE (Document Oriented Service Engine) implementation in Java, WAS and WSA oriented (WSDL 2.0, that is), with no RPC. The programming model should be quite different that the proxy and RPC model, plus a set of articles to show how design using document orientation for services should be done.

So, stay tuned.

What if everybody is wrong? Well, not everybody, but majority.

Reading Kelly Water’s Post about “Agile Project Management: Avoiding The Waterfall” I came to think about this. Almost nobody claims today that they are following waterfall, because of the very bad connotation that word now has, but not actually because they understand what are they doing. Same thing for Agile followers.

From the post, I rescue that any process that has phases reassembles a waterfall. From actual presentations, blogs and discussions, Agile often means just following methodologies like Scrum, Lean or XP. Now I question: are we right about this? Let me see if we are, or not.

Waterfall is a term, it seems nobody knows who coined it, related to a phased methodology. Dr. Winston W. Royce wrote about it (without given that name), describing an ideal form that was not so good. The phases were the requirements gathering, analysis on those requirements, system design, coding, testing and operation. The ideal was each step may iteratively cycle with the next down and the next up, in little steps. The flaw was it was usually the case that testing will not just go back to coding, but it may go up till design, and from there, jump back to requirements gathering again. That flaw (the large back jump) was caused not because the phases, but because the testing phase is where the actual product is finally validated against design and requirements.

The problem is, there is no solution to this. We can dilute or minimize the impact of the testing findings, using several techniques, but we cannot avoid it. At any level, team or individual, the same steps are repeated: I see what it is needed, I analyze what is possible, I think on how to do it, I do it and finally I check if it works. That can describe the process of developer creating a simple three line method! That is, phased development is there, everywhere, and that is not the problem. The problem is when that phased process is done in a way that the impact of facing reality with needs is too big that multiplies the work and the cost of the back jump.

So, the waterfall illusion is when you think that rain is not a waterfall, simply because the raindrops are so tiny you don’t see them fall individually. But they fall anyway. It means, people that think eliminating the analysis and design steps and jump into coding breaks the phased process, I’m sorry, it doesn’t, it may avoid a large body of water fall into one place, but will make millions of little bodies fall everywhere. See the point?

Ok, there is another idea into this, and that is to make the water flow up. You start coding, with no idea of what you need, and then the phases are completed backwards. (Hey! If you want to start really backwards, then start by testing! Oh, yes, we have TDD). The code will make a design “emerge”. The next logical steps if to create requirements that match the design. Well, I guess that does not work. In that case, the requirements are still presented before coding. So the new phases are: requirements, coding, design, testing. Or, requirements, testing, coding, design, testing, coding, design, etc. Anyway, we always have phases.

Pages: 1 · 2 · 3

From a recent discussion with Duncan Cragg, related to his proposed FOREST, a get-only-rest-integration-pattern I came to think that it was good to create a little post to explain the difference between an Architecture Style and a Design Pattern. So, let’s do it.

It was Christopher Alexander, in his book A Pattern Language: Towns, Buildings, Construction. (Oxford University Press, 1977) that coined the Pattern term to denote ideas or solutions that were proven to be successful and widely used.

Well, that book was about buildings, and Alexander was an architect, of the building class. When mapped to the IT field, we have three levels at what we develop: Strategic, Tactical and Operational, or Architectural, Design, Implementation. So, we have those “ideas or solutions” in all those three levels. For Architectural level, we have Architectural Styles. For Design level, we have Design Patterns (or just plain patterns) and for Implementation level we have Idioms. You can read the definitions in POSA1 (Pattern-Oriented Software Architecture:A System of Patterns from Buschmann et al, Wiley 1996).

The style name is discussed by Roy T. Fielding in his dissertation about REST. He actually dislikes a little the name since he says it represents more a particular way of doing things (that singer’s style) rather than a general way of doing things. Coming from the art world myself, I can say it has that other meaning: a style is a way someone follows, and thus “followable”. Given someone is so original that creates a new style that others follow, does not take away that followable quality, only makes it richer.

So, as you know, the architecture of a system is the organization of its elements and their relationships, guided by principles. When defining a style, you define what types of elements would you use, what do they do, the relationships, and principles to guide their construction. A Simple example is the Client Server style. It identifies two architectural elements, the client and the server, indicates what each one does and who will the interact. Not this is a global solution: the whole system works Following this style.

BTW, Roy defines the Styles more like a set of constrains, coordinated, that define the elements and relationships. Actually, REST is defined that way in this dissertation.

When defining a Pattern, you work closer to implementation. There, you define a solution in the domain context, identifying actors and processes, relationships and results, aimed to solve a localized problem. The example here is the Factory. To solve the problem of needing different but similar objects depending on some parameter, the factory proposes the creation of the right type of object. Note this is in an object oriented context, and the solution is given in terms of objects, and it is to solve a particular problem in the whole system, not to define the whole system. Finally, note this is an idea that can be implemented in any OO language, and particularly in Java you can do it using inheritance or interfaces, meaning the pattern does not impose an specific implementation.

The Idioms are little patterns related to languages. These are proven ideas of how to accomplish things in one particular language. Say, to use StringBuffer in Java to concatenate strings.

As you can see, in general, patterns are good things, but quite different depending on the development level you are using. Now, to define a pattern, you need to provide at least five things, per Nick Rozanski and Eoin Woods, in Software Systems Architecture: Working with Stakeholders Using Viewpoints and Perspectives (Addison Wesley 2005).:

1. A name
2. A context where the problem and solution may be presented.
3. A problem to solve
4. The actual solution, given at the appropriate level
5. The consequences of applying that solution.

So, it is important to check all that before defining a new pattern, or before applying it. This last part is very important, since you may end up using a pattern that is not suitable for the context, for the problem or that has unwanted consequences.

Hope this helps clarify the concepts!

Through all these years, the service concept has been a confusing one. At the edge of Service Era, even before the service name was coined, services existed and were implemented using the tools at hand, and the tools helped to define the concept. I mean Services in all their presentations, those including SOA and Web Services, REST and CORBA, etc.

Today, after all this time, hundreds of articles, papers and news had been written explaining all facets of Services. Yet, many of them presented conceptual differences, and in the community’s mind a mix of myths started to grow.

The following is a list of myths I’ve encountered to be still present nowadays. Some are innocuous, some others are not. Furthermore, some may not be considered myths at all for all people, since their opinions may differ from mine. Anyway, here they are.

Myth 1: SOA = Web Services.
More that a myth, this is a general confusion enforced by tool vendors. Why? Because of selling SOA ready tools that offered Web Services construction, so conceptually having SOA was simply achieved by having Web Services
.
Reality: A Service is a bunch of functionality that is cohesively integrated into a locatable resource and accessed through a port using messages. (No, it’s not an object).

SOA is an architecture style that simply indicates the use of Services (special functional units) in the architecture. That is, you can have several very different architectures that are SOAs, with the common characteristic of having services in them.

Web Services Architecture is a special architecture that defines the description, interaction and messaging definition for a service that uses the web to communicate. Note that the service concept is part of both things, but Services are not limited to be Web Services. Web Services is just a way to create services. And Services may not use that architecture, nor be implemented trough the web.

Myth 2: Web Services = RPC.
SOAP (Simple Object Access Protocol) was born to make remote objects accessible through web. Its initial goals were clear: allow to encode a method call through web using XML. Later on, the service definition and concepts were improved, not limiting it to remote object method invocation but to a messaging system (even SOAP acronym changed to Service Oriented Architecture Protocol), but the “traditional” way of doing things kept implementing the services as simple remote method calls. In other words, a simple RPC.

The Service concept states the communication should be using messages, and those are all one way. If you want a request-response behavior, you need two messages: one in-one out. So, at the end, the services concept accepts RPC, but services are not RPC.

Martin Fowler states that difference in the book "Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions (The Addison-Wesley Signature Series) by Gregor Hohpe and Bobby Woolf", when he writes about integration by sharing. RPC is good to share functionality, but services will help to share functionality and data, and they are the less coupled form of integration.

Pages: 1 · 2 · 3

Prior the Internet days (some people talk about prior television), we had different villages, local ones, with no communication. Now, we live in a global village. We need to communicate and share knowledge, even in different languages and cultures. Furthermore, we learned it is not just us and the "my limited world", but any thing we do affects the globality.

Now, when working on a real world problem, working to find a solution of course, we need to communicate and all or our work is to complement the one from others. There is no one single person that fights all the villains and the rest of the team observes and cheers.

Solving a problem with a software intensive system, involves working with all parts of the system: Hardware, Software, people and processes. No one lives alone with its computer, everybody interacts with all the other components of the system, and the developers are part of that system too!.

Making sense of that, the strategy of working only in my local line forgetting there is something beyond that, goes back to the local world and denies globality. Does it mean you need to work all at the same time? No. It means you work in your local line knowing there is something bigger of which that line is an important part.

What does this matter in Steve Yegge’s post about NOOBs? Well, going extreme, the post indicates there is much waste, fat we need to remove to be healthy. The comments, even the modeling and describing language structures are needless for the seasoned, non-noob programmer. I agree, as long as the programmer is working in a system that has only two components: the programmer and his lines of code.

Unfortunately, that is not the reality. A system not only requires the work of many other components, some non-software and some non-programmers, but also requires all those components to communicate at design and execution time. Robert Martin once said that one of the software functions is actually communicate with people, readers that may have not participated in the development. Does that mean the comments should stay? Not exactly, but that the code itself should be readable, understandable.

One way of doing that, for the development group, is using sentences that make sense and do the function (you can scramble the sentences and do the same thing but being not readable).

Globalization in the System means all components will need to communicate and work with other components. Creating the product with those characteristics, because they are needed, is not being NOOB, but being aware of the fact the software is not an island.

William Martinez.

In a presentation by Cedric Beust and Alexandru Popescu at QCon San Francisco, about "Designing for Testability", called "Next Generation Testing", and in his post, Cedric talks about "Test Driven Development" (TDD), and does not talk to promote it.

Actually, he does not talk against it, but simply questions about using it, as if it is a good idea or apply-able to all cases. Cedrics points several problems of using Test-Up-Front technique, and one of his lines is: "Promotes micro-design over macro-design". Follow me on this...

Let's change the last D from Development to Design. A Test Driven Design is that one that is accomplished by basing the decisions on the test (The principle may read: follow the option that is best testable). No one is saying when to test, nor if you create the test before or after the actual coding. We are actually thinking on design, and taking into account one -ility: the Testing one.

Now, from my post about the design levels, (and here is what drove my attention over that Cedric's line), the term "micro-design" is defined as the design decisions the developer makes during the implementation of the "tactical design". Those design decisions should be in accordance to the tactical design, and those should support the strategic design.

Here, the question is now if the Test Drive Design is driving a design level that is not the micro design! Tests, if we talk about Unit and Functional tests as I think TDD means them, are a developers level thing. They, in this case, should drive Micro-Design only, and should never drive other design levels.

Let me explain it in other words: TDD shouldn't tell you what classes to use, which artifacts (queues, files, etc) or which relations between functional units there are. But it may help you decide which call sequence to use, which functions should a class have, which structures to rely on. Those are micro-design decisions, that may later on be refactored.

TDD, used at micro-design level will help define interfaces, improve cohesion and even enforce encapsulation and information hiding (if used correctly, of course, since we can create tests that violates all those best practices). But that is when the developer is creating the operational level descriptions (code). There should be a tactical design in place before doing this. And there should be an strategic design in place before that. We can use the testability to drive some decisions at those levels, but they cannot be the main -ility at all.

William Martinez

W3C has published the Web Services Description Language (WSDL) Version 2.0 Recommendation. That means we can talk about a new standard now.

From history, you can recall SOAP was not a standard by committee but a de facto one, and WSDL 1.1 was a member submission done by Microsoft and IBM. Seven years later, WSDL 2.0 comes to see the light.

I mentioned before in this blog that WSDL offered four types of messaging, a combination of style/encoding parameters. There was an RPC style, and a Document Style, and for a backward compatibility, RPC could be Encoded or Literal.

Also, WSDL allowed a extension to define the transport, but it was just to say we were going to use HTTP, and that to send a SOAP structure as its payload. It was fixed to use the POST method, and it was also fixed the headers contents and such. Not much flexibility.

Well, WSDL 2.0 changes that. It now offers only one style: document. With that you can simulate your RPC calls if you want. Also, it allows to define the content of the document using an XML Schema, but that doesn't mean you will be sending XML through the wire (more on this later)!
But it doesn't stop there. WSDL 2.0 expands significantly the HTTP binding spec. Now you can specify if using GET instead of POST, indicate if you want you parameters (defined in the schema in types, remember?) to be encoded in the URL! Even, you can indicate that you want multi-part messages!

How friendly is that to REST? Keep on reading...

Pages: 1 · 2

One of my students recently asked me a question ORM related. He had this little project he was working on to present in another class, and there was a little screen where he had some data. Let say he had a table with clients (relational, you may guess), and the client had a reference to the company table, since a client belongs to a company. Of course, the reference field in database was a Company ID.

His question was very simple, although very useful to explain some concepts on the spot. He was, of course, converting that client row into an object, to be later displayed in the little window. So he asked me: “Should I add a company ID attribute to the object, or should I add the company name directly?”. I looked at him and asked him back: “How do you fell adding that ID to the object? Is it natural?” He glanced at his computer and then said… “No.”

Then we began to discuss. Actually, we started by defining what was that object’s function in the application, what was its role, what was it there for. The answer was very simple: it was there to provide info to the screen to display the data. Then, we went into discussing what the user needs, and the answer was the user needs the company name, not an ID. Lastly we went into discussing if we needed the to objects, one Client object with a company ID and one Company object with a matching ID and a Name. We found out that, having those two objects, will require us to match them. That is a relational operation that was natural to the database. So, it was obvious we better let the database do its job and send us a new hybrid object with the client information and the company name already in place.

But wait a minute: It that ok? Weren’t we breaking the relational rules and normalization of the data? Thinking a couple of seconds I would say no. We weren’t.

Data is data, and it is stored and related in the database. You can process and produce new information by manipulating that data using the DB tools. That new information is not to store, but to consume by the business logic layers. That is BL information, it is not simple data anymore.

So the idea of ORM one-to-one does not fit in this example. If we have data and we have tools to process that and transform it into business information, then we should allow the BL layers to consume that BL info and not to spend cycles in processing by hand the data. Now the objects model the reality, not the data structure, and we work with cooked information and not raw data.

William.

I a recent comment, Paul mentions he is not familiar with the Document Style concept. This is a great opportunity to touch the theme and clarify some ideas about WSDL and the WSA idea of a service.

WSDL is meant to describe the service definition and interaction models. WSDL defines:

• The Service’s visible endpoint. That is, the address where the port of the service leaves.
• The Binding. That is, which transport to use and how.
• The Operations. Each operation with a name and a message flow (a named message inbound or outbound).
• The Messages. Each named message has a number of named parts that conform the payload.
• The Parts. Each one is defined either by indicating the primitive type or a schema defined xml.
• The Schema. A Types section that is defined using an XML schemas.

Using the above sections, we can define the service by indicating the port, the available operations, which message flow each operation has, and what is the structure of each message.

With the WSDL, the client is able to know how to construct the message and where to send it. So far so good for our service notion.

The problem comes when WSA came out with the style/encoding idea. SOAP was originally created to access Objects remotely (Simple Object Access Protocol). Here is the story.

Those were the times when RPC was the idea of remote access. So, SOAP was created a requirement for the body of the message. It had to be constructed with a fixed structure, where the root element’s name should be the same as the method name. The children elements should be one per parameter, and each one containing the type as an attribute. That structure was known as the encoding.

Pages: 1 · 2 · 3

Funny article in SOA Masters Class site. It explains the vision of SOA based in two variables: the Architecture involvement scope, and the Service concept scope.
Yep, there is a big difference when you see the Service concept as an exposed component, or as business functionality. Same for involving the architecture simply as a glue to interoperate disparate systems, or as a business driven structure.

Funny indeed!

William Martinez Pomares